@lhoestq
We recently worked with @databricks to make the best @huggingface × @ApacheSpark integration, and went a step further Introducing support for HF Storage Buckets in Spark ! Enabling fully optimized access to both AI datasets and buckets on HF on Spark https://t.co/pgvWj7XCot