Many organizations are leveraging EMR to run big data analytics on public cloud. However, reading and writing data to S3 directly can result in slow and inconsistent performance. Alluxio is a data orchestration layer for the cloud, and in this use case it caches data for S3, ensuring high and predictable performance as well as reduced network traffic.
In this office hour, you will learn about:
- How to set up Alluxio with the EMR stack so that Presto jobs can seamlessly read from and write to S3
- Compare the performance between Presto on EMR with Presto and Alluxio on EMR
- Open Session for discussion on any topics such as solving the separation of compute and storage problem, and more
ALLUXIO COMMUNITY OFFICE HOUR
Many organizations are leveraging EMR to run big data analytics on public cloud. However, reading and writing data to S3 directly can result in slow and inconsistent performance. Alluxio is a data orchestration layer for the cloud, and in this use case it caches data for S3, ensuring high and predictable performance as well as reduced network traffic.
In this office hour, you will learn about:
- How to set up Alluxio with the EMR stack so that Presto jobs can seamlessly read from and write to S3
- Compare the performance between Presto on EMR with Presto and Alluxio on EMR
- Open Session for discussion on any topics such as solving the separation of compute and storage problem, and more
Video:
Slides:
Many organizations are leveraging EMR to run big data analytics on public cloud. However, reading and writing data to S3 directly can result in slow and inconsistent performance. Alluxio is a data orchestration layer for the cloud, and in this use case it caches data for S3, ensuring high and predictable performance as well as reduced network traffic.
In this office hour, you will learn about:
- How to set up Alluxio with the EMR stack so that Presto jobs can seamlessly read from and write to S3
- Compare the performance between Presto on EMR with Presto and Alluxio on EMR
- Open Session for discussion on any topics such as solving the separation of compute and storage problem, and more
Videos:
Presentation Slides:
Complete the form below to access the full overview:
.png)
Videos
Deepseek’s recent announcement of the Fire-flyer File System (3FS) has sparked excitement across the AI infra community, promising a breakthrough in how machine learning models access and process data.
In this webinar, an expert in distributed systems and AI infrastructure will take you inside Deepseek 3FS, the purpose-built file system for handling large files and high-bandwidth workloads. We’ll break down how 3FS optimizes data access and speeds up AI workloads as well as the design tradeoffs made to maximize throughput for AI workloads.
This webinar you’ll learn about how 3FS works under the hood, including:
✅ The system architecture
✅ Core software components
✅ Read/write flows
✅ Data distribution/placement algorithms
✅ Cluster/node management and disaster recovery
Whether you’re an AI researcher, ML engineer, or infrastructure architect, this deep dive will give you the technical insights you need to determine if 3FS is the right solution for you.