Many companies are working with development architectures for AI platforms but have concerns about efficiency at scale as data volumes increase. They use centralized cloud data lakes, like S3, to store training data for AI platforms. However, GPU shortages add more complications. Storage and compute can be separate, or even remote, making data loading slow and expensive:

1. Optimizing a developmental setup can include manual copies, which are slow and error-prone
2. Directly transferring data across regions or from cloud to on-premises can incur expensive egress fees

This webinar covers solutions to improve data loading for model training. You will learn:

• The data loading challenges with distributed infrastructure
• Typical solutions, including NFS/NAS on object storage, and why they are not the best options
• Common architectures that can improve data loading and cost efficiency
• Using Alluxio to accelerate model training and reduce costs

Video:

Presentation slides:

Many companies are working with development architectures for AI platforms but have concerns about efficiency at scale as data volumes increase. They use centralized cloud data lakes, like S3, to store training data for AI platforms. However, GPU shortages add more complications. Storage and compute can be separate, or even remote, making data loading slow and expensive:

1. Optimizing a developmental setup can include manual copies, which are slow and error-prone
2. Directly transferring data across regions or from cloud to on-premises can incur expensive egress fees

This webinar covers solutions to improve data loading for model training. You will learn:

• The data loading challenges with distributed infrastructure
• Typical solutions, including NFS/NAS on object storage, and why they are not the best options
• Common architectures that can improve data loading and cost efficiency
• Using Alluxio to accelerate model training and reduce costs