Why NFS/NAS on Object Storage May Not Solve Your AI Problems
November 15, 2023
By 
Beinan Wang

Many companies are working with development architectures for AI platforms but have concerns about efficiency at scale as data volumes increase. They use centralized cloud data lakes, like S3, to store training data for AI platforms. However, GPU shortages add more complications. Storage and compute can be separate, or even remote, making data loading slow and expensive:

  1. Optimizing a developmental setup can include manual copies, which are slow and error-prone
  2. Directly transferring data across regions or from cloud to on-premises can incur expensive egress fees

This webinar covers solutions to improve data loading for model training. You will learn:

  • The data loading challenges with distributed infrastructure
  • Typical solutions, including NFS/NAS on object storage, and why they are not the best options
  • Common architectures that can improve data loading and cost efficiency
  • Using Alluxio to accelerate model training and reduce costs

Many companies are working with development architectures for AI platforms but have concerns about efficiency at scale as data volumes increase. They use centralized cloud data lakes, like S3, to store training data for AI platforms. However, GPU shortages add more complications. Storage and compute can be separate, or even remote, making data loading slow and expensive:

  1. Optimizing a developmental setup can include manual copies, which are slow and error-prone
  2. Directly transferring data across regions or from cloud to on-premises can incur expensive egress fees

This webinar covers solutions to improve data loading for model training. You will learn:

  • The data loading challenges with distributed infrastructure
  • Typical solutions, including NFS/NAS on object storage, and why they are not the best options
  • Common architectures that can improve data loading and cost efficiency
  • Using Alluxio to accelerate model training and reduce costs

Video:

Presentation slides:

Complete the form below to access the full overview:

Videos

Sign-up for a Live Demo or Book a Meeting with a Solutions Engineer