What is Alluxio Enterprise for Data Analytics?

Alluxio Enterprise for Data Analytics accelerates query performance for large-scale analytics workloads,  reduces cloud storage costs, and simplifies data access. Alluxio’s highly distributed, intelligent cache improves data-intensive query performance and reduces the number of costly cloud storage API and egress charges. Alluxio’s unified namespace provides seamless and secure access to data spread across disparate sources.

‍

What's New in Alluxio Enterprise for Data Analytics 3.2?

1. Evolved Architecture to Maximize Speed and Scale

Alluxio’s next-generation architecture, DORA (Decentralized Object Repository Architecture), dramatically enhances the performance and scalability of large-scale data analytics workloads. Learn more about DORA in this post from our engineering team.

‍

Unlimited Scalability with Decentralized Metadata

With DORA, metadata management is distributed across all Alluxio worker nodes. This decentralized approach enables unlimited scalability, supporting tens of billions of files within a single Alluxio cluster. By eliminating the bottleneck of centralized metadata management, DORA paves the way for unprecedented scalability in data-intensive environments.

Reduced Read Amplification with Page Store

DORA’s Page Store introduces a fine-grained caching system for more efficient data storage and retrieval. This innovative approach reduces read amplification by up to 150 times, significantly improving overall system efficiency. Furthermore, it enhances unstructured file parallel read performance by up to 9 times and boosts structured file position read speed by 2 to 15 times. These improvements translate to faster data access and improved analytics performance across a wide range of workloads.

Improved Performance with Zero-copy Network Transmission

This new release implements a Netty-based data transmission solution, replacing the previous gRPC-based system. This zero-copy approach improves large file sequential read performance by 30-50%, enhances memory efficiency, and boosts overall read performance. As shown in the TPC-DS benchmark results below, compared with not using Alluxio, Alluxio DA 3.2 delivers 2x performance when accessing remote region S3 storage.

‍

Chart: Alluxio DA 3.2 versus No Alluxio remote region S3 (time: ms) 

2. Reduced Cloud Storage Egress and API Costs

This latest version of Alluxio substantially reduces operational costs for organizations by minimizing cloud storage API and egress charges. Alluxio Distributed Cache reduces cloud storage API calls and data transfers lowering cloud storage costs while improving query performance.

‍

‍

3. Enhanced Reliability

Reliability gets a major boost in this new release with improved fault tolerance mechanisms. The system now features automatic fallback to the underlying file system, making it more robust and adaptable to Kubernetes and cloud environments. Read more about this feature in the I/O resiliency documentation.

4. Improved Ease of Use

This release introduces Kubernetes-based deployment enhancements, including support for rolling upgrades, making it even easier to manage Alluxio in container orchestration environments. Enhanced metrics visualization provides deeper insights into system performance and resource utilization. The addition of RESTful cache control APIs on DORA gives administrators more flexible and programmatic control over the caching layer, further simplifying management tasks. Read more about Kubernetes integration starting with the install documentation among other pages in the same section.

Try or Upgrade to Alluxio Enterprise for Data Analytics 3.2 Today

Get a personalized demo and see how Alluxio can transform your data infrastructure.

‍

Schedule a Demo Today!

‍

For an exhaustive list of major features in Alluxio Enterprise for Data Analytics 3.2, please refer to our release notes.

Join our community Slack channel with over 10,000 members to ask questions and provide feedback: https://alluxio.io/slack.