We are extremely excited to announce the release of Alluxio 2.2.0!
With this release comes the General Availability (GA) of Alluxio Structured Data Services (SDS), the subsystem of Alluxio responsible for managing and transforming structured data, such as databases, tables, and partitions.
Alluxio 2.2.0 also contains several major improvements to the stability, reliability, and performance of the core system. It is intended to be backwards compatible with previous releases on the Alluxio 2.x line.
Downloads can be found here. Thanks to everyone who has contributed to this release! And here are some release highlights:
Alluxio Structured Data Service (SDS)
The Alluxio SDS provides database, table, and partition level metadata to compute frameworks, such as Presto. Based on the data access pattern or explicit user request, it invokes the transformation engine to transparently reorganize data to best fit the workload’s needs, which can provide up to 5x performance improvement.
Alluxio 2.2.0 marks the GA of SDS for read-only workloads. A native integration with Presto is available in the latest Presto release. For older versions of Presto, the connector can be found in the Alluxio release artifact.
Welcome to try it out by reading more about how to get started with Alluxio's structured data service in the documentation!
Alluxio Data Service
The new data service includes two main components - the Data Service Engine and Data Service Monitoring.
Data Service Engine
The Data Service Engine has been rearchitected to support high performance and scalability required in large scale deployments, especially when utilizing Alluxio SDS.
Data Service Monitoring
In this release, Alluxio Data Service also includes a new CLI for querying information about the state of the job service and throttling capabilities based on the load a node is experiencing. See the documentation for more details.
Alluxio Core
Journal
Online journal backups are now available. A secondary master will take a backup of the journal while the primary master continues to serve requests, allowing for backups to be taken with no downtime.
DevOps - Log Collection
The Alluxio CLI packages a cluster diagnostic tool, `collectInfo`. This allows users to easily gather relevant cluster information for debugging purposes. See the docs for more information.
You can see full details of Alluxio 2.2.0 in the release notes.
As a next step, take a look at our first blog in the Serving Structured Data in Alluxio series.
Enjoy the new release and look forward to hearing your feedback (community slack channel)!
Bin, Calvin, Gene, Zac and the Alluxio Developer Community
.png)
Blog
We are thrilled to announce the general availability of Alluxio Enterprise for Data Analytics 3.2! With data volumes continuing to grow at exponential rates, data platform teams face challenges in maintaining query performance, managing infrastructure costs, and ensuring scalability. This latest version of Alluxio addresses these challenges head-on with groundbreaking improvements in scalability, performance, and cost-efficiency.
.jpeg)
We’re excited to introduce Rapid Alluxio Deployer (RAD) on AWS, which allows you to experience the performance benefits of Alluxio in less than 30 minutes. RAD is designed with a split-plane architecture, which ensures that your data remains secure within your AWS environment, giving you peace of mind while leveraging Alluxio’s capabilities.
PyTorch is one of the most popular deep learning frameworks in production today. As models become increasingly complex and dataset sizes grow, optimizing model training performance becomes crucial to reduce training times and improve productivity.