Integrating Open Source Alluxio in AWS EKS with Terraform
April 1, 2021
By 
Bin Fan
Vasista Polali

Alluxio is an open source Data orchestration platform that can be deployed on multiple platforms. However, it can require a lot of thinking and experience to integrate Alluxio into an existing Data Architecture adhering to minimally required DevOps principles meeting Organizational standards.

The presentation talks about the best practices to set up and techniques to build a cluster with open source Alluxio on AWS EKS, for one of our clients, which made it Scalable, Reliable, and Secure by adapting to Kubernetes RBAC.

Our speaker Vasista Polali will show you how to :  

  • Bootstrap EKS cluster in AWS with Terraform.  
  • Deploy open source Alluxio in a Namespace with persistence in AWS EFS.  
  • Scale up and down the Alluxio worker nodes as Daemon sets by Scaling the EKS nodes with Terraform.  
  • Accessing data with S3 mount.  
  • Controlling the access to Alluxio with Kubernetes port-forwarding, “setfacl” functionality, and Kubernetes service accounts.  
  • Re-using the data/metadata in the persistence layer on a new cluster.

Alluxio is an open source Data orchestration platform that can be deployed on multiple platforms. However, it can require a lot of thinking and experience to integrate Alluxio into an existing Data Architecture adhering to minimally required DevOps principles meeting Organizational standards.

The presentation talks about the best practices to set up and techniques to build a cluster with open source Alluxio on AWS EKS, for one of our clients, which made it Scalable, Reliable, and Secure by adapting to Kubernetes RBAC.

Our speaker Vasista Polali will show you how to :  

  • Bootstrap EKS cluster in AWS with Terraform.  
  • Deploy open source Alluxio in a Namespace with persistence in AWS EFS.  
  • Scale up and down the Alluxio worker nodes as Daemon sets by Scaling the EKS nodes with Terraform.  
  • Accessing data with S3 mount.  
  • Controlling the access to Alluxio with Kubernetes port-forwarding, “setfacl” functionality, and Kubernetes service accounts.  
  • Re-using the data/metadata in the persistence layer on a new cluster.  

Video:

Slides:

Integrating Open Source Alluxio in AWS EKS with Terraform from Alluxio, Inc.

Complete the form below to access the full overview:

Videos

Sign-up for a Live Demo or Book a Meeting with a Solutions Engineer