Hybrid and Multi-Cloud, AI and Deep Learning, Services, Data Sharing and New Table Formats

As we near the end of 2021, it’s a good time to take a deep breath, think about what we’ve learned as well as the trends we’re seeing, and share our thoughts around the top data predictions for 2022. 

As more organizations advance their data revolution strategy, and run more diverse workloads on a wider variety of platforms across clouds and hybrid clouds, 2022 will see even more advances in AI, machine learning and analytic workloads and technologies and services to support them. There are five major trends I predict for 2022:

Hybrid Cloud a Reality & Multi-Cloud Strategy a No-Brainer

We’ve already seen a hybrid-cloud strategy with multiple data centers and public cloud providers emerge as the standard for large enterprises as the operational toolset continues to evolve and simplify cloud migrations. In 2022, we will see organizations grow their digital footprint by embracing the hybrid and multi-cloud model to enjoy elasticity and agility in the cloud, while maintaining tight control of the data they own. Cloud vendors will keep innovating and competing with differentiated capabilities in network connectivity and physical infrastructure improvements because organizations wouldn’t want being locked-in.

Mainstream AI and Deep Learning

As the toolset for AI applications continues to evolve, machine learning and deep learning platforms have entered the mainstream and will attain the same level of maturity as specialized data analytics. Just like we currently see a plethora of fully integrated managed services based on Apache Spark and Presto, in 2022 we will see vertical integrations emerging based on the likes of PyTorch and TensorFlow. MLOps for pipeline automation and management will become essential, further lowering the barriers and accelerating the adoption of AI and ML.

Services for Everything

Operational complexity was the demise of Hadoop on-premises. Cloud services offer the ease of elasticity of infrastructure provisioning with little operational hassle. In 2022, we will see the emergence of managed services not just for cloud environments but also hybrid-cloud and on-premises deployments to eliminate complexity from integrations of myriad components such as data catalog, data governance, computational frameworks, visualization and notebooks. 

Data Sharing Across the Cloud

With SaaS and managed services in the cloud creating more data silos, improved governance and catalog with a data fabric spanning multiple services will come to the rescue in 2022. Sharing data across tenants and multiple service providers efficiently and securely will make data exchange easier than ever before.

Rise of Table Formats for Data Lakes

New stacks in both the storage and the compute layers keep evolving and innovating. Data Lakes are rising to prominence and structured data is transitioning to new formats. In 2022, open source projects like Apache Iceberg and Apache Hudi will replace more traditional Hive warehouses in cloud-native environments, enabling Presto and Spark workloads to run more efficiently on a large scale.