Alluxio, developer of open-source cloud data orchestration software, has released Alluxio 2.5 of its cloud data orchestration platform. The latest version comes with access via POSIX and S3 interfaces, enabling data platform teams to accelerate data pipelines for both business intelligence and model training using frameworks such as Tensorflow and PyTorch. It is intended to help improve compatibility and performance with Machine Learning data pipelines.
Version 2.5 of Alluxio provides cloud data platform teams the ability to accelerate data pipelines for model training with frameworks like PyTorch or Tensorflow, while it can also be used for business intelligence.
Alluxio believes they have significantly improved the support for model training pipelines with this new version. This is due to the accelerated POSIX API used to unify storage access, ease of management, and performance. Improved machine learning and better AI support with Kubernetes are a part of Alluxio 2.5, along with new enhancements to the FUSE interface. All of these upgrades help to make the model training experience better.
When running analytics jobs, such as Spark in Tencent Cloud, the Alluxio Data Orchestration System helps cut query run times significantly. The new version also provides better compatibility with S3 API.
Improved Storage Connectors
Alluxio Version 2.5 gives users better storage connectors with Azure Cloud, Google Cloud, and AWS Cloud platforms. The onboarding process would be smoother and the overall performance has also become better, according to Alluxio. The software solution can “easily” integrate data lakes from top cloud platforms to provides better data management. A QuickStart guide is provided for multi-cloud, single-cloud, and hybrid-cloud orchestration, along with support on Kubernetes.
– story continues below the image –
New Features from the Alluxio 2.5 Community and Enterprise Edition
The Alluxio 2.5 Community and Enterprise edition provides new features including the JNI Based POSIX API. This JNI-based FUSE integration would provide up to 5 times better performance with high-concurrency workloads.
Another feature is the S3 Northbound API, which provides better compatibility with S3 browsing and improved support.
The ADLS Gen2 Connector found in the new version of Alluxio would help to improve support for the Azure cloud. The connector for the Azure Data Lake Storage Gen 2 was introduced in this version providing even more benefits when using Azure object storage.
This new version also offers updated connectors for Google Cloud Storage and Amazon Web Services. The Orchestration Hub from Alluxio would make it easy to gain the support necessary on Kubernetes. It helps to aid in cluster configuration and connectivity with public clouds or private data centers.
As a creator of open-source data orchestration software for the cloud, Alluxio uses intelligent data tiering and data management to deliver higher performance for those in the telecommunications, retail, financial services, and high-tech industries. Alluxio is used by many of the top internet companies and has been backed by several partners including Seven Seas Partners and Volcanics Venture. It was founded at the UC Berkeley AMPLab.