WANdisco recently announced that its LiveData Migrator platform can now automate the migration of Apache Hive metadata directly into Databricks to help users save time, quickly enable new artificial intelligence and machine learning capabilities along with reducing the costs.
Talking about the LiveData Migrator platform, it automates the migration and republication of Hadoop data from on-premises to the cloud. Enterprises that want to migrate Spark and Hadoop content to Databricks from Hive can do the same with high efficiency while mitigating the various risks associated with large-scale cloud migrations.
WANdisco’s LiveData Migrator platform benefits would include:
- Use a single pane of glass to manage both Hive metadata and Hadoop data migrations. Accelerate time to business insights by eliminating the requirement for manual data mappings with direct, native access to structured data in Databricks from on-premises environments.
- No need to migrate data sets in full before converting into Delta format. LiveData Migrator automates the incremental transformation to Delta Lake.
- All the live changes to source data and metadata are reflected immediately in Databricks’ Lakehouse platform, and on-premises data formats used in Hive and Hadoop are automatically made available in Delta Lake on Databricks.
Users can also eliminate migration tasks that previously required constructing data pipelines to transform, adjust and filter data by combining data, metadata, and making on-premises content immediately usable in Databricks. To execute this, users need significant up-front staging and planning. This would also save the efforts that would have been used to set up auto-load pipelines to identify newly-landed data and convert it to final form as part of the processing pipeline.
LiveData Migrator automates cloud data migration at any scale by allowing companies to “easily” migrate data from on-premises Hadoop-oriented data lakes to any cloud within minutes, even while the source of data sets is under active change. Without the requirement of consultants and the expertise of engineers, businesses can migrate their data to initiate their digital transformation.
LiveData Migrator works without any negative aspects like business disruption or production system downtime. On the other hand, it would also ensure that the migration is complete and continuous and any ongoing data changes are replicated to the target cloud environment.
WANdisco CTO’s Take
WANdisco CTO Paul Scott-Murphy said that this brand-new feature by the brand will bring the power of WANdisco and Databricks together. He further said that metadata and data are migrated automatically without any disruption or change to existing systems. He concluded his statement by saying that teams can implement their cloud modernization strategies without risk, immediately employing workloads and data that were locked up on-premises, now in the cloud using the Lakehouse Platform offered by Databricks.
Databricks VP’s Take
Databricks Vice President, Pankaj Dugar, said that Enterprises want to break silos and bring all their data into a lakehouse for analytics and AI but the on-premises infrastructure of these enterprises is restricting them from doing so. He concluded by saying that it will be much easier to take leverage of Databricks’ Lakehouse Platform with the new Hive metadata capabilities in WANdisco’s LiveData Migrator.