iQ Media, a company delivering its cloud-based media intelligence platform called cliQ which lets marketers and media analysts, has implemented Caringo Swarm scale-out cloud & object storage platform for storing over 6 petabytes of video files, primarily TV broadcast clips dating back to 2011, growing at a rate of 6 terabytes per day.
iQ Media maintains a cloud-based media intelligence platform called cliQ which lets marketers and media analysts quickly find and measure the impact of various campaigns and branded or topic-related content over an extended period.
With the cost curve of traditional storage technologies straining iQ Media’s budget and only two IT staff members available, iQ Media CTO and co-founder Lakshmi Venkataswamy was looking for a storage solution that was appropriately priced, seamlessly scalable, and low-maintenance. The solution needed to extract every cent of value from the underlying hardware by enabling it to operate well beyond its warranty coverage without increasing the risk of data loss.
Mr. Venkataswamy and his staff first installed Windows Storage Server, an installation that consisted of 15–16 clustered Windows Storage Nodes. As the size of hard disk drives grew from 2TB and 3TB capacities, RAID rebuilds of failed drives increased in time. They then moved on to using a NAS array with an expansion chassis, but that solution quickly ran out of capacity. Their continued search for a hassle free, scale-out storage solution led them to Caringo.
In September of 2015, Mr. Venkataswamy and his team started with three Caringo nodes, migrating data to them from their Windows Servers. “We could just plug a new node in and it would boot itself up automatically,” said Lakshmi Venkataswamy. “There was no prep work. We had no OS installs. We could migrate disk groups between nodes and Swarm would decide if the disk was part of the node or not and move it around to the correct node.”
The benefits that resulted in selecting Caringo over other object storage solutions like SwiftStack, Hitachi Content Platform and HGST (Amplidata) were:
- Drives could be replaced and automatically be recognized by the cluster and load-balanced without much human intervention.
- Caringo Swarm was storage-agnostic, letting Venkatswamy use storage nodes from any vendor at prices and capacities he chose.
- Files stored in the Caringo Swarm cluster could be automatically protected in the event of drive failure by employing Caringo’s 5:2 erasure coding, and avoiding the cost and complexity of replication.
- FileFly automatically moved files from filers to Swarm with no modification to the existing application.
A more detailed description of the benefits provided by Caringo software is in the complete use case available here.