Google BigQuery Data Replication

When faced with a performance issue that couldn’t be resolved with an ODBC-based solution, a global leader in online auctions and remarketing services reached out to Stelo for support.

With a focus on facilitating purchases and sales through virtual auction technology, the company seeks to create a seamless marketplace that can process millions of transactions annually, support real-time bidding, and manage an extensive inventory.

The company works with both structure and unstructured data, including listings, photos and videos, customer communications, transaction records, customer profiles, and inventory statuses.

Back in 2015, the company engaged Stelo to leverage SQDR, now Stelo V6.3, for data replication from their IBM iSeries system, now IBM i, to MariaDB as part of a multi-year plan to gradually migrate their data infrastructure. With Stelo’s technology and support, they’ve been successful in this portion of their transition strategy. Most recently, the company decided to add Google BigQuery into the mix.

“Our partnerships with both Google and the company have been exceptional. At Stelo, we believe proactive communication and dedicated support are the keys to delivering impactful solutions.”

Do Nguyen

Engineering Manager at Stelo

Need

In revising their data management strategy, the company needed a partner that could replicate their structured and unstructured data from both IBM i and MariaDB for enhanced data analysis capabilities.

Solution

After it became clear that an ODBC-based solution wouldn’t be suitable, Stelo supported the company through an exploratory phase where we assessed a variety of solutions. Google BigQuery’s API was selected for its ease of use, scalability, support for real-time analytics, and cross platform compatibility. From there, Stelo applied change data capture (CDC) expertise alongside Google’s engineering expertise to fine-tune the solution using the Storage Write API.