Penn Foster Delta Lakes Case Study

Penn Foster Deploys Stelo V6.1 for Data Replication to Azure Databricks

Penn Foster is an online career training and educational institution whose mission is to help students gain the knowledge and skills they need to advance in their field or start a new career. With growing enrollment, the institution decided it was time to transition from a traditional, relational data management solution to a cloud-based, big data solution.

Penn Foster predominantly works with structured data (e.g., student, financial data) but expects to incorporate unstructured data like video and audio from a variety of sources in the future. Integrating different sources and data types will offer their data scientists the chance to better leverage machine learning to identify prospective students and improve learner analytics.

Back in 2015, Penn Foster engaged Stelo to leverage our software for data replication from their on- premises IBM for i (formerly iSeries) to Microsoft SQL and MySQL for change data capture (CDC) data warehousing. To manage tables of up to 70 million rows, performance was a key criterion. Even with a new baseline replication, the solution was fast and reliable, inspiring confidence in Stelo’s technology and support team. In 2022, Stelo supported the migration from an on-prem SQL server instance to an Azure-based environment. As promised, Stelo managed upgrades and maintenance to support their evolving data management strategy.

Penn Foster was able to make both of those transitions with their existing Stelo infrastructure—an entry-level production deployment expected to handle up to 1 million transactions per hour (TPH) and a siloed quality assurance (QA) deployment. Most recently, the team at Penn Foster decided to move to Azure Databricks.

“Stelo was willing to engage with us on our unique needs, and the team was very responsive when we asked for extended functionality.”
Stelo IT Director Dan-1
Stephen Tiepel

Senior Director of Data

Need

As part of their most recent data transformation process, Penn Foster needed a partner that could bring their structured and unstructured data into the Databricks platform without intensive coding efforts from their engineering team.

Solution

Initially, all files were dropped into Microsoft Azure Data Lake Storage (ADLS). It became clear that cycling through individual files downstream was more of a coding exercise than desired. Armed with this feedback, Stelo adjusted the technology roadmap, offering Penn Foster a pre-release deployment of Stelo V6.1. Given Stelo's proven technology and outstanding support, Penn Foster accepted with the intention to leverage the newest functionality of Stelo V6.1—delta lakes support.

“With us moving from standard CDC capture to capturing lossless data, I was concerned about scalability. I can tell you scalability hasn’t been a problem.” - Stephen Tiepel, Senior Director of Data at Penn Foster

Featured Technology

  • Azure Databricks is a data analytics platform that’s been optimized for Microsoft Azure cloud services. It offers all the benefits of a cloud-based solution—built-in redundancy, consumption-based cost models, versatility, power advantages, and incremental development possibilities.
  • Stelo is a flexible, high-performance data integration solution for automatically merging high-volume enterprise data streams (structured and unstructured) into a delta lake without cumbersome coding. Stelo V6.1 includes support for non-SQL destinations, support for Linux, user role-based authentication control (RBAC), PowerShell Scripting, and backwards compatibility for relational database management system (RDBMS) destinations.

Learn More

SCHEDULE A DEMO

Results

Confidence

Proof of Concept

Stelo's targeted replication is proving out Penn Foster’s cloud-based architecture at scale.

Integration

Delta Lakes Support

Combining technologies and integrating an open-source storage layer offers faster data access, faster updates, and improved data reliability.

Focus

Efficiency

By focusing on configuration, Stelo V6.1 minimizes hands-on engineering efforts.

Benefits

Time and Labor Icon_Noun 4636078_White Cost Reduction

By moving from a traditional on-premise solution to a consumption-based, cloud-based solution, Penn Foster’s investments are more targeted in scope.

Integrated Data Insights Icon_Noun 2198129_WhiteBetter Support for the Learner Analytics Team

With more meaningful insights, Penn Foster can (1) better identify and serve prospective students and (2) track success rates to support enrolled students on their individual paths to graduation.

Data Scalability Icon_Noun 1304652_WhiteFuture Proofing

With Stelo, Penn Foster integrated technologies that allow them to continue advancing their data management strategy without displacing their current investments.

Get Started

These three steps will help you ensure Stelo works for your needs, then seamlessly deploy your solution.

1

Schedule a Demo

Our expert consultants will guide you through the functionality of Stelo, using your intended data stores.

2

Try Stelo

Test the full capability of the software in your own environment for 15 days. No obligations.

3

Go Live

When you're ready, we can deploy your Stelo instance in under 24 hours with no disruptions to your operations.

SCHEDULE A DEMO