From scheduled queries to robust dbt transformations
How wholesomeco director of analytics partnered with systematik to augment their staff and migrate to dbt.
“I would definitely recommend systematik. Our goal was to consult with experts on how to implement dbt, and build a robust, modern data architecture, and they do that really well.”
The problems
Lack of scalability
Based on BigQuery scheduled views, the existing data pipeline was not built to scale, limiting wholesomeco’s ability to adapt to increasing data demands.
Poor maintenance and error-prone
The lack of version control introduced inconsistencies and errors, complicating maintenance efforts.
Inefficient costs
Rising costs were becoming a concern, as the existing pipeline was not optimized for the scale of data being processed.
Absence of standard definitions
Without standard definitions for metrics, departments faced inconsistencies and misunderstandings in interpreting data.
Poor documentation and data quality monitoring
The initial system had limited documentation and lacked data quality monitoring features, leading to poor data quality and redundancies.
The solutions
Fast deployment through staff augmentation
One of the key reasons wholesomeco partnered with Systematik was to speed up the deployment process. systematik augmented wholesomeco’s team with their data engineers, achieving a swift transition to the new dbt-based pipeline.
Transitioning to dbt
Systematik’s dbt consulting helped wholesomeco transition smoothly, creating a scalable, robust pipeline.
Data quality monitoring
The team implemented data tests and quality monitoring within dbt, ensuring reliable, high-quality data.
Cost-efficiency
By streamlining queries and reducing redundancies, the new system reduced operational costs.
Comprehensive documentation and standard definitions
Systematik also helped create a detailed documentation framework and standard definitions for each metric, ensuring consistency across the organization.
The results
By leveraging systematik’s dbt expertise and staff augmentation services, wholesomeco was able to achieve a rapid and efficient transition to a dbt-based data pipeline. This new system not only met but surpassed the goals set by their director of analytics, mitchell moody. The benefits were manifold: faster deployment, greater scalability, fewer errors, cost savings, and improved data quality. Furthermore, the introduction of comprehensive documentation and standard definitions have ensured operational consistency across the organization.
All in all, the partnership with systematik marked a pivotal advancement in wholesomeco’s data management capabilities. If you’d like to hear more, watch mitchell moody’s video testimonial. He shares his own experience on the project’s success.
