Improving operational integration

The Problem

Scott Technology uses several ERP systems to run various aspects of their business in NZ and Worldwide. The systems are a mix of on-premise and cloud-based solutions. The absence of integration between the different systems means that data often needs to be entered more than once across systems, or manually exported out of one system and then imported into another. There is also an increased reliance on Excel for integrated data reporting and process monitoring.

Scott Technology wanted a solution that would:

• Improve operational integration between the various systems they use.

• Create a platform for data integration and warehousing that will:

  • help improve process integration and/or automation

  • fill gaps in ERP systems reporting functionality

  • enable data sharing between Scott Technology companies, business units, and other parties

  • enable reporting on different levels (consolidated, regional)

  • adapt to changing ERP systems landscape and assist in providing continuity during transitions.

• Automate the extraction of data from data sources into data integration and warehouse platform.

 The Solution

The solution is an Azure-based data integration and warehouse platform that consolidates ERP systems data. The platform uses Azure Data Factory to automate ELT pipelines that move data from on-premise and cloud-based sources into a data warehouse hosted on Azure SQL where further transformation, integration and other data operations can be performed.

The data integration and warehouse platform implement metadata-driven pipelines to automate the ingestion of data from the source through to the various data layers in the data warehouse. Metadata-driven pipelines provide Scott with the ability to rapidly ingest new data sources through configuration changes with minimal pipeline development.

Two environments were set up, one for development and another for production workloads. DevOps is used for continuous integration and deployment across environments.

The Outcome

The initial benefit of having deployed the cloud-based data platform is that they are now able to reliably report on their ERP data in Power BI.

In the long term, the data platform has given Scott Technology a way to integrate and analyse its data in a secure, reliable, and accessible platform that can adapt to its ever-changing systems integration and reporting needs.