logo for information-management-architect.com
leftimage for information-management-architect.com

Data Warehouse ETL Solution

Need an information management data warehouse ETL solution and want practical standards and best practices for data governance?

What is a data warehouse?

A data warehouse solution should be based on the following information management disciplines:

  • Information management;
  • Project management;
  • Data management; and
  • Change management.
What is required for a data warehouse data movement solution?

Data movement solutions should be based on:
  • A project management framework; and
  • A data movement discipline

What is a project management framework?

This framework defines roles and responsibilities, tasks and deliverables for the following data warehouse project phases:
  • Business case analysis;
  • Project planning;
  • Requirements analysis;
  • Architecture and design;
  • Development;
  • Quality assurance testing; and
  • Transition to production.
What is a data movement discipline?

The data movement (ETL) discipline focuses on the movement of data between systems. They are primarily involved in the following project phases:

  • Requirements analysis;
  • Architecture and design;
  • Development;
  • Quality assurance testing; and
  • Transition to production.
Data Movement Team Project Responsibilities
Data Warehoouse ETL


The data warehouse ETL team or data movement discipline is involved in with the following project phases, tasks and deliverables:

Requirements Analysis
  • Data flow diagram;
  • Source system definition;
  • Interface definition;
  • Data conversion requirements;
  • Change data movement requirements;
  • Data requirements; and
  • Data profiling.
Architecture & Design
  • Data movement model;
  • Interface specifications;
  • Data acquisition and load strategy;
  • Failure management and error handling;
  • Data conversion strategy;
  • Data reconciliation approach; and
  • Source to target mapping.
Development
  • Data movement code;
  • Automated ETL process;
  • System processes code;
  • Data quality code;
  • Unit testing; and
  • System testing.
Quality Assurance Testing
  • Integration testing;
  • Performance testing; and
  • Load testing.
Transition to Production
  • Release test; and
  • Transition to production.
What is the scope of data warehouse ETL?


Data Warehousing Scope
Data Warehouse Architecture

What design and  development work is involved?

The data warehouse data movement team is involved with design of:
  • Source systems extract;
  • Staging area; and
  • Clearing house.
They are involved with all aspects of design, development and testing data movement among and between the following environments:
  • Source systems;
  • Staging area;
  • Data Warehouse;
  • Clearing house; and
  • Data marts.

Summary....

A data warehouse ETL solution should be based on an approved data warehouse architecture and the following management frameworks:
  • Information management;
  • Data management;
  • Project management; and
  • Change management.
Data movement solutions should be based on:
  • A project management framework; and
  • A data movement discipline
Key responsibilities for data movement solutions include:
  • Information management architect, who  has overall responsibility for designing the management frameworks;
  • Data architect, who has overall responsibility for data warehousing solution architecture; and
  • Data movement designers and developers, who have responsibility for design, development and system and integration testing.




footer for Information management page