logo for information-management-architect.com
leftimage for information-management-architect.com

Data Integration Design Specification

Improve architecture and design process with David Bowman’s information management guidelines for data integration design specification

This site is designed for information technology professionals and information management consultants who want to learn more about data warehouse and business intelligence architecture

It provides information management guidelines for data  integration design specification and template that can help jump-start a project.

What is a Data Integration Specification?


The objective of an data integration specification is to specify the detail extract, transform and load design so that development can commence.

Architecture Guidelines

Architecture Diagram

Should refer to the architecture diagram in the Architecture Specification.
Module Definition

Should refer to the module definition in the Architecture Specification.
Job Configuration
Job Scheduling
  • Should specify the schedule name;
  • Should specify the schedule file;
  • Should specify job step;
  • Should specify job name;
  • Should specify job usage;
  • Should specify return codes;
  • Should specify start time;
  • Should specify end time; an
  • Should specify failure action.
Job Configuration
  • Should specify inbound directory;
  • Should specify archive directory; and
  • Should specify distribution list.
Source Transport and Scheduling
  • Should specify transport protocol;
  • Should specify push/pull;
  • Should specify frequency;
  • Should specify start time;
  • Should specify end time; and
  • Should specify size/volume.
Source File Conventions
  • Should specify file naming convention;
  • Should specify file type e.g. delimited, COBOL, XML, etc;
  • Should specify delimiter;
  • Should specify header format;
  • Should specify trailer format; and
  • Should specify location of row count.
Source File Layout
  • Should specify field number or position;
  • Should specify field name;
  • Should specify data type; and
  • Should specify description.
Header Format
  • Should specify field number or position;
  • Should specify field name;
  • Should specify data type; and
  • Should specify description.
Trailer Format
  • Should specify field number or position;
  • Should specify field name;
  • Should specify data type; and
  • Should specify description.
Staging Area

ETL Tool Specifics e.g. Informatica
  • Should specify folder name;
  • Should specify work flow name;
  • Should specify session name;
  • Should specify mapping name; and
  • Should specify fields requiring formatting.
Expression/Transformation Logic

Should specify transformation logic
Stored Procedures
  • Should specify procedure name;
  • Should specify usage; and
  • Should specify pre/post usage;
Load to Clearing House or Each Load Area

Tool Specifics
  • Should specify folder name;
  • Should specify work flow name;
  • Should specify session name;
  • Should specify mapping name; and
  • Should specify fields requiring formatting.
Expressions
  • Should specify stage table column;
  • Should specify expression name;
  • Should specify logic; and
  • Should specify data mapping reference.
Look Up/Reference Tables (For each reference table)
  • Should specify source column;
  • Should specify source of input;
  • Should specify column;
  • Should specify in/out;
  • Should specify data mapping reference; and
  • Should specify null handling.
Data Integration Desihn Specification Success Conditions

Should list success conditions e.g. all rows loaded into??
Data Integration Design Specification Template

Under Construction  

Summary...

The objective of an data integration design specification is to specify the detail extract, transform and load design so that development can commence.

This site provided information management guidelines for data  integration design specification and a template that can help jump-start a project.