logo for information-management-architect.com

Data Architecture

Improve roadmap planning process with David Bowman’s information management guidelines for data architecture assessment

This site is designed for Information Technology professionals who need to improve roadmap planning process and require guidance and direction to help teams consistently produce error free results.

It provides a checklist of information management guidelines for data architecture assessment.

What is Data Warehpuse Architecture?

Data warehouse architecture describes how data is stored and used at every step of the data warehousing and business intelligence process from the operational systems to the analytic applications. 

The following diagram depicts a typical data warehouse and business intelligence environment.

data warehouse architecture

Information management solutions are usually involved with some, or all of, the following:
  • Extracting data from operational source systems;
  • Transforming data as required to ensure consistency;
  • Loading data into a data warehouse or some form of storage optimized for efficient storage;
  • Extracting data from the data warehouse;
  • Loading departmental data marts optimized for reporting and analytics; and
  • Some form of reporting tool architecture e.g. universe design or some middle ware design to insulate users from the data warehouse.
Information is a valuable corporate asset and getting the right information, to the right people at the right time is a key management objective.

Achieving this objective requires robust technology and optimal data architecture based on industry best practices.

What is Data Architecture Roadmap Planning?

The objective of a data architecture assessment is gain a good understanding of the current data architecture and database management technology and compare this with best practices to determine what changes are required.

The plan to achieve these changes is called a roadmap and the tasks involved with creating this plan are called roadmap planning.

Roadmap Planning Consulting Firms

Organizations frequently engage Information Technology management consulting firms to help with data architecture roadmap planning.

They usually provide a team which includes a solution architect or management consultant with extensive experience in data warehouse or business intelligence planning, architecture and delivery. These resources are highly skilled and bring many years of successful architecture experience.

A solution architect must be supported by a set of well established data architecture best practices that are typically only available from  Management Consulting firms that specialize in information management and data warehouse solutions.

Consulting firms such as Deloitte Consulting, Ernst & Young, KPMG and Accenture have evolved data warehouse best practices and usually have the internal expertise required to assess data architecture and complete roadmap planning.  

Acquiring new technology and/or upgrading existing technology is an expensive proposition. Organizations seeking  data architecture recommendations should look for consulting firms that provide vendor neutral recommendations. These will tend to be objective and unbiased.

Data Architecture Assessment Process

Should determine the technical readiness for information management by comparing the current situation with the following checklist of information management guidelines.

Data Modeling Tools
  • Should capture all artifacts related to the design of logical and physical data models;
  • Should be used to generate the physical data model from the logical data model using the tool along with translation rules for selective denormalization;
  • Should use a standard industry logical data model as a foundation for the modeling efforts e.g. an enterprise logical data model (ELDM) provides a roadmap for populating content into the information management environment. Some organizations have developed their own models and others may have purchased a commercial model specific for their industry. Companies such as ARDM Software, Teradata and IBM Global Services have well established enterprise logical data models that can provide a jump-start to an ELDM;
  • Should provide common warehouse metamodel format to export meta data from the design tool into downstream recipients e.g. data movement technology, business intelligence tools, metadata management tool; and
  • Should consider common tools such as ER/Studio from Embarcadero, ERWin, from CA, PowerDesigner, from SAP, or Rational Rose Data Modeler, from IBM or other lower cost options, such as Oracle SQL Developer Data Modeler, which is a free download.
Relational Database Platform
  • Should provide scalable hardware using an ANSI SQL compliant relational database;
  • Should provide separate systems for development, test and production;
  • Should provide the capability to implement change without impacting the production environment;
  • Should provide a test environment to test new versions of software before promotion into the production environment; and
  • Should consider common relational database management systems such as DB2, from IBM, Oracle, Teradata, SQL Server, from Microsoft, and other specialized systems such as Aster Database, from Teradata, SQL Server Parallel Data Warehouse, from Microsoft, Sybase IQ, from SAP, Appache Cassandra, Greenplum, from Pivotal and Hadoop or Hortonworks.
Metadata Repository
  • Should provide the capability to store design metadata;
  • Should provide the capability to store business metadata;
  • Should provide the capability to store technical metadata;
  • Should provide the capability to store operational metadata;
  • Should provide the capability to store semantic layer meta data;
  • Should provide the capability to exchange metadata using common warehouse metamodel standards;
  • Should provide the capability to provide visibility to data currency, data quality, data definitions, and data lineage;
  • Should provide the capability for knowledge workers to browse metadata; and
  • Should consider common metadata management tools such as Informatica or Rochade, from ASG Software Solutions
Security
  • Should use LDAP, or an equivalent standard, to manage a centralized directory of security and authorization information used by the business intelligence environment, database level and applications level, as well as other Information Technology  infrastructure within the enterprise;
  • Should use role-based security so that individuals have access rights in alignment with their job functions, rather than to all data;
  • Should have row and column level security for all data warehouse database platforms; and
  • Should encrypt anything that is sensitive or governed by regulations such as HIPPA.
Backup and Recovery
  • Should specify expectations for backup and recovery service levels to ensure that recovery from failures is timely and free of any significant data loss;
  • Should specify expectations for any specific form of transportable media for system data back ups;
  • Should specify expectations for how frequently and when the system data should be backed up; and
  • Should specify expectations for how critical system data should be backed up more frequently.
Disaster Recovery
  • Should specify expectations for disaster recovery service levels so that mission critical systems have plans to recover from natural disasters within a reasonable amount of time;
  • Should specify expectations for the disaster recovery time required.
Service Level Agreements
  • Should be documented in detail for performance, availability, data currency, and data quality;
  • Should be negotiated with business users and should be used as the basis for capacity planning and architecture considerations;
  • Should be used to measure performance of actual service level delivery; and
  • Should be reviewed with business on a quarterly basis.
Methodology
  • Should include an Enterprise logical data model e.g. an enterprise logical data model (ELDM) provides a roadmap for populating content into the information management environment;
  • Should include logical data model tasks e.g. data modeling is a critical part of any information management solution and data models should be available for existing systems and should be created for new applications and data warehousing solutions. Logical data models should be based on the enterprise logical data model;
  • Should include physical data model tasks e.g. physical data models should be available for each database environment and the models should be synchronized with the database. These models should be derived from, or based on, a translation of the logical data model; and
  • Should include a semantic layer e.g. semantic models are a combination of dimensional models or views that have been created for end user access. Users should be provided access to the semantic layer but in most cases, only power users should have the ability to directly query the physical  database.
Organization
  • Should have resources accountable for solution architect roles and responsibilities;
  • Should have resources accountable for data architect roles and responsibilities;
  • Should have resources accountable for data analyst roles and responsibilities;
  • Should have resources accountable for database administrator roles and responsibilities; and
  • Should have resources accountable for metadata specialist roles and responsibilities.
Data Architecture Guidelines
  • Should segment data warehouse components such as data integration, reporting or analysis;
  • Should support the Information Technology subject area which is just like any other business function and has similar analytic requirement;
  • Should provide an optimal balance between data storage, data integration and data access e.g. the best structure for data storage is often a poor choice for data access and vice-versa.
  • Should support business requirements;
  • Should provide both audit metadata columns and processing metadata columns to facilitate error detection, error resolution, merge/purge activities, archiving, unloading, and change data capture; and
  • Should make data models readily available  to everyone who needs them
Summary...

Data architecture describes how data is stored and used at every step of the data warehousing and business intelligence process from the operational systems to the analytic applications. 

Information is a valuable corporate asset and getting the right information, to the right people at the right time is a key management objective. Achieving this objective requires robust technology and optimal data architecture based on industry best practices.

This site provided a checklist of information management guidelines for data architecture assessment.