logo for information-management-architect.com
leftimage for information-management-architect.com

Data Warehouse Management 

Improve roadmap planning process with David Bowman’s information management guidelines for data warehouse management assessment

This site is designed for Information Technology professionals who need to improve roadmap planning process and require guidance and direction to help teams consistently produce error free results.

It provides a checklist of information management guidelines for data warehouse management assessment.

What is Data Warehouse Operations?

Data warehouse operations requires a dedicated production operations staff with appropriate tools for monitoring production, tracking incidents and responding to customer calls.

It also requires a specialized solution delivery methodology, quality assurance process and change management methodology.

Data Warehouse Management Assessment Process

Should determine the organizational readiness for information management by comparing the current data warehouse management situation with the following checklist of information management guidelines.

Performance Monitoring
  • Should provide visibility to workload executing in the data warehouse environment;
  • Should provide aggregated resource e.g. CPU, I/O, memory, locks, etc, with drill down to individual users, queries, etc. 
  • Should provide historical play-back of performance trends; and
  • Should include common tools such Database Performance Analyzer, from solarwinds.
Workload Management
  • Should provide gatekeeper, dynamic workload prioritization, and governors to manage workload executing in the EDW environment;
  • Should evaluate the priority and resource requirements for incoming queries and assess available resources for execution to decide if a query should be run immediately or be deferred;
  • Should efficiently allocate resources within the platform according to query demands aligned to defined service level goals for each query; and
  • Should provide governors to monitor for queries that exceed resource expectations or have performance issues.
Usage Analysis
  • Should provide automated tracking and reporting of data warehouse usage by user and application;
  • Should track resource consumption;
  • Should track detailed tracking down to the level of specific tables and columns within the data warehouse; and
  • Should track the use of historical data e.g. detailed data usage helps the database management team design archiving strategies and partitioning strategies.
Capacity Planning
  • Should analyze trends in resource utilization and service level compliance combined with workload profiling for current and future applications as input to modeling/simulation tools for capacity planning;
  • Should provide what-if analysis to perform sensitivity analysis across a range of workload assumptions;
  • Should suggest optimal investment strategies in terms of CPU, memory, disk spindle, etc. configurations for maximizing performance for each dollar spent.
Impact Analysis
  • Should provide the capability to determine the impact of any changes to source systems, target database structures in the data warehouse, or business transformation rules;
  • Should provide a listing of all extract programs, transformation rules, target table structures, and business intelligence tool impacts resulting from a change e.g. a column length  increases from six bytes to twelve bytes; and
  • Should typically be provided by the data integration technology e.g Informatica.
Incident and Problem Tracking
  • Should log all incidents in a logging tool with classification and a description of the incident;
  • Should provide search capabilities so that new incidents can be easily linked and tracked along with existing and previous incidents;
  • Should provide the capability to report patterns of repeated root causes; and
  • Should consider common tools such Service Desk Manager, from CA Technologies.
Defect and Change Request Tracking
  • Should provide the capability to document and track all defects and change requests;
  • Should provide the capability to document and track all production incidents;
  • Should provide the capability to review and categorize change requests to ensure completeness of the documentation e.g. defects should not be closed without adequate documentation of root cause; and
  • Should consider common tools such as ServiceDesk Plus, from ManageEngine.
Data Quality Management
  • Should provide the capability to automate data profiling;
  • Should provide the capability to measure data quality defects;
  • Should provide the capability to create alerts when data quality does not meet requirements;
  • Should provide the capability to report on trends;
  • Should provide the capability to track defects versus target service level agreements; and
  • Should consider common tools such as Informatica Data Quality, from Informatica.
Service Level Agreements
  • Should be documented in detail for performance, availability, data currency, and data quality;
  • Should be negotiated with business users and should be used as the basis for capacity planning and architecture considerations;
  • Should be used to measure performance of actual service level delivery; and
  • Should be reviewed with business on a quarterly basis.
Release Management
  • Should be a key component of data warehouse management.
  • Should provide a single point of control to coordinate new hardware and software releases into the EDW environment; and
  • Should provide coordination among various teams e.g., database administrators, data integration technical services, business intelligence technical services, etc. to ensure compatibility of all configuration items and that releases are coordinated
Security and Administration   
  • Should provide a centralized function to administrator user ids, passwords, roles, etc; and
  • Should ensure that LDAP, or an equivalent standard, is used to manage a centralized directory of security and authorization information
Service Desk    
  • Should provide assistance to business end users in using analytic applications and accessing the business intelligence environment; and
  • Should ensure that end users do not get direct access to business intelligence developers without some form of triage through a service desk function.
Change Management
  • Should provide a formalized team process to manage change in the hardware and software environments;
  • Should provide the capability to assess all changes and recommend how, when or if to proceed with changes;
  • Should plan the change; and
  • Should plan the change back out the event of unexpected results.
Business Intelligence Technical Services   
  • Should fix business intelligence incidents and problems that cannot be handled by the service desk; and
  • Should have trouble shooting skills combined with extensive business intelligence tool experience.
Data Integration Technical Services
  • Should fix data integration incidents and problems that cannot be handled by the service desk; and
  • Should have trouble shooting skills combined with extensive data integration tool experience.
Software Support
  • Should maintain the software infrastructure in the environment; and
  • Should manage software upgrades and patches to operating systems, databases, etc.
Dedicated Production Staff
  • Should share some production support staff with the operational systems;
  • Should provide specialized support;
  • Should include key operational resources for data warehouse, and business intelligence support
  • Should have more thorough knowledge of the data warehouse applications and databases than shared production support resources.
Summary...

Data warehouse management requires a dedicated production operations staff with appropriate tools for monitoring production, tracking incidents and responding to customer calls.

This site provided a checklist of information management guidelines for data warehouse management assessment.