logo for information-management-architect.com
Home
Strategy
Framework
Business Case Analysis
Project Planning
Requirements Analysis
Architecture & Design
Build Phase
Quality Assurance
Transition to Production
Management Information
Business Intelligence
Data Warehouse
Tools
Jobs
Contact David Bowman
leftimage for information-management-architect.com

Data Conversion Projects

Need to plan data conversion projects for an information management requirements specification and want practical timesaving suggestions?

Data conversion is a key component of an information managementData Conversion Projects requirements specification.

Typical data conversion projects involve the creation of a data warehouse and the loading of history data, which the business owner hopes to use for historical trend analysis. 

I have seen some requirements for five years of history but three years might be a good average.

Sounds easy

If we can load the data warehouse with current data, why can’t we just use the same process to load history data?

Nothing is really simple these days. Even with home computers, we sometimes have challenges converting data. Remember the days when we had “floppy” disks and then we got a new computer that only used CD’s.

A lot of people decided to move all the data off “floppies” and store it on CD’s for back-up purposes. That’s when the fun started—The old files did not open properly, or the right version of software to read them was trashed long ago ….You are probably aware of some of the issues.

Now think of some of the things that might have happened to a computer system during the past three or four years.

The system may have been modified. Early versions of back-up data may have different data structures than later versions and documentation explaining these changes might be hard to locate.

Data quality improvements might have been introduced, and there could be different business rules that need to be considered to ensure quality data in the warehouse.

A company might have been acquired during the history period and there might even be two or three sets of the same related data e.g. billing data, and each set of data might have different ways of using reference data.

What does all this have to do with a requirement specification?

There are some fairly easy requirements that can be determined e.g.
  • Should data be excluded from conversion based on age?
  • Are there any audit reporting requirements for data conversion?
  • Do we need to maintain any linkages with the source system?
There are also some more difficult things e.g.
  • Interface specifications, although data conversion is usually a “one-time” effort, we still need to specify the interfaces and this might involve re-storing several versions of history data to get an idea of what’s involved—We might find that we have several interfaces for the same history file or table;
  • Data mapping needs to address any changing source data structures. It’s possible that data cleansing and data transformation might be required just to get the data into a usable format; and
  • Data quality and data profiling is a big concern. Data profiles on history data will need to ensure that possible changes in data quality are considered.
Data conversion requirements analysis considerations
  • Ensure that the business case clearly defines the use of history data as a business objective. It is very easy to have someone say that we need history data, and then find that the cost of obtaining it is far greater than any potential business benefit;
  • Consider history data conversion as a sub-project within the requirements analysis phase and “time-box” the work effort—It is very easy to spend a lot of time on requirements analysis and this effort needs to be carefully managed; and
  • Consider establishing a small design/development team specifically dedicated to this task.
Summary…

Completion of the requirements specification provides the project manager an opportunity to update project estimates. Data conversion projects and history data requirements must be identified early in the project to ensure clear understanding of the anticipated development work effort.  Anticipate technical challenges with history data and plan for them.




footer for Information management page