logo for information-management-architect.com
leftimage for information-management-architect.com

Release Testing

Improve deployment process with David Bowman’s information management guidelines for data warehouse release testing

This site is designed for information technology professionals and need to improve deployment process and want guidelines

It provides a checklist of information management guidelines for release test

What is a software release?

Deploying a release into the production environment involves risks to the availability and reliability of that environment.

All stakeholders need to be aware of the potential risks involved in the deployment.

Recognizing this, a release manager should ensure that the appropriate managers agree on and sign off on a release plan before the release moves into the design and build phase.

What is a release test?

The objective of a production support user acceptance test is to certify that an information management solution meets production support user acceptance expectations and is considered ready for production.

Scope
  • Should focus on testing a set of requirements (that is expected to move to production as a Production Release) to ensure that it meets user expectations.
  • Should certify that the solution meets production support requirements.
Release Testing Entrance Guidelines
  • Should commence upon approval of the user Acceptance Test Lead, Quality Assurance Test Lead and Data Architect/Project Manager who should ensure that test entrance criteria have been met. 
  • Should verify that a configuration management process has been established and documented to handle requested changes to project documentation
  • Should verify that the code migration process has been documented and arrangements made to accommodate testing.
  • Should verify that a turnover package has been prepared and all code changes are identified in and made available to the Requirements Manager and test team. This includes database changes, changes to install scripts and instructions, structure changes, etc.
  • Should verify that all code to be tested is locked-down and ready to be moved into the test area.
  • Should verify that a testing environment baseline has been updated with the release baseline. (i.e. the code and program versions in the acceptance test environment "mirror" the production environment).
  • Should verify that all release team members have been involved in project meetings since the quality assurance and User acceptance testing
  • Should verify that at least one acceptance test team member is on all applicable project communications lists.
  • Should verify that the requirements specification has been base lined, is under configuration control, and has been made available to the release team for review.
  • Should verify that all design documentation has been base lined, is under configuration control, and has been made available to the release team for review.
  • Should verify that all development resources have been identified and assigned to support the testing effort and are listed in the Test Plan.
  • Should verify that independent code reviews and review/checkpoints have been completed and documented.
  • Should verify that unit testing results have been documented and reviewed by the testing team.
  • Should verify that Unit test defects have been fixed, validated or otherwise properly finalized, and documented.
  • Should verify that integration testing results have been documented and reviewed by QA Test Team.
  • Should verify that integration test defects have been fixed, validated or otherwise properly finalized, and documented.
  • Should verify that quality assurance testing results have been documented and reviewed by the UAT Test Team.
  • Should verify that quality assurance test defects have been fixed, validated or otherwise properly finalized, and documented.
  • Should verify that a development team representative from each subject area has been identified and assigned to support the release testing effort (and is listed in the Test Plan).
  • Should verify that all required support files (such as account lists, flat file inputs, ini files, etc.) have been documented in the Release Object Migration Plan.
  • Should verify that the acceptance test defect management process has been documented and implemented.
  • Should verify that a the release team has been trained in and have appropriate access to the defect management tracking tool.
  • Should verify that a defect manager has been identified, allocated, and trained.
  • Should verify that a Test Plan has been completed and approved.
  • Should verify that acceptance test cases and scripts have been completed and approved.
  • Should verify that a unique acceptance test environment has been reserved and under Configuration Management control.
  • Should verify that a the test environment has been verified as accessible and functioning including connectivity to all applicable systems.
  • Should verify that all applicable test tools have been installed, verified as functional, and user guide/documentation up-to-date.
  • Should verify that proper user ids and permissions required for testing have been obtained and verified.
  • Should verify that test data has been received and pre-tested for validity.
  • Should verify that deployment and installation instructions have been successfully executed and validated.
  • Should verify that contacts have been assigned for their various component responsibilities and are listed in the Test Plan.
  • Should verify that all applicable service level agreements are in place and updated if needed, including clear ownership of Incident Resolution.
  • Should verify that any training required by the acceptance test team has been completed.
  • Should verify that any waivers or exceptions to the above have been documented and approved.
  • Should verify that a release schedule has been established and documented. This schedule must include periodic planned builds for defect fixes while in test.
Test Cases

Should ensure that acceptance test scenarios are documented in the release plan.

Test Execution

Should ensure that acceptance testing is performed in the testing environment by team members assigned from production support.

Test Data
  • Should be based on a sub-set of production data, produced by the production support team.
  • Should not be augmented as it is should be assumed that user  testing has certified that the release meets business requirements.
Defect Management
  • Should ensure that  defects are identified, and recorded in the defect management tracking tool, e.g HO Quality Center
  • Should ensure that defects are identified as either procedural, i.e. the documented procedures in the release plan do not produce the desired results or cannot be followed by the release team or as technical defects i.e. the object migration plan is incomplete or there are technical issues with migration code/scripts.
  • Should be assigned to the release manager who should determine impact on project deliverables, approve/reject change; and create and manage Change Requests as appropriate to ensure that requirements are changed, reviewed, and approved before commencing technical analysis design, build, test and deploy.
  • Should follow project methodology  e.g. analysis, design, build, review/checkpoint process before defects are certified ready for re-testing in the release test environment.
  • Should manage defect resolution
Release Testing Exit Guidelines
  • Should be considered complete upon sign-off and approval of the Production Support/Release Manager, Project Manager, UAT Test Lead and business owner who shall ensure that release test exit criteria have been met.
  • Should ensure updates to test documentation have been completed and are under Configuration Management  control.
  • Should ensure all defects have received a final disposition (i.e. open, closed, postponed, invalid, duplicate).
  • Should ensure all closed defects have an assigned defect root cause. 
  • Should ensure sign-off has been obtained from designated stakeholders indicating test completion.
  • Should ensure other test completion and validation conditions, as specified in the Test Plan, have been met.
Summary...

The objective of a release test is to certify that a release meets production support user acceptance expectations and is considered ready for production.

The release plan should identify user acceptance test roles and responsibilities, tasks, deliverables and entrance/exit criteria.

This site provided a checklist of information management guidelines for release testing.