logo for information-management-architect.com
leftimage for information-management-architect.com

Software Testing Process

Improve quality with David Bowman’s information management guidelines for software testing process

Information management applications require:Software Testing Process
  • Standard test plan;
  • Test traceability;
  • Standard tests and evaluation methods;
  • Standards for describing tests; and
  • Standards for managing, and re-testing, defects.
What is included in a software testing process test plan?

A test plan should include the following:
  • Test objectives, should summarize project objectives, test objectives, scope and roles and responsibilities;
  • Test strategy, should refer to the requirements acceptance criteria, that were developed as part of the requirements definition phase, and the test environments that will be used;
  • Test plan, should list the scope of each test, including entrance criteria, how test cases will be prepared, what test data will be used, what test scripts are required, who will execute test scripts,  how defects will be managed, how test results will be managed, and test exit criteria;
  • Test data strategy, should list the overall approach for creating test data. Try not to use large volumes of data for unit, system, integration, regression and quality assurance testing. 
You can adequately complete these tests with a sub-set of data. Failure to do this will add significantly to the test period  and will not add additional benefit.

The extra time it takes to create a complete set of test data early in the project will be paid back many times by reducing the time to back-up/restore data during testing;
  • Test deliverables should specify what will be produced;
  • Resource plan should list all project roles and responsibilities, level of effort, and other test resource requirements;
  • Training considerations, should list any training that might be required so the test team can complete testing; and
  • Test schedule, should identify a test schedule that clearly defines when all testing is expected to occur—This schedule may be included in the project schedule.

What is a software testing process requirement traceability?

It is important to have a process to ensure that all requirements are tested and that time is not wasted testing for things that are not specified as requirements.

A requirements traceability matrix will normally outline all testing required for each requirement. These tests may involve software testing.
Possible methods for ensuring that requirements are accounted for in each succeeding lifecycle stage include:
  • Inspection, requires a careful, critical review of documentation, code and/or data to verify that a requirement has been addressed;
  • Analysis, examines alternative solutions in detail so that the nature and function of each alternative may be judged as to best fit for achieving the requirements.  One example is to analyze third party products to determine which of the products should be purchased.  Another example is to review business process metrics to determine if the business process has improved as hoped because of implementing a new solution;
  • Demonstration, a practical demonstration showing how something works to provide evidence that a requirement has been achieved via examples or experiments; and
  • Test, performs a pre-defined examination or trial consisting of one or more steps intended to ensure that a requirement has been properly implemented.
What are software testing process test and evaluation methods?

The following standard tests should be performed before new application code is moved to production.
  • Unit testing should focus on testing individual modules to ensure that they perform to specification, handle all exceptions as expected, and produce the appropriate alerts to satisfy error handling. This level of testing is normally performed in the development environment and is conducted by the software developer who develops the code. The test validates the module's logic, adherence to functional requirements and adherence to technical standards.
The objective if this test is to ensure that all module source coded has been executed and each conditional logic branch followed.

The test data and test results should be recorded and form part of the release package when the code moves to production.
  • Code review should focus on reviewing code and unit test results to provide additional verification that the code conforms to data movement best practices and security requirement. The code review should verify that test results confirm that all conditional logic paths were followed and that all error messages were tested properly
  • System testing should focus on testing a set of components  to ensure that it performs to specification.
System testing is generally performed by a senior developer or lead developer in a system test or integration test environment.

Errors, or defects, should be documented, classified by severity,  and monitored by the defect management process to ensure timely resolution.
  • Integration testing should focus on testing software release i.e. a set of components intended to move to production as a planned release.
Integration testing is generally managed by the data architect or software designer in an integration test environment which "mirrors" the intended production environment.

Errors, or defects, should be documented, classified by severity,  and monitored by the defect management process to ensure timely resolution.

Integration testing is a key component of the software testing process and should include:
  • System and volume testing to confirm that the system is operating correctly and can handle the required data volumes;
  • Reconciliation tests to manually confirm the validity of data;
  • Regression testing to ensure that the new software does not cause problems with existing software; and
  • Performance testing to ensure that data can be loaded in the available “load window”;
  • Load manager testing;
  • Warehouse manager testing; and
  • Infrastructure and operations component testing.
  • Security testing should be performed in the production environment  under the direction of the security officer.
This test should certify compliance with system security and integrity and should address:
  • system back-up;
  • recovery;
  • security audit trails and tracking.
  • Quality assurance testing should focus on testing a set of components to ensure that they meets requirements.
These tests should be performed by an independent test team in a quality assurance environment that "mirrors" the production environment.

Every requirement should be tested and all system, user and production support documentation should be tested.

Test cases, test data and test results should be defined as configuration items and handed over to the production support team as part of the release package.
  • User acceptance testing (UAT) should focus on testing a set of requirements to ensure that they meet user expectations.
These tests should be performed in the quality assurance test environment or in a separate user acceptance test environment.

What are software testing process methods for documenting tests?

Each test should be documented to include the following:
  • Test objective should state what the test is expected to achieve;
  • Methodology should describe the general methodology or strategy for each test;
  • Conditions should specify the type of input that shall be used;
  • test progression should describe the manner in which progression is made from one test to the next;
  • Test results should specify how test results will be documented and where they will be stored;
  • Constraints should indicate any limitations on the test;
  • Criteria should describe the rules used to evaluate test results e.g. range of data values, combinations of input types used, maximum  number of errors allowed.
  • Data reduction should describe the techniques used to manipulate or prepare the test data.
Software testing process test cases should specify:
  • Test name;
  • Test description;
  • Test control;
  • Inputs;
  • Outputs; and
  • Test scripts (or procedures).
Defect Management Standard

Software testing process defect management involves:
  • Documenting defects;
  • Analyzing defects to determine if they are development issues or if they are requirement issues;
  • Assigning the defect to the appropriate team for remedy;
  • Monitoring to ensure that appropriate tests are conducted after software is modified;
  • managing the migration of changed code into the appropriate environment; and
  • Monitoring and reporting defect status.
Software testing process summary...

A test plan is a communications tool, which clearly sets testing expectations for all team members. It is important to complete the plan early in the project and manage it carefully to ensure on time, within budget project delivery.

A software testing process should be followed to ensure that all tests follow the same methodology.