How to Build Test Environments

How to Build The Test Environment

1. Test Standards
1. External Standard - Familiarity with and adoption of industry test standards from organizations such as IEEE, NIST, DoD, and ISO.
2. Internal Standards - Development and enforcement of the test standards that testers must meet.
2. Test Environment Components
1. Test Process Engineering - Developing test processes that lead to efficient and effective production of testing activities and products.
2. Tool Development and/or Acquisition - Acquiring and using the test tools, methods, and skills needed for test development, execution, tracking, and analysis (both manual and automated tools including test management tools).
3. Acquisition or Development of a Test Bed/Test Lab/Test Environment - Designing, developing, and acquiring a test environment that simulates the real world, including capability to create and maintain test data.
3. Test Tools
1. Tool Competency - Ability to use 1) automated regression testing tools; 2) defect tracking tools; 3) performance/load testing tools; 4) manual tools such as checklists, test scripts, and decision tables; 5) traceability tools; and 6) code coverage tools.
2. Tool Selection (from acquired tools) - Select and use tools effectively to support the test plan; and test processes.

While there are no generally accepted categories of test tools, experience has shown that the most commonly used tools can be grouped into these eight areas:

• Automated Regression Testing Tools - Tools that can capture test conditions and results for testing new versions of the software.
• Defect Management Tools - Tools that record defects uncovered by testers and then maintain information on those defects until they have been successfully addressed.
• Performance/Load Testing Tools - Tools that can “stress” the software. The tools are looking for the ability of the software to process large volumes of data without either losing data, returning data to the users unprocessed, or have significant reductions in performance.
• Manual Tools - One of the most effective of all test tools is a simple checklist indicating either items that testers should investigate, or to enable testers to ensure they have performed test activities correctly. There are many manual tools such as decision tables, test scripts to be used to enter test transactions and checklists for testers to use when performing such testing techniques as reviews and inspections.
• Traceability Tools - One of the most frequently used traceability tools is to trace requirements from inception of the project through operations.
• Code Coverage - Tools that can indicate the amount of code that has been executed during testing. Some of these tools can also identify non-entrant code.
• Test Case Management Tools - This category includes test generators and tools that can manage data being processed for online assistance.

Common tools that are applicable to testing Testers have access to a variety of work tools, many included with operating software such as “Windows.” These include such things as word processing, spreadsheets, computer graphics used for reporting and status checking, and tools that can measure the reading difficulty of documentation.

1. Quality Assurance / Quality Control
1. Quality Assurance versus Quality Control - Being able to distinguish between those activities that modify the development processes to prevent the introduction of flaws (QA) and those activities that find and correct flaws (QC). Sometimes this is referred to as preventive versus detective quality methods.
2. Process Analysis and Understanding - Ability to analyze gathered data in order to understand a process and its strengths and weaknesses. Ability to watch a process in motion, so that recommendations can be made to remove flaw-introducing actions and build upon successful flaw- avoidance and flaw-detection resources.
2. Building the Test Environment Work Processes
1. Concepts of work processes - understanding the concepts of policies, standards and procedures and their integration into work processes.
2. Building a Test Work Process - an understanding of the tester’s role in building a test work process.
3. Test Quality Control - Test quality control is verification that the test process has been performed correctly.
4. Analysis of the Test Process - The test process should be analyzed to ensure:
1. The test objectives are applicable, reasonable, adequate, feasible, and affordable.
2. The test program meets the test objectives.
3. The correct test program is being applied to the project.
4. The test methodology, including the processes, infrastructure, tools, methods, and planned work products and reviews, is adequate to ensure that the test program is conducted correctly.
5. The test work products are adequate to meet the test objectives.
6. Test progress, performance, processes, and process adherence are assessed to determine the adequacy of the test program.
7. Adequate, not excessive, testing is performed.
5. Continuous Improvement - Continuous improvement is identifying and making continuous improvement to the test process using formal process improvement processes.
3. Adapting the Test Environment to Different Technologies - The test environment must be established to properly test the technologies used in the software system under test. These technologies might include:

a. Security/Privacy b. Client Server c. Web Based Systems d. E-Commercee. E-Business f. Enterprise Resource Planning (ERP) g. Business Reengineeringh. Customer Relationship Management (CRM) i. Supply Chain Management (SCM) j. Knowledge Management k. Applications Service Providers l. Data Warehousing