Tarea: Run Tests
Run the appropriate collections of tests required to evaluate product quality. Capture test results that facilitate ongoing assessment of the product.
Disciplinas: Test
Objetivo
To execute tests and evaluate the test results.
Relaciones
RolesRealizador principal: Otras actividades adicionales que realiza:
EntradasObligatoria: Opcional:
  • Ninguno
Salidas
Descripción principal
Run the system test, which addresses functional and system integration tests and, potentially, user acceptance tests.
Pasos
Schedule test execution

Run the system tests as often as possible. Ideally, run the tests whenever new code is checked into the version control tool.

For larger systems, this will be too expensive. The tests may take several hours to run; therefore, you'll need to schedule tests less frequently. If possible, however, run the tests several times a day. As a minimum, run automated tests each night.

Run the test

Run the test at the scheduled time based on the instructions in the Test Script. It is best that the script be automated.

Good practices:

  1. Run the test in a separate test environment.
  2. Ensure that you run the test against the latest clean build.
  3. The first step of the test should be to set up the test environment (ensure that the network is available, that the test database is available and reset to a known state, and so on).
Close test run
Close the actual run as the last step of running the test. To do this:
  1. Close the test logs. The test log files should be closed and placed in the appropriate folder or directory.
  2. Announce results. Send a notice to everyone involved in the project informing them of the result of the test run and where they can find the test logs.
Examine the test log

Collect and compile information from test execution logs so you can:

  • Capture the high-impact and risk issues discovered in running the tests.
  • Identify errors in test creation, data inputs, or integrating applications and any architectural anomalies.
  • Isolate the target of the test to determine failure points.
  • Diagnose failure symptoms and characteristics.
  • Assess and identify possible solutions.

After completing these steps, verify that you have enough details to determine the impact of the results. In addition, make sure that enough information exists to assist individuals who are performing dependent tasks.

Identify failures and propose solutions

Identify whether or not the test has failed and propose a solution based on the type of test and category of failure.  The approach to testing will determine the identified failures and candidates for solutions.

Tests that are programmer supporting are used to help prepare and ensure confidence in the code. When identifying failures and proposing solutions for programmer supporting tests:

  • Failures will be identified at an object or element level.
  • Solutions will be to help clarify the problem.

Test that are business supporting are used to uncover prior mistakes and omissions.

  • Failures will identify omissions in requirements.
  • Solutions will help to clarify expectations of the system.

After you have this information and the steps proposed to resolve the failures, you can effectively categorize the type of failure and the appropriate type of solution.

See [MAR03] for more information.

Communicate test results

Communicate the test results to the team. For failed tests this might involve adding bugs to the Work Items List.

Communicating test results can affect the perception of the effectiveness of the tests. When communicating test results, it is important that you:

  • Know the audience, so that appropriate information is communicated appropriately
  • Run tests or scenarios that are likely to uncover the high-impact and risk issues or represent actual use of the system

When preparing test result reports, answer the following questions:

  • How many test cases exist, and what are their states (pass, fail, blocked, and so on)?
  • How many bug reports have been filed, and what are their states (open, assigned, ready for testing, closed, deferred)?
  • What trends and patterns do you see in test case and bug report states, especially opened and closed bug reports and passed and failed test cases?
  • For test cases that were blocked or skipped, why are they in this state?
  • Considering all test cases not yet run (and perhaps not even created yet), what key risks and areas of functionality remain untested?
  • For failed test cases, what are the associated bug reports?
  • For bug reports ready for confirmation testing, when can your team perform the test?
Factores clave
  • Run the test regularly. Ideally, that means whenever the code changes but, minimally, once a day.
  • It should be possible for anyone on the test team to run the test at any time.
Más información