Please enable JavaScript.

Coggle requires JavaScript to display documents.

Chapter 1 - Fundamental of testing (2.What is testing? (Terms (Review…

- - - - Defect (bug, fault)
        
        A flaw in a component or system that can cause the component or system to fail to perform its required function.
        
        e.g. an incorrect statement or data definition
        
        A defect, if encountered during execution, may cause a failure of the component or system.
        
        Defects may result in failures, or they may not, depending on inputs and other conditions.
      - mistake (error)
        
        A human action that produces an incorrect result
        
        Who can make errors?
        
        Bussiness Analysis
        
        :red_cross:Requirement
        
        System Architecture
        
        :red_cross:System Design
        
        Programmer
        
        :red_cross:Code
        
        Technical Writer
        
        :red_cross:Help document
      - Failure
        
        Deviation of the component or system from its expected delivery, service or result.
      - Quality
        
        The degree to which a component, system or process meets specified requirements and/or user/customer needs and expectations
    - - human beings are fallible
        
        Random Error
        
        Time Pressure(deadline)
  - - - False-fail result
        
        A test result in which a defect is reported although no such defect actually exists in the test object.
      - False-positive result
        
        See false-fail result.
  - - - Risk
        
        A factor that could result in future negative consequences; usually expressed as impact and likelihood.
      - False-negative result
        
        See false-pass result.
      - False-pass result
        
        A test result which fails to identify the presence of a defect that is actually present in the test object
    - - Since software and the associated work products are written by fallible humans
      - all of these work products will always have defects.
      - Where there are defects, there are risks
        of failure.
    - - How does testing measure software quality?
        
        Testing does provide a way of measuring the system’s quality
        
        So, which attributes reflect quality of system?
        
        the extent of test coverage
        
        The rate of defect discovery
        
        the percentage of tests which have passed all
        
        the number of known defects
      - However, debugging, not testing, is the activity that changed the quality of the system
      - Testing does not change the quality of the system under test directly
      - When testing finds defects and those defects are repaired, the quality of the system is increased
      - Testing also provides a learning opportunity that allows for improved quality if lessons are learned from each project
    - - Testing is part of how these risks of failure can be reduced.
      - reducing the risk in other areas by determining that the software works properly under the tested conditions
      - the testers are more likely to find defects in some areas
  - - - Selecting which test conditions to cover is a fundamental problem of testing
      - Reason
        
        the number of possible test cases is infinite for all but the simplest systems
        
        in the real world, we can only select a finite number of test cases for design, implementation, and execution
        
        Time and budget are limited
      - Solution
        
        Instead of trying to find every possible test case
        
        we should focus on what tests give us the greatest value by covering the most important parts of the system
        
        How can coverage be measure?
        
        coverage of code structures
        
        coverage of risks to the quality of the product
        
        coverage of requirements
- - - - Test planning
        
        What does it identify?
        
        the roles and responsibilities of the participants and
        stakeholders
        
        the relationship between the testers and the developers of the test items
        
        the features to be tested and not tested
        
        the extent to which testing is independent of development of these work items
        
        test items
        
        the test environments required, the appropriate test
        design techniques
        
        entry
        
        exit criteria
        
        how we’ll handle project risks
        related to testing
        
        What does it do?
        
        establish (and update) the scope, approach, resources, schedule, and specific tasks
      - Test control
        
        develop and carry out corrective actions to get a test project back on track when we deviate from the plan
      - Test analysis
        
        identify what to test, choosing the test conditions need to cover
        
        These conditions are any item or event that we can and should
        verify using one or more test cases.
        
        Test conditions can be functions, transactions, features, quality attributes, quality risks, or structural elements
      - Test design
        
        determine how we will test what we decided to test during test analysis
        
        transform general test conditions and general testing objectives from the test plan into tangible(hữu hình) test cases at the appropriate level of detail
      - Test Implement
        
        carry out the remaining activities required to be ready for test execution
        
        Include
        
        developing and prioritizing our test procedures
        
        creating test data
        
        and setting up test environments
    - - run our tests against the test object
    - - Checking result
        
        compare actual results against expected results to determine the pass/fail status of the test
      - Evaluating exit criteria
        
        exit criteria are a set of conditions that would allow some part of a process to complete
        
        Exit criteria are defined during test planning by working with the project and product stakeholders to balance quality needs against other priorities and various project constraints
        
        criteria ensure that everything that should be done has been done before we declare some set of activities finished
        
        test exit criteria help us report our results as well as helping us plan when to stop testing
      - Test result reporting
        
        report our progress against exit criteria
        
        This often involves details related to the status of the test project, the test process, and quality of the system under test
      - Test closure
        
        Include
        
        collecting test process data related to the various completed test activities in order to consolidate our experience
        
        re-useable testware
        
        important facts
        
        and relevant metrics.
  - - - Testing that runs test cases that failed the last time they were run, in order to verify the success of corrective actions.
    - - The process of finding, analyzing and removing the causes of failures in software.
    - - A condition or capability needed by a user to solve a problem or achieve an objective that must be met or possessed by a system or system component to satisfy a contract, standard, specification, or other formally imposed document.
    - - An evaluation of a product or project status to ascertain discrepancies from planned results and to recommend improvements.
      - Include
        
        informal review
        
        technical review
        
        inspection
        
        management review
        
        walkthrough
    - - developed for a particular objective or test condition, such as to exercise a particular program path or to verify compliance with a specific requirement.
      - Include
        
        execution preconditions
        
        expected results
        
        A set of input values
        
        execution postconditions
    - - A test management task that deals with developing and applying a set of corrective actions to get a test project on track when monitoring shows a deviation from what was planned.
    - - A document specifying the test conditions (coverage items) for a test item, the detailed test approach and identifying the associated high level test cases.
    - - A reason or purpose for designing and executing a test.
    - - The process consisting of all lifecycle activities, both static and dynamic, concerned with planning, preparation and evaluation of software products and related work products to determine that they satisfy specified requirements, to demonstrate that they are fit for purpose and to detect defects
  - - - Dynamic testing is an evaluation of that software or related work products that does involve executing the software.
      - As such, the ISTQB definition of testing not only includes a number of pre-execution and postexecution activities that non-testers often do not consider ‘testing’, but also includes software quality activities (e.g. requirements, reviews and static analysis of code) that non-testers (and even sometime testers) often do not consider ‘testing’ either.
    - - such as requirements specifications or user stories
      - Static testing is any evaluation of the software or related work products that occurs without executing the software itself.
- - - - One strategy for selecting tests is risk based testing
      - Another strategy for selecting tests is requirements based testing
- - - - Any event occurring that requires investigation.
    - - Testing of a previously tested program following modification to ensure that defects have not been introduced or uncovered in unchanged areas of the software, as a result of the changes made.
      - It is performed when the software or its environment is changed.
    - - The implementation of the test strategy for a specific project.
      - It typically includes
        
        starting points regarding the test process
        
        the test design techniques to be applied
        
        the decisions made that follow based on the (test) project’s goal and the risk assessment carried out
        
        exit criteria
        
        test types to be performed.
    - - All documents from which the requirements of a component or system can be inferred.
      - The documentation on which the test cases are based.
      - If a document can be amended only by way of formal amendment procedure, then the test basis is called a frozen test basis.
      - include
        
        interface specifications
        
        the system design and architecture
        
        risk analysis reports
        
        requirements and design specifications
    - - Exit criteria are used to report against and to plan when to stop testing.
      - The purpose of exit criteria is to prevent a task from being considered completed when there are still outstanding parts of the task which have not been finished.
      - The set of generic and specific conditions, agreed upon with the stakeholders, for permitting a process to be officially completed.
    - - An item or event of a component or system that could be verified by one or more test cases,
      - e.g. a function, transaction, feature, quality attribute, or structural element.
    - - The degree, expressed as a percentage, to which a specified coverage item has been exercised by a test suite.
    - - Data that exists (for example, in a database) before a test is executed, and that affects or is affected by the component or system under test.
    - - The process of running a test on the component or system under test, producing actual result(s)
    - - A chronological record of relevant details about the execution of tests.
    - - A test management task that deals with the activities related to periodically checking the status of a test project.
      - Reports are prepared that compare the actuals to that which was planned.
    - - A document describing the scope, approach, resources and schedule of intended test activities.
      - It identifies among others test items
        
        and the rationale for their choice
        
        and any risks requiring contingency planning.
        
        the test design techniques and entry and exit criteria to be used
        
        the test environment
        
        who will do each task, degree of tester independence
        
        the features to be tested, the testing tasks
      - It is a record of the test planning process.
    - - A document specifying a sequence of actions for the execution of a test.
    - - A set of several test cases for a component or system under test, where the post condition of one test is often used as the precondition for the next one.
    - - A document summarizing testing activities and results.
      - It also contains an evaluation of the corresponding test items against exit criteria.
    - - Artifacts produced during the test process required to plan, design, and execute tests,
      - such as documentation, scripts, inputs, expected results, set-up and clearup procedures, files, databases, environment, and any additional software or utilities used in testing.
  - - - What will we do?
        
        we transform the more general testing objectives defined in the test plan into tangible test conditions and test cases.
        
        The way in which and degree to which the test conditions and test cases are made tangible – that is to say, specifically documented – depends on the needs of the testers, the expectations of the project team, any applicable regulations, and other considerations.
      - Major tasks
        
        Review the test basis
        
        Evaluate the testability of the test basis and test objects
        
        Identify and prioritize specific test conditions
        
        Design and prioritize high level (i.e. abstract or logical) test cases
        
        Identify the necessary test data to support the test conditions and test cases as they are identified and designed.
        
        Design the test environment
        
        Create traceability between the test basis documents and the test cases
    - - What will we do?
        
        This involves transforming the high level (or abstract or logical) test cases into low level (concrete) test cases, which may be done more formally and written into a test script, or it may be done ‘on the fly’ as a tester is executing tests from a list of high level test conditions
        
        Test implementation also involves setting up
        the test environment
        
        This involves combining the test cases in a particular order, as well as including any other information needed for test execution.
        
        During test execution, of course, we run the tests.
        
        we specify test procedures (or test scripts)
      - Test Implementation
        
        Major Tasks
        
        Identify and create specific test data,
        
        Develop and prioritize test procedures
        
        Finalize, implement and prioritize the test cases.
        
        If automation is to occur, prepare test harnesses and write automated test scripts
        
        Verify that the test environment has been set up correctly
        
        Verify and update the bi-directional traceability prepared previously
      - Test Execution
        
        Major Tasks
        
        Verify and update the bi-directional traceability prepared previously
        
        Compare actual results with expected results
        
        Log the outcome of test execution
        
        Analyze the incidents in order to establish their cause.
        
        repeat test activities when actions are taken to resolve discrepancies.
    - - Test planning
        
        metaphor
        
        test planning as similar to figuring out how to get from one place to another
        
        For small, simple, and familiar projects, finding the route merely involves taking an existing map, highlighting the route, and jotting down the specific directions
        
        For large, complex, or new projects, finding the route can involve a sophisticated process of creating a new map, exploring unknown territory, and blazing a fresh trail.
        
        What does it do?
        
        planning involves defining the overall strategic and tactical objectives of testing
        
        discovering and specifying the specific test activities required to satisfy those objectives and the general mission of testing
      - Test control
        
        What will we do in test control?
        
        So, in test control we continuously compare actual progress against the plan
        
        To some extent, test control involves re-planning, which should take into account the feedback from these test monitoring and control activities.
        
        take whatever actions are necessary to meet the mission and objectives of the project.
        
        monitor test activities
        
        adjust the plan,
        
        report the test status and any necessary deviations from the plan
        
        Metaphor
        
        To continue our metaphor, even with the best map and the clearest directions, getting from one place to another involves careful attention, watching the dashboard, minor (and sometimes major) course corrections, talking with our companions about the journey, looking ahead for trouble, tracking progress towards the ultimate destination, and coping with finding an alternate route if the road we wanted is blocked
    - - What will we do?
        
        we assess test execution against the objectives which we defined in the test plan
        
        We should do so for each test level
      - Major tasks
        
        Assess if more tests are needed or if the exit criteria specified should be changed.
        
        Write a test summary report for stakeholders,
        
        Check the test logs gathered during test execution against the exit criteria specified in test planning.
    - - What will we do?
        
        Test closure activities should occur at major project milestones
        
        we collect data from completed test activities to consolidate experience, testware, facts and numbers.
      - Major tasks
        
        Document the acceptance of the system, particularly as part of the closure of the user acceptance test level.
        
        Finalize and archive testware, the test environment and the test infrastructure for later reuse.
        
        Ensure that all incident reports are resolved, and possibly log change requests for any that remain open
        
        Hand over the testware to the maintenance organization, if that is a different group.
        
        Check which planned deliverables have been delivered, and possibly create any deliverables we might have missed
        
        Analyze lessons learned to determine changes needed for future releases and projects (i.e. perform a retrospective).
        
        Use the information gathered to improve test maturity, especially as an input to test planning for future projects.
      - When can we include test closures?
        
        a test project is completed (or cancelled)
        
        a milestone has been achieved, or a maintenance release has been completed
        
        a software system is released
        
        though the specific milestones that involve closure activities should be specified in the test plan
    - - However, in many cases, the activities in the process may overlap or take place concurrently, provided that these dependencies are fulfilled.
      - These activities are logically sequential
- - - - A test design technique where the experience of the tester is used to anticipate what defects might be present in the component or system under test as a result of errors made, and to design tests specifically to expose them.
  - - - Good testers couple this professional pessimism with a natural inclination to doubt the correctness of software work products and their behaviours as they look at them
    - - Good testers notice everything, even the smallest details.
      - Sometimes these details are cosmetic problems like font-size mismatches, but sometimes these details are subtle clues that a serious failure is about to happen
    - - Good testers expect to find defects and failures
      - They understand human fallibility and its implications for software development
    - - Good testers not only know a defect when they see one, they also know where to look for defects.
      - Experienced testers have seen a veritable parade of bugs in their time, and they leverage this experience during all types of testing, especially experience-based testing such as error guessing
    - - Good testers are curious about why systems behave the way they do and how systems are built
      - When they see unexpected behaviour, they have a natural urge to explore further, to isolate the failure, to look for more generalized problems, and to gain deeper understanding.
    - - Good communicators know how to explain the test results, even negative results such as serious defects and quality risks, without coming across as preachy, scolding or defeatist.