Please enable JavaScript.

Coggle requires JavaScript to display documents.

Software Testing, :, Chap 4 : Test Design Tenique, Examination of…

- - - - Review participants will work separately from each other using relevant documents, processes, regulations and checklists
        
        All issues raised are documented using declarative forms/ reports
        
        Spelling errors will also be recorded in the document but will not be mentioned in the meeting.
    - - Logging Phase
        
        Discussion Phase
        
        Decision Phase
    - - Unified view of documents and timelines.
        
        Possible discussion of role division, test speed, test coverage, process and other questions..
        
        The relationships between the reviewed document and the source document will be explained
    - - On the results of these failure has been found, the author will gradually tweak the document updates.
    - - Define review criteria
        
        Assign roles to each person
        
        Define input and output standards for formal reviews (eg inspections)
        
        Select parts of the document to review
        
        Check input standards
    - - Assure all the reported error, defects have been fixed
        Collect a number of metrics/ stats at each step of the process to control and optimize the review process
  - - - Within a review team, 4 types of participants:
        
        Moderator
        
        Leads the review process.
        
        Determines the type of review, approach and the composition of the review team.
        
        Performs entry check and follow-up on the rework to control quality of the input and output of review process.
        
        General management of the team (coaching team member, lead discussion.)
        
        Schedule meetings.
        
        Stores collected data.
        
        Author
        
        Writer of the document to be reviewed.
        
        Understand defects found, clear confusing areas.
        
        If several people is involved, one person should be appointed to be responsible as author.
        
        Scribe
        
        Also known as The Recorder
        
        Records each defect mention, and any suggestions for process improvement
        
        In practice, the author often also is the scribe.
        
        However, having another person as the scribe can have advantages.
        
        Reviewer
        
        Check any materials for defects (prior to the meeting).
        
        Depending on the type of review, the level of thoroughness and the knowledge of reviewers will be different.
        
        Reviewers should be chosen to represent different perspectives and roles in the review process.
        
        Additionally, Management needs to play a role in the Review process.
        
        Manager
        
        Decides the execution of reviews, allocates time in project schedule, and determines if review objectives have been met.
        Also takes care of any review training requested by the participants.
  - - - Walkthrough
        
        Key characteristics
        
        The author of the document guide the participants through the document and their thought process.
        
        Key characteristics:
        
        The meeting is lead by the authors. Often, a separate scribe is presented.
        
        Scenarios and dry runs may be used to validate the content.
        
        Pre-meeting preparation for reviewers Is optional.
        
        Goal
        
        To present the document to stakeholders.
        
        To explain and evaluate the contents of the document.
        
        To establish a common understanding of the document
        
        To examine and discuss the validity of proposed solutions and possible alternatives.
      - Inspection
        
        The most formal type of review.
        
        Typically led by trained moderator.
        
        In Inspection meeting, defects found are logged.
        
        Discussion is postponed until discussion phase.
        
        Goal
        
        Key characteristics:
        
        Usually led by a trained moderator.
        
        Uses defined roles during the process.
        
        Involve peers to examine the product.
        
        Rules and checklists are used in preparation phase.
        
        Metrics are gathered and analyzed to optimized the process.
        
        Goals:
        
        Help author improve the quality of the inspecting document.
        
        Improve product quality.
        
        Learn from defects, improve process.
      - Technical Review
        
        Discussion meeting that focuses on technical content of a document
        
        Can be quite informal to very formal
        
        Led by: Trained Moderator/ Technical Expert
        
        Participants:
        
        Architects
        
        Chief designers
        
        Key users
        
        Typically a peer-review (no management involved)
      - Success factors for reviews
        
        Implementing reviews is not easy. There is no one way to success and there are numerous ways to fail.
        
        Here are some success factors that can help improve the chances of success when implementing reviews:
        
        Find a ‘Champion’
        
        Pick things that really count
        
        Plan and track review activities
        
        Train participants
        
        Manage people issues
        
        Follow the rules, keep it simple
        
        Continuous improve
        
        Report results
        
        Just do it!
  - - - DATA-FLOW ANALYSIS
        
        Definition
        
        Data Flow Testing is a specific strategy of software testing that focuses on data variables and their values. It makes use of the control flow graph.
        
        Data flow structure follows the trail of a data item as it is accessed and modified by the code. Many times, the transactions applied to data are more complex than the instructions that implement them.
        
        Thus, using data flow measures it is shown how the data act as they are transformed by the program. Defects can be found such as referencing a variable with an undefined value and variables that are never used.
        
        Steps of Data Flow Testing
        
        1.Creation of a data flow graph.
        
        2.Selecting the testing criteria.
        
        3.Classifying paths that satisfy the selection criteria in the data flow graph.
        
        4.Develop path predicate expressions to derive test input.
        
        What is Data flow Testing?
        
        Dynamic data flow identifies program paths from source code.
        
        The static data flow testing process involves analyzing the source code without executing it. Static data flow testing exposes possible defects known as data flow anomaly.
        
        It is performed at two abstract levels: static data flow testing and dynamic data flow testing.
        
        The programmer can perform numerous tests on data values and variables.
        This type of testing is referred to as data flow testing.
        
        Advantages of Data Flow Testing
        Variables used but never defined
        Variables defined but never used
        Variables defined multiple times before actually used
        IDE allocating variables before using.
        
        Data Flow Testing Limitations
        
        Time-consuming
        Testers require good knowledge of programming.
        Costly process.
      - CONTROL-FLOW ANALYSIS
        
        In Testing, control-flow analysis (CFA) is a static-code-analysis technique for determining the control flow of a program.
        Control flow (or flow of control) is the order in which individual statements, instructions or function calls of an program are executed
        The control flow is expressed as a control-flow graph (CFG):
        
        Control-flow analysis’s objective :
        
        Find nodes that not accessible from the start node
        
        Find Infinite loops
        
        Whether the code Is well-structured, reducible
        
        Cyclomatic-complexity
      - Advantages
        
        It can evaluate all the code in an application, increasing code quality.
        
        It provides speed in using automated tools compared to manual code review
        
        Paired with normal testing methods, static testing allows for more depth into debugging code.
        
        Automated tools are less prone to human error.
        
        It will increase the likelihood of finding vulnerabilities in the code, increasing web or application security.
        
        It can be done in an offline development environment.
      - False positives can be detected.
        
        A tool might not indicate what the defect is if there is a defect in the code.
        
        Not all coding rules can always be followed, like rules that need external documentation.
        
        Static analysis may take more time than comparable methods.
        
        Static analysis can't detect how a function will execute.
        
        System and third-party libraries may not be able to be analyzed.
    - - Was developed by Thomas J.McCabe in 1976.
        Software metricused to indicate the complexity of a program.
        Maximum number of linearly independent paths.
      - Application
        
        1.Limiting complexity during development
        
        Measuring the "structuredness" of a program
        
        Implications for software testing
        
        Correlation to number of defects
    - - Lines of code (LOC)
        
        Operands & operators (Halstead’s metrics)
        
        Fan-in & fan-out
        
        Nesting levels
        
        Function calls
        
        OO metrics: inheritance tree depth, number of methods, coupling & cohesion
    - - TYPICAL DEFECTS DISCOVERED BY STATIC ANALYSIS TOOLS
        
        referencing a variable with an undefined value
        
        inconsistent interface between modules and components
        
        variables that are never used
        
        unreachable code or dead code
        
        missing or erroneous logic
        
        complicate structure
        
        programming standards violations
        
        security vulnerabilities
        
        syntax violations
      - Static analysis
        
        *Performed on requirements, design or code
        
        To find defects in code, software model
        
        Ideally performed before formal review
        
        Unrelated to dynamic properties of requirements, design, code
        
        Finds defects rather than failures
        
        Finds defects rather than failures
        
        By tool, without executing code :
      - A compiler statically analyses code
        
        Static analysis tool
        
        Graphic depictions of control flow, data flow
        
        Distinct paths of code line
        
        Structural attributes, coding standards
        
        When
        
        Designer
        
        During software modeling
        
        Developers
        
        Before and during component testing, integration testing
      - Why is static analysis valuable?
        
        Early detection of defects prior to test execution
        Early warning about suspicious aspects of code, design, requirements
        Identify defects not easily found by dynamic testing
        Detecting dependencies and inconsistencies in software models
        Improved maintainability of code and design
        Prevention of defects
  - - - 2.Test Analysis: Identifying test conditions
        
        est conditions is to try to define tests to meet those requirements.
        When identifying test conditions, we want to identify as many as we can.
        
        -The test conditions that are chosen will depend on the test strategy or detailed test approach.
        
        -Test analysis is the process of looking at something that can be used to derive test information. This basis for the tests is called the ‘test basis’.
        
        -Test conditions should be able to be linked back to their sources in the test
        basis - this is called traceability.
      - 3 Test design: specifying test case
        
        Role: Test cases are very important in any project as this is the first step in the testing process and if something goes wrong at this step it will have consequences in the next stages in the test life cycle.
        
        2 Important steps of a test case:
        
        Identify the purpose of the test
        
        Underatand the requirements
        
        Design a file test case
        
        3 Types of test case
        
        -GUI test case
        
        -Positive test case
        
        -Negative test case
        
        -Combination test case
        
        1 Component of a test case
        
        Test case ID
        
        Date excute test
        
        Function name
        
        Test data
        
        Pre-condition
        
        Test steps
        
        Expected results
        
        Actural results
        
        Execute test by
        
        In software engineering, a test case is a specification of the inputs, execution conditions, testing procedure, and expected results that define a single test to be executed to achieve a particular software testing objective, such as to exercise a particular program path or to verify compliance with a specific requirement.
      - 1 . Formality of test documentation
        
        -The right level of formality for you
        depends on your context.
        
        The thoroughness of your test
        documentation depend on external factors.
        
        -Testing may be performed with
        varying degrees of formality.
      - 4 Test implementation
        
        4.1 Specifying test procedures or scripts
        
        The next step is to group the test cases in a sensible way for executing them and to specify the sequential steps that need to be done to run the test.
        
        Test procedures, or test scenario, specifies a sequence of actions to perform a test.
        
        You need to use "build" software to create an executable program and the process is called " Software Build"
        
        There are 2 types of test procedure: smoke test and sanity check.
        
        -Smoke Testing is a software testing technique performed post software build to verify that the critical functionalities of software are working fine.
        
        -Sanity testing is a kind of Software Testing performed after receiving a software build, with minor changes in code, or functionality, to ascertain that the bugs have been fixed and no further issues are introduced due to these changes.
        
        4.2 Test implementation: specifying test procedures or
        
        Smoke test
        
        Smoke Testing is performed to ascertain that the critical functionalities of the program is working fine
        
        The objective of this testing is to verify the "stability" of the system in order to proceed with more rigorous testing
        
        This testing is performed by the developers or testers
        
        Smoke testing is usually documented or scripted
        
        Smoke testing is a subset of Acceptance testing
        
        Smoke testing exercises the entire system from end to end
        
        Smoke testing is like General Health Check Up
        
        Sanity test
        
        Sanity Testing is done to check the new functionality/bugs have been fixed
        
        The objective of the testing is to verify the "rationality" of the system in order to proceed with more rigorous testing
        
        Sanity testing in software testing is usually performed by testers
        
        Sanity testing is usually not documented and is unscripted
        
        Sanity testing is a subset of Regression Testing
        
        Sanity testing exercises only the particular component of the entire system
        
        Sanity Testing is like specialized health check up
    - - Black-box(Specification-based technique)
      - White-box (Structure-based technique)
      - Experience-Based techniques
    - - 4.4.1 Using structure-based techniques to measure coverage
        and design tests
        
        What is White-box Testing
        
        White Box Testing is software testing technique in which internal structure, design and coding of software are tested to verify flow of input-output and to improve design, usability and security
        
        Usually done by development team
        
        Characteristics of White-box Testing
        
        Is a software testing strategy based on the algorithm, the functional internal structure of the respective software.
        
        Testers must have a certain knowledge of coding, the internal structure of the function, know how to program the software.
        
        The testing is conducted based on checking whether the algorithm or code is correct.
        
        This level of testing usually requires testers to write test cases full of branches in the code; When testing, will set conditions and data to run on all branches in the algorithm, ensuring full implementation.
        
        What is Test Coverage?
        
        Type of Test Coverages
        
        Code Coverage
        
        Statement Coverage
        
        +Branch Coverage
        
        +Condition Coverage
        
        +Path Coverage
        
        White-box Test Design and Test
        
        When writing test cases: Based on the requirements and content of the Source Code (interfering inside the Code of the program)
        
        When executing the test: Execute the test in the code (no need to execute the program, because the white box test execution will use some framework that supports it Ex: Test debug framework, Mockup)
        
        In this testing, it is required that the tester has certain knowledge and skills about the programming language used, understanding the algorithm in the software component, to be able to understand the details of the code to be tested.
      - 4.4.2 Statement Coverage and
        Statement Testing
        
        Statement Coverage
        
        Statement coverage technique is used widely in Software testing.
        
        Used to design white-box test cases.
        
        Statement Coverage
        
        It involves execution of all statement of the source code at least once, then calculate the total number of executed statements out of total statements present in the source code.
        
        Statement Coverage - Goal
        
        Cover all the possible executing statements and path lines in the code.
        
        Statement Coverage - Example
        
        Consider the code snippet here.
        To achieve 100% coverage, just 1 test case is required: A > B
        
        Statement Coverage – Example 2
        
        Here we have divided the code snippet into 6 statements.
        
        With each line as a statement, we have:
        
        2 read statement
        
        1 assignment statement
        
        1 IF statement (3 lines), with 1 PRINT statement inside.
        
        Let’s analyze the coverage of a set of tests on this code:
        
        TEST SET 1
        
        Test 1_1: A = 2, B = 3
        
        => C = 8
        
        Test 1_2: A = 0, B = 25
        
        => C = 50
        
        Test 1_3: A = 47, B = 1
        
        => C = 49
        
        In 1_1 (C = 8), we will cover the statements on lines 1,4 and 6.
        
        In 1_2 (C = 50), we cover the same statements as 1_1
        
        In 1_3 (C = 49), we still cover the same as 1_1
        
        Since we have covered 5 out 6 statements, we have 83% coverage.
        
        To cover statement 5, we will design another test case:
        
        TEST 1_4: A = 20, B = 25
        
        With this, C = 70, statement 5 will be executed.
        
        Now, our statement coverage is 100%.
      - 4.4.3 Decision coverage and
        Decision testing
        
        Decision Coverage Testing
        
        Decision coverage technique gives decision coverage to Boolean values.
        
        It cover all possible outcomes of each and every Boolean condition of the code
        
        Whenever there is a possibility of two or more outcomes from the statements like do while statement, if statement and case statement (Control flow statements), it is considered as decision point because there are two outcomes: either true or false.
        
        With IF statement, exit can be TRUE or FALSE
        With a loop statement, outcome is either to perform the code within the loop or not – TRUE or FALSE
        
        Advantages & Disadvantages
        
        Advantages
        
        To validate that all the branches in the code are reached.
        
        To ensure that no branches lead to any abnormality of the program’s operation.
        
        It eliminate problems that occur with statement coverage testing.
        
        Disadvantages
        
        This metric ignores branches within Boolean expressions which occur due to short-circuit operators.
        
        Decision Coverage Testing - Example
        
        Suppose we have a test that gives 100% statement coverage:
        
        TEST SET 2
        
        Test 2_1: A = 20, B = 15
        
        Here, C = -10, so the condition C < 0 is TRUE. We print ‘C Negative’.
        
        With this, we have executed the decision outcome of condition FAIL.
        
        Let’s add another test:
        
        TEST SET 2
        
        Test 2_1: A = 20, B = 15
        
        Test 2_2: A = 10, B = 2
        
        This covers both outcome:
        
        TRUE with 2_1 (C = -10)
        
        FALSE with 2_2 (C = 6)
      - 4.4.4 Other structure-based Techniques
        
        Other structure-based Techniques
        
        There are other structure-based techniques that can be used to achieve testing to different degrees of thoroughness.
        
        Some techniques are requires more test to achieve 100% coverage, and are therefore strong. Some are weaker.
        
        Branch Coverage
        
        Aims to ensure each one of the possible branch from each decision is executed at least once.
        
        Every branch is either true or false
        
        Helps validating all the branches in the code.
        
        Some control-flow code coverage techniques
        
        Linear Code Sequence and Jump
        
        Technique to identify code coverage. Equivalent to statement coverage.
        
        Condition coverage
        
        Ensures each one of the Boolean expression has been evaluated to both TRUE and FALSE.
        
        Multiple Condition coverage
        
        All the possible combinations of outcomes of conditions in a decision are tested at least once.
        
        Since there are only two possible outcomes of a condition (TRUE or FALSE), 2 is the basis for the number of test situations that can be created.
        
        Condition determination coverage
        
        The percentage of all single condition outcomes that independently affect a decision outcome that have been exercised by a test case suite.
        
        100% modified condition / decision coverage implies 100% decision condition coverage.
        
        Path Coverage
        
        Deals with the total number of paths that could be covered by a test case.
- - - - When we mention to testing, we can find out a lot of terms such as:
        
        failure
        
        fault
        
        defect
        
        quality
        
        mistake
        
        error
        
        bug
        
        risk
    - - Error: a human action that produces an incorrect result
      - Failure: deviation of the software from its expected delivery or service
      - Fault: a manifestation of an error in software
        also known as a defect or bug
        if executed, a fault may cause a failure
    - - Reliability: the probability that software will not cause the failure of the system for a specified time under specified conditions
        
        Can a system be fault-free? (zero faults, right first time) - No
        
        Is a “fault-free” software application always reliable? – No
        
        Can a software system be reliable but still have faults? - Yes
      - 🡺 But why do faults occur in software?
        
        Software is written by human being
        
        The pressure of deadlines…
    - - Very little or nothing at all:
        Minor inconvenience
        No visible or physical detrimental impact
      - Huge sums:
        
        Ariane 5 ($7billion)
        
        Mariner space probe to Venus ($250m)
        
        American Airlines ($50m)
      - Software is not “linear”:
        Small input may have very large effect
      - In order to prevent these from happening, we need testing
    - - Total for “exhaustive” testing:
        20 x 4 x 3 x 10 x 2 x 100 = 480,000 tests
        If 1 second per test, 8000 mins, 133 hours, 17.7 days
        10 secs = 34 weeks, 1 min = 4 years, 10 min = 40 years
      - Testing everything (all combinations of inputs and preconditions) is not feasible except for trivial cases. Instead of exhaustive testing, we use risks and priorities to focus testing efforts
    - - It depends on RISK:
        
        Risk of missing important faults
        
        Risk of incurring failure costs
        
        Risk of releasing untested or under-tested software
        
        Risk of losing credibility and market share
        
        Risk of missing a market window
        
        Risk of over-testing, ineffective testing
      - Use RISK to:
        
        Determine what to test first
        
        Determine what to test most
        
        Determine how thoroughly to test each item
        
        Determine what not to test (this time)
        
        Allocate the time available for testing by prioritising testing ...
    - - What is quality?
        
        Projects aim to deliver software to specification.
        
        For the project to deliver what the customer needs requires a correct specification.
        
        The delivered system meet the specification.
      - What does testing test?
        
        System function, correctness of operation
        
        Non-functional qualities
  - - - According to ISTQB Glossary, the definition for testing is:
        The process consisting of all lifecycle activities, both static and dynamic, concerned with planning, preparation and evaluation of system and related work products to determine that they satisfy specified requirements, to demonstrate that they are fit for purpose and to detect defects.
    - - All life cycle activities: software testing can help us build the right software and reduce the cost overall
      - Both static and dymanic: dynamic testing is the process where the code is executed to show the results of running tests, while reviewing documents without executing the code is called static testing
      - Planning: We need to choose what we want to test, by selecting test conditions and designing test cases
      - Preparation: choose testing condition and design test case
      - Evaluation: check the result and evaluate the software to decide if we have finish the test and if the software has passed the test
      - Software products and related word products: test requirement and design specification as well as related document
    - - Determine if the software products satisfy specified requirements
      - Detect defects
      - Demonstrate if the software products are fit for purpose
    - - We can use both dynamic testing and static testing as a means for achieving similar test objectives.
      - Testing objective often include:
        
        Finding defects
        
        Gaining confidence in and providing information about the level of quality
        
        Preventing defects
    - - Defects tend to cluster -> knock-on defects
      - By focusing on clusters, we can :
        
        Identify risky modules
        
        Contribute to the planning for the tests executed later
        
        Prevent defects and failures happening again
        
        Identify the cause of clusters and potential future clusters
    - - Solution :
        
        the test cases need to be regularly reviewed and revised
        
        adding new and different tests
    - - What is debugging?
        
        The process of identifying and removing errors from computer hardware or software
    - - Testing Principle - Testing shows presence of defects
        “Testing can show that defects are present, but cannot prove that there are no defects.
        Testing reduces the probability of undiscovered defects remaining in the software but, even if no defects are found, it is not a proof of correctness”
    - - Testing Principle - Absence of errors fallacy
        “Finding and fixing defects does not help if the system built is unusable and does not fulfill the users' needs and expectations.”
      - The people using software are more interested in the software supporting them in completing tasks efficiently and effectively.
      - The customers for software are not interested in defects or numbers of defects, except when they are directly affected.
      - 🡺 The software must meet their needs.
  - - - Testing shows presence of defects
        
        Testing only proves that defects are presence.
        
        Even if no defects are found, we can not say that the program is defect-free.
      - Exhaustive testing is not possible
        
        Optimal testing based on assessments.
        
        Testing every scenarios and inputs is not possible.
      - Early testing
        
        Testing should start soon in the development life cycle.
        
        Designing test cases and reviewing tests early.
      - Defect clustering
        
        Errors within one test subject is not equally distributed.
        
        A small number of modules contain most of the defects detected.
      - Pesticide paradox
        
        If the same test is repeated over and over again, eventually no new bugs will be found.
        
        Test cases need to be reviewed and revised.
      - Context-dependent
        
        Different types of software need different types of testing.
      - Absence of errors fallacy
        
        A bug-free software does not mean it is ready to be released.
        
        Software is built to match user’s requirements and needs.
  - - - Fundamental process
      - Test Planning and Control
      - Test Analysis and Design
      - Test Implementation and Execution
      - Evaluating exit criteria and reporting
      - Test closure activities
    - - Test planning: Understanding of the requirements from customers, stakeholders and the project, risks that testing is meant to address.
        
        Major tasks
        
        Determine the objectives, scope and risks of testing.
        
        Decide the overall approach of testing.
        
        Allocate resources required for testing.
        
        Schedule test analysis and design tasks, implementation, execution and evaluation.
        
        Determine the exit criteria.
      - Test control :
        Comparing progress made against the plan.
        Taking necessary actions to meet the objectives of testing.
        
        Major tasks
        
        Measure results of reviews and testing.
        
        Document progress.
        
        Provide information on testing.
        
        Take action and make decisions.
    - - The process where general testing objectives are transformed into test conditions and test designs.
      - Major tasks
        
        Review test basis.
        
        Identify test conditions.
        
        Design tests.
        
        Design test environment set-up, identify required tools and infrastructures.
    - - Implementation
        
        Creating data and instructions for written tests.
        
        Creating test suites.
        
        Implementing test environment.
      - Execution
        
        Execute test suites and test cases.
        
        Check actual results against expected results.
        
        Report error and create Incident report.
    - - Defining when to stop testing
      - Factors:
        
        Maximum test cases executed, with certain pass percentage.
        
        Bugs rate falls below certain level.
        
        Deadline is reached.
      - Major tasks
        
        Assessing if more tests are needed or if the exit. criteria should be changed.
        
        Writing test summary report for stakeholders.
    - - Activities done when software is ready for delivery (Project is cancelled, target is achieved, etc…)
      - Major tasks
        
        Check which planned deliverables are delivered.
        
        Ensure all incident reports are resolved.
        
        Evaluate how testing went.
        
        Handover Testwares to maintenance organization.
  - - - Independent testing corresponds to an independent team, who involve in testing activities other than developer to avoid author bias and is often more effective at finding defects and failures.
      - Test your own work?
        Find 30% - 50% of your own faults
        Same assumptions and thought processes
        See what you meant or want to see, not what is there
        Emotional attachment
        Don’t want to find faults
        Actively want NOT to find faults
    - - Tests by the person who wrote the item under test;
      - Tests by another person within the same team, such as another programmer;
      - Tests by a person from a different organizational group, such as an independent test team;
      - Tests designed by a person from a different-organization or company, such as outsourced testing or certification by an external body.
    - - Explain that by knowing about this now we can work round it or fix it so the delivered system is better for the customer.
      - Start with collaboration rather than battles. Remind everyone of the common goal of better quality systems.
      - Communicate findings on the product in a neutral, fact-focused way without criticizing the person who created it.
        For example, write objective and factual incident reports and review findings.
- - - - NON-FUNCTIONAL TESTING is defined as a type of Software testing to check non-functional aspects (performance, usability, reliability, etc) of a software application. It is designed to test the readiness of a system as per nonfunctional parameters which are never addressed by functional testing.
    - - Survivability
      - Reliability
      - Scalability
      - Security
      - Availability
      - Usability
      - Interoperability
      - Efficiency
      - Reusability
      - Portability
      - Flexibility
    - - Performance Testing
      - Load Testing
      - Failover Testing
      - Usability Testing
      - Compatibility Testing
      - Stress Testing
      - Maintainability Testing
      - Compliance Testing
      - Disaster Recovery Testing
      - Security Testing
      - Volume Testing
      - Scalability Testing
      - Portability Testing
      - Endurance Testing
      - Efficiency Testing
      - Baseline Testing
      - Reliability Testing
      - Documentation Testing
      - Recovery Testing
      - Internationalization Testing
      - Localization Testing
  - - - Applied mainly in component testing, integration
      - White-box testing derives tests based on the system’s internal structure or implementation. Internalstructure may include code, architecture, work flows, and/or data flows within the system
      - This type of testing carried out to test the structure of code, because weare interested in what is happening 'inside the box'
      - It focuses primarily on verifying the flow of inputs and outputs through the application, improving design and usability, strengthening security
    - - Two basic steps
        
        Understand the source code
        
        Create the test cases and execute
      - Code Coverage analysis
      - Branch Coverage
  - - - Unit Testing
      - Smoke Testing
      - Integration Testing
      - Systems Testing
      - Interface Testing
      - Sanity Testing
      - Regression Testing
      - Acceptance Testing
    - - To check whether the system is functionally perfect,
        the functions are “what” the system should do
  - - - The purpose of regression testing is to verify that modifications in the software or the environment do not cause unintended adverse effects and that the system still meets its requirements.
- - - - This kind of testing is performed by testers
      - Integrating testing checks the data flow from one module to other modules
      - It aims at finding interfacing issues between the modules
      - Top down, Bottom up, Big Bang, Hyrbird
      - Integration testing is the testing of a group of related modules.
    - - A Unit is a smallest testable portion of system or application which can be compiled, liked, loaded, and executed
      - Usually performed by the developers
      - It checks that component are fulfilling functionalities or not
      - The Unit tests are always automated and more specifically use White-box testing technique as the knowledge of the piece of code and internal architecture is required to test the code
      - Tools: JUnit, TestNG, xUnit, NUnit...
    - - Testing each aspect of the application as seen from the outside without looking at the actual source code
      - It is carried out by the QA team
      - System test falls under the Black-box testing category of software testing
      - Both the functional and non-functional requirements of the system are evaluated
      - Involves the external workings of the software from the user's perspective
      - Determining if the application conforms to its business requirements
      - Where the complete integrated application is tested as a whole
    - - Where the system is tested for compliance to its business requirements
      - With the intent to see if the product is fit for delivery
      - is basically done by the user or customer and other stakeholders
      - Types of Acceptance Testing
        
        Alpha Testing
        
        can also be white-box along with black-box tests
        
        can be carried out by both in-house developers and QAs as well as the potential end users as well
        
        takes place at the developer's site and is not open to the world
        
        Beta Testing
        
        is performed after alpha testing and in the real world environment without the presence or control of developers
        
        are normally open to the whole world (or client)
        
        takes place at the customer's or the end user's site
        
        are only black-box