Please enable JavaScript.

Coggle requires JavaScript to display documents.

ISTQB Certified Tester Advanced Level Test Automation Engineering (CTAL…

- - - - Advantages
        
        Complex tests beyond manual capability
        
        Faster execution
        
        Less human error
        
        Better resource efficiency
        
        Quicker feedback on SUT quality
        
        Improved system reliability
        
        Consistent execution across cycles
        
        More tests per build vs manual
      - Disadvantages
        
        Additional project costs
        
        Initial investment required
        
        Development and maintenance time
        
        Need for clear objectives
        
        Test rigidity and less adaptability
        
        Potential introduction of additional defects
      - Limitations
        
        Not all manual tests can be automated
        
        Only verifies what programmed to do
        
        Limited to machine interpretable results
        
        Requires automated test oracle
        
        es la parte del script de prueba automatizado que decide si el resultado de una prueba es correcto o no, comparando el resultado real con el esperado
  - - - Waterfall Model
        
        Cada fase tiene su documentacion
        
        Test Automation Implementation in parallel or after implementation phase
        
        Test runs during verification phase
        
        Linear and sequential
      - V-Model
        
        Sequential process execution
        
        Test levels derived from requirements
        
        En paralelo se diseñan las pruebas con la
        parte de verificación pero se ejecutan cuando
        llegan a la parte de validación
        
        Component, integration, system, acceptance testing in validation phase
        
        TAF recommended for each level
      - Agile Development
        
        Countless automation possibilities
        
        TAEs and business decide roadmap
        
        Best practices: code reviews, pair programming
        
        Frequent automated execution
        
        In-sprint automation goal
    - - Tool Selection Criteria
        
        Project requirements as baseline
        
        Team composition and experience
        
        Low-code/no-code for non-programmers
        
        Language matching with SUT
        
        SUT analysis required
        
        Costs must be considered
        
        What the project wants to achieve over time
        to select a UI, API or other test tool
- - - - Design for Testability has these aspects
        
        Observability
        
        Determine actual vs expected results
        
        SUT provides interfaces for insight
        
        Controllability
        
        SUT provides interfaces for actions
        
        UI elements, function calls, protocols
        
        Architecture Transparency
        
        Clear documentation of components
        
        Understandable interfaces
        
        Foster (alentar) quality at all test levels
      - Configuration Solutions
        
        Accessibility identifiers
        
        locators del UI basicamente
        
        System environment variables
        
        Modifica el comportamiento de la app para facilitar las pruebas
        
        Deployment variables
      - Testability of the SUT should be design and implemented in parallel with other features of the SUT. Its done by the Architecture sometimes with help of the TAEs. Its a non-functional requirement of the system.
    - - Local Development Environment
        
        Initial software creation
        
        Component testing with automation
        
        Can happen GUI and API testing
        
        White box testing with IDE
      - Build Environment
        
        Software building and testing
        
        DevOps ecosystem
        
        Low-level tests execution
        
        Static analysis
      - Integration Environment (es QA)
        
        System integration testing
        
        Fully integrated SUT
        
        Black-box testing only
        
        Monitoring presence required
      - Preproduction Environment
        
        Non-functional quality assessment
        
        Performance efficiency focus
        
        User acceptance testing
        
        Resembles production closely
      - Production Environment
        
        Real-time assessment
        
        Live user interactions
        
        Monitoring and best practices
        
        Canary release, blue/green deployment
  - - - Test process activities to automate
      - Test levels to support
      - Test types to support
      - Test roles and skill sets
      - Software products and families
      - SUT compatibility requirements
      - Test data availability and quality
      - Methods for unreachable cases
    - - Language and technology
      - Configuration capabilities
      - Test data management
      - Test type support
      - Reporting capabilities
      - Integration with other tools
      - Architecture scalability
      - Maintainability and reliability
- - - - Generic Test Automation Architecture (gTAA)
        
        Abstract communication view
        
        Interfaces
        
        SUT interface
        
        Project management interface -> Describes the test automation development progress
        
        Test management interface -> Describes the mapping of test cases definitions and automated test cases
        
        Configuration management interface -> Describes the CI/CD pipelines, enviroments and testware, control de versiones ( es github, ADO, etc)
        
        High-level design concept
      - Capabilities provided by test automation tools and libraries
        (Core test automation capabilities should be identified and selected from available tools as required for a given project)
        
        Test Generation
        
        Automated design from test model
        
        Model-based testing tools
        
        Optional capability
        
        Test Definition
        
        Implementation and definition of test cases/suites
        
        Separation from SUT and tools, no esta casada con una herramienta de automation como selenium
        
        High-level and low-level tests
        
        Es el proceso y conjunto de artefactos que permiten definir e implementar casos de prueba y/o suites de prueba, de manera que puedan ser reutilizados, mantenidos y ejecutados automáticamente. En otras palabras, es donde se describe qué se va a probar y cómo, sin preocuparse aún por la ejecución o adaptación técnica.
        
        Test Execution
        
        Automated test execution
        
        Test logging and reporting
        
        Test Adaptation
        
        Functionality to adapt test in diferent ways
        
        Different adaptors for APIs, protocols, UI, Mobile
    - - Requirements Consideration
        
        Functional requirements
        
        Non-functional requirements
        
        Technical requirements
        
        Existing or required tools that are necessary to implement a solution
      - Architecture Components for TAS
        
        Tool and library selection
        
        Plugin and component development
        
        Connectivity and interface requirements (e.g. database, mocks, connections, etc)
        
        Connecting to the test management and defect management tools
        
        Version control systems
    - - Test Automation Framework Structure
        
        TAF is the foundation of TAS
        
        Test harness (test runner)
        
        Test libraries, scripts, suites
      - TAF Layers Architecture
        TAF layers define a distinct border of
        classes that have similar purposes such
        as test cases, test reporting, test logging,
        encryption, and test harnesses
        
        Test Scripts Layer
        
        Test case repository
        
        Test suite annotations
        
        Calls business logic services, como por ejemplo metodo login.
        Son los test
        
        No direct core library calls
        
        Business Logic Layer
        
        SUT dependent libraries
        
        Inherits from core libraries
        
        TAF configuration for SUT
        
        Core Libraries Layer
        
        SUT independent libraries
        
        Reusable across projects
        
        Same development stack
    - - Capture/Playback
        
        Pros: Initially easy setup and use
        
        Cons: Hard to maintain, scale, evolve
        ● The SUT needs to be available while capturing a test case
        ● Only feasible for a small scope and an SUT that rarely changes
        ● The captured SUT execution depends highly on the SUT version from which the capture has been taken
        ● Recording each individual test case instead of reusing existing building blocks is time consuming
      - Linear Scripting
        
        Pros: Easy setup, modifiable scripts
        Compared to capture/playback, the test scripts can be modified more easily
        
        Cons: Hard maintenance, programming knowledge needed
        ● The SUT needs to be available while capturing a test case
        ● Only feasible for a small scope and an SUT that rarely changes
        ● Compared to capture/playback, some programming knowledge is necessary
      - Structured Scripting
        
        Pros: Easy maintenance, business logic separation
        
        Cons: Programming knowledge, initial investment into TAF development its time consuming
      - Test-Driven Development (TDD)
        
        Red-Green-Refactor process
        
        Pros: Better code quality, testability, easier code coverage, Reduces defect propagation, better comunication between devs testers and business
        
        Cons: Initial learning curve, proper implementation needed (Not following TDD properly can result in false confidence in code quality )
      - Data-Driven Testing (DDT)
        
        Test scripts with external data
        
        Pros: Quick test expansion, reduced costs
        
        Cons: Test data management required
      - Keyword-Driven Testing (KDT)
        
        Test cases as keyword tables/list of steps
        
        Pros: Non-technical involvement possible
        
        Cons:Implementing and maintaining keywords is a complex task, high effort for small systems
      - Behavior-Driven Development (BDD)
        
        Given-When-Then format
        
        Natural language scenarios
        
        Pros: Better communication, specification coverage
        BDD can be leveraged in producing multiple test types on different levels of the test pyramid
        
        Cons: Additional test cases needed, complex implementation
        Need hole team approach
        Overly complex test steps will turn debugging into a difficult and costly activity
    - - Object-Oriented Principles
        
        Encapsulation
        
        Mostrar solo los detalles esenciales y ocultar la complejidad. Ej. Interfase Vehiculo tiene mentodos como acelerar() y frenar() sin importar internamente como lo hace un Auto o Moto
        
        Abstraction
        
        Consiste en mostrar solo lo esencial de un objeto y ocultar los detalles innecesarios. Clases abstractas
        
        Inheritance
        
        Permite que una clase hija herede atributos y métodos de una clase padre.
        
        Polymorphism
        
        metodo heredado puede ser sobreescrito por el hijo completamente por uno nuevo
      - SOLID Principles
        
        Single responsibility
        
        Open-closed
        
        Liskov substitution
        
        Interface substitution
        
        Dependency inversion
      - Design Patterns
        
        Facade Pattern
        
        Hides implementation details
        
        Exposes only necessary elements
        
        Singleton Pattern
        
        Single driver communication
        
        Page Object Model
        
        Class file per page
        
        Centralized locator updates
        
        Flow Model Pattern
        
        Extension of page object model
        
        Additional facade over page objects
        
        Improved abstraction and maintainability
- - - - Scope validation definition, should be done in short time
      - Evaluation criteria
        
        Programming languages
        
        Commercial/open-source tools
        
        Test levels coverage
        
        Test case selection
        
        Test case Development approach
      - Timeline definition
      - CI/CD integration during pilot
      - Non-technical aspects evaluation
        
        Team knowledge and experience
        
        Team structure
        
        Licensing and organization rules
        
        Planned testing types and targeted test levels to cover
  - - - Deployment Risks
        
        Resource utilization (CPU, RAM)
        
        Network connection reliability
        
        Device-specific requirements
        
        Firewall openings
      - Technical Deployment Risks
        
        Packaging
        
        Version control importance
        
        Repository sharing
        
        Cloud or on-premises
        
        Logging
        
        Multiple logging levels
        
        Fatal, Error, Warn, Info, Debug, Trace
        
        Test result information
        
        Test Structuring
        
        Test harness and fixtures
        
        Test environment control
        
        Preconditions and postconditions
        
        Repeatable and atomic tests
        
        Updating
        
        Automatic updates risks
        
        Version changes on devices
        
        Mitigation strategies
  - - - Clean Code Principles
        
        Logical and common project structure
        
        Avoid hardcoding
        
        Limit method parameters
        
        Avoid long complex methods
        
        Use logging
        
        Apply beneficial design patterns
        
        Focus on testability
        
        Common naming conventions
      - Additional Factors
        
        Data-driven approach
        
        Constants usage
        
        Design pattern leverage
        
        Static analyzers usage
        
        Code formatters
        
        Agreed Branching strategies for versioning
        (Using different branches for features, releases, and defect fixes is helpful in understanding the branch contents.)
- - - - Test Level Integration
        
        Configuration Tests
        
        File path verification
        
        These tests are run during the build of a test automation project (TAF/TAS)
        
        Component Tests
        
        Build step of the pipeline
        
        Ejemplo de Pipeline
        jobs:
        build:
        runs-on: ubuntu-latest
        steps:
        - name: Checkout repo uses: actions/checkout@v3 - name: Setup Node.js uses: actions/setup-node@v3 with: node-version: '18.x' - name: Install dependencies run: npm install - name: Run configuration tests run: npm run config-test - name: Run component tests run: npm run component-test - name: Run component integration tests run: npm run component-integration-test
        
        Individual component testing
        
        Crucial pipeline quality gates
        
        Component Integration Tests
        
        CI pipeline integration
        
        Low-level component testing
        
        Build step of the pipeline
        
        System Tests
        
        CD pipeline integration
        
        Final quality gate
        
        System Integration Tests
        
        CD pipeline quality gates
        
        Separate component integration
        
        Execution Approaches
        
        Test cases are executed as a separate pipeline, triggered by the successful deployment
        
        tests do not act as a quality gate
        
        it requires other, usually manual, actions to roll back an unsuccessful deployment.
        
        Test cases are executed as part of the deployment phase
        
        In case tests need to be rerun a redeplyment needs to be done
        
        Based on the tests results, the deployment can fail and be rolled back automatically
        
        Acceptance testing
      - Pipeline Purposes
        
        Periodic test suite execution
        
        Non-functional testing
    - - Configuration Management
        
        Components
        
        Test Environment Configuration
        
        Stored with testware
        
        Core library sharing
        
        Different URLs and credentials
        
        Test Data
        
        Environment specific data
        
        Release specific data
        
        Data management systems
        
        Test Suites/Cases
        
        Purpose-based suites (e.g. smoke, regression)
        
        Different pipeline execution for each test level
        
        Release Management
        
        Each release of the SUT
        determines a feature, that can be
        tested with these optiones
        
        Feature Toggle Configuration
        
        2 more items...
        
        Testware Versioning
        
        2 more items...
    - - API Infrastructure Dependencies
        
        Required Information
        
        API Connections
        
        relationship between APIs
        
        Business logic understanding
        
        API Documentation
        
        Baseline for automation
        
        Parameters and headers
        
        Request-response objects
        
        Contract Testing
        (es un tipo de prueba
        de integracion)
        
        Integration testing type
        
        Service communication verification
        
        Data consistency rules
        
        Compatibility across systems
        
        Interaction capturing in contracts
        
        Consumer-driven approach
        
        El proveedor debe adaptarse a esas expectativas.
        Ventaja: asegura que el consumidor nunca se rompa.
        Riesgo: si hay muchos consumidores distintos, el proveedor puede terminar atado a mil contratos diferentes.
        
        Provider-driven approach
        
        Quién define el contrato: el proveedor (dueño de la API).
        El proveedor dice:
        “Mi endpoint /users/{id} siempre devuelve estos campos.”
        Los consumidores deben adaptarse a lo que el proveedor define
- - - - Data Sources
        
        SUT Logs
        
        APIs and applications
        
        Web and database servers
        
        Web/mobile UI
        
        TAF Logs
        
        Audit trail provision
        
        Build and Deployment Logs
        
        Production Logs
        
        Performance monitoring
        
        Trend analysis
        
        Load, stress, spike testing
        
        Screenshots and Recordings
        
        Native tool features
        
        Third-party solutions
      - Test Automation Features
        
        Scripting Language Support of many test tools support
        
        Record and log Before, during, after execution of tests and suites
        
        Test Reporting Analysis
        
        Previous test run consideration
        
        Trend highlighting
        
        Success rate changes
        
        Automated Verification
        
        Actual vs expected comparison
        
        Tool-supported assertions
        
        Clear pass/fail determination
        
        Failure cause information
      - Logging Categories
        
        TAS Logging
        
        Test case identification - start and end time
        
        Execution status tracking - passed, failure or TAS failure
        
        Test step details with timing
        
        Dynamic SUT information (e.g. memory leaks)
        
        Counter for reliability or stress testing
        
        Si hay datos generados random para el test, loggearlos
        
        Action replay capability
        
        Screenshot capture
        
        Failure information storage
        
        Color coding usage
        
        SUT Logging
        
        Defect analysis information
        
        Configuration logging
        
        Searchable logs to match logs of SUT with TAS t failures
        
        Time stamp synchronization
      - Integration and Visualization
        
        Third-party Tool Integration
        
        Spreadsheets, XML, documents
        
        Database and report tools
        
        Traceability information
        
        Export format compatibility
        
        Visual Results
        
        Chart utilization
        
        Traffic light indicators
        
        Management summaries
        
        Drill-down capabilities
    - - Data Analysis
        
        Test Result Analysis
        
        Test environment data analysis
        
        Resource management (CPU, RAM)
        
        Single vs multi browser test excecution
        
        Cloud sizing support
        
        Previous execution comparison
        
        Web log usage monitoring
        
        Failure Analysis Process
        
        Historical failure checking (por si es por ejemplo un known defect)
        
        Test case identification
        
        Test step failure location
        
        analysis wheter it matches state of the SUT and expected result with screen shots, api or network logs.
        
        Defect logging procedures
        
        Root Cause Identification
        
        TAS defect possibility
        
        Environment availability issues
        
        we can see SUT logs for search root cause
    - - Test Progress Reporting
        
        Report Content
        
        SUT information
        
        Test environment documentation
        
        Stakeholder-appropriate format
        
        Failure reasons
        
        Test execution history
        
        Responsible person identification
        
        Test results information
        
        Report Publishing
        
        Stakeholder distribution
        
        Website or cloud upload or Mailing list distribution or Test management tool integration
        
        Stakeholder Categories
        
        Management Stakeholders
        
        Solution/enterprise architects
        
        Project/delivery managers
        
        Test managers/directors
        
        Operational Stakeholders
        
        Product owners/managers
        
        Business representatives
        
        Business analysts
        
        Technical Stakeholders
        
        Team leaders, scrum masters
        
        Administrators and developers
        
        Test leaders, TAEs, testers
        
        Dashboard Creation
        
        Modern reporting tools
        
        Colorful charts and visualizations
        
        Data aggregation capabilities
        
        Trend identification
        
        Decision-making support
        
        AI/ML Analysis
        
        Machine learning algorithms
        
        Large data analysis
        
        Automated failure analysis
        
        Common defect grouping
        
        Time reduction benefits
- - - - Environment Verification Planning
        
        Test Tool Setup
        
        Automated vs manual setup
        
        Repository advantages
        
        Version control importance
        
        Installation and configuration
        
        Repeatability in setup/teardown
        of the test environment
        
        Tiene que poder ejecutarse igual igual que antes
        
        Connectivity with internal and
        external systems/interfaces
        
        Internal and external system connections are OK
        
        Verify TAS can access SUT
        
        TAF Own Component Testing
        
        Individual component verification
        
        Functional and non-functional testing
        
        Object verification testing
    - - Test Suite Verification
        
        Check the composition of the test suite
        
        Test cases with all expected results
        
        Test data is present
        
        Correct TAF and SUT versions
        
        Verify new tests that focus
        on new features of the TAF
        
        First-time a new feature of the TAF is used in TC should be verified
        
        Close monitoring requirement
        
        Test Repeatability
        
        Consistent test results
        
        Unreliable test identification
        
        Consider the intrusiveness
        of automated test tools
        
        SUT integration impact
        
        Performance considerations
    - - Root Cause Analysis
        
        Test script failure investigation
        
        Test log inspection
        
        Performance data analysis
        
        Setup and teardown review
      - Isolation Testing
        
        Individual test execution
        
        Intermittent failure are more difficult to analyze
        
        Defect can be in TC, SUT, TAF, hardware or network
        
        System resource monitoring may yield clues for the root cause
      - Assertion Verification
        
        Verify if all the assertions are in place. Missing assertions may result in inconclusive test results.
    - - Automated Code Scanning
        
        Vulnerability detection
        
        Defect identification
        
        Code quality standards
        
        Proactive defect detection
        
        DevSecOps implementation
      - Output Categorization
        
        Severity levels
        
        Critical, High, Medium, Low
        
        Prioritization support
        
        Suggested fixes provision
- - - - Data Collection and Analysis
        
        Test Histogram
        
        Improvement area identification
        
        CI/CD tool capabilities
        
        Fragile test case identification
        
        Visual test data representation
        
        Artificial Intelligence
        
        UI testing support
        
        Locator change detection
        
        Self-healing algorithms
        
        Image recognition usage
        
        Maintenance acceleration
        
        Schema Validation
        
        API data analysis
        
        Business specification matching
        
        Mandatory element verification
    - - TAS Enhancement Areas
        
        Scripting Improvements
        
        Technique upgrades
        
        Linear scripting
        
        Data driven testing
        
        Keyword driven testing
        
        Implementation improvements
        
        Create libraries for similar actions, insted for having multiple times and parametrize it
        
        Establish failure recovery process for the TAS
        TAS should be able to recover from this to continue with next possible test.
        
        Wait mechanism evaluation
        
        Hard coded waits
        
        Dynamic waiting by polling
        
        The best option: subscribe to the event mechanism of the SUT but test scripting language needs to support event subscription and the SUT needs to offer these event to the TAS
        
        Test Execution Optimization
        
        Concurrent testing on different envs
        
        Regression suite splitting
        
        Duplication removal
        
        Parallel batch jobs in CI/CD
        
        Automated scheduling
        
        Verification Standardization
        
        Standard method adoption
        
        Re-implementation avoidance
        
        Parameterization usage
        
        Architecture Changes
        
        SUT testability improvements
        
        For example, if the SUT is going to be changed to provide APIs for testing then the TAS should also be refactored accordingly.
        
        Framework Updates
        
        When core libraries used in TAF has major updates that cannot be immediately since it would break the tests. It is advisable to perform a pilot and an impact analysis before adoption plan creation.
        
        Setup and Teardown
        
        Repeated action optimization
        
        Single-place updates for common code
        
        Documentation Enhancement
        
        Test automation documentation
        
        User documentation
        
        Test reports and logs
        
        TAS features Addition
        
        Detailed reporting
        
        System integration with other systems
        
        Value-focused features (only ones that will be used)
        
        TAF updates and upgrades
        
        Updating or upgrading to a new version of the TAF may have new function availability
        
        Impact assessment
        
        Sample testing before upgrade
    - - Identify changes in the
        test environment components
        
        Change Identification
        
        Evaluate what changes and improvements need to be made
        
        The overall goal is to ensure automated tests continue to run in an efficient manner. Changes should be made incrementally, with a minimum viable product mindset, so that the impact on the TAS can be measured through a limited run of test scripts. Once it is found that no side effect exists, changes can be fully implemented
        
        A full regression run is the last step toward verifying that the change did not adversely affect the automated test scripts
      - Increase efficiency and effectiveness
        of core TAS function libraries
        
        Evaluation of existing test scripts
        for SUT revision/elimination
        
        Continued value of test script assessment
        
        Complex test decomposition in smaller test
        
        Infrequent test elimination
        
        Complexity reduction
        
        As a TAS matures, new ways are discovered to perform tasks more efficiently.
      - Refactor the TAA to accommodate changes in the SUT
        
        As the SUT evolves and matures, the underlying TAA will have to evolve as well to ensure that the capability is
        there to support the SUT
      - Naming conventions and standardization
        
        Naming convention consistency
        
        Previously defined standards
      - Target multiple functions that act on the
        same control type for consolidation
        
        For example, there may be a specific function that works only on dropdown lists.
        Alternatively, there may be a function that works with several functions by specifying a function as one of its parameters. Therefore, a TAE may use several functions that can be consolidated into fewer functions, achieving the same results, and minimizing maintenance.
    - - Test automation can help in
        nonspecific test activities such as
        
        Environment Setup and Control
        
        Test data creation automation
        
        User profile generation
        
        Infrastructure setup
        
        Cleanup process automation
        
        Data Aging
        
        Test data manipulation
        
        Database date field control
        
        Year perspective maintenance
        
        Screenshot and Video Generation
        
        Built-in tool capabilities
        
        Documentation support
        
        Marketing material creation