Please enable JavaScript.
Coggle requires JavaScript to display documents.
Tool Support for Testing - Coggle Diagram
Tool Support
for Testing
Test Tool
Considerations
Tools that help to manage requirements, test cases, test procedures, automated test scripts, test results, test data, and defects, and for reporting and monitoring test execution
Tools that are used for analysis and evaluation
Tools that are directly used in testing, such as test execution tools and test data preparation tools
y tool that assists in testing (a spreadsheet is also a test tool in this meaning) - bảng tính
Test Tool
Classification
Purpose:
Automate activities that cannot be executed manually (e.g., large scale performance testing)
Improve the quality of test activities by allowing for more consistent testing and a higher level of defect reproducibility
Improve the efficiency of test activities by supporting manual test activities throughout the test process
Improve the efficiency of test activities by automating repetitive tasks or tasks that require significant resources when done manually (e.g., test execution, regression testing)
Increase reliability of testing (e.g., by automating large data comparisons or simulating behavior)
Tools can be classified based on several criteria such as purpose, pricing, licensing model (e.g., commercial or open source), and technology used
6 test
tool types
Tool support for test
design and implementation
Test design tools aid in the creation of maintainable work products in test design and implementation,
including test cases, test procedures and test data
Example:
Model-Based testing tools
Test data preparation tools
In some cases, tools that support test
In some cases, tools that support test design and implementation may also support test execution and logging, or provide their outputs directly to other tools that support test execution and logging.
Tool support for specialized
testing needs (chuyên biệt)
support the general test process
support more specific testing for non-functional characteristics
Tool support for
static testing
Static analysis tools (D)
Tool support for test
execution and logging
Coverage tools (requirements coverage, code coverage (D))
Test harnesses (D)
Test execution tools (e.g., to run regression tests)
Tool support for management
of testing and testware
Defect management tools
Configuration management tools
Requirements management tools (traceability to test objects)
Continuous integration tools (D)
Test management tools and application lifecycle management tools (ALM)
Tool support for performance
measurement & dynamic analysis
these activities cannot effectively be done manually
Examples
Performance testing tools
Dynamic analysis tools (D)
Benefits & Risks of Test Automation
(only apply to test execution tool)
Benefit
Greater consistency and repeatability (e.g., test data is created in a coherent manner (nhất quán), tests are executed by a tool in the same order with the same frequency, and tests are consistently derived from requirements)
More objective assessment (static measures, coverage) - khách quan
Reduction in repetitive manual work (e.g., running regression tests, environment set up/tear down tasks, re-entering the same test data, and checking against coding standards) -> saving time
Easier access to information about testing (e.g., statistics and graphs about test progress, defect rates and performance)
Risk
The effort required to maintain the test work products generated by the tool may be underestimated
The tool may be relied on too much (seen as a replacement for test design or execution, or the use of automated testing where manual testing would be better)
new platform or technology is not supported by the tool
Version control of test work products may be neglected (bỏ qua)
The time and effort needed to achieve significant and continuing benefits from the tool may be under-estimated (including the need for changes in the test process and continuous improvement in the way the tool is used)
Relationships and interoperability issues (khả năng tương tác) between critical tools may be neglected, such as requirements management tools, configuration management tools, defect management tools and tools from multiple vendors
open source project is suspended (đình chỉ)
There is no clear ownership (tuyền sở hữu) of the tool (e.g., for mentoring, updates, etc.)
The time, cost and effort for the initial introduction of a tool may be under-estimated (including training and external expertise) - chưa được ước tính, chuyên môn)
The tool vendor go out of business, retire the tool, or sell the tool to a different vendor
Expectations for the tool may be unrealistic (including functionality and ease of use) (thực tế, tính dễ sử dụng)
vendor provide a poor response for support, upgrades, and defect fixes
Special Considerations for Test
Execution and Test Management Tools
Test execution
tools
execute test objects using automated test scripts
requires significant effort in order to
achieve significant benefits
Data-driven test
approach (theo hướng):
separates out the test inputs and expected
results into a spreadsheet
uses a more generic test script that can read the input
data and execute the same test script with different data.
Keyword-driven
test approach
a generic script processes keywords describing the actions to be taken (also called action words), which then calls keyword scripts to process the associated test data
Capturing test approach
(cách tiếp cận kiểm tra nắm bắt)
does not scale to large numbers of test scripts (mở rộng đến)
A captured script is a linear representation (biểu diễn) with specific data and actions as part of each script
recording the actions of a manual tester
This type of script may be unstable (không ổn đinh) when unexpected events occur, and require ongoing maintenance as the system’s user interface evolves over time (phát triển)
The above approaches require someone to have expertise in the scripting language (testers, developers or specialists in test automation)
When using data-driven or keyword-driven test approaches testers who are not familiar with the scripting language can also contribute by creating test data and/or keywords for these predefined scripts.
Model-Based
testing (MBT) tools
enable a functional specification to be captured in the form of a model, such as an activity diagram
This task is generally performed by a system designer
interprets the model in order to create test case specifications (thông số kỹ thuật) which can be saved in a test management tool and/or executed by a test execution tool
Test management
tools
interface with other tools or
spreadsheets for various reasons:
maintain consistent traceability to requirements in a requirements management tool
link with test object version information in the configuration management tool
produce useful information in a format that fits the needs of the organization
Effective Use
of Tools
Pilot Projects for Introducing
a Tool into an Organization
After completing the tool selection and a successful proof-of-concept, introducing the selected tool into an
organization generally starts with a pilot project, which has the following objectives:
Gaining in-depth knowledge about the tool, understanding both its strengths and weaknesses
Evaluating how the tool fits with existing processes and practices, and determining what would need to change
Deciding on standard ways of using, managing, storing, and maintaining the tool and the test work products (e.g., deciding on naming conventions for files and tests, selecting coding standards, creating libraries and defining the modularity of test suites)
Assessing whether the benefits will be achieved at reasonable cost
Understanding the metrics that you wish the tool to collect and report, and configuring the tool to ensure these metrics can be captured and reported
Main Principles
for Tool Selection
Consideration of whether or not the tool is available for a free trial period (and for how long)
Consideration of pros and cons of various licensing models (e.g., commercial or open source)
Evaluation of the tool against clear requirements and objective criteria (khánh quan)
Evaluation of the vendor (including training, support and commercial aspects) or support for noncommercial (e.g., open source) tools
Understanding the build and continuous integration tools already in use within the organization, in order to ensure tool compatibility and integration (tính tương thích)
Identification of internal requirements for coaching and mentoring in the use of the tool
Understanding of the technologies used by the test object(s), in order to select a tool that is compatible with that technology (tương thích)
Evaluation of training needs, considering the testing (and test automation) skills of those who will be working directly with the tool(s)
Identification of opportunities for an improved test process supported by tools
Estimation of a cost-benefit ratio based on a concrete business case (if required)
Assessment of the maturity of the own organization, its strengths and weaknesses
a proof-of-concept evaluation should be done to establish whether the tool performs effectively with the software under test and within the current infrastructure or, if necessary, to identify changes needed to that infrastructure to use the tool effectively
Success Factors
for Tools
Defining guidelines for the use of the tool (e.g., internal standards for automation)
Implementing a way to gather usage information from the actual use of the tool
Providing training, coaching, and mentoring for tool users
Monitoring tool use and benefits
Adapting and improving processes to fit with the use of the tool
Providing support to the users of a given tool
Rolling out the tool to the rest of the organization incrementally (cung cấp)
Gathering lessons learned from all users
ensure that the tool is technically and organizationally integrated into the software development lifecycle, which may involve separate organizations responsible for operations and/or third party suppliers