Please enable JavaScript.
Coggle requires JavaScript to display documents.
Chapter 6: SYSTEM INTEGRATION QUALITY - Coggle Diagram
Chapter 6: SYSTEM INTEGRATION QUALITY
Performance Testing and Monitoring
Performance Testing:
Definition
: Type of software testing ensuring applications perform well under expected workload.
Focus Areas:
Software application's performance:
Response time
Reliability
Resource usage
Scalability
Objective
: Not to find bugs but to eliminate performance bottlenecks.
Commonly Known As: "Perf Testing"
Reasons for Performance Testing:
Provides stakeholders with information about application speed, stability, and scalability.
Uncovers areas for improvement before product launch.
Determines if software meets speed, scalability, and stability requirements under expected workloads.
Essential for mission-critical applications like space launch programs or medical equipment.
Downtime costs for Fortune 500 companies can amount to millions annually.
Google.com experienced a 5-minute downtime costing an estimated $545,000.
Amazon Web Service outage led to companies losing sales worth $1100 per second.
Types of Performance Testing:
Load testing
: Checks application performance under anticipated user loads, identifying bottlenecks before going live.
Stress testing
: Tests application under extreme workloads to assess how it handles high traffic or data processing, finding the breaking point.
Endurance testing
: Ensures software can sustain expected load over a prolonged period.
Spike testing
: Evaluates software's response to sudden large spikes in user-generated load.
Volume testing:
Populates a database with a large volume of data to monitor software performance under varying loads.
Scalability testing:
Determines software's ability to "scale up" effectively to support increased user loads, aiding capacity planning.
Common Performance Problems:
Long Load Time:
Initial time for application startup should be minimized, ideally under a few seconds.
Poor Response Time:
Time from user input to application response should be quick to maintain user interest.
Poor Scalability:
Inability to handle expected number of users or accommodate a wide range of users.
Bottlenecking:
Coding errors or hardware issues causing decreased throughput under certain loads.
Performance Testing Process:
Identify Testing Environment:
Understand physical test environment and available testing tools.
Anticipate challenges that may arise during performance testing.
Identify Performance Acceptance Criteria:
Establish goals and constraints for throughput, response times, and resource allocation.
Plan & Design Performance Tests:
Determine usage variations among end users.
Identify key scenarios for testing all possible use cases.
Simulate various end users and plan performance test data.
Define metrics to be gathered during testing.
Configure the Test Environment:
Prepare the testing environment prior to execution.
Implement Test Design:
Develop performance tests based on the test design.
Run the Tests:
Execute and monitor the performance tests.
Analyze, Tune, and Retest:
Conduct retesting as needed.
Make necessary adjustments and optimizations.
Consolidate and analyze test results.
This structured process ensures thorough performance testing and optimization of software applications.
Example Performance Test Cases:
Verify response time remains under 4 seconds when 1000 users access the website simultaneously.
Ensure the response time of the Application Under Load stays within an acceptable range during slow network connectivity.
Determine the maximum number of users the application can handle before crashing.
Evaluate database execution time with 500 records read/written simultaneously.
Monitor CPU and memory usage of the application and database server under peak load conditions.
Test response time of the application under varying load conditions: low, normal, moderate, and heavy.
Performance Test Tools:
LoadNinja: Cloud-based load testing tool facilitating recording and playback of comprehensive load tests in real browsers.
NeoLoad: Performance testing platform designed for DevOps, integrating seamlessly into existing Continuous Delivery pipelines.
HP LoadRunner: Popular performance testing tool capable of simulating large user loads to evaluate application behavior.
JMeter: Leading tool for load testing web and application servers, offering flexibility and extensive capabilities.
Service Level Agreements (SLA)
Contract between service provider and customers.
Documents services provided and service standards.
Defines obligations of the service provider.
Importance of SLAs:
Manage customer expectations and define liability for outages or performance issues.
Describe performance characteristics for comparison with other vendors' SLAs.
Set means for addressing service issues, such as through service credits.
SLA Checklist:
Statement of objectives.
Scope of services to be covered.
Service provider responsibilities.
Customer responsibilities.
Performance metrics (response time, resolution time, etc.).
Penalties for contract breach/exclusion.
Benefits of SLA Implementation:
Creation of a customer-oriented environment, focusing on service outcomes.
Increased service delivery efficiencies.
Improved resource utilization.
Clear performance expectations for both customer and service provider.
Greater clarity regarding roles, responsibilities, and priorities.
Continuous improvement of service quality.
Better service provision to the TRIS University community, especially students and staff.
Performance Metrics in SLAs:
Availability and uptime percentage:
Tracks the time services are accessible to customers.
Specific performance benchmarks:
Defined standards periodically compared against actual performance.
Service provider response time:
Time taken by the service provider to respond to customer issues or requests.
Resolution time:
Duration to resolve an issue once it's logged by the service provider.
Schedule for advance notification of network changes affecting users.
Usage statistics provided to users.
Penalties for Breaking SLA Terms:
Plan for addressing downtime and compensation for customers in case of a breach.
Compensation may include credits proportional to the duration of SLA breaches.
Service providers may set a maximum dollar amount for performance penalties.
Exclusions section detailing situations where SLA guarantees and penalties don't apply.
Excluded events might include natural disasters or terrorist acts.
Entities that Establish SLAs:
IT service providers, managed service providers, and cloud computing service providers.
Corporate IT organizations practicing IT service management (ITSM).
IT departments seeking to measure, justify, and compare services, including outsourcing vendors.
Performance Testing Metrics: Parameters Monitored
Memory Use:
Amount of physical memory available to processes on a computer.
Disk Time:
Amount of time disk is busy executing a read or write request.
Bandwidth:
Shows the bits per second used by a network interface.
Private Bytes:
Number of bytes a process has allocated that can't be shared amongst other processes. Used to measure memory leaks and usage.
Committed Memory:
Amount of virtual memory used.
Memory Pages/Second:
Number of pages written to or read from the disk to resolve hard page faults.
Page Faults/Second:
Overall rate at which fault pages are processed by the processor.
CPU Interrupts per Second:
Average number of hardware interrupts a processor is receiving and processing each second.
Disk Queue Length:
Average number of read and write requests queued for the selected disk during a sample interval.
Network Output Queue Length:
Length of the output packet queue in packets.
More than two indicates delay and potential bottlenecking.
Network Bytes Total per Second:
Rate at which bytes are sent and received on the interface, including framing characters.
Response Time:
Time from when a user enters a request until the first character of the response is received.
Throughput:
Rate at which a computer or network receives requests per second.
Amount of Connection Pooling:
Number of user requests met by pooled connections.
More requests met by connections in the pool indicate better performance.
Maximum Active Sessions:
Maximum number of sessions that can be active at once.
Hit Ratios:
Number of SQL statements handled by cached data instead of expensive I/O operations.
Helps identify and address bottlenecking issues.
Hits per Second:
Number of hits on a web server during each second of a load test.
Rollback Segment:
Amount of data that can rollback at any point in time.
Database Locks:
Monitoring of table and database locks, crucial for tuning database performance.
Top Waits:
Monitored to determine which wait times can be minimized for faster data retrieval from memory.
Thread Counts:
Health of an application measured by the number of running and active threads.
Garbage Collection:
Process of returning unused memory back to the system.
Efficiency of garbage collection needs to be monitored.