Английская Википедия:Concurrent testing
Research[1] and literature[2] on concurrency testing and concurrent testing typically focuses on testing software and systems that use concurrent computing. The purpose is, as with most software testing, to understand the behaviour and performance of a software system that uses concurrent computing, particularly assessing the stability of a system or application during normal activity.
Research and study of program concurrency started in the 1950s,[3] with research and study of testing program concurrency appearing in the 1960s.[4] Examples of problems that concurrency testing might expose are incorrect shared memory access and unexpected order sequence of message or thread execution.[5]Шаблон:Rp[1] Resource contention resolution, scheduling, deadlock avoidance, priority inversion and race conditions are also highlighted.[6]Шаблон:Rp
Selected history & approaches of testing concurrency
Approaches to concurrency testing may be on a limited unit test level right up to system test level.[7]
Some approaches to research and application of testing program/software concurrency have been:
- This was considered to be ineffective for testing concurrency in a non-deterministic system and was equivalent to the testing of a sequential non-concurrent program on a system
- Considered likely to find some issues in non-deterministic software execution.
- This later became called non-deterministic testing.[9]
- This is an approach to set the system into a particular state so that code can be executed in a known order.
- An attempt to test synchronisation sequence combinations for a specified input (shared variable access not being corrupted, effectively testing race conditions variables). The sequence is typically derived for non-deterministic test execution.
- Structural Approaches / Static Analysis
- Analysis of code structure and static analysis tools.
- An example was a heuristic approach[11]
- This led to code checker development, for example jlint.[12] Research and comparison of static analysis and code checkers for concurrency bugs [13]
- See also List of tools for static code analysis
- Multi-user approach
Testing software and system concurrency should not be confused with stress testing, which is usually associated with loading a system beyond its defined limits. Testing of concurrent programs can exhibit problems when a system is performing within its defined limits. Most of the approaches above do not rely on overloading a system. Some literature[6]Шаблон:Rp states that testing of concurrency is a pre-requisite to stress testing.
Lessons learned from concurrency bug characteristics study
A study in 2008[11] analysed bug databases in a selection of open source software. It was thought to be the first real-world study of concurrency bugs. 105 bugs were classified as concurrency bugs and analysed, split as 31 being deadlock bugs and 74 non-deadlock bugs. The study had several findings, for potential follow-up and investigation:
- Approximately one-third of the concurrency bugs cause crashes or hanging programs.
- Most non-deadlock concurrency bugs are atomicity or order violation.
- I.e. focusing on atomicity (protected use of shared data) or sequence will potentially find most non-deadlock type bugs.
- Most concurrency bugs involve 1 or 2 threads.
- I.e. Heavy simultaneous users/usage is not the trigger for these bugs. There is a suggestion that pairwise testing may be effective to catch these types of bugs.
- Over 20% (7/31) deadlock bugs occurred with a single thread.
- Most deadlock concurrency bugs (30/31) involved only one or two resources.
- An implication that pairwise testing from a resource usage perspective could be applied to reveal deadlocks.
See also
- Software testing
- Scalability testing
- Load testing
- Software performance testing
- Scenario analysis
- Simulation
- Stress test (hardware)
- System testing
References
General References
- ↑ 1,0 1,1 Ошибка цитирования Неверный тег
<ref>
; для сносокwang
не указан текст - ↑ 2,0 2,1 Ошибка цитирования Неверный тег
<ref>
; для сносокdustin
не указан текст - ↑ Ошибка цитирования Неверный тег
<ref>
; для сносокpilot
не указан текст - ↑ Ошибка цитирования Неверный тег
<ref>
; для сносокdijkstra
не указан текст - ↑ Ошибка цитирования Неверный тег
<ref>
; для сносокbrito
не указан текст - ↑ 6,0 6,1 6,2 Ошибка цитирования Неверный тег
<ref>
; для сносокbinder
не указан текст - ↑ Ошибка цитирования Неверный тег
<ref>
; для сносокmelo
не указан текст - ↑ 8,0 8,1 8,2 Ошибка цитирования Неверный тег
<ref>
; для сносокtai
не указан текст - ↑ 9,0 9,1 Ошибка цитирования Неверный тег
<ref>
; для сносокhwang
не указан текст - ↑ Ошибка цитирования Неверный тег
<ref>
; для сносокqi
не указан текст - ↑ 11,0 11,1 Ошибка цитирования Неверный тег
<ref>
; для сносокlu2
не указан текст - ↑ Ошибка цитирования Неверный тег
<ref>
; для сносокartho
не указан текст - ↑ Ошибка цитирования Неверный тег
<ref>
; для сносокmanzoor
не указан текст