10 Testability 可测试性

Testing leads to failure, and failure leads to understanding
测试导致失败，失败导致理解
—Burt Rutan

Industry estimates indicate that between 30 and 50 percent (or in some cases, even more) of the cost of developing well-engineered systems is taken up by testing. If the software architect can reduce this cost, the payoff is large.

行业估计表明，开发精心设计的系统的成本中有30%到50%（在某些情况下，甚至更多）由测试承担。如果软件架构师能够降低这一成本，回报将是巨大的。

Software testability refers to the ease with which software can be made to demonstrate its faults through (typically execution-based) testing. Specifically, testability refers to the probability, assuming that the software has at least one fault, that it will fail on its next test execution. Intuitively, a system is testable if it “gives up” its faults easily. If a fault is present in a system, then we want it to fail during testing as quickly as possible. Of course, calculating this probability is not easy and, as you will see when we discuss response measures for testability, other measures will be used.

软件可测试性是指通过（通常是基于执行的）测试，软件可以很容易地证明其故障。具体地说，可测试性是指假设软件至少有一个故障，它在下一次测试执行中失败的概率。直观地说，如果一个系统很容易“放弃”其故障，那么它是可测试的。如果系统中存在故障，那么我们希望它在测试过程中尽快失败。当然，计算这种概率并不容易，正如我们讨论可测试性的响应度量时所看到的，将使用其他度量。

Figure 10.1 shows a model of testing in which a program processes input and produces output. An oracle is an agent (human or mechanical) that decides whether the output is correct or not by comparing the output to the program’s specification. Output is not just the functionally produced value, but it also can include derived measures of quality attributes such as how long it took to produce the output. Figure 10.1 also shows that the program’s internal state can also be shown to the oracle, and an oracle can decide whether that is correct or not—that is, it can detect whether the program has entered an erroneous state and render a judgment as to the correctness of the program.

图10.1显示了程序处理输入并产生输出的测试模型。oracle是一种代理（人工或机械），通过将输出与程序规范进行比较来决定输出是否正确。输出不仅仅是功能上产生的价值，它还可以包括质量属性的衍生度量，比如产生输出所需的时间。图10.1还显示，程序的内部状态也可以显示给oracle，oracle可以决定是否正确，也就是说，它可以检测程序是否进入错误状态，并对程序的正确性做出判断。

Setting and examining a program’s internal state is an aspect of testing that will figure prominently in our tactics for testability.

设置和检查程序的内部状态是测试的一个方面，它将在我们的可测试性策略中占据重要地位。

[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-y4l9qupO-1643793509448)(fig.10.1)]
Figure 10.1 A model of testing
图10.1 测试模型

For a system to be properly testable, it must be possible to control each component’s inputs (and possibly manipulate its internal state) and then to observe its outputs (and possibly its internal state, either after or on the way to computing the outputs). Frequently this control and observation is done through the use of a test harness, which is specialized software (or in some cases, hardware) designed to exercise the software under test. Test harnesses come in various forms, such as a record-and-playback capability for data sent across various interfaces, or a simulator for an external environment in which a piece of embedded software is tested, or even during production (see sidebar). The test harness can provide assistance in executing the test procedures and recording the output. A test harness can be a substantial piece of software in its own right, with its own architecture, stakeholders, and quality attribute requirements.

为了使系统能够正确测试，必须能够控制每个组件的输入（并可能操纵其内部状态），然后观察其输出（以及可能的内部状态，无论是在计算输出之后还是在计算输出的过程中）。通常，这种控制和观察是通过使用测试工具来完成的，测试工具是专门设计用于测试软件的软件（或在某些情况下是硬件）。测试工具有多种形式，例如用于记录和回放通过各种接口发送的数据的功能，或用于测试嵌入式软件的外部环境的模拟器，甚至在生产过程中（参见侧栏）。测试工具可以在执行测试程序和记录输出方面提供帮助。测试工具本身可以是一个重要的软件，具有自己的体系结构、涉众和质量属性需求。

Testing is carried out by various developers, users, or quality assurance personnel. Portions of the system or the entire system may be tested. The response measures for testability deal with how effective the tests are in discovering faults and how long it takes to perform the tests to some desired level of coverage. Test cases can be written by the developers, the testing group, or the customer. The test cases can be a portion of acceptance testing or can drive the development as they do in certain types of Agile methodologies.

测试由不同的开发人员、用户或质量保证人员执行。可以对系统的部分或整个系统进行测试。可测试性的响应度量涉及测试在发现故障方面的有效性，以及执行测试到某种期望的覆盖水平所需的时间。测试用例可以由开发人员、测试组或客户编写。测试用例可以是验收测试的一部分，也可以像在某些类型的敏捷方法中那样推动开发。

Netflix’s Simian Army
奈飞的猿猴大军

Netflix distributes movies and television shows both via DVD and via streaming video. Their streaming video service has been extremely successful. In May 2011 Netflix streaming video accounted for 24 percent of the Internet traffic in North America. Naturally, high availability is important to Netflix.

奈飞通过DVD和流媒体视频分发电影和电视节目。他们的流媒体视频服务非常成功。2011年5月，Netflix流媒体视频占北美互联网流量的24%。当然，高可用性对Netflix很重要。

Netflix hosts their computer services in the Amazon EC2 cloud, and they utilize what they call a “Simian Army” as a portion of their testing process. They began with a Chaos Monkey, which randomly kills processes in the running system. This allows the monitoring of the effect of failed processes and gives the ability to ensure that the system does not fail or suffer serious degradation as a result of a process failure.

奈飞在亚马逊EC2云中托管他们的计算机服务，他们将所谓的“猿猴大军”作为测试过程的一部分。他们从一个混沌猴子开始，它随机杀死运行系统中的进程。这允许监控失效过程的影响，并确保系统不会因过程失效而失效或严重退化。

Recently, the Chaos Monkey got some friends to assist in the testing. Currently, the Netflix Simian Army includes these:

最近，混沌猴得到了一些朋友来协助测试。目前，奈飞的猿猴大军包括：

• The Latency Monkey induces artificial delays in the client-server communication layer to simulate service degradation and measures if upstream services respond appropriately.
  延迟猴子在客户机-服务器通信层诱导人为延迟，以模拟服务降级，并测量上游服务是否正确响应。
• The Conformity Monkey finds instances that don’t adhere to best practices and shuts them down. For example, if an instance does not belong to an auto-scaling group, it will not appropriately scale when demand goes up.
  一致性猴子会发现不符合最佳实践的实例，并将其关闭。例如，如果一个实例不属于自动缩放组，那么当需求增加时，它将不会适当地进行缩放。
• The Doctor Monkey taps into health checks that run on each instance as well as monitors other external signs of health (e.g., CPU load) to detect unhealthy instances.
  医生猴子会访问每个实例上运行的健康检查，并监控其他外部健康迹象（例如CPU负载），以检测不健康的实例。
• The Janitor Monkey ensures that the Netflix cloud environment is running free of clutter and waste. It searches for unused resources and disposes of them.
  看门人猴子确保奈飞云环境没有杂乱和浪费。它会搜索未使用的资源并进行处理。
• The Security Monkey is an extension of Conformity Monkey. It finds security violations or vulnerabilities, such as improperly configured security groups, and terminates the offending instances. It also ensures that all the SSL and digital rights management (DRM) certificates are valid and are not coming up for renewal.
  安全猴子是一致性猴子的延伸。它会发现安全违规或漏洞，例如未正确配置的安全组，并终止违规实例。它还可以确保所有SSL和数字版权管理（DRM）证书都是有效的，并且不会更新。
• The 10-18 Monkey (localization-internationalization) detects configuration and runtime problems in instances serving customers in multiple geographic regions, using different languages and character sets. The name 10-18 comes from L10n-i18n, a sort of shorthand for the words localization and internationalization.
  10-18猴子（本地化国际化）使用不同的语言和字符集，在为多个地理区域的客户提供服务的实例中检测配置和运行时问题。10-18这个名字来自L10n-i18n，是本地化和国际化的缩写。

Some of the members of the Simian Army use fault injection to place faults into the running system in a controlled and monitored fashion. Other members monitor various specialized aspects of the system and its environment. Both of these techniques have broader applicability than just Netflix.

猿猴大军的一些成员使用故障注入，以受控和监控的方式将故障注入运行系统。其他成员监控系统及其环境的各个专门方面。这两种技术都比奈飞具有更广泛的适用性。

Not all faults are equal in terms of severity. More emphasis should be placed on finding the most severe faults than on finding other faults. The Simian Army reflects a determination by Netflix that the faults they look for are the most serious in terms of their impact.

并非所有故障的严重性都相同。应更加重视发现最严重的故障，而不是发现其他故障。猿猴大军反映了奈飞的决心，即他们寻找的故障在影响方面是最严重的。

This strategy illustrates that some systems are too complex and adaptive to be tested fully, because some of their behaviors are emergent. An aspect of testing in that arena is logging of operational data produced by the system, so that when failures occur, the logged data can be analyzed in the lab to try to reproduce the faults. Architecturally this can require mechanisms to access and log certain system state. The Simian Army is one way to discover and log behavior in systems of this ilk.

这种策略说明，有些系统过于复杂，适应性太强，无法进行全面测试，因为它们的一些行为是突发的。该领域测试的一个方面是记录系统产生的运行数据，以便在发生故障时，可以在实验室分析记录的数据，以尝试重现故障。在架构上，这可能需要访问和记录特定系统状态的机制。猿猴大军是发现和记录这类系统行为的一种方法。

—LB

Testing of code is a special case of validation, which is making sure that an engineered artifact meets the needs of its stakeholders or is suitable for use. In Chapter 21 we will discuss architectural design reviews. This is another kind of validation, where the artifact being tested is the architecture. In this chapter we are concerned only with the testability of a running system and of its source code.

代码测试是验证的一种特殊情况，即确保工程工件满足其利益相关者的需求或适合使用。在第21章中，我们将讨论架构设计审查。这是另一种验证，测试的工件是架构。在本章中，我们只关注正在运行的系统及其源代码的可测试性。