寫了幾百次的測試講義…就是書上的摘録!!

一般科學，真理 的把關者就是鐵面無私的「檢驗」。

 1.軟體測試的發展簡史

 

1975, IEEE上發表了 “ 軟體數據選擇原理”，將軟體測試定論為一種研究方向。證實
1979, Glenford J. Myers 發行了 “軟體測試的藝術” , 證偽
1983, 軟體測試完全指南，缺陷預防
2000, 自動化測試技術盛行，向體系化發展, TMM 測試成熟度模型， TCMM 測試能力成熟度模型 等等出現
2002, 系統化軟體測試，測試是為了量測和提高軟體品質，對測試軟體進行工程設計、實施和維護的整個生命周期過程。

【Note】

2001, 敏捷軟體開發宣言, DevOps 等逐漸發展
2008, 敏捷軟體開發模式轉型，瀑布式開發測試開始轉型為敏捷式開發測試。手動式測試及自動化測試同步發展。

DevOps : 開發即運維

CI : 持續集成

CD : 持續交付

CICD 自動化流水線，再次縮短產品發布周期!

VUCA : Volatility 易變、Uncertainty 不定、Complexity 複雜、Ambiguity 模糊

2. 測試架構

與開發相比，測試更需具有系統和全局的視野視角，一個專業的測試人員可以就 基於 “對行業的理解”、”對用戶行為的剖析”、”對使用場景的狀況”、”對競爭對手的了解”等，對測試是否通過做出預測性的判斷。

要能夠對不同的組織、產品、研發模式做出最適合當下狀況的選擇並進行剛剛好的測試。

好的產品是設計出來的，測試分析不僅能夠幫助幫助測試人員更好的認識產品，還能更深入的幫助開發者確認需求設計。測試的意義不僅在通過測試發現缺陷、為產品發佈提供信心，還在於缺陷的預防，切實的的提升產品的質量。

2.1 需求分析的階段

需求是測試的源頭

理解需求 (商業目標和核心的價值)

制定測試策略 (以確定測什麼及怎麼測)

【Note】風險識別和有效的風險應對能力

2.2 測試分析和設計

以此確認測試設計中的測試的覆蓋度 (廣度與深度) 剛剛好。

測試設計大綱

【Note】
IEEE 2476-2010的定義，產品的品質是指在特定的使用條件下，產品滿足明示和隱含的需求的固有特性”，簡單來說是品質是滿足需求，但需要不是一個容易被了解的事情。

2.3 測試執行的階段

手動測試或自動化測試並分析當前測試項目和計劃的偏差

確認和計劃的偏差

選擇合適的測試案例

跟踪測試過程

【Note】明確測試目標、測試重點的能力

2.4 測試品質的評估

需求覆蓋度分析

code 的覆蓋度分析

【Note】品質分析和評估的能力

3. 系統測試計劃

基本測試約可分為 單元測試、集成測試、系統測試、驗收測試，每個測試階段又包含了 測試計劃、測試設計、測試實現、測試執行等活動。

3.1 軟體測試生命周期

軟體工程中有軟體生命周期，同樣測試也有軟體測試生命周期，它是指一個測試如何完成，
像測試計劃-> 測試設計->測試實現->測試執行就是一個典型的軟體測試生命周期。

3.2 系統測試的四個階段

系統測試是針對軟體產品系統進行的測試，在總體上包含有功能測試和非功能測試兩個部份。
．功能性測試：是驗證軟體系統功能是否符合軟體系統的需求規格的測試過程。
．非功能性測試：是在驗證軟體系統是否符合軟體系統規格的基礎上，進而驗證測試系統的容錯性、穩定性、可用性…等等的測試過程。

具體的系統測試過程與軟體組織的具體過程定義相關。通常系統測試過程可分為：

1.系統測試計劃階段：完成系統測試計劃

2. 系統測試設計階段：完成系統測試方案

3. 系統測試實現階段：完成系統測試用例、腳本和規程.

4. 系統測試執行階段：執行系統測試用例，發現問題並回歸測試，提交系統測試日報和系統測試報告。

IEEE 1028 - Software Reviews and Audits. 2008.
IEEE 829 - Standard for Software and System Test Documentation, 2008
IEEE 730 - Software Quality Assurance Processes, 2014.
IEEE 1012 - System, Software, and Hardware Verification and Validation, 2016
ISO/IEC/IEEE 15939 - Systems and software engineering - Measurement process, 2017.

 

 

Beautiful Testing Leading Professionals Reveal How They Improve Software

Beautiful Testing Leading Professionals Reveal How They Improve Software

測試之美 領略頂尖專家改善軟體的測試法則

Successful software depends as much on scrupulous testing as it does on solid architecture or elegant code. But testing is not a routine process, it's a constant exploration of methods and an evolution of good ideas. 

Beautiful Testing offers 23 essays from 27 leading testers and developers that illustrate the qualities and techniques that make testing an art. Through personal anecdotes, you'll learn how each of these professionals developed beautiful ways of testing a wide range of products -- valuable knowledge that you can apply to your own projects.

Test Plan

撰寫測試計劃書的目的有三個；一個就是要讓軟體測試進行的更加順利，另一個就是可以讓參與人員之間的溝通更加的暢通，最後一個就是可以讓軟體測試採取系統化的方式來進行，同時也易於管理。 計劃書可以分成三種、單一文件測試計劃書 （STP：Single Test Plan）、主要方針測試計劃書（MTP：Master Test Plan）、詳細運作測試計劃書（DTP：Detail Test Plan）。

STP
所謂的STP（單一文件測試計劃書），就是將測試計劃所有的議題，撰寫在同一份的文件上。 這樣的做法是將所有軟體測試的常數與變數集中，以方便管理掌控。 對於管理人員而言，能夠集中管理當然是最好不過了，但是相對的這份測試計劃書的內容，就會顯的更加的複雜。

MTP and DTP
MTP（主要方針測試計劃書）與DTP（詳細運作測試計劃書），這兩個種類通常是伴隨在一起使用。 基本上MTP的內容是將測試分成為不同的進行階段，對於每個階段規劃出概略的測試方針，至於各階段的詳細測試計劃則是撰寫在DTP內，因此通常一個MTP會伴隨好幾個DTP一起出現，這種方式相當適合大型的軟體開發。

IEEE 829
IEEE 829的標準建議，測試計劃書包括了16個大綱要項：

項目綱要	解釋
Test plan identifier	測試計劃書的目標
Introduction	基本介紹
Test items	測試的項目
Features to be tested	需要測試的產品功能
Features not to be tested	不需要測試的產品功能
Approach	採用的測試模式
Item pass/fail criteria	項目通過測試的標準
Suspension criteria and resumption requirements	測試中斷與開始的規定
Test deliverables	測試完成所要提交的項目
Testing tasks	測試工作項目
Environmental needs	測試環境的設定
Responsibilities	人員的工作分配
Staffing and training needs	人員的能力與所需要的訓練
Schedule	測試的時程
Risks and contingencies	潛在問題與風險
Approvals	文件的認可

  

Top 10 Negative Test Cases

Top 10 Negative Test Cases

Negative test cases are designed to test the software in ways it was not intended to be used, and should be a part of your testing effort. Below are the top 10 negative test cases you should consider when designing your test effort:

1. Embedded Single Quote - Most SQL based database systems have issues when users store information that contain a single quote (e.g. John's car). For each screen that accepts alphanumeric data entry, try entering text that contains one or more single quotes.
2. Required Data Entry - Your functional specification should clearly indicate fields that require data entry on screens. Test each field on the screen that has been indicated as being required to ensure it forces you to enter data in the field.
3. Field Type Test - Your functional specification should clearly indicate fields that require specific data entry requirements (date fields, numeric fields, phone numbers, zip codes, etc). Test each field on the screen that has been indicated as having special types to ensure it forces you to enter data in the correct format based on the field type (numeric fields should not allow alphabetic or special characters, date fields should require a valid date, etc).
4. Field Size Test - Your functional specification should clearly indicate the number of characters you can enter into a field (for example, the first name must be 50 or less characters). Write test cases to ensure that you can only enter the specified number of characters. Preventing the user from entering more characters than is allowed is more elegant than giving an error message after they have already entered too many characters.
5. Numeric Bounds Test - For numeric fields, it is important to test for lower and upper bounds. For example, if you are calculating interest charged to an account, you would never have a negative interest amount applied to an account that earns interest, therefore, you should try testing it with a negative number. Likewise, if your functional specification requires that a field be in a specific range (e.g. from 10 to 50), you should try entering 9 or 51, it should fail with a graceful message.
6. Numeric Limits Test - Most database systems and programming languages allow numeric items to be identified as integers or long integers. Normally, an integer has a range of -32,767 to 32,767 and long integers can range from
   -2,147,483,648 to 2,147,483,647. For numeric data entry that do not have specified bounds limits, work with these limits to ensure that it does not get an numeric overflow error.
7. Date Bounds Test - For date fields, it is important to test for lower and upper bounds. For example, if you are checking a birth date field, it is probably a good bet that the person's birth date is no older than 150 years ago. Likewise, their birth date should not be a date in the future.
8. Date Validity - For date fields, it is important to ensure that invalid dates are not allowed (04/31/2007 is an invalid date). Your test cases should also check for leap years (every 4th and 400th year is a leap year).
9. Web Session Testing - Many web applications rely on the browser session to keep track of the person logged in, settings for the application, etc. Most screens in a web application are not designed to be launched without first logging in. Create test cases to launch web pages within the application without first logging in. The web application should ensure it has a valid logged in session before rendering pages within the application.
10. Performance Changes - As you release new versions of your product, you should have a set of performance tests that you run that identify the speed of your screens (screens that list information, screens that add/update/delete data, etc). Your test suite should include test cases that compare the prior release performance statistics to the current release. This can aid in identifying potential performance problems that will be manifested with code changes to the current release.

Application Testing Life Cycle

Application Testing Life Cycle

This Life Cycle is used for standard applications that are built from various custom technologies and follow the normal or standard testing approach. The Application or custom-built Lifecycle and its phases is depicted below:

Test Requirements
* Requirement Specification documents
* Functional Specification documents
* Design Specification documents (use cases, etc)
* Use case Documents
* Test Trace-ability Matrix for identifying Test Coverage


Test Planning
* Test Scope, Test Environment
* Different Test phase and Test Methodologies
* Manual and Automation Testing
* Defect Mgmt, Configuration Mgmt, Risk Mgmt. Etc
* Evaluation & identification – Test, Defect tracking tools

Test Environment Setup
* Test Bed installation and configuration
* Network connectivity's
* All the Software/ tools Installation and configuration
* Coordination with Vendors and others

Test Design
* Test Traceability Matrix and Test coverage
* Test Scenarios Identification & Test Case preparation
* Test data and Test scripts preparation
* Test case reviews and Approval
* Base lining under Configuration Management

Test Automation
* Automation requirement identification
* Tool Evaluation and Identification.
* Designing or identifying Framework and scripting
* Script Integration, Review and Approval
* Base lining under Configuration Management

Test Execution and Defect Tracking
* Executing Test cases
* Testing Test Scripts
* Capture, review and analyze Test Results
* Raised the defects and tracking for its closure

Test Reports and Acceptance
* Test summary reports
* Test Metrics and process Improvements made
* Build release
* Receiving acceptance