diff --git a/README.en.md b/README.en.md deleted file mode 100644 index 3040453..0000000 --- a/README.en.md +++ /dev/null @@ -1,508 +0,0 @@ -# XTS - -- [Introduction](#section465982318513) -- [System Types](#section125090457443) -- [Directory Structure](#section161941989596) -- [Constraints](#section119744591305) -- [Usage Guidelines](#section137768191623) -- [Test Case Development Guidelines](#section3695134065513) - - [C-based Test Case Development and Compilation \(for the Mini System\)](#section198193336544) - - [C-based Test Case Execution \(for the Mini System\)](#section13820233175418) - - [C++-based Test Case Development and Compilation \(for Standard and Small Systems\)](#section3822123311540) - - [C++-based Test Case Execution \(for Standard and Small Systems\)](#section128222336544) - - -## Introduction - -The X test suite \(XTS\) subsystem contains a set of OpenHarmony certification test suites, including the currently supported distributed compatibility test suite \(DCTS\). - -This subsystem contains the DCTS and **tools** software package. - -- The **dcts** directory stores the source code and configuration files of DCTS test cases. The DCTS helps device vendors detect the distributed scenario incompatibility as early as possible and ensures that the software is compatible to OpenHarmony during the entire development process. -- The **tools** software package stores the test case development framework related to **dcts**. - -## System Types - -OpenHarmony supports the following system types: - -- Mini system - - A mini system runs on the devices whose memory is greater than or equal to 128 KiB and that are equipped with MCU processors such as ARM Cortex-M and 32-bit RISC-V. This system provides multiple lightweight network protocols and graphics frameworks, and a wide range of read/write components for the IoT bus. Typical products include connection modules, sensors, and wearables for smart home. - -- Small system - - A small system runs on the devices whose memory is greater than or equal to 1 MiB and that are equipped with application processors such as ARM Cortex-A. This system provides higher security capabilities, standard graphics frameworks, and video encoding and decoding capabilities. Typical products include smart home IP cameras, electronic cat eyes, and routers, and event data recorders \(EDRs\) for smart travel. - -- Standard system - - A standard system runs on the devices whose memory is greater than or equal to 128 MiB and that are equipped with application processors such as ARM Cortex-A. This system provides a complete application framework supporting the enhanced interaction, 3D GPU, hardware composer, diverse components, and rich animations. This system applies to high-end refrigerator displays. - - -## Directory Structure - -``` -/test/xts -├── dcts # Test code -│ └── subsystem # Source code of subsystem test cases for the standard system -│ └── subsystem_lite # Source code of subsystems test cases for mini and small systems -│ └── common # Source code of Test cases rely on shared memory for mini and small systems -│ └── BUILD.gn # Build configuration of test cases for the standard system -│ └── build_lite -│ └── BUILD.gn # Build configuration of test cases for mini and small systems -└── tools # Test tool code -``` - -## Constraints - -Test cases for the mini system must be developed based on C, and those for the small system must be developed based on C++. - -## Usage Guidelines - -**Table 1** Test case levels - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

Level

-

Definition

-

Scope

-

Level0

-

Smoke

-

Verifies basic functionalities of key features and basic DFX attributes with the most common input. The pass result indicates that the features are runnable.

-

Level1

-

Basic

-

Verifies basic functionalities of key features and basic DFX attributes with common input. The pass result indicates that the features are testable.

-

Level2

-

Major

-

Verifies basic functionalities of key features and basic DFX attributes with common input and errors. The pass result indicates that the features are functional and ready for beta testing.

-

Level3

-

Regular

-

Verifies functionalities of all key features, and all DFX attributes with common and uncommon input combinations or normal and abnormal preset conditions.

-

Level4

-

Rare

-

Verifies functionalities of key features under extremely abnormal presets and uncommon input combinations.

-
- -**Table 2** Test case granularities - - - - - - - - - - - - - - - - - - - - -

Test Scale

-

Test Objects

-

Test Environment

-

LargeTest

-

Service functionalities, all-scenario features, and mechanical power environment (MPE) and scenario-level DFX

-

Devices close to real devices

-

MediumTest

-

Modules, subsystem functionalities after module integration, and DFX

-

Single device that is actually used. You can perform message simulation, but do not mock functions.

-

SmallTest

-

Modules, classes, and functions

-

Local PC. Use a large number of mocks to replace dependencies with other modules.

-
- -**Table 3** Test types - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

Type

-

Definition

-

Function

-

Tests the correctness of both service and platform functionalities provided by the tested object for end users or developers.

-

Performance

-

Tests the processing capability of the tested object under specific preset conditions and load models. The processing capability is measured by the service volume that can be processed in a unit time, for example, call per second, frame per second, or event processing volume per second.

-

Power

-

Tests the power consumption of the tested object in a certain period of time under specific preset conditions and load models.

-

Reliability

-

Tests the service performance of the tested object under common and uncommon input conditions, or specified service volume pressure and long-term continuous running pressure. The test covers stability, pressure handling, fault injection, and Monkey test times.

-

Security

-
  • Tests the capability of defending against security threats, including but not limited to unauthorized access, use, disclosure, damage, modification, and destruction, to ensure information confidentiality, integrity, and availability.
  • Tests the privacy protection capability to ensure that the collection, use, retention, disclosure, and disposal of users' private data comply with laws and regulations.
  • Tests the compliance with various security specifications, such as security design, security requirements, and security certification of the Ministry of Industry and Information Technology (MIIT).
-

Global

-

Tests the internationalized data and localization capabilities of the tested object, including multi-language display, various input/output habits, time formats, and regional features, such as currency, time, and culture taboos.

-

Compatibility

-
  • Tests backward compatibility of an application with its own data, the forward and backward compatibility with the system, and the compatibility with different user data, such as audio file content of the player and smart SMS messages.
  • Tests system backward compatibility with its own data and the compatibility of common applications in the ecosystem.
  • Tests software compatibility with related hardware.
-

User

-

Tests user experience of the object in real user scenarios. All conclusions and comments should come from the users, which are all subjective evaluation in this case.

-

Standard

-

Tests the compliance with industry and company-specific standards, protocols, and specifications. The standards here do not include any security standards that should be classified into the security test.

-

Safety

-

Tests the safety property of the tested object to avoid possible hazards to personal safety, health, and the object itself.

-

Resilience

-

Tests the resilience property of the tested object to ensure that it can withstand and maintain the defined running status (including downgrading) when being attacked, and recover from and adapt defense to the attacks to approach mission assurance.

-
- -## Test Case Development Guidelines - -You should select the appropriate programming language and your target test framework to develop test cases. - -**Table 4** Test frameworks and test case languages for different systems - - - - - - - - - - - - - - - - - - - - -

System

-

Test Framework

-

Language

-

Mini

-

HCTest

-

C

-

Small

-

HCPPTest

-

C++

-

Standard

-

HJSUnit and HCPPTest

-

JavaScript and C++

-
- -### C-based Test Case Development and Compilation \(for the Mini System\) - -**Developing test cases for the mini system** - -The HCTest framework is used to support test cases developed with the C language. HCTest is enhanced and adapted based on the open-source test framework Unity. - -1. Access the **test/xts/dcts** repository where the test cases will be stored. - - ``` - ├── dcts - │ └──subsystem_lite - │ │ └── module_hal - │ │ │ └── BUILD.gn - │ │ │ └── src - │ └──build_lite - │ │ └── BUILD.gn - ``` - -2. Write the test case in the **src** directory. - - 1 Import the test framework header file. - - ``` - #include "hctest.h" - ``` - - 2. Use the **LITE\_TEST\_SUIT** macro to define names of the subsystem, module, and test suite. - - ``` - /** - * @brief Registers a test suite named IntTestSuite. - * @param test Subsystem name - * @param example Module name - * @param IntTestSuite Test suite name - */ - LITE_TEST_SUIT(test, example, IntTestSuite); - ``` - - 3. Define Setup and TearDown. - - Format: Test suite name+Setup, Test suite name+TearDown. - - The Setup and TearDown functions must exist, but function bodies can be empty. - - 4. Use the **LITE\_TEST\_CASE** macro to write the test case. - - Three parameters are involved: test suite name, test case name, and test case properties \(including type, granularity, and level\). - - ``` - LITE_TEST_CASE(IntTestSuite, TestCase001, Function | MediumTest | Level1) - { - // Do something - }; - ``` - - 5. Use the **RUN\_TEST\_SUITE** macro to register the test suite. - - ``` - RUN_TEST_SUITE(IntTestSuite); - ``` - -3. Create the configuration file \(**BUILD.gn**\) of the test module. - - Create a **BUILD.gn** \(example\) build file in each test module directory. Specify the name of the built static library and its dependent header file and library in the build file. The format is as follows: - - ``` - import("//test/xts/tools/lite/build/suite_lite.gni") - hctest_suite("DctsDemoTest") { - suite_name = "dcts" - sources = [ - "src/test_demo.c", - ] - include_dirs = [ ] - cflags = [ "-Wno-error" ] - } - ``` - -4. Add build options to the **BUILD.gn** file in the **dcts** directory. - - You need to add the test module to the **test/xts/dcts/build\_lite/BUILD.gn** script in the **dcts** directory. - - ``` - lite_component("dcts") { - ... - if(board_name == "liteos_m") { - features += [ - ... - "//xts/dcts/subsystem_lite/module_hal:DctsDemoTest" - ] - } - } - ``` - -5. Run build commands. - - Test suites are built along with version build. The DCTS is built together with the debug version. - - >![](figures/icon-note.gif) **NOTE:** - >The DCTS build middleware is a static library, which will be linked to the image. - - -### C-based Test Case Execution \(for the Mini System\) - -**Executing test cases for the mini system** - -Burn the image into the development board. - -**Executing the test** - -1. Use a serial port tool to log in to the development board and save information about the serial port. -2. Restart the device and view serial port logs. - -**Analyzing the test result** - -View the serial port logs, whose format is as follows: - -The log for each test suite starts with **Start to run test suite:** and ends with **xx Tests xx Failures xx Ignored**. - -### C++-based Test Case Development and Compilation \(for Standard and Small Systems\) - -**Developing test cases for small-system devices** \(For examples of the standard system, go to the **global/i18n\_standard directory**.\) - -The HCPPTest framework is enhanced and adapted based on the open-source framework Googletest. - -1. Access the **test/xts/dcts** repository where the test cases will be stored. - - ``` - ├── dcts - │ └──subsystem_lite - │ │ └── module_posix - │ │ │ └── BUILD.gn - │ │ │ └── src - │ └──build_lite - │ │ └── BUILD.gn - ``` - -2. Write the test case in the **src** directory. - - 1. Import the test framework header file. - - The following statement includes **gtest.h**. - - ``` - #include "gtest/gtest.h" - ``` - - 2. Define Setup and TearDown. - - ``` - using namespace std; - using namespace testing::ext; - class TestSuite: public testing::Test { - protected: - // Preset action of the test suite, which is executed before the first test case - static void SetUpTestCase(void){ - } - // Test suite cleanup action, which is executed after the last test case - static void TearDownTestCase(void){ - } - // Preset action of the test case - virtual void SetUp() - { - } - // Cleanup action of the test case - virtual void TearDown() - { - } - }; - ``` - - 3. Use the **HWTEST** or **HWTEST\_F** macro to write the test case. - - **HWTEST**: definition of common test cases, including the test suite name, test case name, and case annotation. - - **HWTEST\_F**: definition of SetUp and TearDown test cases, including the test suite name, test case name, and case annotation. - - Three parameters are involved: test suite name, test case name, and test case properties \(including type, granularity, and level\). - - ``` - HWTEST_F(TestSuite, TestCase_0001, Function | MediumTest | Level1) { - // Do something - } - ``` - -3. Create a configuration file \(**BUILD.gn**\) of the test module. - - Create a **BUILD.gn** build file in each test module directory. Specify the name of the built static library and its dependent header file and library in the build file. Each test module is independently built into a **.bin** executable file, which can be directly pushed to the development board for testing. - - Example: - - ``` - import("//test/xts/tools/lite/build/suite_lite.gni") - hcpptest_suite("DctsDemoTest") { - suite_name = "dcts" - sources = [ - "src/TestDemo.cpp" - ] - - include_dirs = [ - "src", - ... - ] - deps = [ - ... - ] - cflags = [ "-Wno-error" ] - } - ``` - -4. Add build options to the **BUILD.gn** file in the **dcts** directory. - - Add the test module to the **test/xts/dcts/build\_lite/BUILD.gn** script in the **dcts** directory. - - ``` - lite_component("dcts") { - ... - else if(board_name == "liteos_a") { - features += [ - ... - "//xts/dcts/subsystem_lite/module_posix:DctsDemoTest" - ] - } - } - ``` - -5. Run build commands. - - Test suites are built along with the version build. The DCTS is built together with the debug version. - - >![](figures/icon-note.gif) **NOTE:** - >The DCTS for the small system is independently built to an executable file \(.bin\) and archived in the **suites\\dcts** directory of the build result. - - -### C++-based Test Case Execution \(for Standard and Small Systems\) - -**Executing test cases for the small system** - -Currently, test cases are shared by the NFS and mounted to the development board for execution. - -**Setting up the environment** - -1. Use a network cable or wireless network to connect the development board to your PC. -2. Configure the IP address, subnet mask, and gateway for the development board. Ensure that the development board and the PC are in the same network segment. -3. Install and register the NFS server on the PC and start the NFS service. -4. Run the **mount** command for the development board to ensure that the development board can access NFS shared files on the PC. - - Format: **mount** _NFS server IP address_**:/**_NFS shared directory_ **/**_development board directory_ **nfs** - - Example: - - ``` - mount 192.168.1.10:/nfs /nfs nfs - ``` - - -**Executing test cases** - -Execute **DctsDemoTest.bin** to trigger test case execution, and analyze serial port logs generated after the execution is complete. diff --git a/README.md b/README.md index a7d0e5b..3040453 100644 --- a/README.md +++ b/README.md @@ -1,261 +1,260 @@ -# XTS子系统 +# XTS -- [简介](#section465982318513) -- [系统类型](#section125090457443) -- [目录](#section161941989596) -- [约束](#section119744591305) -- [使用说明](#section137768191623) -- [用例开发指导](#section3695134065513) - - [C语言用例开发编译指导（适用于轻量系统产品用例开发）](#section198193336544) - - [C语言用例执行指导（适用于轻量系统产品用例开发）](#section13820233175418) - - [C++语言用例开发编译指导（适用于小型系统、标准系统用例开发）](#section3822123311540) - - [C++语言用例执行指导（适用于小型系统、标准系统用例开发）](#section128222336544) - -- [相关仓](#section1371113476307) - -## 简介 - -XTS子系统是OpenHarmony生态认证测试套件的集合，当前包括dcts（distributed compatibility test suite）分布式兼容性测试套件。 - -XTS子系统当前包括dcts与tools软件包： - -- dcts，存放dcts相关测试用例源码与配置文件，其目的是帮助终端设备厂商尽早发现在分布式场景下与OpenHarmony的不兼容性，确保软件在整个开发过程中满足OpenHarmony的兼容性要求。 -- tools，存放dcts相关测试用例开发框架。 - -## 系统类型 - -OpenHarmony支持如下几种系统类型： - -- 轻量系统（mini system） - - 面向MCU类处理器例如Arm Cortex-M、RISC-V 32位的设备，硬件资源极其有限，支持的设备最小内存为128KiB，可以提供多种轻量级网络协议，轻量级的图形框架，以及丰富的IOT总线读写部件等。可支撑的产品如智能家居领域的连接类模组、传感器设备、穿戴类设备等。 - -- 小型系统（small system） - - 面向应用处理器例如Arm Cortex-A的设备，支持的设备最小内存为1MiB，可以提供更高的安全能力、标准的图形框架、视频编解码的多媒体能力。可支撑的产品如智能家居领域的IP Camera、电子猫眼、路由器以及智慧出行领域的行车记录仪等。 - -- 标准系统（standard system） - - 面向应用处理器例如Arm Cortex-A的设备，支持的设备最小内存为128MiB，可以提供增强的交互能力、3D GPU以及硬件合成能力、更多控件以及动画效果更丰富的图形能力、完整的应用框架。可支撑的产品如高端的冰箱显示屏。 +- [Introduction](#section465982318513) +- [System Types](#section125090457443) +- [Directory Structure](#section161941989596) +- [Constraints](#section119744591305) +- [Usage Guidelines](#section137768191623) +- [Test Case Development Guidelines](#section3695134065513) + - [C-based Test Case Development and Compilation \(for the Mini System\)](#section198193336544) + - [C-based Test Case Execution \(for the Mini System\)](#section13820233175418) + - [C++-based Test Case Development and Compilation \(for Standard and Small Systems\)](#section3822123311540) + - [C++-based Test Case Execution \(for Standard and Small Systems\)](#section128222336544) -## 目录 +## Introduction + +The X test suite \(XTS\) subsystem contains a set of OpenHarmony certification test suites, including the currently supported distributed compatibility test suite \(DCTS\). + +This subsystem contains the DCTS and **tools** software package. + +- The **dcts** directory stores the source code and configuration files of DCTS test cases. The DCTS helps device vendors detect the distributed scenario incompatibility as early as possible and ensures that the software is compatible to OpenHarmony during the entire development process. +- The **tools** software package stores the test case development framework related to **dcts**. + +## System Types + +OpenHarmony supports the following system types: + +- Mini system + + A mini system runs on the devices whose memory is greater than or equal to 128 KiB and that are equipped with MCU processors such as ARM Cortex-M and 32-bit RISC-V. This system provides multiple lightweight network protocols and graphics frameworks, and a wide range of read/write components for the IoT bus. Typical products include connection modules, sensors, and wearables for smart home. + +- Small system + + A small system runs on the devices whose memory is greater than or equal to 1 MiB and that are equipped with application processors such as ARM Cortex-A. This system provides higher security capabilities, standard graphics frameworks, and video encoding and decoding capabilities. Typical products include smart home IP cameras, electronic cat eyes, and routers, and event data recorders \(EDRs\) for smart travel. + +- Standard system + + A standard system runs on the devices whose memory is greater than or equal to 128 MiB and that are equipped with application processors such as ARM Cortex-A. This system provides a complete application framework supporting the enhanced interaction, 3D GPU, hardware composer, diverse components, and rich animations. This system applies to high-end refrigerator displays. + + +## Directory Structure ``` /test/xts -├── dcts # 测试代码存放目录 -│ └── subsystem # 标准系统子系统测试用例源码存放目录 -│ └── subsystem_lite # 轻量系统、小型系统子系统测试用例源码存放目录 -│ └── common # 测试用例依赖共享内存源码存放目录 -│ └── BUILD.gn # 标准系统测试用例编译配置 -│ └── build_lite # 轻量系统、小型系统测试用例编译配置存放目录 -│ └── BUILD.gn # 轻量系统、小型系统测试用例编译配置 -└── tools # 测试工具代码存放目录 +├── dcts # Test code +│ └── subsystem # Source code of subsystem test cases for the standard system +│ └── subsystem_lite # Source code of subsystems test cases for mini and small systems +│ └── common # Source code of Test cases rely on shared memory for mini and small systems +│ └── BUILD.gn # Build configuration of test cases for the standard system +│ └── build_lite +│ └── BUILD.gn # Build configuration of test cases for mini and small systems +└── tools # Test tool code ``` -## 约束 +## Constraints -轻量系统用例开发语言是C，小型系统用例开发语言是C++。 +Test cases for the mini system must be developed based on C, and those for the small system must be developed based on C++. -## 使用说明 +## Usage Guidelines -**表 1** 用例级别说明 +**Table 1** Test case levels -

级别名称

+ - - - - - - - - - - - -

Level

基本定义

+

Definition

测试范围

+

Scope

Level0

冒烟

+

Smoke

验证关键功能点基本功能/最基本DFX属性在最常见输入下的表现，通过表示功能基本可运行。

+

Verifies basic functionalities of key features and basic DFX attributes with the most common input. The pass result indicates that the features are runnable.

Level1

基本

+

Basic

验证各功能点基本功能/基本DFX属性在常见输入下的表现，通过表示功能基本可测试。

+

Verifies basic functionalities of key features and basic DFX attributes with common input. The pass result indicates that the features are testable.

Level2

重要

+

Major

验证各功能点的基本功能/基本DFX属性在常规输入/常见异常情况下的表现，通过表示功能基本正常可用，可开展Beta。

+

Verifies basic functionalities of key features and basic DFX attributes with common input and errors. The pass result indicates that the features are functional and ready for beta testing.

Level3

一般

+

Regular

验证各功能点的全部功能/全部DFX属性在各种常规/非常规输入组合下，或各种正常/异常预置条件组合下的表现。

+

Verifies functionalities of all key features, and all DFX attributes with common and uncommon input combinations or normal and abnormal preset conditions.

Level4

生僻

+

Rare

验证关键功能点在极端异常预置条件下、用户难以触及的异常输入组合下的表现。

+

Verifies functionalities of key features under extremely abnormal presets and uncommon input combinations.
-**表 2** 用例粒度说明 +**Table 2** Test case granularities -

用例规模

+ - - - - - - - -

Test Scale

被测试对象

+

Test Objects

测试环境

+

Test Environment

LargeTest

业务功能/全场景特性/整机及场景级DFX

+

Service functionalities, all-scenario features, and mechanical power environment (MPE) and scenario-level DFX

尽量使用贴近真实的环境设备

+

Devices close to real devices

MediumTest

模块/子系统集成至设备后的功能/DFX

+

Modules, subsystem functionalities after module integration, and DFX

使用真实的单设备进行验证，可进行消息模拟，尽量不对函数进行MOCK

+

Single device that is actually used. You can perform message simulation, but do not mock functions.

SmallTest

模块/类/函数

+

Modules, classes, and functions

在开发者个人环境进行测试，尽量不依赖其他模块，存在大量的MOCK

+

Local PC. Use a large number of mocks to replace dependencies with other modules.
-**表 3** 测试类型说明 +**Table 3** Test types -

测试类型名称

+ - - - - - - - - - - - -

Type

测试类型定义

+

Definition

Function

验证被测对象提供给用户的业务功能实现正确性的测试项，这里的“用户”可以是终端用户或开发者，功能包括业务功能及平台功能

+

Tests the correctness of both service and platform functionalities provided by the tested object for end users or developers.

Performance

验证被测对象在特定预置条件/负载模型下的处理能力的测试项，“处理能力”一般以单位时间内可处理的业务量来衡量，如呼叫/秒，帧率/秒，事件处理量/秒等

+

Tests the processing capability of the tested object under specific preset conditions and load models. The processing capability is measured by the service volume that can be processed in a unit time, for example, call per second, frame per second, or event processing volume per second.

Power

验证被测对象在特定预置条件/负载模型下在一定时间内能源消耗量的测试项

+

Tests the power consumption of the tested object in a certain period of time under specific preset conditions and load models.

Reliability

验证被测对象在正常/异常输入情况下，或业务量压力和长时间连续运行压力情况下业务表现的测试项，含稳定性、压力、故障注入、Monkey测试项

+

Tests the service performance of the tested object under common and uncommon input conditions, or specified service volume pressure and long-term continuous running pressure. The test covers stability, pressure handling, fault injection, and Monkey test times.

Security

验证系统对恶意威胁的防护能力，威胁包括但不限于未授权访问、使用、泄露、破坏、修改、毁灭，以保障信息的机密性、完整性和可用性; 验证系统对用户隐私的保护能力，保障用户的隐私数据被收集、使用、保有、披露和处置符合法律规范，保障用户的隐私权; 验证对各类安全规范的遵从情况，如安全设计规范、安全红线、工信部安全认证规范等，保障安全相关法律法规的合规。

+
  • Tests the capability of defending against security threats, including but not limited to unauthorized access, use, disclosure, damage, modification, and destruction, to ensure information confidentiality, integrity, and availability.
  • Tests the privacy protection capability to ensure that the collection, use, retention, disclosure, and disposal of users' private data comply with laws and regulations.
  • Tests the compliance with various security specifications, such as security design, security requirements, and security certification of the Ministry of Industry and Information Technology (MIIT).

Global

验证被测对象在是否具有国际化数据支持和本地化能力的测试项，包括语言显示、输入/输出习惯、时间显示、区域特性如货币时间禁忌等等

+

Tests the internationalized data and localization capabilities of the tested object, including multi-language display, various input/output habits, time formats, and regional features, such as currency, time, and culture taboos.

Compatibility

当被测对象为应用时，包括被测对象对于自身数据的后向兼容性、对于系统的前后向兼容性、对于不同用户数据(如播放器之音频文件格式/智能短信之用户短信内容)的兼容性测试项； 当被测对象为系统时，包括被测系统对于系统自身数据的后向兼容性、以及对于生态中常用应用的兼容性测试项；当被测对象为软件时，包括被测系统对于相关的硬件的兼容性；

+
  • Tests backward compatibility of an application with its own data, the forward and backward compatibility with the system, and the compatibility with different user data, such as audio file content of the player and smart SMS messages.
  • Tests system backward compatibility with its own data and the compatibility of common applications in the ecosystem.
  • Tests software compatibility with related hardware.

User

验证被测对象在真实用户场景下的用户体验感受的测试项，注意此种情况下没有客观的“正确”与“失败”，所有的结论及评价都应该来自于用户

+

Tests user experience of the object in real user scenarios. All conclusions and comments should come from the users, which are all subjective evaluation in this case.

Standard

验证被测对象对于行业及公司内标准/协议/规范的遵从情况的测试项，注意此处的“标准”不包含任何安全标准，针对安全标准的测试项划归为“安全测试”类型

+

Tests the compliance with industry and company-specific standards, protocols, and specifications. The standards here do not include any security standards that should be classified into the security test.

Safety

验证被测对象的Safety属性，避免产品可能对人身安全、健康以及产品本身带来的危害。

+

Tests the safety property of the tested object to avoid possible hazards to personal safety, health, and the object itself.

Resilience

验证被测对象的韧性属性，确保系统受攻击时承受并保持在有定义的运行状态（包括降级）、恢复并适应攻击以保障Mission达成。

+

Tests the resilience property of the tested object to ensure that it can withstand and maintain the defined running status (including downgrading) when being attacked, and recover from and adapt defense to the attacks to approach mission assurance.
-## 用例开发指导 +## Test Case Development Guidelines -根据测试系统选择测试框架和对应测试用例语言。 +You should select the appropriate programming language and your target test framework to develop test cases. -**表 4** 系统和测试框架、开发语言对应关系 +**Table 4** Test frameworks and test case languages for different systems -

系统

+ - - - - - - - - - - -

System

测试框架

+

Test Framework

语言

+

Language

轻量系统

+

Mini

hctest

+

HCTest

c

+

C

小型系统

+

Small

hcpptest

+

HCPPTest

c++

+

C++

标准系统

+

Standard

HJSUnit、hcpptest

+

HJSUnit and HCPPTest

js、c++

+

JavaScript and C++
-### C语言用例开发编译指导（适用于轻量系统产品用例开发） +### C-based Test Case Development and Compilation \(for the Mini System\) -**示例：轻量系统测试用例开发** +**Developing test cases for the mini system** -当前使用的测试框架是hctest，hctest测试框架支持使用C语言编写测试用例，是在开源测试框架unity的基础上进行增强和适配。 +The HCTest framework is used to support test cases developed with the C language. HCTest is enhanced and adapted based on the open-source test framework Unity. -1. 用例目录规范：测试用例存储到test/xts/dcts仓中 +1. Access the **test/xts/dcts** repository where the test cases will be stored. ``` ├── dcts @@ -267,52 +266,52 @@ OpenHarmony支持如下几种系统类型： │ │ └── BUILD.gn ``` -2. src目录下用例编写样例。 +2. Write the test case in the **src** directory. - 1.引用测试框架 + 1 Import the test framework header file. ``` #include "hctest.h" ``` - 2. 使用宏定义LITE\_TEST\_SUIT定义子系统、模块、测试套件名称 + 2. Use the **LITE\_TEST\_SUIT** macro to define names of the subsystem, module, and test suite. ``` /** - * @brief register a test suit named "IntTestSuite" - * @param test subsystem name - * @param example module name - * @param IntTestSuite test suit name + * @brief Registers a test suite named IntTestSuite. + * @param test Subsystem name + * @param example Module name + * @param IntTestSuite Test suite name */ LITE_TEST_SUIT(test, example, IntTestSuite); ``` - 3. 定义Setup与TearDown + 3. Define Setup and TearDown. - 命名方式：测试套件名称+Setup，测试套件名称+TearDown。 + Format: Test suite name+Setup, Test suite name+TearDown. - Setup与TearDown必须存在，可以为空函数。 + The Setup and TearDown functions must exist, but function bodies can be empty. - 4. 使用宏定义LITE\_TEST\_CASE写测试用例 + 4. Use the **LITE\_TEST\_CASE** macro to write the test case. - 包括三个参数：测试套件名称，测试用例名称，用例属性（测试类型、用例粒度、用例级别）。 + Three parameters are involved: test suite name, test case name, and test case properties \(including type, granularity, and level\). ``` LITE_TEST_CASE(IntTestSuite, TestCase001, Function | MediumTest | Level1) { - //do something + // Do something }; ``` - 5. 使用宏定义 RUN\_TEST\_SUITE注册测试套件 + 5. Use the **RUN\_TEST\_SUITE** macro to register the test suite. ``` RUN_TEST_SUITE(IntTestSuite); ``` -3. 测试模块的配置文件（BUILD.gn）样例： +3. Create the configuration file \(**BUILD.gn**\) of the test module. - 在每个测试模块目录下新建BUILD.gn编译文件，用于指定编译后静态库的名称、依赖的头文件、依赖的库等；具体写法如下： + Create a **BUILD.gn** \(example\) build file in each test module directory. Specify the name of the built static library and its dependent header file and library in the build file. The format is as follows: ``` import("//test/xts/tools/lite/build/suite_lite.gni") @@ -326,9 +325,9 @@ OpenHarmony支持如下几种系统类型： } ``` -4. dcts下BUILD.gn增加编译选项。 +4. Add build options to the **BUILD.gn** file in the **dcts** directory. - 需要将测试模块加入到dcts目录下的编译脚本中，编译脚本路径：test/xts/dcts/build\_lite/BUILD.gn。 + You need to add the test module to the **test/xts/dcts/build\_lite/BUILD.gn** script in the **dcts** directory. ``` lite_component("dcts") { @@ -342,38 +341,38 @@ OpenHarmony支持如下几种系统类型： } ``` -5. 测试套件编译命令。 +5. Run build commands. - 随版本编译，debug版本编译时会同步编译dcts测试套件 + Test suites are built along with version build. The DCTS is built together with the debug version. - >![](figures/icon-note.gif) **说明：** - >dcts测试套件编译中间件为静态库，最终链接到版本镜像中 。 + >![](figures/icon-note.gif) **NOTE:** + >The DCTS build middleware is a static library, which will be linked to the image. -### C语言用例执行指导（适用于轻量系统产品用例开发） +### C-based Test Case Execution \(for the Mini System\) -**示例：轻量系统测试用例执行** +**Executing test cases for the mini system** -将版本镜像烧录进开发板。 +Burn the image into the development board. -**测试步骤** +**Executing the test** -1. 使用串口工具登录开发板，并保存串口打印信息。 -2. 重启设备，查看串口日志。 +1. Use a serial port tool to log in to the development board and save information about the serial port. +2. Restart the device and view serial port logs. -**测试结果分析指导** +**Analyzing the test result** -基于串口打印日志进行分析； +View the serial port logs, whose format is as follows: -每个测试套件执行以Start to run test suite开始，以xx Tests xx Failures xx Ignored结束。 +The log for each test suite starts with **Start to run test suite:** and ends with **xx Tests xx Failures xx Ignored**. -### C++语言用例开发编译指导（适用于小型系统、标准系统用例开发） +### C++-based Test Case Development and Compilation \(for Standard and Small Systems\) -**示例：小型系统测试用例开发**（标准系统参考具体样例目录：global/i18n\_standard） +**Developing test cases for small-system devices** \(For examples of the standard system, go to the **global/i18n\_standard directory**.\) -当前使用的测试框架是hcpptest，hcpptest测试框架是在开源的googletest测试框架的基础上进行的增强和适配。 +The HCPPTest framework is enhanced and adapted based on the open-source framework Googletest. -1. 规范用例目录：测试用例存储到test/xts/dcts仓中。 +1. Access the **test/xts/dcts** repository where the test cases will be stored. ``` ├── dcts @@ -385,17 +384,17 @@ OpenHarmony支持如下几种系统类型： │ │ └── BUILD.gn ``` -2. 测试模块src下用例编写样例： +2. Write the test case in the **src** directory. - 1. 引用测试框架： + 1. Import the test framework header file. - 需要引用gtest.h 如：\#include "gtest/gtest.h" + The following statement includes **gtest.h**. ``` #include "gtest/gtest.h" ``` - 2. 定义Setup与TearDown + 2. Define Setup and TearDown. ``` using namespace std; @@ -419,25 +418,25 @@ OpenHarmony支持如下几种系统类型： }; ``` - 3. 使用宏定义HWTEST或HWTEST\_F写测试用例 + 3. Use the **HWTEST** or **HWTEST\_F** macro to write the test case. - 普通测试用例的定义：HWTEST（测试套名称， 测试用例名称， 用例标注）。 + **HWTEST**: definition of common test cases, including the test suite name, test case name, and case annotation. - 包含SetUp和TearDown的测试用例的定义 ：HWTEST\_F（测试套名称， 测试用例名称，用例标注）。 + **HWTEST\_F**: definition of SetUp and TearDown test cases, including the test suite name, test case name, and case annotation. - 宏定义包括三个参数：测试套件名称，测试用例名称，用例属性（测试类型、用例粒度、用例级别）。 + Three parameters are involved: test suite name, test case name, and test case properties \(including type, granularity, and level\). ``` HWTEST_F(TestSuite, TestCase_0001, Function | MediumTest | Level1) { - // do something + // Do something } ``` -3. 测试模块下用例配置文件（BUILD.gn）样例： +3. Create a configuration file \(**BUILD.gn**\) of the test module. - 每个测试模块目录下新建BUILD.gn编译文件，用于指定编译后可执行文件的名称、依赖的头文件、依赖的库等；具体写法如下。每个测试模块将独立编译成.bin可执行文件， 该文件可直接push到单板上进行测试。 + Create a **BUILD.gn** build file in each test module directory. Specify the name of the built static library and its dependent header file and library in the build file. Each test module is independently built into a **.bin** executable file, which can be directly pushed to the development board for testing. - 举例： + Example: ``` import("//test/xts/tools/lite/build/suite_lite.gni") @@ -456,12 +455,11 @@ OpenHarmony支持如下几种系统类型： ] cflags = [ "-Wno-error" ] } - ``` -4. dcts目录下增加编译选项（BUILD.gn）样例： +4. Add build options to the **BUILD.gn** file in the **dcts** directory. - 将测试模块加入到dcts目录下的编译脚本中，编译脚本为：test/xts/dcts/build\_lite/BUILD.gn。 + Add the test module to the **test/xts/dcts/build\_lite/BUILD.gn** script in the **dcts** directory. ``` lite_component("dcts") { @@ -475,79 +473,36 @@ OpenHarmony支持如下几种系统类型： } ``` -5. 测试套件编译命令。 +5. Run build commands. - 随版本编译，debug版本编译时会同步编译dcts测试套件 + Test suites are built along with the version build. The DCTS is built together with the debug version. - >![](figures/icon-note.gif) **说明：** - >小型系统dcts独立编译成可执行文件（bin格式）， 在编译产物的suites\\dcts目录下归档。 + >![](figures/icon-note.gif) **NOTE:** + >The DCTS for the small system is independently built to an executable file \(.bin\) and archived in the **suites\\dcts** directory of the build result. -### C++语言用例执行指导（适用于小型系统、标准系统用例开发） +### C++-based Test Case Execution \(for Standard and Small Systems\) -**示例：小型系统测试用例执行** +**Executing test cases for the small system** -目前的用例执行采用nfs共享的方式，mount到单板去执行。 +Currently, test cases are shared by the NFS and mounted to the development board for execution. -**环境搭建** +**Setting up the environment** -1. 使用网线或无线网络将开发板与PC进行连接。 -2. 开发板配置IP、子网掩码、网关，确保开发板与PC处于同一个网段。 -3. PC安装nfs服务器并完成注册，启动nfs服务。 -4. 开发板配置mount命令，确保开发板可以访问PC端的nfs共享文件。 +1. Use a network cable or wireless network to connect the development board to your PC. +2. Configure the IP address, subnet mask, and gateway for the development board. Ensure that the development board and the PC are in the same network segment. +3. Install and register the NFS server on the PC and start the NFS service. +4. Run the **mount** command for the development board to ensure that the development board can access NFS shared files on the PC. - 格式：mount \[nfs服务器IP\]:\[/nfs共享目录\] \[/开发板目录\] nfs + Format: **mount** _NFS server IP address_**:/**_NFS shared directory_ **/**_development board directory_ **nfs** - 举例： + Example: ``` mount 192.168.1.10:/nfs /nfs nfs ``` -**用例执行** - -测试套件执行 DctsDemoTest.bin 触发用例执行，基于串口打印日志进行分析。 - -### 全量编译指导（适用于标准系统） - -全量编译test/xts/dcts目录下执行编译命令: ./build.sh suite=dcts system_size=standard - -测试用例输出目录：out/release/suites/dcts/testcases - -测试框架&用例整体输出目录：out/release/suites/dcts(编译用例时会同步编译测试套执行框架) - -### 全量用例执行指导（适用于小型系统、标准系统） - -搭建测试环境 Windows工作台下安装python3.7及以上版本，确保工作台和测试设备正常连接。 - -测试执行目录（对应编译生成的out/release/suites/dcts目录） - - ``` -├── testcase # 测试套文件存放目录 -│ └──xxx.hap # 测试套可执行hap文件 -│ └──xxx.json # 测试套对应执行配置文件 -├── tools # 测试框架工具目录 -├── run.bat # window平台测试套启动执行文件 -├── report # 测试报告生成目录 - - ``` -用例执行 - -在Windows工作台上，找到从Linux服务器上拷贝下来的测试套件用例目录，在Windows命令窗口进入对应目录，直接执行dcts\run.bat。 - -界面启动后，输入用例执行指令。 - -全量执行：run dcts - -模块执行(具体模块可以查看\dcts\testcases)：run –l DctsSamgrTest - -查看测试报告。 进入dcts\reports\，获取当前的执行记录，打开“summary_report.html”可以获取到测试报告。 - - -## 相关仓 - -xts\_dcts - -xts\_tools +**Executing test cases** +Execute **DctsDemoTest.bin** to trigger test case execution, and analyze serial port logs generated after the execution is complete. diff --git a/README_ZH.md b/README_ZH.md new file mode 100644 index 0000000..a7d0e5b --- /dev/null +++ b/README_ZH.md @@ -0,0 +1,553 @@ +# XTS子系统 + +- [简介](#section465982318513) +- [系统类型](#section125090457443) +- [目录](#section161941989596) +- [约束](#section119744591305) +- [使用说明](#section137768191623) +- [用例开发指导](#section3695134065513) + - [C语言用例开发编译指导（适用于轻量系统产品用例开发）](#section198193336544) + - [C语言用例执行指导（适用于轻量系统产品用例开发）](#section13820233175418) + - [C++语言用例开发编译指导（适用于小型系统、标准系统用例开发）](#section3822123311540) + - [C++语言用例执行指导（适用于小型系统、标准系统用例开发）](#section128222336544) + +- [相关仓](#section1371113476307) + +## 简介 + +XTS子系统是OpenHarmony生态认证测试套件的集合，当前包括dcts（distributed compatibility test suite）分布式兼容性测试套件。 + +XTS子系统当前包括dcts与tools软件包： + +- dcts，存放dcts相关测试用例源码与配置文件，其目的是帮助终端设备厂商尽早发现在分布式场景下与OpenHarmony的不兼容性，确保软件在整个开发过程中满足OpenHarmony的兼容性要求。 +- tools，存放dcts相关测试用例开发框架。 + +## 系统类型 + +OpenHarmony支持如下几种系统类型： + +- 轻量系统（mini system） + + 面向MCU类处理器例如Arm Cortex-M、RISC-V 32位的设备，硬件资源极其有限，支持的设备最小内存为128KiB，可以提供多种轻量级网络协议，轻量级的图形框架，以及丰富的IOT总线读写部件等。可支撑的产品如智能家居领域的连接类模组、传感器设备、穿戴类设备等。 + +- 小型系统（small system） + + 面向应用处理器例如Arm Cortex-A的设备，支持的设备最小内存为1MiB，可以提供更高的安全能力、标准的图形框架、视频编解码的多媒体能力。可支撑的产品如智能家居领域的IP Camera、电子猫眼、路由器以及智慧出行领域的行车记录仪等。 + +- 标准系统（standard system） + + 面向应用处理器例如Arm Cortex-A的设备，支持的设备最小内存为128MiB，可以提供增强的交互能力、3D GPU以及硬件合成能力、更多控件以及动画效果更丰富的图形能力、完整的应用框架。可支撑的产品如高端的冰箱显示屏。 + + +## 目录 + +``` +/test/xts +├── dcts # 测试代码存放目录 +│ └── subsystem # 标准系统子系统测试用例源码存放目录 +│ └── subsystem_lite # 轻量系统、小型系统子系统测试用例源码存放目录 +│ └── common # 测试用例依赖共享内存源码存放目录 +│ └── BUILD.gn # 标准系统测试用例编译配置 +│ └── build_lite # 轻量系统、小型系统测试用例编译配置存放目录 +│ └── BUILD.gn # 轻量系统、小型系统测试用例编译配置 +└── tools # 测试工具代码存放目录 +``` + +## 约束 + +轻量系统用例开发语言是C，小型系统用例开发语言是C++。 + +## 使用说明 + +**表 1** 用例级别说明 + + + + + + + + + + + + + + + + + + + + + + + + + + + + +

级别名称

+

基本定义

+

测试范围

+

Level0

+

冒烟

+

验证关键功能点基本功能/最基本DFX属性在最常见输入下的表现，通过表示功能基本可运行。

+

Level1

+

基本

+

验证各功能点基本功能/基本DFX属性在常见输入下的表现，通过表示功能基本可测试。

+

Level2

+

重要

+

验证各功能点的基本功能/基本DFX属性在常规输入/常见异常情况下的表现，通过表示功能基本正常可用，可开展Beta。

+

Level3

+

一般

+

验证各功能点的全部功能/全部DFX属性在各种常规/非常规输入组合下，或各种正常/异常预置条件组合下的表现。

+

Level4

+

生僻

+

验证关键功能点在极端异常预置条件下、用户难以触及的异常输入组合下的表现。

+
+ +**表 2** 用例粒度说明 + + + + + + + + + + + + + + + + + + + + +

用例规模

+

被测试对象

+

测试环境

+

LargeTest

+

业务功能/全场景特性/整机及场景级DFX

+

尽量使用贴近真实的环境设备

+

MediumTest

+

模块/子系统集成至设备后的功能/DFX

+

使用真实的单设备进行验证，可进行消息模拟，尽量不对函数进行MOCK

+

SmallTest

+

模块/类/函数

+

在开发者个人环境进行测试，尽量不依赖其他模块，存在大量的MOCK

+
+ +**表 3** 测试类型说明 + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +

测试类型名称

+

测试类型定义

+

Function

+

验证被测对象提供给用户的业务功能实现正确性的测试项，这里的“用户”可以是终端用户或开发者，功能包括业务功能及平台功能

+

Performance

+

验证被测对象在特定预置条件/负载模型下的处理能力的测试项，“处理能力”一般以单位时间内可处理的业务量来衡量，如呼叫/秒，帧率/秒，事件处理量/秒等

+

Power

+

验证被测对象在特定预置条件/负载模型下在一定时间内能源消耗量的测试项

+

Reliability

+

验证被测对象在正常/异常输入情况下，或业务量压力和长时间连续运行压力情况下业务表现的测试项，含稳定性、压力、故障注入、Monkey测试项

+

Security

+

验证系统对恶意威胁的防护能力，威胁包括但不限于未授权访问、使用、泄露、破坏、修改、毁灭，以保障信息的机密性、完整性和可用性; 验证系统对用户隐私的保护能力，保障用户的隐私数据被收集、使用、保有、披露和处置符合法律规范，保障用户的隐私权; 验证对各类安全规范的遵从情况，如安全设计规范、安全红线、工信部安全认证规范等，保障安全相关法律法规的合规。

+

Global

+

验证被测对象在是否具有国际化数据支持和本地化能力的测试项，包括语言显示、输入/输出习惯、时间显示、区域特性如货币时间禁忌等等

+

Compatibility

+

当被测对象为应用时，包括被测对象对于自身数据的后向兼容性、对于系统的前后向兼容性、对于不同用户数据(如播放器之音频文件格式/智能短信之用户短信内容)的兼容性测试项； 当被测对象为系统时，包括被测系统对于系统自身数据的后向兼容性、以及对于生态中常用应用的兼容性测试项；当被测对象为软件时，包括被测系统对于相关的硬件的兼容性；

+

User

+

验证被测对象在真实用户场景下的用户体验感受的测试项，注意此种情况下没有客观的“正确”与“失败”，所有的结论及评价都应该来自于用户

+

Standard

+

验证被测对象对于行业及公司内标准/协议/规范的遵从情况的测试项，注意此处的“标准”不包含任何安全标准，针对安全标准的测试项划归为“安全测试”类型

+

Safety

+

验证被测对象的Safety属性，避免产品可能对人身安全、健康以及产品本身带来的危害。

+

Resilience

+

验证被测对象的韧性属性，确保系统受攻击时承受并保持在有定义的运行状态（包括降级）、恢复并适应攻击以保障Mission达成。

+
+ +## 用例开发指导 + +根据测试系统选择测试框架和对应测试用例语言。 + +**表 4** 系统和测试框架、开发语言对应关系 + + + + + + + + + + + + + + + + + + + + +

系统

+

测试框架

+

语言

+

轻量系统

+

hctest

+

c

+

小型系统

+

hcpptest

+

c++

+

标准系统

+

HJSUnit、hcpptest

+

js、c++

+
+ +### C语言用例开发编译指导（适用于轻量系统产品用例开发） + +**示例：轻量系统测试用例开发** + +当前使用的测试框架是hctest，hctest测试框架支持使用C语言编写测试用例，是在开源测试框架unity的基础上进行增强和适配。 + +1. 用例目录规范：测试用例存储到test/xts/dcts仓中 + + ``` + ├── dcts + │ └──subsystem_lite + │ │ └── module_hal + │ │ │ └── BUILD.gn + │ │ │ └── src + │ └──build_lite + │ │ └── BUILD.gn + ``` + +2. src目录下用例编写样例。 + + 1.引用测试框架 + + ``` + #include "hctest.h" + ``` + + 2. 使用宏定义LITE\_TEST\_SUIT定义子系统、模块、测试套件名称 + + ``` + /** + * @brief register a test suit named "IntTestSuite" + * @param test subsystem name + * @param example module name + * @param IntTestSuite test suit name + */ + LITE_TEST_SUIT(test, example, IntTestSuite); + ``` + + 3. 定义Setup与TearDown + + 命名方式：测试套件名称+Setup，测试套件名称+TearDown。 + + Setup与TearDown必须存在，可以为空函数。 + + 4. 使用宏定义LITE\_TEST\_CASE写测试用例 + + 包括三个参数：测试套件名称，测试用例名称，用例属性（测试类型、用例粒度、用例级别）。 + + ``` + LITE_TEST_CASE(IntTestSuite, TestCase001, Function | MediumTest | Level1) + { + //do something + }; + ``` + + 5. 使用宏定义 RUN\_TEST\_SUITE注册测试套件 + + ``` + RUN_TEST_SUITE(IntTestSuite); + ``` + +3. 测试模块的配置文件（BUILD.gn）样例： + + 在每个测试模块目录下新建BUILD.gn编译文件，用于指定编译后静态库的名称、依赖的头文件、依赖的库等；具体写法如下： + + ``` + import("//test/xts/tools/lite/build/suite_lite.gni") + hctest_suite("DctsDemoTest") { + suite_name = "dcts" + sources = [ + "src/test_demo.c", + ] + include_dirs = [ ] + cflags = [ "-Wno-error" ] + } + ``` + +4. dcts下BUILD.gn增加编译选项。 + + 需要将测试模块加入到dcts目录下的编译脚本中，编译脚本路径：test/xts/dcts/build\_lite/BUILD.gn。 + + ``` + lite_component("dcts") { + ... + if(board_name == "liteos_m") { + features += [ + ... + "//xts/dcts/subsystem_lite/module_hal:DctsDemoTest" + ] + } + } + ``` + +5. 测试套件编译命令。 + + 随版本编译，debug版本编译时会同步编译dcts测试套件 + + >![](figures/icon-note.gif) **说明：** + >dcts测试套件编译中间件为静态库，最终链接到版本镜像中 。 + + +### C语言用例执行指导（适用于轻量系统产品用例开发） + +**示例：轻量系统测试用例执行** + +将版本镜像烧录进开发板。 + +**测试步骤** + +1. 使用串口工具登录开发板，并保存串口打印信息。 +2. 重启设备，查看串口日志。 + +**测试结果分析指导** + +基于串口打印日志进行分析； + +每个测试套件执行以Start to run test suite开始，以xx Tests xx Failures xx Ignored结束。 + +### C++语言用例开发编译指导（适用于小型系统、标准系统用例开发） + +**示例：小型系统测试用例开发**（标准系统参考具体样例目录：global/i18n\_standard） + +当前使用的测试框架是hcpptest，hcpptest测试框架是在开源的googletest测试框架的基础上进行的增强和适配。 + +1. 规范用例目录：测试用例存储到test/xts/dcts仓中。 + + ``` + ├── dcts + │ └──subsystem_lite + │ │ └── module_posix + │ │ │ └── BUILD.gn + │ │ │ └── src + │ └──build_lite + │ │ └── BUILD.gn + ``` + +2. 测试模块src下用例编写样例： + + 1. 引用测试框架： + + 需要引用gtest.h 如：\#include "gtest/gtest.h" + + ``` + #include "gtest/gtest.h" + ``` + + 2. 定义Setup与TearDown + + ``` + using namespace std; + using namespace testing::ext; + class TestSuite: public testing::Test { + protected: + // Preset action of the test suite, which is executed before the first test case + static void SetUpTestCase(void){ + } + // Test suite cleanup action, which is executed after the last test case + static void TearDownTestCase(void){ + } + // Preset action of the test case + virtual void SetUp() + { + } + // Cleanup action of the test case + virtual void TearDown() + { + } + }; + ``` + + 3. 使用宏定义HWTEST或HWTEST\_F写测试用例 + + 普通测试用例的定义：HWTEST（测试套名称， 测试用例名称， 用例标注）。 + + 包含SetUp和TearDown的测试用例的定义 ：HWTEST\_F（测试套名称， 测试用例名称，用例标注）。 + + 宏定义包括三个参数：测试套件名称，测试用例名称，用例属性（测试类型、用例粒度、用例级别）。 + + ``` + HWTEST_F(TestSuite, TestCase_0001, Function | MediumTest | Level1) { + // do something + } + ``` + +3. 测试模块下用例配置文件（BUILD.gn）样例： + + 每个测试模块目录下新建BUILD.gn编译文件，用于指定编译后可执行文件的名称、依赖的头文件、依赖的库等；具体写法如下。每个测试模块将独立编译成.bin可执行文件， 该文件可直接push到单板上进行测试。 + + 举例： + + ``` + import("//test/xts/tools/lite/build/suite_lite.gni") + hcpptest_suite("DctsDemoTest") { + suite_name = "dcts" + sources = [ + "src/TestDemo.cpp" + ] + + include_dirs = [ + "src", + ... + ] + deps = [ + ... + ] + cflags = [ "-Wno-error" ] + } + + ``` + +4. dcts目录下增加编译选项（BUILD.gn）样例： + + 将测试模块加入到dcts目录下的编译脚本中，编译脚本为：test/xts/dcts/build\_lite/BUILD.gn。 + + ``` + lite_component("dcts") { + ... + else if(board_name == "liteos_a") { + features += [ + ... + "//xts/dcts/subsystem_lite/module_posix:DctsDemoTest" + ] + } + } + ``` + +5. 测试套件编译命令。 + + 随版本编译，debug版本编译时会同步编译dcts测试套件 + + >![](figures/icon-note.gif) **说明：** + >小型系统dcts独立编译成可执行文件（bin格式）， 在编译产物的suites\\dcts目录下归档。 + + +### C++语言用例执行指导（适用于小型系统、标准系统用例开发） + +**示例：小型系统测试用例执行** + +目前的用例执行采用nfs共享的方式，mount到单板去执行。 + +**环境搭建** + +1. 使用网线或无线网络将开发板与PC进行连接。 +2. 开发板配置IP、子网掩码、网关，确保开发板与PC处于同一个网段。 +3. PC安装nfs服务器并完成注册，启动nfs服务。 +4. 开发板配置mount命令，确保开发板可以访问PC端的nfs共享文件。 + + 格式：mount \[nfs服务器IP\]:\[/nfs共享目录\] \[/开发板目录\] nfs + + 举例： + + ``` + mount 192.168.1.10:/nfs /nfs nfs + ``` + + +**用例执行** + +测试套件执行 DctsDemoTest.bin 触发用例执行，基于串口打印日志进行分析。 + +### 全量编译指导（适用于标准系统） + +全量编译test/xts/dcts目录下执行编译命令: ./build.sh suite=dcts system_size=standard + +测试用例输出目录：out/release/suites/dcts/testcases + +测试框架&用例整体输出目录：out/release/suites/dcts(编译用例时会同步编译测试套执行框架) + +### 全量用例执行指导（适用于小型系统、标准系统） + +搭建测试环境 Windows工作台下安装python3.7及以上版本，确保工作台和测试设备正常连接。 + +测试执行目录（对应编译生成的out/release/suites/dcts目录） + + ``` +├── testcase # 测试套文件存放目录 +│ └──xxx.hap # 测试套可执行hap文件 +│ └──xxx.json # 测试套对应执行配置文件 +├── tools # 测试框架工具目录 +├── run.bat # window平台测试套启动执行文件 +├── report # 测试报告生成目录 + + ``` +用例执行 + +在Windows工作台上，找到从Linux服务器上拷贝下来的测试套件用例目录，在Windows命令窗口进入对应目录，直接执行dcts\run.bat。 + +界面启动后，输入用例执行指令。 + +全量执行：run dcts + +模块执行(具体模块可以查看\dcts\testcases)：run –l DctsSamgrTest + +查看测试报告。 进入dcts\reports\，获取当前的执行记录，打开“summary_report.html”可以获取到测试报告。 + + +## 相关仓 + +xts\_dcts + +xts\_tools + diff --git a/figures/icon-caution.gif b/figures/icon-caution.gif new file mode 100755 index 0000000..6e90d7c Binary files /dev/null and b/figures/icon-caution.gif differ diff --git a/figures/icon-danger.gif b/figures/icon-danger.gif new file mode 100755 index 0000000..6e90d7c Binary files /dev/null and b/figures/icon-danger.gif differ diff --git a/figures/icon-note.gif b/figures/icon-note.gif new file mode 100755 index 0000000..6314297 Binary files /dev/null and b/figures/icon-note.gif differ diff --git a/figures/icon-notice.gif b/figures/icon-notice.gif new file mode 100755 index 0000000..86024f6 Binary files /dev/null and b/figures/icon-notice.gif differ diff --git a/figures/icon-tip.gif b/figures/icon-tip.gif new file mode 100755 index 0000000..93aa720 Binary files /dev/null and b/figures/icon-tip.gif differ diff --git a/figures/icon-warning.gif b/figures/icon-warning.gif new file mode 100755 index 0000000..6e90d7c Binary files /dev/null and b/figures/icon-warning.gif differ