mirror of
https://github.com/openharmony/developtools_integration_verification.git
synced 2026-07-24 04:48:00 -04:00
Description:[feature] optimize readme opensource validate content
Bug: https://gitee.com/openharmony/developtools_integration_verification/issues/IAT8BB Test: run validate_readme_opensource.py Signed-off-by: Jinguang Dong <dongjinguang@huawei.com>
This commit is contained in:
Jinguang Dong
@@ -1,83 +1,42 @@
|
||||
# 开源软件README.OpenSource工具使用文档
|
||||
# 开源软件 `README.OpenSource` 工具使用文档
|
||||
|
||||
## 目录
|
||||
|
||||
- [开源软件README.OpenSource工具使用文档](#开源软件readmeopensource工具使用文档)
|
||||
- [目录](#目录)
|
||||
- [简介](#简介)
|
||||
- [功能概述](#功能概述)
|
||||
- [工作流程概览](#工作流程概览)
|
||||
- [生成工具工作流程](#生成工具工作流程)
|
||||
- [验证工具工作流程](#验证工具工作流程)
|
||||
- [安装与环境配置](#安装与环境配置)
|
||||
- [环境要求](#环境要求)
|
||||
- [安装步骤](#安装步骤)
|
||||
- [使用指南](#使用指南)
|
||||
- [生成 `README.OpenSource` 文件](#生成-readmeopensource-文件)
|
||||
- [步骤](#步骤)
|
||||
- [示例](#示例)
|
||||
- [生成工具流程图](#生成工具流程图)
|
||||
- [验证 `README.OpenSource` 文件](#验证-readmeopensource-文件)
|
||||
- [步骤](#步骤-1)
|
||||
- [示例](#示例-1)
|
||||
- [验证工具流程图](#验证工具流程图)
|
||||
- [处理验证错误](#处理验证错误)
|
||||
- [测试与验证](#测试与验证)
|
||||
- [自动化测试](#自动化测试)
|
||||
- [运行测试](#运行测试)
|
||||
- [预期结果](#预期结果)
|
||||
- [手动测试](#手动测试)
|
||||
- [生成工具测试](#生成工具测试)
|
||||
- [验证工具测试](#验证工具测试)
|
||||
- [目录结构](#目录结构)
|
||||
- 开源软件 `README.OpenSource` 工具使用文档
|
||||
- 简介
|
||||
- 功能概述
|
||||
- 安装与环境配置
|
||||
- 环境要求
|
||||
- 安装步骤
|
||||
- 使用指南
|
||||
- 生成 `README.OpenSource` 文件
|
||||
- 验证 `README.OpenSource` 文件
|
||||
- 验证格式
|
||||
- 验证内容
|
||||
- 命令行参数
|
||||
- 测试与验证
|
||||
- 自动化测试
|
||||
- 运行测试
|
||||
- 预期结果
|
||||
- 手动测试
|
||||
- 目录结构
|
||||
- 工作流程概览
|
||||
- 生成工具流程图
|
||||
- 验证工具流程图
|
||||
|
||||
## 简介
|
||||
|
||||
本开源软件工具旨在为项目中的每个开源组件创建标准化的 `README.OpenSource` 文件，并提供验证这些文件格式和内容的功能。该工具仅使用 Python 标准库开发，易于安装和使用。
|
||||
本工具旨在为项目中的每个开源部件创建标准化的 `README.OpenSource` 文件，并提供验证这些文件格式和内容的功能。该工具仅使用 Python 标准库开发，易于安装和使用。通过合并生成和验证功能，用户可以使用单个脚本完成所有操作，并根据需要选择不同的功能场景。
|
||||
|
||||
## 功能概述
|
||||
|
||||
- **生成工具**：`generate_readme_opensource.py`
|
||||
- 通过交互方式，用户输入开源组件的信息，支持多个组件的输入。
|
||||
- 生成符合规范的 `README.OpenSource` 文件，包含所有输入的组件信息。
|
||||
- **验证工具**：`validate_readme_opensource.py`
|
||||
- 验证项目中所有 `README.OpenSource` 文件的格式和内容。
|
||||
- 检查必需字段是否完整，JSON 格式是否正确。
|
||||
|
||||
### 工作流程概览
|
||||
|
||||
下面是对应的工具的工作流程示意图
|
||||
|
||||
#### 生成工具工作流程
|
||||
|
||||
```mermaid
|
||||
flowchart TD
|
||||
A[开始] --> B[输入输出目录和开源部件信息]
|
||||
B --> C{是否继续添加开源部件?}
|
||||
C -->|是| D[输入开源部件信息]
|
||||
D --> C
|
||||
C -->|否| E[生成README.OpenSource文件]
|
||||
E --> F[结束]
|
||||
|
||||
```
|
||||
|
||||
#### 验证工具工作流程
|
||||
|
||||
```mermaid
|
||||
flowchart TD
|
||||
A[开始] --> B[输入要验证的目录]
|
||||
B --> C[搜索README.OpenSource文件]
|
||||
C --> D{找到文件?}
|
||||
D -->|是| E[读取并验证文件]
|
||||
E --> F{验证通过?}
|
||||
F -->|是| G[输出验证成功信息]
|
||||
F -->|否| H[输出错误信息]
|
||||
D -->|否| I[输出未找到文件信息]
|
||||
G & H & I --> J[结束]
|
||||
|
||||
```
|
||||
|
||||
- 生成工具 ：
|
||||
- 通过交互方式，用户输入开源部件的信息，支持多个部件的输入。
|
||||
- 生成符合规范的 `README.OpenSource` 文件，包含所有输入的部件信息。
|
||||
|
||||
- 验证工具：
|
||||
- **格式验证**：验证项目中所有 `README.OpenSource` 文件的格式，检查必需字段是否完整，JSON 格式是否正确。
|
||||
- **内容验证**：对 `README.OpenSource` 文件中的特定字段，如 `"Name"`、`"License"`、`"Version Number"`、`"Upstream URL"`，与参考数据进行比对，确保其内容符合预期。还会验证 `"License File"` 字段指向的文件是否存在，路径应相对于 `README.OpenSource` 文件所在目录。
|
||||
|
||||
## 安装与环境配置
|
||||
|
||||
@@ -112,9 +71,9 @@ flowchart TD
|
||||
|
||||
### 生成 `README.OpenSource` 文件
|
||||
|
||||
运行 `generate_readme_opensource.py` 脚本，按照提示输入开源组件的信息。
|
||||
运行 `generate_readme_opensource.py` 脚本，按照提示输入开源部件的信息。
|
||||
|
||||
#### 步骤
|
||||
**步骤：**
|
||||
|
||||
1. **进入项目目录**
|
||||
|
||||
@@ -131,7 +90,7 @@ flowchart TD
|
||||
3. **按照提示输入信息**
|
||||
|
||||
- 脚本将提示您输入输出目录，默认为当前目录。
|
||||
- 输入每个组件的详细信息，包括：
|
||||
- 输入每个部件的详细信息，包括：
|
||||
- Name
|
||||
- License
|
||||
- License File
|
||||
@@ -139,15 +98,17 @@ flowchart TD
|
||||
- Owner
|
||||
- Upstream URL
|
||||
- Description
|
||||
- 输入完成后，选择是否添加另一个组件。
|
||||
- 输入完成后，选择是否添加另一个部件。
|
||||
|
||||
4. **完成生成**
|
||||
|
||||
- 脚本将在指定的输出目录下生成 `README.OpenSource` 文件。
|
||||
|
||||
#### 示例
|
||||
**示例：**
|
||||
|
||||
```
|
||||
python generate_readme_opensource.py
|
||||
|
||||
请输入输出目录（默认当前目录）：
|
||||
Name: elfutils
|
||||
License: LGPL-2.1, LGPL-3.0, GPL-2.0
|
||||
@@ -156,7 +117,7 @@ Version Number: 0.188
|
||||
Owner: opensource@sourceware.org
|
||||
Upstream URL: https://sourceware.org/elfutils/
|
||||
Description: A collection of tools and libraries.
|
||||
是否添加另一个组件？(y/n): y
|
||||
是否添加另一个部件？(y/n): y
|
||||
Name: OpenSSL
|
||||
License: Apache-2.0
|
||||
License File: LICENSE
|
||||
@@ -164,38 +125,19 @@ Version Number: 1.1.1
|
||||
Owner: opensource@openssl.org
|
||||
Upstream URL: https://www.openssl.org/
|
||||
Description: A toolkit for TLS and SSL protocols.
|
||||
是否添加另一个组件？(y/n): n
|
||||
是否添加另一个部件？(y/n): n
|
||||
已生成 ./README.OpenSource
|
||||
```
|
||||
|
||||
#### 生成工具流程图
|
||||
|
||||
```mermaid
|
||||
sequenceDiagram
|
||||
participant User as 用户
|
||||
participant Script as generate_readme_opensource.py
|
||||
User->>Script: 运行脚本
|
||||
Script-->>User: 请输入输出目录
|
||||
User->>Script: 输入输出目录
|
||||
Script-->>User: 请输入部件信息
|
||||
User->>Script: 输入开源部件信息
|
||||
loop 添加开源部件
|
||||
Script-->>User: 请输入部件信息
|
||||
User->>Script: 输入开源部件信息
|
||||
Script-->>User: 是否添加另一个开源部件？(y/n)
|
||||
User->>Script: y 或 n
|
||||
end
|
||||
Script->>FileSystem: 生成README.OpenSource文件
|
||||
Script-->>User: 已生成 README.OpenSource
|
||||
```
|
||||
|
||||
|
||||
|
||||
### 验证 `README.OpenSource` 文件
|
||||
|
||||
运行 `validate_readme_opensource.py` 脚本，验证指定目录中的 `README.OpenSource` 文件。
|
||||
运行 `validate_readme_opensource.py` 脚本，使用不同的参数进行格式或内容验证。
|
||||
|
||||
#### 步骤
|
||||
#### 验证格式
|
||||
|
||||
验证项目中所有 `README.OpenSource` 文件的格式和必需字段。
|
||||
|
||||
**步骤：**
|
||||
|
||||
1. **进入项目目录**
|
||||
|
||||
@@ -203,10 +145,10 @@ sequenceDiagram
|
||||
cd src
|
||||
```
|
||||
|
||||
2. **运行验证脚本**
|
||||
2. **运行格式验证脚本**
|
||||
|
||||
```
|
||||
python validate_readme_opensource.py [目录路径]
|
||||
python validate_readme_opensource.py --validate-format [目录路径]
|
||||
```
|
||||
|
||||
- 如果不指定目录路径，默认验证当前目录。
|
||||
@@ -216,47 +158,70 @@ sequenceDiagram
|
||||
- 脚本将输出验证结果，指示文件是否有效。
|
||||
- 如果有错误，脚本会列出具体的错误信息。
|
||||
|
||||
#### 示例
|
||||
**示例：**
|
||||
|
||||
```
|
||||
python validate_readme_opensource.py .
|
||||
python validate_readme_opensource.py --validate-format .
|
||||
|
||||
./README.OpenSource is valid.
|
||||
./README.OpenSource format is valid.
|
||||
```
|
||||
|
||||
#### 验证工具流程图
|
||||
#### 验证内容
|
||||
|
||||
```mermaid
|
||||
sequenceDiagram
|
||||
participant User as 用户
|
||||
participant Script as validate_readme_opensource.py
|
||||
User->>Script: 运行脚本
|
||||
Script->>FileSystem: 搜索 README.OpenSource 文件
|
||||
alt 找到文件
|
||||
Script->>Script: 验证文件内容
|
||||
alt 验证通过
|
||||
Script-->>User: 文件有效
|
||||
else 验证失败
|
||||
Script-->>User: 文件无效，输出错误信息
|
||||
end
|
||||
else 未找到文件
|
||||
Script-->>User: 未找到 README.OpenSource 文件
|
||||
end
|
||||
```
|
||||
对 `README.OpenSource` 文件中的特定字段与参考数据进行比对。
|
||||
|
||||
**步骤：**
|
||||
|
||||
1. **准备参考数据**
|
||||
|
||||
#### 处理验证错误
|
||||
创建一个包含参考数据的 JSON 文件，例如 `reference_data.json`，内容如下：
|
||||
|
||||
如果验证失败，脚本会输出类似以下的错误信息：
|
||||
```
|
||||
[
|
||||
{
|
||||
"Name": "elfutils",
|
||||
"License": "LGPL-2.1, LGPL-3.0, GPL-2.0",
|
||||
"Version Number": "0.188",
|
||||
"Upstream URL": "https://sourceware.org/elfutils/"
|
||||
},
|
||||
{
|
||||
"Name": "OpenSSL",
|
||||
"License": "Apache-2.0",
|
||||
"Version Number": "1.1.1",
|
||||
"Upstream URL": "https://www.openssl.org/"
|
||||
}
|
||||
]
|
||||
```
|
||||
|
||||
2. **运行内容验证脚本**
|
||||
|
||||
```
|
||||
python validate_readme_opensource.py --validate-content --reference-data reference_data.json [目录路径]
|
||||
```
|
||||
|
||||
3. **查看验证结果**
|
||||
|
||||
- 脚本将输出验证结果，指示文件是否有效。
|
||||
- 如果有错误，脚本会列出具体的错误信息。
|
||||
|
||||
**示例：**
|
||||
|
||||
```
|
||||
./README.OpenSource is invalid:
|
||||
- Component 1 is missing required field: License
|
||||
- JSON decode error: Expecting value: line 10 column 5 (char 250)
|
||||
python validate_readme_opensource.py --validate-content --reference-data reference_data.json .
|
||||
|
||||
Validating: ./README.OpenSource
|
||||
./README.OpenSource: Content validation passed.
|
||||
Validation process completed.
|
||||
```
|
||||
|
||||
请根据提示修复 `README.OpenSource` 文件中的错误，然后重新运行验证脚本。
|
||||
### 命令行参数
|
||||
|
||||
- `--validate-format`：验证 `README.OpenSource` 文件的格式和必需字段。
|
||||
- `--validate-content`：验证 `README.OpenSource` 文件的内容，与参考数据比对。
|
||||
- `--reference-data`：指定参考数据的 JSON 文件路径（内容验证时必需）。
|
||||
- `--log-file`：指定日志文件路径，保存验证结果。
|
||||
|
||||
**注意**：`--validate-format` 和 `--validate-content` 可以组合使用，顺序执行格式和内容验证。
|
||||
|
||||
## 测试与验证
|
||||
|
||||
@@ -292,26 +257,65 @@ OK
|
||||
|
||||
#### 生成工具测试
|
||||
|
||||
- **测试正常输入**：按照使用指南运行生成脚本，输入多个组件的信息，检查生成的 `README.OpenSource` 文件内容是否正确。
|
||||
- **测试正常输入**：按照使用指南运行生成脚本，输入多个部件的信息，检查生成的 `README.OpenSource` 文件内容是否正确。
|
||||
- **测试边界情况**：输入空值、特殊字符、超长字符串，观察脚本是否能正常处理。
|
||||
|
||||
#### 验证工具测试
|
||||
|
||||
- **验证正确的文件**：使用生成的正确的 `README.OpenSource` 文件，运行验证脚本，确保验证通过。
|
||||
- **验证错误的文件**：手动修改 `README.OpenSource` 文件，引入格式错误或缺少字段，运行验证脚本，检查是否能正确捕获错误。
|
||||
- **验证内容不匹配**：修改 `README.OpenSource` 文件的字段，使其与参考数据不一致，运行内容验证，检查是否能正确报告不匹配项。
|
||||
|
||||
## 目录结构
|
||||
|
||||
以下是项目的目录结构：
|
||||
|
||||
- src/
|
||||
- **src/**
|
||||
- `generate_readme_opensource.py`：生成README.OpenSource开源不见配置信息脚本
|
||||
- `validate_readme_opensource.py`：验证工具脚本，包含JSON格式验证和内容验证功能。
|
||||
|
||||
- `generate_readme_opensource.py`：生成工具脚本
|
||||
- `validate_readme_opensource.py`：验证工具脚本
|
||||
|
||||
- test/
|
||||
- **test/**
|
||||
- `test_generate_readme_opensource.py`：生成工具测试用例
|
||||
- `test_validate_readme_opensource.py`：验证工具测试用例
|
||||
- `test_validate_readme_opensource.py`：工具的测试用例。
|
||||
- **README_OSS.md**：使用文档（本文件）。
|
||||
|
||||
- **README_OSS.md**：使用文档
|
||||
## 工作流程概览
|
||||
|
||||
### 生成工具流程图
|
||||
|
||||
```mermaid
|
||||
sequenceDiagram
|
||||
participant User as 用户
|
||||
participant Script as validate_readme_opensource.py
|
||||
User->>Script: 运行脚本 (--generate)
|
||||
Script-->>User: 请输入输出目录
|
||||
User->>Script: 输入输出目录
|
||||
loop 添加开源部件
|
||||
Script-->>User: 请输入部件信息
|
||||
User->>Script: 输入开源部件信息
|
||||
Script-->>User: 是否添加另一个开源部件？(y/n)
|
||||
User->>Script: y 或 n
|
||||
end
|
||||
Script->>FileSystem: 生成 README.OpenSource 文件
|
||||
Script-->>User: 已生成 README.OpenSource
|
||||
```
|
||||
|
||||
### 验证工具流程图
|
||||
|
||||
```mermaid
|
||||
sequenceDiagram
|
||||
participant User as 用户
|
||||
participant Script as validate_readme_opensource.py
|
||||
User->>Script: 运行脚本 (--validate-format/--validate-content)
|
||||
Script->>FileSystem: 搜索 README.OpenSource 文件
|
||||
alt 找到文件
|
||||
Script->>Script: 验证文件内容
|
||||
alt 验证通过
|
||||
Script-->>User: 文件有效
|
||||
else 验证失败
|
||||
Script-->>User: 文件无效，输出错误信息
|
||||
end
|
||||
else 未找到文件
|
||||
Script-->>User: 未找到 README.OpenSource 文件
|
||||
end
|
||||
```
|
||||
|
||||
@@ -1,8 +1,8 @@
|
||||
# src/validate_readme_opensource.py
|
||||
|
||||
import os
|
||||
import json
|
||||
import argparse
|
||||
import logging
|
||||
from typing import List, Dict, Optional
|
||||
|
||||
REQUIRED_FIELDS = [
|
||||
"Name",
|
||||
@@ -14,50 +14,214 @@ REQUIRED_FIELDS = [
|
||||
"Description"
|
||||
]
|
||||
|
||||
def validate_readme_opensource(file_path):
|
||||
"""
|
||||
验证 README.OpenSource 文件的格式和内容。
|
||||
class OpenSourceValidator:
|
||||
def __init__(
|
||||
self,
|
||||
project_root: str,
|
||||
log_file: Optional[str] = None,
|
||||
reference_data: Optional[List[Dict[str, str]]] = None
|
||||
):
|
||||
self.project_root = project_root
|
||||
self.reference_data = reference_data or []
|
||||
self.log_file = log_file
|
||||
|
||||
:param file_path: README.OpenSource 文件路径
|
||||
:return: 是否验证通过，错误信息列表
|
||||
"""
|
||||
errors = []
|
||||
try:
|
||||
with open(file_path, 'r', encoding='utf-8') as f:
|
||||
data = json.load(f)
|
||||
if not isinstance(data, list):
|
||||
errors.append("The file does not contain a JSON array.")
|
||||
return False, errors
|
||||
# 设置日志配置
|
||||
if self.log_file:
|
||||
logging.basicConfig(
|
||||
filename=self.log_file,
|
||||
level=logging.INFO,
|
||||
format="%(asctime)s - %(levelname)s - %(message)s",
|
||||
)
|
||||
else:
|
||||
logging.basicConfig(
|
||||
level=logging.INFO,
|
||||
format="%(asctime)s - %(levelname)s - %(message)s",
|
||||
)
|
||||
|
||||
for idx, component in enumerate(data):
|
||||
for field in REQUIRED_FIELDS:
|
||||
if field not in component:
|
||||
errors.append(f"Component {idx + 1} is missing required field: {field}")
|
||||
except json.JSONDecodeError as e:
|
||||
errors.append(f"JSON decode error: {e}")
|
||||
return False, errors
|
||||
except Exception as e:
|
||||
errors.append(f"Unexpected error: {e}")
|
||||
return False, errors
|
||||
def find_all_readmes(self) -> List[str]:
|
||||
"""递归查找所有 README.OpenSource 文件"""
|
||||
readme_paths = []
|
||||
for dirpath, _, filenames in os.walk(self.project_root):
|
||||
if "README.OpenSource" in filenames:
|
||||
readme_paths.append(os.path.join(dirpath, "README.OpenSource"))
|
||||
return readme_paths
|
||||
|
||||
def validate_format(self, readme_path: str) -> bool:
|
||||
"""验证 README.OpenSource 文件的格式和必需字段"""
|
||||
errors = []
|
||||
try:
|
||||
with open(readme_path, 'r', encoding='utf-8') as f:
|
||||
data = json.load(f)
|
||||
if not isinstance(data, list):
|
||||
errors.append("The file does not contain a JSON array.")
|
||||
return False
|
||||
for idx, component in enumerate(data):
|
||||
for field in REQUIRED_FIELDS:
|
||||
if field not in component:
|
||||
errors.append(f"Component {idx + 1} is missing required field: {field}")
|
||||
except json.JSONDecodeError as e:
|
||||
errors.append(f"JSON decode error: {e}")
|
||||
return False
|
||||
except Exception as e:
|
||||
errors.append(f"Unexpected error: {e}")
|
||||
return False
|
||||
|
||||
if errors:
|
||||
for error in errors:
|
||||
logging.error(f"{readme_path}: {error}")
|
||||
return False
|
||||
else:
|
||||
logging.info(f"{readme_path} format is valid.")
|
||||
return True
|
||||
|
||||
def load_reference_data(self, reference_data_path: str):
|
||||
"""从 JSON 配置文件中加载参考数据"""
|
||||
try:
|
||||
with open(reference_data_path, "r", encoding='utf-8') as f:
|
||||
self.reference_data = json.load(f)
|
||||
except Exception as e:
|
||||
raise ValueError(
|
||||
f"Failed to load reference data from {reference_data_path}: {e}"
|
||||
)
|
||||
|
||||
def find_reference_data(self, name: str) -> Optional[Dict[str, str]]:
|
||||
"""在参考数据中根据名称查找对应的开源软件信息"""
|
||||
for reference in self.reference_data:
|
||||
if reference.get("Name") == name:
|
||||
return reference
|
||||
return None
|
||||
|
||||
def validate_content(self, readme_path: str) -> bool:
|
||||
"""校验 README.OpenSource 文件的内容，并与参考数据进行比对"""
|
||||
# Step 1: 读取 JSON 文件
|
||||
try:
|
||||
with open(readme_path, "r", encoding='utf-8') as f:
|
||||
readme_data = json.load(f)
|
||||
if not isinstance(readme_data, list):
|
||||
logging.error(f"{readme_path}: JSON data is not an array.")
|
||||
return False
|
||||
except json.JSONDecodeError as e:
|
||||
logging.error(f"{readme_path}: JSON decode error: {e}")
|
||||
return False
|
||||
|
||||
# Step 2: 校验 JSON 数组中的每个开源软件元数据
|
||||
all_valid = True
|
||||
for software_data in readme_data:
|
||||
name = software_data.get("Name")
|
||||
if not name:
|
||||
logging.error(f"{readme_path}: Missing 'Name' field in software data.")
|
||||
all_valid = False
|
||||
continue
|
||||
|
||||
reference_data = self.find_reference_data(name)
|
||||
|
||||
if reference_data is None:
|
||||
logging.error(
|
||||
f"{readme_path}: Software '{name}' not found in reference data."
|
||||
)
|
||||
all_valid = False
|
||||
continue
|
||||
|
||||
# 比对 "Name", "License", "Version Number", "Upstream URL"
|
||||
for field in ["Name", "License", "Version Number", "Upstream URL"]:
|
||||
expected_value = reference_data.get(field)
|
||||
actual_value = software_data.get(field)
|
||||
if actual_value != expected_value:
|
||||
logging.error(
|
||||
f"{readme_path}: Field '{field}' mismatch for '{name}'. Expected: '{expected_value}', Found: '{actual_value}'"
|
||||
)
|
||||
all_valid = False
|
||||
|
||||
# 校验 "License File" 路径是否存在
|
||||
if not self.validate_license_file(readme_path, software_data.get("License File")):
|
||||
all_valid = False
|
||||
|
||||
if all_valid:
|
||||
logging.info(f"{readme_path}: Content validation passed.")
|
||||
else:
|
||||
logging.error(f"{readme_path}: Content validation failed.")
|
||||
return all_valid
|
||||
|
||||
def validate_license_file(self, readme_path: str, license_file: str) -> bool:
|
||||
"""校验 LICENSE 文件是否存在，路径相对于 README.OpenSource 文件所在目录"""
|
||||
if not license_file:
|
||||
logging.error(f"{readme_path}: 'License File' field is missing.")
|
||||
return False
|
||||
|
||||
readme_dir = os.path.dirname(readme_path)
|
||||
license_file_path = os.path.join(readme_dir, license_file)
|
||||
|
||||
if not os.path.exists(license_file_path):
|
||||
logging.error(
|
||||
f"{readme_path}: License file '{license_file}' not found at: {license_file_path}"
|
||||
)
|
||||
return False
|
||||
else:
|
||||
logging.info(f"{readme_path}: License file '{license_file}' exists.")
|
||||
return True
|
||||
|
||||
def run_validation(self, validate_format: bool = True, validate_content: bool = False):
|
||||
"""运行完整的校验流程，递归处理所有 README.OpenSource 文件"""
|
||||
try:
|
||||
readme_paths = self.find_all_readmes()
|
||||
if not readme_paths:
|
||||
logging.error("No README.OpenSource files found in the project directory.")
|
||||
return
|
||||
|
||||
for readme_path in readme_paths:
|
||||
logging.info(f"Validating: {readme_path}")
|
||||
if validate_format:
|
||||
if not self.validate_format(readme_path):
|
||||
logging.error(f"{readme_path}: Format validation failed.")
|
||||
continue # 如果格式验证失败，跳过内容验证
|
||||
if validate_content:
|
||||
if not self.validate_content(readme_path):
|
||||
logging.error(f"{readme_path}: Content validation failed.")
|
||||
|
||||
logging.info("Validation process completed.")
|
||||
|
||||
except Exception as e:
|
||||
logging.error(f"Validation failed: {e}")
|
||||
|
||||
return len(errors) == 0, errors
|
||||
|
||||
def main():
|
||||
parser = argparse.ArgumentParser(description='Validate README.OpenSource files.')
|
||||
parser.add_argument('directory', help='Directory to search for README.OpenSource files.')
|
||||
parser = argparse.ArgumentParser(
|
||||
description="Validate README.OpenSource files in a project."
|
||||
)
|
||||
parser.add_argument("project_root", help="The root directory of the project.")
|
||||
parser.add_argument(
|
||||
"--validate-format", action='store_true', help="Validate the format of README.OpenSource files."
|
||||
)
|
||||
parser.add_argument(
|
||||
"--validate-content", action='store_true', help="Validate the content of README.OpenSource files against reference data."
|
||||
)
|
||||
parser.add_argument(
|
||||
"--reference-data", help="Path to the reference data JSON file (required for content validation)."
|
||||
)
|
||||
parser.add_argument("--log-file", help="Path to the log file for validation results.")
|
||||
|
||||
args = parser.parse_args()
|
||||
|
||||
for root, dirs, files in os.walk(args.directory):
|
||||
if 'README.OpenSource' in files:
|
||||
file_path = os.path.join(root, 'README.OpenSource')
|
||||
valid, errors = validate_readme_opensource(file_path)
|
||||
if valid:
|
||||
print(f"{file_path} is valid.")
|
||||
else:
|
||||
print(f"{file_path} is invalid:")
|
||||
for error in errors:
|
||||
print(f" - {error}")
|
||||
if args.validate_content and not args.reference_data:
|
||||
parser.error("--reference-data is required for content validation.")
|
||||
|
||||
if __name__ == '__main__':
|
||||
# 初始化验证器对象
|
||||
validator = OpenSourceValidator(
|
||||
project_root=args.project_root,
|
||||
log_file=args.log_file
|
||||
)
|
||||
|
||||
if args.validate_content:
|
||||
# 从配置文件中加载参考数据
|
||||
validator.load_reference_data(args.reference_data)
|
||||
|
||||
# 执行校验流程
|
||||
validator.run_validation(
|
||||
validate_format=args.validate_format or not (args.validate_format or args.validate_content),
|
||||
validate_content=args.validate_content
|
||||
)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
|
||||
|
||||
@@ -0,0 +1,14 @@
|
||||
[
|
||||
{
|
||||
"Name": "elfutils",
|
||||
"License": "LGPL-2.1, LGPL-3.0, GPL-2.0",
|
||||
"Version Number": "0.188",
|
||||
"Upstream URL": "https://sourceware.org/elfutils/"
|
||||
},
|
||||
{
|
||||
"Name": "OpenSSL",
|
||||
"License": "Apache-2.0",
|
||||
"Version Number": "3.0.9",
|
||||
"Upstream URL": "https://www.openssl.org/"
|
||||
}
|
||||
]
|
||||
@@ -1,65 +1,168 @@
|
||||
# test/test_validate_readme_opensource.py
|
||||
|
||||
import sys
|
||||
import os
|
||||
sys.path.append(os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
|
||||
|
||||
import unittest
|
||||
import os
|
||||
from unittest.mock import patch, mock_open, MagicMock
|
||||
import json
|
||||
import shutil
|
||||
from src.validate_readme_opensource import validate_readme_opensource
|
||||
import os
|
||||
|
||||
class TestValidateReadmeOpenSource(unittest.TestCase):
|
||||
# 确保导入 OpenSourceValidator 类
|
||||
from src.validate_readme_opensource import OpenSourceValidator
|
||||
|
||||
|
||||
class TestOpenSourceValidator(unittest.TestCase):
|
||||
def setUp(self):
|
||||
self.test_dir = 'test_validate'
|
||||
if not os.path.exists(self.test_dir):
|
||||
os.makedirs(self.test_dir)
|
||||
self.valid_readme = [
|
||||
self.project_root = "/fake/project/root"
|
||||
self.validator = OpenSourceValidator(self.project_root)
|
||||
self.readme_content_valid = json.dumps(
|
||||
[
|
||||
{
|
||||
"Name": "TestLibrary",
|
||||
"License": "MIT",
|
||||
"License File": "LICENSE",
|
||||
"Version Number": "1.0.0",
|
||||
"Owner": "TestOwner",
|
||||
"Upstream URL": "http://example.com",
|
||||
"Description": "A test library.",
|
||||
}
|
||||
]
|
||||
)
|
||||
self.readme_content_missing_fields = json.dumps(
|
||||
[
|
||||
{
|
||||
"Name": "TestLibrary",
|
||||
# Missing 'License'
|
||||
"License File": "LICENSE",
|
||||
"Version Number": "1.0.0",
|
||||
"Owner": "TestOwner",
|
||||
"Upstream URL": "http://example.com",
|
||||
"Description": "A test library.",
|
||||
}
|
||||
]
|
||||
)
|
||||
self.invalid_json_content = "{ invalid json }"
|
||||
self.reference_data = [
|
||||
{
|
||||
"Name": "elfutils",
|
||||
"License": "LGPL-2.1, LGPL-3.0, GPL-2.0",
|
||||
"License File": "COPYING-GPLV2",
|
||||
"Version Number": "0.188",
|
||||
"Owner": "zhanghaibo0@huawei.com",
|
||||
"Upstream URL": "https://sourceware.org/elfutils/",
|
||||
"Description": "A collection of tools and libraries."
|
||||
"Name": "TestLibrary",
|
||||
"License": "MIT",
|
||||
"Version Number": "1.0.0",
|
||||
"Upstream URL": "http://example.com",
|
||||
}
|
||||
]
|
||||
self.invalid_readme = [
|
||||
{
|
||||
"Name": "elfutils",
|
||||
"License": "LGPL-2.1, LGPL-3.0, GPL-2.0",
|
||||
# Missing 'License File'
|
||||
"Version Number": "0.188",
|
||||
"Owner": "zhanghaibo0@huawei.com",
|
||||
"Upstream URL": "https://sourceware.org/elfutils/",
|
||||
"Description": "A collection of tools and libraries."
|
||||
}
|
||||
]
|
||||
self.valid_path = os.path.join(self.test_dir, 'valid_README.OpenSource')
|
||||
self.invalid_path = os.path.join(self.test_dir, 'invalid_README.OpenSource')
|
||||
|
||||
with open(self.valid_path, 'w', encoding='utf-8') as f:
|
||||
json.dump(self.valid_readme, f, indent=2, ensure_ascii=False)
|
||||
@patch("os.walk")
|
||||
@patch("builtins.open", new_callable=mock_open, read_data="")
|
||||
def test_validate_format_valid(self, mock_file, mock_os_walk):
|
||||
# 模拟 os.walk 返回一个 README.OpenSource 文件
|
||||
mock_os_walk.return_value = [(self.project_root, [], ["README.OpenSource"])]
|
||||
# 模拟打开文件并返回有效内容
|
||||
mock_file.return_value.read.return_value = self.readme_content_valid
|
||||
|
||||
with open(self.invalid_path, 'w', encoding='utf-8') as f:
|
||||
json.dump(self.invalid_readme, f, indent=2, ensure_ascii=False)
|
||||
self.validator.run_validation(validate_format=True, validate_content=False)
|
||||
# 如果没有异常，则测试通过
|
||||
|
||||
def tearDown(self):
|
||||
if os.path.exists(self.test_dir):
|
||||
shutil.rmtree(self.test_dir)
|
||||
@patch("os.walk")
|
||||
@patch("builtins.open", new_callable=mock_open, read_data="")
|
||||
def test_validate_format_missing_fields(self, mock_file, mock_os_walk):
|
||||
mock_os_walk.return_value = [(self.project_root, [], ["README.OpenSource"])]
|
||||
mock_file.return_value.read.return_value = self.readme_content_missing_fields
|
||||
|
||||
def test_validate_valid_readme(self):
|
||||
valid, errors = validate_readme_opensource(self.valid_path)
|
||||
self.assertTrue(valid)
|
||||
self.assertEqual(len(errors), 0)
|
||||
self.validator.run_validation(validate_format=True, validate_content=False)
|
||||
# 检查日志或假设如果出现异常，测试将失败
|
||||
|
||||
def test_validate_invalid_readme(self):
|
||||
valid, errors = validate_readme_opensource(self.invalid_path)
|
||||
self.assertFalse(valid)
|
||||
self.assertIn("Component 1 is missing required field: License File", errors)
|
||||
@patch("os.walk")
|
||||
@patch("builtins.open", new_callable=mock_open, read_data="")
|
||||
def test_validate_format_invalid_json(self, mock_file, mock_os_walk):
|
||||
mock_os_walk.return_value = [(self.project_root, [], ["README.OpenSource"])]
|
||||
mock_file.return_value.read.return_value = self.invalid_json_content
|
||||
|
||||
if __name__ == '__main__':
|
||||
self.validator.run_validation(validate_format=True, validate_content=False)
|
||||
# 检查是否正确处理 JSON 解码错误
|
||||
|
||||
@patch("os.walk")
|
||||
@patch("builtins.open", new_callable=mock_open)
|
||||
def test_validate_content_valid(self, mock_open_builtin, mock_os_walk):
|
||||
# 模拟 os.walk 返回一个 README.OpenSource 文件
|
||||
mock_os_walk.return_value = [(self.project_root, [], ["README.OpenSource"])]
|
||||
|
||||
# 模拟打开文件并返回有效内容
|
||||
def side_effect_open(file, mode="r", encoding=None):
|
||||
if "README.OpenSource" in file:
|
||||
return mock_open(read_data=self.readme_content_valid)()
|
||||
elif "reference_data.json" in file:
|
||||
return mock_open(read_data=json.dumps(self.reference_data))()
|
||||
else:
|
||||
raise FileNotFoundError
|
||||
|
||||
mock_open_builtin.side_effect = side_effect_open
|
||||
|
||||
# 初始化验证器并加载参考数据
|
||||
self.validator.reference_data = self.reference_data
|
||||
|
||||
# 模拟 os.path.exists 返回 True（表示许可证文件存在）
|
||||
with patch("os.path.exists", return_value=True):
|
||||
self.validator.run_validation(validate_format=False, validate_content=True)
|
||||
# 如果没有异常，则测试通过
|
||||
|
||||
@patch("os.walk")
|
||||
@patch("builtins.open", new_callable=mock_open)
|
||||
def test_validate_content_license_file_missing(
|
||||
self, mock_open_builtin, mock_os_walk
|
||||
):
|
||||
# 与上一个测试类似
|
||||
mock_os_walk.return_value = [(self.project_root, [], ["README.OpenSource"])]
|
||||
|
||||
def side_effect_open(file, mode="r", encoding=None):
|
||||
if "README.OpenSource" in file:
|
||||
return mock_open(read_data=self.readme_content_valid)()
|
||||
elif "reference_data.json" in file:
|
||||
return mock_open(read_data=json.dumps(self.reference_data))()
|
||||
else:
|
||||
raise FileNotFoundError
|
||||
|
||||
mock_open_builtin.side_effect = side_effect_open
|
||||
|
||||
self.validator.reference_data = self.reference_data
|
||||
|
||||
# 模拟 os.path.exists 返回 False（表示许可证文件不存在）
|
||||
with patch("os.path.exists", return_value=False):
|
||||
self.validator.run_validation(validate_format=False, validate_content=True)
|
||||
# 检查是否正确处理许可证文件缺失
|
||||
|
||||
@patch("os.walk")
|
||||
@patch("builtins.open", new_callable=mock_open)
|
||||
def test_validate_content_field_mismatch(self, mock_open_builtin, mock_os_walk):
|
||||
# 修改 README 内容以包含不匹配的字段
|
||||
readme_content_mismatch = json.dumps(
|
||||
[
|
||||
{
|
||||
"Name": "TestLibrary",
|
||||
"License": "Apache-2.0", # 应为 'MIT'
|
||||
"License File": "LICENSE",
|
||||
"Version Number": "1.0.0",
|
||||
"Owner": "TestOwner",
|
||||
"Upstream URL": "http://example.com",
|
||||
"Description": "A test library.",
|
||||
}
|
||||
]
|
||||
)
|
||||
|
||||
mock_os_walk.return_value = [(self.project_root, [], ["README.OpenSource"])]
|
||||
|
||||
def side_effect_open(file, mode="r", encoding=None):
|
||||
if "README.OpenSource" in file:
|
||||
return mock_open(read_data=readme_content_mismatch)()
|
||||
elif "reference_data.json" in file:
|
||||
return mock_open(read_data=json.dumps(self.reference_data))()
|
||||
else:
|
||||
raise FileNotFoundError
|
||||
|
||||
mock_open_builtin.side_effect = side_effect_open
|
||||
|
||||
self.validator.reference_data = self.reference_data
|
||||
|
||||
# 模拟 os.path.exists 返回 True
|
||||
with patch("os.path.exists", return_value=True):
|
||||
self.validator.run_validation(validate_format=False, validate_content=True)
|
||||
# 检查是否正确处理字段不匹配
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
|
||||
Reference in New Issue
Block a user