langgenius/dify
1
0
Fork 0
mirror of https://github.com/langgenius/dify.git synced 2026-07-22 20:45:26 -04:00
Code Issues 775 Packages Projects Releases 10 Wiki Activity

[Chore/Refactor] #19088

New Issue
Closed
opened 2026-02-21 19:55:32 -05:00 by yindo · 0 comments
yindo commented 2026-02-21 19:55:32 -05:00
Owner
Copy Link
Copy Source

Originally created by @Enriquejaja on GitHub (Oct 11, 2025).

Self Checks

  • I have read the Contributing Guide and Language Policy.
  • This is only for refactoring, if you would like to ask a question, please head to Discussions.
  • I have searched for existing issues search for existing issues, including closed ones.
  • I confirm that I am using English to submit this report, otherwise it will be closed.
  • 【中文用户 & Non English User】请使用英语提交，否则会被关闭 ：）
  • Please do not modify this template :) and fill in all the required fields.

Description

When running Python in the code execution module, an error message appears because the BeautifulSoup 4 library is not installed in the sandbox.

Image

Motivation

The code is as follows:

pip install beautifulsoup4

import json
import re
from bs4 import BeautifulSoup

您提供的 HTML 内容

html_content = url

def parse_hacker_news(html):
"""
解析Hacker News HTML内容并提取新闻条目。
"""
soup = BeautifulSoup(html, 'lxml')
news_items = []

# 查找所有包含新闻信息的 'athing' 行
item_rows = soup.select('tr.athing.submission')

for item_row in item_rows:
    # 获取紧随其后的元数据行 (包含分数、作者等)
    metadata_row = item_row.find_next_sibling('tr')
    
    # --- 提取主要信息 ---
    rank_tag = item_row.select_one('span.rank')
    title_tag = item_row.select_one('span.titleline > a')
    site_tag = item_row.select_one('span.sitestr')
    
    item_id = int(item_row.get('id', 0))
    rank = int(rank_tag.text.strip('.')) if rank_tag else None
    title = title_tag.text if title_tag else None
    url = title_tag.get('href') if title_tag else None
    site = site_tag.text if site_tag else None
    
    # --- 提取元数据信息 ---
    score = None
    author = None
    age = None
    comments_count = None
    
    if metadata_row:
        score_tag = metadata_row.select_one('span.score')
        author_tag = metadata_row.select_one('a.hnuser')
        age_tag = metadata_row.select_one('span.age > a')
        
        # 评论数是 subline 中的最后一个 a 标签
        comment_link = metadata_row.select('.subtext > span.subline > a:last-of-type')

        if score_tag:
            # 使用正则表达式从 '123 分' 中提取数字
            score_match = re.search(r'\d+', score_tag.text)
            if score_match:
                score = int(score_match.group(0))
        
        author = author_tag.text if author_tag else None
        age = age_tag.text if age_tag else None
        
        if comment_link:
            comment_text = comment_link[0].text
            if '评论' in comment_text or 'comment' in comment_text:
                #   会被 BeautifulSoup 转换为空格
                comment_match = re.search(r'\d+', comment_text.replace('\xa0', ' '))
                if comment_match:
                    comments_count = int(comment_match.group(0))
            # 如果文本是“讨论”或“discuss”，则评论数为0
            elif '讨论' in comment_text or 'discuss' in comment_text.lower():
                 comments_count = 0
        
    news_items.append({
        "id": item_id,
        "rank": rank,
        "title": title,
        "url": url,
        "site": site,
        "score": score,
        "author": author,
        "age": age,
        "comments_count": comments_count,
    })
    
return {"news_items": news_items}

--- 主程序 ---

if name == "main":
parsed_data = parse_hacker_news(html_content)

# 将结果转换为格式化的JSON字符串
# ensure_ascii=False 确保中文字符能正确显示
json_output = json.dumps(parsed_data, indent=2, ensure_ascii=False)

print(json_output)

Additional Context

Searched and checked, no previous solutions found

Originally created by @Enriquejaja on GitHub (Oct 11, 2025). ### Self Checks - [x] I have read the [Contributing Guide](https://github.com/langgenius/dify/blob/main/CONTRIBUTING.md) and [Language Policy](https://github.com/langgenius/dify/issues/1542). - [x] This is only for refactoring, if you would like to ask a question, please head to [Discussions](https://github.com/langgenius/dify/discussions/categories/general). - [x] I have searched for existing issues [search for existing issues](https://github.com/langgenius/dify/issues), including closed ones. - [x] I confirm that I am using English to submit this report, otherwise it will be closed. - [x] 【中文用户 & Non English User】请使用英语提交，否则会被关闭 ：） - [x] Please do not modify this template :) and fill in all the required fields. ### Description When running Python in the code execution module, an error message appears because the BeautifulSoup 4 library is not installed in the sandbox. <img width="2266" height="1250" alt="Image" src="https://github.com/user-attachments/assets/e573946b-e282-41e1-a913-77cd06a4dee9" /> ### Motivation The code is as follows: pip install beautifulsoup4 import json import re from bs4 import BeautifulSoup # 您提供的 HTML 内容 html_content = url def parse_hacker_news(html): """ 解析Hacker News HTML内容并提取新闻条目。 """ soup = BeautifulSoup(html, 'lxml') news_items = [] # 查找所有包含新闻信息的 'athing' 行 item_rows = soup.select('tr.athing.submission') for item_row in item_rows: # 获取紧随其后的元数据行 (包含分数、作者等) metadata_row = item_row.find_next_sibling('tr') # --- 提取主要信息 --- rank_tag = item_row.select_one('span.rank') title_tag = item_row.select_one('span.titleline > a') site_tag = item_row.select_one('span.sitestr') item_id = int(item_row.get('id', 0)) rank = int(rank_tag.text.strip('.')) if rank_tag else None title = title_tag.text if title_tag else None url = title_tag.get('href') if title_tag else None site = site_tag.text if site_tag else None # --- 提取元数据信息 --- score = None author = None age = None comments_count = None if metadata_row: score_tag = metadata_row.select_one('span.score') author_tag = metadata_row.select_one('a.hnuser') age_tag = metadata_row.select_one('span.age > a') # 评论数是 subline 中的最后一个 a 标签 comment_link = metadata_row.select('.subtext > span.subline > a:last-of-type') if score_tag: # 使用正则表达式从 '123 分' 中提取数字 score_match = re.search(r'\d+', score_tag.text) if score_match: score = int(score_match.group(0)) author = author_tag.text if author_tag else None age = age_tag.text if age_tag else None if comment_link: comment_text = comment_link[0].text if '评论' in comment_text or 'comment' in comment_text: # &nbsp; 会被 BeautifulSoup 转换为空格 comment_match = re.search(r'\d+', comment_text.replace('\xa0', ' ')) if comment_match: comments_count = int(comment_match.group(0)) # 如果文本是“讨论”或“discuss”，则评论数为0 elif '讨论' in comment_text or 'discuss' in comment_text.lower(): comments_count = 0 news_items.append({ "id": item_id, "rank": rank, "title": title, "url": url, "site": site, "score": score, "author": author, "age": age, "comments_count": comments_count, }) return {"news_items": news_items} # --- 主程序 --- if __name__ == "__main__": parsed_data = parse_hacker_news(html_content) # 将结果转换为格式化的JSON字符串 # ensure_ascii=False 确保中文字符能正确显示 json_output = json.dumps(parsed_data, indent=2, ensure_ascii=False) print(json_output) ### Additional Context Searched and checked, no previous solutions found <!-- Failed to upload "CleanShot 2025-10-11 at 20.51.04.png" --> <!-- Failed to upload "CleanShot 2025-10-11 at 20.48.28.png" -->
yindo added the refactor label 2026-02-21 19:55:32 -05:00
yindo closed this issue 2026-02-21 19:55:33 -05:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Knowledge Pipeline
Proposal
WIP
cloud
dependencies
feat:webapp
good first issue
outdated
performance
pull-request
Mirrored from GitHub Pull Request
 python:uv
refactor
size:L
stale
status: accepting prs
status: blocked
status: in discussion
stt & tts
vulnerablity
web
wont-fix
🌊 feat:workflow
🌚 invalid
🐞 bug
👻 feat:rag
💪 enhancement
📚 documentation
🔨 feat:tools
🙋‍♂️ question
🙏 help wanted
🤔 cant-reproduce
🤖 feat:agent
No Label refactor
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: langgenius/dify#19088
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?