langgenius/dify
1
0
Fork 0
mirror of https://github.com/langgenius/dify.git synced 2026-07-21 20:15:31 -04:00
Code Issues 775 Packages Projects Releases 10 Wiki Activity

New Tool: Screenpipe for screen/audio context in agents #21882

New Issue
Open
opened 2026-02-21 20:14:42 -05:00 by yindo · 0 comments
yindo commented 2026-02-21 20:14:42 -05:00
Owner
Copy Link
Copy Source

Originally created by @louis030195 on GitHub (Jan 25, 2026).

Feature Request: Screenpipe Tool

Add Screenpipe as a built-in tool in Dify!

What is Screenpipe?

Screenpipe (16k+ GitHub stars) is a 24/7 local screen & mic recording tool:

  • Screen OCR (Apple Vision, Windows OCR, Tesseract)
  • Audio transcription (Whisper)
  • Local SQLite storage
  • REST API

Why Add to Dify?

Dify agents lack context about what users are actually doing. Screenpipe provides:

  • Visual context: What's on screen
  • Audio context: Meeting transcriptions
  • Historical context: Past activity

Proposed Tool

Screenpipe Search

  • Description: Search user's screen recordings and audio transcriptions
  • Parameters: query, content_type, time_range
  • Returns: Relevant content with timestamps

Use Cases

  1. Context-aware chatbot: "What was that error message?"
  2. Meeting assistant: "Summarize my last meeting"
  3. Research helper: "Find the article I was reading about X"

API

GET http://localhost:3030/search?q=query&content_type=all

Happy to contribute the implementation!

Links:

Originally created by @louis030195 on GitHub (Jan 25, 2026). ## Feature Request: Screenpipe Tool Add **[Screenpipe](https://github.com/mediar-ai/screenpipe)** as a built-in tool in Dify! ## What is Screenpipe? Screenpipe (16k+ GitHub stars) is a 24/7 local screen & mic recording tool: - Screen OCR (Apple Vision, Windows OCR, Tesseract) - Audio transcription (Whisper) - Local SQLite storage - REST API ## Why Add to Dify? Dify agents lack context about what users are actually doing. Screenpipe provides: - **Visual context**: What's on screen - **Audio context**: Meeting transcriptions - **Historical context**: Past activity ## Proposed Tool **Screenpipe Search** - Description: Search user's screen recordings and audio transcriptions - Parameters: query, content_type, time_range - Returns: Relevant content with timestamps ## Use Cases 1. **Context-aware chatbot**: "What was that error message?" 2. **Meeting assistant**: "Summarize my last meeting" 3. **Research helper**: "Find the article I was reading about X" ## API ``` GET http://localhost:3030/search?q=query&content_type=all ``` Happy to contribute the implementation! **Links:** - GitHub: https://github.com/mediar-ai/screenpipe - TypeScript SDK: https://www.npmjs.com/package/@screenpipe/js
yindo added the 💪 enhancement🤖 feat:agent labels 2026-02-21 20:14:42 -05:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Knowledge Pipeline
Proposal
WIP
cloud
dependencies
feat:webapp
good first issue
outdated
performance
pull-request
Mirrored from GitHub Pull Request
 python:uv
refactor
size:L
stale
status: accepting prs
status: blocked
status: in discussion
stt & tts
vulnerablity
web
wont-fix
🌊 feat:workflow
🌚 invalid
🐞 bug
👻 feat:rag
💪 enhancement
📚 documentation
🔨 feat:tools
🙋‍♂️ question
🙏 help wanted
🤔 cant-reproduce
🤖 feat:agent
No Label 💪 enhancement 🤖 feat:agent
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: langgenius/dify#21882
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?