Mintplex-Labs/anything-llm
1
0
Fork 0
mirror of https://github.com/Mintplex-Labs/anything-llm.git synced 2026-07-19 14:13:35 -04:00
Code Issues 328 Packages Projects Releases 10 Wiki Activity

[GH-ISSUE #5713] [FEAT]: #5253

New Issue
Closed
opened 2026-06-05 14:52:56 -04:00 by yindo · 3 comments
yindo commented 2026-06-05 14:52:56 -04:00
Owner
Copy Link
Copy Source

Originally created by @Samuelyilma on GitHub (May 27, 2026).
Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/5713

What would you like to see?

Do you think it would be possible to extend the AI’s capabilities so that it can interact with desktop applications as well? If we could explore a design where the AI can perform actions like opening, editing, or controlling apps, I think that could really broaden the tool’s potential

Originally created by @Samuelyilma on GitHub (May 27, 2026). Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/5713 ### What would you like to see? Do you think it would be possible to extend the AI’s capabilities so that it can interact with desktop applications as well? If we could explore a design where the AI can perform actions like opening, editing, or controlling apps, I think that could really broaden the tool’s potential
yindo added the enhancementfeature request labels 2026-06-05 14:52:56 -04:00
yindo closed this issue 2026-06-05 14:52:56 -04:00
yindo commented 2026-06-05 14:52:57 -04:00
Author
Owner
Copy Link
Copy Source

@timothycarambat commented on GitHub (May 27, 2026):

Yes, of course, but eventually you would wind up with what is functionally computer use which for OSS is..well not quite there yet.

However, we did an experiment once for basically what amounts to RPA (Remote Process Automation) with a bit of AI to fill in the blanks so at least we could navigate and click around GUI apps to a degree and run automations.

This had benefits like:

  • Repeatable process
  • Granular controls on action points (like dynamically filling out a field or something)

But obvious downsides like:

  • You need to "train" the flow at least once
  • It would make the computer unusable while it runs since it needs to control the mouse/keyboard while running.

This ultimately never shipped though as we just didnt see the big use case. However it seems to become more apparent now since computer-use has made very little real progress compared to other segments of AI and it does seem like there is an audience for it. The code still exists, it is just super out of date.

<!-- gh-comment-id:4556058807 --> @timothycarambat commented on GitHub (May 27, 2026): Yes, of course, but eventually you would wind up with what is functionally [computer use](https://github.com/trycua/acu) which for OSS is..well not quite there yet. However, we did an experiment once for basically what amounts to RPA (Remote Process Automation) with a bit of AI to fill in the blanks so at least we could navigate and click around GUI apps to a degree and run automations. This had benefits like: - Repeatable process - Granular controls on action points (like dynamically filling out a field or something) But obvious downsides like: - You need to "train" the flow at least once - It would make the computer unusable while it runs since it needs to control the mouse/keyboard while running. This ultimately never shipped though as we just didnt see the big use case. However it seems to become more apparent now since computer-use has made very little real progress compared to other segments of AI and it does seem like there is an audience for it. The code still exists, it is just super out of date.
yindo commented 2026-06-05 14:52:57 -04:00
Author
Owner
Copy Link
Copy Source

@Samuelyilma commented on GitHub (May 27, 2026):

i was curious if you have any plans to bring back the computer use functionality in the near future and what kind of things should we expect to see in the next update

<!-- gh-comment-id:4556899101 --> @Samuelyilma commented on GitHub (May 27, 2026): i was curious if you have any plans to bring back the computer use functionality in the near future and what kind of things should we expect to see in the next update
yindo commented 2026-06-05 14:52:57 -04:00
Author
Owner
Copy Link
Copy Source

@timothycarambat commented on GitHub (May 27, 2026):

Image

It is still there! Its where it has always been, under Experimental features (since support is so limited)

<!-- gh-comment-id:4557561050 --> @timothycarambat commented on GitHub (May 27, 2026): <img width="1713" height="982" alt="Image" src="https://github.com/user-attachments/assets/6bb27079-e5b4-4dbf-8396-bcebb9a7bd95" /> It is still there! Its where it has always been, under Experimental features (since support is so limited)
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Desktop
Docker
Integration Request
Integration Request
OS: Linux
OS: Mobile
OS: Windows
UI/UX
blocked
bug
bug
core-team-only
documentation
duplicate
embed-widget
enhancement
feature request
github_actions
good first issue
investigating
needs info / can't replicate
possible bug
pull-request
Mirrored from GitHub Pull Request
 question
stage: specifications
wontfix
No Label enhancement feature request
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: Mintplex-Labs/anything-llm#5253
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?