Show HN: a Rust lib to trigger actions based on your screen activity (with LLMs)
The GitHub project "Screen Pipe" uses Large Language Models to convert screen content into actions. Implemented in Rust + WASM, inspired by `adept.ai`, `rewind.ai`, and `Apple Shortcut`. Open source under MIT license.
Read original articleThe GitHub project "Screen Pipe" focuses on utilizing Large Language Models to turn screen content into actions. It is implemented in Rust + WASM and draws inspiration from various sources like `adept.ai`, `rewind.ai`, and `Apple Shortcut`. The tool enables screen capture and text extraction, showcasing a TypeScript server-side example using a Large Language Model for analyzing sales conversations. Currently in the Alpha stage, the project operates on the creator's computer, capturing screen elements to trigger actions. To test the tool, users can clone the repository and follow the instructions in the README. The project is open source under the MIT license, welcoming contributions through forks and pull requests. Additionally, a public Discord channel is available for community engagement and collaboration on related projects. More details and access to the project can be found on the GitHub repository for "Screen Pipe."
Related
AI-powered conversion from Enzyme to React Testing Library
Slack engineers transitioned from Enzyme to React Testing Library due to React 18 compatibility issues. They used AST transformations and LLMs for automated conversion, achieving an 80% success rate.
React Lua
The GitHub repository focuses on translating ReactJS 17.x into Lua for Roblox and Lua community. It aims for performance and accuracy, welcoming contributions from notable developers like Paul Doyle and Matt Hargett.
Sloth search for Ruby Weekly – a 100 minute hack turned 20h open sauce project
Sloth Finder, a Ruby and Rails tool, curates niche articles on API and automation. It emphasizes simplicity, slow loading times, and plans to upgrade its tech stack for efficiency. Open source on GitHub.
Update on GNOME Newton, the Wayland-native accessibility project
The Newton project aims to enhance accessibility on free desktops, focusing on Wayland-protocols, AccessKit, Mutter, Orca, GTK, and libadwaita components. Progress includes running GTK 4 apps in Flatpak sandboxes, with ongoing work to improve Orca functionality on Wayland. Challenges involve optimizing performance and addressing architectural issues for assistive technologies. Future plans may extend support to GNOME Shell UI.
My weekend project turned into a 3 years journey
Anthony's note-taking app journey spans 3 years, evolving from a secure Markdown tool to a complex Electron/React project with code execution capabilities. Facing challenges in store publishing, he prioritizes user feedback and simplicity, opting for a custom online deployment solution.
For the time being it doesn’t do much other than intercepting the input/output, and is a bit buggy at that - so, no LLM there :)
But posting anyhow in case it can be useful for someone to avoid reinventing the plumbing.
EDIT: To make it clear, my point is that the description could have been clearer, especially regarding the Python dependency.
Related
AI-powered conversion from Enzyme to React Testing Library
Slack engineers transitioned from Enzyme to React Testing Library due to React 18 compatibility issues. They used AST transformations and LLMs for automated conversion, achieving an 80% success rate.
React Lua
The GitHub repository focuses on translating ReactJS 17.x into Lua for Roblox and Lua community. It aims for performance and accuracy, welcoming contributions from notable developers like Paul Doyle and Matt Hargett.
Sloth search for Ruby Weekly – a 100 minute hack turned 20h open sauce project
Sloth Finder, a Ruby and Rails tool, curates niche articles on API and automation. It emphasizes simplicity, slow loading times, and plans to upgrade its tech stack for efficiency. Open source on GitHub.
Update on GNOME Newton, the Wayland-native accessibility project
The Newton project aims to enhance accessibility on free desktops, focusing on Wayland-protocols, AccessKit, Mutter, Orca, GTK, and libadwaita components. Progress includes running GTK 4 apps in Flatpak sandboxes, with ongoing work to improve Orca functionality on Wayland. Challenges involve optimizing performance and addressing architectural issues for assistive technologies. Future plans may extend support to GNOME Shell UI.
My weekend project turned into a 3 years journey
Anthony's note-taking app journey spans 3 years, evolving from a secure Markdown tool to a complex Electron/React project with code execution capabilities. Facing challenges in store publishing, he prioritizes user feedback and simplicity, opting for a custom online deployment solution.