Launch HN: Skyvern (YC S23) – open-source AI agent for browser automations

Congrats on the launch! I've been keeping up with you folks since you last posted (a few months ago, I believe). How does Anthropic's recent announcement of Claude's "computer use" abilities grab you? What key differentiators does Skyvern have, at this point in time ("computer use" with Claude being relatively new)?

Probably not the first AI wrapper around Playwright this week, and certainly not the first this month.

I think this use case of automation in a BPA sense is more compelling than using it for test automation, because the latter is much more concerned with the precision and repeatability of the process. For the BPA task, arguably you care only about the outcome and it often doesn't matter if it gets there via some crazy route.

Part of the problem for me is that your example video shows a big wodge of prompt that had to be written to make this work and then a few kb of payload data (parameters) in a plaintext, non-csv format. If the expectation is that this replaces someone just using Playwright with codegen due to that being too technical, I'm not convinced there is a huge group of people who can manage one task but not the other.

Furthermore, you are expecting them to pass over their website login credentials and apparently their credit card details too, in plain text. You had better have a very solid idea of how to handle that sensitive data to avoid serious consequences if your users' skyvern accounts are compromised.

I think the frequency of website redesigns is oversold by people producing these LLM-driven Playwright wrappers, especially when targeting old-fashioned or government sites. As an example, we have had a suite of lengthy Playwright browser automations to interact with a government site for a few years and have had to maintain them only once, when the agency's business process changed. The prompt would also have needed to change had we used Skyvern, as would the payload, because the process was different. The difference with the Playwright automation, though, is that we could use assertions to verify steps had succeeded/failed and data had been recorded correctly, so we would know the process needed updating. I can't see that option in Skyvern which would have me worrying that process changes would be overlooked and we would unknowingly start entering the wrong data or missing steps.

Anyone building a start-up on 3rd party LLMs at this point has to have some big cajones. Or you need a smash-and-grab business model. Serious risk if your horizon is measured in years instead of months.

Anthropic threw their hat in this ring yesterday, and it will very likely be followed by OpenAI and Google soon. Godspeed.

Congrats!!! And super cool that you've open sourced it under the AGPL. Sorry if this is answered in the docs but I did a brief search on the source and noticed you're not using LangChain but do plan to integrate it so it can be offered to that community. I'm curious if you wouldn't mind talking about what you did use to create the chain of thought/actions logic in Skyvern and if you had to start work today if you'd consider going the LangChain/Graph route? Thanks.

In case anyone else is confused as to what "browser automations" is : this is about making a program that drives a target web site (owned by someone else typically), in the manner of selenium or the like --- inserting key press events and mouse move/click events, to make that target web site do something. Once you know that the rest of the description makes sense.

As with any of these LLM workflow automation tools, it raises a few questions about each potential use case, and the likely long-term outcomes.

1. Is this working around friction due to a lack of interoperability between tools? For example, is this something that would be more efficient if the owner of the website exposed a REST service? Will the existence of this tool disincentivize companies from exposing services when it makes sense?

2. If there is a good reason for the lack of a service endpoint, perhaps for security reasons, will your automation workflow be used to bypass those security measures? Could your tool be used by malicious actors to disable major services? Are you that malicious actor yourself? Will your tool be used by scalpers to prevent consumers from buying high-demand products?

3. If this is being used to work around deferred maintenance with internal tools and processes, will the existence of these kind of tools be used by management to justify further deferral of that maintenance? Will your tool become a critical piece of the support staff's workflow?

4. If your tool is being used in good faith to work around anti-patterns in website design, will the owner of the website be incentivized to break your workflow? Is your use case just a step in an arms race?

These are the thoughts that go through my head whenever I hear about software being laid on top of complicated processes, where instead of simplifying the underlying processes, we add another layer of complexity to sweep it under the rug. I'm sure that people will find your project useful, but I wonder what the longer-term effects will be.

Is Skyvern able to scrape data from multiple websites with different structures and combine this data into structured data in one CSV or JSON file? Example: scrape interest rates offered on savings accounts from multiple bank websites and extract the name of the bank, bank logo, product name and interest rate for each account and run this saved query on a regular schedule (daily, weekly etc)?

Congrats on the launch. I'm curious if you had any experience running skyvern on airline websites (for example to extract award availability for miles tickets from point A to B)? It seems like airlines always change things around and have robust anti scraping measures.

Awesome, I've been working on a similar thing at a smaller scale and I think this area is very promising.

I've limited my problem scope to single page interactions / scraping which has been very reliable and useful for my company. But agentic automation does sound fun.

Congrats! Do you have numbers on WebArena (https://webarena.dev) or VisualWebArena (https://jykoh.com/vwa)?

Congrats on the launch! This is really cool - one of the applications of LLM I find most compelling. I've seen so many back office processes that have hundreds of steps, are incredibly error prone, and traditionally couldn't be automated due to API limitations. Solutions like Skyvern are going to supercharge businesses that have had historically low margins due to the number of humans required. (Not as a replacement for a human, but as a force multiplier)

Hi, looks cool! Congratulations. Will check it out and maybe add it to https://ai-tools.directory for people looking for such solutions!

This is great, and I can think of several business uses and some personal.

Like this: Could I use this to pull screenshots or PDFs of my grocery receipts from a major grocery chain?

The plaintext version of your signup email replaces the ampersand in the url with an & XML entity. You probably don't want that.

> You won't be able to run Skyvern unless you enable at least one provider.

Any plans on bundling a local LLM / supporting local LLMs?

awesome work. I had the github starred from the day I saw on Show HN but never got around to using it.

I want to use this to automate approving/declining group members for our facebook group which is approaching half million members and fb admin tools are pretty lacking

> how can I fill out a contact-us form on hundreds of different websites?

What's the use case here exactly? Sorry for being a bit pessimistic, but this sounds like an easy way to automatically send a lot of spam.

Congratulations on the launch. This is really cool. I was recently tinkering with the same idea. But based on a browser extension.

There are many back office tasks where people copy data from page 1 into a form of page 2.

Looks super interesting!

Unfortunately the mobile experience is pretty bad - practically unusable. I'd expect any web application made in the last decade to be mobile-first.

Sounds good.

Question, if it's computer vision based, does that mean that it can be trivially ported to support desktop automations?

Do I understand correctly that this is an open source of the GUI only, you don't show the model itself?

Cool but pricing is utterly insane

Would be great to have a fixed blockchain based event log, ideally encrypted.

Quick question: What does DataDog's ddtrace do in the opensource version?

Can I use this to make changes to a Wordpress website if given login?

This is very cool. Do you think I could use to do UX/UI testing?

Has someone run this on modal.com yet?

Congrats on the launch - love the tool

But will Cloudflare brick it?

Congrats!! This is super neat. I've been looking for good ways to have AI browse the internet on my behalf - the way I normally do, and give me a presentation / summary of the highlights, so that I don't have to open myself up as much to social media and the chance for doomscrolling, etc.

I'm going to be playing with this.

Wait until the media gets wind of what the industries been doing this fall… a whole repo on using AI to autonomously use other people’s websites, and not a single paragraph on safety — for the websites or for us. Technically incredible ofc, and it’s a beautiful repo. I wish it didn’t make so anxious.