An "oh fuck" moment in time

This is a tired trope, that engineers who don’t use software assistants will be left in the dust and need to hop onboard asap.

It’s really easy to learn how to use software assistants. Those engineers can hop on or off any time. It’s really hard to build up fundamental engineering skills.

The problem with Windsurf is that it works great until it doesn't. And the time you spend checking and fixing its mistakes is longer than it would take for you to implement that yourself.

Some of those cases are:

* Sometimes the Accept changes button doesn't work

* Sometimes it DELETES a lot of your code, without you noticing

* Sometimes it duplicates code

* Sometimes it gets stuck in a loop trying to fix an error, or it can't fix trivial errors

* Sometimes it suddenly can not access your files.

* Sometimes it adds "// ...rest of code goes here" comments instead of writing the code.

In my experience, the best use-cases for AI assistants/LLMs are:

* Generate boilerplate/scaffold for a new project

* Refactor code (rename variables, functions, or slightly alter functionality like use "while loop instead of for")

* Quick way to find the name/signature of an API/function without having to google the docs of that library (e.g. "add a new React MaterialUI list")

I have used it for about 2 months, but in my experience it actually gets worse with time, not better. The problem now doesn't seem to be with the actual writing of code, but with the UI and how it applies the changes.

My experience is sometimes it works amazingly well like you say, and sometimes it doesn't.

LLMs are great at translations, like "convert this SQL to JooQ" or "convert this Chinese text to English". If you were going from Haskell -> Rust it might have trouble fighting with the borrow checker, or it might not if your memory usage patterns were straightforward or you can "RC all the things". Haskell's a particularly good target because less state means less complexity thinking about state.

> Then took my kids down to the local pool. When I got back, I had a fully functional Haskell library with autogenerated C bindings, FFI to CoreAudio and my jaw was on the ground.

With a tricky copyright declaration and licensing.

A mechanical translation of Rust code under the copyright of ___?

Maybe also involving stealing bits of Haskell code without attribution from material copyrights of ___, ___, and ___?

Cleanroom engineering is a thing when it comes to "make me one like that, except I have copyright of it". The LLM laundering approach isn't that.

Ripping off open source code isn't new. Various companies and individuals would do it. Occasionally they got caught.

Microsoft/GitHub had to reassure companies that cared about LLM copyright violations, by indemnifying them. I guess so that open source projects ripped off would have to face the legal resources of Microsoft.

Making a wrapper is commodity knowledge (there is a ton of wrappers out there as examples, tons of literature about it). It's more about hard work than intelligence.

Eventually, AI (as it is today) will run out of commoditizable knowledge.

It is impressive, but there is a man inside the machine writing wrappers and talking about them to allow this magic to happen.

Dunno, man. I'm using Claude 3.5 via Cody on IntelliJ for work, or via Kagi for other cases. When it works, it works great - especially for easy, well-understood things like boilerplate code or the typical simple tasks that I don't bother memorizing because they were already available in search engines anyway.

For anything more complex or esoteric, LLMs shit the bed pretty quickly. A recent example: I needed to implement a particular setup for an Argo CD ApplicationSet (default values plus a per-stage override, at the generator level). Every single suggestion by the LLM was wrong. I decided to dig into the documentation because I assumed there was no way this wasn't possible, and lo and behold - matrix generators are a thing, and my exact use case was an example in the doc. I asked the LLM about it and it knew what they were, so it clearly had indexed it before.

It's far from the only issue - the most common one is the below-average quality of code in some cases, with plenty of wrong implementations strictly due to hallucinations. Then Cody has its own issues on top of that - not applying changes correctly is the most common one. Put that together with engineers that are in a rush or are using the LLM as a crutch... and I can see a future where devs with little experience can put together huge codebases that will be as brittle as a house of cards.

> convert this rust library

…

> This wasn't something that existed, it wasn't regurgitating knowledge from Stackoverflow. It was inventing/creating something new.

But it was something that existed… it took a rust library and converted it to haskell. That’s not invention, that’s conversion

What adaptation is actually necessary for competent software developers? If it is as easy as writing what you want the llm to do, I'm pretty sure any competent software developer can learn that within a few days. And the necessity of precise prompting techniques will quickly diminish over time.

> It's currently really hard for newbies to break into the software industry as it's still recovering from the en-mass layoffs two years ago. With coding assistants this good, now, and getting better daily I'm left wondering...

This can definitely help with some of the tech debts. But I wonder how do we deal with changing requirements? Use Cases? Experiments & new directions from Product? Software has a life after building.

If the output is so amazing why leave out a link to the generated library?

Why leave out an explanation of why you needed, or how you’ll use, this Haskell audio api wrapper?

In the absence of these this just feels like resume slop.

If it is, do recruiters see “I asked a thing to do the work for me” as desirable in a potential hire?

The original library’s license is Apache 2.0. Does that have any bearing on the authors opinion on sharing this generated code?

It would never have occurred to me to ask an LLM to convert an entire library from one language to the other, with tests, and everything.

I've started to think of them merely as smarter auto-corrects, like for when you forget the syntax or which order a functions arguments go in, you can just use copilot's inline chat to write "Convert this array of tuples to a dict" and it saves you the 20 seconds it takes to look up the syntax. Or "See above line where I deserialized a uint32 from a binary file. Do the same thing for all the other fields in the foobar struct".

I just never even thought to ask it to simply build my whole app for me.

It’s impressive, but I’m not sure I’d characterize a Haskell wrapper around C APIs as “something new that never existed before”.

> N period on from now, software engineers who haven't adopted or started exploring software assistants, are frankly not gonna make it. Engineering organizations are right now are split between those who have had that "oh fuck" moment and are leaning into software assistants and those who have not.

I'm looking at coders coming out of University who chat-GPT'd every coursework and they can't program their way out of a paper bag. I think what is being lost is the ability to solve non-chat-GPT-able problems. I'm having my 'oh fuck' moment, but I'm looking at the new coders.

Maybe I am just super behind the progress train, and that programmers will become a thing of the past. Maybe we can setup some adversarial LLM training scenario and have LLMs that reliably code better than humans. My concern remains valid though, that there is now a generation being created that turns to an LLM rather than reasoning about a problem [1].

[1] https://www.reddit.com/r/ChatGPT/comments/1hun3e4/my_little_...

I've found that having the coding assistant do large tasks like this unsupervised is the worst way to use it. Asking the assistant to do narrowly defined tasks and then checking the work is the only reliable way.

I've been using Cursor since the summer and wrote up a few tips here https://betaacid.co/blog/cursor-dethrones-copilot

In some ways, refusing to use LLM's to assist with coding is like that funny commercial where they show a guy trimming grass one blade at a time using a nail clipper. Using LLM's responsibly is a massive force multiplier. You still have to know what you are doing or the result could be in a range between not maintainable to disastrous.

In my experiments with LLM's one of the most obvious issues is project structure and organization, particularly in the context of future objectives. It is way too easy to have an LLM go in circles when analyzing ideas for project structures. I have seen them propose layouts that, while workable, are far from ideal.

The same applies to code. It can write usable code faster than a human. However, and, again, in my experience, you are constantly refactoring the results.

In all, this is a new tool and, yes, it is useful. It's amazing to me considering the idea that the whole thing is based on guessing the next token. It doesn't really understand anything, yet, you'd be hard-pressed to prove this to those who don't know how we got here. I've had a few of those conversations with non-coders using ChatGPT for a range of tasks. It's magic.

With LLMs I am in this strange place where I read all of these amazing things that they supposedly all can do. And I have no doubt in my mind that they probably can. And then I ask ChatGPT a simple question such as:

  Here is a logic puzzle that I need some help solving: Samantha is a girl and has two brothers and four sisters. Alex is a man and also one of Samantha's brothers. How many brothers and sisters does Alex have? Assume that Samantha and Alex share all siblings.

And I get back a very well written, multi-step response that leaves no doubt in anyones mind that:

  To solve this logic puzzle:
  
     Samantha has 2 brothers and 4 sisters.
         This means there are 7 children in total (Samantha, her 2 brothers, and her 4 sisters).
  
     Alex is one of Samantha's brothers. Since Samantha and Alex share all siblings, Alex has:
         1 brother (the other brother besides himself).
         4 sisters.
  
  Final Answer:
  
  Alex has 1 brother and 4 sisters.

Maybe it's like with Apple and I am using it wrong.

To get back to the "intern"-comparison. I could usually tell when an intern was struggling, there just were human telltale signs. When AI is wrong, it still presents its results with the confidence of someone who is extremely deep in the Dunning-Kruger hole but can still write like a year-long expect on the topic.

So I've been doing a lot of thinking and I've gone through a rollercoaster of feelings concerning AI and becoming redundant.

Recently though, I've had a different "oh fuck" moment: what if the period of general global prosperity is coming to an end, that Fukuyama's end of history is coming to an end, that climate change won't come under control. Maybe I'll look back and wish that I could be a redundant SWE in a 2010s economy. With such a backdrop, how scary could AI really be?

The only advantage AI code assistants have over competent humans is speed in generating code.

But speed doesn’t matter. When code is going to live for several years, no one will care if it took minutes to create vs a week.

If you are an incompetent human, you might never be able to generate the same code some LLM generates. This will cause you to sing it’s praises and worship the LLM as your savior.

In general, there is a correlation between weak developers and how hyped they are by LLM coding assistants.

I don't care that much for the commentary about who will or will not make it. After all, I think that if AGI is really just around the bend, then we don't have to worry about programming jobs: they probably won't exist. But if it's not, then these tools will probably continue to improve at a rate that makes what we have today irrelevant. I believe that will happen with all of the image generation knowledge people are accumulating with Stable Diffusion derived models.

That said, forgetting all of the AI doom or hype, this seems frankly incredibly cool. I haven't managed to find a great use case for AI code generation where it hasn't failed miserably (except for minor boilerplate uses, like with IntelliJ's LLM auto complete for Java), but actually, generating FFI libraries seems like a pretty kickass use case for automation. I dig this. I will definitely try it.

Does the library work?

Also, this is an incredible achievement, but I don't understand this statement:

> This wasn't something that existed, it wasn't regurgitating knowledge from Stackoverflow. It was inventing/creating something new.

I mean you literally told it to copy an existing library. I understand the Haskell code is novel, but translation is not the same as inventing.

Binary, polarizing language for clicks. I've had a few mild versions of the "oh fuck" moment they're talking about in the past 1-2 years but outside of some specific cases (writing test data generators, scaffolding, etc.) LLMs havent massively changed the way I do things. They're cool and all but, c'mon. I've been programming for 25 years, 15 of which were professional. This is a majorly creative job and while I'm down for Butlarian Jihad as much as the next guy you need to actually have a machine which can be self-sufficiently creative to start talking about it.

There is a psychology paper waiting to be written about humans perceiving intellect as the ability to communicate knowledge convincingly and thus being amazed by LLMs. I’m by no means immune to it

Thanks for this -- have been using Cody a lot and just tried Windsurf on my hobby project. So far it seems immediately like a step up. Has anyone paid for it? The free version is doing good work.

> N period on from now, software engineers who haven't adopted or started exploring software assistants, are frankly not gonna make it. Engineering organizations are right now are split between those who have had that "oh fuck" moment and are leaning into software assistants and those who have not.

Mine was InstructGPT, a few months before the public release of ChatGPT. Everything that followed was an unsurprising extrapolation of what I saw in that moment.

Question is, how far will it continue? That I cannot say. At least a factor of 3 from foreseeable compute improvements… unless a war is started over the manufacture of compute. But also, Koomey's law (energy) is slower than Moore's (unit cost + feature size), and if one presupposes that manufacture of compute is unbounded, then we rapidly hit limits of manufacture of power sources for the compute, and that in turn causes huge economic issues well before there's enough supply to cause mass unemployment… unless there's a lot of huge algorithmic improvements waiting to happen, because that might be enough to skip electricity shortages and go straight to "minimum and cheapest calories to keep human alive are more expensive than the electricity to run the servers that produce economic value equal to the most productive human".

I have no confidence in any forecast past the end of 2028.

This is no ingenious feat by any standard, except probably for someone in the Dunning–Kruger dip.

https://gwern.net/creative-benchmark

I can't wait to see if AI can get past its centripetal plateau.

Listen close, please. The end goal of AI coding assistants is not

> convert this rust library at https://github.com/RustAudio/cpal/tree/master to a haskell library using ghc2024.

The end goal is something more like

> Build me a website where users can make and share a link to a text document like a shopping list and edit it collaboratively

This guy thinks LLMs are so great, but if he grew up with LLMs, not only would he not have the skill set to competently ask the LLM to produce that output, he wouldn't have any idea the output he's trying to produce.

OpenAI, Google, Microsoft are not trying to sell the idea of helping out your senior engineers. They are trying to sell the idea that any of your interns could be senior engineers, so that you no longer have to pay for expensive senior engineers.

I'm confused.

Assistants are so easy to use yet we should drop everything to "learn"/"adopt" them? Why not just wait until they're not complete dumpster fires then start using them?

what HN post were they referring to ?

TLDR: ai can be helpful, anyone who doesn’t use it hasn’t realized it’s helpful, SWE’s who disagree “are frankly not gonna make it”

Review: great fun story, I agree with the first part of the take and I like the post and I like some ai tools, I disagree with the last bit. There are really good SWE’s who are in spaces where ai is not that useful. SWE is too big now, it’s too vague, ai doesn’t apply to all. You don’t need to use ai to survive, but I agree you should try it out and see before dismissing it. And retry it every so often cuz it’s developing and changing.