AI-assisted coding will change software engineering: hard truths

> It's becoming a pattern: Teams use AI to rapidly build impressive demos. The happy path works beautifully. Investors and social networks are wowed. But when real users start clicking around? That's when things fall apart.

Oh no. We will have to endure another sort of AI slop infesting the web? It's bad enough as it is. Most smaller web sites are already broken in tiny ways. Who hasn't had to break out the browser debugger just get past some web sites broken order page?

Sloppy reviews, images, chat bots, and phishing are everywhere now. In this brave new world someone no computer experience tinkering a home with can produced a beautiful looking web site that's broken it 1000's of ways, we are going to be overrun with this crap. And they are going to be harvesting login email addresses and passwords.

It's going to be a rough decade.

My personal belief is just that AI assistants feel faster, not that they actually are. I'm sure they are in some specific circumstances but on average. They feel faster because your work is different, you put in different energy. I don't want to say easier (probably is) because just context switching has similar effects. People are really not reliable self evaluators. It's always the top comment on any post about some psychological study but never for these AI things. Truth is it's hard to find objective measurements. Lines of code, commits, things shipped, etc aren't strong metrics.

But I think the opening of the article is important, it asks why products aren't getting better [0]. We all feel this, right? There's so much low hanging fruit that could make our lives less frustrating but is never done because it isn't flashy. Like Apple, you got all that AI but you can't use a regex to merge calendars? Google, you can't allow a test task to be made in the past (extra helpful when it repeats). Wikipedia still uses the m address and you go to the mobile site from desktop if you don't manually remove? I could go on and on but I think we're just in the wrong headspace.

[0] imo products are getting worse, but that decline started before GPT

I have been using LLMs (chapt-gpt, perplexity, claude) for development for over a year. It is helpful for summary explanations of concepts and boilerplate for frameworks and library APIs. But it makes errors within those consistently.

Its a great tool and saves a great deal of time, but I have yet to go beyond generating snippets I have to vet, typically finding a made up library API call or misunderstanding of my natural language prompt.

I find it hard to pare down these LLM evangelizing articles into take aways that improve my day to day.

Different take: AI writes a lot of mediocre code for us so we don't have to and we're impressed at that.

But that's not the problem we need to solve. All our programming languages are verbose and stupid. It takes too much effort to solve the problems we do in them.

I heavily started using windscribe and cursor editor. Mind you, I have 10 years of experience.

But lately I have cut down their usage and gone back to writing stuff by hand.

Because it is so easy to apply the changes the AI suggests but there's this subtle shift over time in the code base towards a non-optimal architecture.

And it becomes really hard to get out of.

The place where I still use AI quite a lot is autocomplete but of the intelligent kind like if I am returning a different string for each enum, all of that gets autocompleted really fast.

So line completion models like what JetBrains provides for free is I think the right balance. Supermaven also works well.

The problem with this is that it's supposedly about predicting the future but actually bases everything on LLM's current capabilities.

It's incredible that people still haven't figured out or won't accept or plan for technology to continue to improve. Especially given how obvious the rapid improvement in this area has been.

Having said that the article seems to accurately reflect what it's like using the current tools.

But how could anyone reasonably expect the situation to be similar 3-5 years down the line?

If they just didn't frame it as a prediction then it would make sense.

> The irony? AI tools might actually enable this renaissance. By handling the routine coding tasks, they free up developers to focus on what matters most: creating software that truly serves and delights users.

How very naive. All productivity and efficiency gains will be utilized to push out an ever-increasing stream of new features because that is what drives sales and that is what the business needs.

It is the exact same reason why widening a highway does not actually reduce traffic congestion.

Echos with my experience. LLMs work great if you micromanage them aggressively, but the moment I put too much trust, it backfires terribly.

I've tried using Claude for the first time to help me write some wifi code for an ESP32. It gets like 90% of the way there and I can't get the last 10% to where it works how I need it. Every time I mention an issue with the code, it rewrites it and a new issue pops up. Then I me tiin that and it rewrites it with the original issue again. It's like it forgets the past mistakes it's made.

Also there's been multiple times where it just forgets a closing parenthesis in a function or tells me a function definition doesn't exist in the code even though it's literally right there.

Tangentially related, I get strong Metaverse/NFT vibes around predictions on agents.

Namely, a lot of predictions were made around NFTs that just didn’t make sense or were kind of dumb. My pet favorite was this notion that in the future you could bring your NFTs with you to different games and the like. You could buy a Batman NFT costume and have your guy wear it while playing metaverse World of Warcraft. They basically took Ready Player One and ran with it. Besides the fact that this is much harder to do than they could imagine, it’s also kind of a goofy idea.

I feel the same way with predictions made around AI agents. My pet favorite is the notion that we stop using the internet and delegate everything to our agents. Planning a trip? Let an AI agent handle things for you. Shopping? Likewise, let an agent handle your purchases. In the future ads won’t even be targeted at people, they’ll target agents instead, and pretty soon agents won’t even browse the internet but talk to other agents instead.

Is it feasible? I can’t say. I’m more interested in how goofy it all sounds. The notion that you no longer have buyer preferences while your agent gets served ads, or the notion of planning that trip to Rome or whatever and just entrusting the agent with the itinerary as if it won’t come up with unoriginal suggestions.

Work agents make more sense in general, but the sentiment remains.

Flow state code != production quality code. Not yet.

From personal experience, writing real-life production code is like running a marathon; it requires endurance and rigor. I've seen AI-generated code — it’s more like a treadmill run, fine for practice only. Unpredictable issues, hallucinations pop up all the time with AI code, I have to rely on my own skills to navigate and solve problems.

The unintended consequence of this is well captured in Bainbridge's piece from 1983 'Ironies of Automation'

https://www.sciencedirect.com/science/article/abs/pii/000510...

- The more tasks you automate the less practice people get in doing those tasks themselves and developing the experience in executing them.

- Yet, experience becomes more important as issues/exceptions occur (which they will)

- Ironically, when people are needed the most they are least prepared to step in because automation has taken over their day-to-day.

Net result might be a reduced supply of 'experience' but demand remains strong thus increasing the price of it.

The even harder truth not mentioned here is that existing tools have a hard time understanding large codebases with well-establishd internal patterns and libraries.

The article mostly talks about how AI tools can help with new things, but a large amount of software development is brownfield, not greenfield.

It is a somewhat interesting take.

What I am interested in as a person teaching a computing course, what is the best way to force people to understand/interact with the code coming from the LLM. I.e. when I give computing problems to students, it is often easy to put the problem in chatgpt and get an answer. In a very significant fraction of cases the code would be somewhat sensible and would pass the tests. In some cases the output would use the wrong approach or would fail the test, but not often enough to completely discourage cheating.

In the end this comes down to the question of what skills we want from people writing code with the help of the LLM, and how to test for those skills. (here I'm not talking about professional programmers, but scientists rather)

I was reading the book "How Big Things Get Done," which is immensely applicable to the field of software engineering. The book is about how and why big projects fail. The book mentioned that IT projects were among the worst in cost and time overrun. I see essentially a win-win situation for software developers:

Either,

AI will enhance the work of software engineering on a fundamental level, helping SWE projects to be delivered (more) on time and with high(er) quality (I can't state how amazing this would be)

OR

things won't get significantly better, projects still can't reliably be delivered, software quality doesn't get better, etc. (the robots won't be taking our jobs)

It will be interesting to see which future we'll end up.

Here's a related HN discussion about the 70% article that Addy wrote on the same topic:

https://news.ycombinator.com/item?id=42336553

Here are some more hard truths to add to the pile.

> this kind of crawling and training is happening, regardless of whether it is ethical or not

Glad we've established that it's going to change our profession regardless of ethics.

> Software engineers are getting closer to finding out if AI really can make them jobless

The capital class is definitely interested in this. They would love to pay fewer of us or pay us less and still get the same results. The question in 2025 might be: why would I pay you if you're not using GenAI assistants? Bob over there accepts a lower salary and puts out more code than anyone else on this team!. They may not care what the answer is: profit is all that matters.

After all, they clearly don't care about the ethics of training these models, exploiting labor in countries with weak worker protections, soaking up fresh water during local droughts, etc. Why would they care about you and your work?

Personally I don't find that generating code is where I do most of my programming work. I spend more of my time thinking and making sure I'm working on the right thing and that I'm building it correctly for the intended purpose. For that I want tools that aid me in my thinking: model checkers, automated theorem provers, and better type systems, etc. I need to talk to people. I don't find reviewing generated code to be especially productive even though it feels like work.

I think code synthesis will be more useful. Being able to generate a working implementation from a precise specification in a higher-level language will be highly practical. There won't be a need to review the code generated once we trust the kernel since the code would be correct by construction and it can be proven how the generated code ties to the specification.

We can't use GenAI to do the synthesis and replace a kernel as we still haven't solved the "black box" problem of neural nets.

The problem I find with GenAI and programming is that human language is sufficiently vague for communicating with folks but too imprecise for programming.

I suspect that in a few years there could be a gold mine for consulting: fixing AI-generated "house of cards" code.

Hope we're all good with the coming wave of security errors, breaches, and general malfeasance that's coming with the wave of GenAI code. You think software today could be better? The current models have been trained on all the patterns that make it the way it is now. And they will generate more of it. We have to hope that "software engineers" can read enough code, fast enough, and catch those errors before they ship. Should be good times.

The big change: no more big universally used frameworks like React, Vue, etc.

Every company has its own little special framework crafted by an AI with its own nuances you need to learn, and your skills will no longer transfer from company to company. Gone will be the days when you can swap out a software engineer for a similar one with the same experience in a framework you use. Every engineer coming in has to start from zero and learn exactly how to work with your special paradigms, DSLs, etc.

You can kinda see an agent that automates a 'best practices' AI-assisted workflow to iterates with AI to generate and run tests, optimize code, and feed in the right examples or signatures so it can generate code that properly uses existing APIs.

Maybe trying to use cheaper models first and then calling the more expensive models to iterate and get through tests or errors.

I haven't really seen anything like this so I imagine it's a lot harder than I'm imagining.

LLM’s have replaced Stack Overflow for me. Occasionally I can use them to write a simple bash script for me or some bit of terraform that I don’t feel like looking up. Useful but not exactly life changing.

What I would consider a game changer would be generating USEFUL unit and integration tests. Ideally that used the existing fixtures and utilities already in place. I’ve yet to see that happen even with code the LLM had just generated.

I think this might be like those static analysis tools.

people run their code through it, and it finds a LOT of problems, some of them serious.

and then you fix them and you feel good.

But after that, you have a bunch of problems that aren't real. You either ignore them or the tool starts creating more work exponentially.

I suspect AI will be like that. It will help you a bit, but don't get caught up in it because you'll spend time distracted doing AI things.

This is an amazing breakdown. I can't believe how quickly AI tools have become integral to our workflows. Completely agree with the idea that experienced developers will be even more valuable in the future.

I found these ideas / analogies to be helpful to cut out the chatter for GenAI

1) Analogy - Using chat GPT to do code is like deciding to cross the amazon. You start moving, and half way through you realize the map is wrong. Now you are in the middle of the Amazon, without a map.

2) Reliable Matte Painting / Rough work -> I sketch, so matte painting is what GenAI reminds me of, quite a bit. It’s going to get you half way… somewhere, faster. You have to get to the end yourself.

It’s easier for me to assume that GenAI is going to be mostly correct 70% of the time, and never a 100% of the time. Build and use accordingly.

I’m tired about the chatter about the chatter about GenAI at this point.

the biggest reason not to use LLMs is the customer noncompete clauses where they’re learning from the conversations but you agree not to “develop models that compete” —

makes me think everyone using OpenAI, Anthropic, Gemini API, Mistral, Copilot, Perplexity, etc is dumb or addicted to short term benefits while totally ignoring the long term consequences of paying to train your own replacement while agreeing not to compete

LLMs are an ad-free version of google search. If you use LLMs with this expectation, it'll be improvement of software engineering productivity.

There are many ways of thinking and reasoning about the profession and what it means to each and every one of us.

Some of the buckets:

* The builders, don't care how they get the result.

* The crafters, those who care how they get to the results (vim vs emacs), and folks who enjoy looking at tiny tiny details deep in the stack.

* The get-it-done people, using standard IDE tools, stick with them, and it's a strict 9-5.

...

And many with types, and subtypes of each ^^.

In my opinion, many people have a passion for making computers to do cool things. Along the way, some of us have fallen into the mentality of using a particular tool or way of doing things and get stuck in our ways.

I think it's another tool that you must know how to utilize and utilize in a way that does not atrophy your skills in the long run. I personally use it for learning and allowing me to get an in on a knowledge topic which I can then pull on and verify that the information is correct.