These days LLMs are generating a lot of code. Code is a languages designed to enable human-computer collaboration, so it makes sense that large-language models would be good at generating it. Anthropic recently analyzed interactions in Claude and found that the most common type of conversation was “Web and Mobile App Development.”
In the WordPress ecosystem, I’m seeing plenty of this AI-generated code. If you need some specific task accomplished, why check the plugin directory for an existing solution when you can have an LLM generate it for you almost instantly. I often use it for simple tasks like generating CPTs and custom fields or taking a first pass at some CSS that I’m too lazy to write.
I’m seeing it used for a lot of deeper functionality, too. Need to migrate some post types or bulk manipulate some data? Why futz around in a plugin like WP All Import when you can get the exact functionality you need written just for you, often as a slick WP-CLI command.
When AI-generated code started to get good, there was a common refrain that open source software had the most to gain from this technological advancement. In some ways, we’ve seen this bear out. LLMs love generating code for open source frameworks like React, Express, and WordPress. They have so many examples to learn from, and the license is open, so of course they can build with it. But is “quantity of private downstream code generated” a tangible benefit? And if so, is it exclusive to open source software?
I don’t know what’s happening inside companies running more proprietary tech stacks, but I assume that if they have a large enough codebase (or are leveraging many of these same open source stacks themselves), they’re seeing these benefits, too. In fact, LLMs are probably getting powerful enough to understand the underlying structure of programming in general, whether they’ve seen a lot of that specific framework/language or not.
Not that AI is a perfect software engineer, by any means.
Right now, LLMs are great at small concrete tasks, but they tend to make more mistakes inside bigger codebases, bigger projects, bigger features. They do ‘new features’, but don’t do ‘maintenance tasks’ nearly as well and often get caught in bug-inducing infinite loops. And of course they tend to make strategic decisions that the user may not understand the effects of until it’s too late. For a project like WordPress, with over a thousand open pull requests waiting to be reviewed, code quantity isn’t the most pressing issue.
While I’m happy that more people are able to ask ChatGPT to build them that one-off WordPress plugin, I wonder if there’s something that open source is losing as a result of generative code?
In the past, if I needed to solve a problem with code, my options were to write it myself OR to find an example from someone else. I could use their code or I could fork it and tweak it. Maybe I’d publish my fork or even send it back to the author as a pull request into their project. This is the foundational transfer of value generated by open source software.
Open source thrives when we collaborate, when we publish our code, when we take the extra few minutes and make whatever we just built a bit more universal so that it can hopefully solve other people’s problems. It builds in public so that ultimately the best solutions can gain widespread adoption.
I don’t think we’re going to see large open source projects suffering any time soon, but I do think that the onramp for new contributors (much like the onramp for junior developers) is disappearing.
WordPress in particular has become much less open to new contributors in recent years. Part of that is the seismic shift from PHP to React/JavaScript for the bulk of the new admin interface. But even more than that, the Gutenberg repository is still run like a skunkworks team quickly prototyping and iterating (in public). Features are picked up and dropped randomly, documentation is sparse, and the best way to get an answer about the massive codebase is not RTFM or ChatGPT – it’s tagging strangers on Twitter.
What open source projects need are more knowledgable contributors who can lead projects, make decisions, have good taste, have a sense of ownership over features, and who can onboard new contributors. I’m reminded of Brooks’ Mythical Man month, which argues that additional manpower only slows a project down, mainly because it increases the amount of intercommunication needed faster than it can decrease the outstanding coding needs. At this point, it’s hard to see where AI is going to solve that issue. Reading an AI-generated summary of an issue thread is not going to lead you to the kind of deep understanding needed when making decisions.
The pathway for new contributors is not just understanding code, but also understanding the culture of collaboration. There’s nothing wrong with AI-generated code, I’m using it all the time, too. But I would encourage developers not to lose sight of the actual value of open source software – it’s not just the code you get, but the code you give away. And more importantly, the strangers across the web that you collaborate with to make it even better.
Leave a Reply