The performance not only depends on the tool, it also depends on the model, and the codebase you are working on (context), and the task given (prompt).
And all these factors are not independent. Some combinations work better than others. For example:
- Claude Sonnet 4 might work well with feature implementation, on backend code python code using Claude Code.
- Gemini 2.5 Pro works better for big fixes on frontend react codebases.
...
So you can't just test the tools alone and keep everything else constant. Instead you get a combinatorial explosion of tool * model * context * prompt to test.
16x Eval can tackle parts of the problem, but it doesn't cover factors like tools yet.
I think it's just written by someone who reads a lot of LLM output - lots of lists with bolded prefixes. Maybe there was some AI-assistance (or a lot), but I didn't get the impression that it was AI-generated as a whole.
The thing that sucks about it is maybe his english is bad (not his native language) so he relies on LLM output for his posts. Im inclined to cut people slack for this. But the rub is that it is indistinguishable from spam/slop generated for marketing/ads/whatever.
Or it's possible that he is one of those people that _realy_ adopted LLMs into _all_ their workflow, I guess, and he thinks the output is good enough as is, because it captured his general points?
LLMs have certainly damaged trust in general internet reading now, that's for sure.
I don't know why you do. I found the article interesting, derived value from it. I don't care if it's an LLM or a human that gave me the value. I don't see why it should matter.
It matters to me for so many reasons that I can't go over them all here. Maybe we have different priorities, and that's fine.
One reason why LLM generated text bothers me is because there's no conscious, coherent mind behind it. There's no communicative intent because language models are inherently incapable of it. When I read a blog post, I subconsciously create a mental model of the author, deduce what kind of common ground we might have and use this understanding to interpret the text. When I learn that an LLM generated a text I've read, that mental model shatters and I feel like I was lied to. It was just a machine pretending to be a human, and my time and attention could've been used to read something written by a living being.
I read blogs to learn about the thoughts of other humans. If I wanted to know what an LLM thought about the state of vibe coding, I could just ask one at any time.
