> The way you test code cannot (and should not) be decoupled by the way in which you architect the code itself.
I'm not sure I fully understand. In my view, it's pretty evident that tests should test behavior and not structure. Coupling to implementation details is unavoidable but should be managed, and ideally minimized, to keep the test suite robust and maintainable.
Given the choice between a test suite that monkey patches out some dependency (redis, say) on a per-test basis, and one which substitutes a fake for this dependency in some centralized composition root, which one do you prefer, and why?
> I'm not sure I fully understand. In my view, it's pretty evident that tests should test behavior and not structure.
Not OP, but I think I know what OP has in mind. There are some trivial ways in which code can be made easier to test as well as more complex (let's call them architectural) ways.
Examples of trivial conveniences include: making tested functionality accessible from the test (i.e. private class methods in Java present a problem for testing), coding against an interface rather than an implementation (this allows substituting a test implementation for a production implementation that can fish out some bugs), having internal to the code correctness checks that don't necessarily influence program execution (simply having runtime type checks, like it's done in eg. Java, compared to eg. C makes testing easier and more productive). Of course, there's more, these are just very common and easy to understand.
The architectural properties that improve the ability to test a program might be: the so-called "observability" programmed into the product, i.e. the product generates metrics data that's not part of the desired output. The product is split into modules with formally defined interfaces, which allows testing modules in isolation and lowers the number of possible combinations to test.
Improving testability isn't necessarily a good thing because it's likely to negatively impact complexity, size or speed of the program. So, depending on what's more important for any given program, the testing strategy will be different...
You are not wrong but if we go deeper we will find more nuances.
> tests should test behaviour and not structure.
Of what?
The whole idea of software architecture, and underlying of my messages, is to structure the code so that it is easy, but before easy, possible, to work at the right abstraction level.
Which allows to tests the behaviour of components and not their structure.
Trivial example, say your code read from a socket and manage the bytes with some CPU operations and write them to another socket. (You may recognise it is basically what a compressor do)
The way in which you manage read and write to the sockets will make dramatically simpler or much more difficult to write good tests.
If you adopt strategies like sans-io, you will see that the testing is almost trivial.
If all the logic sprawl up from the read syscall in a loop, you will notice how more challenging testing becomes.
---
To answer your question, the way I let LLMs write code is very DI (dependency injection) based.
A class never instantiates another class - all the dependencies are passed to the constructor. Including time. Including whatever DB iteraction.
The reason why I prefer this is that I can test each component at every level of abstraction. I don't have to. But I can.
My current approach is to force coverage higher than say 80% as default and then when a bug is discovered drill down to the components.
The question "for which functions/modules/classes should I write tests?" and the question "how do I structure my functions/modules/classes?" are very closely related questions, which I think is what the other comment was getting at.
That follows largely from what you're saying though, so I think the two of you agree, you're just coming at the same thing from different perspectives. Like you say, you want to avoid/manage coupling tests to implementation details, and typically the best way to do that is to bundle a significant, testable chunk of code into a single module, define a clear public interface to that module, and then test that interface. The internals of that module are then free to change, but the test interface should stay the same.
But the corollary of that is that you can't just design your modules independently of your tests, you need to design them so that they are testable. Which is why the previous comment links testing to code architecture/design.
Regarding dependencies like that, the best situation is where you can either:
(a) design a module so that the dependency is injected with a clear interface (not just "here's an instance of libredis.rs" but "here's a series of callbacks for saving data, those callbacks might save to Redis, but they might be in-memory only, or they might save everything to the blockchain, either way it doesn't matter").
(b) just spin up an instance of Redis and test directly against that — it should be quick enough, and there's plenty of ways to ensure that the tests don't affect each other even while accessing a shared resource.
> Teams, em dash bla bla bla em dash have quietly bla bla bla. It's not this, it's _that_.
I find it hard to read articles where the agentic writing is this obvious. It's a distraction from the message of the text, which I'm sure is worth my time. Is there no way to stop generated writing from sounding like this?
The prompt was actually huge, containing all the central ideas of the articles in details. I actually found my first prompt in case you're curious:
---
Human contexts are way more limited compared to computers. We can't reason about complex software when they go through many branches with so many implications. It's just too hard for humans to keep track of all interconnected pieces. So humans have historically split the system parts into manageable modules that can be understood in isolation and then we spend some time connecting those parts. That's how we can keep the context reasonable for human understanding.
So, when senior engineers found themselves lost while trying to debug an issue in a complex system they would naturally decide to pause and rewrite or refactoring the confusing piece of the system to make it manageable so developers can easily understand what's going on and review future changes.
Usually a system doesn't start that confusing. But as requirements change developers add additional branches and code until the code is no longer manageable. Sometimes the requirements changed significantly since the code was first written and all we have in the code are exceptions rather than the rule. That's usually when historically senior developers would take the time to rewrite that part of the system so they can reason about it.
But AI agents are not as limited as humans context-wise and they can reason about those confusing (to humans) systems and make sense of it. So they simply keep adding additional branches to the existing mess without ever suggesting a major refactoring like a senior developer would do in those cases, unless there's specific harness to tell agents to act like that.
This article is about bringing this into attention so that developers can policy themselves and keep asking themselves whether it's time for a major refactoring instead of relying on the AI agents and trust them because they no longer understand the code because it's too complex for humans to follow. Can you draft an article focused on this concern?
Thank you, I appreciate you sharing this very much! I wish I could read the entire chat, but I imagine there might be some NDA stuff in there if you talk abt real world experiences.
So, I get the idea of having a layer of abstraction between your tests and the system under test. You decouple _what_ is being done from _how_ it is done, which leads to robust, easily maintained, and even more re-usable test suites. I've seen it in action, it's a great idea, I've reaped the benefits myself. Great stuff.
What I've never really understood is why some people choose natural language for the abstraction. It just seems like such an odd and expensive choice. Most of the layers of abstraction we add in our codebase are written in the programming language of choice itself, they're functions, classes, what have you. But for this one specific flavor of abstraction, folks reach for natural language.
The one potential benefit I could think of for this over a more standard DSL is that non-technical colleagues would be able to read or even write these specs. But in practice I've only seen this pan out once in my life, and that was with a more traditional DSL rather than natural language.
Because the intent has always been that Product delivers the BDD statements not Engineering, but Product people have (for the most part in my experience) refused to deliver what BDD flows need to be tested. That’s why most of the time the test suites aren’t as resilient because the engineer is guessing at which flows to test.
In that context it makes more sense that Llms are a push to replace Product people not Engineering people.
I don't think so. Natural language is vague and repetitive.
I used hitchstory (YAML based stories) for doing this coz it's terse and typed.
I've tried using cucumber before and it reminds me of COBOL - which also had the same idea of making code more accessible via a "natural language-ish" interface.
> If coding is easy, how come programmers were in high demand, and have demanded large salaries for years (even before ZIRP)?
There's probably a lot wrong with the "coding was never the hard part" take, but this quote right here shows that the author is not willing to engage with the actual idea that folks who say this are espousing. Because if the author was discussing these ideas on good faith, he'd know that the answer is obvious: there's much, much more to a software engineer's job than just coding, and that other stuff is very hard to do well, and people pay for that.
Again, I'm not saying the "coding was never the hard part" folks are right, but I really, really hate straw men.
It was a commercial slogan for a toilet cleaning agent. An English equivalent would be "mr clean recommends you use mr clean". Today, it is used to point out when someone tells you they themselves delivered good work. Anytime you'd use the meme of Obama giving himself a medal, you could use this phrase.
Actually the existence of the French name is confusing for English speakers as although many would know enough school French to get "canard" == "duck" if prompted, in English this word ends up meaning some sort of fabrication or hoax, apparently from an old French joke where somehow that punchline of that joke was popularized in England so long ago we don't have records. Human culture is weird.
So the first thing an English native might get from "Canard WC" is probably hoax toilet, I expect the duck imagery of the product would bring the "canard" == "duck" meaning across though.
I don't see how it does. Claude Code is an extremely widely used product; the preceding comment offered an objective evaluation target, not a "trust me it's good" argument.
I don't see how Claude code being a widely used product is relevant to the person who orchestrated the Rust rewrite of bun saying that the orchestration of the Rust rewrite of bun went well. Wc eend is a widely used product as well, if that helps.
Ah, for context, which I suspect you may be unaware of, Jarred (the person who said the rewrite to rust went well) is the creator of bun, and the guy behind the rewrite.
I think the chain of reasoning is not hard to follow:
1. Assume the Rust rewrite of Bun went badly
2. Then something must be grievously wrong with a released bun runtime based on that code
3. Claude Code uses the released bun runtime based on that code
4. From #2 and #3, something must be grievously wrong with Claude Code
5. If something were grievously wrong with Claude Code, users would reduce use of Claude Code and use alternative tools
6. From #4 and #5, users are reducing use of Claude Code and using alternative tools
7. Claude Code has wide use and use is growing across all software engineering verticals
8. #6 and #7 contradict
9. From the contradiction, the assumption in #1 is false
The parts that are not explicitly spelled out here are an exercise for the reader. It doesn't really matter if the guy who wrote Bun said this or my uncle said this.
Ok, thanks for explaining! Doesn't really explain how the wc eend expression _doesn't_ apply here (which would be hard to do, because it _does_ apply, since this is someone praising their own work, and it doesn't get more straightforwardly applicable than that), but I do really appreciate the effort.
I always thought it was common sense and just basic critical thinking to take people paid by Anthropic praising products of Anthropic with a grain of salt (and let's be clear, that's what this is), but apparently it's not.
It reveals the exact type of mindset that could produce an article like this. All they know is waterfall, so they see agentic coding and think: "you know what we need? More waterfall"
reply