100% and I’m a software developer and have been for ~30 years. Good QA people know how to find regression and bugs _that you didn’t think about_ which is the whole reason why it shouldn’t be under “engineering” and that it should exist. One of the QA people I work with currently is one of my favorite people. They don’t always make me happy (in the moment) with their bugs or with how they decide to break the software, but in the end it makes a better, more resilient product.
Agreed. QA specialists are there to think about what the engineer didn't think about. Unless the engineer is incompetent or the organization is broken, the engineer has already written tests for everything they could think of, but they can't think of everything.
More importantly, it is almost impossible for engineers to be as well incentivized to spend extra time exploring edge cases in something they already believe to work than to ship a feature on time.
Like everything else though, its contextual. Complexity of domain, surface area and age of product, depth of experience on team and consequences of failure are all so variable that there cannot be only one answer.
I have done it both ways for many years. I have worked on teams where QA is a frustrating nuisance, and teams where they were critical to success. I have worked on teams that did pretty good without them, and probably those were the highest throughput, most productive teams because the engineers were forced to own all the consequences - every bug they shipped was a production issue they were immediately forced to track down and resolve.
But those were very small teams, and eventually I was the only founding engineer left on the team and far too many mistakes by other people made it to my desk because I was the only person who could find them in review or track them down quickly in production. That was when I started hiring QA people.
Ive almost never worked on a project where there was the right number of QAs who were doing the right thing.
Usually there either arent any in which case bugs get missed or there are 5 very cheap ones running mindless scripts who are standing in for the devs' inability or unwillingness to write decent automated tests but dont catch the really deep level thorny stuff.
QA has a reputation problem. Because it is considered as unimportant role, good people don't get attracted to it. The average people who do don't do a good job.
> QA has a reputation problem. Because it is considered as unimportant role, good people don't get attracted to it.
Hard disagree. QAs' main tasks are acceptance testing and V&V. This means their work is to act as independent parties whose role is to verify that the results are indeed being delivered. Their importance is up there next to Product Owners.
The problem is that some QA roles are relegated to running the same old test scripts and do some bullshit manual exploratory tests that catch and assert nothing. It's the ultimate bullshit job as it's performative. This is clear in the way that LLM agents are seen as threatening the very existence of the role.
> More importantly, it is almost impossible for engineers to be as well incentivized to spend extra time exploring edge cases in something they already believe to work than to ship a feature on time.
Personal liability and professional insurance works for all the actual “professions” in the US, to some extent, right?
It might be time to start the considerations for professional licensing for platform scale or commercially published software.
More like certified products. New ISO standard may require professional liability for software products, which will be adopted as requirement by big consumers and will pull the industry into certification loop, because insurers will ask for it. This will obviously put a high entry barrier to many product categories, slowing down innovation.
Yes, but slowing down to avoid hazards is sometimes important.
Medical devices and such are the only places I’d expect to see the need for certified products. By extension, in the new era, we really ought only expect certified software where we expect a duty to care from the software system (or any other assigned duty)
My point exactly, embedded devices are the closest software gets to actually being built by licensed engineers. The expectation can often be that you are an electrical engineer by training, where licensure is a viable path, unlike in software engineering.
Couldn't agree more strongly. The best QA people have the old-school “test to break” mentality. They do weird things, like pulling the network cable out of their machine mid-transaction, or kicking off a series of performance scripts and then powering off servers in a distributed system just to see what happens.
When I got started in software, QA was already in a heavy decline. A mentor who had been a QA manager at Apple told me that being a QA person (in the industry) was once seen as a high-trust position. He was sad at what it became.
When I got into software that employer was pretty small (50 people overall I think).
Their approach to QA was to have this be a optional thing the service people could do.
It worked surprisingly well, with the caveat that they never created regression tests.
The employer eventually switched someone from that team to establish these regression tests full time, but they had no programming experience and by the time I left no real progress was done in around 6 month. No idea what came of that, and a few years later they fired a lot of the team
Yes, QA is important. My code will always "work" in that everything I tested is bug free. But having someone other test, especially someone who knows the service is gold.
But there is also bad QA: The most worthless QA I was forced to work with, was an external company, where I, as developer, had to write the test sheet and they just tested that. Obviously they could not find bugs as I tested everything on the sheet.
My most impressive QA experience where when I helped out a famous Japanese gaming company. They tested things like press multiple buttons in the same frame and see my code crash.
> But there is also bad QA: The most worthless QA I was forced to work with, was an external company, where I, as developer, had to write the test sheet and they just tested that. Obviously they could not find bugs as I tested everything on the sheet.
This was my sole experience at the one place I worked with an internal QA team. They absolutely could never find bugs that devs missed, often mis-marked ones that didn't exist, and failed to find obvious edge cases that did exist.
Multiple devs fired because the CEO believed the QA over the engineering team; if they marked a bug as present, it was the engineer's fault for writing it. If they didn't catch a bug that made it to prod, it was the engineer's fault for not including it in the test plan. They represented nothing but red tape and provided no value.
Good QA sounds great! I'd love to know what that's like someday! It'd be great to have someone testing my code and finding breakages I missed! I'm only slightly (incredibly) bitter about my bad experience with its implementation.
I do think the type of testing where QA just follows pre-generated script has place. But it is about long term regression. The first round absolutely should not find anything. But with complex system it also should find nothing in a year or three or five years... Offloading this to dedicate resource could be useful in certain industries.
I did not think of that. Maybe for some industries, it might make sense. But if I want a regression test, I would probably set it up as automated test. In the case I mentioned above it was the only test beside my own for a new service.
The tension is that QA is important. But most QA practitioners are not good. The world is filled with QA people who couldn't make it as an SWE and now are button pushers. But high quality QA people are amazing. These are the ones who understand how to break apart a system, push it to its limits, and engineer the quality plan.
This is an area where I expect AI to create a bimodal future. The smaller group of high quality QA people will now be able to offload the activity to agents instead of the QA drones. They'll still be worth their weight in gold, whereas the drones will be redundant.
I've worked with a lot of QA folks that just repackage up the unit tests the dev already writes. And I've met a few that strikes out on their own and comes up with real tests.
The latter is much more high touch, but they're often worth their weight in gold. The former is kinda pointless.
Exactly. I spent 20 years split between MS and Apple. Some of the best people I ever worked with were in QA. One guy in particular was an extremely talented engineer who simply didn't enjoy the canonical "coding" role; what he did enjoy was finding bugs and breaking things. ;-)
Really? The best people I worked with were never QA.
Moreover, the best QAs would almost always try to be not QA - to shift into a better respected and better paid field.
I wish it werent so (hence my username) but there is a definite class divide between devs and QA and it shows up not just in terms of the pay packets but also who gets the boot in down times and who gets listened to. This definitely affects the quality of people.
I think it's overdue an overhaul much like the sysadmin->devops transition.
We have differing experiences, which shouldn't be surprising. My example explicitly referred to someone who was a good engineer who enjoyed the QA role.
This might have been an Apple/MS thing, but we always had very technical QA people on the dev tools team. For example, the QA lead for the C++ compiler had written their own compiler from scratch and was an amazing contributor.
In the Windows team (back before the test org was decimated) I saw the described "class divide". Anybody who was good enough would switch from SDET to SDE [disclaimer: obviously there were some isolated exceptions]. The test team produced reams of crappy test frameworks, each of which seemed like a "proving project" for its creators to show they could be competent SDEs. After the Great Decimation my dev team took ownership of many such frameworks and it was a total boondoggle; we wasted years trying (and mostly failing) to sort through the crappy test code.
This was all unfortunate, and I agree in principle with having a separate test org, but in Windows the culture unfortunately seemed to be built around testers as second-class software developers.
I spent most of my time working on Visual Studio (in the Boston time frame) so we got to interact with pretty much every team. I absolutely hated interacting with the Windows team. Everything was a fight for no reason.
As I said above, everyone has their own experiences but the QA folks I worked with at MS were fantastic.
Not sure if you're aware but Dave Plumber now has a really good YT channel [0] where he talks about MS back in those days. It's a fun walk down memory lane.
> Really? The best people I worked with were never QA.
> Moreover, the best QAs would almost always try to be not QA - to shift into a better respected and better paid field.
That sort of seems circular. If they're not respected or paid well, of course most of the talented people would not want to remain in QA, and eventually you'd just have mediocre QA. That doesn't really give you any insight into whether high quality QA would be useful though.
(edit: I see now that's basically the point you're trying to make, so I guess we're in agreement)
We don't have dedicated QA at my job, but personally I'm thrilled when a coworker finds a bug in my code. It's much more preferable to the customer finding it
> They don’t always make me happy (in the moment) with their bugs or with how they decide to break the software, but in the end it makes a better, more resilient product.
Did you try with 2 QA people? 65536 QA people? 0 QA people? NULL QA people?
> Good QA people know how to find regression and bugs _that you didn’t think about_ which is the whole reason why it shouldn’t be under “engineering” and that it should exist.
I think the core of the issue is the weasel word "Good QA" instead of just "QA". You're underlining the importance of having someone in a team who has good product understanding and has good ownership skills. How many QAs fit that description? Not many, unfortunately. Some FANGs outright eliminated the role and replaced it with a mix of product owners and team ownership, and some QAs are just doing their 9-to-5 job going through their manual test scripts to verify it something meets a definition of done. What happens when you can have a LLM agent doing the same instantly as a step in a random CICD pipeline?
Yes, QA and test engineers tend to have a better ability to specify the correct behavior of a system than anyone else. It's literally their job.
This is a big asset in the current paradigm where a LLM agent can effectively implement behavior only when given a robust test suite to iterate against.
The red team and blue team should not share supervisors.
Nor, in the case of QA, should the audit team be engineers trained to act and think like the ones who wrote the software. A fresh perspective is useful.
But in the long run, supervisory independence is the real deal. I know of a QA manager who shut down an entire factory's output until a major safety issue (that had been kicked down the road several times) was addressed. It took chutzpah, and serious power, to do that. The Dir. of Engrg. would NEVER have allowed it.
Frankly, calling software development engineering is quite debatable. We should be calling less things engineering that aren't actually engineering qualifications.
Engineering like the guy in the booth at a show is a sound engineer. Talented: check; challenging work: check; valuable: check; creative: check. "Engineering" like designing a building, bridge, or power line? Nope.
It's not a protected term in the US so it's jarring to those of us living where it is.
It should be under engineering but as a seperate skill/role. To ship fast you want QA aligned with milestones, sprints etc. You want QA to feel the deadlines, and then like an Eng push up if things aren't realistic.
What I have seen before with QA is a queue like system. Hand to QA. get back in 3 days. Not the QA persons fault. It is the setup if the org that is wrong.
Imagine if you had a code review role and code review team and you had to wait to get your code OKed. We wouldn't put up with it.
QA teams seperate from Eng worked when the software was burned to CD rather than uploaded to s3.
Oh wow. That's wide open then, just about any of them. I run Mint on older mac hardware (happily cruising along on an 11" MacBook Air), I'm a sucker for Ubuntu's default look and feel a lot of the time - and they have a pretty large user community (as do all the major distros), and love Fedora too.
One note - a lot of the distros are Debian based (Mint/Ubuntu/etc.) vs. Red Hat (Fedora, Asahi - M1/2 Apple hardware Fedora, etc.) vs. Arch Linux.
If you're just getting started it's probably easier to get into the Debian world, but I'd advise you to dip your toes in other distros too unless you just deeply fall in love with your first pick.
(I have my 80 year old neighbor happily running Mint on their old computer to save them from having to buy a new Windows machine.)
I'll add that Libre Office can be great but if you already have an ecosystem you like, most of those have online editors too which can handle most of the average user needs.
this. despite all the ghost stories and war stories. it’s how apple sells you the watch to save you from that bear attack or that time you got trapped somewhere.
the stories are real, and in some cases you may need it — in most cases you don’t. and it clearly doesn’t always protect you.
Sure, lots of people who are white and deny systemic racism or the effects of it exist. That’s literally the problem we’re fighting and why we need diversity efforts.
Lets say what it is, there aren't enough African Americans in the white collar workplace.
Its not that we don't have Asians, Mexicans, or Indians. At least half the companies I worked for, white people were less than 50% of the workers.
Its not a current racism issue, its likely a relic of racism issue. A large number of African Americans grow up in poor areas, get terrible education, are among a non-work-ethic driven culture, and never make it to the American Dream. Repeat generation after generation. Sure there are racists from the south, but talk to an African American of privilege(middle/upper middle class) living in a city. They have little bad things to say, and mostly blame it on the individual.
The issue is the zip code. You can see it in how white people living in these same areas perform.
you make good points but i'd argue it's both. i'm talking about the subconscious bias aspect of racism (to your point it's a relic of racism, but it's an active relic) as well - this is the very reason we need DE&I. we're 3 to 4 years into it and still non-marginalized folks don't seem to fully understand the problem.
i feel like "talk to an African American of privilege" and what follows is incorrect. i know and have known a lot of well off Black friends and associates and they all have stories - there are a lot of racists in the north too, they just act a little different.
My issue isn't with the ostensible spirit or intent of DEI programs. The concept sounds great. The problem is the corporate implementations are transparently inauthentic and insincere. It's abundantly clear that the real intent is PR and compliance, not some altruistic desire for fairness.
Admittedly didn't read the thin article. I do think there's something to the premise that people from diverse backgrounds end up unhappy.
Poorly run diversity efforts are upsetting to POC. In a way they all tend to be poorly run. It can be infantilizing and condescending.
Ultimately, we end up falling short of the numbers, so the people behind the programs are unhappy. POC are unhappy because of the quality of how the programs tend to be run. Obviously the fascists/republicans aren't happy. The centerist people tend to fall between the people running the program and the right.
So who's supposed to be happy. I will say that in my experience, these diversity programs don't exist. That is to say the really overbearing ones people complain about. I imagine it's a bigger thing for SV and some big companies (although, I have worked for big companies).
In my experience, programs are nearly non-existent and the minorities they hire are at least "pretty competent", which is hard to find in software. If anything, it looks like they're still being pretty choosy. So people like OP get their wish, and the programs are pretty laid back.
i agree with a lot of that. i run a Pleroma server (less overhead than Masto), i think the winning combination will be the group that comes up with a universal baseline for online behavior. the intellectually dishonest purveyors of hate speech have been making a lot of inroads in the fediverse and in part (imo) due to the fact that we don't have a good way to trust other admin's decisions to block/ban servers and users and relay those quickly.
Dorsey isn't really "actively" in any of this process or direction beyond the initial setup of things at Blue Sky btw. Jay is doing a great job and a lot of the "protocol" hubbub is actually some unfortunate (in this scenario) nomenclature and less actual new protocols.
i'm for whatever thing(s) allow for a healthier online experience for folks and whatever keeps _us_ from being the product.
(i'd add that as someone who's made most of my living developing integration software - i'm hoping once things are cemented a bit that there's a non-ridiculous path to having a 2-way relay for sites that want it)
