It is more just that there are a wide variety of existing ways available to harm or kill the population. No need to invoke the specter that AI magic is going to engineer some super bug. Plenty of virulent pathogens already around.
You could trivially cause enormous harm by sabotaging electrical distribution, water treatment, hospitals, airports, etc. Go to any food manufacturer/distributor and contaminate the products with listeria or such.
> Oh no, now this website has information on how to kill someone.
That's just a bad faith response.
> I'm not a bioweapons expert
That's the whole point, innit? You don't know how to do, so you can't. Publishing functioning recipes using easily accessed materials would provide you and countless others with a bio-weapon. Thus the publication is the problem, not the tech stack (internet, pcs, servers, etc.).
Therefore, OpenAI publishing risky information is equally at fault.
Oh no, now this website has information on how to kill someone.
I'm not a bioweapons expert, but I think there's a lot of equipment and materials required to build a bioweapon of sufficient lethality and efficacy.
Look at anthrax for an example of a bioweapon with only one of those.