What follows next is purely speculation and it is based on my own observations and thoughts but based on what I've seen the old RBAC models, while being almost broken before, now it is fully broken, with the fact that now coding assistants and engineers are working on multiple unrelated projects simultaneously - especially working on wild experiments they had no time for previously. The risk of supply chain issue has increased dramatically in the enterprise.
Again, I am not saying it is related but I think it has an impact.
Now in many places it is encouraged by coders and managers to vibe stuff on their own devices. Soon or later it will become a problem, especially for those that have no idea what they are doing.
I am not saying it is related but I feel that it coincides perfectly.
I just cannot believe there is no underlaying thread going through all of these recent supply chain issues, and yes there are some hacking groups that specialise in this, sure, but it is because the bounty is plentiful.
Just to clarify, and I know you weren't saying they are related, but this has absolutely nothing to do with AI or vibe coding or manager code.
It's a continuation of the Shai Halud worm and the lack of security around developer dependnecy installations, which has existed for a very long time.
Hackers have figured out that developers themselves are an ideal target due to how easy it is to trick them into installing something and how much private information they have on their machines (creds, cloud clis, mcps, etc.).
> due to how easy it is to trick them into installing something
You have tools from large corporations where the official installation procedure involves copy pasting a command from a random blog post, run it with sudo and watch it download and execute a script from a random filehost. This is somehow deemed acceptable by everyone involved.
Meanwhile I can't use teams in our meeting rooms, since any form of internet access was deemed a security risk in rooms where customer projects could be discussed. This is in a day and age where 90% of customer meetings are done over the internet.
Anyone trying to follow sane practices in this industry just asks to end up in a padded cell.
> Meanwhile I can't use teams in our meeting rooms, since any form of internet access was deemed a security risk in rooms where customer projects could be discussed. This is in a day and age where 90% of customer meetings are done over the internet.
I hope this is in jest. Are you saying in order to discuss any customer project you have to book a meeting room? So no discussions of customer projects at the open plan desks or even in your boss' office for fear that something might overhear that conversation? Or is this only when the customer happens to be on-site to discuss their project? Does your organization assign U.S. Military style NICKA code names to everything?
As with many other things, AI exacerbates this problem. It’s so easy for many more of things things to happen unattended and in greater volume, and the AIs themselves can be tricked into doing these things, not helped by their patten of “prompt the user to approve 30 different inscrutable pythons and bash scripts”.
> Now in many places it is encouraged by coders and managers to vibe stuff on their own devices. Soon or later it will become a problem, especially for those that have no idea what they are doing.
Yes in our place too. "You better do as much as possible with AI or you will be left behind" dogmas etc.
It's the stupid IoT hype all over again. No concern for security, just trying to be the first in the pack.
I argued for years that we had too few workers for our total project count and management argued that most projects were idle and so it was fine to have so many per worker.
I think web-based IDEs like GitHub Codespaces (but even VSCode with tunnels) is part of the solution because at the very least you can get an isolated dev environment per project. I've been advocating for this for as long as I remember.
Unfortunately, most developers don't like them so it is a though sell.
Why do most developers not like it? Is it because the browser is a terrible platform for text editors since there is no proper key mapping, or access to proper debuggers, or there is too much latency, and no access to cli tools?
You make it sound like you are surprised, but everyone who has tried this knows it's crap and a band aid at best.
What is a real text editor, by your estimation? NVim? Emacs? Genuinely curious.
I use VSCode/Codium since I maintain a GUI stack for general usage. But I have all the terminal tools installed for my work there as well. I hate customizing things too, which I find is necessary if you want to get the most out of terminal text editors. VSCode is pretty good out of the box, with terminal access and everything built in.
Jeez, I hope this doesn't turn into a text editor flame war...
Why would I ever want to use a browser based solution instead of local VMs? If you're worried about VM escapes then you have bigger problems (and hopefully a full time security team supporting you).
Edit: I realize in hindsight this comes across as overly negative. I think those are great solutions to have available for when you are working with a suboptimal local setup for whatever reason. I just don't think they're the default choice let alone any sort of ideal to strive for.
Is the theory here that the browser cannot be co-opted to infect web-based repositories?
Also: thinking of how yt-dlp can integrate with browser cookies now and the malware paths that opens up. (This is part of why Chrome wants HSM cookies, I expect: DRM and opsec!)
In this scenario the malware will not be on the device but in an isolated dev environment on a remote machine. So it will have access to whatever was configured in that repo but hopefully the project is isolated enough to ensure containment and prevent cross-pollination.
I don't think the cloud (someone else's computer) is the best solution. The sanitation problem can be mitigated by compartimentization but the cloud aspect also adds brittleness and new attack vectors.
Why not set up proper containers (or VMs) locally? And why not wait a little till local LLMs catch up?
Maybe just a personal itch, but having your dev environment elsewhere feels so gross to me..
This is the leg of the cycle when we go back to mainframes & centralized computing? With all the datacenter build out; why wouldn't you want your services adjacent to the LLM processing centers?
ABAC/Capability and very granular policies for both actions and actions on behalf of others with the right sort of resource-based policies as well. And the apps need to be capability constrained and sandboxed.
Gonna be a hard nut to crack to implement this across the supply chain.
> Now in many places it is encouraged by coders and managers to vibe stuff on their own devices. Soon or later it will become a problem, especially for those that have no idea what they are doing.
I am not saying vibe coding is the issue. The issue is that a typical developer might be working on a lot more projects that run concurrently then they used to. And because of the various nature of the project the risk is significantly increased.
Scale this across the workforce and you not just doubled the problem.
You can vibecode docs and tests also but I'm truly not seeing more of those.
In the end it can just be a culture thing. A dev who was going to write docs and tests before is going to have a LLM generate docs and tests today. Same with safe practices and defensive coding. The machine does whatever you want from it, for most that's "just get the job done I don't care". So that's the output.
If I vibe code a project, that involves docs and tests as well. Obviously I do not, at any point, do anything blindly and there are some iterations for everything. I always double-check, and I do not use "agents", I do everything manually. I always check what the LLM is thinking, in real-time. I might be old school, but that allows me to write code that is not a pile of shit. :P I am still conscious about quality.
I think that the numerical example you gave appears to be wrong unless you intended 1% rather than 0.01%.
In any case, fair enough. The concern is that organizations will build processes around AI where many people do not review outputs carefully. I do not disagree with this.
I also agree that my particular workflow is anecdotal and does not work at scale.
Yes my bad I even checked it in the calculator but then typed in .01 again but added % again. I meant to do it to serve as an example of how bad humans are at thing.... right...
You can also fork everything and maintain local versions that you much more easily resolve conflicts with upstream with AI and get the best of both worlds while you work through the backlog of internally reimplementing all dependencies, which even with AI will take a long time.
We used AI to build our AI platform and now we are using the AI platform to build the tools that we need for AI. :)
But no honestly, unfortunately most tools I did for myself are not for hobbies but something that I needed for work... like this one (https://github.com/crmkit/crmkit) most recently.
It will be cool to measure models based on their RAW performance and measure them in terms of ROI - not some benchmark but something meaningful like we used this model to solve X.
That will be a massive mind shift and might justify the token expenditure.
I made a tiny ai bug hunting harness (<4MB) that has everything (except the model obviously). It was designed for pentesting purposes where the tiny size matters to make it more portable between environments.
The intended purpose is not to be used as a worm but it does not take a genius to figure out that with small modifications such a thing could work relatively well - especially if it uses AI keys from compromised targets. Making the agent self-modifiable is relatively straightforward task and in fact I already did that in another project.
Every Windows computer has a small rwkv model on it. Wouldn't be hard at all to get decent cpu performance from a tiny malicious harness, especially one that used the self-evolving skills features and open source models.
Malware is going to be crazy, people aren't ready for the revelation of how insecure and broken things are. Everything is held together by bubblegum, duct tape, and panicked engineers putting out fires.
reply