We are not running out of IPv4 space because NAT works. The price of IPv4 addresses has been dropping for the last year.
I know this because I just bought another /22 for exe.dev for the exact thing described in this blog post: to get our business customers another 1012 VMs.
Your NAT traversal article is amazing, but sadly the long tail (ha) means any production quality solution has to have relays, which is a huge complexity jump for people who just want to run some p2p app on their laptop.
And it's not clear it will ever be better than it is now with CGNAT on the rise.
IPv6 does not work on the only ISP in my neighborhood that provides gigabit links. I will not build a product I cannot use.
Even when IPv6 is rolled out, it is only tested for consumer links by Happy Eyeballs. Links between DCs are entirely IPv4 even when dual stacked. We just discovered 20 of our machines in an LAX DC have broken IPv6 (because we tried to use Tailscale to move data to them, which defaults to happy eyeballs). Apparently the upstream switch configuration has been broken for months for hundreds of machines and we are the first to notice.
I am a big believer in: first make it work. On the internet today, you first make it work with IPv4. Then you have the luxury of playing with IPv6.
Whenever I see a comment that says "if you don't do the thing in the most efficient way possible, someone else will steal your lunch", I think that people vastly overestimate the likelihood that this will actually happen.
It's similar to "open source is the most secure because it has the most eyeballs on it", but in reality security bugs will exist for years with no one noticing because people vastly overestimate how any developers will actually spend their time analyzing any given open source software.
Sure, bugs are more likely to be caught in open source and it's more likely someone will take your market share with a more efficient and competitively priced product, but you're overblowing the likelihood of both by a large margin.
> "if you don't do the thing in the most efficient way possible, someone else will steal your lunch"
Well you’re leaving behind both a serious pain point for your users AND you’re leaving in the open a clearly more compute- and money-efficient way to achieve the objective on the table.
It’s literally giving your eventual competitors (because there will be competitors, eventually) a competitive advantage.
Then sure, the market is very wide but… just look at stackoverflow vs chatgpt. As soon as a better alternative came on the market, stackoverflow died to irrelevance within months.
Have you looked at each service running through a cloudflare tunnel or (HE offers something similar too)?
(PS: I use exe.dev quite a lot whenever I want to have a project and basic scripting doesn't work and I want to have a full environment, really thanks for having this product I really appreciate it as someone who has been using it since day one and have recommended/talked about your service in well regards to people :>)
You can get this effect today by installing Tailscale on your exe.dev VM. :)
The reason we put so much effort into exposing these publicly is for sharing with a heterogeneous team without imposing a client agent requirement. The web interface should be easy to make public, easy to share with friends with a Google Docs-style link, and ssh should be easy to share with teammates.
That said, nothing wrong with installing tunneling software on the VM, I do it!
Nice to see this work! I experimented with this for exe.dev before we launched. The VM itself worked really well, but there was a lot of setup to get the networking functioning. And in the end, our target are use cases that don't mind a ~1-second startup time, which meant doing a clean systemd start each time was easier.
That said, I have seen several use cases where people want a VM for something minimal, like a python interpreter, and this is absolutely the sort of approach they should be using. Lot of promise here, excited to see how far you can push it!
The thing people tend to gloss over is how CoW shines until you need to update the base image, then you start playing whack-a-mole with stale memory and hotpatching. Snapshots give you a magic boot, but god help you when you need to roll out a security fix to hundreds of forks with divergent state.
Fast startup is nice. If the workload is "run plain Python on a trusted codebase" you win, but once it gets hairier the maintenance overhead sends you straight back to yak shaving.
Wouldn't you need to restart a process anyways if there's a security update? Sounds like you'd just need to kill all the VMs, start up the base again, and fork (but what do I know).
That is very true. We use copy on write for exe.dev base images right now, and are accumulating a lot of storage because of version drift.
We believe the fix here is to mount the base image as a read-only block device, then mount a read-write block device overlay. We have not rolled it out yet because there are some edge cases we are working through, and we convinced ourselves we could rework images after the fact onto a base image.
Right now our big win from copy-on-write is cloning VMs. You can `ssh exe.dev cp curvm newvm` in about a second to split your computer into a new one. It enables a lot of great workflows.
I’ve been a big fan of “what’s the thinnest this could be” interpretations of sandboxes. This is a great example of that. On the other end of the spectrum there’s just-bash from the Vercel folks.
exe.dev | Full time | SF Bay Area | multiple roles
Support Engineer - If you want to use Claude or Codex (or Shelley!) to trawl through our code base, augmented with (carefully scoped) API keys to make our customers lives better, we are hiring.
Designer - If you want to use Claude or Codex (or Shelley!) to make our product functional and beautiful, we are hiring. You do not ship mocks or assets to other teams. There is no other team. You ship by pushing to production. Don't worry, we've got your back.
Software Engineer - If you see a pattern here, we are hiring. Expect to design, build, and run entire subsystems. What matters is attention to product detail and overall architecture, we have agents for writing code.
With significant industry experience, pay will be over $200,000 with meaningful equity.
We are a small team and going to stay one. The focus is building a high-trust environment. Success for us is if you say "I'm going to fix the load balancer" we all sigh with relief, because you are on it and we can rely on you to take on and solve large projects.
The concern is not losing access to some new IDE for operating outside the terms of service. The concern is when you lose access to the IDE, you also lose access to your 20 year old Gmail account.
A general problem for Google products is that everything is mixed together.
But you also want smart phones, electric cars, and a navy. There needs to be a path towards doing things other than foisting them on people who are out of sight.
Texans seem more than happy to host these industries. Let them, they have no public land left to protect anyway. The environment is arguably California’s most valuable asset. May as well preserve it so people continue to want to actually live here.
Texans often try to regulate these industries at the local level. The state government has tried to put a stop to most of that by passing the Texas Regulatory Consistency Act which took away the ability of local communities to protect themselves. The state has ruled that Texans will be exploited by industry in order to protect profits and the citizens aren't allowed to vote to save themselves.
All we have is the weather? California is the largest agricultural producer of any state, and it's not even close. Plants like growing here for the same reason people do.
You act as though California is no longer one of the largest populations or one of the largest economies.
The “snowball fallacy” is a fallacy because there is no reason California s can’t swing the regulatory pendulum back the other direction if there is too much economy / freedom impacted.
When I took a machining course, the instructor sat in the corner and showed us YouTube videos in Mandarin with English subtitles to teach us the equipment.
You are making lots of projections from a single anecdote and conflating a state’s policies/economy with that of a country of 25x the population.
Detroit was once one of the US’s largest population cities, at nearly 2.5 million residents in the late 1960s, falling to less than 1 million by the 2010s. On this scale, California is still in the peak days of the 1960s, but we aren’t showing any current signs of shrinking. Maybe AI will be the catalyst for massive job losses, but that’s for the future to unfold.
Machining is a low value part of the economic supply chain, like sweat shop clothing. While I don’t want to lose it, it’s being dominated by countries (China, Taiwan) which are willing to throw MASSIVE money at the industry. TSMC was literally a whole-of-country effort to centralize the entire world’s supply chain of cutting edge semiconductors on one island. China is winning because they have cut-throat competition between companies and they don’t slow down for legal concerns such as regulation or intellectual property. That is only going to last for a certain amount of time before people will demand better living environments (which is partly why they have such a terrible fertility rate).
China probably caught up the same way starting 40 years ago. Watching VHS tapes in English (or German, Japanese, or French) with Mandarin subtitles*. Clearly "never" is untrue because it's been done once already.
IMO this is all cyclical.
* This is metaphorical. Obviously there were also textbooks and research papers and technical manuals and everything else. The point is much of it came from abroad and they learned it all to the point that they're the experts today.
Maps of California are dotted with SuperFund sites where these companies left the taxpayers with the bill to clean up their toxic messes. We don’t “foist” these externalities on other people; they choose to hold lower value on a clean environment than regions which regulate pollutants and other negative externalities.
Most of the complaints from this website aren't about things being outright banned. It's mostly stuff where the regulation is so strict that's it's "nearly impossible". But the regulation seems fair to me wrt what's actually required to keep TCE, asbestos, Freon, chloroform, etc out of our soil and water.
Companies that are complaining are complaining that they can't treat the environment as an economic externality anymore in California. Therefore the price of all of these goods are being subsidized with our health and our ecosystems' health.
I hope more of the world follows California's lead and we eventually have a price of these goods that represents what it actually takes to manufacture them in a fair way
But you also want smart phones, electric cars, and a navy
This is kind of disingenuous.
I mean, not everything used in California, needs to be manufactured in California. Why not manufacture it in New Mexico? Or Arkansas for that matter?
What you're implying, is that Wisconsin, Nebraska, Maine, Florida, etc, etc, etc, should all build out the manufacturing base to manufacture things that are used in those states. That's not really how a healthy economy should work.
I guess what I'm pointing out is that, we don't need to manufacture smartphones in South Dakota. It's perfectly acceptable to manufacture them in, say, New Jersey, and then ship them to South Dakota. Similarly, we don't need to manufacture everything in California.
Sure. Everything's built overseas. We all already know that.
Why can't things be be built here, in a state other than California?
What is the fixation with California? Tennessee or Nebraska would love to build smart phones, or even just the chips for smart phones. What is the reasoning for not building these things in other states?
I agree but I fail to see how bad water infrastructure that allows poop to get into the water supply in Mexico has anything to do with this topic. Nobody is arguing that you should be able to spew cancer causing chemicals into the air. It is possible to do all these industrial processes responsibly. It just costs more to do it. So either you can allow businesses to do these things with reasonable amounts of regulation locally or you can prevent those businesses (what CA does) and import these products made somewhere where they won't follow your regulations. And since pollution notoriously doesn't honor borders, perhaps its best not to use simplistic scarecrow arguments and instead have a nuanced understanding of the topic. But don't let me stop your partisan hackery, I'm sure you enjoy it.
> But you also want smart phones, electric cars, and a navy.
I would like far less of all of these to exist than we currently produce (I use a 5 year old phone, an 11 year old car, and think the US Navy could function just fine with a lot less budget and warships).
I don't, because I care about security updates, and I don't want to have to choose between a highly degraded battery and giving up waterproofing.
> an 11 year old car
Crash safety has improved by leaps and bounds in recent years. I suspect you're more likely to be killed in a car accident that you wouldn't be in a new car, than to be killed by one of the industries that California bans.
> think the US Navy could function just fine with a lot less budget and warships
If a powerful adversary goes to war with us, then we'd want a lot more, and only increasing then would be too late, because we'd lose the war first.
Lot of things could be added to this list. Good luck getting permission to start a hospital, or permission to mine/refine anything with a slightly messy process (e.g. rare earth metals). You can't build a new port. The California Coastal Commission won't let you open a new hotel anywhere on the water. You can't even keep a bar open late in San Francisco.
I just searched "new hospital opened in CA" on Google and see that there were two new hospitals opened in Irvine in December, half of a new hospital complex in Santa Clara opened in October, more being built and slated to open this year or next...
Hospitals always take long time, both are non-profit and had to raise ton of money. They are both large multi-building complexes. And I think the UCI one is a trauma center (even more complexity) to deal with the fact that the previous (UCI) trauma center no longer meets earthquake standards.
I do think sandboxes as a concept are oversold for agents. Yes we need VMs, a lot more VMs than ever before for all the new software. But the fundamental challenge of writing interesting software with agents is we have to grant them access to sensitive data and APIs. This lets them do damage. This is not something with a simple solution that can be written in code.
That said, we (exe.dev) have a couple more things planned on the VM side that we think agents need that no cloud provider is currently providing. Just don't call it a sandbox.
It is a good theory, but does it hold up in practice? I was able to prototype and thus argue for and justify building exe.dev with a lot of help from agents. Without agents helping me prove out ideas I would be doing far more boring work.
Hi, author here. I mean the piece of code that calls the model and executes the tool calls. My colleague Philip calls it “9 lines of code”: https://sketch.dev/blog/agent-loop
We have built two of them now, and clearly the state of the art here can be improved. But it is hard to push too much on this while the models keep improving.
the harness being "9 lines of code" is deceptive in the same way a web server is "just accept connections and serve files."
the hard part isn't the loop itself — it's everything around failure recovery.
when a browser agent misclicks, loads a page that renders differently than expected, or hits a CAPTCHA mid-flow, the 9-line loop just retries blindly. the real harness innovation is going to be in structured state checkpointing so the agent can backtrack to the last known-good state instead of restarting the whole task. that's where the gap between "works in a demo" and "works on the 50th run" lives.
We are not running out of IPv4 space because NAT works. The price of IPv4 addresses has been dropping for the last year.
I know this because I just bought another /22 for exe.dev for the exact thing described in this blog post: to get our business customers another 1012 VMs.
reply