Will the code be theoretically runnable elsewhere though?
I guess my two concerns about solutions like this are always A) vendor lock-in B) whether it's too big of a project for any one company and requires a community of contributors to realize the vision.
In the short term it doesn't really matter because clearly being able to build an app is a huge step up from not being able to build an app regardless (which is why companies like Bubble.is are doing so well), but in the long term it seems like there are 10 - 15 companies working on this so eventually there will be more options and those factors will be key differentiators.
Our goal is making it super easy to build and run backends, and so if the first step was "install kubernetes" I don't think we'd have realized our goal. We don't have a goal of running these apps elsewhere. However, we don't like the lock-in either, as it's a blocker to people trying Dark, so we're trying to figure out ways to avoid that.
A current thought is perhaps allowing you to export your application compiled to Node or similar if you decide to stop using Dark. (Data is already easy to export, so we're mostly worried about code).
This is probably going to be the next blog post I write about this, because it's pretty front and center to the business.
This is the first time I've heard the concern of the project being too big for one company. I personally don't worry about it because it's so hard to move a small team in the same direction that I don't think a large community of volunteers will be a better fit.
Yeah I agree Dark should definitely run apps by default. And just to be clear I'm guessing that currently vendor lock-in is probably mostly a concern for folks who already know how to code, who wouldn't necessarily use it anyway if that specific blocker were removed. E.g. since I'm already a full stack developer, the things I'd be looking at are more like whether it would make hiring easier rather than whether it would be better than hiring a dev shop to build an MVP.
I guess another option would be having Dark function more as an app-store-like intermediary between the hardware and software. E.g. telling users they need X, Y, and Z infrastructure and letting them deploy it to whatever cloud they wanted, and having dark just charge for orchestrating that and for providing whatever other services.
In terms of a community, I guess I meant more in terms of getting developers to create libraries for data processing, in terms of things like HTML parsers for scraping webpages and other stuff that is essential for choosing a language but where it's hard to see how it would be viable for one company to build out all that stuff. Unless the plan is just to let folks run code in other languages.
> mostly a concern for folks who already know how to code, who wouldn't necessarily use it anyway if that specific blocker were removed
Yes, I think that's right. There's a lot of folks who have strong feelings about this but who aren't necessarily our target user. I know that's going to cause a lot of problems but what can you do :-/
> I guess another option would be having Dark function more as an app-store-like intermediary between the hardware and software.
Yeah, this very much isn't what we're trying to build, or the business we want to be in :)
> In terms of a community, I guess I meant more in terms of getting developers to create libraries for data processing
Gotcha! Yes indeed, there will be a package manager very much like npm/cargo/rubygems, for just this sort of thing. We plan to make it extremely easy to create and share packages.
> I know that's going to cause a lot of problems but what can you do
That's a good point, in the sense that even if less technical people don't care about certain issues, they're often not going to adopt the product without first soliciting advice from people who do care about those issues. So that just means potentially needing to build out a lot of stuff knowing that people won't actually use it for a long time.
Sort of like our product and the OAuth issue, which we're finally on track to (mostly) solve with a Gmail add-on but it's obviously been several years at this point.
We haven't decided on transactional consistency yet - if there are particular goals there, I'd love to hear them. Ostensibly, we want to allow users to design the right distributed system for themselves, and provide ways of saying "we want this DB to prioritize availability over consistency", for example. But we haven't yet decided how to do that or what to support at launch, nor where the bounds of a transaction will be.
We have a DB migration tool. The DB has a current type, and you specify the new type, and add rollforward and rollback functions. That creates a new name for the datastore: any use of the old name uses the old type, and the new name uses the new type (both on the same datastore, applying rollforward and rollback as necessary for the interface you need). Once you've removed all the references to the old name, you mark the migration as complete, and we'll handle the underlying transformation of the data behind the scenes.
I think operation level consistency policies are more useful than DB level. For instance, saying bounded stale reads are OK for some operation could be useful for the high read scenarios and hopefully the system will materialize a cache.
Gotcha. Yeah, we're planning on allowing users to tweak their performance by providing this sort of constraint to the system, and allowing the system optimize it. We are far from that, but that's where we're aiming for.
> In Dark, making an API call is as simple as a function call. Our tools for building connectors to 3rd party services handle rate limiting, authentication, error handling, and retries, with great default behaviour. Visibility around costs and failures are built into the editor, as is support for secret keys and Personally Identifying Information.
The history of RPC is...checkered. It's possible that every attempt to make remote process calls as simple as local function calls has resulted in terrible abstractions that leak like a sieve.
My suggestion (for what it's worth) is to bake the notion of message passing into the language, and make it as simple as possible.
I'm talking about using 3rdparty APIs, like Twilio or Stripe or something, that we don't have control over and that we need to handle. So we want it to feel like calling functions, though in practice it needs to be able to handle all the stuff that goes wrong.
Can you link to something about message passing? I'm familiar with a concept with this name, but I don't quite see what you mean.
I don't think it's really similar to Eve at all, except that it's an outside the box look at how we code. We're extremely focused on the backend coding use case, and solving known problems. I think Eve was on the path of experimenting to come up with a new way of doing computing, while we basically know what we're aiming to build already (and mostly have for over a year now).
That said, it has a lot of similarities to LightTable, especially regarding coding with live values.
Great! That got us our first two customers, and it allowed us prioritize really effectively for what they actually needed. And it cost us almost nothing, like $15k total - a great investment :) Would you read a blog post on this?
Probably September, though no promises. We're still redoing some core metaphors. Codewise, you can expect it to look similar to Elm/OCaml - functional/imperative language with structured error handling and no null.
What's the business model? E.g. open tooling, pay for hosting/monitoring/optimization?