My experience with AppEngine over the last couple of years shows that what they are likely doing internally is dealing with years and years of technical debt for things like scaling and deployment. As a reasonably important customers, we had meetings with higher-ups where they'd discuss the work that they had been doing on improving the reliability at both high and low scale which was ignored for years on end.
The past model for AppEngine involved some pretty heavy development work modifying the JVM and Python VM to implement multi-tenant sandboxing. This obviously doesn't scale very well from a development perspective, and shows why new platform versions have always lagged far behind release.
If I were to infer what they are doing internally now, it would be replacing "legacy AppEngine" with "container AppEngine". In a few years you'll see them migrate everyone to containers that will scale in unsurprising ways to users of AppEngine today.
The reason they haven't moved everyone over to containers is that the dev experience for containers on AppEngine sucks horribly right now and they want it to be at least as seamless as it is for Python and Java right now. They also need to migrate 100% of the legacy AppEngine console functionality over to the new cloud dashboard before they could consider this complete.
So my guess is that AppEngine as we know it is "dead", but being replaced by a product that is a pure superset of what we have today.
This seems accurate to me. There are thousands of bugs on the platform that nobody ever fixes. They have yet to add Python 3 support, too. I have been waiting for over two months for a fix to a permissions problem in Cloud Storage that makes it difficult to use (if somebody leaves the org, any files they owned are essentially unusable).
It's deeply frustrating at times. They are often unresponsive on many fronts. I certainly hope they are fixing the platform... or fixing something, at least. That'd be refreshing.
> If I were to infer what they are doing internally now, it would be replacing "legacy AppEngine" with "container AppEngine".
The fact that in addition to support for custom runtimes, App Engine Managed VMs has Google-provided VMs corresponding to all of the existing supported sandboxed runtimes except PHP is, I think, a pretty good indication that Google's plan for official runtimes going forward is to move them to official Managed VMs rather than what you seem to be calling (accurately, I think) "legacy App Engine" -- the sandboxed platforms.
But all the signs are, IMO, there that this, while it may take a while, the same kind of transition that the move off the old legacy datastore was, something that's optional while it the new platform matures, but eventually will be the only way.
It is not a pure superset. With the current app engine, Google takes care of security patches for the language. With container engine, the customer is now responsible for security patches for their language.
Define "operating system"? It's not crazy to think that you might have another layer inside your VM, not unlike how a hypervisor was one layer that didn't exist outside of mainframes until adapted for regular computers.
Not necessarily. One of the ideas behind unikernels is very fast boot time, so that you can start a new instance for every tenant. Another idea is reduced complexity...
I work in Product Management on App Engine, and I'd like to assure you that we're continuing to invest in the product. We already support Java 8 in beta on Managed VMs (https://cloud.google.com/appengine/docs/managed-vms/), and the same is true for Python 3. Once Managed VMs goes GA, we expect that most of our customers will find that it provides all the benefits of App Engine with increased flexibility around the programming language, native code, etc.
What about free tier users? Are you going to send them packing and cause another Reader-esque outcry among developers? There is currently no free tier for Managed VMs.
As much as you may hate supporting them, they will cause a PR headache if you stop.
The more I use it (and see minimial improvements), the more I'm sure AppEngine is a stale project, kept alive by the potential wrath of the internet if they canned it.
[edit] I'll give them credit, They've redone the console. But it's terrible. And you still need the old one to do things such as turn on cloud-storage.
Disclaimer: I'm a software engineer on Google Cloud SQL, not on App Engine, I'm not senior management and I don't have any particular visibility into App Engine that other Googlers don't.
App Engine is very far from dead, either in the "time to turn off the service" or the "it's destaffed but the servers still run" use of the term. App Engine is well-staffed and, AFAICT, is actually growing. They're working extremely hard on getting system reliability to a place where they want it to be. [1] seems to indicate the fruits of that labor, paying off in the last month or so. Now, reliability means reliable for _all_ their customers, from the one person bedroom coder to SnapChat. They have had a big task meeting the requirements of everyone.
I can't talk to the bug triaging system of App Engine nor their future plans. That said, work continues apace to get Managed VMs [2] out of beta, and it seems like a pretty exciting new road which may be a custom better fit for larger companies.
They rarely handle bugs well. The linked issue is typical. They don't even need to do any work to do a whole lot better; they just need to give developers an indication about what to expect. That does mean that they need to make decisions internally, which might be the problem. Potentially, management is umm'ing & arr'ing about whether to kill App Engine.
They recently threw a bit of a contribution in the direction of AppScale, but it's difficult to read that. It could be to soften the blow in the event of discontinuation, or it could be to remove the perception of App Engine as a service with no alternative.
There have also been new features recently introduced, albeit projects that seem like they should have lower prioritization than ensuring folk can use SSL with App Engine, or use modern versions of their programming language. For example, this recent addition: http://googlecloudplatform.blogspot.com.au/2015/02/using-goo...
My reading of the situation is that Google Cloud folk have resources (headcount) to do routine maintenance on App Engine, but not embark on any infrastructural advancements, like new versions of Python & Java, or significant new functionality.
Sounds like their answer for Java8 is probably the same for Python3:
"To echo comment #66, App Engine in fact is not going away. To the contrary, we're investing heavily, and the new hosting environment that is based on containerized virtual machines (what we're currently calling "App Engine Managed VMs") allows you to utilize Java 8 as well as Servlet 3.x."
> They've redone the console. But it's terrible. And you still need the old one to do things such as turn on cloud-storage.
This is no longer correct. As a long-time GAE user, I was one of the last holdouts on the old console. Recently, however, I spun up a large new project - and discovered that I haven't needed (or wanted to use) the old console at all. Including for GCS.
The new console is vastly more usable than the AWS console, and significantly more useful than Heroku's. Just the logging system alone is priceless.
Do you think they gave up on it completely? They haven't updated anything python related for a really long period of time. I have been using webapp2, but the lack of improvements and general care for the framework, makes me rethink if I should continue using it.
Are you talking about the lack of improvements in webapp2 or the lack of improvements in the App Engine Python SDK in general?
webapp2 was started outside of Google by Rodrigo Moraes, who has since moved on to other projects. Googlers have contributed to the webapp2 but have never taken the lead.
There are probably dozens of WSGI-compatible web frameworks that are under active development and you can choose any one that you like.
As a slight aside, we've been using retrolambda[1] in our Android game for a little while now (>10M downloads), and it's been fantastic. There's an "Auto convert to lambdas" in Eclipse that shrunk our code down nicely, and then we've transitioned into designing our code _for_ Java 8, and I'd argue it's made things much easier to work with.
The only niggle is that when you set your IDE to Java 8 (be it Eclipse or IntelliJ) it exposes Java 8's new methods - and allows those on our desktop machine builds. We've been caught out a few times as it compiles just fine for Android, then explodes at runtime on device - but never anything that didn't get caught in QA. If you just want the lambdas and method referencing, it's wonderful.
Curiously, we do get crash reports coming in from select Android devices that still don't have some of the standard library from Java 7 implemented. That catches us out in production more often but, fortunately, it looks like >99% have no issue with 7.
I'm glad to hear this works well, as I was doubtful of adding another layer of complexity (I'm not using an IDE). I just started learning the Android API and working on an app idea, and I was really surprised when a lambda expression I used failed to compile. I was even more surprised when it wasn't just a configuration problem on my end.
The tricky part was the integration really. We're using Eclipse (our project is getting kinda old), and we can deploy to a PC or device.
We use Gradle to do our real builds; CI and to devices at our desks. And we just use Eclipse set up to run with Java 8 for everything else. Once it's setup, it's pretty seamless, we haven't had to think about it for a long time.
I am confused by this tool. They talk about taking all the invokeDynamic instruction into old style anonymous classes. I had assumed that the invokeDynamic instruction was introduced in Java 7, its not clear if the transformation is really necessary on Android. Am I making a terrible assumption here?
Just to make things awkward for ourselves; we have tools that _do_ use real Java 8, as do some of our unit tests :)
It's not so bad, I was just trying to come up with negatives -- we've made the mistake of pushing something to a device with the bad APIs perhaps 4-5 times in a year.
It seems like the Java community isn't too concerned with JDK7's EOL. I recently did a survey of Java 8 support for the JVM services we use and only one service explicitly supported JDK8:
* Cassandra - supports JDK8, docs only referenced 7 (I contacted DataStax)
* Elasticsearch - explicitly mentions exactly what JDK 7 & 8 versions are supported. Yay!
* Zookeeper - Only mentions "JDK 6 or up" so... yay? Seems to work as promised.
Point being: Only 1 out of 4 major Java services even bothered explicitly mentioning Java 8 support (ES) -- and it doesn't even discourage use of the EOL'd 7.
Java is a complex ecosystem. I think a JDK being EOL'd means something different in the Java ecosystem then it does to the rest of us.
Java puts massive effort into backwards compatibility; (almost) anything that runs on JDK7 will run without modification on JDK8. And "big" frameworks naturally try to be compatible for as many users as possible - if you look at smaller one-person libraries you're more likely to find JDK8-only code.
The problem is that bugs in the JDK itself will no longer be fixed in v7; Oracle's line is (reasonably) "you should update to JDK8, all your code will work fine".
There's a subtle distinction here... It's pretty common for a widely used Java library to be compiled and/or coded against an older JDK version e.g. JDK 5 or 6. This is seldom an issue as most well written libraries written using JDK5 or JDK6 will work perfectly with and have been tested on JDK 8.
I would definitely NOT say that the Java community isn't too concerned with JDK7 EOL. No responsible developer is going to run a network-facing application on an EOL unpatched JVM as it's a security risk (unless you fork out for a support contract for older versions).
> Java is a complex ecosystem. I think a JDK being EOL'd means something different in the Java ecosystem then it does to the rest of us.
JDK Forward compatibility makes this mostly a moot point for the lib developers. Upgrading your JDK is your own responsibility, the rest is expected to just work.
I guess that either the complexity of large updates (like JDK 7 to 8) is high or that Google aren't investing many resources into GAE. The release notes (or lack of) says a lot:
Given the focus on other cloud products (like being able to run Docker images) I would be surprised if Google continued to add new features to GAE. It seems like it's growing cobwebs...
I actually like GAE as a concept, but I had a very bad experience with their Google Cloud endpoints: bad documentation, bad tooling and bad bugs. Even when I got it up and running to implement a custom authenticator was a mess and the performance wasn't what I'd expect. This got much better, especially with the official Maven plugin: but I'm really concerned that Google aren't 100% behind GAE and development could cease at any moment.
An official statement would be good. But like so much at Google it's not forthcoming.
I've been using GAE for 6 years now and while the pace of supporting/fixing stuff has always been slow, it has indeed been on a steady decline ever since Compute Engine got announced.
GAE for me is a nice lesson to not rely on closed source solutions, where I can't go and fix mission critical stuff myself. I've got confirmed bug reports over 4 years old that still aren't fixed [1], and I got serious deadlock bugs on Windows over a year old, that got applied low priority with the recommendation to use another OS. [2]
In general I really like GAE, but the complete lack of support & dedication from Google and the shifting of resoures to other products makes me sad.
It's not the same thing exactly but you might want to check out AppScale which is an "Open Source Implementation of Google App Engine": https://github.com/AppScale/appscale
Do people really use App engine? I mean that is the most limited cloud platform of all, its interface is just horrible and confusing and it supports just a handful of technologies with drastic limitations. Who in their right mind would want to pay for this? It is a bad product, and the very definition of vendor lock-in. I'm curious about actual users.
Yes. I'm one of them, and I keep coming back to it. The short answer for why is that no matter what you think of it, GAE delivers on one basic promise: No ops/devops. I've scaled applications to thousands of QPS with teams of one or two people, and I never even had to adjust the # of instances (scaling is automatic). I've never had to worry about the database getting overloaded. It's just "write code and deploy".
Out of curiosity, what cloud provider interface do you think is better? Certainly not AWS. And which limitations do you consider so drastic?
What you mention is what I love about it, too. What I don't like is the uncertainty; feels as though Google may kill it at any moment, and they do a really poor job of working on features that people care about. (SSL on naked domains, modern language support, etc.)
I get the impression that Google has been putting a lot of resources into GAE all this time, but mostly into retiring technical debt and integrating it with the new expansive "cloud platform" vision. I also get the impression that nothing happens quickly in Google-land :-(
That said, even "as-is" without any major changes on the horizon, GAE is pretty solid. My only major complaint right now is Java8, but given the choice between suffering with Java7 for another year and hiring an ops staff, I'll pick the former. I had to make "the platform" choice again just a couple weeks ago for a new startup, and still picked old-style GAE - not even the new Managed VMs, which I don't think are reliable enough yet.
Technically, there are vendors other than Google. Red Hat have an implementation called CapeDwarf. There's also AppScale. There was another called TyphoonAE, but I haven't noticed any signs of life from that one in quite some time.
The big unknown for me is how usable any of those implementations are, and whether there is such a thing as a decent migration path (for data) from Google to them, and vice-versa.
For a system that serves up webpages and has a decent database back end, it works and works well.
The overhead required to fit what you are serving to fit within App Engine isn't major. With a lot of other systems you have a lot of infrastructure type issues that you have to deal with. What it costs on setting up App Engine, you save on configuring a different system.
You are not going to get App Engine to do what you would normally use for a server and yes it is limited.
I have two web apps running on App Engine since a couple of years ago. It's nice enough, but I wonder why Google isn't doing much with it. App Engine could be positioned as the full service PaaS here you don't have to worry about load balancers and all that.
Google has also broken backwards compatibility multiple times, forcing me to spend time just to keep the apps running.
For your really small hobby app.. its a pain sometimes to deal with all the config.
For a real app with real traffic, you want to scale in different ways. What you can do with dockr and a bunch of VMs gives you a TON of flexibility.
I know a few people with apps on App Engine.. I used it myself for a toy project. It's "ok".. but compared to buying a $5 digital ocean VM and getting a real server for less money? Makes no sense if you know linux at all.
The rationale behind App Engine is very valid, though. You are one person, or perhaps a small company focused on developing particular software. Can you do as good a job as Google's App Engine's security team at keeping your servers securely running your application? That includes potentially urgent patches to the kernel, libraries, web server, etc.
Even if you can match Google's efforts in doing all that, why would you want to waste your time doing that maintenance work rather than working on your application itself?
Docker VMs are an appealing concept to me, but it occurs to me that I would be dependent upon some third-party that I know very little about to release an updated image whenever something in the stack needs to be updated. Additionally, I need to be involved in that deployment. With App Engine, Google have my code, so they update the image and my code is always running in a VM that ought to be far more bullet-proof than I could manage. (even with some Linux knowledge)
> That includes potentially urgent patches to the kernel, libraries, web server, etc.
but given that java 7 is getting EOL'ed, there will be no more security patches for it. So how do you expect their jvm is secure? The basis for google's ability to patch better than yourself is pure faith.
Google's execution of App Engine has been sloppy and half-hearted, but the concept makes sense, doesn't it?
That said, I'm assuming their JVM is still secure despite being old. Oracle are EOLing their Java 7 JVM, but that doesn't directly affect App Engine unless that's the JVM Google are using. The indirect effects are inexcusable, though. That developers need to stick to an old language to use App Engine while the rest of the world has moved on is ridiculous.
I thought the whole appeal of App Engine is that you dont get a real server and thus dont have to worry about managing them. It seems like if you prefer a real server you aren't the target market.
It's just a matter of using Linux: you must know also about setting up a database server, replication, load balancing, security, logging... things that App Engine gives you for "free".
2. Google's given what seems to me to be a pretty clear signal that managed VMs are the future of app engine; if expect new official runtimes to be delivered that way once managed VMs are out of beta.
Seems that managed VM's are the future of App Engine. Looking at their feature matrix, it appears that the primary benefit of old-sandboxed apps is that they launch within milliseconds, versus minutes in managed VM's (https://cloud.google.com/appengine/docs/managed-vms/). I'm only now learning about their capabilities. From what I understand, they allow arbitrary environments using Docker files. How else are they different from normal compute engine VM's? Is it because they are spawned and load balanced? Does that mean they are more analogous to Docker on Elastic Beanstalk?
I have several appengine projects. The java ones need 20+ seconds to startup. Python ones need two seconds to startup. Golang ones need 0.5 seconds to startup.
My later appengine projects wlll be all golang based.
Dalvik (the android JVM) is supported by Google, not Oracle; it's up to Google which versions they support. (For people who want to write code that works on both Android and desktop Java, remember that Java 7 code runs fine on a Java 8 JVM).
Basically Oracle no longer freely supports their version of Java7. For Android, Google is responsible, and as an end user you don't get any support at all.
If Google isn't going to support future versions of Java, either for its cloud service or on Android, then at least it should be making an aggressive push for Go as its replacement. Right now it's kind of neither this, nor that, and it's falling behind in both (including on Android, where Apple is making a strong push for Swift).
ugh, i smell an Internet Explorer standard compliance story approaching in the next year. Not just the app engine but also android:
Big player stays on old version for years, standard is moving forward with tons of long awaited features, nobody can use those new features because supporting big player is too important.
Software development in a nutshell, these kind of stories just keep repeating, python 2vs3, C99 in MSVC, etc etc. I guess i should just stop pretending that a new version has been released until 5 years after the actual announcement.
Java versions are 100% backwards compatible, or close enough. I think you're asking them to make Java forwards compatible..? If so, no, that doesn't make any sense.
Code written for Java 7 runs perfectly on Java 8. Code that uses new Java 8 features doesn't run on Java 7 though, because those features didn't exist then. You upgrade the Java runtime to Java 8 and then it supports the new features. Simple, no?
Could you elaborate on which issues you ran into. We are in the process of upgrading to from jdk7 to jdk8 (at-least for the services we see having a long lifespan) and it would be great to hear some real world experience.
Java 6 was released in 2006. Java 7 was released in 2011. Java 6 was EOL'd in 2013. That seems like a long enough timeline for people to have migrated sensibly.
I'm crying laughing right now. Businesses rarely see the advantages of upgrading something "Already Works". Now you could agree (and be right to a point) that it's the responsibility of developers/ops/etc to convince the business side but in my experiance the people writing the code are far enough detached from the people making decisions that they have next to no voice. My last job I saw an upgrade from Drupal 5 -> 7 and currently I work on framework that was last actively developed on 5 years ago (never versions have been released, they are now 2 major versions ahead of us which is a lifetime in PHP as of late with all it's changes). A lot of times you come into a place that is so far behind that to move up to the newest version would be equivalent to a full re-write (whereas moving up over time would have been possible had there been a bigger push for it).
I don't know if that contradicts what I said. If your upgrade strategy is "never," then no matter what the long-term support window is, you're going to go over it. If it's "only when I have to," making one obvious marker for when one might "have" to sooner rather than later keeps some people from falling so far behind that "to move up to the newest version would be equivalent to a full re-write."
2 years is not a long time to migrate large system that works fine except for minor maintenance. It isn't just time, it is finding budget for churn work.
I had one experience in java6 to java7 migration. The issue was a partly overriding of a list implementation. That list implementation was used in a for loop. The way the for loop was iterated across over was different in the two java versions.
e.g.
for(Object a:list){
a.doSomething();
}
In java6 it was equivalent to
for (int i:i<list.length();i++){
list.get(i).soSomething();
}
In java7 it was done like this
Iterator iter = list.iterator();
while(iter.hasNext() {
iter.next().doSomething();
}
The original programmer had overridden the get method but not the next method in the iterator method. (Actually just called toLowerCase on a list of strings).
Oracle did nothing wrong, the programmer only partly overriding the list implementation was broken. This kind of programmer error is too common and is normally the reasons that upgrades become an issue, i.e. its the equivalent of using undefined behaviour in C and then scream when your compiler upgrade burns down your house.
Ah, this is why I never see these kind of issues. He subclassed a list just to modify the list to return lowercased output always, preventing any method of ever accessing the original multi case text.
At any rate, all of the java 6/7 code I've ever had has ported to java 8 without any changes. Assumed it was the normal experience.
In my case in 500,000 lines of code or so it was the only example. Nothing like the stuff I found in a different job going from 4 to 5! The enum keyword was painful! and then we found class names in the database instantiated at runtime with reflection once the migration was "finished". Some programmers are mean to those who come after.
I've run into a few, but they've been mostly of the kind where a library/framework infers JDK version and assumes 1.8 is < 1.4 or whatever is the earliest they support.
I mean... sure. Oracle doesn't OWE me anything. They can do what they want. But if they want a positive relationship with their community, they could not break compatibility with every Java release.
I don't know if they care. At some point, people will rebel, but I can't predict where that tipping point is.
Just no.
Their hesitancy to not break backwards compatibility is why Java got so stale. They have gotten better since 1.5 but I would prefer if they broke it more often.
You are talking about Sun really.
But Java got really stale when Oracle bought Sun and no one cared about Java language for a few years. That's why 7 was so late and had almost no features. Everything in 8 was supposed to be in 7.
Declaring a product "end of life" is utterly meaningless to every entity that does not pay you to support the product. You can declare whatever you'd like; people will stop using it when they're done using it.
No, this is still very much meaningless. "Java 7" is a platform specification. Oracle has declared they will not develop their implementation further. There are other implementations, none of which are affected by any EOL announcement by Oracle.
Outside of specialty use cases, OpenJDK and Oracle JDK are effectively all that exist for mainline, non-Android Java. OpenJDK 7 is going to get very few patches without Oracle working on it. And a rounding error's worth of people are going to switch to another JVM because Oracle doesn't support Java 7 anymore--they're either going to sit on an unsupported VM or upgrade.
So, yes, it is important to the vast majority of using Java except maybe you and that's great but you're being That Guy.
> they're either going to sit on an unsupported VM or upgrade.
No, they're going to just sit on an unsupported VM, which was my original point. All the theater about pretending that Oracle have any effect on these decisions is ludicrous, no matter how huge a majority you pretend to speak for, nor no matter how much ad hominem you ladle onto your replies.
The past model for AppEngine involved some pretty heavy development work modifying the JVM and Python VM to implement multi-tenant sandboxing. This obviously doesn't scale very well from a development perspective, and shows why new platform versions have always lagged far behind release.
If I were to infer what they are doing internally now, it would be replacing "legacy AppEngine" with "container AppEngine". In a few years you'll see them migrate everyone to containers that will scale in unsurprising ways to users of AppEngine today.
The reason they haven't moved everyone over to containers is that the dev experience for containers on AppEngine sucks horribly right now and they want it to be at least as seamless as it is for Python and Java right now. They also need to migrate 100% of the legacy AppEngine console functionality over to the new cloud dashboard before they could consider this complete.
So my guess is that AppEngine as we know it is "dead", but being replaced by a product that is a pure superset of what we have today.