The day before, Discord did high-quality a write-up on why they chose Cassandra [1], and now this post hits explaining how one of the world's most popular and trafficked service has engineered their infrastructure; it's like a dream.
I'll echo the praise I wrote earlier, that insights like this aren't only some of the best content to hit HN, but become some of the most valuable resources for designers who have yet to face a scaling issue, but know they will soon.
Since you have developed custom layers on top of open-source software to fit your particular usecase and load profile, and host all this in-house, have you considered monetizing your infrastructure for outsiders who may have similar needs?
Today, one has limited, unpleasant choices: either pay out the nose for something like AWS or Google Cloud to get elastic scaling and the captive storage systems that can be made to handle these kinds of workloads and still have to write a fair bit of custom glue to get all pieces to play nice, or you can build out the servers yourself, but have to employ dedicated talent with the requisite expertise. Either way, the barriers are fairly steep; you could tap into an under-served market should you choose to sell IaaS (edit: or, more accurately, PaaS). Has this conversation come up in the past?
I'll echo the praise I wrote earlier, that insights like this aren't only some of the best content to hit HN, but become some of the most valuable resources for designers who have yet to face a scaling issue, but know they will soon.
Since you have developed custom layers on top of open-source software to fit your particular usecase and load profile, and host all this in-house, have you considered monetizing your infrastructure for outsiders who may have similar needs?
Today, one has limited, unpleasant choices: either pay out the nose for something like AWS or Google Cloud to get elastic scaling and the captive storage systems that can be made to handle these kinds of workloads and still have to write a fair bit of custom glue to get all pieces to play nice, or you can build out the servers yourself, but have to employ dedicated talent with the requisite expertise. Either way, the barriers are fairly steep; you could tap into an under-served market should you choose to sell IaaS (edit: or, more accurately, PaaS). Has this conversation come up in the past?
[1] https://news.ycombinator.com/item?id=13439725