The minimum would be 100 conversions per month below which we cannot perform experimentation. Obviously better works better, and 500 would be better. But even without experiments, we have default curves which you can use. Regarding A/B test, you do not need to A/B test in the same country, you can always perform experiments across countries and get 100% of the traffic of a country to the same experiment.
Thank you! this might be interesting for us. It won’t be real A/B tests across different countries though unless they are extremely similar, right? our service is also seasonal and different countries are on different schedules so it makes even more of a difference I imagine.
The way we operate is by putting countries on a curve based on their characteristics (PPP, tech salaries etc.) and find which curve maximizes your KPIs. When you say seasonal do you mean yearly seasonality? I'd definitely be curious to hear more! My email: abel@corrily.com
Regarding your last point on feeling cheated, there are two aspects to it. Right now if you check the price of Slack with an indian IP, you will see $2.67 for the lower tier (6.67 for the US), Netflix will give wildly different prices per country [1], and Survey monkey gives me the same USD price in Euros. People expect to pay different prices in different countries.
The other aspect is to prevent a user from receiving different prices. To prevent that we do a few things. We remember the prices that were displayed to a user and can make sure an entire country gets the same prices during an experiment.
I would be very worried about customers finding out that other people in the same country got offered different prices. Here's an example of a story from the Atlantic[1] that describes it in a very poor light. Making sure someone always gets the same price on refresh doesn't hold up if they are browsing at a friends house for example. I would be very worried about getting shunned for price discrimination.
In my experience, airline “price optimization” is wildly overblown a boogeyman tale.
Airlines price classes of seats individually. If one is sold, the next user sees a different price. A rule based system is fairly common. They’re not using RL or dynamic pricing algorithms. In fact, I’d wager most don’t have the technical expertise or even data (with the rise of meta-search) to do so.
I can confirm that some customers are apparently very keenly aware of prices. I've recently been experimenting with my prices, and 2 days after lowering after going way too high, a customer sent me a message asking why they were paying more
> and Survey monkey gives me the same USD price in Euros.
European prices usually include VAT, while US prices don't include state taxes, since they are applied at checkout based on your location. That ends up being the same price in the end.
Just checked here from London, Survey Monkey's "Team Premier" prices are $75|£75|€75 per user per month [1]. I don't think the reason behind the price differences is whether they are VAT inclusive/exclusive?
So I am super interested in your comment and I think you will be a bit surprised by my response.
I am not a strong believer in capitalism. On the contrary. Neither is Andrej. We both grew up in leftist families and I must have been the only portfolio manager that reads Das Kapital at night. We actually came at the problem from a very very different direction. We thought pricing everything from a US perspective was problematic. The price levels in India are about a 10th of the US.
So we thought "Ok but that doesn't even make sense for a SaaS to price the same way everywhere. Because they lose out on poorer markets". We built Corrily with the thought that more personalized pricing based on an individual's willingness to pay also reflected an individual's capacity to pay.
We think it's pretty cool to align a business incentive (reach more markets) with an economic one (poorer nations and richer nations paying different amounts).
I actually wonder a lot what other people think about this because I think it is an important point. Is it fairer to price differently according to willingness to pay given not everyone has the same purchasing power, or to give the same price to everyone?
That's a really neat quickstart! We actually allow to go outside of the Stripe integration and have an API to ping purchases here (https://doc.corrily.com/openapi/reference/operation/setSucce...)... But I wonder how to make it more user friendly. We did get a js SDK, but it wasn't very neat. Do you have any thought on what UX would make the most sense?
We currently handle VAT for services (in this case we pass the correct VAT to send to Stripe via their tax rate API [1]). We have not yet expanded it to physical goods as they are a bit trickier to manage from a tax perspective. However you make a really good point on exploring this. I'd be curious to hear more about your experience actually, what were the biggest pain points you faced when growing internationally?
Full disclosure: This experience is mostly gathered from a side business run by a small team in a niche market over a decade or so. I am probably a lot more “by the book” in how I like to run my businesses than your average startup founder. My personal experience and views may be atypical.
Adding VAT to bills at local standard rates is only one small aspect of VAT compliance. You also have the actual reporting and remittance processes everywhere they apply. You might have some complications if any special rate applies to your product or service. Then there are the perennial questions of jurisdiction and extra-territorial enforceability, which I will leave to the lawyers and accountants because they make my brain hurt and I just want a definitive answer to what our true obligations are so we can comply with them. The worse part, particularly for a smaller business with no dedicated staff to work on this stuff, is how often the situation changes, sometimes at very short notice, and with no central authority you can monitor or that will notify you when you need to act.
I have spent far too much time over the years updating our systems to get the implementation as close to exactly right as we reasonably can. I have spent even more time just watching things and trying to keep up with what our obligations were. The opportunity cost of that time alone must have been several orders of magnitude greater than the total difference in taxes ever collected and remitted as a result. That was, with hindsight, an absurd way to run the business. Being by-the-book, we should probably have just stopped all sales to the rest of the EU, which has only ever been a relatively small market for us, as soon as they introduced the more onerous VAT rules. (No doubt many businesses instead took a pragmatic view and simply ignored the rules and didn’t comply, and I suspect few of them have suffered significantly for it in practice.)
That is a lesson learned, and it’s not a mistake I will make again. Fortunately, we don’t have to, because there are now several services that will collect payments but also handle sales tax/VAT compliance transparently. Their value proposition appears compelling: we can sell everywhere, be fully compliant, and not be responsible for maintaining the implementation, all at the same time. So these services are where we are looking as we start the new business, and time will tell whether the reality meets expectations. I imagine that if we do settle on one and we are still happy with it after using it for a while, we will start using it for other businesses any of us run as well, and services like Stripe will move to a legacy role and ultimately be phased out entirely.
there are now several services that will collect payments but also handle sales tax/VAT compliance transparently
An interesting case of privatizing the enforcement of government regulation! But yes it does sound like a compelling value-add; note that this value is required by any world-wide marketplace, even the small ones, if they don't want to be at risk. It kind of sucks that VAT is like 20% but that's still not enough to pay for a public version of one of those VAT compliance services.
Just my personal opinion now: I think VAT as a form of taxation doesn’t fit into the modern global economy very well. VAT has long been criticised as a regressive tax anyway, but the basic principle of calculating value added by looking at the difference between the incoming and outgoing columns and then taxing that amount becomes difficult both to implement and to practically enforce if those columns fall in different jurisdictions. Then you end up with additional measures for cross-border trade that cause headaches for businesses and consumers alike, and you can create perverse incentives that punish businesses for trying to comply while rewarding those that do not with a competitive advantage. At the same time, we do live in that global economy now, and enforcing VAT domestically if you didn’t try to impose analogous rules internationally would also cause a big competitive disadvantage for domestic suppliers.
I don’t know how you solve those problems without scrapping VAT entirely, with some profound implications for your entire tax system given how much tax revenue governments currently bring in this way that would presumably be shifted to other sources. (I can’t help wondering whether removing VAT would also increase productivity enough by itself that it would offset at least some of the lost tax revenues automatically.) Whatever the plausible alternatives might be, it seems obvious to me that the current system doesn’t work very well for anyone.
I'd be curious to get examples of actual successful lawsuits resulting from A/B testing!
Regarding the legality of it, in the US it is illegal to perform price discrimination when the intent is to directly harm your competition ([1], [2], [3]). However, first, Robinson-Patman does not apply to services [4], and secondly this is trying to avoid predatory pricing, and the supreme court ruled that price differentials are prohibited when the price differential "may be substantially to lessen competition".
Similarly in the EU, Article 102 c) of the Treaty on the Functioning of the European Union prohibits price discrimination for companies in dominant position. It has however been significantly relaxed in United Brands v. Commission [5], where the court recognized that a dominant firm may charge different prices to reflect the competitive market.
In short, price A/B testing is legal and common practice for most companies, and VWO's article seems a bit too quick to jump to declaring it illegal.
You might also like to look into the EU consumer protection rules, particularly the Consumer Rights Directive and its respective implementations in the member states and the UK. There is quite a lot of specific information that must be provided to customers during the buying process, and there are some serious penalties available if a merchant is found not to have done so.
In particular, if a customer might have seen multiple prices for the same thing around the time of purchase, for example on a pricing page checked on a phone and then on a checkout process completed on a PC, you would surely want to have solid evidence that the actual price charged had been clearly understood and accepted at the time of payment.
Absolutely! I remember reading this paper [0] about personalized pricing. The fact that personal data (an IP) is being used to inform prices has to be disclosed in the GDPR disclosure of a website. A very simple way to avoid a lot of those issues however is to have time-limited experiments in which 100% of Country's flow is directed towards one experiment (in the case of Corrily we keep track of which user saw what price when to insure the user will always see the same price).
The usage really differs depending on the size of the company. For larger companies, they tend to have entire pricing teams and the pricing is never "Done". In this case we are adopting a subscription-based model and are trying to optimize prices in a very granular fashion (per country, tradeoffs between usage-based and subscription based fees etc.). Those customers tend to need experiments for the rest of their company's life.
Then there are smaller companies. Think Seed or Series A companies that focus on getting prices in the right ballpark. For them, we provide a lot of value at first by finding the right price (and country adjustments) at first, and then provide a bit less value when they found prices that seem to work well. We adapted our pricing to reflect it with a low-ish subscription fee to maintain our infrastructure and show localized pricing to their users, and an experiment-linked usage based fee. We're happy with this system as it aligns well with the value we are providing.
Later we will add dynamic promotions (think personalized time-limited promotions based on the user's usage of the app, or country holidays promotions) to increase conversions and help companies be local at scale, which should help us generate ongoing value, even to smaller companies.
