It's about control of preloaded content, specifically. If you don't care about your content being easily preloaded from the SERP page, then sure, no need to use AMP. And if, as a user, you don't care about quick access to preloaded content (because e.g. you're on a low-latency connection), feel free to scroll down from the top-stories carousel.

Preloading is a waste of data on mobile. If you're on a metered connection then this costs more. Ranking sites by performance would already ensure a fast UX for users.

Also position on the SERP page is a major indicator of relevancy. Biasing results because they use AMP is unfair and inaccurate.