The downside is the prune command is a giant hammer. In a lot of cases you don't want to clean everything, just things you know you won't be using. For instance, in a CI environment there's going to be some images that will always be there, but if you run a prune in-between jobs you've now made it necessary to re-pull all images.