Hmmm, maybe we're reading different papers. Ok, I admit that a lot of papers compare only against other metaheuristics, but only in cases where it is already accepted that those other metaheuristics far out-perform weaker methods.
About NFL, I agree that a lot of authors misuse it in that way. But again, in most papers in good journals, what we see is either a comprehensive experiment with a wide enough range of instances, or a real industry problem, not cherry-picking.
> Ok, I admit that a lot of papers compare only against other metaheuristics, but only in cases where it is already accepted that those other metaheuristics far out-perform weaker methods.
Those methods are invariably other evolutionary methods, and benchmarks are cherry-picked to show only encouraging results under the convenient guise of the "no free lunch" theorem. That' pretty much the norm, such as the repetitive recipe for inventing a metaheuristic algorithm of a) coming up with a clever nature-inspired metaphor with a catchy name, b) put together an algorithm that is arguably inspired on the metaphor, c) come up with a benchmark that arguably portrays the algorithm as being any improvement, even if only on a fortuitous corner case.
No, I'm using "weaker methods" in the technical sense, not "performs worse".
I agree that many papers of that recipe type exist -- Sorensen and Weyland have skewered them effectively -- they are just froth, to be ignored in a discussion of the true merits of evolutionary computation.
About NFL, I agree that a lot of authors misuse it in that way. But again, in most papers in good journals, what we see is either a comprehensive experiment with a wide enough range of instances, or a real industry problem, not cherry-picking.