We use it extensively, essentially for anything that doesn't need to happen "just this second". It's also replaced everything we used to do in cron, as it easy to carry around in source control.
Most of our emails, calculating expensive things (we have some match-making algorithms that run as tasks), sending notifications and all sorts of fun things are all done in tasks. By moving as much of our code into celery as possible it's been much easier to scale (adding a new node is trivial) and our pages are much more responsive.
That Celery makes it all so very easy to do is icing on the cake.
I've set it up for a project where incoming HTTP requests can trigger outgoing HTTP requests, something I found useful was a wrapper function that attempts to execute the task asynchronously (the_task.apply_async) but falls back on running it in-process (the_task.apply). This way if RabbitMQ isn't running everything still basically works. The downside is that if RabbitMQ is running but celeryd isn't, all the tasks just pile up in the queue.
Most of our emails, calculating expensive things (we have some match-making algorithms that run as tasks), sending notifications and all sorts of fun things are all done in tasks. By moving as much of our code into celery as possible it's been much easier to scale (adding a new node is trivial) and our pages are much more responsive.
That Celery makes it all so very easy to do is icing on the cake.