Interesting tool. I recently got my Rails 2.3 app running in 1.9.2. This utility wouldn't quite cut it though because most of my effort was devoted to getting remarkable_rails working just so I could get my specs to run.
After all that effort, I found out I'm better off sticking with REE. While my specs all did pass, they took 3 - 4 times longer to run in 1.9.2. I ran experiments on that for about a day, not believing the results because I've been told repeatedly that 1.9.2 is unequivocally faster. Alas, that is not the case. So, that's my "excuse" for not running 1.9.2.
In production use, we found we could run double the load on our servers with 1.9 than we could with 1.8. Switching to 1.9 kept us running our app on a cluster of 8 servers as our user base grew, saving us the monthly rental of 8 servers and the overhead of running twice as many web servers.
We never measured how much longer our tests took to run though, since our customers were only ever running in Production mode
If your changes were method changes, one9 should catch them. As explained in the Rails section of this post, http://tagaholic.me/2011/03/05/one9-upgrade-to-ruby-19-now.h..., you can always jump into a troublesome gem's source, install its dependencies and run one9 on its tests.
I kind of believe slowness in test runs vs 1.8MRI or REE depends on bytecode compiling (though i have not proof of this).
rails and rspec include a massive amount of code that gets loaded/compiled, and that is done every time you run the tests without giving time to YARV to express itself in the faster paths, so this difference is more noticeable.
