Rails was just accepted into Google Summer of Code, one of the projects on the list is improving rubybench which myself and @schneems will be mentoring.
@tgxworld has already put a tremendous amount of effort into the project, he has a guaranteed spot for the summer. There may be another spot or two more open and we need to start thinking about what the project will include and how we can measure success.
If you are interested in getting involved the best thing to do is to start hacking on rubybench you do not need to wait till summer for that.
Top level there are a few big goals I would like us to have:
Automatically detect performance regressions in Rails and report back to the Rails team
- Work on improving micro and macro Rails benchmarks
Benchmark other Ruby implementations.
- At the moment we only test MRI. When it comes to “other implementation” testing JRuby and JRuby 9k are the highest priority followed by Rubinius and others.
Improve RubyBench architecture to allow for very simple backfilling of benchmarks, create an admin UI for RubyBench so we can track how stuff is progressing, what is running and where?
Improve documentation and front end, we do need some designer love.
- look at pypy speed center are there any concepts we can borrow?
Any other ideas? Keep in mind this is just a brain dump.