RubyBench weekly report


#1

We’ll like to allow people to subscribe and unsubscribe to a weekly report to view all the benchmark results.

Things to tie down:

  • Report format

#2

I think version 0 should be pretty simple.

Walk through all the be benches - if any bench got faster or slower beyond a threashold, include a link to it with details EG:

The following benchmarks got faster

  • app_answer - 2% faster - link to report

The following benchmarks got slower

  • app_mandelbrot - 2% slower - link to report

If nothing changed no need to send an email, we need to be super careful we only alert when there is a material change and it is not purely “noise”


#3

Noticed this today which was due to one of the commits which was fixed by the commit right after.

TODO: We need a good way of identifying/filtering anomalies like this in order to prevent false positives


#4

Confirmed that this was caused by a commit made accidentally. I think the way we should handle this is to just invalidate the particular commit in the future.


#5

We need something like this :grin:


#6

Relevant information for the future mir.aculo.us JavaScript with Thomas Fuchs » Blog Archive » Embedding Canvas and SVG charts in emails