RubyBench weekly report

We’ll like to allow people to subscribe and unsubscribe to a weekly report to view all the benchmark results.

Things to tie down:

  • Report format

I think version 0 should be pretty simple.

Walk through all the be benches - if any bench got faster or slower beyond a threashold, include a link to it with details EG:

The following benchmarks got faster

  • app_answer - 2% faster - link to report

The following benchmarks got slower

  • app_mandelbrot - 2% slower - link to report

If nothing changed no need to send an email, we need to be super careful we only alert when there is a material change and it is not purely “noise”

Noticed this today which was due to one of the commits which was fixed by the commit right after.

TODO: We need a good way of identifying/filtering anomalies like this in order to prevent false positives

Confirmed that this was caused by a commit made accidentally. I think the way we should handle this is to just invalidate the particular commit in the future.

1 Like

We need something like this :grin:

1 Like

Relevant information for the future JavaScript with Thomas Fuchs » Blog Archive » Embedding Canvas and SVG charts in emails