I think version 0 should be pretty simple.
Walk through all the be benches - if any bench got faster or slower beyond a threashold, include a link to it with details EG:
The following benchmarks got faster
- app_answer - 2% faster - link to report
The following benchmarks got slower
- app_mandelbrot - 2% slower - link to report
If nothing changed no need to send an email, we need to be super careful we only alert when there is a material change and it is not purely "noise"