Currently, ManualRunner can be used from production console although it’s not convenient way to do it. So as @system proposed, admin UI could be handy.
Therefore I’ve created dedicated topic to hear some suggestions what do you find handy having in admin UI?
Maybe being able to see suite logs (without ssh to server) could make our lives easier when some error with running suite occurs? Storing logs for last 10 runs would be sufficient.
I’m intentionally not being too demanding with most of your time right now – I think Sam’s goal of measuring framework overhead is an excellent one, and I don’t want to interfere.
But at some point, I’m strongly in favor of not only a “last X commits” admin UI, but more specificially a “run against this commit in this (potentially new) repo” admin UI so that we can run the full RubyBench suite easily against new patches, such as Shyouhei’s deopt branch or Vladimir Makarov’s MJIT branch.
Another thing we want to add is to be able to backfill based on the version as well.
I wonder, do you maintain any board with TODOs? I think it could be handy to sync work.
We could use Projects · ruby-bench-web · GitHub or something like trello board.