Comparing all Rails benchmarks to raw counterparts

1

A general pattern I want us to follow is include a “baseline” for almost all Rails benchmarks that shows how long “raw” would take compared to AR. The would provide two very important functions:

  1. It provides the team with a performance goal.
  2. It allows us to easily isolate gem performance regressions (like pg or sqlite gems getting slower).

For example yesterday I was running this:

ENV['RAILS_ENV'] = 'production'
require 'benchmark/ips'

require File.expand_path("../../config/environment", __FILE__)


Benchmark.ips do |b|
  b.report("raw") do
    map = {}
    builder = SqlBuilder.new <<SQL
    SELECT p.topic_id, p.post_number
    FROM post_actions pa
    JOIN posts p ON pa.post_id = p.id
    WHERE p.deleted_at IS NULL AND pa.deleted_at IS NULL AND
       pa.post_action_type_id = :post_action_type_id AND
       pa.user_id = :user_id AND
       p.topic_id IN (:topic_ids)
    ORDER BY p.topic_id, p.post_number
SQL

    builder.map_exec(OpenStruct, user_id: 3, post_action_type_id: 2, topic_ids: [5,6,7,14]).each do |row|
      (map[row.topic_id] ||= []) << row.post_number
    end

  end

  b.report("AR") do
     map = {}
     PostAction.where(user_id: 3, post_action_type_id: 2, deleted_at: nil)
              .references(:post)
              .includes(:post)
              .where('posts.topic_id in (?)', [5,6,7,14])
              .order('posts.topic_id, posts.post_number')
              .pluck('posts.topic_id, posts.post_number')
              .each do |topic_id, post_number|
                (map[topic_id] ||= []) << post_number
               end

  end
end

# Raw   2.146k (± 5.7%) i/s -     10.865k
# AR    837.687  (± 5.6%) i/s -      4.182k

So in this example we are paying a 2.5x performance tax by using AR.

Thoughts?

2

I think thats a good idea. Will look into it soon.