Loading files in Rails

This post assumes that you’ve already read Part 1 - Loading Files In Ruby . If you haven’t go follow that link and go and read it, then come back here.

So we’ve gone through how Ruby loads files. But we’re still trying to work out how Rails does its magic trick of knowing about all your classes.

# app/controllers/invoice_controller.rb

class InvoiceController < ApplicationController # 🪄🌈❓❗❓
  # some business logic
end

I want to introduce you to two characters, let’s call them Classic and Zeitwerk. Here’s a picture of them:

As you can probably tell, they’re very different characters.

• Zeitwerk is organised, proactive and never makes mistakes. He’s a joy to work with (although some people think he’s a bit fussy).

• Classic is the complete opposite. He does everything last minute, makes a lot of mistakes and is always one step away from being fired.

In this post, we’re going to look at the way that Rails used to load files (Classic), then we’re going to look at the new way (Zeitwerk).

How autoloading used to work

Back in the day, Rails got Classic to do all the autoloading.

Classic’s approach for this was to read through all the code, and then when he hit a constant he hadn’t seen before, he quickly tried to find a file which contained that constant. For this, he relied pretty heavily on usingObject.const_missing.

I’m sorry, what’s missing?

const_missing is a cool ruby meta-programming trick that allows you to determine what happens when a constant is referenced that doesn’t exist. Normally, Ruby would raise a NameError if you refer to a non-existent constant.

But we can change that…

class Object
  def self.const_missing(c)
    puts "there is no such constant #{c}!"
  end
end

BANANA_NAME => "there is no such constant BANANA_NAME!"
APPLE_NAME => "there is no such constant APPLE_NAME!"
GRAPE_NAME => "there is no such constant GRAPE_NAME!"

Here, we’re re-opening the Object class and overriding const_missing. Now, instead of raising of raising an error, it prints a message to the user.

Where things really get interesting is where we use const_missing in combination with const_set. Object.const_set allows you to define (or redefine) any constant. So with these two methods working together, we can dynamically set a constant if we find that it doesn’t exist yet. 🤯

Imagine a customer walking into a shop and asking for something that the shop doesn’t have in stock. Then imagine the shop owner going into the back and quickly making the item on the spot. They bring it out and the customer buys it.

That’s basically what we’re doing here.

Here’s an example of this sort of thing:

class Object
  def self.const_missing(constant)
    # Turn `BANANA_NAME` into "Banana"
    fruit_name = constant.to_s.split('_').first.capitalize  
    new_value = "#{["Tim", "Ann", "Bob", "Ada"].sample} the #{fruit_name}" 
    
    # Sets the value of the constant (for any future references)
    const_set(constant, new_value)
    
    # Now we return the new value of `BANANA_NAME`
    new_value
  end
end

BANANA_NAME => "Tim the Banana"
APPLE_NAME => "Ann the Apple"
GRAPE_NAME => "Bob the Grape"

This is the trick that Rails (and Classic) relied on for many years.

So how did this work in Rails?

When Ruby hits ApplicationController, it tries to find this class using its Constant Lookup Hierarchy.

If Ruby hasn’t seen the constant before, then it will go all the way through the Constant Lookup Hierarchy, and find nothing. Ruby’s next step is to trigger Object.const_missing. This effectively hands the task of finding this class over to Rails, because Classic has overridden const_missing.

In Classic’s version of Object.const_missing , it would convert the class name to snake_case then try to load a file of that name.

Here’s a rough idea of how that worked:

class Object
  def self.const_missing(constant)
    # translates `ApplicationController` to `application_controller`
    underscored_version_of_constant_name = constant.to_s.underscore
    
    # here we `require` the file `application_controller`, thus loading 
    # the missing constant `ApplicationController`
    require constant.to_s.underscore 
    
    # now we fetch (and return) the - no longer missing! - `
    # ApplicationController` constant
    Object.const_get(constant) 
  end
end

What was wrong with how Classic did things?

This approach worked for many years, but it came with some problems.

I said earlier that Classic made a lot of mistakes. You can see a long list of these mistakes in the Rails Guide for version 5.2 (the last version that still exclusively used the Classic autoloader).

Here’s an example of one of them, imagine the code below:

# date.rb
class Date
  def initialize
    puts "Would you like to go on a date with me ❤️?"
  end
end

# supermarket/date.rb
module SuperMarket
  class Date
    def initialize
      puts "Would you like to buy one of these tasty dates? 🌴"
    end
  end
end

# supermarket/dried_fruits.rb
module Supermarket
  class DriedFruits
    def initialize
      @date = Date.new
    end
  end
end

So we have a couple of Date classes, which we definitely don’t want to mix up (🌴 != ❤️)

If we first call Supermarket::DriedFruits.new, before any references are made to either of the Date classes, everything works as expected:

> Supermarket::DriedFruits.new
Would you like to buy one of these tasty dates 🌴?
✅

Ruby doesn’t know about either of the Date classes, so we trigger const_missing. The Classic autoloader does its thing and loads the correct class.

But! If you first referenced one of the top-level (romantic) dates, then we get different behaviour.

> Date.new
Would you like to go on a date with me ❤️?
✅

> Supermarket::DriedFruits.new
Would you like to go on a date with me ❤️?
❌

This happens because Classic autoloading is only triggered if Ruby is unable to first find a constant of that name in it’s constant lookup hierarchy. In the above example, Ruby looks in the following places:

Supermarket::DriedFruit::Date # nope 
Supermarket::Date # nope - `supermarket/date.rb` has not been loaded yet!
::Date # aha! I found a Date class!

It already knows about a Date class, so we never get to const_missing and supermarket/date.rb doesn’t get loaded. 😞

Having the value of constants depend on the order that are referenced is some pretty crazy behaviour.

So the old autoloader was a bit of a hack. It also completely ignored the perfectly nice autoload functionality that already existed in Ruby (as we discussed in Part 1…). You’d think if you were going to implement an autoloader you would use autoload - right?

And, as we’ll see, that’s exactly how Zeitwerk likes to do things.

So how does zeitwerk do it?

Zeitwerk doesn’t wait to find a constant that is has not seen before. Zeitwerk is proactive.

Before looking at any of the application code, Zeitwek first scans all the autoload paths when the app loads.

It then creates a bunch of autoload statements, based on the files it has seen, and assumptions about what the classes in those files are likely to be called.

# Zeitwerk sees:
`application_controller.rb`
# Zeitwerk assumes this class exists:
=> ApplicationController
# So Zeitwerk autoloads:
autoload(:ApplicationController, 'application_controller.rb')

# Zeitwerk sees:
`supermarket/date.rb`
# Zeitwerk assumes this class exists:
=> Supermarketr::Date
# So Zeitwerk autoloads:
Supermarket.autoload(:Date, 'supermarket/date.rb')

So to go back to our earlier example, this is Ruby’s new search through the Constant Lookup Hierarchy.

Supermarket::DriedFruit::Date # nope 
Supermarket::Date # I have an autoload for that! Let's load the file!

By using Ruby’s in-built autoload functionality, rather than hacking around it, we bypass all the problems we saw with the Classic autoloader.

By creating these autoload statements when your app first boots, the required classes can then be loaded while Ruby is searching the Constant Lookup Hierarchy, rather than waiting for this search to finish

So, the new autoloader is much better than the old one, and everyone is happy about it. 🎉🌈

In 2021, the Rails wizard finally fired Classic (Rails 7 no longer supports the classic autoloader), and the long list of autoloading “Gotchas” was removed from the Rails guides.

One small problem

However, remember I mentioned before that Zeitwerk was a bit fussy? It’s not his fault really, it’s just the way he likes to work.

You see, while Classic found the class name, then guessed the file name:

# Classic
"ClassName".underscore => "class_name"

Zeitwerk finds the file name and then guesses the class name:

# Zeitwerk
"class_name".camelize => "ClassName"

But actually, it’s harder to guess class names from file names than the other way round, because of Rails conventions about file and class naming. Files are always follow the snake_case format, eg api.rb. However, the class name in this file could be either API or Api.

It’s not a big problem. Zeitwerk can handle either situation, you just need to let him know 🙂.

There are a few other things to be aware of if you’re upgrading a Rails app that currently uses the classic autoloader, which you can find here.

Still hungry for more autoloading?

If you want to keep learning about how Rails loads files, below is a great talk from Xavier Nora (who created zeitwerk) about how he did it.