Migration Magic ✨, Handling Not-Null Constraints in Production with Ruby on Rails

Originally posted on my blog ahmednadar.com.

We've all been there-happily adding new tables and columns to our applications in the development environment. And when things go wrong, we drop the table and start fresh, no harm, no foul 😎. But what happens when we encounter issues in the production stage? That's when the challenge begins. In this post, we'll tackle a tricky migration scenario involving not-null constraints and share practical tips to avoid losing data in production. So buckle up, and let's dive in! 🚀

😌 Development Stage Bliss

While i'm working on validateok.click application I needed to add a new column category to an existing table ideas. Here's an example migration:

class AddCategoryToIdeas < ActiveRecord::Migration[7.1]   
    def change     
        add_reference :ideas, 
                      :category, 
                      default: 'Tools', 
                      null: false, 
                      foreign_key: true, 
                      type: :uuid   
    end 
end

In the development environment, running this migration is a breeze, and everything works as expected. But what about production? 😰

😬 Production Stage Hurdles

In production, dropping a database isn't an option unless we're willing to lose everything - and nobody wants that! 😱 Using the same migration steps as we did in development above 👆🏼 can lead to errors, like this one:

ActiveRecord::NotNullViolation: PG::NotNullViolation: ERROR: column "category_id" of relation "ideas" contains null values

The error occurs because the existing records in the ideas table don't have a category_id value, not only that, and we've set the null: false constraint. What can we do to fix this issue? 🤔

🪄 The Solution

No worries! We have the perfect solution for you💡. Let's walk through the steps needed to handle this migration gracefully in production.

Step 1: Remove the default value and split the migration into two parts

First, remove the default value 'Tools' from the migration, as it should be a UUID, not a string. This is my mistake because I didn't pay attention that I'm using UUID. Am I the only one🤦‍♂️ Then, split the migration into two steps:

1. Add the category_id column without the null: false constraint and backfill the existing ideas with a default category.

2. Finally, add the null: false constraint.

Here's the updated migration file:

class AddCategoryToIdeas < ActiveRecord::Migration[7.1]
  def up
    add_reference :ideas, :category, foreign_key: true, type: :uuid #1

    default_category = Category.find_or_create_by!(name: 'Tools')
    Idea.update_all(category_id: default_category.id)

    change_column_null :ideas, :category_id, false #2
  end

  def down
    remove_reference :ideas, :category
  end
end

Step 2: Understand the purpose of each step

1. Add the category_id column without the null: false constraint. This allows us to update the existing records without violating the non-null constraint.

2. Find or create a default category named 'Tools' and set its ID as the default category_id for all existing ideas. This ensures that all existing ideas have a valid category_id value.

3. Add the null: false constraint to the category_id column. Now that all existing ideas have been updated with a default category_id, we can safely enforce the non-null constraint.

4. And now you can confidently run rails db:migrate

🗞 Wrapping Up

And there you have it, folks! Thanks to these steps, we've gracefully handled a potentially tricky migration in the production environment without losing any data 🎉. By splitting the migration into two parts, updating the existing records, and then enforcing the non-null constraint, we've made our application more robust and reliable.

Remember, as Rails developers, it's essential to think about the impact of our db migrations in both development and production environments. With a bit of vision, and a few well-placed emojis 😜, we can overcome these challenges and continue building fantastic applications.

As always, Happy Coding, and until next time! 🚀