Introduction

This talk, "Taming the Beast: Safely Managing Database Operations in Rails in a Team of 100s," presented by Miles McGuire at Rails World 2025, discusses the challenges and solutions Intercom has developed for managing large-scale database schema migrations within a sizable engineering team using Rails and MySQL.

Main Theme

The main theme revolves around maintaining developer productivity and system safety while enabling fast, safe schema changes across massive databases, specifically focusing on processes, tooling, and automation to facilitate zero-downtime operations, prevent mistakes, and reduce risk.

Key Points

• Context and Scale: Intercom uses Rails and MySQL to manage hundreds of terabytes of data with tables ranging up to tens of billions of rows, posing significant migration safety and productivity challenges as compared to smaller-scale projects.
• Productivity and Developer Experience:
  • New engineers should be able to use standard Rails patterns without running into undocumented or unique process hurdles, crucial for onboarding and now especially important as more migrations are generated by LLMs.
• The Evolution of Safety Processes:
  • Checklists and Runbooks: Intercom began with operational checklists to document migration safety steps, emphasizing that each checklist item should be justified by real experience to avoid productivity drains.
  • In-PR Automation: Checklists were integrated into GitHub PR reviews via GitHub Actions, meeting developers within their workflow and ensuring wider adoption.
• Limitations of Checklists:
  • Checklists alone are fallible; often ignored or misunderstood, especially by experienced engineers.
  • Checklists do not prevent long-running, dangerous migrations or handle scale-specific risks.
• Proactive Risk Reduction:
  • Linting with RuboCop: Custom RuboCop cops warn developers about risky migration patterns—such as forgetting to bulk changes, using non-BIGINT foreign keys, or using raw SQL via execute—helping catch errors before code review.
  • Blocking Risky Operations: Some operations (like execute in migrations) are blocked outright due to their bypassing of safety checks and the inability to inspect their contents programmatically.
• Runtime Safety Checks:
  • Monkey-Patching Migrations: Intercom monkey-patches ActiveRecord migration execution in production to analyze migrations in real time, blocking those that fail heuristics (e.g., affecting tables beyond 5 million rows or 5GB in size).
  • Automated Decision Points: The system programmatically intercepts and blocks unsafe actions (such as drop table on large tables or change_column operations that could break deployments), relying on production data for accuracy.
  • Escape Hatches: Expert users can override checks if absolutely necessary, maintaining flexibility for advanced use-cases.
• Migration Execution Practices:
  • Only single, explicitly-specified migrations are allowed in production runs to avoid conflicts and overlapping risky operations.
• Move to PlanetScale:
  • The transition to PlanetScale will further shift workflows, leveraging built-in schema change review processes and abstracting many manual or custom controls away from the core team.

Conclusions and Takeaways

• Safe schema migrations at scale require layered controls: documentation, code linting, automation, and runtime checks.
• Automating safety net steps where possible minimizes productivity loss and reduces the burden on specialized database teams.
• As database infrastructure evolves (e.g., migration to PlanetScale), many manual interventions will become obsolete, emphasizing the value of aligning with built-in tooling where possible.

Relevant Examples

• Migration linting and blocking implemented via RuboCop and monkey-patching.
• In-PR checklists posted via GitHub Action.
• Runtime analysis blocking risky migrations in production (e.g., big table alters, risky drops, or unsupported changes).

Closing

The strategies developed by Intercom have reduced operational risk and improved developer experience in a large, fast-moving Rails organization, offering practical code examples and processes for others to adapt to their own scale and infrastructure.