00:00:28
I'm here to introduce you to a gem, ActiveRecord::Tenanted, that 37signals is open-sourcing. I actually pushed the publish button on it about two minutes before this talk. This gem provides multi-tenancy support, and I'll go into the details today.
00:00:49
Why do we need a new library for multi-tenancy? Multi-tenancy is not new in the Rails world — people have been talking about it for years. As early as 2009, Guy Neger gave a talk on Rails 2 and multi-tenancy. It's a great, if somewhat dated, talk that discusses many of the problems that need solving in ActiveRecord and the rest of Rails.
00:01:19
Data isolation has been solved several times and there are gems that do this already. For example, IBM's definition of multi-tenancy emphasizes that each tenant's data is isolated from and invisible to other tenants sharing the application instance; they mention privacy and security. I'm focusing on that core idea of strict data isolation. I won't dive into database architecture or specific database technologies like materialized views; instead I'm focusing on Rails applications and what the Rails framework could do better to support you when you build a multi-tenant app.
00:01:59
I'm also not here to sell you on multi-tenancy — I'll assume you already know you want it. In the Rails ecosystem data isolation generally takes two forms along a spectrum: co-mingled data (one database with tenant IDs) or separate databases per tenant. I'll cover both for shared context, but my preference and the focus of this talk is on separate databases.
00:02:36
Co-mingled data means you have a single database where every table includes a tenant or account ID column. Each row is scoped by that tenant ID, and your queries must filter by that ID. For example, a users table might include an account_id column to indicate which account each user belongs to. Gems like ActsAsTenant provide much of this functionality: you add the account ID, declare the association, and scope uniqueness constraints to a tenant if needed.
00:03:16
ActsAsTenant provides an API like with_tenant to set the current account for a block; inside the block queries are automatically scoped (ActsAsTenant often uses default scopes). If you attempt a query outside a with_tenant block it can raise an error so you don't accidentally query without a tenant context. If you don't mind co-mingled data, ActsAsTenant is a great option — you can stop listening now. The downside is you must add account_id to all models and ensure developers never forget to set or respect it. That makes tenancy an application-level concern and increases the risk of accidentally leaking data between tenants, which is why I prefer separate databases.
00:04:22
Separate databases means one database (or one schema or one file) per tenant. In SQLite this is a separate file on disk for each tenant; in Postgres you might use schemas, and in MySQL you might create a named database per tenant. Safety comes from having only one connection active at a time, so that connection only has access to one tenant's data. The drawback is managing many databases and connections, but those feel like things the framework could automate. There is a gem called Apartment (around since 2011) that models this isolation, providing an API to create a tenant and switch connections inside a block.
00:05:52
Apartment will go create the database and, inside a switch block, use the right connection for ActiveRecord queries; if you query without context it raises an error. One thing I don't like about some approaches is the explicit switch/with_tenant call everywhere, though in practice middleware can handle switching for requests so your app code doesn't need to call it constantly. At a high level we're trying to ensure you can query the right database at the right time.
00:06:36
So why not just use Apartment? Two reasons. First, connection management: Apartment's connection handling predates Rails 6.1's thread-safe multi-database and sharding features and isn't thread-safe in some cases (there are open issues with threaded servers). Second, completeness: Apartment and similar libraries tend to handle ActiveRecord and request middleware, but they don't integrate with the rest of Rails — Action Cable, Active Job, Active Storage, Action Mailer, caching, testing, and so on.
00:07:08
Rails 6.1 introduced built-in horizontal sharding and multi-database support implemented with thread safety in mind. Others noticed this opportunity too: earlier this year, Julik published a one-file implementation called Shardine that uses Rails' improved connection management. I didn't use Shardine or Apartment because I wanted a more complete solution: thread-safe connection management plus tight integration with every Rails component.
00:08:03
The defaults I chose for the gem are opinionated but simple: treat everything inheriting from ApplicationRecord as tenanted (your app's default connection class), and use the request subdomain to resolve the tenant name. If you accept those defaults, the gem inflates the necessary context and it's dead simple to get a multi-tenant app running. You can override those defaults if you want a different configuration.
00:10:04
Making a real app multi-tenant was remarkably simple in practice. In our Fizzy app we added the gem, mixed tenanted into ApplicationRecord (which we could probably omit later), and modified database.yml to include a %tenant format specifier. That specifier tells Rails where to put the SQLite database file for the current tenant. Everything I describe here focuses on SQLite, but the gem can be adapted to Postgres or MySQL as well.
00:11:04
After these changes, the app becomes multi-tenant with no further steps: designers and developers can work on a normal Rails app without special knowledge of tenancy. Because there's no opportunity for application-level accidental leaks, tenant data safety is easier to guarantee.
00:11:36
Let me explain what the gem does inside ActiveRecord and why Rails 6.1's multi-database support is so useful. In database.yml you can define shards: groups of databases that share the same schema. Models can pull their data from any of those shards if you configure ApplicationRecord accordingly. You can switch connections with connected_to and pass the shard name; it behaves as you expect.
00:12:25
I experimented: what if we treated each tenant as a shard and declared many shards in database.yml? I generated 10,000 tenants and added their entries as shards. Surprisingly, it basically worked: the app booted, managed many connections, and I saw fast response times — around 7 ms in my test. But the drawbacks were clear: shards must be declared statically at boot, boot time becomes very slow (it took around two minutes to boot 10,000 connections), and memory use increased significantly. So I wanted a way to do this dynamically without declaring every shard at boot.
00:14:09
I dug into Rails' connection management internals and quickly got lost in classes like AbstractAdapter, ConnectionHandler, ConnectionPool, PoolManager, DatabaseConfig, and others. The connection pool is the important piece: it represents a connection (or set of connections) to a database. What I wanted was a new connection pool per tenant, created on the fly when that tenant is first referenced.
00:15:17
The first problem to solve was avoiding the need to declare thousands of entries in database.yml. I created a RootConfig class and used Rails' register_db_config_handler hook to register a custom handler for database configurations. The handler looks for tenanted: true in the config and returns the RootConfig object, effectively injecting our custom config object into Rails' database configuration stack at boot.
00:16:27
RootConfig still contains the %tenant format specifier, but it exposes a new_tenant_config method that produces a concrete configuration for a specific tenant name. We pass that concrete config into establish_connection to create a connection pool, and the pool ends up with the right shard name.
00:17:05
When you mix tenanted into ApplicationRecord you get methods such as current_tenant. Tenants and shards are treated equivalently, but the API uses "tenant" for clarity. The connection_pool method checks whether a pool for the current tenant already exists; if not it creates one using the dance I described. There are many edge cases to handle — for example, during boot there's no tenant so you can't connect to a tenant database yet, which leads us to rely on the schema cache heavily — but I've implemented much of the necessary handling.
00:18:16
The gem's API is centered on ApplicationRecord. You can list tenants, check existence, and create tenants. current_tenant is nil until you set a tenant context; attempting to query without a tenant raises an exception. If you wrap code in with_tenant, current_tenant returns the tenant name, queries work, and returned objects also carry a tenant attribute. That attribute enables safety checks so it's hard to accidentally operate on the wrong tenant's data.
00:19:06
For example, if you fetch a user record from tenant "foo" and try to update it outside a with_tenant block, you'll get an error because there's no tenant context. If you try to update that user while connected to a different tenant ("bar"), you'll get a wrong-tenant error. Associations behave the same way. You can also enforce Rails' prohibit_shard_swapping behavior to prevent changing the tenant mid-request unless explicitly allowed. Middleware normally takes care of calling with_tenant for incoming requests, so application code typically doesn't need to call it directly. The middleware is a Rack middleware that inspects the request, uses a configurable tenant resolver to map the request to a tenant name (by default the subdomain), and wraps request processing in with_tenant. In Fizzy we initially used subdomain tenanting and later switched to a path-based tenant resolver; all that complexity is encapsulated in the tenant resolver configuration.
00:21:31
I've talked about the ActiveRecord part, but we need to make the rest of Rails work too — Action Cable, Active Job, Active Storage, caching, mailers, Turbo Streams, and testing. I'm going to run through the list of integrations I added to make multi-tenancy feel seamless.
00:22:04
The connection class controls Rails integrations. As long as you set the connection class, the gem provides many integrations for free. One challenge is Rails' internal models: Action Text, Active Storage, and Action Mailbox create their own ActiveRecord subclasses, so you can't automatically inherit behavior from ApplicationRecord. To handle this, Rails provides subtenant_of, which forwards connection_pool and current_tenant from those Rails models to your tenanted class. In practice, calling subtenant_of(ApplicationRecord) tells Active Storage, for example, to use the same tenant connection, allowing joins across tenant models, attachments, and blobs.
00:23:54
Fragment caching presents a different conflict: Rails cache keys often look like "users/1" and every tenant will have a record with ID 1. If you use a shared cache like Memcached or Redis, fragments could collide and tenants could see cached content from others. One solution is to use a tenant-local cache store (e.g., SQLite via ActiveSupport::Cache) with subtenant_of, but that's only for specific cache stores. A more general solution is to modify Rails' cache keys to include the tenant ID, ensuring keys are unique per tenant.
00:24:49
Active Storage also needs tenant isolation because blob keys determine how files are stored in S3 or on disk. If blob keys collide across tenants, their files mix together. We address this by prefixing the blob key with the tenant name so each tenant's blobs live in a distinct folder or key namespace on S3 or in a subdirectory on disk.
00:25:44
Active Job raises an important question: what happens when you enqueue a job with a model instance from a tenant? We extend ActiveJob::Base with a tenant attribute. When a job is created we capture the current tenant (if any) and serialize it with the job. When the job runs, perform_now is wrapped inside with_tenant if a tenant attribute exists, so the job executes in the intended tenant context. This preserves non-tenanted jobs as well. However, there's a tricky case: if you fetch a user from tenant foo, then switch context and call perform_later in a different tenant, safety checks alone don't prevent misuse — so we had to dive into GlobalID.
00:27:04
GlobalID is the gem Rails uses to serialize object references. Normally a GlobalID looks like app://AppName/ModelName/id. That would conflict across tenants. We add the tenant name to the GlobalID to make it unique, and we modify the locator so GlobalID deserialization is tenant-aware. If you try to deserialize a tenanted GlobalID without a tenant context or in the wrong tenant, it raises an error. These checks ensure jobs can't accidentally load objects from the wrong tenant.
00:28:01
Action Cable can use the same tenant resolver you configure for HTTP requests to determine the tenant for each connection, and connection commands are wrapped in with_tenant. Many Rails features (Turbo Frames, Turbo Streams, etc.) use GlobalID under the hood, so once GlobalID is tenant-aware, they become tenant-aware too. Action Mailer supports the %tenant specifier in configuration, and load_async is handled intelligently: if you call load_async in a tenant context but resolution happens later, the gem ensures the query runs on the correct tenant connection. SQL logs include the tenant name and normal tagged logging works; I plan to add structured logging support as well.
00:29:28
Your test suite shouldn't have to know about tenanting either. Tests run with tenants available and the integrations behave the same; I won't cover all the testing details in this talk, but I have considered test patterns and helpers to make writing tests straightforward.
00:30:10
Looking ahead before a 1.0 release, the main work is managing connection pools: right now, if you receive one request each from 10,000 customers you could end up with 10,000 connection pools in memory. I want to add a cap on the number of pools and a reaping mechanism for unused pools to make this manageable. With SQLite creating a new connection is cheap, but I want to understand performance characteristics with Postgres and MySQL and would welcome collaborators. There are also rough edges around database tasks (rake db:*), which are complicated to override cleanly. The project is open source now — I pushed it — so please try it at github.com/basecamp/activerecord-tenanted and send feedback. If you're considering SQLite in production you'll likely want replication; Kevin McConnell will give a talk on SQLite replication later in this room. Thank you for your patience and for coming to my talk on multi-tenant Rails.
