00:00:08.800
multi-tenant rails. Uh this is the uh presentation where I talk to you about giving a database to each one of your
00:00:15.519
customers. So like you get a database and you get a database and you get a database and you get a database.
00:00:21.199
Everybody gets a database. Now that I've got the Oprah meme out, we can focus on business. Um I'm here to introduce you
00:00:28.720
to a gem uh active record tenanted that 37 signals is open sourcing. In fact, I
00:00:34.079
just pushed the publish button on it about two minutes ago. Uh and this gem is uh intended to um yeah it provides
00:00:42.000
multi-tenency and I'll I'll go into all those details today. Um I do want to talk a little bit about why we need a
00:00:49.200
new library for multi-tenency. Uh so multi-tenency is not a new topic in the
00:00:54.719
Rails world. As early as 2009, uh Guy Ner gave a talk on Rails 2 and
00:01:01.520
multi-tenency. Uh this is a really great talk. It's a little dated now, but if you want to go take a look at it, uh he
00:01:07.040
talks a lot about all of the problems that have to be solved both in active record, but also the rest of Rails. Um
00:01:13.840
and uh so so anyway, so this is this is not a new problem. Uh and in fact um
00:01:19.759
data isolation has been solved several times. There are a couple of gems already that do this. So if you go to
00:01:25.280
IBM's website because IBM invented multi-tenency. Everyone knows this. Uh IBM says each
00:01:32.720
tenants data is isolated from and invisible to the other tenants sharing the application instance. And they use words like privacy and security. Uh and
00:01:40.400
this is really the the core bit that I want to talk to today about. Um I'm not going to talk about like database
00:01:46.320
architecture. Um, a lot of people want to talk about uh using things like materialized views. Um, I'm not going to
00:01:53.040
be talking about any of the database technology necessarily. I'm going to be focused on the Rails applications that you all
00:01:59.600
maintain and what the Rails framework could be doing better to support you when you build a multi-tenency app. I'm
00:02:05.520
also not going to sell you on multi-tenency. I'm going to assume that you know if you want multi-tenency, so
00:02:11.440
I'm not going to try to sell you on it. So this is just going to assume that you're here because you're really interested and uh don't need to be sold.
00:02:18.879
Uh where am I going with this? Right. So data isolation uh at least in the Rails
00:02:24.480
uh ecosystem uh takes two forms generally. There's a spectrum here, but I'm going to break it down into into
00:02:30.720
two. Uh co-mingled data or separate databases. Uh separate databases is what I'm going to be talking about, but I'm
00:02:36.319
going to cover both of these just so that we all have some shared context. So co-mingled data is you have one database
00:02:42.720
and all of your customers data is mingled together in this database. Um the idea is you have a tenant ID or like
00:02:49.760
an account ID of some kind in every table. So every row has an ID that you
00:02:55.840
use to filter by uh your your tenant ID or your customer ID. Um and the
00:03:00.879
constraint is you have to remember to specify your tenant ID in every query.
00:03:06.080
Here's what this looks like in practice. Um, you might have a users table. We got some people in here. We've got my
00:03:11.920
friends Adriana and Peter who both work for the same company. It's Shopify. Um,
00:03:16.959
then we got three different Jeremies who all work for different companies because we have a lot of Jeremies in the Ruby world. Um, this is our data. You can see
00:03:24.319
we have an account ID column telling us where it is. And um, the access tenant gem provides a lot of this
00:03:30.400
functionality. Uh, but you have to do the normal Rails thing. You add an account ID column to your table. you
00:03:35.840
call access tenant which sets up the belongs to association with the account.
00:03:40.879
Um and if you need to do extra things like uh scope your uniqueness constraint to uh unique within a tenant as opposed
00:03:47.519
to unique within an entire table uh access tenant lets you do that too.
00:03:52.640
uh the API that it provides for you is um you call a with tenant block and you
00:03:57.840
provide your current account and then uh inside that block if you're going to do a query to your database it's going to
00:04:04.319
automatically scope it. So like I think access tenant uses like default scopes for this magic uh it's pretty good and
00:04:10.080
if you don't um if you don't have a a with tenant block it'll raise an error and tell you I don't know which tenant
00:04:15.680
you're querying on and this is a good thing this provides the safety mechanism. Now, if you don't mind comingling your data, access tenant is
00:04:22.479
great and you should totally use it and you can like leave and you don't need to listen to the rest of the talk. Um, but
00:04:27.600
if you want to um, oh, sorry, one more thing. Uh, the downside of this is you
00:04:33.280
have to add the account ID to all of your models. Uh, which in my mind makes it really kind of an application level
00:04:40.160
concern. You've got to make sure you've set up all of your models the right way. If you forget, things are going to go bad. And worse is um sometimes your
00:04:47.280
developers need to know that all the data is comingled together which means that there could be a bug introduced
00:04:53.199
where accidentally you might get data from one tenant being shown to another tenant and this is bad and I think the
00:04:59.199
framework could be doing a little bit more which is why I like separate databases. So the idea with separate
00:05:04.400
databases is you have a separate maybe if it's SQL light it's actually a separate file on disk for each of your
00:05:10.720
tenants but this also works for Postgress or my SQL for Postgress it'll create a new schema uh for my SQL you'd
00:05:17.280
create a new named database just for each tenant um and the isolation
00:05:22.479
constraint the safety is provided by you only have one connection active at a time so that connection that database
00:05:29.280
connection only has access to one tenants's data so it's very very hard to accidentally show data to the wrong
00:05:35.360
tenant. Now, the drawback though is um you now have a lot of databases to
00:05:40.479
manage and you have a lot of uh connections to manage as well, but those feel like things the framework could be
00:05:45.600
doing for you. And so, this is why I like this too. Um there's actually a gem that um presents this isolation model.
00:05:52.560
It's called apartment. It's been around since 2011. And the API provides us pretty simple. Uh you can create a new
00:05:58.639
tenant and this will do exactly what you think. it'll go create the database uh inside a switch block. Uh if you make a
00:06:05.440
query, an active record query, it will use the right connection to go serve your query, which is great. And if you
00:06:12.319
do it without that context, it'll raise an error. Very cool. You can see that there are actually no other application
00:06:18.720
concerns and the framework is doing all of this work for you. One thing I don't like uh is the fact
00:06:25.919
that you need to do this uh switch here or with tenant uh for the previous gem I
00:06:31.039
showed you. Uh in reality though, there's middleware that does the switching for you. So pretend you don't see that for right now, right? So all
00:06:36.639
we're really talking about is making sure that you can query the right database at the right time.
00:06:42.960
Okay, once again, coingle data kind of an application level concern. Separate
00:06:48.080
databases kind of a framework concern. Um I know what you're thinking. Hey
00:06:54.240
smart guy, why don't you just use apartment then? So there are there are two reasons I think when we were we started to build fizzy which is our new
00:07:01.520
um our new application at 37 signals uh that apartment wasn't really fitting our needs. And the first one has to do with
00:07:08.560
uh connection management. So rail 6.1 uh came out in 2020 and it provided
00:07:14.639
horizontal sharding and multi-database support and these features were implemented with thread safety in mind.
00:07:20.160
Thank you GitHub. Thank you, Ailen. So, the um the idea here is that oh sorry,
00:07:26.720
apartment uh dates back to 2011 and the way it manages connections predates all
00:07:32.000
of this magic and it's actually not thread safe. So, there's like like an open issue on apartment dealing with
00:07:38.240
Akuma threads where it doesn't work properly. And so, I I I saw this and I saw like a path to uh tenanting support
00:07:45.520
that used thread safe connection management. Maybe it's a little bit more performant as well.
00:07:51.120
I'm not the only person who saw this path. Uh earlier this year, uh Julik
00:07:56.560
Tarkinoff, is Julik here? Julik. Anyway, uh Julik published a uh short one file
00:08:03.520
implementation called Chardine, which is a really great name. I wish I had thought of it. Um basically, you know,
00:08:09.199
he says the same thing here, like, hey, Rail 61 has better connection management. Um and so it felt really
00:08:14.479
good to know that other people had seen this opportunity as well. But I didn't use shardine and I didn't use apartment
00:08:19.520
and that has to do with the second reason which is um completeness of the solution. So uh apartment and shardine
00:08:27.520
both uh handle active record and they handle the rack middleware necessary to figure out which tenant uh to use for a
00:08:34.320
request. But it doesn't really do anything with the rest of rails. What about what about action cable? What
00:08:39.760
about active job? What about uh active storage? Um, doesn't really do anything
00:08:45.440
there. So, it felt to me a little bit like I keep hitting the wrong button. It felt a little bit like this to me where
00:08:51.440
like step one was, you know, you configure your models in the middleware, super easy, and then step two is like
00:08:57.360
make the rest of rails work. Uh, make the rest of Rails work. Uh, so
00:09:04.399
uh what I wanted to do is I wanted to make multi-tenency easy. Specifically, I want to have
00:09:09.600
modern thread safe connection management and I want to have tight integration with every Rails component. Uh so the
00:09:16.800
omocas defaults that I chose for the gem and these are it's just uh just a default. You can change the
00:09:22.480
configuration is to take everything that inherits from application record, right? So this is your your application's
00:09:28.640
default uh connection class. Assume that everything that inherits from that is tenanted.
00:09:34.720
Um, and then also to say that uh we're going to use the subdomain of the request to figure out which tenant it
00:09:40.240
is. So if the request comes into fu.ample.com, foo is going to be the name of the
00:09:45.279
tenant that gets used. And so the the gem will will inflate all of the context for you. Uh so if you if you assume
00:09:51.760
these two defaults, um then it's dead simple to get up and running with a
00:09:57.200
multi-tenant app. How simple you ask? Good question. This is a slightly simplified version of
00:10:04.160
the pull request uh to fizzy which is our new product changing it from a single tenant app to a multi-tenant app.
00:10:11.200
Like no This is it. At the top we obviously add the gem. In the middle
00:10:16.480
we kind of annotate our application record class with tenanted. And to be honest we could probably skip that too.
00:10:22.079
I just haven't gotten there. And at the bottom we are modifying our database YAML file. And specifically what we're
00:10:28.079
doing here is we stick in a percent tenant format specifier which means that for whatever tenant is currently in
00:10:34.720
context that's where that SQLite database file is going to live. Uh side note here uh everything I'm
00:10:41.920
going to be talking about today is about SQLite support but there's no reason that this gem can't be made to work with
00:10:47.440
Postgress or my SQL. So if you're interested in doing some of that work and helping out let me know.
00:10:53.200
So if you make these changes uh your tenant will be multi-tenant there like there is no step two like this is the
00:10:58.720
dream this is what I wanted to build um and fizzy in particular this is um just
00:11:04.079
a screenshot but you can say it's like really nicely designed we have a team of developers and designers working on this
00:11:09.440
and none of them know anything about tenanting they just do their job like it's a normal vanilla Rails app and it
00:11:16.000
just works so for me that's like the key I don't need to worry about tenant data safety if there's actually no opport
00:11:22.800
opportunity for developers or designers to accidentally leak data. Uh it it just works. So
00:11:30.399
let me talk a little bit about the work that the gem does inside active record to make this magic happen. And I want to
00:11:36.880
start with explaining why the multi- database support inside Rails 6.1 is so
00:11:42.320
amazing. So if you're not familiar with it, here's the brief version. You can in
00:11:47.440
your database YAML file, you can define shards. And in this case, shards are a group of databases that share the same
00:11:55.120
schema. So the idea is you can have a set of models that can pull their data
00:12:00.320
from any of those shards. So in this case, I've got two shards,
00:12:05.839
shard one and shard two. In my application recordbased class, I say these are the shards that the models can
00:12:12.639
possibly read from. And then if I need to, I can switch away from the primary connection by calling connected to and
00:12:18.800
pass in shard one or shard two. And it does exactly what you think it does. So I took a look at this and I thought to
00:12:25.040
myself, what if we turned this to 11,
00:12:30.480
could I create 10,000 databases using vanilla rail sharding? And so this is
00:12:37.040
what I did. I created 10,000 tenants in the database YAML file. I used Herb to just generate 10,000 entries. Uh
00:12:44.079
application record, I passed all of those entries into the connect to call. And then inside the app, I was calling
00:12:49.600
connected to. This time it's tenant one and tenant two. So you can see where I'm going with this. Like each shard is
00:12:55.120
going to be a tenant. Now the surprising thing is this kind of worked. Um two things highlighted. These
00:13:02.160
are my notes from the experiment. There are good reasons why we should absolutely not use vanilla Rails tenanting out of the box for this. But
00:13:09.360
the really interesting thing is that it totally worked. Like once the app booted, I was able to get response times of like 7 milliseconds. Like it just
00:13:15.760
worked. It managed 10,000 connections to databases. And I was like, "Okay, there's something here." To summarize,
00:13:21.760
the pros and cons. The pros are that it's it's battle tested. It's been in Rails for 5 years now, totally thread
00:13:27.279
safe, and it has the same kind of block calling semantics that I really liked where you call connected to do, and you
00:13:32.800
have a block where that tenant is in context, and I like that API. The cons all have to do with the the fact that
00:13:39.839
you have to declare all of these shards statically like at boot time. Um you
00:13:45.040
can't dynamically add a new tenant or a new shard. Really slow boot time for the process. It took about 2 minutes to boot
00:13:51.519
up uh 10,000 uh 10,000 database connections and it needed about 2 gigs of memory to do all that. So like all
00:13:58.000
right, what if you know can we um can we come up with a way to do this dynamically? I was convinced that we
00:14:03.839
could and then I started to dive into the code and I started looking at
00:14:09.360
all of the classes that coordinate to do connection management in Rails abstract
00:14:15.440
adapter connection handler connection pool config pool manager database configuration this database config and I
00:14:21.839
got lost it was really really confusing so the sense making exercise I like to do is to draw pictures I'm not going to
00:14:29.120
explain this picture to you don't panic um this is mainly just illustrative of the complexity. Um, these are all the
00:14:36.079
different classes that I just named that have to coordinate to manage your connections. And I'm going to call your attention up here in the middle, which
00:14:41.680
is connection pool. Connection pool is what represents uh a connection or a set of connections to the database. And so
00:14:48.480
this is what I wanted. I wanted a new connection pool for every tenant and I wanted to be able to create that on the
00:14:53.760
fly when the tenant was first referenced. In the upper lefthand corner you have
00:14:58.880
active record base and pretty much any active record query is going to call super small. I'm sorry about that. That
00:15:05.519
method is connection pool. So there's a method called connection pool that does a little dance and then returns the
00:15:11.519
connection pool object. So looking at this I was like okay I can I can reimplement some of this dance. Um
00:15:17.680
but what we really need to do is avoid having to inject a database configuration at boot time. Okay pretty
00:15:24.160
pictures go away. This is this is the first problem I ran into. Um I don't
00:15:29.600
want to have to specify 10,000 entries in my database yl file. I want to have one entry and have that act like a like
00:15:35.440
a abstract base config kind of a thing. So how do I actually get Rails to
00:15:40.800
understand what I'm trying to do here with a percent tenant format specifier.
00:15:46.000
Uh so here's what I did. I created a new class called root config. The details of that aren't necessarily interesting,
00:15:52.399
but I used a register db config handler. So, Rails has this little known method
00:15:58.240
that allows you to register your own handler to create database configurations at boot time. And the way
00:16:04.399
it works is I look for that tenanted equals true uh key in the config. And if
00:16:09.920
that's true, I return the root config object. And when I ask Rails for the
00:16:15.519
configurations after boot time, you can see that first line is the gems root config. So what I've done is I've
00:16:20.880
managed to inject my own class into the lowest possible level of the database configuration stack. And this is this is
00:16:27.199
the shim that I needed to do some magic. So now that I have this root config, what what can I do with it? Um, if I
00:16:34.399
just take a look at the root config and I ask it for its database, it's still going to have the percent tenant format specifier, which is not useful, but
00:16:40.880
there's a method on it called new tenant config that creates that concrete config for a specific tenant. And if we ask it,
00:16:47.680
it'll have foo as the tenant name, which is exactly what we want. And we pass that into establish connection to create
00:16:53.440
our connection pool. We can see that it's got the right shard name. So I'm skipping a lot of magic here, but the
00:16:59.519
idea here now here is I've got a tenant specific connection.
00:17:05.039
Um when you mix in tenanted into application record, this is the magic that happens. Uh there's a method called
00:17:12.880
current tenant. So tenants and shards are the same thing. This is just a a nice little uh method to refer to
00:17:19.039
everything as a tenant instead of having to mix and use shard in some places and tenant in other. uh and the connection
00:17:25.120
pool. This is exactly what I just showed you. I check to see if the connection pool has already been exist already exists. If it doesn't, then I go through
00:17:31.440
the dance and I create the new pool and I return it. Uh and the I'm skipping a lot of complications though. That's very
00:17:38.240
simplified code. There are a lot of edge cases that I had to handle. Uh maybe the most notable one is we can't connect to
00:17:45.280
the database during boot because there's no tenant. If there's no tenant, there's no database. And so this kind of this
00:17:51.280
this bootstrapping problem where we end up relying on the schema cache really heavily. Um I'm not going to go into the
00:17:57.760
detail here because I think it's a little boring, but I've done a lot of the heavy lifting and the API that it
00:18:02.799
presents is a little bit different than the gems I showed you already. So you can see that first of all, all the
00:18:08.160
methods hang off of application record. So we've mixed in the behavior into your base class uh instead of into a um like
00:18:16.559
a a constant or a module that the gem is bringing. So you can ask what tenants
00:18:21.600
exist, it'll tell you. You can check for tenant existence. You can create a tenant. By default, this current tenant
00:18:28.000
method is going to say nil because there's no tenant context yet. We don't know which tenant we're talking to. And
00:18:33.360
if in that lack of context, we make a query, it raises an exception. If we specify with tenant in that block,
00:18:39.120
everything does what you think it does. current tenant returns the name of the tenant. We can make a query and then the object that gets returned also has a
00:18:46.000
tenant attribute now and that works for both the model instances and associations.
00:18:52.799
Uh and we can use the fact that all these objects have an a uh a tenant attribute to do some safety checks. Uh I
00:18:59.840
want to make it really hard to accidentally uh connect to the wrong database. So here's an interesting use
00:19:06.400
case. If I uh fetch a user record out of the foo database and I try to do an
00:19:12.880
update, it's going to raise an error because I'm not connected to a database yet. I'm not in that with tenant block.
00:19:17.919
Um if I do it in the wrong tenant, I'm in tenant bar here. It's going to raise another exception telling you you've
00:19:24.000
connected to the wrong database and not let you do it. These are great safety mechanisms. Uh it also does this for
00:19:29.120
associations. If I try to call comments, it's going to behave the exact same way. And finally, uh, the gem is not going to
00:19:36.160
let you change tenants willy-nilly, like whenever you want. Uh, if you're in tenant fu and you try to switch to bar,
00:19:42.480
it's going to raise an exception unless you've specified prohibit shard swapping. This is just vanilla rails
00:19:47.600
horizontal sharding functionality. Um, but you can also, you know, if you can call with tenant a million times, it's
00:19:53.200
okay. And this makes it a little bit easier to provide uh your own method calls that can that will set tenant. And
00:19:59.600
this is this is okay. All those examples had with tenant being
00:20:05.039
called all the place. And I and I mentioned that middleware should usually take care of this. And like I want to just explain really quickly how the
00:20:11.280
middleware works. It's just a a rack rack middleware class. Uh if you're
00:20:16.320
familiar with rack middleware, there's a call method that wraps um everything
00:20:21.760
else that happens during request processing. And so in this case, we take a look at the request and we use uh the
00:20:28.880
tenant resolver, which is I'll explain in a second. It's a configurable lambda. Uh we map the request to a tenant name
00:20:36.400
and then we wrap the rest of the request processing in with tenant. So this way in your app you don't have to call with
00:20:42.080
tenant everywhere. It's already been called for you and so that way you can act like it's just a normal vanilla Rails app and you don't have to your
00:20:48.720
code doesn't need to know about tenanting. Uh the way this is configured there are
00:20:53.760
two configuration params. One is which class are we going to tenant which is connection class and tenant resolver
00:21:00.640
here is a lambda that just returns the subdomain and this is the default omccas config uh and as long as that connection
00:21:06.960
class is present uh the middleware is injected automatically for you. So all of this just is there for you and if you
00:21:13.520
want to do something more complicated you override tenant resolver. uh oh maybe worth noting uh in fizzy we
00:21:19.600
started with a subdomain tenanting and then we moved to like a path base where the the account ID is in a path we have
00:21:24.960
to extract it but all that all that um complication is still limited to the tenant resolver it's still in one place
00:21:31.360
in our initializer uh so I think this model works really well okay so
00:21:38.880
I've talked about the active record bit but now we need to talk about how to how to draw the rest of the owl um so this
00:21:45.360
is going to be a little bit I'm going speed up because I'm low on time. But uh this is going to feel a little bit like I'm a salesman. I'm just running down a
00:21:51.679
list of features. I'm gonna slap the hood of the car and tell you this baby draws the whole owl.
00:21:58.799
Okay, real fast. Here we go. So again, back to the connection class. This is what controls all the Rails
00:22:04.159
integrations. You can set this to nil if you want to do have special behavior, but as long as that connection class is
00:22:09.919
set, you'll get a bunch of stuff in Rails for free from the gem. So one great feature that's built in is
00:22:18.559
uh handling all the Rails records. So if you use uh action text active storage or action mailbox, you know that these
00:22:25.600
Rails libraries create their own models and we want these models to still be in the tenant database, right? We want to
00:22:32.320
be able to do joins between a comment and an attachment and a blob, for example. So everything kind of needs to
00:22:38.880
be in the same database. But Rails in makes all of these subclass
00:22:44.400
active record based directly. So there's no way for us to inject all of our behavior in from application record. Uh
00:22:52.080
the the exception that gets raised has a hint here. It says make sure to use subtenant of. Oh, interesting.
00:22:57.600
Subtiting. That sounds like exactly what you think it is. subtenant of basically forwards
00:23:03.919
connection pool from the active record uh sorry from the rails records over to
00:23:09.600
our tenanted class. So when you call subtenant of and pass in application record uh under the hood we just
00:23:16.960
delegate current tenant and connection pool to application record. And this this is kind of a neat hack uh you can
00:23:23.600
use for a lot of interesting things if you were evil but um we're we're using
00:23:28.640
it for good here. It's fine. Um but the the end result of this though is that if we are in a tenanted context we can act
00:23:36.080
ask active storage record what's your current tenant and it will know and the connection pool is identical and this is
00:23:42.880
how we do joins across all of our models and uh if we ask for the actual connection we can see that shard fu is
00:23:49.600
set. So all these rails records are still are going to be using your attendant connection and this is exactly
00:23:54.720
what we want. Let's talk about the fragment cache. The
00:24:00.320
fragment cache for uh most Rails objects by default is going to be something like
00:24:06.240
here it says users one. It's like the table name and then the ID. And if you've been paying attention, you know
00:24:11.919
that that's going to be uh a conflict. We're going to have a lot of records in the application with ID1. So there's
00:24:18.720
going to be a user ID one in every tenant. So something like this could totally happen where you're going to get
00:24:23.760
a um you're going to display a cached version of foo's data to a bar user and
00:24:29.120
you don't you don't want that if you're using solid cache. One solution to this is that you could use
00:24:35.279
subtenant of on solid cache which means that essentially all of solid cache is going to be in that tenant database too.
00:24:41.840
So the the cache isn't mingled with all of your tenant data together. It's one cache database per tenant and that works
00:24:49.440
okay but it only works if you're using solid cache. What if you're using memcache or reddus
00:24:55.520
the solution is to actually modify the cache key that rails returns and we just jam the tenant ID into the end of the
00:25:01.039
cache key. Now it's unique and now if you go back to the situation where you've got a fragment cache, everybody's
00:25:06.400
going to get their own entry. Works pretty good. Not too hard. Um, Active Storage has a similar problem where the
00:25:12.000
blob ID, sorry, the blob key is what's used to actually store it in S3 or in
00:25:17.440
your disk store. And that means all of your tenant blobs are going to be mixed together. Makes it hard to delete a tenants or uh
00:25:24.720
one tenants uh blobs or to maybe give them a zip file full of their their blobs. So, what we do is we do the same
00:25:31.679
thing. We stick the tenant ID on the front though so that everything is in a distinct folder on S3 or it's in a
00:25:38.080
subdirectory if you're using the disk store. So this is good kind of similar patterns coming up. Um but a really
00:25:44.640
interesting case is active job. So code like this we're kicking off a job with
00:25:50.240
tenant fu. We have a user record being passed in as an argument that's from tenant fu. Like how the hell does this
00:25:57.120
work when this arrives in the job worker? How does the job worker have any idea what you're talking about? So the
00:26:03.440
way we do this is uh we extend active job base. We add a tenant attribute and
00:26:09.120
we make sure that when the job instance is created we check to see if there's a current tenant in context and if so we
00:26:15.039
save it off as the attribute. When we serialize the job the tenant is serialized and deserialized. So when it
00:26:21.360
gets stuck into the job Q table uh that that will be there and then perform now checks and says oh is there a tenant
00:26:27.520
attribute? If so, I'm going to wrap perform now in a with tenant. And that way, the job just knows what tenant you
00:26:34.720
you intended to run this for. Um, and you also haven't broken the untened case where you just want to run a job and you
00:26:40.240
don't care about tenanting. That still works. Imagine this scenario though. You've got
00:26:45.520
a user that you pulled out of tenant fu and then accidentally you're in you're in a different tenant context and you
00:26:51.600
call perform later. Like, do the safety checks cover this? They don't actually.
00:26:56.960
So we have to dig into global ID and global ID is the gem used to serialize Rails objects and normally similar
00:27:04.720
patterns everywhere right normally the ID is uh application name slashclass
00:27:10.880
name slash id and this is going to have the obvious conflicts so if we don't do anything this is actually going to
00:27:16.720
delete the wrong user's comments we don't want that so we go into global ID same pattern we add the tenant name onto
00:27:23.120
the end of the global ID make it unique and then We have to make the locator class also unique. So locator is what
00:27:30.000
actually goes to the database and fetches the thing for you. If you give it a global ID, you ask for an object back. It needs to know that in this
00:27:36.960
first case user is a tenanted model and we haven't given it a tenant attribute.
00:27:43.440
So it's going to refuse to to handle that. It'll give you an error saying this is an invalid ID. Uh if we try to
00:27:50.720
load a tenanted record not in a context, it'll raise an error. If we do it in the right place, we'll get the object back. And if we use the wrong tenant, we'll
00:27:56.720
get the rook error. So all the safety checks from active record are also present in global ID, which is great.
00:28:01.840
And so if we go back to this case, the job is going to fail with a literate exception saying wrong tenant error.
00:28:07.520
This is exactly what we want. Uh action cable action cable um is a lot like the
00:28:14.159
middleware where we can use that tenant resolver proc that you've defined uh to
00:28:19.679
figure out on an action cable connection which tenant is in the context for that connection. So every user will have a
00:28:25.919
tenanted action cable connection and then any commands will get wrapped with with with tenant as well. Uh and finally
00:28:33.760
to turbo frames and streams and a lot of other pieces of rails use global ID under the hood and so everything just
00:28:40.960
gets tenanted kind of automatically like making global ID support tenanting really got me a long way.
00:28:47.520
Action mailer base supports the percent tenant specifier as do a couple of other components. uh and load async
00:28:54.880
interesting edge case. If I call load async in fu context, but that doesn't
00:29:00.799
get resolved until later when maybe I'm in a different context, it's smart enough to handle that as well. It'll give you back the right user from the
00:29:07.840
right uh database connection. So if anything goes wrong, SQL query
00:29:14.080
logs will include the tenant in the log. And also normal tag logging kicks in as well. Uh, and if you're going to go to
00:29:21.360
uh Adriana's talk about uh structured logging later, I still have to make it support structured logging, but I'll get there.
00:29:28.159
Okay, we're almost done. Uh, testing framework. Your tests shouldn't have to know about tenanting either. Like, by
00:29:34.159
default, all of your tenants are going to get all of these things. I'm not going to go into detail. Just uh trust
00:29:40.320
me that I've thought about how to write tests for all of this. And in summary,
00:29:45.840
this baby draws the whole owl. Okay, a lot of details. I'm sorry about that.
00:29:51.919
Uh, looking ahead, uh, I would like to do a few things
00:29:57.679
before I ship a 1.0 of this, but really what it boils down to is trying to
00:30:02.720
manage the connection pools. Like right now, um, it's possible if you have one request each from 10,000 customers, uh,
00:30:10.880
that you're going to have 10,000 connection pools in memory, that's not going to be a good scene. So what I want to be able to do is set a cap on the
00:30:16.320
number of connection pools and then reap unused ones over time. And this will make it a little bit more manageable
00:30:21.679
with SQLite. That's not a big deal because creating a new connection is pretty fast. It's just a file descriptor. Uh I'm not sure how that's
00:30:27.440
going to perform with Quest or MySQL, but I would like to find out with somebody here who will help me with it.
00:30:32.799
Um a couple of rough edges specifically around database task support. Um, and this is because database tasks are um
00:30:40.720
I'm not going to make a judgy comment, but like it's really complicated to make database tasks work well and to override their behavior. So, I really need to put
00:30:47.039
some work in on that before I cut a 1.0. And with that, I just want to let you know this is open source now. I just
00:30:53.279
pushed the button. Um, if you'd like to go to basecap active director tenanted, you can try it out. I would love
00:30:58.320
feedback on this to see if it solves anybody else's problem as well. Uh if you're thinking about using SQLite uh in
00:31:05.120
production, you're probably going to want replication. So hang out in this room after lunch. Kevin McConnell is
00:31:10.399
going to give a talk about SQLite replication. Um and thank you for your patience and for coming to my talk on
00:31:16.320
multi-tenant rails.