00:00:10.080
I feel like I'm supposed to start with something upbeat like, "Woo! Yeah, we're here in Amsterdam. It's amazing." But
00:00:17.199
you know what? I'm not going to do that. I'm going to give it a little rant just at the start here instead because I
00:00:23.600
continue to be amazed by this curious paradox we find ourselves in in our
00:00:29.279
industry. In many ways, this is the best time it's ever been to be a software
00:00:34.320
developer. There has never been more free open-source available. Computers have never been faster. Everything is
00:00:42.000
supposed to be wonderful. So why is it sometimes that things don't feel quite as wonderful? Why is it sometimes that
00:00:48.879
we can't recognize the progress we're all supposed to be enjoying and celebrating? Why is it in fact that many
00:00:56.879
days we are frustrated by things going not forward but backwards? I think one
00:01:03.359
of the reasons is that we've forgotten or pushed back the idea of figuring out
00:01:09.280
what is the actual problem we're solving. We have sliced so many of the big problems into tiny little problems
00:01:16.240
and then we work on that tiny little slice in our little isolated area and we
00:01:21.600
may make progress, we may make it better, but when you add all the slices up, we've actually gone backwards. It's
00:01:28.000
a very curious phenomenon and it's something that I've dedicated my career
00:01:33.119
to and my work on rails to arresting thinking not of problems as tiny little
00:01:39.280
slices that are isolated from each other but thinking about the whole problem all
00:01:44.880
of web development and as you'll see today more than even that
00:01:50.479
now I've been around long enough in this industry to remember some of the things
00:01:57.600
where we've gone backwards. When I got started in 99, we were
00:02:04.240
editing the web application I was working on with PHP HTML in one CVS
00:02:10.479
repository. And you know what? Somehow we managed to ship great software that was used by lots of people on 1999
00:02:18.239
hardware both on the development side and the machines. Today it seems like everything needs everything all of the
00:02:25.920
time with everyone involved to ship anything at all. What happened? How did we go from big dog to little dog?
00:02:36.319
Even more curious to me is the deployment story. A single developer in
00:02:42.560
1999 could deploy a change to their web application in literally 5 seconds. I
00:02:48.239
know because that's how I did it. I would drag my little file in my FTP manager to production and it would be
00:02:55.120
live in 5 seconds. Today, this is actually wrong. It says a 50
00:03:01.840
strong team in 2025 could deploy in 15 minutes. I've been talking to people for the last few days. No one's deploying in
00:03:08.319
15 minutes. They're deploying in 30 minutes, in an hour, in three hours, in 8 days.
00:03:14.720
Something has gone seriously madly wrong when we have regressed from the ancient
00:03:21.120
ways of deploying with FTP in 5 seconds to the modern world where it takes an
00:03:27.840
hour. What the hell?
00:03:33.040
The idea of how many moving parts and components we need to create our web
00:03:38.239
applications today have also absolutely exploded.
00:03:43.440
to do what exactly? Are we creating applications that are magically more
00:03:49.120
wonderful than they were in 1999 using space age technologies to accomplish
00:03:54.560
glorious things? No, we're doing the same [ __ ] we always did. We're reading
00:03:59.840
from a database and we're writing to a database and then we're formatting that output into some HTML tables. Oh yeah,
00:04:06.319
we use CSS now. What wonderful progress. And in that complication and in all
00:04:13.200
those moving parts, we have made things more brittle. We have gone backwards. It
00:04:19.199
was very common in 99 to have three nines of uptime. Today it seems like people can barely manage that even with
00:04:25.759
all their Kubernetes, all their AWS, all their data dogging, all the everything.
00:04:31.759
We aren't making things better. The common explanation for this is we're
00:04:39.680
being conned by merchants of complexity. These devious companies who convince us
00:04:47.040
to use their complicated tools and products and somehow they just snooker
00:04:52.639
us. We're really smart and we wouldn't do that normally, but just they're so
00:04:58.240
devious. They're so clever. I wrote a blog post a little while back
00:05:04.080
saying, do you know what? I don't actually think that's the problem. I do think there are merchants of complexity.
00:05:09.120
I do think most of the companies I mentioned before are merchants of complexity. Then they love to sell you
00:05:14.160
complicated [ __ ] that makes the world go backwards. But you know
00:05:20.080
what? The only reason they can sell that and grow so big is because you don't buy it.
00:05:27.759
It takes someone to purchase from a merchant of complexity for the merchant of complexity to make any money at all.
00:05:33.360
And they're making a lot of money. And why are they making all that money is because we have a psychological need to
00:05:42.560
be seen as more than just crut monkeys.
00:05:49.360
I'm a crut monkey. My career is crutoning. It is reading things from a
00:05:55.680
database, creating records, updating those records, and occasionally deleting them. That's a very simple job as it at
00:06:03.680
its essence. Using the web to do that job is a wonderful way of making a
00:06:08.720
living. But you know what? It feels too simple. It doesn't feel computer sciency enough. I have to employ my CS degree
00:06:15.759
here. And now we need more complication. I actually think that's one of the root
00:06:21.600
causes. We have to get back to embracing happily, proudly the monkey.
00:06:29.759
And we are that monkey. And when we do and we embrace the simplicity of the
00:06:36.479
crud, we are free to begin solving problems end to end. And when we solve
00:06:44.400
problems end to end, because we don't over complicate things, we can think about things at a higher level. Now that
00:06:50.160
doesn't mean pairing everything back to the bone because that's actually not endto-end problem solving. Ruby and Rail
00:06:56.319
since the start has been a mega framework. It has tried to do everything
00:07:02.160
required for most people to create a web application with everything they need.
00:07:08.880
And you know what? They need a lot of [ __ ] This isn't about minimalism. In fact, in many ways, it's about
00:07:14.560
maximalism. It's about solving all the little annoying problems that we all
00:07:20.160
share when we're working making web applications. And it takes a lot. It takes HTML templates. It takes database
00:07:27.520
access and routing and sending emails and background jobs and and all this
00:07:32.880
stuff. I think this has fallen out of favor.
00:07:38.160
And it's fallen out of favor from people who make these kinds of frameworks. The
00:07:44.800
new frameworks you see announced every day, none of them are trying to do this. None of them are trying to address the
00:07:51.440
whole problem. And if they are, it's because they just raised a series A funding and just started a million
00:07:56.479
projects. And well, good luck with that. But in the open source world, I think we
00:08:01.759
need to get back to solving full problems as Rails has done since the beginning.
00:08:07.599
And one way of thinking about this is the Roman Empire.
00:08:14.800
I like to think not about the end of the Roman Empire as the meme goes. I like to
00:08:20.879
think about the height of the Roman Empire and Pax Reelsana is that height of the
00:08:29.120
Roman Empire that we have all these domains, all these provinces and they're
00:08:34.719
all working quite nicely. And the reason they are working quite nicely is because it's not just based on
00:08:41.680
technology. It's not just a collection of frameworks. It's not just a collection of solutions. It is a
00:08:48.800
philosophy of life. And that philosophy of life, I found, has given me the
00:08:55.600
interest and the motivation and the ambition to keep going, to keep expanding this glorious peaceful empire
00:09:03.600
further north, west, east, and south. And the way I think about that in the
00:09:11.120
most glorious terms possible, libertas, proprietas, pas. We have freedom to do
00:09:18.640
whatever we want with this framework that we all share. It comes in a form of
00:09:24.160
a macass, but you can change it however you want. It's just Ruby code. You can literally do bundle open, see how
00:09:30.560
everything works, and if you don't like it, you can just change it. Hell, you can monkey patch string to do something
00:09:36.560
nutty. That degree of freedom to shoot yourself in the head is wonderful.
00:09:44.160
I pride the fact that we use sharp tools in the Rails community and we are not
00:09:49.839
afraid of our citizens propriet. We have full ownership of what
00:09:55.760
we work on. No one is coming down to tell us that this is how it has to be done. This is how it has to work. There
00:10:01.200
are suggestions. I will occasionally tell you that I think RSpec syntax is
00:10:06.240
ugly as sin and you should stop using it. But you know what? You're free to ignore me as many of you have done for
00:10:11.680
many years. You bastards.
00:10:17.760
And finally, pas duty. We are not just
00:10:22.800
consumers of a framework. We're not just beneficiaries of some magic open-source
00:10:29.040
vendor in the sky dropping little packages on us and gifts. No, no. You're
00:10:35.600
supposed to do your part for the empire. You're supposed to do your part for the framework. You're supposed to do your
00:10:40.880
part for open source. So, that's where I'm going to start. I'm going to start with my part. what I am
00:10:47.680
doing for the empire, what I am doing for the open source, what we have been doing in the past year at 37 signals to
00:10:54.880
bring gifts to all of you and we will start
00:11:00.480
or we will commence in two ways. First more
00:11:05.839
we will expand the borders of the empire. We will conquer new territories,
00:11:11.360
new problems, new provinces. And let's start with a little more.
00:11:19.760
Little more is Markdown.
00:11:24.959
Markdown is wonderful. It's the new lingua frana of AI. AI loves markdown.
00:11:31.279
And it also, for some curious reason, really loves emojis. Wonderful. Well, let's make it even
00:11:36.720
easier for people to use markdown in Rails to produce markdown to make that a
00:11:42.480
common expectation. I've been writing a lot of markdown recently. I've been writing a various manuals for Linux
00:11:48.720
adventures. It's all written in markdown using a editor here we call house. I'll
00:11:56.160
talk about that later. But that's the input part. And what I found is we need
00:12:02.800
to make all the stuff that we're making easier to consume by AI in a format that AI prefers and AI prefers markdown. So
00:12:12.240
this is what it's going to look like in Rails 8.1. When you want to do markdown, you will simply do format.md.
00:12:20.079
You can do a render markdown and you can pass it any object and we will try to duct type that object to call to
00:12:27.040
markdown. This is a PR that's still pending for writebook. I am going to ship this maybe during the conference
00:12:32.880
such that you can download an entire writebook, the manual for a march or anything else as a single markdown file
00:12:38.560
and it'll be running this exact code. But let's go up a little bit.
00:12:45.839
We have a new text editor coming for action text.
00:12:52.000
It's called Lexi. And the reason we need that might not come
00:12:58.000
as any surprise for people who've been watching the evolution of Trixs, our current editor in action text. Trixs has
00:13:05.760
been a project that we at 37 signals have been operating and using for many
00:13:11.120
years. Didn't find a lot of people wanted to help us out improving that. So
00:13:16.240
it ended up being just a project on the back boner back boner on the back burner
00:13:22.000
of everything else we are making at 37 signals and it did not get a lot of
00:13:27.920
love. So, do you know what? Sometimes you just got to accept when to throw in the towel. And we are throwing in the
00:13:33.680
towel on tricks as the new default. And we will make Lexi the new default um
00:13:40.800
editor in action text and therefor
00:13:48.800
you kind of have to squint to tell the difference from tricks because tricks actually covered all the basics
00:13:54.320
adequately. But we can cover it better than adequately. One example here for the keynup server is that this is a live
00:14:00.880
view of the text editor that has live syntax highlighting. Live syntax
00:14:06.160
highlighting for Ruby, live syntax highlighting for a lot of different languages. That is actually really nice
00:14:11.600
when you're inserting code into your base camp or your other app that's using uh action text. What's also nice,
00:14:18.959
following on the markdown announcement, you can paste markdown straight into this editor and it'll convert it to the
00:14:25.279
Vizywick and you'll be able to do that. You can get Markdown out as well, too. Um,
00:14:31.600
we're still working on this at 37 Signals. I actually just found a bug when we were uh I was recording this and
00:14:38.800
I captured this. This is a screenshot from a new application we're working on at 37 Signals called Fizzy. We're using
00:14:45.839
the Lexi text editor in Fizzy. Um, and it's pretty wonderful. But what is Lexi?
00:14:52.560
Lexi is a implementation or a use of Lexical.
00:14:58.320
Lexical is a text editing framework made by the fine folks at Meta. It powers
00:15:04.639
WhatsApp and Instagram and Facebook and about I think they brag 1.7 billion
00:15:10.320
people every day use it. They have a team just doing that. Wonderful. Let's
00:15:16.079
give some of our work and some of our load to to other folks who are really motivated for it. Um and the people at
00:15:22.320
Meta are very motivated to maintain Lexical. Lexical powers web text editing
00:15:27.680
experience for hundreds of millions of users every day across Facebook, Messenger, WhatsApp, and Instagram. Love
00:15:33.199
it. Love having an upgrade where we get all the benefits and someone else does the work. Or what I like to call
00:15:40.399
performance optimization. And hello Aaron. Um, but what what uh what about this guy
00:15:47.760
House? Last year I talked about House, a new text editor for markdown that we've been
00:15:53.440
using actually in rightbook. And um we tried to use House in two new products
00:15:59.360
we were working on. One was Fizzy and the other one hasn't been announced yet. And in both of those two cases, we found
00:16:05.440
out that uh most people actually don't want Markdown. Markdown is a wonderful tool for developers. It's a wonderful
00:16:12.959
tool for AI and normal people go, "What's that hashtag again that you have to push to be the a the big headline?"
00:16:20.880
We switched over to Lexical for both of those two products. House is probably
00:16:26.240
not going to be something we're going to invest further into. Um, we're going to move forward with Lexical.
00:16:32.000
Another wonderful addition, the more we're adding here is action job continuations.
00:16:37.680
It is a new way of structuring your jobs such that they can be interrupted and
00:16:44.480
resumed without losing all progress. This was something we found was very needed at 37 signals when we
00:16:51.680
moved out of the cloud which we did a couple years ago. We moved to Kamal. We moved to a setup wherever we deploy a
00:16:58.880
new version of our app, we will shut down any container that's running with the old code within 30 seconds. Now,
00:17:05.520
those 30 seconds are not very compatible if you have a long job that's doing a big task like exporting all of your base
00:17:12.400
camp. That might take actually a few hours if you have a huge account. And being able to resume that in a
00:17:17.839
step-by-step manner just means we're not throwing away work. It's a wonderful way of cutting these things up. Making sure
00:17:23.679
that all your jobs are resumable. Check that out. It's going to be in Rails 8.1 as well.
00:17:32.640
But those ideas are sort of within the known borders of the empire.
00:17:38.640
We also happen to border another empire, the empire of the native apps. And you
00:17:44.240
know what? They're a little hostile over there. But we do have to trade with them. We cannot just seal off the border
00:17:50.480
and pretend that the native frontier isn't there. We cannot pretend that you
00:17:57.679
don't need native applications to make competitive web applications. We found
00:18:02.799
this to our slight surprise when we launched Hey about four years ago and
00:18:09.840
Apple unceremoniously tried to murder us. Now thankfully they were not successful and I don't hold a grudge.
00:18:20.080
I very very much hold a grudge. Um we'll get back to that in a little bit. But
00:18:25.840
grudge or no grudge, we still need to make native applications and we need to
00:18:31.760
make them better and better because that's what the competition is. And you can't just throw in your towel and fold
00:18:38.160
your hand and pout and say, "I don't want to." If you have a [ __ ] business and I happen to have a [ __ ] business
00:18:44.480
and I'd like that [ __ ] business to make some money and that requires customers and customers want native
00:18:49.520
apps. So we suck it up and we do it. One of the things we've been building
00:18:54.640
with for since the start is hybrid applications, hotwire native, everything
00:18:59.840
integrated. And the first question anyone asks when they look at that and say, "This looks amazing."
00:19:05.919
But why do people need their travel email when they're on a plane and they don't have internet? What do they do
00:19:12.880
then? Well, you could build something. That's your problem. No, it's not. Everything has
00:19:19.200
our problem. And if offline is a thing everyone needs, we should solve offline.
00:19:25.120
So that's what we've been working on. Turbo offline is coming. We've been
00:19:30.160
working on it for a while. This has been the work of uh Rosa working on it. We're going to tackle the problem.
00:19:37.360
Yes. And the wonderful thing about solving it
00:19:43.600
at this layer, solving it at the turbo layer, is we don't just solve it for native. We actually solve it for all web
00:19:50.480
applications wherever they're being used. And it's being solved in two steps. V1, which is shipping very
00:19:55.600
shortly, it'll be a cache as you go. If you go to an email in Hey, that email
00:20:01.360
will be cached on that device and you can return to it without having network connection. Step two will be cache as I
00:20:08.320
say so you we can preemptively cache certain pages like your travel folder
00:20:13.760
without you having to visit individual emails. That's going to be awesome. Now, Turbo is integrated into Hotwire Native,
00:20:21.760
which is the all-encompassing framework for dealing with everything native on
00:20:28.000
Hotwire. And I'm going to talk exactly zero about that because later today we
00:20:34.159
have the expert Joe in the last keynote of the day talking about Hotwire Native
00:20:40.159
1.3 that's shipping very shortly. It's going to be awesome. The other thing that's going to be
00:20:46.080
awesome in the native frontier is finally solving push.
00:20:53.760
Finally solving push notifications to native devices and more.
00:20:59.760
This was known as action notifier when I was teasing it last year. Action notifier
00:21:05.280
um morphed into action push. And today we actually have the first part of
00:21:11.039
action push, the action push native part because we needed to solve that equation
00:21:18.000
very quickly because we are nuking our AWS account in just a few moments. And
00:21:25.039
that required we stop using AWS Pinpoint to send out our native push
00:21:30.400
notifications and switch to using directly notifications sent to Apple and
00:21:35.440
sent to Google. action push native does that for you. Wraps it up beautifully.
00:21:40.640
Um reduces all your push notifications to this lovely syntax. You create a new
00:21:46.080
device with a new token that you will send to on a given platform that can be either be Apple or or Google. And you
00:21:52.640
can then design these uh or use these action push notifications attaching metadata that's relevant
00:21:59.600
either only on Apple devices or on Google devices and deliver those to the
00:22:04.640
um devices. You can even do things like silent notifications if your app depends on that for certain types of updates.
00:22:12.159
It's really wonderful. Um, this is the configuration schema for it. There's going to be a new config push YAML where
00:22:19.200
you set things up for your Apple integration and your Google integration. You put in your keys and off you go.
00:22:27.120
It's pretty awesome. And we've been using that in production for a few weeks after we did the cut over. and it's
00:22:32.799
sending I don't know I'm just gonna make up a number millions of notifications out
00:22:38.559
already. It works really well. The other part of the equation is web push. I was
00:22:45.360
super excited about web push when we launched Campfire a little while ago. Campfire has a
00:22:52.880
native web app. It's not a native application you install through the um app stores, but it uses all the latest
00:22:58.960
and greatest web native uh technology, including web push. And it works pretty
00:23:05.280
well. It's not perfect, and that's one of the reasons why action push is not
00:23:10.559
just action web push. Native notifications still have some advantages. You can pull them back and
00:23:16.080
deliverability is slightly improved. But if you're using campfire during this conference and you're getting push notifications on your phone, it's coming
00:23:22.960
through essentially uh action web push. Um
00:23:30.159
that code looks something like this. This is code from inside of campfire. This is the action web push code inside
00:23:37.919
of campfire. We have not yet extracted it. It's a little more cumbersome and a
00:23:42.960
little more complicated to use than the native stuff. You have to do a little JavaScript dance to get the right keys
00:23:48.640
and everything out. I'd very much like to see this extracted. I had meant to do
00:23:53.840
the extraction, but you know what? I have a lot of [ __ ] on my plate. And therefore, I'm going to do something even better. Instead of charging $2.99
00:24:02.080
for Campfire, we're going to give it away for free.
00:24:13.760
because free asterisk. I need action web push extracted and one of you is going
00:24:20.320
to do that. That's the deal. That's why you're getting it for free. We need this code out. Please go check it out. This
00:24:27.039
URL is going to be active right after the keynote or shortly thereafter. We need this extracted. I'd love for one of
00:24:32.640
you to work on that. So, that's a lot of more. That's a lot
00:24:37.840
of more we're adding into the framework. There's a bunch of ideas here. Some of this is shipping already in Rails 8.1.
00:24:43.520
Some of it is going to ship in the next version of Rails. But to have a healthy empire, you need
00:24:49.919
more. You need no more territories, more conquest. And you also need to prune. You need to retract in certain uh areas.
00:24:57.440
You need to realize where do we need less?
00:25:03.520
Because this is the yingyang of framework development. You can't just grow grow grow without also pruning away
00:25:10.159
the dead branches and the dumb code. And we do have dumb code and we do have dead
00:25:16.000
ends and we do have areas of the framework where things have evolved to a point where we don't need it anymore.
00:25:21.039
Sometimes because we simply realized we were wrong. other times because the
00:25:26.480
underlying technology moves forward. For many many years I used something called Puma Dev to run all our local
00:25:34.320
development and it was a whole dance of setting that up and it didn't work very well on Linux and then when I arrived on
00:25:39.440
Linux I said what is this [ __ ] and then I ripped it out and replaced it with the glory that is
00:25:46.240
local host. You don't need Pumadeev. You don't need any process to run your local
00:25:51.760
applications in Rails. You just need localhost. And the reason you just need localhost is that in 2017, localhost was
00:26:00.080
made a secure context. There is no reason for you to run a SSL certificate
00:26:06.000
locally during development if you run off local host. I didn't know this. I
00:26:12.240
found this out when I was annoyed by the fact that PumaDev on Linux is six steps
00:26:17.279
to set up and they're all cumbersome and annoying and I didn't want to do it. So I found out that local host can actually
00:26:22.799
do all this work. We don't need it at all. We just rip it out. And that's what we did at 37 Signals. This is what our
00:26:28.400
development URLs now look like. They have explicit ports, which at first I
00:26:33.520
thought that's a little weird. And then I thought, no, it's [ __ ] not. The
00:26:38.640
port corresponds to a process running under my machine. So if I want to run uh Basecam 3 plus launchpad plus queen B,
00:26:46.720
that is three different processes. They should each run on their own ports. And why can't we just make them explicit?
00:26:53.120
Half the challenge of dealing with software is what is the problem? Is this
00:26:59.200
worth a convoluted solution to get rid of these ports at the end? No, it's not.
00:27:04.320
You can just write them down. Oh, yeah. 30 10. That's the one for queen bee. So,
00:27:09.919
we just all use that and then poof. whole problem of how to boot up multiple
00:27:16.000
local instances of your Rails applications just go away. Wonderful.
00:27:22.480
The other thing was that we over several years were toying with how to use
00:27:28.320
Docker. Docker is incredible in a million ways and a total pain in the ass
00:27:34.320
to use for development. That took us a little while to come to
00:27:39.520
the realization that that pain in the ass didn't need to be there. We could get the best of Docker. What Docker does
00:27:46.400
so exceptionally well is when you have services that are sort of all encompassing for the system like running
00:27:53.279
my SQL and you need to run that in different versions because one of your apps needs 80 or 80, another one needs
00:28:01.039
804. It's really nice to be able to have multiple versions of that and you can't do that if you install it natively. Now
00:28:07.440
that led to a realization for a lot of people, well, we should just do all our development inside of Docker. And then
00:28:12.720
they ran into all the multiple problems there are with doing just that. But you don't have to choose. We can just run
00:28:18.720
native Ruby, which by the way now is managed beautifully by a project called
00:28:23.760
MI that makes it exceptionally easy to install different Rubies of different versions and keeping everything
00:28:29.520
separately. And you don't need Docker for that at all, but you still need Docker for your DBs. So that's what we
00:28:34.880
did. We simply share a set of Docker DBs that are versioned such that if an app
00:28:41.440
needs 8, that's the process it'll go to, that's the port it'll attempt running on. And
00:28:48.640
if it needs 71 of Reddit, it'll do that. And if a different project needs something else, it'll do that. really
00:28:54.159
just solved the main hurt and the main problem we had with using Docker in development isolated just for the
00:29:00.000
databases. Even more consequential for us, an even
00:29:06.000
bigger painful I was about to say branch, but I actually almost feels like a broken tooth. [ __ ] system tests.
00:29:14.159
I stood up on a stage not too dissimilar for this 10 years ago and proudly declared that system tests were the
00:29:20.399
future. System tests in theory are wonderful. You get to test everything
00:29:26.559
end to end and flow through the browser and you test your HTML and you test your back end and the [ __ ] doesn't work. It
00:29:34.000
never worked. It never works. It's always brittle. It's always broken. It's
00:29:39.600
always slow and it's totally not worth it. That realization took even longer to
00:29:45.760
arrive at than fumbling around with Docker. But finally
00:29:51.440
uh last year we merged this pull request for Hey. Hey
00:29:56.480
had a lot of system tests and they took a long time to run and developers were constantly battling how to make them
00:30:02.559
stable. And we said enough. What if we just nuke the whole thing from orbit?
00:30:09.440
It's the only way to be sure and leave only 10 system tests left as a smoke
00:30:15.440
path of just does the whole thing work. don't use it to test anything in depth. And I've never heard of a single bug we
00:30:23.600
pushed to production in hay that this [ __ ] would have caught. It was not worth it. Therefore, we are yanking the
00:30:31.600
recommendation in Rails 8.1 that all new controllers should have system tests. We
00:30:36.880
are no longer going to be generating them. The framework is still in there. I think it can make sense for smoke tests
00:30:44.240
in a very small proportion but for us it was absolutely not worth the trade-off
00:30:49.760
otherwise and these are some of the realizations I think are really important to have that you go in in a
00:30:56.000
new domain a new territory and think this is this is going to be great and then when it's not great
00:31:03.279
you like you go back into the little hedge and say like system test
00:31:10.480
No. Speaking of tests,
00:31:17.440
do you know what really uh pissed me off as we were getting out of the cloud
00:31:24.480
was this [ __ ] thing. The cloud test runner. This is small small text. So
00:31:30.799
I'll read it for you. This was a test run on BCX, the second version of Basecam that we're still running. a
00:31:37.120
relatively small app, smaller than hey, smaller than base cam 3. It took 10 [ __ ] minutes to run these and it ran
00:31:43.760
it in, let's see, eight different paralyzed projects that were all carefully set up on a cloud runner and
00:31:51.760
it took 10 minutes. Total pain in the ass. Totally too long. Um,
00:31:57.840
but I didn't like what do you do? Like what what do you have a lot of tests? We have to run the test. We can't just kill all of them. maybe we can kill the
00:32:04.399
system test, but we're not going to kill everything. Um, and then I was like, you know what, that made sense in 2018,
00:32:12.080
2019. We had Intel chips in our MacBooks and they sucked and they basically
00:32:18.960
hadn't changed for like six years because Intel just ran into the dirt and could not figure out how to get their
00:32:24.720
new node out. You had all this cloud CPUs, AMD epics going awesome.
00:32:30.399
Everything was great in the cloud. the development scene sucked and then something magic happened. Um TSMC
00:32:39.279
through the grace of Apple and others really improved the chips we have in our
00:32:45.120
local development machines to the point that the chips the CPUs we now have are way better and way faster than the [ __ ]
00:32:52.880
we rent in the cloud. Here's the hey test suite. The basecam
00:33:00.399
test suite, as I just showed you, have 13,000 assert uh assertions. The Hey test suite has 30,000 assertions.
00:33:09.760
I saw a cloud run for the hey test suite that took well over 15 minutes.
00:33:15.519
You can't maybe read it, but you know what the fastest one is on the fastest AMD you can buy today? 1 minute 12
00:33:22.000
seconds. Even on an Apple chip, an M4, it's under four
00:33:28.640
minutes. This is totally doable to run locally
00:33:34.640
for even a relatively large app like hey. And many of you have smaller apps
00:33:40.399
than that. And a few of you have much bigger apps than that. And none of this [ __ ] applies to you. You're not going to be able, sorry, to run the Shopify test
00:33:46.880
suite locally. That's not happening tomorrow. But that's the 0.001% 001% of
00:33:51.919
everyone making Rails applications. The vast majority of you should be able to replicate these numbers and run your
00:33:58.559
entire test suite locally. And if you do it on Linux, it's about twice as fast as doing it on
00:34:05.279
a Mac. Um, you got the M4 Max here, the fastest Apple machine I've tested, 2
00:34:12.560
minutes and 22 seconds. And right below it, you have 2 minutes and 5 seconds for my trusted framework uh laptop. And then
00:34:20.480
you have the framework desktop at 2 minutes and 23 seconds. This is totally fast enough. You just run it locally and
00:34:26.159
you can just blow up all that complexity of running cloud runners. Now
00:34:32.240
here comes the uh common objection. Well, if we allow developers to run things on their own machines, first of
00:34:37.919
all, you're going to get it works on my machine. Yeah. You think just because it [ __ ]
00:34:43.440
works in the cloud that it it works in the cloud runner like that doesn't mean anything extra. What means actually is
00:34:48.800
you pin down your dependencies, you pin down your versions. This is why we did the other stuff. We use the same version
00:34:54.079
of the database, the reddest, the everything in Docker and we can pin it exactly. We're not dependent on a random
00:34:59.839
version in homebrew to bail us out. We use mi to pin exactly the version of
00:35:06.160
Ruby that we're going to use. And between those two things, the Ruby version and the
00:35:11.760
database versions, that's it. any esoteric language that you or um library
00:35:19.040
you might find that isn't the exact version is not going to [ __ ] do anything. We've been running this setup for quite a long time. We haven't had a
00:35:24.800
single issue. And if you are, weigh it against the joy it would be to know
00:35:30.079
whether your PR is ready for deployment in 1 minute and 12 seconds. Now, we
00:35:35.599
actually do protect the PR is that they can't just be merged now that there's no cloud runner. We use a wonderful feature
00:35:42.320
of GitHub where you can require sign off from a past uh test. You can install
00:35:48.640
that with a tool that we have and when you do it requires a human to certify that they have signed off, that they ran
00:35:55.280
the test and they all passed. This sounds like it shouldn't work because it sounds like it depends on trust. But you
00:36:01.040
know what? Trust works. In most organizations, your employees are not [ __ ] They're actually programmers
00:36:08.560
who want to ship working software and they know that if they lie on the attestation here, they will get fired.
00:36:15.200
It's a really effective enforcement mechanism, I actually say. So, not a problem. In Rails 81, we have
00:36:23.520
baked in a way to do all of this locally, not just a test. There's a brand new DSL for declaring your local
00:36:31.280
CI run where you can run your Robocop, you can run your gem audit, uh, and you can run all your tests and you can do
00:36:36.960
the sign off at the edge to clear the PR for merging. When we run this locally, it takes no time at all.
00:36:44.160
Um, let's shift gears. All this benchmarking I did with
00:36:50.640
different local computers around how fast they can run your local tests and realizing why are we paying cloud
00:36:57.359
providers to run slow tests when we can run them real fast on the machines we already have. Reminded me of this. This
00:37:03.200
is Fortnite. Fortnite has to render an entire scene of 3D graphics
00:37:11.440
120 times a second. They have 8 millisecond to render this [ __ ]
00:37:16.880
scene. And you're telling me it's got to take 15 minutes to pass some goddamn paper tests in the cloud.
00:37:24.160
No, that's unreasonable. And when I thought about that, I thought like, you know what? There's a lot of things that
00:37:29.280
are really unreasonable. We are waiting way too damn long on way too many damn things. The example I gave at the start
00:37:36.000
is a great illustration. FTP. We could deploy in 5 seconds. Now we're confident
00:37:41.520
or content. deploying in 15 minutes, half an hour, three hours, eight days.
00:37:47.839
No, no, no. We're going to solve all this. We're going to apply engineering principles and we're going to have a
00:37:54.160
frame budget for everything. And where I'm going to start is we're going to have a bootstrap budget. The idea that a
00:38:00.720
new developer joining a new company should spend a few hours setting up their damn laptop is preposterous.
00:38:08.960
At 37 Signals, this is our bootstrap budget. In 15 minutes, you should be able to go from cold machine that has
00:38:15.760
nothing on it to having everything set up and a change deployed in production.
00:38:22.079
That's our budget. To make that budget happen, I listened
00:38:27.599
to Ellen Kish. People who are really serious about software should make their own operating
00:38:35.200
system. Let's do that. Let's make our own
00:38:40.560
operating system.
00:38:46.640
Let's do it right now. Let's see if we can be within budget. We
00:38:52.400
have 5 minutes to set up Umachi from this little USB key on a brand new
00:39:00.000
framework desktop. Here's the stopwatch. Let's go. I'm going to plug in the USB
00:39:07.200
key. I can fiddling in here. I'm gonna turn on the machine
00:39:14.400
and I'm gonna start the stopwatch. Let's go. Now, we're gonna install Machi on a
00:39:20.240
brand new framework desktop. As I said, there's um nothing we're going to keep forward. We're just going to blow everything away. This is a project I
00:39:26.960
have I was about to say spent the last two months about. That's not true. Obsessed the last two months about. let
00:39:34.320
my life be consumed entirely by a Linux god for the last two months as I've
00:39:40.480
dived into the idea that it should not take hours to set up a new machine. And
00:39:46.240
also, I should not have to ask Apple for [ __ ] permission about anything at
00:39:51.440
all. So, we're going to set up this machine. I'm going to enter my username. I'm going to give it a big quick
00:39:57.440
password here. I don't need to enter the name or email address. The host name, we just do Amachi. We are in Amsterdam.
00:40:03.280
That's true. Everything looks good. And we're going to install it on the internal NVME drive. Yeah, we're going to blow it away. Let's go. So, as this
00:40:10.800
thing is installing, let me tell you a little bit about Machi. I switched to Linux about uh 18 months
00:40:17.680
ago, and I did what everyone else does when they switch to Linux. I used Ubuntu. That is the most popular Linux
00:40:24.720
distribution. It's really nice. Canonicle did a great job at putting a
00:40:31.040
Linux distribution together that will not be too foreign or too scary for someone moving over for Mac. That's
00:40:37.359
exactly what I needed at that time. Something vaguely familiar but built on
00:40:42.400
the underpinnings of Linux. And I had a really good time for about a year tinkering with that and changing things.
00:40:49.040
And then I saw a YouTube video and the bastard who made it and dragged
00:40:55.520
me into this obsession is actually sitting right over there. It is Mr. Typcraft himself who dragged me into the
00:41:03.200
mysterious world of Linux ricing and tiling window managers and Hyperland in
00:41:10.720
particular. And what I found was like tumbling down the rabbit hole. What even
00:41:17.839
is this place? What do you mean I'm not using a mouse to move windows around anymore? What do you mean everything
00:41:24.079
just resizes when I open things? And why does it look so good? Why does it look
00:41:29.280
so magical? And why is everything a tuy uh terminal user interface? Wasn't that
00:41:36.400
[ __ ] we gave up in the 80s? And then I thought, oh, the 80s.
00:41:44.640
I like the8s. I remember the 80s.
00:41:50.000
They were cool. I remember the 90s when I was on the
00:41:55.520
AmIGGA and running a BBS and we had cool asy graphics. Oh, like
00:42:02.560
that. That's like the '9s. I remember BBS's.
00:42:08.640
They were really fun. What if computers were fun again? Wouldn't that be great?
00:42:14.720
Yes, it would be great. Computers should be fun. They should be cool. They should look a little bit like a hacker movie.
00:42:21.119
That's actually an aesthetic that's worth aspiring to. And what if we built an entire operating system in that
00:42:28.000
image, in those cool vibes, in that cool aesthetic? Oh, and also built on [ __ ]
00:42:34.400
Linux. The operating system that runs 95% of every server in the world that
00:42:40.560
drives the web. The system that has been optimized by a Finnish lunatic with a
00:42:47.520
foul mouth for the last 34 years. Yeah, that's the system actually I would
00:42:53.119
like to run. That guy is obsessed. And that obsession has produced the
00:42:59.839
greatest open-source project of all time. Linux is truly amazing. I've been
00:43:04.880
laing links for 30 years on the server and I've totally been sleeping on the fact that you can actually run it on
00:43:10.560
your computer, too, and it's gotten really damn good. Now, I know a lot of you out there tried some version of
00:43:17.359
Linux at some time and you couldn't get your [ __ ] Wi-Fi to work or your
00:43:22.640
printer wouldn't print or any of the other anecdotes that everyone who's ever tried Linux in the last 30 years will
00:43:29.119
have like a rolodex of. Do you know what? First, I will say this. Um, almost
00:43:34.319
all of it has been fixed. Do you know what? This is what happens if you just keep working on a problem for 34 [ __ ]
00:43:39.440
years. It ends up being really good in the end if you're that obsessed. We've seen it with Ruby. When I started using
00:43:45.760
Ruby, it was version 168. Do you know how much faster, how much better, how
00:43:51.200
much more featureful Ruby is today? Of course. Oh [ __ ] we're done. Are you
00:43:56.480
kidding me? That's not even five minutes. That's 4 minutes and 44 seconds. And then I just got to unplug
00:44:15.119
Whoa. That's pretty fast. Things get good if obsessed people keep working on them.
00:44:21.760
Especially open source. Especially if you can gather up an entire community of
00:44:27.359
people who are as interested as you are in making technology better. This is why
00:44:33.599
open source is winning in every domain that we care about when it comes to web
00:44:39.040
development. Do you buy editors anymore? Do you buy databases? Do you buy web
00:44:44.880
frameworks? No, you don't. You use open source ones that if you they don't do as you like, you just [ __ ] change them.
00:44:50.960
Isn't that amazing? Oh, here's Amachi. Hello. Let's make a Rails app.
00:44:57.200
Um, because this, of course, has everything you need to make a damn Rails app within the first five minutes of
00:45:02.400
booting this ship. So, we're going to hop in here. We're going to do Rails new blog. Of course, it's going to be a
00:45:07.920
blog. Not a blog in 15 minutes, but a blog in 5 seconds. And when we start
00:45:13.440
this, we also realized, you know what? This is still a little bit of work in progress. And if you run it on a low resolution screen like this, you're
00:45:19.599
going to have to hop into a config file and change that a little bit. But does
00:45:24.800
that matter? Are you afraid of a little config? No, you're not afraid of a little config. You're actually excited when you see Neo Vim pop up and like,
00:45:30.400
"Oh, that's what all the cool kids use. Can I do it, too?" Yes, you can. It's free.
00:45:36.480
So, we're going to do this. We now have our Rails application here as a blog. We're going to generate R, by the way,
00:45:42.640
because of course this has the aliases built in that I love to use. Just R, not rail, not bin rail, just R. R, generate
00:45:50.000
scaffold post title string. We got that going. Then we need to
00:45:57.119
migrate the database. And then we need to start it just on dev. And when we do this, we can hop
00:46:04.640
over here. Oh, hello Rails. Hello Rails. Hello Rails. Hello Rails. Oh, isn't that
00:46:12.480
nice? That's a nice air screen. And here's the real thing.
00:46:17.680
Hello world with a brand new OS in about six and a half minutes.
00:46:31.119
And not only just like hello world, but [ __ ] cool hello world that can change
00:46:36.319
it to green. Like, what? What about that, huh? Like, whoa, look at those windows. What if
00:46:44.800
[ __ ] just looked so cool everywhere? What if you had like a screen saver that
00:46:50.400
was like the '9s?
00:46:55.440
Wouldn't that be amazing? That is Omachi. Um, I've been dedicating the last two months on it. I'm going to make
00:47:01.839
it the damn best way to develop Ruby on Rails on it. And I hope you will give it
00:47:07.839
a go. Check it out even if you love your Mac. Um, we actually have 200 USB keys
00:47:13.839
with Umachi on it out at the base camp 37 SQL booth if you are serious about
00:47:19.280
installing it. And before you get that key, it cost me $7. So I will look you straight in the eye and say, "Are you going to actually install it? I'm not
00:47:25.440
giving you a [ __ ] souvenir here." So this is for people who actually want
00:47:30.720
to install it. Come out um and see us at the booth. So, let's get back to the presentation.
00:47:38.480
Here are my backup slides. In case the live demo blew up, it didn't. So, we
00:47:44.400
don't need them. Isn't that nice? I like when things work out. Uh, the great thing about making your own damn
00:47:50.160
operating system is you can put in all the stuff you want. I'll give you just one quick example here. Omachi ships
00:47:57.359
with git. It ships with mis. It ships with docker. It ships with databases. It ships with everything a web developer
00:48:03.040
would ever want. And let's dive into just one example. Git. So it ships with u lazy git which is an amazing tuy for
00:48:11.119
managing your git interactions. This is running inside of neovim. I used to use
00:48:16.240
the github desktop app. Nice app. Not a tuy. This is much nicer. This is much
00:48:22.480
quicker. I cannot believe that this project existed for years. and I didn't use it. We set up common aliases like
00:48:29.040
that R you saw. We can set up common aliases for all the Git operations. We
00:48:34.400
can even preconfigure your username and your email addresses that your commits will actually work out of the box right
00:48:39.920
away. We pick those out of the signup form. Um we can do everything. We can
00:48:45.440
set up those aliases. As I said, we can have a nice Starship prompt out of the box that will tell you if you have any
00:48:52.240
new files in the repo that are dirty and you need to check in. We can just do everything. We can do everything cuz
00:48:58.480
it's ours. And we don't have to [ __ ] beg Apple for a crumb. No, no, no. We can have a loaf of bread the size of
00:49:04.960
your head. And then we can realize we don't have to stop there.
00:49:11.040
After creating Umachi, I realized actually why isn't the entire setup of a
00:49:16.240
new machine at 37 Signals exactly like this. It should be. So, we've created a tailoring script that installs
00:49:22.960
everything on the setup in just a few minutes. Checks everything out. The developer is ready to use. As I said on
00:49:30.079
that budget, we want to deploy in 3 minutes. That sounds crazy to a lot of people who might be spending hours if
00:49:35.520
not days. But here's a deploy of Fizzy, the new project we're working on. You can see it up in the corner. 58 seconds.
00:49:42.880
We can make this so much faster. Running local CI deploying everything. Although I will say this is base camp 4. Few more
00:49:48.319
servers. It takes about four minutes. I know, sad little dog. I'm a little ashamed of it, too. We will shave off
00:49:54.079
that minute and get down to three minute. I guarantee it. Um, and we will get back to this. We will
00:50:00.880
get back to a way of developing that feels way more like 99, that feels like it's free of needless complexity, and we
00:50:07.520
will move much faster. And then when we deploy things,
00:50:12.640
I just little aside here, I've gotten really into these little mini PCs and it's really amazing just how much
00:50:18.000
performance you can crack into these boxes. These stacks here have 76 cores
00:50:23.359
and I think about 300 GB of RAM. It's pretty cool. I took one of those boxes
00:50:29.040
and do you know what um this uh campfire you guys have been using uh all week?
00:50:34.240
Yeah. head runs in my closet on literally that [ __ ] machine.
00:50:39.359
That's not stock photo. That's me in my dirty utility closet taking a picture of the machine that runs Campfire for
00:50:45.680
everyone here and barely breaks a sweat. Oh, and it's 300 bucks. And do you know how much that would cost if I ran it in
00:50:51.680
cloud? Oh, here's a droplet. Oh [ __ ] that amount of performance would be 300 bucks a month. And I mean, I know really
00:50:58.160
this is bad and this is just inest. I know they're out there. Heroku, this would cost like upwards of 1,200 bucks a
00:51:04.240
month to run this amount of performance that $300 runs in my closet. Do you know what? You can you can you can just you
00:51:10.640
can just do things. And this gave me an idea. And the idea
00:51:15.760
was if this amount of performance is available to mere humans for hundreds of
00:51:21.440
dollars, what could we do at an enterprise level? We've been running two main data centers for 37 Signals for
00:51:27.839
many years and we just created an outpost in Amsterdam. Actually in Amsterdam right here we have some Hay
00:51:34.480
servers standing right here. So if you're using hay in Amsterdam it's going to be pretty [ __ ] fast. Um but what
00:51:40.319
if we went further? What if instead of having these mega data centers, we had micro data centers? What if it looked
00:51:45.440
more like my utility closet than it did like this big honking thing? So as I
00:51:51.119
said a few times, we're working on a new product called Fizzy. And whenever I work on a new product, I want to push
00:51:56.240
the boundaries. Sometimes that aligns very well with something that's good for the product. Other times that's just hell of fun. And one of the boundaries I
00:52:03.119
wanted to push was this idea that the internet the internet just like packages
00:52:09.520
fly around the world and it's super fast because you know what packages over fiber they fly at the speed of light.
00:52:17.599
little light bouncing inside of these glass cables and it's about 300 million
00:52:22.880
um meters per second. Well,
00:52:29.920
what was this math? I made this math like four days ago. The point of the math was Oh, the circumference of the
00:52:37.040
earth. The circumference of the earth
00:52:42.720
is 40 million m. If you divide those two things, you have the theoretical time it takes for light to travel one time
00:52:49.440
around the globe. It's 137 milliseconds. And that should kind of be the
00:52:54.480
theoretical boundaries for packages as they ship around. Well, um,
00:53:00.160
it takes a little longer because not all the cables go straight around the Earth and not all the light goes in a straight
00:53:05.839
line. It actually bounces inside of these cables. So, when you actually measure it computer to computer, this is
00:53:12.079
about how long it takes. New York to LA, RTT is 70 milliseconds. If you get all
00:53:17.520
the way from New York to Sydney, it's 220 to send one package and get a
00:53:22.960
return. That's a lot of time. You guys have probably been looking at your performance charts for how long your X
00:53:28.960
run times are. 220 milliseconds for a lot of people, that's their budget. And you just blew it on a single [ __ ]
00:53:34.319
package. And you don't get to send one package. When you're making a post request and it's the first one, first
00:53:40.559
you got to do a TS TLS handshake. Oh, well that's a package and that's even highly optimized. It used to be like two
00:53:46.079
or three packages. Now we got it down to one package but 220 milliseconds. You
00:53:51.119
post something to that, you get a response back with a location of where the browser should go to next. Oh, 220
00:53:56.960
millisecond. You do a new get request to get to that location. Up 220 milliseconds. New York to Sydney on one
00:54:03.359
post have spent almost 700 milliseconds just on the lack of light.
00:54:11.760
Um, if you do that experiment in in other locations, if you do New York to LA,
00:54:17.760
it's still pretty damn meaningful because these are long distances. America is actually a pretty big
00:54:23.440
country. There's like 210 milliseconds here of just light lag going back and
00:54:28.640
forth between New York and LA. But I made some other tests, too. What if you move things a little closer? I'm in
00:54:34.319
Copenhagen at the moment. When I ping my local Copenhagen website that is literally
00:54:41.599
down the street, it's two milliseconds. That's pretty quick. Even when I travel
00:54:46.800
the entire country, it's 8 milliseconds. When I go to Falconstein, where Hner has
00:54:52.000
a data center in Germany, it's 17 millonds. And when I go all the way here, it's 25 milliseconds. That's
00:54:58.160
actually not that bad. And when we add that up, the lag of light on all of
00:55:03.359
these connections is way less. 660 milliseconds from New York to Sydney,
00:55:08.480
six milliseconds. If I could just stay inside of Copenhagen, that's a huge
00:55:13.520
speed upgrade. Now, if you make an entire actual request here, you say your
00:55:19.119
X run time on the server for a post is 150 milliseconds and your get is 75 millonds, the entire operation of making
00:55:25.760
that new post New York to Sydney is almost a second. coping to Copenhagen 200 millconds only
00:55:32.160
one that do you know what else is about 200 millconds a Formula 1 driver's reaction time when all the lights go out
00:55:38.800
that's what we should target that should be the goal that we do not exceed the um
00:55:45.280
response time of a Formula 1 driver. How do we do that? We chased the white rabbit that the
00:55:52.640
cloud industry has been chasing for a decade plus. Edge computing. And I saw a
00:55:59.040
funny tweet this morning from Peter Level saying, "EDge computing is all [ __ ] and no one has made it work yet." And you know what? I'm inclined to
00:56:05.520
agree, but it also sounds like a fun challenge. If no one else can figure it out, let's figure it out. And the way we
00:56:11.440
are going to try to figure it out with Fizzy is with this pattern of many small outposts. Essentially, the idea of the
00:56:18.240
server in the closet being so cheap, couldn't we just have a lot of them? Yes, we could. But the first thing we
00:56:23.520
need to do is we need to ensure that every data center we stand up does not need the entire beefy database that we
00:56:29.119
need for every single customer and running that for base camper. Hey, that's what makes it expensive. It's actually the data. It's not the compute
00:56:35.920
because once you start subdividing your users, you will have less need for compute in that area. Um, but you still
00:56:42.079
need the whole database. No, what we need instead is for every database to store only a store everything all the
00:56:49.520
data but only serve a fraction of the queries. If we could do that, if we can accomplish that, we could have mini data
00:56:56.880
centers everywhere. We could have these little drops everywhere. If we didn't need all that complicated infrastructure
00:57:02.319
for Fizzy, we could maybe have it everywhere here. And if we put Fizzy in all of these cities I have on this list,
00:57:08.640
almost all customers would be within 25 milliseconds of an outpost. and their response times would feel a lot faster.
00:57:15.440
So to make that happen, we have moved heaven and earth. The first piece of earth we've moved is
00:57:22.960
in a project called active record teniting because to make this whole setup work, we need to blow up the
00:57:29.760
single database for everyone and turn it into one database per customer so that we can put the database next to the
00:57:36.640
customer or as close as we can to it. that has required an enormous amount of digging inside of the internals of
00:57:44.079
active record and a million other frameworks. Mike has been working on this. We're going to push it out.
00:57:49.119
There's been a few uh updates already resolved, but the fundamental idea is creating tenants and tenants is one
00:57:55.680
database to one customer and being able to flow all that through it. Not only does that have benefits for performance,
00:58:02.319
it also have benefits for isolation, for backup, for privacy, for a million other
00:58:07.680
reasons. And now you can place these databases around the world. But when you do and you place all these databases around the world, you need to sync them.
00:58:13.920
That's the hard part. And to do that, we have spawned another project um that
00:58:19.119
Kevin has been working on and we'll be talking about later today, which is Beamer. And Beamer is a simple way of
00:58:24.319
replicating SQLite databases, which is what we're using for this project. You could use MySQL 2, but I think SQLite is
00:58:31.200
uniquely suited for the idea that every customer should have their own database.
00:58:36.319
This setup keeps it in replication. If that sounds vaguely like light FS, it's
00:58:42.079
because it sort of is. But it's built for an entirely different scale. It's built for tens of thousands of
00:58:48.400
individual databases per customers. It's built for tens of thousands of transactions per second because we're
00:58:55.359
not just replicating one, two, or a 100 databases. We're replicating one for every customer. And then finally, we
00:59:02.400
need a way to route all of these things around the internet. is that people land on an outpost that's near them. We've
00:59:09.599
extended Kamal proxy adding in this geo component so that we can get a setup
00:59:14.720
that looks like this that you might have databases in the US and you might have them in Europe and if you're using uh if
00:59:21.119
you're hitting a customer who's in the US on a on a get you can either get routed based on geo routing to the app
00:59:27.920
server in Copenhagen and the one in US East. That's all the readers. The readers are safe to access from
00:59:33.359
anywhere, but we still have a dominant writer. That's the hard problem. We're not solving all of the distributed
00:59:39.760
writers yet. Um, but when you do that, you're redirected to the right uh
00:59:44.799
writer. To make all that happen, we need to go
00:59:50.400
back to that magical year of 1999 where everyone knew everything and designed
00:59:55.440
everything with perfect foresight for all of the internet. This is from RFC
01:00:01.520
2616 that defines the HTTP11 protocol. And what does it tell us? It
01:00:07.440
tells us that get and methods should not have the significance of taking action other than retrieval. It's a really long
01:00:14.079
way of saying don't [ __ ] write when you're doing a get that blows the whole thing up. So we are going to restrict
01:00:20.720
all writes only to happen on posts and the other methods we piggyback off post.
01:00:26.480
And when you do that, you get the wisdom from 1999 um forward and you can make all this
01:00:32.720
thing work. Finally, we use session affinity of course because if you do a
01:00:38.319
write to a writer, you can get a stale read. If you're send to a reader that doesn't yet have the updates, the
01:00:44.960
standard way of doing this is just kind of putting a finger in the air and saying like, "Ah, I think in like a
01:00:50.480
second it's probably okay." That's a really poor way of doing it because it means when you write that comment,
01:00:56.240
you're very often redirected to the read for that comment back on the writer
01:01:01.359
which may be far from you. You want to get redirected onto the reader as soon as the reader is ready. If you run a
01:01:08.160
really fast replication and you have this technique of doing it, you can do that. So instead of just taking a static
01:01:14.000
number one second, we will boil it down to a transaction ID and if that
01:01:19.920
transaction ID what we have written into the database is available on the reader, we're good to go. And if we're not good
01:01:26.079
to go and the reader will know this, we will return a header that will redirect you somewhere else. All right, that's
01:01:33.839
our gift exchange. It's full of more. It's action job continuations in rails
01:01:41.680
8.1. It's action native push out in beta right now. It's turbo offline dropping
01:01:47.359
very soon. It's hotwire native13. Go see the keynote at the end of the day. It
01:01:53.760
action text lexi which is shipping today in beta. It's active record tening that
01:01:59.440
we still need to wrap up. So too with beamer and kamal geo proxy. It's less.
01:02:06.480
It's the idea that you can just run native Ruby and Docker DBs. It's the fact that you can use local host for all
01:02:12.480
your development. Is the realization that we know need system tests, that we can run our testing locally, that we can
01:02:19.200
do scripted setups, that we can have a new computer ready in 15 minutes, that we can deploy from local.
01:02:26.160
That's a lot of stuff. Some of the good stuff is shipping in Rails 8.1 right now
01:02:32.559
after this keynote. That's Raphael. He's the release manager. He's going to push this out.
01:02:39.040
I invite you to check out Umachi, check out Linux, update your quite possibly uh
01:02:45.680
ancient ideas of what Linux is, works, or what it could look like. I invite all
01:02:51.359
of us to do more end-to-end problem solving to do more endtoend open source
01:02:58.319
not to accept that there are parts of our stacks whether it's the computer or the server or the framework where we
01:03:04.799
can't just change [ __ ] if we want to change it we should be able to change everything we should own everything
01:03:13.200
because we want end to- end freedom we don't want people telling us what to do I don't want people telling us what to
01:03:18.640
do I certainly don't want Apple telling me what to do or what I can do or where I can't be or what I can't distribute.
01:03:25.760
[ __ ] Apple.
01:03:32.799
I want libertas. I want proprietas. I want Pas. I want freedom, ownership, and
01:03:39.520
duty. And what you do with those virtues? Well, that's up to you, isn't it? Thank you so much.
