00:00:16.720
Hey everyone, how's it going? Are we
00:00:19.279
having fun creating connections?
00:00:22.160
Awesome. Awesome. Because that's why we
00:00:24.640
do these things. Um, so this is my first
00:00:27.680
RailsCom and my last Railscom, so it's a
00:00:30.480
little bit bittersweet, but we're in
00:00:32.719
Philadelphia. It's one of my favorite US
00:00:34.719
cities. U, fun fact, it Philadelphia has
00:00:38.399
the oldest houses in the United States.
00:00:41.040
So definitely go and check them out. I
00:00:43.600
want to talk to you uh today about um my
00:00:47.200
learnings uh about the asset pipeline
00:00:49.440
and how to use them effectively in um
00:00:52.320
apps and gems.
00:00:54.399
The first disclaimer is that I've been
00:00:56.480
wrangling assets in Rails for the past
00:00:59.120
nine years and I still have to reference
00:01:01.039
my past articles, tweets, and code. So,
00:01:03.600
a lot of love went into this. So, if you
00:01:05.920
think this is uh this is cool and you
00:01:07.840
like it, please, you know, clap and say,
00:01:09.600
"Wow, I I love that." Uh, the second
00:01:12.640
disclaimer is that I might I might get
00:01:14.479
into some unorthodox
00:01:16.560
approaches. I don't want you to use
00:01:18.400
them, but it's good to know them. And
00:01:21.200
the third disclaimer is that some of the
00:01:24.000
these things are mind-numbing. So when I
00:01:26.240
see you falling asleep, I'll throw one
00:01:28.159
of these on the screen so I know you'll
00:01:31.600
pay attention.
00:01:33.600
Um, so is everyone ready?
00:01:36.880
Perfect. So we'll talk about all of
00:01:39.119
these today, but one of my missions is
00:01:41.360
to keep an appropriate level of wellage.
00:01:44.560
I don't want to overwhelm you, but not
00:01:46.799
underwhelm you either. I want to keep it
00:01:48.320
somewhere in the middle.
00:01:50.960
So maybe you're wondering why I'm
00:01:52.640
talking about these. I've been coding
00:01:54.560
since 2010. So I've been hitting uh most
00:01:57.920
of these problems. Um I came to Rails in
00:02:01.280
2016 and I had all kinds of questions
00:02:03.119
like what is an asset pipeline? What is
00:02:05.119
sprockets? Why do we need it? Can I not
00:02:07.040
use it? Can I just do my own thing? Uh
00:02:09.599
then I built AO which is a distributed
00:02:12.080
rails engine. So I had to distribute it
00:02:14.160
to others. Uh so I learned a lot uh
00:02:16.720
about them about the assets. Then I
00:02:19.120
built some more open source work like
00:02:20.400
Marksmith a markdown editor for Rails.
00:02:23.520
So this is me Adrian and I also co-host
00:02:26.160
a boutique Ruby conference in September
00:02:28.640
in Bucharest. So I would love to host
00:02:31.040
you. What are assets? Generally speaking
00:02:34.000
images, CSS, JavaScript, favicons, JSON,
00:02:37.519
XML and some other types of documents.
00:02:39.920
Uh how did we used to do it before? So
00:02:42.319
we would used to uh so maybe some of you
00:02:44.800
would used to do this the same things.
00:02:46.239
We would have like an HTML file or maybe
00:02:47.840
an PHP entry point and then we would
00:02:49.680
load up uh jQuery and then the jQuery
00:02:52.319
plugins and then our own plugins and
00:02:54.080
then vars.js was a good thing apparently
00:02:57.040
and then main.js and then some other
00:02:59.200
things that we didn't know where to put
00:03:01.200
and these are actually some some
00:03:03.040
snippets from my past code so don't
00:03:04.959
judge please. Um and that approach you
00:03:09.599
know kind of ran ran the web. This is
00:03:11.680
how things were back in the day. And
00:03:13.920
from what I hear from my WordPress
00:03:15.360
friends, it kind of still does in some
00:03:17.040
places. Uh but that that approach led to
00:03:19.680
a few problems. Um a few problems that
00:03:21.840
an asset pipeline uh could really help.
00:03:24.080
And the first one is bundling, which is
00:03:26.239
a fancy word for concatenation.
00:03:28.959
So we figured fast we pretty fast
00:03:31.760
figured out that HTTP1 had a problem
00:03:33.840
with multiple connections. So whenever
00:03:36.239
we had multiple assets in one of our
00:03:38.400
pages, it would have to do like multiple
00:03:40.480
back and forth and HTTP one just wasn't
00:03:42.400
good enough for that. Uh so then we
00:03:44.480
thought okay let's just move instead of
00:03:46.640
having 10 files let's just move
00:03:48.400
everything inside one file and we had
00:03:50.799
tooling for that. it was called gulp or
00:03:52.879
grunt or broccoli. And we would have uh
00:03:56.080
these kinds of pipelines where we it
00:03:58.319
would take all the JavaScript and it
00:03:59.840
would go pass them through a minifier
00:04:01.920
and an ugly fire and then rename them
00:04:03.760
and move them into a special place and
00:04:05.680
we had these tasks for local development
00:04:07.680
as well and for images and um you know
00:04:10.239
it was a mess. Um then the second
00:04:12.560
problem that an asset pipeline solves is
00:04:14.879
fingerprinting
00:04:16.479
uh which uh is so we figured out that
00:04:20.079
once we had that one JavaScript file it
00:04:22.720
was not very optimal to load it on every
00:04:25.199
page load because it's the same file. So
00:04:26.880
we want to cache that but um because uh
00:04:30.560
we were caching it we need to do a cache
00:04:32.320
busting. So whenever we change it we
00:04:33.759
redeploy we need something else. So what
00:04:35.680
our asset pipeline does whenever you run
00:04:38.080
the asset pre-ompile step and when you
00:04:40.240
deploy to production it will go it will
00:04:42.639
uh scan all of your files and create
00:04:44.400
this unique hash for that unique uh
00:04:46.960
contents and then we can cach that since
00:04:49.919
for forever and because it will have
00:04:51.600
that special name attached to it and we
00:04:54.400
have that special cache busting uh
00:04:56.320
mechanism set up. And the third problem
00:04:59.120
is transpiling which is fancy talk of
00:05:01.840
get all of this content through a map uh
00:05:04.080
function and do something to it. And we
00:05:06.639
used to uh transpile SAS as CSS to CSS
00:05:11.440
apply vendor prefixes if you know if you
00:05:14.080
remember those and then um try to and
00:05:16.800
then transform coffees script to
00:05:18.240
JavaScript and minify script and
00:05:19.919
optimize images and some other things.
00:05:22.160
And that was sprockets. That is
00:05:24.240
sprockets. This is what it does and it
00:05:25.840
did it very very well. And at some point
00:05:28.560
uh the web evolved and sprockets had to
00:05:31.520
do some some more things. Uh it would
00:05:34.000
scan the bower file and it would do JS
00:05:37.280
min c and zopfle and whatever that is.
00:05:39.759
So it's starting to do a lot uh of
00:05:41.680
things. Where are we now? So we have
00:05:44.400
http2 which solved that problem of
00:05:46.639
having multiple connections. The browser
00:05:48.960
doesn't care anymore. Uh if you just
00:05:50.639
have you can just have 20 or 30 files
00:05:52.400
and it'll download it without a problem.
00:05:54.400
CSS became smarter. So we don't need SAS
00:05:56.560
to CSS JavaScript became smarter smarter
00:05:59.360
so we don't need uh coffecript uh and uh
00:06:03.039
in 2021 2022 the HH released uh prop
00:06:07.600
shaft uh which is a drop-in replacement
00:06:10.400
for sprockets and it only does four
00:06:13.120
things. It does fingerprinting for cache
00:06:15.360
busting. It uh gives us a dev server for
00:06:17.680
whenever we do things on our local
00:06:19.759
machine. uh just a little bit of
00:06:21.520
transpiling if you need it and adds a
00:06:23.840
configurable load path which is fancy
00:06:25.600
work of telling prop shaft watch this
00:06:27.919
folder for me and and use those uh
00:06:30.800
assets as well. Uh and then it got a
00:06:33.440
little bit better with import maps JS
00:06:35.840
bundling and CSS bundling. Import maps
00:06:38.400
is a rather new technology which
00:06:40.479
basically tells the browser gives it
00:06:43.360
gives it a list of files and tells it to
00:06:45.600
fetch them and it helps you still keep
00:06:48.960
that import functionality inside your
00:06:51.520
JavaScript files and browser the browser
00:06:53.280
knows where to uh pick those up in
00:06:55.919
import maps works perfectly with prop
00:06:58.240
shaft. Um import maps tells the browser
00:07:00.960
which files to fetch and prop serves the
00:07:04.160
files to the browser. uh Ray Danski has
00:07:07.360
a nice article about it so you can uh
00:07:09.520
definitely check that out. CSS uh
00:07:12.960
bundling and JS bundling um these are
00:07:15.280
special gems which leverage existing uh
00:07:17.680
libraries to actually bundle your CSS
00:07:20.319
for you. So for example for CSS it will
00:07:22.000
install Tailwind, Bulma or Bootstrap and
00:07:24.240
for JavaScript it will install rollup,
00:07:26.479
bun, Webpack or um another one or ES
00:07:31.120
build and it basically they are
00:07:33.599
essentially filling the gaps of what
00:07:35.680
sprockets used to do and prop because
00:07:38.720
they are special tools made for uh those
00:07:41.280
um um those pieces of code. Uh another
00:07:45.440
two tool two two bigger tools that you
00:07:47.840
can use is shaka packer which is a fork
00:07:50.160
of webpacker it's maintained by a third
00:07:52.080
party and v rails which is amazing it'll
00:07:54.560
do basically everything for you it will
00:07:56.479
bundle the code transpile it will build
00:07:58.400
it for production it will give you a
00:07:59.919
great dev server so definitely try that
00:08:02.240
as well so we have an overview two asset
00:08:05.199
pipelines sprockets and propshot we have
00:08:07.120
to use propshhat now bundling tools JS
00:08:10.080
bundling CSS bundling vitrails and the
00:08:12.800
import mapsish
00:08:14.960
If you want to serve JavaScript, you can
00:08:16.479
do it using propshaft import maps or vit
00:08:18.639
rails. If you want to serve CSS, you can
00:08:20.240
do it using propsh and vit rails. If you
00:08:22.720
want to go fast, you use propshaft and
00:08:24.720
import maps. If you want a little bit of
00:08:26.479
power, you use propsh bundling and CSS
00:08:29.039
bundling. And if you want a little bit
00:08:30.639
more power and a little bit of elegance,
00:08:32.399
you use propshaft and um vit rails.
00:08:36.719
That's it. Thanks. Thanks for all Was it
00:08:39.039
that easy?
00:08:41.360
No, it wasn't easy.
00:08:43.839
It was quite complex actually
00:08:47.200
but I'm glad some people thought it was
00:08:49.120
easy. Um so it was a little bit complex
00:08:52.640
but even though uh we have that
00:08:55.200
complexity the fact that we have those
00:08:58.000
tools uh we have the guardrails this is
00:09:00.480
this is amazing for us. So as long as
00:09:02.240
you follow the the guardrails you're
00:09:04.399
going to be good. So let's get a little
00:09:05.839
bit into the talk and talk about how we
00:09:08.399
can leverage the asset pipeline to build
00:09:10.640
Rails plug plugins effectively. And the
00:09:13.200
question is is it easy or not? And the
00:09:16.399
answer is kind of so so
00:09:19.920
what are we building? Uh my buddy Jeremy
00:09:22.800
um recently uh built Liinal which is a
00:09:27.040
new t new take on forums and he has an
00:09:30.320
amazing markdown editor and I always
00:09:32.560
wanted that to use in my apps and I
00:09:34.399
called him up like do you have this as a
00:09:35.760
gem plugin he said no I asked can I
00:09:38.240
build it and said yes let's do that and
00:09:40.399
this is how Mark Smith um you know came
00:09:43.040
around u the fact that Jeremy's last
00:09:46.800
name is Smith and this is called Mark
00:09:48.240
Smith is that a coincidence I I guess
00:09:50.880
we'll never know.
00:09:52.959
So we we're going to build Consmith.
00:09:54.959
Actually, we're not going to build it,
00:09:56.080
but we're going to show how we can use
00:09:58.080
the asset pipeline to build the assets
00:10:00.720
and serve them to all kinds of Rails
00:10:03.200
apps. Uh at the end of the talk the
00:10:05.519
talk, you'll have links to the comsmith
00:10:08.000
editor. So all the code there and both
00:10:10.320
the import maps and prop shaft versions
00:10:12.160
of the Rails apps that uses it. So uh
00:10:14.480
you can follow along there. For the rest
00:10:16.720
of the talk, I want you to put yourself
00:10:19.200
in the shoes of an library author. So
00:10:23.279
you're not building an engine where
00:10:25.279
which you control and put inside an app
00:10:27.680
which you control as well. You only
00:10:29.279
control the engine. So the app can be
00:10:30.720
built in multiple ways. So that imposes
00:10:33.519
some kind of a little bit of restriction
00:10:35.519
for you as an uh library author.
00:10:44.959
asset work. So let's start from simple
00:10:47.760
to complex. Why complex?
00:10:50.800
Because we don't have just one story
00:10:52.800
around assets in Rails. And um if you
00:10:55.839
plan to release a library, you probably
00:10:57.360
have to cover the whole spectrum. So
00:10:59.200
these are some of the requirements of an
00:11:01.839
app. Some some users might have these,
00:11:03.760
some some don't. But you want to cover
00:11:06.000
everybody, right? So there are a couple
00:11:08.320
of decisions that you can take. So the
00:11:11.200
first lowest hanging fruit is approach
00:11:13.839
that you can do is use import maps. Uh
00:11:16.640
this will get you this will read you of
00:11:18.560
any build step that you uh might want to
00:11:21.680
do. So using these two snippets of code
00:11:24.959
will help you push a JavaScript file on
00:11:27.440
an import maps app. So what we're doing
00:11:30.240
here is creating our own import map J uh
00:11:32.880
RB file inside inside our engine. Then
00:11:36.399
we are telling the parent app to
00:11:38.240
actually use that import map rb file and
00:11:41.440
then we're telling uh prop shaft to
00:11:43.519
actually serve the comsmith js file from
00:11:47.839
our directory from our engine. Um
00:11:52.880
next, okay, we ticked a few boxes off of
00:11:55.360
that list, but what if we need an npm
00:11:58.480
package? We have two options. We can
00:12:00.399
bundle it into our gem or we can tell
00:12:02.640
the user to bundle it. Solution one,
00:12:05.519
let's bundle it. Uh we're going to
00:12:07.040
install J import map rails. Um
00:12:09.839
unfortunately the uh the template for
00:12:12.480
installing this does not work in an
00:12:14.160
engine. You have to do a little bit of
00:12:15.279
manual work, but it's all there in the
00:12:17.200
repo. Then we pin that dependency
00:12:20.320
stimulus confetti which I invite you to
00:12:22.399
use. We pin that dependency inside our
00:12:24.399
engine and then we tell PropSa to
00:12:26.880
actually uh watch for the
00:12:28.560
vendor/javascript
00:12:30.160
directory where the dependency is
00:12:32.320
actually pinned. Um, and then the import
00:12:36.000
map RB file is going to have that
00:12:37.839
dependency and it's going to show up uh
00:12:40.399
in the hash import map hash inside the
00:12:42.720
browser. Solution two, you just tell the
00:12:45.519
user to pin it in their app. So it's a
00:12:47.279
little bit simpler for you. You just
00:12:48.560
give it uh give the user this uh
00:12:50.800
instruction. So they'll they'll just run
00:12:52.880
it and it'll be there. The difference
00:12:54.639
between these two approaches is for
00:12:56.800
example when you do the second solution,
00:12:58.639
it's when you want to give control to
00:13:00.399
the user to the versioning of the asset.
00:13:03.200
So you might have a dependency that you
00:13:04.959
use in your engine and they use in their
00:13:06.480
app. So you want might want to give them
00:13:08.320
control to use whatever version they uh
00:13:10.560
see fit.
00:13:12.320
We ticked a few more boxes off of that
00:13:14.560
list. Uh but what if we need a little
00:13:17.360
bit more power? Maybe some transpiling,
00:13:19.279
maybe a little bit of typescript. Well,
00:13:21.519
I said we don't want to have we don't
00:13:23.680
want to make the user have a build step,
00:13:25.680
but we can have a build step inside our
00:13:28.160
engine. Um so let's do that. uh we're
00:13:31.279
going to add JS bundling. Uh then we're
00:13:34.240
by just running yarn build we have the
00:13:36.959
we'll we'll have that JavaScript file uh
00:13:39.600
put inside our app assets builds
00:13:41.680
directory inside our engine. So that
00:13:43.680
file is ready to go out uh in inside to
00:13:47.600
go out into the browser. Uh and we can
00:13:49.920
even remove these instructions for
00:13:51.680
propshot because propshot will follow
00:13:54.079
will um watch that directory.
00:13:57.920
we ticked a few more boxes off of that
00:13:59.760
list. Um, so import maps are great, but
00:14:02.480
not everybody's using uh that feature.
00:14:04.639
Uh, the next big thing is JS bundling.
00:14:07.440
Um, so the parent app might be might
00:14:10.639
maybe might have been generated with
00:14:12.079
something like this like that-
00:14:13.600
JavaScript/s
00:14:15.519
build. Uh, and you might want to import
00:14:17.839
that file like this import conment.
00:14:20.639
Unfortunately, that's not possible out
00:14:22.720
of the box. you're going to have to do a
00:14:24.720
few a few things, a little bit of work.
00:14:27.040
Um the reason is even though propshaft
00:14:30.160
um exposes that file to Rails, you have
00:14:32.480
to remember that ES build and and the
00:14:34.880
other um build tools, they are separate
00:14:37.680
processes and separate pipelines. So
00:14:39.199
they don't know about that file. So
00:14:41.519
there are a few solutions here. We can
00:14:43.519
create an npm package. Uh it seems uh a
00:14:46.480
big lift, but it's actually the one of
00:14:48.240
the most popular ways of doing uh this
00:14:50.240
thing. Uh it's not that difficult. You
00:14:52.480
run npm in it. You hit enter, enter,
00:14:54.160
enter. You're going to get this kind of
00:14:55.760
uh package.json. We already have the
00:14:58.240
bundled file so we can use it inside uh
00:15:01.199
module. And then you hit npm published.
00:15:03.839
Voila, it's up on npmjs.
00:15:06.959
Um then you tell the user use it like
00:15:09.040
this. Yarn add and use it inside your
00:15:11.279
app.
00:15:13.839
Now we've basically ticked all of the
00:15:15.920
solutions. So now we can use npm
00:15:18.560
packages inside our consid. Uh it's we
00:15:22.160
can use fingerprinting, we can use
00:15:24.079
caching. U it's it's a really good setup
00:15:26.800
and it will hold you forever. You can
00:15:28.720
cover the whole spectrum or at least
00:15:30.720
until we'll have another asset pipeline
00:15:33.120
introduced and then we'll figure things
00:15:34.720
out. Um I talked about some unorthodox
00:15:38.720
approaches. Again, I don't urge you to
00:15:40.959
use them. uh and one approach is to
00:15:43.839
actually write a rate task to copy that
00:15:46.560
JavaScript file from the engine inside
00:15:48.560
the main app and they can run it and
00:15:50.560
they can organize it however they want
00:15:52.160
it. That's not perfect because that
00:15:54.959
doesn't stay in sync whenever you update
00:15:56.639
the package. The next best thing is to
00:15:59.279
write a great task to um do a sim link
00:16:02.399
between the engine and your app. Again,
00:16:05.279
not the best thing ever. you can inject
00:16:07.759
your the file yourself if you're not
00:16:09.759
doing stimulus controllers and I'm going
00:16:11.759
to talk about that. Um so what you can
00:16:13.920
do is use a JavaScript include tag and
00:16:16.399
just include the config confids file
00:16:20.320
because we in uh we instructed uh
00:16:22.959
propshot to watch that directory. The
00:16:26.000
bad thing is if you want to if you want
00:16:28.720
to uh use a stimulus controller you
00:16:30.720
won't be able it won't be able to hook
00:16:32.639
into your application right away. These
00:16:36.000
are a few lines of code which show you
00:16:38.320
how we do it at AVO and I'm going to
00:16:40.399
show you uh in a snippet. It takes a
00:16:42.560
little bit of work. So what we actually
00:16:44.320
do is we have a global config uh
00:16:47.600
JavaScript object. We have stimulus
00:16:49.199
controllers an array and then through a
00:16:51.199
plug-in system we inject the JavaScript
00:16:53.600
from our plug-in inside the application
00:16:55.600
HTML. Then again through the plug-in
00:16:57.920
system we can push on that stimulus
00:16:59.839
controllers the name of the stimulus
00:17:01.279
controller. We in our bundle file we we
00:17:04.240
take that array and just register all of
00:17:06.880
those controllers from the plug-in and
00:17:08.720
then you can um you do your own thing
00:17:10.959
you know when application js and you
00:17:12.240
register your you know the parent app u
00:17:14.640
controllers and you have everything
00:17:16.400
running and this is um actually this is
00:17:19.439
amazing because it's a we don't whenever
00:17:23.039
we tell people to install a plug-in we
00:17:25.120
don't have to tell them oh you have to
00:17:26.319
do yarn install and don't forget to do
00:17:27.760
yarn update and all of that stuff you
00:17:29.280
just do bundle uh add that gem
00:17:32.000
and you're off the hook and that's a big
00:17:33.600
flex in the author you know space
00:17:36.480
library author space.
00:17:38.799
Um another unorthodox approach which we
00:17:41.360
use uh when we started AO propshift
00:17:43.679
wasn't around there wasn't too much
00:17:45.120
documentation around sprockets uh
00:17:47.120
webpacker was around the corner and uh
00:17:50.640
there was a lot of you know talk about
00:17:52.160
how to do that inside an engine so what
00:17:55.039
we wanted to do is give the user an an
00:17:58.240
engine with assets specifically like
00:18:01.039
JavaScript and CSS and very advanced
00:18:02.480
ones and not impact their deployment
00:18:05.360
pipeline. So if they don't have
00:18:06.960
Webpacker that's fine. We have it but we
00:18:09.360
don't you know if they have it that's
00:18:11.200
fine. So whenever they run assets
00:18:12.880
pre-ompile uh our assets they don't even
00:18:15.760
go in there. So what we did instead of
00:18:18.480
hacking into the instead of uh you know
00:18:21.280
yeah hacking into the asset pipeline we
00:18:24.080
are using a middleware. So we compile
00:18:25.919
over all of our scripts into a special
00:18:27.600
directory called public/avo assets. And
00:18:30.320
then we instruct using the engine we
00:18:32.480
create a middleware to expose that
00:18:34.720
directory to the public. Um and then
00:18:38.240
whenever you use this these um these
00:18:41.120
paths uh whenever we use these assets we
00:18:43.440
use it use them from those paths and
00:18:46.160
again we don't impact their build
00:18:47.919
pipeline. And now when import maps came
00:18:49.760
around if they don't want to have a
00:18:51.520
build build uh uh asset pipeline a build
00:18:55.679
um step inside their asset pipeline
00:18:57.679
that's just fine. Um, what about CSS? I
00:19:01.360
didn't talk a lot about it because it's
00:19:02.720
very easy using propshaft. You just add
00:19:04.880
that stylesheet link tag and it will
00:19:07.360
prop
00:19:08.960
uh pick it up. Uh, you can even use
00:19:11.280
imports inside your CSS files. You just
00:19:14.000
have to use this URL uh function inside
00:19:16.799
your CSS to be able to get the uh
00:19:19.039
fingerprinted um assets.
00:19:23.600
That was my talk for today and I'm going
00:19:25.440
I want to see you tomorrow at uh our
00:19:27.039
hack spaces. We have four repos uh with
00:19:30.720
um different issues we can talk about
00:19:32.960
for JavaScript and Ruby and uh from
00:19:35.200
beginner to intermediate. Um and we have
00:19:38.400
a very special friendly RB collectible.
00:19:40.880
It's that frame which my sister and my
00:19:43.440
dad handmade and we don't give these we
00:19:46.080
give them only to speakers. So if you
00:19:47.520
want something rather unique, you should
00:19:49.919
go to the silent action and um uh check
00:19:52.640
that out. Thank you everyone.