00:00:15.200
hi oh too loud so um i realize i'm the last person between you guys and the bar so i'm
00:00:21.359
really impressed that you're still here um there's been some amazing talks today i've seen a lot of them some amazing
00:00:27.599
talks going on right now so thanks a lot for coming i really appreciate it uh and thanks to ruby central i don't know if
00:00:33.280
any of you guys are in the room um for giving me the chance to talk at rubyconf it's like a once in a lifetime opportunity i'm really excited
00:00:40.160
uh it's it's thrilling for me to be here so thanks a lot my name is pat shaughnessy and i'm going to be talking
00:00:45.520
about blocks so that's right i'm going to talk for 45 minutes about ruby blocks
00:00:51.360
in fact together we're going to be dissecting a ruby block so don't worry there'll be no blood no gore
00:00:57.920
no lab animals will be harmed in this experiment what what i'm going to do is look at how
00:01:03.280
ruby implements blocks internally and see what we can learn about blocks by doing that and i'll be talking about
00:01:10.080
max's ruby implementation today not about j ruby rubinius other versions of ruby
00:01:15.680
before i get into blocks let me just start by saying a little bit about myself you probably haven't heard of me before i work at a company called
00:01:22.080
mckinsey which is a management consulting firm but i'm not a consultant i just i'm just a ruby developer i write
00:01:27.520
code for them um in my spare time i read a blog about ruby development at pat shaughnessy.net
00:01:34.240
and i'm really excited i actually just finished my first book i wrote a book i've been working for the last hey thank
00:01:40.320
you so i just released a book uh this week on monday called ruby under a microscope
00:01:45.520
and it's all about ruby internals so i've taken six months off my my job and i went into my just
00:01:53.040
sat at my house and i wrote about how ruby works and the way i the way i like to think of
00:01:58.399
it is i went on a long road trip it was kind of like visiting japan or a foreign country where you don't really know the
00:02:04.000
language and you're seeing all these amazing exotic sites and you're um you know but
00:02:09.280
you begin to learn a little bit of local language you get to know people there and you can kind of appreciate their customs and their food and that's what
00:02:15.360
it was like for me so the book is sort of a travel journal about ruby mri internals i also touch on j
00:02:22.239
ruby and robinius in the book so if you're into internal stuff and actually the presentation today this
00:02:28.640
is really a synopsis of one of the chapters from the book please check it out
00:02:34.560
so the first question you probably have for me is okay pat why ruby internals who cares about how ruby works all i
00:02:41.040
care about is that ruby does what it's supposed to do you know if i it takes a value if i insert something into a hash
00:02:46.879
or if i save it into array all i need to know is that i'm going to get it back when i give it the same key um i i don't
00:02:53.200
care how ruby works i'm busy i have a day job i have to get stuff done i have to um
00:02:58.239
you know maybe i have an open source project at night i work on in my spare time or on the weekends you know who has time to look into ruby internals and
00:03:04.640
figure it out you know i took six months off and i did it i'm kind of crazy maybe but um here are the reasons why i did it
00:03:10.640
and maybe why you should learn about ruby internals too so i can think of three reasons um the first one is i
00:03:16.080
think if you study ruby internals it can help you become a better ruby developer so i think you can learn
00:03:22.159
about the language more deeply than you may have you can learn it more deeply than you did before
00:03:28.239
you can learn about how ruby was intended to be used how matts and the rest of the core team intended the language to be used not
00:03:34.799
just use it the way that you happen to learn how to use it and i think you can learn a lot and just
00:03:40.000
become a better developer for it maybe a second reason could be there's a lot to learn from the core team you know
00:03:46.159
the koichi and the other team people on team are they're really smart people they've solved difficult problems
00:03:51.440
and maybe you can learn from their solutions their algorithm their code and learn about you know maybe pull some
00:03:57.040
ideas from them that you can try on your own projects so there's a lot of amazing solutions in there that we can all learn
00:04:02.239
from and it's a little inaccessible unless you take the time to learn it and i think the last reason the most
00:04:07.599
important one is just a lot of fun i had a blast i had so much fun learning about ruby internals and i hope you guys get a
00:04:13.680
little taste of that today if you if you like it then check out the book there's a lot of really amazing stuff going on
00:04:19.440
in there it's fascinating to me to learn how a language works internally you know i never knew this i'm not a computer
00:04:25.040
science major or anything and so for me to learn about how language worked internally it was just it was just super
00:04:30.800
fun so that's the most important reason have fun um so today i'm going to talk about blocks you know here's a block and it
00:04:38.160
seems kind of hard to believe that there's really that much interesting to say about blocks you know what can we say about them well we can call them you
00:04:43.919
can call a block you can pass in an argument to a block like this um and you know most of us especially
00:04:50.000
here rubyconf all you guys can tell me how this you know what this code will do in two seconds or less you can tell me
00:04:55.680
what this code is going to do you know what am i possibly going to say what can we say about blocks that's that
00:05:00.960
interesting so hold that thought i'll get to the blocks in a second the real reason however i want to talk about blocks is
00:05:08.560
when i think back to when i first came across ruby and i imagine a lot of you here had the same experience you know
00:05:14.160
you first saw ruby for me was in 2008 maybe and before that i was a php
00:05:19.199
uh java c plus plus developer i did a few other random things too you know so if i wanted to write code
00:05:25.199
that did this in c i would have to say you know int i semicolon uh four
00:05:30.800
parentheses i equals zero semicolon i less than ten semicolon i plus plus
00:05:35.840
parentheses brace blah blah blah it was a much more verbose way of doing the same thing and when i first came across
00:05:41.440
this it might have been 2008 i thought to myself wow this is something different i haven't seen anything like
00:05:47.280
this before you know it was it seemed really elegant and beautiful to me and it got me to
00:05:52.560
think twice about ruby i thought to myself ruby is something special and when i think back about that it it
00:05:58.960
really was the blocks the way that you can use blocks and ruby that got me to fall in love with the language so you
00:06:04.800
know aside from all the technical stuff which we're about to go into for me the real reason i wanted to talk to you guys today about blocks was
00:06:11.440
it's what i like the most about ruby uh and you know think back to your own history where you discovered the
00:06:16.479
language and what it was about ruby that made you fall in love with it enough to come to ruby conf
00:06:22.000
so for me it was blocks so what am i going to say about blocks well the first thing i want to do is define
00:06:27.440
what is a block you know if we ask that question what is a block what kind of answer do we get you know and so the way
00:06:33.440
i want to do that is imagine if i could slice open the block and look inside you know what's inside of a block are there
00:06:39.039
moving parts in there am i going to see blood guts chromosomes you know amoebas if i were to slice a piece of a block
00:06:44.880
add look it under look at it under a microscope you know what would i see and so the first half the presentation
00:06:51.039
i'm going to talk about what blocks are i'm going to answer that question what is a block
00:06:56.880
and we'll find out in a few minutes that blocks are ruby's implementation of closures and we'll define what a closure
00:07:03.440
is along the way uh and then at the second half i'm going to switch gears and talk about closures and meta programming and how those two
00:07:10.000
things are related so two different very different concepts but i think they're actually related to each other and so um
00:07:15.759
we'll talk about that second so let's get into it so if you were to look inside the ruby c source code and
00:07:22.000
you were to try to figure something out about what blocks are the first thing that you would see is that there's a c
00:07:27.360
memory structure called rb underscore block and i'm not showing the contents of it
00:07:32.720
here and the reason why is i want to sort of discover as we go along what's in this structure
00:07:38.319
and but the nice thing about studying ruby internals is if you have a question like this you know what is a block or
00:07:44.479
what is an object what is a module you know you can actually directly answer that all you need to do is go into the c
00:07:49.919
code find the definition of this or that or the other thing and that's your answer you know you
00:07:54.960
don't have to think about it very much you can just look at how it was implemented and so that's maybe another reason for studying ruby internals is
00:08:01.120
you can answer all these sorts of questions that you might have about how the language works or how it was intended to be used
00:08:07.759
so before we go any further i worry about my diagrams uh you know today and also in my ebook i have a lot of
00:08:12.800
pictures it's like an illustrated book um my pictures are not definitive exhaustive definitions of
00:08:19.680
exactly how ruby works i'm not going to show you every single thing inside the block structure
00:08:25.440
and there's the reason why is i want to point out to you the details about ruby internals that i think are important for
00:08:31.440
for you for ruby developers to know there's a lot of other very technical things and a lot of nitty gritty details
00:08:37.279
that maybe aren't as interesting and so i'm going to gloss over ignore some of those details and so my diagrams are
00:08:43.680
over simplified versions of reality so that'll make them hopefully easier for you to understand as well and made it
00:08:49.600
easier for me to draw them too so what we're going to do is uh we're going
00:08:54.800
to do a series of experiments and the reason i call the book ruby under a microscope is i wanted to use sort of a
00:09:00.320
scientific method for discovering and learning things about ruby you know it's one thing to just start talking about
00:09:05.839
how it works and what it does it's another thing to test your hypothesis with an experiment um so i'm not going
00:09:11.279
to do real experiments today i'm just going to do a you know a few simple thought experiments wouldn't have much time but in the book i go into more
00:09:17.200
detail about one thing or another and actually run some code the experiments are written in ruby code so you can download them and
00:09:23.440
run them yourself if you're interested so the first thing i'm going to do is really the most basic thing you can do
00:09:28.959
with a block is call a block so how do we call a block so not rocket science all of you know
00:09:34.240
how to call a block here's an example um here's a block that takes a string value
00:09:39.519
the quick brown fox the rest of that of course is jumps over the lazy dog that didn't fit on the
00:09:44.880
slide but we're going to come back to that string over and over again and you're going to get very bored and tired of hearing me say that
00:09:51.200
and we're going to save it in a string variable call a local variable called str and then we're going to print that
00:09:57.279
out and we'll do that 10 times so not very interesting code but what it does is it allows me to answer the question
00:10:03.360
what is a block and so the first the most obvious answer to that question is well it's right there on the screen it's
00:10:09.600
everything between the do keyword and the end keyword you know the first most obvious definition of a block is it's
00:10:16.000
just a code snippet it's something that um it's code that you type in it's sort of like a function
00:10:22.160
if we were to look inside of ruby you know through a microscope so to speak this is what you would see
00:10:28.240
so that rb block c structure i'm showing that on the right and now i'm showing you one of the
00:10:33.839
fields or one of the values inside that and there are many more but here's one of them it's called iseq and i think and
00:10:40.320
you know help me out here core team i think that stands for instruction sequence and that's a pointer and it points to
00:10:46.800
an instruction sequence that's what i'm showing on the left side um and an instruction sequence is a series
00:10:53.760
of bytecode or virtual machine instructions the first one is put string set dynamic et cetera et cetera don't
00:11:00.160
worry don't try to read that don't try to understand it i'm not going to explain any of that today i have a lot
00:11:05.360
more of that in my book but what i wanted to point out is this is really what a block is it's a snippet
00:11:12.560
of code and internally this is what it looks like um just taking a step back for a moment i will say just one uh you know maybe
00:11:18.560
one minute about how the virtual machine works and how these by code instructions work at a ten
00:11:23.760
thousand foot level when you look at ruby from you know from a distance the way ruby works at least starting with
00:11:28.959
ruby one nine is you give it some your code ruby parses that and compiles your ruby code into
00:11:34.800
these instructions and then executes them with a virtual machine and that's what koichi was talking about earlier
00:11:40.839
today so so anyway this is our first answer to the question what is a block it's a
00:11:47.760
it's a snippet of code or a function now let's do another experiment this time i want to look at how you can
00:11:54.240
reference variables from the parent scope and that sounds kind of complicated but it's really actually obvious so if you were to look at a
00:12:00.240
block like this so now i'm doing the same thing i'm putting the same string out ten times i'm just doing it in a
00:12:05.760
more complicated silly fashion and so what i've done here is i've taken the first half of the string the quick
00:12:12.000
round fox and i've saved that in the same str variable but i'm i'm doing that in the outer scope
00:12:18.320
um then i call the block 10 times and in the block i say str2 so i have a second variable and on that one i'm saving the
00:12:25.200
second half of the string jumps over the lazy dog and then i have a put a statement that
00:12:30.240
says put s you know string and string two so pretty simple code you can all understand this
00:12:36.320
and in fact you know what's so interesting about this it's all kind of obvious all you guys write this this sort of code probably every day this is
00:12:42.399
probably a reflex you probably just type this in without even realizing it um you know why am i pointing this out to you
00:12:48.560
well for me there's something actually kind of profound and interesting going on here it has to do with the question again what is a block so the way i look
00:12:55.839
at it is blocks have kind of this dual personality they're kind of schizophrenic you know they have on the
00:13:01.680
one hand they're a separate function that i'm calling on the other hand they're part of the surrounding function
00:13:07.760
and so that put a statement can access equally well it can access str
00:13:13.360
and it can access str2 it doesn't it doesn't matter and that's kind of obvious we all take that for
00:13:18.639
granted but it's not you know don't take it for granted a lot goes on inside of ruby to make that work so let's take a
00:13:24.399
look inside of ruby and see how that's implemented um so what i'm going to do is i'm going to step through this slowly and i'm
00:13:30.240
going to go through this in great detail and explain how this is implemented so the first thing we do is we take that string the fox string and we save it in
00:13:37.360
a variable str so what does ruby do when you save a value in a variable so if we were to
00:13:43.519
look inside of ruby again you can tell these are super oversimplified diagrams but what i'm
00:13:49.360
trying to get across here are the essential ideas so first of all on the left we have a stack
00:13:55.040
this is something called the virtual machine's internal stack so the way the virtual machine works and if
00:14:01.199
you heard the presentation earlier you know this it's a stack based machine as it runs those instructions that we saw
00:14:06.880
earlier set local get local whatever is it runs those internal instructions those instructions typically use
00:14:13.360
they take values off of this internal stack they do some kind of operation and they push the result back on the stack
00:14:20.240
so this internal stack is actually not something that you ever see as a ruby developer or really need to know much about
00:14:26.160
but you know just remember that the virtual machine inside of ruby is a stack-based machine
00:14:31.199
and so when we save a value in a local variable like str in this example that value gets saved on that stack
00:14:38.480
internally okay on the right is actually a different stack so on the right there's something called the rb control frame
00:14:44.720
structure and i'm showing one member from that structure and quiche asked earlier you know how many members are in
00:14:49.920
this and i think there were 11. so there's a lot more going on here i'm not showing you but what i wanted to point out is that there's one pointer the dfp
00:14:56.880
which i think stands for dynamic frame pointer that indicates that ruby uses to keep track of where the variables are
00:15:03.839
for the current um for the current scope so what i need to explain here is that as your ruby program runs and you call
00:15:10.480
from one method to another to a block to a lambda you have this ruby call stack that you build up and if you ever run
00:15:16.560
the command put s color that will print out that whole call stack and so each level in that call stack is represented
00:15:22.959
by this rb control frame structure and so what you have to imagine here is i don't really have time to explain all
00:15:28.639
this today is that there's a stack or an array of these control frames that represent your ruby call stack so if you
00:15:35.279
think about it inside of ruby there are two stacks there's the ruby stack on the right and there's the
00:15:40.480
internal virtual machine stack on the left and those two things are related and connected anyway let's get back to blocks so how
00:15:47.360
does this work so now we've saved a value in str and now we're going to call the block so what happens
00:15:53.600
well actually we're not calling the block are we what are we doing here what we're doing here is we're taking a number
00:15:59.279
10 and we're considering that to be an object that's the receiver of a message
00:16:04.320
so 10 is an instance of the fixed num or integer class and we're sending a message to it which is the times method
00:16:11.040
so what we're doing is we're calling the times method on that class fixed num
00:16:16.079
and that has nothing to do with the block in fact we're not calling the block at all what we're doing is we're taking that block and we're passing it
00:16:22.000
into the method as an argument so it's um there's more going on here than you might think so let's look at
00:16:28.000
what happens inside of ruby at the moment when you pass a block into a method call as an argument so again
00:16:34.320
we're not calling the calling the block yet we're simply referring to that block for the first time so when you do that
00:16:39.600
when you when you first refer to or use a block ruby says oh i have a new block
00:16:44.800
i need to create a new rb block structure and so what it does and again i'm
00:16:50.000
showing an oversimplified version of the same structure but it creates a second value in there and it copies it the dfp
00:16:57.199
it copies it from the control frame to the block okay so remember that dfp that was a
00:17:02.720
pointer over to the internal stack where the local variables for that method were
00:17:08.079
stored so what's going on here is interesting is that ruby is saving information inside the block about where you first
00:17:15.760
referred to that block from where you decided to pass it into a method so
00:17:21.280
blocks are not as simple as they seem they're not just a code snippet they have this other value inside of them which is a
00:17:27.439
pointer to the environment where you reference them from so interesting
00:17:32.880
let's hold on to that thought for men and see where where we go okay so now we're calling uh the 10 dot
00:17:38.480
times method so we're calling into the times method inside the fixednum class and so whenever you call a method in
00:17:45.200
ruby it creates another stack frame on that internal stack it pushes a bunch of values on there and there's a lot of
00:17:51.120
details i'm not showing but i just want to get across the fact that as you call deeper and deeper into your code it
00:17:56.559
creates more and more of these stack frames and now this fixednum times class this is implemented in c this is not
00:18:02.320
something that you write yourself but it works the same way that you probably would write it loops over you know the number zero one two up to n
00:18:09.200
minus one so in this case we'll go up to nine and finally now it's time to call a
00:18:14.320
block so that will that code inside of the fixednum class will yield to the block yield is just a fancy word to use
00:18:20.400
for calling the blocks function um and uh and it'll also pass in that integer argument if you want to if you
00:18:26.799
want it to okay so now i'm in the block i'm going to create that str2 variable where i save jumps over the lazy dog and
00:18:34.559
then i'll print out the two strings so let's see how that works okay so now i'm getting a little more
00:18:40.080
complicated in my internal stack my yar of internal stack now i have three stack frames in fact there's more than three
00:18:46.480
i'm glassing that over too there's a fourth stack frame in there but just to keep things simple today on the bottom
00:18:52.480
we have the parent scope where the str variable was in the middle we have another scope for
00:18:58.480
the times method and then on the top we have the top of the stack is now being used as we run
00:19:04.240
the code inside the block and so that's where the str2 variable goes
00:19:09.440
and so the put s call when it says put s str str2 the code in put us is able to
00:19:14.960
access both str2 in the current scope and then using that dfp point to go down
00:19:20.960
to the parent scope farther down on the stack and find that str variable and the reason it can do that is that when you
00:19:27.200
yield to a block in ruby when you call a block ruby takes that dfp pointer out of the
00:19:32.559
block and puts it on to the stack so that your all the code inside your
00:19:37.840
block is able to access the environment where you originally again where you first referred to that block so ruby's
00:19:44.640
in that way sort of merging those two things together the original surrounding environment and the
00:19:50.080
current scope of the block itself so that's uh you know back to that dual personality that's the way blocks are
00:19:56.320
able to behave in this way how they're smart enough to do both things um so just taking a step back and
00:20:02.720
reviewing we found uh so far we've discovered two two fields or values inside the block
00:20:08.080
structure we have the iseq pointer this is the instruction sequence that points over to the again a series of bytecode
00:20:16.080
instructions that the virtual machine is going to run so this is the compiled version of my function
00:20:21.120
and then it has that dynamic frame pointer or dfp that points down to the stack and again not to where the stack
00:20:27.679
is you know at the moment we run the code but to the referencing environment where i first referred to that stack to
00:20:34.159
that block so it turns out in computer science there's a fancy name for this combination of two ideas a function and
00:20:42.000
the environment where you refer to that function from and that was invented a long time ago back in 1975 i don't know if that's off
00:20:49.360
the edge of the slide so back in 1975 uh an mit professor named gerald sussman
00:20:54.880
and his colleague guy steele wrote a paper called scheme an extended an interpreter for
00:21:00.240
extended lambda calculus sounds really complicated and it's pretty it's a pretty hardcore academic paper it's
00:21:06.320
actually online you can download this and read it yourself i did the other day it was really fun
00:21:12.240
i love this stuff i'm kind of a language geek and this paper defined and and explained
00:21:18.559
their implementation of a language called scheme scheme is a dialect of lisp and what's interesting about scheme and
00:21:24.960
scheme is still being used today uh is that it was the first time that in computer science there was a formal
00:21:30.080
definition of the word closure so they define inside this um this academic paper what a closure is and here's how
00:21:36.400
they defined it so we'll just read through this for a second in order to solve this problem i don't remember what problem they were solving
00:21:42.400
we introduced the notion of a closure uh which is a data structure containing a lambda expression whatever that is
00:21:49.760
and an environment to be used when that lambda expression is applied to its arguments so kind of hot you know come
00:21:56.240
you know convoluted language here a lambda expression that's a fancy way of saying a function or a method and um applying that lambda expression
00:22:03.760
to its arguments just means to call the function and pass in the arguments but if you think about this for a minute
00:22:09.840
this is actually what we have in a ruby with a block we have a function and an environment to be used when we
00:22:17.120
call a function you know backing up this is what a block is it's the function on the top left and in the environment on
00:22:23.120
the lower left and so uh and so what we've done here is we've shown that blocks and ruby are are they're ruby's
00:22:30.320
implementation of closures so ruby has taken and i don't know if matt did this on purpose or you know indirectly but
00:22:35.760
ruby's taken um a concept from the 1970s and applied it to a language that was
00:22:41.120
invented in the 1990s and that we're still using today so i find that kind of interesting
00:22:46.880
so let's move on and talk about lambda so lambdas are related to blocks and let's see if we could do experiment to figure out you know what lambdas are
00:22:53.520
but before we get to the lambdas themselves you know where is the word lambda come from you know why in the world does ruby have a keyword that is a
00:23:00.320
greek letter lambda is a letter in the greek alphabet what does it have to do with computer science
00:23:06.159
um so it turns out the same thing that ruby borrowed the idea of lambda or the word lambda from lisp this is the first
00:23:12.559
computer that ever ran uh the lisp language this is from the early 1960s list was invented by john mccarthy in
00:23:19.200
the late 1950s actually so this is what is that almost 50 years ago now
00:23:26.240
and in lisp john mccarthy introduced the idea of a lambda which is a way and so
00:23:31.440
here's a list how you would create an anonymous function i'm not a list developer and many of you here might be
00:23:37.280
what i think this is doing is it's defining a function there's a lot of parentheses in lisp but what i think
00:23:42.720
it's doing is defining a function that takes the argument and then divides that argument by two and returns that value
00:23:48.559
so it's uh but it's not doing that immediately it's returning a function as a value and so what lisp introduced and
00:23:55.039
other functional programming languages do this too they allow you to treat code or functions as data values
00:24:01.679
uh and so they you know quote unquote they treat code as a first class citizen so in ruby you can do the same thing
00:24:07.440
ruby borrowed the lambda key word from lisp and um here's uh you know here's my same
00:24:12.480
silly example um now i'm doing in in an even more complicated and confusing manner so what i want to do is
00:24:19.039
demonstrate how you use lambda in ruby so we're going to do the same thing we're going to print out the same string
00:24:25.039
not 10 times just one time today well for this experiment so how does this work
00:24:32.000
a lot of you guys might know exactly how lambdas work but in case some of you don't or haven't used one in a while let me go through it so the way this works
00:24:38.960
is on the bottom i call a method display message calls into the method in the method we
00:24:45.520
save the quick brown fox the first half of the string in a variable str same thing we did earlier and then we have a
00:24:51.200
block that takes um the first half of the string and prints it out concatenated with the second half of the
00:24:56.480
string except the difference here is i don't call the block immediately instead i say lambda and what lambda does is it
00:25:02.720
returns that block as a value it converts it into a data value of some kind and that becomes since it's the
00:25:08.640
last thing in the display message method that becomes the return value for that method and then finally we say dot call
00:25:16.159
at the very bottom so it's not until the bottom here where i say call that it prints out the full string and you can
00:25:21.919
try this on your own computers so let's go through this and see how this is implemented inside of ruby you
00:25:28.240
know what goes on when you when you use lambda so we'll go through it one line at a time slowly the first thing is i
00:25:34.000
take the fox string and i put it in the str variable so same story here um you know no surprise we save the string on
00:25:41.679
the stack and there's a control frame structure um and the bottom empty one is the outside scope and then we have one
00:25:48.000
for the message uh for that in that method that we're calling this should be display message
00:25:54.080
um okay what do we do next so now we call lambda so what happens inside of ruby when you call the lambda keyword so
00:26:00.799
it's going to be a complicated diagram don't panic i'm going to go through this slowly let's see if we can figure this out together
00:26:06.799
and let me take a couple minutes to just explain this picture so before i get to all the boxes and arrows on the right
00:26:12.559
let me talk about the two words on the left stack and heap so what i'm indicating there is there's really two
00:26:17.679
different types of memory in your ruby process or really in any process for that matter there's stack memory that's
00:26:23.360
what we've been talking about so far where you push values onto a stack you pop them off the stack is used for
00:26:29.120
values that are very temporary in nature so they're only valid or they're only needed for a short period of time
00:26:34.559
usually during the context of one method or function on the on the bottom under the dotted
00:26:40.320
line we have the heap and a lot of you may know the heap is for where as for when you want to save things for a long
00:26:45.520
period of time you know when you um want some value to persist around or you know the lie around for a while so you can
00:26:51.120
use it later and the biggest or most obvious example of using the heap in a ruby program is
00:26:56.559
when you create an object it gets saved in the heap it ruby allocates the memory out of the heap and uses it for that
00:27:02.240
object so what's going on here when i call lambda what does it do well the first thing it does let's start
00:27:08.080
from the top left it takes that internal stack where we have that str variable and it copies it from the stack it
00:27:14.640
copies that stack frame down into the heap so it creates a persistent sort of
00:27:19.840
um a new second copy of all the same information but what's special about that is it's now in the heap so it can
00:27:26.240
persist for a longer period of time and the other thing ruby does it creates on the bottom and i don't know if
00:27:32.000
everyone can see this some of you are pretty far back there's a structure called rb underscore env
00:27:37.760
and this is a what's called the environment object i assume and this is a sort of a structure
00:27:43.279
that ruby uses to manage that heap copy of the stack so interesting what else is going on
00:27:49.279
here the other thing that ruby does on the right is it takes the control frame structure and it copies it into
00:27:54.320
something called an rb underscore proc structure and as you might know some of you the return value of a lambda
00:28:01.440
key of the lambda keyword is actually a proc object and so this is ruby's internal representation of a proc
00:28:07.840
and you can see it puts in there the dfp pointer the iseq pointer and a couple other things
00:28:14.240
and one interesting detail to notice here is that inside the proc structure is a block structure
00:28:20.000
uh and so in fact what we what we're seeing here by looking at the way these things are implemented is that procs and
00:28:25.520
lambdas and blocks are all really the same thing they're all closures they're all that combination of the iseq pointer
00:28:32.559
the code or the function and the dfp pointer which is the referencing environment so there's all really
00:28:38.720
different ways of looking at the same thing and so when you call lambda it creates that new object it copies the stack
00:28:44.399
frame or that environment into the heap and then it um and then it returns that proc to you as
00:28:50.799
a return value okay so finally we're ready to print out the string and we're ready to say
00:28:56.360
displaymessage.call and it's going to print out the string it'll work just fine but what i wanted to point out here is something interesting which is
00:29:02.799
if you look at this carefully and think about it how is it possible that when i call
00:29:08.320
dot call at the bottom it prints out the whole string if you look at that first variable str
00:29:13.919
that's a local variable inside a method normally when a method returns all the variables that you create in a method
00:29:19.919
are you know they're free they're released they're gone forever but obviously here that str variable lives
00:29:25.760
on it's got sort of a second lease on life and it's able to do more you know more good for you do more in your program how does that work well that's
00:29:32.880
really back up to this picture that's the whole reason why ruby is doing all this work all this copying all these new
00:29:39.440
structures and pointers all this is just so that you can continue to refer to the environment where you called lambda from
00:29:47.120
so at the moment i call lambda that environment not only what's inside the lambda but the surrounding values are
00:29:52.559
all saved away so that later when i say call it works fine and looking at how that's implemented
00:29:59.120
inside of ruby it's sort of a similar picture except the opposite so what we do now is ruby takes the dfp
00:30:06.159
pointer from the block that's inside that proc and it copies it up into a new stack frame that it sets up to run the code
00:30:13.279
inside the block and remember in the block we're creating str2 that's the dog part of the string
00:30:19.919
so that is pushed onto the stack now on the top left and it saved that dfp pointer there
00:30:25.440
which points now it points out to the heap where the referencing environment or that you know copy of the reference
00:30:31.520
of that environment is saved so it's just like calling a normal block except for that detail the dfp is now pointing
00:30:37.679
out to the heap so that's how lambdas work so let's move on that's enough about
00:30:43.039
blocks and we've seen how blocks and laminas and procs are all the same thing and they're all
00:30:48.080
they're all closures now let's shift gears and talk about closures and meta programming so two different very very
00:30:53.840
different ideas how are they related and you know what does meta-programming have to do with closures
00:30:58.960
so first of all um so let's do an experiment using a closure to define a method so what in the world does that mean
00:31:05.840
so let's take another example let's say i create a now i'm going to wrap my exciting fox dog string and wrap it up
00:31:11.440
in a class so i'm going to create a class called quote and you all know how to create methods what you do is you say
00:31:16.960
class the name of a class quote and you say def this def that def the other thing you just start typing in method
00:31:22.399
definitions and use the def keyword def all very obvious so how do you do meta programming what
00:31:29.279
is meta programming well one way of doing meta programming in ruby is to dynamically create methods and all that
00:31:35.200
means is instead of saying def def i can say define method display message so the
00:31:41.360
only difference between this what you normally do and this the define method meth method
00:31:47.600
is that you pass the name of the meth of the new method as a parameter to it so the name of my new message my new method
00:31:54.240
is display message i should have thought of a different example that wasn't a tongue twister
00:31:59.360
so here i am defining a method in a more confusing convoluted manner that doesn't make as much sense that's what meta
00:32:04.559
programming really is all about isn't it so um but you know in a real example what
00:32:10.240
you would do is you would probably have a loop here you might create 10 or 20 methods all the same time you're going through the you know each name each
00:32:16.720
method's name is maybe an array or maybe the name of the method is from user input or from a database query so there
00:32:22.320
might be some really valid reasons for doing something like this but what i want to point out here is something different i think there's
00:32:28.000
actually some there's another minor detail here you may not have noticed that's different between this
00:32:33.360
and this which is that when you say define method and you pass in the the name then you have to say do
00:32:39.679
end so the the methods body is actually a block you're passing a block into this define
00:32:47.039
method method and that's not what we did over here we just said def the name and then we started typing in the code
00:32:53.519
and remember blocks are closures so because the block is a closure it means that the code inside of the block
00:33:00.240
can access things in its surrounding environment now in this example there really is nothing interesting around
00:33:06.080
that there's no there's no reason to do this but imagine if in your in your ruby application there was an interesting
00:33:11.679
environment there was a place in your code where you had a lot of interesting values you wanted your new method to be
00:33:17.120
aware of and to have access to so i'll make up a contrived example let's say i only remember to put the fox
00:33:24.080
the first half of the string in the in the instance variable at str but then i discovered later oh i have
00:33:30.240
the dog the second half of the string str2 somewhere else in my code you know how can i how can i allow my exciting
00:33:37.360
method to print out the same thing both strings together well since it's a closure it's able to access
00:33:43.760
the instance variable so it's going to become a method of this quote class it's able to access equally well at str
00:33:51.360
on the top and str2 which is in the surrounding scope there
00:33:57.600
one confusing detail here here is i have to say quote dot send define method that's because the define method method
00:34:04.240
is a is private to the module class but other than that it's actually not that complicated so this is how you can
00:34:09.599
define a method using a closure okay let's take another example um let's
00:34:14.720
look at sort of the classic example of meta programming so meta programming really means to write code that writes
00:34:20.639
code and in ruby the simplest way to do that is to say is to use the eval function if i create
00:34:27.040
a string like put s two plus two uh again this is silly but you can imagine
00:34:32.399
a real example where you're dynamically writing code that based on some kind of data you're getting from somewhere
00:34:38.320
you can pass that string into eval and what it will do is it will recursively call that virtual machine again and
00:34:44.960
compile that code into those bytecode instructions and then run them and so in this case it'll print out four
00:34:50.320
so not really interesting but one one detail about eval that maybe you didn't know is that it's also a closure so the
00:34:56.480
code inside of this string can access the environment around the call to eval
00:35:02.240
so what does that mean and how does that work well another way of doing this is to use the binding keyword this allows you to
00:35:08.960
formally specify that environment that you want the eval to work with um so so imagine for a minute i went
00:35:15.839
back to my quote class and i said def get binding so i create a method called get binding and the reason i want to do
00:35:21.839
that you'll see in a minute the reason i want to do that is to be able to access the scope or the environment inside of
00:35:27.920
this class so well so i can access that at str variable so hold that thought let's move on and
00:35:34.880
look at creating a new instance of the quote class so i'll say object equals quote dot new so now i have a new object
00:35:42.720
what i want to do is so let me take a moment and explain just for for 10 seconds or less how ruby implements
00:35:48.880
objects and i have a lot more about this in my book um there's a um a lot more detail whole chapter on
00:35:54.800
objects but in a nutshell whenever you create an object ruby creates a structure called our object at least for
00:36:01.119
custom objects that you write you know for strings and arrays there's something else but one of the other details i didn't
00:36:07.040
show in that control frame structure is that there's another thing called the self pointer that points to the current
00:36:13.599
value of self the object in which you're running at the moment so all of you are probably familiar with self but if not
00:36:19.440
just imagine that as you go through your ruby call stack and you call one method calls another calls another each one of
00:36:25.040
those methods belongs to some object some instance of some class and so the way ruby implements that is in this
00:36:31.760
control frame structure and there's one of these for each level in your ruby call stack there's a self pointer that
00:36:37.119
points to which instance uh which object instance is currently the cell the current value
00:36:43.119
of self um the r object um structure itself has some other things in it but the most
00:36:48.400
important two values are there's a class with a k that's the class pointer that we'll point out to indicate which class
00:36:55.440
this object is instance of and then there's another thing called iv ptr or the instance variable pointer and
00:37:01.359
this points to an array of the instance variables for this object so in this case we're going to have at
00:37:06.720
str in that in that array okay so back to our example now we know something about how objects work
00:37:12.079
internally and i've created an object obj equals quote.new and now i call eval
00:37:17.680
and what i want to have happen and this of course this will work you can try this if you really want to but what i want to do here is allow the compiler
00:37:25.200
when it compiles that string to compile it in the context of the obj object
00:37:31.520
and so the way you can do that in with the eval method is you can pass in a second parameter which is that binding
00:37:38.640
and remember we got that so we're saying obj.getbinding so backing up to my class at the bottom
00:37:45.200
we had that method that we saw earlier get binding and that just calls this special this mysterious keyword binding
00:37:52.800
and what does that do and what happens at the moment i call binding and what is a binding anyway
00:37:58.160
so well we can see what it's intended to do it's going to tell eval when you compile that code do it in the context
00:38:06.480
of this binding and that binding can be anywhere in your application and so how does that work what happens
00:38:12.400
inside of ruby when i say binding well let's take a look
00:38:17.680
so this is this sort of a similar picture to what we had earlier we have on the top we have the stack on the
00:38:22.880
bottom we have the heap and on the top we have the control frame structure now i'm going to point out there's the self pointer there along
00:38:29.200
with the dfp and all the same things will happen with the stack it'll copy the stack out into
00:38:34.720
the heap and it'll set the dfp pointer to the heap copy i'm not showing that in this picture just to keep things simple
00:38:41.599
but when you call binding ruby will also call it'll also create a new structure called rb underscore binding on the
00:38:48.880
lower right and in that structure it'll save the file name and the line number where you call the binding keyword in case you
00:38:55.599
want to you know debug something you can or if you print out the binding it'll tell you but it also creates this thing
00:39:01.200
called env which points to the same environment structure we saw earlier with the lambda keyword
00:39:07.599
except this time what i'm actually showing inside that environment a block structure so it
00:39:12.640
turns out there's another block structure in the environment and i'm also showing an additional value
00:39:18.000
here inside the block called self so when you call binding it copies not only
00:39:23.520
the dfp from that control frame into the new block it copies also that self pointer and the self pointer will point
00:39:30.240
over to the r object the to the current value itself and that's how when i you know compile
00:39:36.720
that code using eval it's able to access all the way down on the lower left the at str variable
00:39:43.520
so what we've seen here is bindings lambdas procs and blocks are all really the same thing they're all
00:39:49.839
closures and they're all different ways of looking at the same thing and i find you know i think this is all
00:39:55.520
fascinating okay so and then just reviewing you know going back to my original question what
00:40:01.280
is a block now we've seen there's uh three things in this block structure rb block structure we have first the iseq
00:40:07.839
the instruction sequence that points to the function we have the dfp the dynamic frame pointer that
00:40:13.520
points to the referencing environment and now we have this third thing which is the cell pointer that points to our
00:40:18.800
object structure which is the current value itself so this is really the best definition of
00:40:24.960
you know what is a block and it's also the way uh the best it's the way that ruby has implemented closures
00:40:31.440
so in ruby the cl closure in ruby is a function combined with its referencing
00:40:36.560
environment and i guess what you could call the object environment so the con the objects context or the context of uh
00:40:43.599
in which you're calling that block so that's it for today i'll let you guys go at the end of the day if you like
00:40:49.760
this stuff and you're into ruby internals you want to learn more you know buy my book ruby under a microscope
00:40:55.359
and then you'll find it online on my website thanks a lot
00:41:31.440
you