Search
Talks
Keeping Ruby Reasonable
Edit
#ruby
#metaprogramming
#ruby-internals
#programming-language-design
#dynamic-scoping
#closure
Summarized using AI

Keeping Ruby Reasonable

Joshua Ballanco • September 29, 2011 • New Orleans, Louisiana • Talk

The talk "Keeping Ruby Reasonable" by Joshua Ballanco examines the challenges of reasoning about Ruby code due to its flexible and dynamic nature, particularly relating to the concept of first-class environments. The session at RubyConf 2011 starts with an introduction to the speaker's background and the intention to discuss how Ruby’s treatment of bindings and open classes complicates developers' ability to predict code behavior.

Key Points:

- Definition of Reasonable: Ballanco defines reasonable not only in terms of sensible programming but also in terms of the ability to easily reason about what Ruby code will do.

- Dynamic Nature of Ruby: Ruby allows for changes in classes and methods at runtime, posing challenges for developers as the only way to ascertain the actual behavior of a program is to run it.

- First-Class Environments: The talk distinguishes between bindings and first-class environments, emphasizing that Ruby’s implementation allows bindings to be mutable and accessible throughout the program, which can lead to unpredictable behavior.

- Ruby vs. Scheme: Ballanco compares Ruby's handling of environments with that of Scheme, referencing discussions in the Scheme community regarding the benefits and dangers of first-class environments. He highlights the conclusion that while flexibility in binding environments seems beneficial, it makes code reasoning almost impossible.

- Implications for Performance: The chaotic nature of Ruby's flexible environments affects performance, as Ruby's runtime cannot safely optimize code without guarantees about bindings. This is exacerbated by the challenge of compiling Ruby due to the unpredictable behaviors caused by first-class environments.

- Examples & Case Studies: Throughout the talk, Ballanco presents examples of coding scenarios (like the Fancy Greeter story) that illustrate how bindings can be dynamically modified, leading to unexpected errors and behaviors in the code.

- Final Thoughts and Discussion Starter: The talk ends without a definitive solution but emphasizes the need for a conversation on how to manage the issue of first-class environments in Ruby better to enhance comprehension and optimization. Ballanco invites questions and aims to engage the audience in a deeper discussion about these complexities.

The overall takeaway from the talk is that while Ruby’s flexibility provides power to developers, it simultaneously hinders the ability to reason about code and optimize performance, suggesting a need for careful management of this flexibility in future language developments.

Keeping Ruby Reasonable
Joshua Ballanco • New Orleans, Louisiana • Talk

Date: September 29, 2011
Published: December 12, 2011
Announced: unknown

Pop quiz, hot shot! What is the value of this Ruby expression: "2 + 3"? What if I told you that somewhere earlier I had done this: "class Fixnum; alias :+ :*; end"? Changes everything, right? We're all familiar with Ruby's open classes. For many of us, they're one of the facets of Ruby that make it so much fun to work with. However, Ruby treats bindings as first class objects, and they are, similarly, open. This can lead to problems. What sorts of problems? Well, effectively, the only way to know what a Ruby program will do is to run it. Stated another way: it is impossible to reason about Ruby. Now, this may not seem like such a big deal for the seasoned Rubyist. So long as we're responsible in how we write our code and make sure not to do anything too crazy, we can have a pretty good idea of what any piece of code will do ahead of time. Unfortunately, compilers and runtimes can't rely on "responsible" programmers and having a "pretty good idea" of what code will do just doesn't cut it for a VM. As a result, method caches get invalidated far too often and whole classes of common optimizations do not work with Ruby. So, what's a programming language to do? Luckily, Ruby is not alone in facing this dilema. The Scheme community has been confronting similar issues from the very beginning, and in the R5RS and R6RS reports, they outlined a solution. This talk will take a look at what makes first-class environments and "eval" so problematic, and what lessons Ruby might learn from Scheme in how to be more reasonable.

RubyConf 2011

00:00:16.340 hi congratulations on making it this far
00:00:21.450 on the first day of rubyconf I know it can sometimes be a little overwhelming that start of the conference all the
00:00:27.570 excitement and everything just wanted to start off with a quick introduction my
00:00:33.390 name is Joshua Belenko if you want to follow me on Twitter I don't say that
00:00:38.460 much but I tweet under Manhattan metric and on github you can find me under jay
00:00:43.650 blank I am a software engineer for patch which is a hyperlocal news service and
00:00:50.730 it is owned and run by AOL I also happen
00:00:56.370 to be a member of the mac Ruby core team and if you grab me after the talk and
00:01:02.520 talk to me for any length of time I will probably end up bringing up matt grupe at some point I really enjoy that
00:01:10.020 project but that's not what I'm here to talk about today today I want to talk about keeping Ruby reasonable okay so
00:01:19.110 before we get into it what do I mean by reasonable well there's the standard
00:01:24.450 dictionary definition something based on good sense a reasonable price reasonable expectations and that applies to Ruby
00:01:33.660 and programming in general but I wanted to take it actually a step further and give it a double meaning and i also want
00:01:41.009 to talk about keeping ruby reasonable in the context of being able to reason
00:01:46.110 about Ruby so to start this discussion
00:01:51.509 and and by the way I should mention I I do want this to be a discussion in fact
00:01:57.360 I had a lot of material I shaved it down left plenty of time at the end for
00:02:02.459 questions comments honestly I will jump to the end and tell you I don't have a
00:02:09.000 good conclusion to this talk I wanted it to be a conversation starter so if I can
00:02:15.750 do that then I will consider it a success but I'd like to start the conversation with the story so the story
00:02:25.049 starts with a Ruby programmer goes by the name Ruby reasoner you wanted to write a class
00:02:31.610 this is a very simple class it's got two methods it's a greeter depending on the
00:02:37.890 time of day it's going to greet you with the appropriate greeting and then we throw in a little loop at the bottom so
00:02:43.590 we can do some IO present a prompt get the response present the proper greeting
00:02:50.370 by calling a method on an instance of the class so Ruby reason I wrote this is
00:02:57.630 pretty proud of it and his friend came along Peter Python Easter and his friend
00:03:03.150 Peter pie that needs to want it to help hey Ruby reasoner I got something for you I've got a library that I call fancy
00:03:10.860 pants here's what you do you call the fancified method and pass it a block whatever you put in that block we're
00:03:18.060 gonna fan safai it so Ruby reasoner said hey that's that sounds like a good idea
00:03:24.209 I'll go ahead I'll throw that in my greeter class make a fancy greeter and then go ahead and run it and this is
00:03:34.260 what happens so he goes into his directory runs his fancy greeter gets a
00:03:44.310 nice prompt said morning okay good night uh Cindy evening oh look hey that's
00:03:51.480 pretty fancy and hmm that's an interesting error let's try this again
00:03:59.720 all right let's start p on this time okay try it again just to make sure yep
00:04:05.370 okay everything looks like it's working fine that was a weird bug I don't
00:04:10.500 understand where that came from but uh
00:04:15.560 that's a really interesting error message wrong number of arguments one
00:04:22.109 for 190 what the heck is going on well
00:04:28.080 at this point Ruby reasoner a little suspicious of his friend decided to take
00:04:34.289 a look inside of that library and here's what he found so at the top we're fanta fighing things
00:04:43.699 right that's easy but then look at what Peter Python East has done to our friend
00:04:49.660 he's going to spawn a thread he's going to sleep for a random amount of time then he's going to take the block that
00:04:57.199 Ruby reasoner so naively passed to this method he's going to steal the binding from that block pick a random method out
00:05:04.370 of it and then redefine it with a somewhat reasonably looking error
00:05:10.370 message right completely redefined the class method and the thing about Ruby is
00:05:20.410 that you can do this is completely legitimate Ruby code Ruby We Need to
00:05:30.770 Talk we need to have a conversation because as programmers we very
00:05:39.110 frequently find ourselves asking this question what does it mean chances are
00:05:46.880 if you do development as a career or even as a hobby you probably end up
00:05:52.070 reading more code than you right and even the code that you do right at some point you're probably going to go back
00:05:57.500 and read it and very often when you read this code you find yourself asking what does it mean so one plus one what does
00:06:04.580 it mean well maybe it's just a math equation that's easy 1+1 2 done maybe
00:06:10.880 it's a program hmm what if we look at
00:06:15.949 this okay now that definitely looks like a program that's no math I'm familiar with so the next thing we might jump to
00:06:24.740 is okay well if it's a program what's the language right and if you tell me what language it is I can start to
00:06:30.949 understand bits and pieces I can pick out the tokens I can find the symbols right the basic tokenization parsing
00:06:39.770 style stuff that a language does but that's not really what the program means what the program means what it does
00:06:47.979 depends on the functions depends on the semantics depends on how we did value
00:06:53.030 transformations and if we take all of these together all of the information that we need in order to understand what
00:07:00.050 a program actually does you can call that an environment so when you want to
00:07:07.550 answer the question what does it mean you need to know something about the
00:07:13.580 environment that code is going to run in
00:07:18.850 okay in honor of the Ruby rogues can we
00:07:24.080 get a definition to start this off okay a couple of definitions in fact because I'm going to go ahead and use language
00:07:32.870 that's a little more sort of universally applicable which is slightly different than the language that Ruby uses to
00:07:38.750 refer to the same things so before I get into that I want to make sure that we're all on the same page so first of all
00:07:45.289 something like this a gets one that's what we call a binding okay we now have
00:07:52.190 a variable a that is bound to a value 1 we have a variable B which is bound to a
00:07:57.620 string object in that string objects values hello world right so when you
00:08:03.050 take all of these together all of the
00:08:10.580 bindings together that are in a scope we can call that an environment and it's
00:08:17.570 these bindings which allow us to make sense of code right because it's all about grabbing the bindings and then the
00:08:23.960 values the functions whatever it might be that associate with them okay so when we say an environment what we're talking
00:08:30.349 about is the collection of bindings now you're all probably familiar with the concept of a closure whether you realize
00:08:36.469 it or not any time you do an do end block and Ruby you've created a closure
00:08:41.740 and in Ruby we have this method unfortunately named binding because it
00:08:47.779 makes things a little confusing but that binding is actually not returning the binding it's returning an environment
00:08:53.830 it's returning all of the bindings right and it does so as an object and this
00:08:59.779 will call a first-class environment so environment is to first class environment the same way function is to first class function if
00:09:06.490 you've ever heard that concept for example in Python you can use functions as values you can assign them to
00:09:12.790 variables you can pass them around it's the same idea in Ruby and first-class environments right an environment is the
00:09:19.389 collection of all the bindings in a scope a first-class environment is an environment that you lift out of place
00:09:24.639 make an object and then you can pass it around and do things with it ok so with
00:09:32.050 those definitions out of the way I want to talk about what motivated me right I been very encouraged a lot of the talks
00:09:37.510 have been people pulling other sources and I think it's good that we learned
00:09:43.360 from other sources and in this case I was originally inspired to do this talk by a series of blog posts from 2009 it's
00:09:52.149 a really good blog by a guy named John Marshall he's a core contributor to MIT
00:09:57.699 scheme and the subject of the series of blog posts that he wrote in this time
00:10:03.269 referred back to the scheme standardization process so if you're not familiar with scheme scheme actually has
00:10:09.430 a standard they have a committee that gets together and agrees on a standard and every soft and they reconvene and
00:10:15.310 they revise it so they're now currently on the sixth revision which we sometimes call the r6 RS and oh yes r 7 is
00:10:24.850 currently in process so r 6 is the most recently ratified i believe so anyway in
00:10:31.510 the r6 RS ratification process of the discussions that were happening surrounding scheme one of the big
00:10:37.300 debates was on the concept of first-class environments should we have them how should they behave and in the
00:10:45.010 series of blog posts Joe Marshall does a really great deconstruction of them and
00:10:50.470 in the final blog post he comes to the conclusion he says I really liked first
00:10:55.990 class environments when I was first exposed to them at this point I believe that first class environments are
00:11:02.019 useless at best and dangerous at worse you kind of saw that at the first
00:11:08.529 example that I showed you but let's delve in a little further and understand what exactly he means by this so
00:11:17.190 in trying to define the behavior and the semantics of first class environments in
00:11:24.660 this blog post Joe breaks it down into four essential questions that you have to be able to answer about the behavior
00:11:30.540 of a first class environment okay so first should you be able to extract an
00:11:38.640 environment from any closure that is anytime we closed around a scope should
00:11:45.390 we expect to be able to get all of the bindings from that scope at any later point in time yes or no right well Ruby
00:11:54.950 Ruby says yes now keep this in mind
00:12:00.870 because we'll come back to the implications maybe the unintended implications of making this decision a little later okay next question when we
00:12:11.730 get a first class environment should all of the bindings that are in scope at the
00:12:17.430 time the closure is formed should all of them be in that first class environment you could say yeah sure everything
00:12:25.020 that's in scope when you make that closure when you get the first class environment from that closure all of
00:12:30.270 those bindings are still available or we could say no no inside your closure
00:12:35.820 you're using some of those bindings we're only going to get those right or maybe we make you tell us which bindings
00:12:44.820 should be explicitly carried across in the first class environment of course
00:12:49.920 Ruby says all of them anything that's in scope at the time you form a closure
00:12:55.740 when you get the environment from that closure they're all still valid okay now
00:13:01.650 that we've got our bindings are they going to be live right that is are they
00:13:07.290 are we just capturing the value all right so so we could say yes all the
00:13:12.690 bindings are live they're all mutable so you can query any value if another part
00:13:19.830 of the program updates the value that's reflected in the first class environment if you have a handle on the first class
00:13:25.740 environment you can go in you can redefine things or we can be more restrictive we can
00:13:30.960 the bindings are are live right so if they change somewhere else in the program then that's reflected in the
00:13:36.660 first class environment but if you have a handle on the first class environment you're not allowed to manipulate them
00:13:42.000 right or we could just make our first class environment snapshots right just collections of values at the time the
00:13:48.960 closure was created or we could do some sort of user specified version of that
00:13:56.420 of course Ruby again says all those bindings are all live there all mutable
00:14:03.110 right that's how I was able to redefine that method just by having the binding for that blog ok what happens if you
00:14:12.720 have a first class environment and you define a new binding right does the new
00:14:18.840 binding shadow any older binding right so when you redefine the method is that method redefined everywhere in that
00:14:25.260 environment maybe you clone the environment and the new binding is only
00:14:30.990 available in the clone right so behind the scenes you're duping the environment and putting the new binding in there or
00:14:36.810 maybe you just simply say look i'll give you the environment but you don't get to manipulate it it's not mutable again
00:14:45.470 right Ruby chooses one because when I
00:14:50.610 grabbed that binding and redefine that method the class which originally had
00:14:56.520 the original method so on my redefined method right so after iterating through
00:15:05.100 these four questions this is the conclusion that Joe comes to in his blog
00:15:10.590 post when someone suggests first class environments I assume they want options 1 1 1 and 1 in this variation though the
00:15:20.370 user simply cannot reason about his code
00:15:25.910 and he goes and he decomposes that a little bit he's saying essentially once
00:15:31.410 you've done that anytime you create a closure and hand that closure off to another piece of code that's it games
00:15:38.580 over you can no longer guarantee the behavior of any piece of your code Ruby chooses options
00:15:49.360 111 and one right and and again it's very understandable when you approach
00:15:56.800 these questions one at a time right why wouldn't I allow all of the bindings to
00:16:02.800 be in scope why wouldn't I allow them to all be mutable it's only when you consider them as a whole and the
00:16:07.899 implication of choosing one for all of those questions that you understand just
00:16:13.329 how dangerous this is okay so this is the problem what do we do about it well
00:16:22.720 as I said these blog posts were part of a discussion around the r6 RS process
00:16:28.269 and they eventually did come to a conclusion this is their conclusion you can find all the details all the gory
00:16:34.750 details on their website which is actually really a great read if you ever have a spare week this is about what it
00:16:42.160 will take but essentially so they've gotten a vowel method right that second
00:16:47.620 line there the third line is what we're eval and then the 4th through 6th lines
00:16:53.889 those are the environment definition and essentially what we're doing is we're
00:16:59.319 using library references so scheme introduced this concept of library references most of your code will be
00:17:05.290 encapsulated in a library and so r NR s is a library and then they have various
00:17:10.390 forms on the library so you can say out of this library only grab these bindings
00:17:15.850 right prefix all the bindings with a token so that's why in our eval line on
00:17:21.939 line three we're calling eval car and Eve are you Val cons right and not a car
00:17:27.730 and cons you can also rename you can also limit right so you can take all the
00:17:34.030 bindings except for these bindings so they have a series of forms which are really nice for manipulating these
00:17:39.630 importantly though you can't create any new bindings in an environment so you can get the environment from a library
00:17:45.520 that's fine you can get all the bindings you can use their values can't change them
00:17:53.269 another another example I wanted to bring up was objective-c and now this is a little weird probably because
00:17:59.309 objective-c doesn't actually have first class environments mostly because
00:18:04.350 objective-c doesn't have eval or any concept of eval but as I was pointing
00:18:12.600 out in the definitions a first class environment is just an environment that you are able to lift out of a closure if you have closures you still have a
00:18:19.289 concept of environments and objective-c does have closures now with blogs and therefore does have a concept of
00:18:26.100 environments for those blogs and I kind of like what objective-c does because
00:18:31.649 when you declare these variables right so here are bindings that got a name we got to create it out we got things then
00:18:37.919 we're going to create a closure right and what happens is if we reference one of these bindings inside the closure
00:18:44.330 name gets automatically retained when we do our block copyright before returning right so we retain it but we retain it
00:18:53.250 in a read-only form if however we declare something under under block then
00:19:00.509 when we reference it in the closure not only do we retain it but it's also live we can manipulate it we can mutate it
00:19:08.539 finally if we have a variable that's declared in scope but is not actually
00:19:13.919 captured in the closure it's released or garbage collected depending on your mode and it goes away all right so this is
00:19:22.230 kind of a nice you know it puts a little bit more work on the program or a little
00:19:27.659 bit more work on the compiler you have to resolve symbols and whatnot but it's
00:19:32.970 a nice I feel happy medium for how to have environments but still keep things sensible okay so so far this is all sort
00:19:44.009 of pie in the sky language design type stuff I just want to write code right
00:19:50.399 I'm not going to do anything stupid like that first file where you know I created a thread and ground to binding and
00:19:55.769 redefine things that's dumb how is this going to affect me well how many times
00:20:00.840 have you done this right you've got let's say you're doing a rails app you need a before filter and round
00:20:06.419 filter something on a name scope alright so you're gonna you're going to create a proc to use for that but to create the
00:20:15.090 proc you have a process that you have to go through there's a lot of data involved maybe you have to go out to a
00:20:20.610 service get stuff and then you have to parse a dom or who knows what until you
00:20:26.340 find the thing that you actually need and then you say but but but trust me this filter is really easy it's just an
00:20:32.429 equivalence check well guess what because anybody who gets a handle on
00:20:39.809 that proc could call binding on it and could reasonably according to Ruby expect to grab any of these other
00:20:45.570 variables they all remain live so long as your proc remains live your giant
00:20:51.179 data your big dumb all the parser they aren't going to get collected so you
00:21:02.460 thought you were safe by just passing a little tiny product mm-hmm that proc
00:21:07.710 carries a lot of baggage with it okay
00:21:14.279 now this is an honest question I said I wanted to start a conversation this is an honest question because I don't have
00:21:19.350 an answer to this but I love for somebody to tell me otherwise so let's
00:21:26.940 say you have a method that's going to return a proc you define a couple of variables and then you return the proc
00:21:31.980 and you don't refer to any of those variables inside of the proc and then later on oops and then later on you take
00:21:42.539 the proc you grab the binding and you eval to change the variable that wasn't referred to in the Prague you can do
00:21:49.559 this in Ruby um I don't know if anybody does I honestly don't even know if
00:21:56.850 that's reasonable this is a really magic you know far-reaching effects and if
00:22:05.429 you're not going to do that then we could probably remove the need to be able to do that now that said if
00:22:12.330 somebody does do something along these lines I'd love to know about I'd love to see what the use case is i
00:22:18.719 think it would certainly make me maybe a little smarter okay lots of talk about
00:22:27.519 first class environments don't want to leave you on a sour note so let's finish up by playing a game we're going to play
00:22:34.719 capture the flag hear the rules you need to provide a class class needs to have a
00:22:41.199 method met the name is flag I'm going to try and capture it class must also
00:22:46.719 contain a class constant called base that constant has to be a prog has to be
00:22:52.629 an instance of a proc object you can only define one class you cannot require
00:22:58.179 a load other source files you can't do any funny stuff no funny stuff just a class with a constant and a method and
00:23:07.769 this is the game we're going to play so you're going to pass me the name of your
00:23:13.179 class on the command line I'm going to require its file and then I'm going to create a new instance of my game class
00:23:19.119 with your class first I'm going to check make sure you're not cheating make sure you've got that class level constant and
00:23:25.690 that it is a proc we're going to play the game I'm going to use that proc to
00:23:31.539 grab a binding first class environment and I'm going to eval a redefinition of your flag method and then we're going to
00:23:39.639 call that method see if I captured your flag okay blue team you're up blue team
00:23:48.309 went for the straightforward solution I got a method I got a product what
00:23:54.759 happens so we run the game ah captured that proc
00:24:11.379 carried around a first class environment which exposed the entire internal workings of that class so the game was
00:24:17.649 able to just redefine methods willy-nilly whenever you like Red Team
00:24:23.590 Europe Red Team a little fly about this
00:24:29.429 red team is going to create a class level method called hidden base then
00:24:35.320 he's going to use the method method to get a handle on that method keep them keeping up here get a method and then
00:24:43.149 call to proc right this is just method to proc if you've ever done a map or in
00:24:48.250 each done that ampersand colon you've used method to Prague okay so we're just
00:24:53.590 writing it out here just method to proc when you do that base is still an
00:24:58.899 instance of the class Prague right what happens when we play the game
00:25:10.870 aha red team wins but why why did that
00:25:17.320 work right why did calling the method method to create a proc from a method why did that work when just simply
00:25:23.530 creating a proc didn't let's send in a spy see what's going on sir spy is real
00:25:30.340 simple our spy is just going to grab the base and instead of trying to redefine
00:25:35.680 things in an eval we're just going to query what what is self and what is the class of self okay so if we do this for
00:25:48.090 the blue team self is blue team the
00:25:53.230 class right and it is an instance of class right that they just exposed their entire class through the binding red
00:26:00.790 team on the other hand that proc that they passed its environment says I am an
00:26:07.840 instance of a method class my class is method not class ah tricky but Ruby
00:26:20.140 actually has two concepts of first class environments when you think about it this way right because the method method
00:26:26.679 in a sense gets an environment by by using the method method and calling to
00:26:32.980 proc so method to prompt gives you a prog right that proc has an environment but it's not mutable it's not even
00:26:40.740 accessible really by calling binding you're getting some method object and
00:26:46.090 method class but when you take that method to prague prague and pass it to
00:26:51.130 another thing you can call it and it has its entire environment to execute its call right so if you do something like a
00:26:57.220 through z dot map up case right method
00:27:02.530 to proc on up case it's taking that up case method and carrying around enough environment to know that this is a
00:27:08.230 string that we need to call the string up case on it right so so method objects
00:27:16.720 are a kind of if you look at them sideways they're a kind of first class environment that's far far far more
00:27:23.800 restrictive then the first class environment you get my calling binding on a prog so Ruby
00:27:30.800 kind of already has both extremes so I
00:27:39.010 told you I didn't have a really satisfying conclusion because I don't really have a patch to to cure the ills
00:27:44.990 of Ruby's first class environments this is a difficult problem and it's going to take a lot of thinking and a lot of work
00:27:51.440 and looking at the internals to see how this how this should be resolved but I
00:27:57.530 think what we can at least learn from looking at this initially is that the first class environments as they exist
00:28:03.620 in Ruby today they're kind of dangerous they're kind of really dangerous it means that once you pass a proc to any
00:28:10.910 other method you can no longer reason about your code and not just you right I
00:28:16.840 trust you're all good programmers you're not going to do anything like that first library class that I showed you but it's
00:28:24.950 not just you that we need to worry about right the Ruby runtime can't trust you
00:28:33.620 there's no mechanism you know you can you can hand me a piece of code and say trust me I didn't do anything ridiculous
00:28:40.900 but the Ruby runtime has no way for you to say Ruby trust me you don't have to
00:28:47.030 get all of those bindings I'm not using them I didn't do anything silly like that right maybe we need to give the
00:28:53.960 Ruby runtime a way to understand that we aren't trying to do anything silly we
00:29:02.000 also kind of have an eye to a way that we might be able to get to something
00:29:07.490 more sensible with the method class right in fact i think if we found a happy medium it would benefit both
00:29:15.650 first-class environments on prox and also the method objects that you get from calling the method method because
00:29:22.490 currently when you call the method method right you can turn that method into a proc you can unbind that method
00:29:28.280 from its class from its object when you unbind that method the only thing you
00:29:34.790 can do is rebind it to another instant of the same class or subclass of that
00:29:40.909 class now let's think about this unless you've dynamically defined that method after instantiation you're saying that a
00:29:48.379 method that I declared in a class I can pull off of an object and put on to another object which probably already
00:29:53.450 has that method or I can put it on a subclass which probably also already has that method not very useful what I
00:30:01.999 really like to do is be able to grab a method from one object put it on a completely different object but I can't
00:30:08.989 do that currently because the method object has this severely tightened down
00:30:14.509 first class environment concept right in fact I think if you look under the hood there is no there isn't actually an
00:30:21.049 environment there there's simply a name of a class and method object says don't bind me to anything that's not this
00:30:26.330 class right so I think by rethinking and
00:30:32.470 formalizing right because the other thing is is I don't think anybody intentionally decided that first class
00:30:41.149 environments and Ruby should operate this way I think it's a consequence of sort of iterative design of a language
00:30:47.389 like I said when you take each of those four questions one at a time it seems
00:30:53.480 reasonable to choose option one for each of them it's only when you look at them in aggregate that you realize what the
00:30:58.850 danger is and just how much trouble you can get in so I think what happened with
00:31:05.840 Ruby is that the language naturally developed and evolved to have first-class environments of a variety
00:31:13.070 that are very dangerous and maybe it's time that we take another look at first class environments it would mean that we
00:31:20.539 could make more sense of our code it would mean that we could optimize our code there are a number of optimizations
00:31:26.509 right part part of the material that I had to cut out of this talk was a deep dive into method call and what actually
00:31:34.309 happens when you call a method in Ruby when you look at that right that's a very long function c function if you
00:31:40.429 pull it up in see Ruby and there are all kinds of checks was the class redefined
00:31:46.220 was metally defined did somebody move my environment did somebody move my chair right there's all this paranoia
00:31:51.890 that ruby has because everything is so flexible right if we can remove some of
00:31:57.110 that by making things slightly less flexible then we could potentially optimized code way more than we do
00:32:04.370 currently okay well that's all I got
00:32:10.390 oops I I did put the code snippets I
00:32:16.310 presented up on my github account in case you wanted to play around with them the blog that I mentioned Joe Marshalls
00:32:22.250 blog very excellent blog he doesn't post frequently but when he does they're almost always gems he has a really good
00:32:28.610 series on persistence to that's at fun called up blogspot com it is all scheme
00:32:34.100 so there is that but and then I actually put up a couple of public links on my
00:32:40.730 pin board under the tag keeping Ruby reasonable it's all of the blog posts
00:32:46.040 that inspired this along with some of the r 6 r sr 5 RS the documents some of
00:32:53.180 the discussion and a couple of other resources if you want to get into this and learn more and with that i would
00:32:59.120 gladly take any questions and hopefully start a discussion
00:33:09.049 the difficulties in ashburn piloting things like I noticed about us and our
00:33:14.460 sins are what really did is it that's a short ability to like talk much where is
00:33:20.009 having dynamic environments and so I think what happened after our secure ms
00:33:27.090 was in a lot of people to write deficiency fireworks which in turn could compile interpreters and so effectively
00:33:35.009 the last email to your petition given that first class of Martin's over here
00:33:41.850 so open I wonder if anybody in the
00:33:48.259 difficulties of effective writing eilers well ok so so the question is regarding
00:33:54.929 the Redmond compilers and how difficult
00:34:01.169 it is given the openness so i can give you the mac ruby perspective um Matt
00:34:07.169 groovy didn't have the binding method for the first two years of its life even
00:34:13.379 today if you try I'm sure you'll find bucket loads of bugs and every time i
00:34:19.859 ask allah ralph was the lead developer on macaroni what about binding he wouldn't talk to me for a week um you
00:34:30.629 can do a lot without the binding method right we got two years into mcnamee we were developing the actual production
00:34:36.720 software so much shipped on lion um never having to touch binding I actually
00:34:43.169 was talking to somebody who wasn't familiar with rubiane i mentioned that i was giving this talk and they asked me
00:34:48.899 about the topic and started going into first class and where's why do you even need that I thought about it why do we
00:34:56.639 even need that the most obvious thing I can think of is er be a templating
00:35:02.640 language were to get so in order to get self of Papa jizz here's right IRB but
00:35:11.099 even IRB you can do without so the other thing that you mentioned was eval and I
00:35:16.410 guess mention that evil existed in scheme long before first class environments because
00:35:21.490 you don't actually need first-class environments for email right you can simply decide that eval will work in the
00:35:28.660 current lexical scope which is the way that evil normally works normally when you call eval in Ruby there is a hidden
00:35:36.750 argument that most people never bother with which is binding which if you don't specify the button it just as the
00:35:43.630 current binding right the current lexical scope is what your eval is using right so eval is different from eval
00:35:52.360 with first-class environments eval with first-class environments implies that
00:35:57.970 you can take one piece of code and execute it as it were in a completely different spot in the program right so
00:36:07.450 many languages without first class environment have rebels yes I RV can do some really fancy things with the first
00:36:16.690 class environment and by the way I mean obviously there was a desire to have first-class environments right it's not
00:36:21.910 that there are no benefits it's just a matter of when you weigh the benefits to
00:36:27.130 the wrist and the openness that a first class environment presents what it does
00:36:33.670 to the ability to form compilers what it does to the ability to reason about code optimize yeah you really seriously got
00:36:41.170 to think about whether or not it's worked out and in terms of writing Ruby compilers uh so i'm pretty sure Rubinius
00:36:50.740 I know Mac Ruby does JRuby does um they'll end up being bytecode compilers
00:36:56.500 and somewhere in the bytecode ends up being a way to for example in Mac movie
00:37:03.370 if you include eval and a string we just put the string in the data section of
00:37:09.970 the object file and we reference it from the bytecode and we spin up a full
00:37:15.550 version of the mac Ruby interpreter and just interpret the code line which is
00:37:21.460 unfortunate right it would be nice to be able to that more efficient but because we we
00:37:27.730 will have no guarantees on the environment at the time you call eval what you're passing in the only way we
00:37:34.119 can do it is to actually run it in situ oh yeah yeah there were a lot of changes
00:38:15.090 it's a little difficult specifically because the scheme community is a bit of
00:38:21.700 a esoteric entity within itself there was a lot of contention around the r6 RS
00:38:28.300 for example MIT scheme just flat-out declared they are not going to implement it they're gonna stick with our five RS
00:38:35.020 and wait and see what happens in the future seven is hopefully addressing a
00:38:40.900 lot of those concerns I'm part of it also is that the skin community I mean
00:38:47.050 actually we could probably learn something from the scheme community because they've always focused on standardizing the smallest subset of
00:38:53.680 scheme that you can basically equate languages and say this is a scheme this
00:39:00.130 is a scheme this is a seat until six right and this is the contention that a lot of people came up with six is six is
00:39:07.450 the first time that they actually for made into trying to standardize the library and there was a lot of contention about whether or not that was
00:39:13.240 a good idea whether or not you know you were doing
00:39:35.490 yes first class yes I will grant you
00:39:42.580 that however it occurs to me that first class environments encapsulate a lot of
00:39:50.380 other issues which people would bring up as other reasons that MRI is slow for example a lot of the confusion
00:39:56.920 surrounding constant resolution is somewhat related in that there are
00:40:04.810 multiple paths to a constant right because you've leave so many nodes live
00:40:10.930 and you can basically ok so i guess i should explain what i mean when you have
00:40:16.780 a constant right when you declare a constant that constant has a certain scope and later on in your program when
00:40:22.810 you want to get that constant you have to come up with a way to refer to it well Ruby actually has a myriad of rules
00:40:29.830 on how to get at those constants there's just a ton of ways because basically
00:40:35.460 constant resolution and Ruby is a matter of walking the nodes where different
00:40:40.570 scopes are defined until you get to the constant that you're looking for and if
00:40:45.790 you walk the nodes in a certain way and don't find the constant then you go back and walk them in a different way to try
00:40:51.160 and find the constant and you go back and you walk on it differently what this means is that so long as those nodes are
00:40:57.370 main live and mutable you have to walk them every time to find this constant you can't do constant folding for most
00:41:03.250 practical purposes now I know Charles nutter has a really neat thing that he's done with invoke dynamic to make it
00:41:09.610 slightly reasonable to constant fold in JRuby and that's a great advance but the
00:41:16.420 bottom line is is that there's still way too many ways that you can screw up that
00:41:22.110 optimization and you're back to walk in all the nodes to find your consequentially so yeah first
00:41:29.089 class environments are not the only reason that Ruby is slow but I feel like addressing it would be a good first
00:41:34.309 foray into figuring out how to make be more efficient both it's really the same
00:41:52.640 thing the advantage that humans have is that they're humans and they can
00:41:58.160 converse and human languages and say no seriously I didn't do anything funky
00:42:03.770 with your environment but I've seen playing places where people unintentionally do funky things with
00:42:10.430 their environment screw things up and those bugs are the hardest to find because you know the first thing you
00:42:16.039 want to do when you hit a bug is find out where where did it occur and I don't
00:42:21.799 know if you noticed but that first example that I ran right when it through
00:42:27.200 that exception if you looked at the back trace of that exception it looked like
00:42:32.720 it was occurring at the original method definition in the code so you would
00:42:39.529 stare at that all day and never understand why that method through that
00:42:45.950 exception you have to walk through all of the other code to figure it out and
00:42:51.799 those are the worst bugs to fix the ones where you really seriously have to investigate the entire source base to
00:42:59.299 figure out what's going on so from the user perspective yes but also from the compilers respective compilers can run
00:43:07.339 algorithms they can run heuristics they can speculatively optimize things but at the end of the day if the compiler can't
00:43:14.930 be confident in the conclusions that it's reached about a piece of code it's going to be slower than if it was
00:43:21.859 confident but piece of code another question
00:43:36.859 good
00:43:57.820 right so yes so the comment was that bindings are not the only reason that Ruby cartas Ruby code is hard to reason
00:44:05.390 about or the first class of artists and that is true right I mean if you look at languages like Haskell where you have a
00:44:11.869 proof system built into the runtime or actually compiler yes there is a lot
00:44:18.530 that you could do to do more reasoning about Ruby code I guess what I was
00:44:25.010 trying to bring a bring up in terms of first class environments is that there's
00:44:31.040 a spectrum of how much reasoning you can do about code right a classic example in Ruby is it's very difficult to determine
00:44:37.940 whether something is a method call or a variable reference right if I right Who
00:44:43.070 am I calling the foo method or am i referring to the foo variable right that's an ambiguity that's based into
00:44:48.950 the Ruby language and I think some would argue that that's an acceptable ambiguity it allows me to not have to
00:44:54.380 put / ends on all my function calls and maybe I like saving two key stories or
00:44:59.780 maybe like the way that coat hooks it does look that I love the way reveals so
00:45:05.300 there is a scale of how much reasoning you can do about your code I think what
00:45:11.300 I was trying to point out with first class environments his first class environment put us at the bob you cannot
00:45:18.109 know anything about your code like let's not even worried about type inference
00:45:26.270 let's not even worried about you know any sort of proof system with first class environments running around you
00:45:32.990 can't guarantee anything
00:45:38.559 so I was looking for mothers how can we take some so I guess the the thing so
00:45:47.809 Matt my man Eddie has a really great blog post that he did recently where he was looking at a dsl that he developed
00:45:53.989 and he was using prox in the dsl and he
00:45:59.059 did some benchmarking and he found out that the the prox were not really a good
00:46:05.479 choice of the way that he was using them it's not great so really I guess the more or less story is be careful when
00:46:12.229 you use progress it's not just first class environments there are other things about props that make them kind
00:46:18.799 of dangerous the other thing that you can do is for example if you find yourself doing anything like this where
00:46:24.380 you're hoping to pass along a little bit of code to be executed later right and
00:46:29.839 you know that you don't need any of these references what you can do here is right before the proc return you could
00:46:36.709 just nil out all those variables and then they would all get collected so
00:46:43.130 that would that would be like if you want to take something practical away from this when you're done with a
00:46:49.910 variable if you're going to return a prog nil adapt or move it to another
00:46:56.900 scope or yeah put it into the other stuff where you can't reach it from the
00:47:02.509 products yeah that's the other problem
00:47:07.849 is that you sometimes close over scopes when you don't intend to
00:47:13.080 yeah yeah self is and and define method
00:47:19.820 if you've ever been to mark the method right right the same method using death and define method and you don't run it
00:47:26.670 the same well you could that would be
00:47:38.460 very interesting so okay I think we're
00:47:43.500 probably way out of time so thank you all
00:48:23.520 you
JB
Joshua Ballanco
3 talks
Explore all talks recorded at RubyConf 2011
+55