Summarized using AI

Grow Your Unix Beard Using Ruby

Jesse Storimer • November 01, 2012 • Denver, Colorado • Talk

Summary of "Grow Your Unix Beard Using Ruby"

In this talk at RubyConf 2012, Jesse Storimer explores the intricacies of using Unicorn, a web server for Rack applications. He aims to teach attendees about Unix systems programming and how Ruby can interface with it effectively. The presentation promises to demystify the Unicorn web server, showcasing its performance benefits and architectural features that adhere to Unix philosophies.

Key Points Discussed:

  • Introduction to Jesse Storimer: He introduces himself and shares his journey from Rails programming to diving into lower-level system programming with Ruby.
  • Unicorn Overview:
    • Unicorn is characterized as a Rack HTTP server designed for fast clients and operates through a pre-forking model.
    • Discussed features such as how Unicorn handles processes and connections.
  • Understanding Forking:
    • Storimer explains the concept of forking processes, emphasizing how fork in Ruby creates a new process which returns values in two different contexts.
    • He illustrates fork by showing how it can be executed in Ruby and its impact on process management.
  • Pre-Forking Mechanism:
    • Pre-forking is highlighted as a way to leverage operating system capabilities by preparing multiple workers to handle incoming connections without needing to reinitialize memory multiple times.
    • Through pseudo-code, the mechanism by which Unicorn boots and processes requests is described.
  • Memory Efficiency with Copy on Write:
    • Explained how copy-on-write optimizations allow for memory savings by sharing rather than duplicating data.
    • Mentioned that Ruby 1.8 and 1.9 do not utilize this feature effectively.
  • Zero Downtime Restart:
    • Explained how Unicorn can replace itself without losing connections during deployments, illustrating how to maintain service availability.
  • The Role of exec:
    • Introduced the exec system call, which allows replaced processes to maintain their listener sockets, enabling seamless updates to applications.
  • Concluding Thoughts on Unix Philosophy:
    • Storimer advises attendees to embrace the Unix philosophy of specialized tool usage and encourages exploring common system calls beyond web programming.
  • Final Challenges to the Audience:
    • Encourages programmers to explore beyond their usual scope, learn new things, and engage with the powerful features of Unicorn and Ruby.

Key Takeaways:

  • The importance of understanding how Unix systems programming can enrich Ruby applications.
  • A strong recommendation to explore Unicorn’s source code and its elegant design patterns that enhance performance for Ruby applications.

Grow Your Unix Beard Using Ruby
Jesse Storimer • Denver, Colorado • Talk

Date: November 01, 2012
Published: March 19, 2013
Announced: unknown

Ruby, like Perl before it, has great support for systems programming and interfaces to common system calls. Unfortunately, this doesn't get talked about much in the Ruby community. In an age where alternative runtimes and threads are getting lots of attention many of the largest Rails deployments are running on single-threaded, blocking I/O, preforking web server called Unicorn.

This talk will be a safari tour through the unique features and internals of Unicorn. By strongly following the Unix philosophy of 'doing one thing well' Unicorn has carved out a reputation as a high-performance, dependable choice for Ruby apps.

The Unicorn codebase is the best, most comprehensive example of Unixsystems programming in Ruby that I've come across. You'll see real-worldexamples of fork(2), accept(2), execve(2), and others. I'll walk throughhow Unicorn distributes connections across a shared socket, how it implements deferred signal handling, how it can replace a running instance of itself without losing any connections. Trust me, by the time it's over you'll begin to feel mysteriously compelled to start growing your Unix beard.

RubyConf 2012

00:00:15.200 this
00:00:15.519 is really experienced with ruby
00:00:19.119 i'm jessie stormer i've had a lot of
00:00:21.439 people tell me they don't recognize me
00:00:22.880 from this picture
00:00:24.720 this is my dirty picture just you can
00:00:26.640 put the person into the picture that's
00:00:41.760 some technical talk let to show you some
00:00:43.520 cool easy stuff
00:00:45.120 you can do with ruby it's often called
00:00:47.520 as the domain of like
00:00:48.960 c programming but you can do so many
00:00:56.840 things
00:00:58.960 so a few years ago in 2009 i was a rails
00:01:01.840 programmer
00:01:03.120 i had a grip on rails and ruby and i
00:01:06.320 came across this blog post of iron
00:01:08.080 tomato
00:01:09.600 unicorn a web server for him for me that
00:01:13.200 does automatically stuff
00:01:15.200 it's a great time for me because i was
00:01:18.320 ready to learn something new
00:01:19.520 and a lot about rails i knew nothing
00:01:21.280 about this lower level stuff
00:01:23.280 didn't have a full computer science
00:01:25.439 education
00:01:26.880 so ryan suggested checking out unicorn
00:01:30.159 link to the main source file which looks
00:01:32.320 like this
00:01:34.400 i opened it up and i was kind of like
00:01:36.720 well what did i do with
00:01:39.280 it's this long a lot of imitation a lot
00:01:41.680 of methods
00:01:42.799 but i read through it and i realized
00:01:44.320 there was a lot of methods i've never
00:01:45.759 seen before
00:01:46.960 there's a lot of stuff i didn't
00:01:48.000 understand
00:01:52.560 the more
00:01:55.759 it was even though it wasn't like a
00:01:57.759 normal ruby code i'd see
00:01:59.360 it was a certain confidence there was a
00:02:01.119 lot of elegance this guy was using stuff
00:02:03.520 that was obviously powerful but that i
00:02:06.159 hadn't seen
00:02:07.520 so if you're looking at roomy
00:02:09.200 documentation looking at manual pages
00:02:11.680 thinking some books i started to figure
00:02:13.200 out what unicorn does how it works
00:02:16.080 um i want to share some of that today
00:02:19.599 so we're talking about unicorn the
00:02:21.280 unicorn if you've never
00:02:22.640 heard of it is a web server for rackax
00:02:25.760 can i see your hands if you ever
00:02:27.360 deployed an app on unicorn
00:02:30.480 cool lots of you now hands if you've
00:02:33.040 ever opened the source code
00:02:44.640 i have studied and learned from the
00:02:47.040 source code but
00:02:48.560 i can't take credit for the projects for
00:02:51.840 the website unicorn is a rack htmp
00:02:54.400 server for fast clients and phoenix
00:02:56.480 for today we're ignore most of it and
00:02:59.280 just so i can teach you some stuff
00:03:01.040 talking about the way manages humans
00:03:02.840 processes
00:03:04.800 that's why it's what makes it special
00:03:07.200 the first feature i want to talk about
00:03:08.480 is called pre-forking
00:03:10.319 before we go there we should talk about
00:03:12.000 forking and making sure we understand
00:03:13.599 what happens
00:03:14.720 so when i say four points i can put a
00:03:16.640 system
00:03:24.319 the most important part work creates a
00:03:26.480 new process
00:03:27.840 so let's do it in the bottom half of the
00:03:30.400 screen i'm just showing you the list
00:03:32.000 of irb processes various very women's
00:03:35.040 selected lists
00:03:36.319 just matching the process ids then i'll
00:03:39.680 call fork and you see i get back a new
00:03:42.799 process id
00:03:43.840 now there's two ids another thing you'll
00:03:47.280 notice
00:03:47.760 is that i kind of have two return values
00:03:50.239 there
00:03:50.959 there's the one right after the fork
00:03:52.879 which is a number
00:03:54.080 and there's a nail that i got
00:03:55.200 highlighted so as far as i know
00:03:57.519 fork is the only ruby method that
00:03:59.760 returns twice
00:04:01.439 and notice it returns two values like an
00:04:03.920 array
00:04:04.480 needed returns two times the reason is
00:04:07.120 the fork
00:04:07.920 creating a process because the umbrella
00:04:10.159 processes now
00:04:11.360 they both return we both very standard
00:04:14.080 that's why you start to return that
00:04:20.799 so because twice any crazy things like
00:04:23.840 challenge your understanding of your
00:04:25.040 students so
00:04:27.040 say it for one thing else another thing
00:04:31.840 and if i remember this
00:04:37.680 she looks like it's true impulse
00:04:40.960 have to think this stuff is dangerous
00:04:42.400 but it's just a matter of understanding
00:04:44.560 what's really happening
00:04:46.080 so let's do it again it's not unique
00:04:50.800 in the true part of the false or open
00:04:53.919 process i need as well really see the
00:05:08.840 difference
00:05:12.560 so this time around i'll run it again
00:05:15.120 and as you can
00:05:16.000 guess you're going to see
00:05:19.600 the same output but both of those blocks
00:05:22.160 have different process ids
00:05:24.000 so it's kind of interesting that even
00:05:25.280 though they're in the same source file
00:05:27.360 they're right beside each other in code
00:05:29.600 they're running instead of processes on
00:05:31.120 the system
00:05:32.479 so i'm going to revise this statement
00:05:33.919 fork doesn't really return twice
00:05:36.000 returns once but in two processes so
00:05:39.039 where there is one
00:05:40.160 there are not two only parallel and
00:05:43.759 the if uh construct is kind of weird
00:05:46.720 it's usually how you do it and see
00:05:49.520 the person who have more elegance
00:05:51.520 solutions more elegant dsls
00:05:53.680 so the way you really use forklift block
00:05:56.800 and that block would get evaluated in
00:05:59.440 the
00:06:00.080 child process with the newly created
00:06:02.000 process you have no good attention
00:06:05.840 the last thing i want to show you about
00:06:07.120 before we continue is how it shares
00:06:09.440 memory i'm going to create an array 105
00:06:14.160 uh process in that process
00:06:28.319 so normal ruby scoping um
00:06:31.680 you know something happens add a block
00:06:33.520 to an instance variable
00:06:35.039 it would affect the same variable you
00:06:37.600 know in that instance
00:06:39.360 but in this case you run it
00:06:46.319 you see that the two have different
00:06:48.400 arrays
00:06:49.599 this is because when you work a process
00:06:52.639 it's something you're asking the
00:06:53.599 operating system to do so
00:06:55.919 when you fork the process it copies
00:06:57.440 everything in memory copies
00:06:59.520 open file descriptors
00:07:09.520 they've got separate copies so let's
00:07:13.199 uh add one more thing here four creates
00:07:15.360 an exact copy
00:07:17.520 we'll talk later about how it can
00:07:20.080 optimize this but
00:07:21.360 from your perspective it's an exact copy
00:07:24.319 this is the code we just saw
00:07:26.319 and you may think oh maybe this is a
00:07:28.400 trick of
00:07:29.440 instance of variables or something this
00:07:30.639 is kind of weird if you change this to
00:07:32.160 globals
00:07:32.960 it's the same effect because effectively
00:07:35.360 the code inside the block is running in
00:07:36.880 a totally separate process
00:07:38.720 cannot be affected by the other code
00:07:40.479 after the block
00:07:44.960 oh and process weight there it's just so
00:07:46.720 that the
00:07:48.000 original process waits for the child to
00:07:50.319 finish its work
00:07:51.440 make sure that we've deleted something
00:07:53.120 from the array
00:07:55.919 so we learned about forking it creates a
00:07:58.000 new process
00:07:59.199 the process is a copy um we can use that
00:08:02.560 to understand what preforking's all
00:08:04.080 about
00:08:05.440 and preforking is really cool it's
00:08:07.680 really cool because it leverages
00:08:09.440 what the operating system has to offer
00:08:10.800 it pushes as much as you can
00:08:12.639 onto the operating system and it's not a
00:08:15.840 new idea
00:08:17.440 the fork man page that i showed earlier
00:08:21.039 if you scroll to the bottom
00:08:22.319 on my mac the date is 1993
00:08:25.759 and i mean the fork api has really been
00:08:27.919 unchanged for
00:08:29.039 30 40 years it's a very old idea
00:08:32.399 apache in some configurations can do it
00:08:35.360 same with passenger
00:08:36.880 nginx and unicorn so unicorn didn't
00:08:40.080 invent this stuff
00:08:42.560 um i'm just going to give you a quick
00:08:45.760 like a pseudo code example of of how
00:08:48.399 unicorn boots up
00:08:52.240 so if you're to open the unicorn gem go
00:08:54.800 to the http server class
00:08:57.120 these are two methods you would see
00:08:58.160 among lots of others uh
00:09:00.399 the start method which gets called
00:09:03.279 that's basically the entry point
00:09:04.720 so what does it do it opens a listener
00:09:07.120 socket
00:09:08.399 and loads your your rack up or your
00:09:10.000 rails app then it
00:09:12.080 however many workers you want it forks
00:09:13.600 them and they enter a loop
00:09:16.160 now we've got two concurrent processes
00:09:18.640 running in this loop right
00:09:20.240 two workers each of them gets
00:09:22.640 connections off the socket
00:09:24.320 process client uh calls the rack app
00:09:27.760 so let's i'm going to explain this a bit
00:09:29.360 more detail i want to show you a diagram
00:09:32.320 so this is the master process it's got a
00:09:34.320 socket and the rack app in memory
00:09:37.839 it forks now it's got two workers ready
00:09:40.160 to handle your app
00:09:41.440 two parallel processes now
00:09:44.959 you might look at this and say okay what
00:09:48.160 it would have really got here you know
00:09:49.279 two processes
00:09:50.000 ready to handle the app why didn't we
00:09:52.160 just like with
00:09:53.279 mongrel or some other server you could
00:09:55.120 just boot up two at the command line
00:09:56.800 what's the real gain here if we're
00:09:58.080 having to copy everything over
00:10:01.040 but thanks to fork and thanks to
00:10:04.000 something called copy on write
00:10:05.600 fork can actually be faster and use
00:10:08.720 less memory than putting these things up
00:10:12.000 by hand and i mean two two processes not
00:10:15.279 exciting
00:10:16.079 at shopify we have uh 70 like 7-0
00:10:20.480 parallel processes per machine
00:10:26.959 so the savings is really magnified so
00:10:29.360 copy and write what is that well ruby
00:10:32.320 has a funny history with copyright right
00:10:34.000 but
00:10:34.959 it's a it's a kernel feature such that
00:10:37.360 when you fork a process
00:10:38.880 it doesn't actually copy all the memory
00:10:40.959 as long as it can
00:10:42.240 it lets them both reference the same
00:10:44.399 part of memory
00:10:46.079 and if one of them decides to change
00:10:47.839 that then it gets its own copy
00:10:49.920 but think about how much of your stuff
00:10:51.839 how much stuff in your rack app never
00:10:53.279 changes
00:10:54.160 when you load rails when you load gyms
00:10:57.519 when you load your your models business
00:10:59.519 logic you don't modify that stuff at
00:11:01.519 runtime
00:11:03.120 so the parent process should be able to
00:11:04.640 load that fork the workers
00:11:07.519 they can share that memory should never
00:11:10.000 need to be copied
00:11:11.040 so you could save a ton of memory that
00:11:12.320 way but
00:11:14.079 if you're using mri 1.8 or 1.9 which
00:11:17.360 probably a lot of us are
00:11:19.680 it's not copy and write friendly so it
00:11:21.760 actually
00:11:22.720 kind of like disables this feature and
00:11:24.880 it's thanks to the garbage collector
00:11:27.040 the garbage collector you may have heard
00:11:29.040 uses a mark and sweep strategy
00:11:31.360 what that means is it looks at all the
00:11:33.120 objects in memory
00:11:34.640 and marks bits on them saying which ones
00:11:36.800 need to be
00:11:37.680 swept so it's it's effectively writing
00:11:40.880 all over the place
00:11:42.240 so when you fork a process
00:11:45.360 copyright right saves you the memory as
00:11:47.760 soon as you're in the garbage collector
00:11:49.760 all the savings are effectively nil
00:11:56.720 so if you're doing using unicorn if
00:11:59.120 you're doing a lot of forking and you
00:12:00.240 want to
00:12:01.360 uh you know make best use of resources
00:12:03.680 mri 2.0 has solved this ruby enterprise
00:12:05.839 edition
00:12:06.639 that's one reason why it exists in the
00:12:08.480 first place the bottom one is a
00:12:12.320 version of mri with some 2.0 stuff
00:12:14.880 back-ported like the
00:12:16.240 copyion right friendly garbage collector
00:12:18.800 and the way that these guys get around
00:12:20.079 it
00:12:20.639 they still use mark and sweep but
00:12:22.320 instead of writing all over the place
00:12:24.639 they store all the little uh the bits in
00:12:27.040 one little part of memory
00:12:28.320 so you can end up preserving all that to
00:12:31.279 all that shared memory
00:12:33.920 so back to this example i said
00:12:35.519 pre-forking was awesome why is it
00:12:37.200 awesome
00:12:38.560 uh well because if you want to spin up a
00:12:40.639 bunch of parallel workers for your app
00:12:42.639 you can save a ton of memory and do it a
00:12:44.560 lot quicker than
00:12:45.920 spawning processes yourself like when i
00:12:48.399 say yourself i mean like
00:12:50.399 trying to do it 70 times from the
00:12:51.839 command line everyone's competing for
00:12:53.680 resources
00:12:54.720 and this way the master loads the app
00:12:56.399 once
00:12:57.839 the fork is pretty quick all the workers
00:12:59.920 can uh
00:13:01.200 share the same piece of memory and
00:13:03.440 theoretically you should get
00:13:04.720 tons of savings that's one reason why
00:13:07.360 it's awesome
00:13:08.079 the other reason is i was talking about
00:13:10.959 memory
00:13:11.680 but i also mentioned that in a fork
00:13:15.440 open files are shared sockets are shared
00:13:19.440 so
00:13:23.680 the master opens a listener socket and
00:13:25.519 it doesn't take any connections
00:13:27.600 and each of the workers as part of its
00:13:29.519 loop they accept connections
00:13:32.240 and so you this is a the accept system
00:13:34.720 call that they use
00:13:37.519 and the kernel does does the job of
00:13:39.680 making sure that
00:13:41.440 only each connection only goes to one
00:13:43.519 worker and things are load balanced
00:13:45.360 nicely things are cued fairly
00:13:47.199 you don't need any kind of a proxy in
00:13:49.440 front of unicorn to
00:13:50.720 talk to each worker it pushes all that
00:13:52.800 down to the kernel and the kernel
00:13:54.079 handles it nicely
00:14:00.240 so yeah pre-forking is awesome because
00:14:02.160 you can get faster boot time
00:14:04.639 uh you can save more memory get great
00:14:08.240 connection load balancing features from
00:14:09.760 the kernel
00:14:12.399 let's do another unicorn feature you can
00:14:15.360 replace an instance of itself
00:14:17.199 without losing any connections this is
00:14:18.639 like a zero downtime restart
00:14:21.839 the most obvious example of when you
00:14:23.120 want to do this
00:14:24.959 is if you want to you know deploy a new
00:14:27.279 version of your app
00:14:28.320 you write some new code you ready to put
00:14:30.480 a new app online
00:14:32.560 so this uh gets triggered by
00:14:36.399 you send a like if you wanted to deploy
00:14:38.959 a new code
00:14:40.880 you would put the code in place you
00:14:43.040 would send a user to
00:14:44.560 a signal to the master which would tell
00:14:47.519 it
00:14:48.320 uh here we'll look at the code send user
00:14:50.480 two signal which would trigger
00:14:51.680 this method again this is pseudo code
00:14:53.920 but if you look uh
00:14:55.120 inside the unicorn http server class
00:14:57.040 you'll see a method called
00:14:58.480 re-exec and
00:15:01.839 you see the the two components here are
00:15:03.839 a fork and an exec
00:15:06.079 we haven't touched on exec yet it's
00:15:08.000 another system call
00:15:09.760 straight from the man page exec
00:15:12.320 transforms the
00:15:13.680 calling process into a new process
00:15:20.720 let's do it in ruby
00:15:24.079 so i print hi i uh
00:15:27.760 executive command print by
00:15:37.279 so when i run this i'm going to see the
00:15:39.440 ls output scroll by
00:15:41.279 if i go back i can see see it says hi
00:15:45.440 but it doesn't say buy and the reason is
00:15:48.800 because
00:15:49.839 what that exact did it printed hi then
00:15:52.480 it called exec
00:15:53.360 which effectively blew away the ruby
00:15:55.120 process it replaced it with ls
00:15:57.920 so ls did its thing and it exited
00:16:01.279 and that was it ruby is effectively gone
00:16:03.360 in that play in that case
00:16:05.839 replaced so how does unicorn replace
00:16:08.959 itself
00:16:10.399 well the what's happening here is
00:16:14.079 forks a new process off the master it
00:16:16.639 grabs the
00:16:18.399 arguments it was called with this is a
00:16:22.160 list of file descriptor numbers that its
00:16:25.040 listening sockets
00:16:26.000 are listening on and then it calls exec
00:16:28.959 the list the listener
00:16:30.240 socket file descriptor numbers is
00:16:31.600 important because just like you can
00:16:33.199 share a socket through fork
00:16:34.800 you can share socket through exec
00:16:38.240 so this is this is like at the
00:16:41.839 the point with the most processes this
00:16:43.680 is what you'll see
00:16:45.199 the old master has forked on the left a
00:16:48.079 new master
00:16:49.519 that new master has called exec on
00:16:51.360 itself with unicorn so we're
00:16:53.519 just going to load up a new process load
00:16:55.759 up the rack app
00:16:56.639 as before this time instead of opening a
00:16:58.639 listener socket
00:17:00.000 it's just going to reopen the one that
00:17:01.759 got passed from the old master
00:17:06.880 so from the outside world the socket
00:17:09.520 never closes
00:17:11.120 the inside world there's only one socket
00:17:12.640 too and if you ask the kernel there's
00:17:14.079 only one socket
00:17:15.280 in this diagram there are six handles on
00:17:17.839 that socket
00:17:18.799 they can all work with it but there's
00:17:21.280 still just one
00:17:22.880 so when the new when the new master and
00:17:25.120 its workers
00:17:26.559 are ready you kill the old one and you
00:17:29.679 end up with
00:17:30.640 the same kind of process tree as before
00:17:33.440 and this way when you deploy new stuff
00:17:35.760 the socket gets passed down the tree the
00:17:38.400 tree grows and drinks
00:17:39.840 but there's never any effect on the
00:17:41.600 inside world
00:17:46.720 so this is the url for unicorn
00:17:50.640 you should definitely check it out
00:17:51.840 besides just what i talked about there's
00:17:53.760 lots of other cool features
00:17:55.679 i talked about pre-forking and it's how
00:17:57.840 it does restarts
00:17:59.360 if you want to look around at it and see
00:18:01.520 what other kind of stuff it does the
00:18:02.880 self-pipe trick is pretty cool
00:18:05.840 really there's lots of cool stuff in
00:18:07.039 there so
00:18:09.919 what did we actually talk about
00:18:13.679 we talked about some unicorn features
00:18:15.919 but
00:18:18.799 the concepts really built around fork
00:18:21.520 and exec
00:18:22.559 so if you only take away one thing today
00:18:24.720 let it be this statement
00:18:25.919 fork and exec because fork and exec
00:18:29.039 you know fork copied the current process
00:18:31.200 exact transform it
00:18:32.720 it's at the heart of process spawning in
00:18:34.880 unix everywhere
00:18:36.559 all these methods in ruby process spawn
00:18:38.720 system
00:18:39.679 backticks p open if you look at the
00:18:42.160 source
00:18:42.799 they all do fork and exec so there's no
00:18:47.440 there's no magic to them that's that's
00:18:49.919 their
00:18:51.120 that's how they do it this is process
00:18:53.520 spawn from rabinius
00:18:55.520 among other things it does fork an exec
00:18:58.640 just as one example and forking exec
00:19:02.559 isn't just for ruby uh it's for
00:19:05.760 lots of programming languages and even
00:19:07.280 for your shell when you do in your shell
00:19:10.480 curl google and send the output to a
00:19:13.120 file
00:19:14.400 if you're implementing that shell in
00:19:15.760 ruby you might do something like this
00:19:21.360 you fork the shell you set up the
00:19:23.280 environment that the command needs
00:19:24.799 in this case you redirect standard out
00:19:26.880 to a file
00:19:28.160 you might also set an environment
00:19:29.919 variable you might
00:19:31.919 you know point standard out to a pipe
00:19:33.520 instead and then you exact the command
00:19:37.280 and typically the shell waits makes you
00:19:40.000 wait till the command is done to do the
00:19:41.280 next one
00:19:42.720 so fork and exec is very uh
00:19:47.039 julia at the heart of unix
00:19:50.160 so i said i was talking about these guys
00:19:51.360 too again on the left is
00:19:54.240 ken the guy on the right is dennis ken
00:19:56.960 is the
00:19:57.520 original author of unix dennis is the
00:20:00.160 original author of c
00:20:01.440 about 40 years ago
00:20:04.720 yeah 40 years ago these guys obviously
00:20:07.440 collaborated a lot
00:20:08.400 and given their history they have credit
00:20:09.840 for the original unix beards
00:20:12.799 now the experience are kind of a funny
00:20:14.799 thing
00:20:16.080 sometimes like in the case of these guys
00:20:18.320 you might think it's
00:20:19.200 denotes wisdom respect sometimes you see
00:20:22.320 a beard like that and think that the guy
00:20:23.760 spends too much time in his parents
00:20:25.039 basement
00:20:26.720 i never really gave it too much thought
00:20:27.919 until i read this a few months ago
00:20:30.480 the unix beard is really an extension of
00:20:32.320 the philosopher's beard and the
00:20:33.520 academics beard
00:20:35.280 and i thought that was so on point to
00:20:37.760 how i see them
00:20:38.880 when i read stuff that they wrote years
00:20:40.640 ago you know they weren't
00:20:43.520 looking to change the world or to make a
00:20:46.400 lot of money
00:20:47.120 they were doing research they were
00:20:48.240 looking to find about find truths about
00:20:50.960 programming about
00:20:56.320 about operating systems about language
00:20:58.480 design
00:21:00.880 so i have a ton of respect for these
00:21:02.240 guys but
00:21:04.000 uh not everyone has to grow a beard
00:21:08.000 the guy on the right is linda torvalds
00:21:10.159 obviously a unix hacker and no beard
00:21:13.039 so i certainly wouldn't want to
00:21:15.039 encourage everyone to go grow a beard
00:21:16.960 if you feel like doing it that's great
00:21:18.799 but you can have a figurative one too
00:21:22.240 um so i'm gonna leave off with a quick
00:21:25.120 story
00:21:26.559 i've got a three-year-old daughter i
00:21:28.400 like to read this book it's a
00:21:30.240 dr seuss book it's about a young guy
00:21:32.720 he's learning the alphabet a to z
00:21:35.200 he uh he learns all the letters a is for
00:21:37.840 apple
00:21:38.559 b is for bears he is for zebra and an
00:21:41.600 older kid comes along and says oh
00:21:42.960 you don't know much let me show you
00:21:44.400 something else and it shows them these
00:21:46.000 crazy dr seuss letters and dr seuss
00:21:48.320 words
00:21:49.280 but in the end the younger kid discovers
00:21:51.039 some stuff the older guy didn't know
00:21:53.520 so then my challenge to you is if you've
00:21:56.320 learned something new today or if you've
00:21:57.520 seen something
00:21:58.000 you haven't seen before if you spend
00:21:59.840 most of your days doing web programming
00:22:02.480 try and find out something you haven't
00:22:04.400 found before looking unicorn is a good
00:22:06.400 place to start
00:22:07.679 and help me learn some more stuff i
00:22:09.600 don't know
00:22:11.120 so i'm a bit early i think but um that's
00:22:14.320 the end
00:22:15.039 if you are interested in my books you
00:22:16.400 can get a discount code there i've got a
00:22:18.640 few
00:22:19.200 copies to give away up here if anybody
00:22:20.720 wants one i think i've got time for
00:22:22.640 questions
00:22:24.840 thanks
00:22:52.840 um
00:22:58.799 to be honest the only thing we've ever
00:22:59.919 seen about it is that there was a commit
00:23:01.520 saying
00:23:02.480 let's do it like engine x does so i
00:23:04.720 guess you'd have to ask
00:23:06.080 someone who knows more about nginx
00:23:09.679 i saw another hand the back the question
00:23:12.320 was why is it necessary to fork a
00:23:13.919 process
00:23:15.039 in order to exact a different one uh are
00:23:17.919 you asking why there isn't
00:23:19.200 uh some something that puts those two
00:23:21.600 together
00:23:22.240 so the question was forks seem to do two
00:23:24.320 things creates a new process
00:23:25.760 and inherits the environment what can
00:23:27.840 you do if you don't need the environment
00:23:29.600 so i'll have a two-part answer the first
00:23:31.200 is there is something called
00:23:32.720 posix spawn which gets you what you want
00:23:35.840 you posix spawn give it a command and it
00:23:39.200 creates a new process that doesn't
00:23:41.039 inherit the environment of the old
00:23:42.559 process which makes it much quicker
00:23:45.039 and spawns the new process it's
00:23:47.440 something that
00:23:48.240 is not supported everywhere it's not
00:23:50.320 supported in core ruby
00:23:51.760 there's a gem for it posix spawn i think
00:23:54.320 it's called
00:23:55.520 and the other reason i'd say is i think
00:23:57.520 a lot of this stuff like i said fork is
00:23:59.200 a very old
00:24:00.080 concept and if you look at a lot of
00:24:02.159 system calls
00:24:03.120 when i look at them i see people who
00:24:05.200 were like
00:24:06.240 basically designing the system they
00:24:07.440 needed to write shells
00:24:09.440 and to write programs i needed a long
00:24:11.120 time ago and that stuff is still useful
00:24:13.360 today so it still sticks around and it's
00:24:14.960 the most
00:24:15.440 widely available
00:24:19.120 but it is written for times past
00:24:24.559 yes if you're running 70 processes do
00:24:27.840 you
00:24:28.240 do you all use ree or like hat or
00:24:31.279 you just have like 11 billion rams
00:24:34.320 both we used we used to use the ree
00:24:37.600 we switched to uh 193
00:24:40.720 like stock 193 and we only saw
00:24:43.919 like the new vm saves so much memory
00:24:47.039 that we only saw a little bit of a loss
00:24:48.880 but we're now using that patched version
00:24:50.720 in production which has
00:24:52.640 different speed patches as well as the
00:24:54.559 copy on right patch
00:24:55.919 and it's been wonderful
00:24:59.440 yeah uh what about the other you
00:25:02.240 mentioned
00:25:02.640 a couple versions of mri what about the
00:25:04.240 other implementations
00:25:06.400 actually is there anybody here who knows
00:25:07.840 if rabinius is copying right friendly
00:25:11.120 i couldn't find out um the jvm doesn't
00:25:13.919 support fork
00:25:15.440 so you know in its efforts to be
00:25:18.240 cross-platform compatible like on
00:25:19.600 windows and stuff
00:25:20.559 windows doesn't support fork jb doesn't
00:25:22.799 support fork
00:25:24.320 um rubinius does but i don't know if
00:25:26.320 it's copying right friendly
00:25:31.760 okay okay
00:25:36.640 yes back there
00:25:43.600 is the question was is the backwards is
00:25:45.600 the back port of stuff from 2.0
00:25:47.279 uh stable we're using it in production
00:25:49.840 at shopify i use it for development
00:25:51.600 every day
00:25:52.640 it's been stable for me
00:26:05.360 yes so just saying that jb
00:26:09.200 is heavily uh encourages using threads
00:26:11.440 instead of processes
00:26:13.360 yeah so the question was isn't that a
00:26:14.720 better model for what needs to be done
00:26:16.640 it really depends on your needs right
00:26:18.559 jrb is great with threads because the
00:26:20.159 jvm is great with threads
00:26:21.760 mri for instance if you're doing
00:26:23.440 threading thanks to the
00:26:25.440 the global vm lock uh threads aren't
00:26:28.000 really happening in parallel
00:26:29.760 multiple processes you do get
00:26:31.360 parallelization and you do get
00:26:33.600 uh you know shared memory which is big
00:26:36.080 savings
00:26:37.039 so i i can't say that one is better than
00:26:39.360 the other it depends on
00:26:41.200 what you're deployed on depends on what
00:26:42.960 your needs are of course the best thing
00:26:44.640 to do is to test and
00:26:46.480 and use metrics
00:27:02.080 great point which is that if you're
00:27:03.600 using fork you don't have to worry about
00:27:05.840 shared state with threads or locking or
00:27:07.520 synchronization
00:27:08.640 it's a much simpler approach
00:27:15.600 favorite system call hmm
00:27:21.120 uh i'd have to say fork just because
00:27:23.279 it's so useful
00:27:24.399 i mean it's been said that it may not do
00:27:26.720 what you want every time but
00:27:28.320 i find it so useful in so many places i
00:27:30.960 know there's some funny names but i
00:27:32.240 can't think of any right now
00:27:36.240 all right anybody else yeah do you find
00:27:39.279 yourself
00:27:39.679 sending signals randomly on your head
00:27:41.440 right now like you did before you
00:27:43.120 learned about
00:27:45.600 i find myself sending signals randomly
00:27:47.679 on the commune line
00:27:49.039 before i knew like now that i know about
00:27:51.360 signals um
00:27:53.760 maybe not randomly but i know how to you
00:27:55.919 know i know what kill dash 9 means now
00:28:01.279 all right who else yeah
00:28:16.799 so the question was do you find yourself
00:28:18.159 using any ipc
00:28:20.000 mechanisms besides sockets or pipes i
00:28:22.480 assume
00:28:23.120 stuff like shared memory or maybe posix
00:28:25.760 cues
00:28:26.640 um i never have in ruby i think there
00:28:30.720 might be gems for some of that stuff but
00:28:32.240 not in ruby core
00:28:33.600 so like if you want to communicate
00:28:35.440 between processes you can use
00:28:37.039 pipes or sockets but if you want to do
00:28:38.320 memory mapping or
00:28:40.880 shared memory i'm not sure how to do it
00:28:42.880 in ruby i never have
00:28:58.320 yeah so when they all call accept that's
00:29:01.440 a blocking call so they're all waiting
00:29:03.520 and i presume that you know first one in
00:29:05.440 gets first in the queue
00:29:06.640 next one next in the queue but i i don't
00:29:08.880 know the kernel source that well
00:29:13.279 who else
00:29:17.200 that's it okay thanks everyone
00:29:51.919 you
Explore all talks recorded at RubyConf 2012
+46