Search
RubyConf 2011
Blueprint: configuration management for busy people
Edit
#configuration-management
#devops
#python
#server-management
#development-environment
Summarized using AI

Blueprint: configuration management for busy people

Richard Crowley • September 29, 2011 • New Orleans, Louisiana • Talk

In this talk titled "Blueprint: Configuration Management for Busy People," Richard Crowley introduces Blueprint, a simple and effective configuration management tool designed to streamline application deployment and system configuration for developers. Unlike complex tools like Puppet and Chef, Blueprint focuses on improving the development workflow by allowing teams of any skill level to manage configurations with ease.

Key Points Discussed:

- Purpose of Blueprint:

- Blueprint aims to democratize configuration management, making it accessible for teams regardless of size or skill level, thus fostering confidence during deployment and stability while scaling.

- How Blueprint Works:

- It automates the reverse engineering of server configurations and stores them locally in a JSON format within a Git repository, making it easy to share configurations across team members.

- Users define dependencies within familiar interactive environments without the cumbersome syntax required by other tools.

- Item Potency:

- Blueprint ensures that configurations are item potent, meaning commands can be run multiple times without adverse effects. This characteristic is crucial for maintaining system stability and predictability during deployment.

- Philosophical Overview:

- Crowley emphasizes the importance of realistic development environments, advocating for consistency between development and production platforms to mitigate unforeseen issues.

- The tool is designed to fit into existing workflows while maintaining standard Unix conventions, thus reducing surprises and enhancing predictability.

- Interactive Features:

- Blueprint supports detailed visibility into configuration files, packages, and services, allowing developers to inspect and modify blueprints easily.

- The tool includes an innovative 'ignore' feature, similar to Git, which streamlines the process of managing dependencies and configurations.

- Cloud Integration:

- The service supports integration with AWS EC2, providing a means to bootstrap new servers automatically based on defined blueprints, thus streamlining cloud deployments.

- Collaboration between Developers and Operators:

- Crowley highlights the importance of a collaborative approach in configuration management, enabling developers to contribute effectively while respecting operational perspectives.

- Future Direction:

- Potential improvements for Blueprint were discussed, such as modularity to enhance usability across diverse configurations and implementation of additional resource types.

Conclusions & Takeaways:

- Blueprint is a promising configuration management tool that bridges the gap between development and operations, significantly enhancing productivity and deployment confidence. Organizations looking to simplify their deployment process would benefit from exploring Blueprint as a flexible and collaborative option for configuration management.

Blueprint: configuration management for busy people
Richard Crowley • New Orleans, Louisiana • Talk

Date: September 29, 2011
Published: December 12, 2011
Announced: unknown

Configuration management shouldn't be a luxury. It should be a right available to teams of every size and skill because it makes you confident when you deploy and calm when you scale up. Until now, however, the tools have been focused on the exotic possibilities seen at enormous scale in sprawling organizations. Blueprint is different. Blueprint reverse engineers servers to understand how they're configured and makes it painless to push the configuration to your coworkers and to production. It understands system packages, gems, config files, source installations, and more. In this talk we'll discuss the architecture of the Blueprint tools, their use and how they fit into most workflows, their limitations and how users address them, and some of the tools being built on top of Blueprint to manage many servers at once.

RubyConf 2011

00:00:13.700 thank you so thanks for braving the escalators
00:00:19.199 coming all the way down here I'm going to talk about blueprint a configuration management software that
00:00:25.619 in many ways is analogous to tools like puppet and chef but I think the focus
00:00:30.779 that I've given it on a good development workflow over operational abstractions
00:00:36.239 is going to be tempting to a lot of you so hi my name is Richard Crowley I work for
00:00:41.760 Dev structure which is the company that produces and and works on blueprint as well as is building some cloud services
00:00:49.020 and other non-free things surrounding blueprint
00:00:55.199 the tool itself is um is written in Python and hold your booze there's no
00:01:00.960 reason to write any python to use blueprint it's implemented in Python purely because that language is
00:01:07.680 available on basically every Linux system and that makes it really easy it's BSD licensed and it runs on Debian
00:01:14.400 and RPM based Linux distributions which is to say that it doesn't really do anything useful on Mac OS X and there's
00:01:21.659 a very good reason for that which we'll get to in just a couple of slides so the point of blueprint the the
00:01:28.200 workflow that it allows you to get into is is that you work on your application and add dependencies and configure your
00:01:34.680 development environment as you please and blueprint's job is to come around and figure out what you did to the
00:01:40.259 server store it locally in a Json format in a local git repository generate code from
00:01:47.159 it so that you can send it to your your neighbor over at the next desk and be able to deploy from it
00:01:54.240 and the uh and the origins of this tool are uh kind of entwined with what now is
00:02:00.780 known as the devops movement so I wanted to start by talking through some of the philosophies that made blueprint the way
00:02:07.560 it is today and the first of those is I believe in realistic development environments and
00:02:14.400 for everyone here I'm willing to bet that means Linux because your production environment is
00:02:20.099 going to be running on Linux and and in addition to using that same operating system in development I think it's
00:02:26.400 important to use the same sorts of web servers and database servers that you use in production
00:02:32.099 there's there's no other way to get a good feeling for the robustness and the
00:02:37.319 the quirky failure scenarios that you're going to come across in production if you're not exercising the same stack in
00:02:43.140 development every single day following closely from that I think it's
00:02:49.260 great to be able to use interactive tools like apt-get or gem install to manipulate how your development
00:02:55.739 environment is configured as a counter example to how easy and great that is think about if you were
00:03:03.120 using puppet or Chef where there's a configuration language off to the side so to add a dependency that you're going
00:03:09.420 to need immediately while you're developing your app you have to have to switch to a different code base and find
00:03:15.180 the correct place to put your your new puppet resource and then test it and check the syntax and then run it and
00:03:21.360 check it in and then you can come back to an application code base where now you can use that dependency you just
00:03:27.720 added so the the context switching overhead is just way too high and I think that me makes a lot of developers
00:03:34.560 hostile towards those tools right from the get-go
00:03:39.980 now configuration management in general this is nothing special to blueprint preaches the notion of managing the
00:03:46.260 entire Stacks configuration and so this means not just your uh your ruby gems and the particular versions of those but
00:03:54.000 also the system packages that they depend on and other other tools that you need this means the configuration files
00:04:01.200 for those services like you'd store and Etsy this means software that you've installed from Source this means the
00:04:07.500 databases and everything and a tool like rubygems for all of its strengths is not by itself capable of
00:04:15.659 handling this um ruby gems and and not to pick on it every language level package manager has
00:04:23.040 chosen a wide compatibility for example with Windows but also with both Debian
00:04:29.520 and RPM based Linux and Mac OS X and everything else so we need a kind of meta dependency declarations to
00:04:36.600 understand the relationship between a set of ruby gems and the underlying system packages that make them work
00:04:43.080 this is where blueprint comes in another thing that uh basically every
00:04:49.320 config management nerd would tell you is that it's important for everything to be item potent and this is this frees you
00:04:55.259 up to run the command over and over again no uh no one's scared to run puppet or Chef twice in a row and expect
00:05:01.860 anything bad to happen and likewise you can run blueprint over and over again without fear that anything bad is going
00:05:08.400 to happen and the reason for this and the the important characteristic is that
00:05:13.500 when we're dealing with commands like installing packages we're dealing with
00:05:19.199 commands that are already item potent that if if things are already in the correct State nothing's going to change
00:05:25.500 when we're dealing with placing files on the file system we're checking and making sure we're placing the entire
00:05:31.380 file's contents and never diffing or appending lines because the first thing that happens when you start applying
00:05:38.100 patches potentially over and over again or appending lines is that you end up appending the same line to your config
00:05:44.340 file but jillions of times you fill up your disk everything's broken nobody's happy so having a notion of item potence means
00:05:52.020 that when you run the config over and over again you're always going to arrive at the same desired state so when you
00:05:59.220 run over and over again you can bring nodes into compliance which is the the buzzword that a lot of Enterprise config
00:06:04.740 management people use the point here is that you won't have the fear at three in the morning when
00:06:10.259 you try to run your blueprint over again to make sure everything's correct
00:06:16.139 now as we've developed blueprint we've come across a lot of decisions where
00:06:21.419 we could do things um in a bit more invasive or custom or
00:06:26.819 unique way or we could do things the Unix way the the Debian way or the red hat way and we try to always choose
00:06:34.919 doing things the most standard and conventional way to to become the least surprising tool that we possibly can be
00:06:42.600 so when when faced with where we look for configuration files for example we
00:06:48.180 limit our search to Etsy because that's where config files belong on Unix so
00:06:53.400 this means that the tool is a lot simpler to understand it's a lot simpler to predict what it's going to do and
00:06:59.039 this means that you can use it without the without fear that it's going to do something crazy
00:07:06.120 um with uh with apologies to a guy named Ryan tomaco from GitHub
00:07:11.400 blueprint means never having to say you're sorry and he he said this about git but I think the same sort of thing applies
00:07:17.520 there's never a case where installing blueprint a month ago instead of five
00:07:23.039 minutes ago will make any difference so there's never a case where I have to say to someone you should have installed
00:07:28.080 blueprint last month before you cowboyed the hell out of that server because the way blueprint works is it's
00:07:34.259 taking an absolute look at the state of the system not a diff between what it is now and what it was last month so no
00:07:41.819 matter what you've done and no matter what order you've done it in we can find out exactly how a server works and
00:07:47.280 reduce it to kind of the minimal State that's necessary to bring it back on another server to do the same thing
00:07:55.919 um so now for a short bit of history when blueprint and really Dev structure the
00:08:02.099 company was first created we were building a development environment that we intended to sell as a service and uh
00:08:09.240 this was a bad terrible idea and the worst of it was that the the data that
00:08:14.759 blueprint was pulling out of systems was being stored in a relational database which seemed awesome at the time but it
00:08:20.879 was actually terrible now all of the data that blueprint extracts from a server is stored in a
00:08:27.419 Json format that is well documented and easy to generate and consume and when
00:08:34.200 necessary we use tarballs to store binary data and the like so no more
00:08:39.779 relational databases no more secret cloud services that uh that are holding onto your data without any ability to
00:08:46.380 get it out it's all very simple and it's all stored locally and of course it's documented and I
00:08:52.860 believe that Unix Man pages are the greatest documentation ever so everything has a man page the commands
00:08:59.940 and this documents what options they take and what they mean but also that file format I was talking about and if
00:09:06.839 the uh if Chrome hadn't crashed on me I'd show you the man pages but imagine and go to the internet and see them and
00:09:14.100 what you'll find in the description of the Json format is that it also describes the algorithm that you should
00:09:20.580 use to apply it so that if you're working from the same blueprint to people's implementation of applying that
00:09:27.120 to a server should work out exactly the same way again it gives you the confidence to do it over and over again
00:09:33.420 without worrying about bad things happening and now the very last bit of philosophy
00:09:39.899 is that I believe uh in a crowded Market that I'm admittedly kind of a late Comer
00:09:45.120 to like config management it's important for tools to interoperate well and so in
00:09:50.519 a bit we'll see how blueprint plays nicely with both puppet and Chef and kind of uh a different sort of
00:09:57.540 interoperability is I believe that the tools we use should take full advantage of the production platforms on which
00:10:04.440 they're deployed so ec2 being kind of the Cadillac of that world right now will show how blueprint integrates
00:10:11.160 tightly with ec2 so philosophy done
00:10:17.300 now we can talk about the workflow that blueprint promotes which I think is the coolest part of it and I think I've said
00:10:23.220 that at least twice now already so I'd like to walk through how blueprint Works how you would use it and
00:10:30.480 what it does behind the scenes while you're using it so since we're now all totally believing My
00:10:37.740 Philosophy we're all starting with a Linux VM in vagrant or virtualbox or in the cloud or in the kitchen and it
00:10:45.120 doesn't matter where it comes from or what flavor it is the important thing is that you have root access and that you
00:10:51.180 use it and so this means that when I get a new VM the first thing I do is install blueprint this doesn't matter if you can
00:10:57.959 read it it sets up a Debian archive and it installs blueprint that's the first thing I do and it's
00:11:03.240 installed system-wide and it's going to be able to appear in a blueprint later in a weird kind of self-hosting way
00:11:09.899 next thing you might want to do with your root access is install some gems I support this system-wide installation
00:11:16.560 means that you have a consistent end and understandable representation of what it
00:11:23.399 is to be that type of system less awesome is if you use uh use a tool
00:11:29.220 like bundler for example to pick on one to install gems often their own directory someplace that's not well
00:11:35.579 known to the system and can't be introspected reliably so given that bundlers basically a
00:11:41.459 requirement for rails install things system wide and then you've got a complete picture of your dependencies in
00:11:48.360 a in a discoverable location now likewise with gems as Ruby itself
00:11:54.980 tools like rvm and rbn have gained a lot of popularity I think mostly because of
00:12:01.380 OS X but once you get to the place where you're developing in a Linux VM there's
00:12:06.839 I don't think any reason not to install Ruby system-wide and and the version of
00:12:11.940 Ruby that you actually want and I'd encourage you to look into a tool yeah
00:12:18.600 very
00:12:28.019 that's he said that some distributions use Ruby for their own means and you
00:12:33.959 might mess that up which is very true and uh it's not so much the location of
00:12:39.600 the files so it's not that it be user bin Ruby it's that it be in the APT or
00:12:44.940 yum database for you to to audit and upgrade and remove that reliably and uh
00:12:50.100 and so here's Josh Timberman from Ops Code uh wrote a little quick thing that
00:12:55.320 uses fpm Jordan's to sells package management tool to uh to build a Debian
00:13:01.019 package for Ruby 192 whatever and this is like I said auditable
00:13:06.420 upgradable downgradable uninstallable and and if you use uh if you run your
00:13:12.779 own Debian archive then you can do this for much more than just Ruby itself you can do this for any particular system
00:13:18.240 dependency you have that you need to deploy outside of the confines of of whatever Debian or or red hat gives you
00:13:27.360 so now that you've got a bunch of stuff on your server and uh and it's all installed system wide we can create a
00:13:33.480 blueprint and so you can imagine you might run blueprint create and give it a name we'll call it Foo this is what
00:13:40.139 kicks off the reverse engineering of the server this is where the magic happens I suppose and uh and it goes through a lot
00:13:48.000 of back ends figuring out what to do and prints a little status along the way so
00:13:53.279 let's talk about what it does at each of these steps so the first thing it looks at are
00:13:58.740 packages in a whole bunch of different package managers the system package managers like apt and yum will in
00:14:06.480 addition to grabbing the full list of all the packages installed and their version numbers
00:14:11.519 it will take a take a hard look at those systems and figure out a representation of the base system the packages that can
00:14:19.620 be ignored because without them you don't even have a functioning Linux server so packages marked essential in Debian
00:14:26.579 and packages descending from the the Yum group called core in Red Hat World uh
00:14:33.420 and there's some others too are taken out of the blueprint so that they don't clutter things up we're reducing noise
00:14:39.860 and at the same time kind of increasing the strength of uh of the statement by
00:14:44.940 that including a package in the blueprint gives if we list every package then it's just these are the packages on
00:14:50.880 the system but if we start to remove stuff that's considered common or or trivial then the blueprint becomes a
00:14:58.980 representation of what's special about your system that means that it's smaller and easier to read and more personal to
00:15:04.199 how your environment works so all these other package managers work roughly the same way grabbing names and
00:15:10.260 versions and storing them in the the Json format for later
00:15:15.540 the next sort of resource that we look for are files unsurprisingly their
00:15:21.360 configuration files for the various packages and services that you've got and by convention those are stored in
00:15:27.720 Etsy so we only look in Etsy and this simplifies the process for blueprint and
00:15:34.019 causes it to be a lot more reliable in picking the correct files again we try to ignore a lot of things
00:15:41.040 like files that are included in a package and haven't been changed from that package as well as a lot of other
00:15:46.680 heuristics about files that are placed by the post install script from a from a
00:15:52.560 package which is a terrible thing don't do that and we have some other dirtier heuristics with c times and uh and some
00:16:01.139 packages in Debian that don't actually include checksums for their files so we do a lot of work to reduce the number of
00:16:06.959 files we include down to just the bare minimum things that are different and special about your system
00:16:15.240 so now another form of resource that we track we call Source installs which is admittedly a little bit of an ambiguous
00:16:21.360 and terrible name but the this feature was invented to handle and allow people to configure
00:16:28.320 make make install on their systems and have that still become part of the blueprint so
00:16:34.560 by convention again those things go into user local and so we only look in user local by default and and this gives you
00:16:41.880 a place to install binary files or custom scripts of your own or even your entire application and and whatever you
00:16:48.660 put in user local we're going to store as a tarball alongside the blueprint so you've got potentially your entire
00:16:54.540 application ready to to deploy via a blueprint
00:17:00.120 and now the last type of resource that blueprint takes a look at are services and these are definitely a little bit
00:17:07.500 different than everything else because if you'll notice everything so far has been file system State it's been one or
00:17:14.699 more files on disk their their metadata Etc whereas a service is a running process
00:17:21.000 it's runtime State and not something that we can describe in the same way so
00:17:26.220 one possibility would be to describe a service by its uh its file that just
00:17:31.559 that determines how it started and stopped but that's in Etsy init or Etsy init.d so that's already a file resource
00:17:38.700 so what we actually care about for a service resource is the conditions under which it should
00:17:45.600 be restarted and this means that we pay attention to things like if a service
00:17:51.360 definition mentions another file or a another directory we restart the service
00:17:57.960 when those files and directories are changed if if a service definition is
00:18:03.179 the member of a package and that package is upgraded we restart the service
00:18:08.760 so this means that when you apply a blueprint you're going to get not only the latest
00:18:13.820 updates to your disk but you're also going to have that reflected in the runtime state of the system so that
00:18:19.440 gives us more like a full deployment system baked into blueprint
00:18:26.220 so remember long ago when this was all the the process of running blueprint
00:18:31.799 create it actually takes about five seconds so that was very long-winded in comparison
00:18:37.260 but once you've got that once it's run it's stored locally in git and you might be tempted and rightfully so to inspect
00:18:44.220 what's in the blueprint so this blueprint show command would tell you in uh in the the Json format what's in
00:18:51.539 the blueprint and so you can see without really needing to read there's a file a
00:18:58.200 package and a service defined here and they Define metadata and versions
00:19:03.780 and uh and you can real quickly inspect that and see what's what's in it and what's not in it
00:19:10.320 likewise another blueprint this one that has the uh systems Ruby some some gems
00:19:16.500 and then a source tarball that we've extracted from user local and again you can see quickly what's in
00:19:22.440 it and what's not in it and uh and and be able to understand if something's missing and we'll get to how
00:19:28.980 you correct that in one moment if the the looking at Raw Json gives you a headache you can use the uh the kind
00:19:36.179 of shell friendly front ends to that blueprint show packages is one that
00:19:41.340 prints the uh the package manager name and version all uh all in a pipeline
00:19:46.860 friendly format that you can send through grep instead and awk and do strange and wonderful things to it
00:19:52.620 uh likewise there's blueprint show sources there's also show files and show services and the sources prints the uh
00:19:59.700 the verbose listing as if from tar TV so you can see what's in those tarballs
00:20:06.059 and you may find after looking at at your blueprint one of many possible ways
00:20:11.580 that you don't like it that there's something missing there's something that's uh too verbose that you you
00:20:16.620 installed a month ago and you don't actually care about and uh inspired basically exactly by git ignore we've
00:20:23.760 got an answer to that in the form of the blueprint ignore file so a file something like this in your
00:20:30.360 home directory would uh would tweak how blueprint determines what files to
00:20:35.760 include what packages to include so the top section looks exactly like a git ignore file it follows all the same
00:20:42.720 rules it ignores directories and files and it unignores things with the the exclamation point in the in the
00:20:49.080 beginning yeah is the blueprint ignore file included in
00:20:55.559 the blueprint not in the same way that another file is but yes it is stored with each revision
00:21:03.240 of the blueprint so you can um a second ago you could say blueprint show packages you can say blueprint show
00:21:09.539 ignore and get the ignore rules for that particular blueprint
00:21:15.660 um so the the second section is where we introduce new syntax uh colon resource
00:21:22.320 type colon so to ignore or unignore packages and likewise at the bottom
00:21:28.039 Source directories we can ignore the very bottom we've unignored the opt directory so that we're now creating a
00:21:34.440 tarball of whatever's in opt and we can deploy whatever we have in opt that way as well
00:21:39.860 there's a curious strange pattern in the middle here where we ignore and then
00:21:45.659 immediately unignor the build essential package and I feel like this deserves explaining because it looks kind of
00:21:51.840 strange when I when I worked on this feature at first we we took a long look at what the
00:21:58.380 least surprising Behavior would be and when you ignore a package the least
00:22:03.600 surprising thing seems to be to also ignore all of its dependencies whereas when you unignore a package the
00:22:10.200 least surprising thing is to only unignore that one package so the
00:22:15.600 behavior that emerged is that if you ignore and then unignore the same package you end up only ignoring its
00:22:22.799 dependencies which is a cosmetic but not functional impact on what actually
00:22:28.440 happens when the blueprint is applied later this just removes the 20 odd dependencies of build essential from the
00:22:35.159 blueprint but make sure that they'll still be there in the end so now with this we run blueprint create
00:22:41.940 again and we've got a blueprint we're real happy with and now you might want
00:22:47.039 to share it with somebody else another developer or or the new hire perhaps and
00:22:52.500 so you can use a free service that we're offering experimentally where you can push it to the cloud and pull it back
00:22:59.940 down on some other service so you run blueprint push and it gives you this nasty long URL that identifies your
00:23:07.080 blueprint uniquely in the cloud and someone else can use that URL to pull it back down
00:23:12.600 and we store them behind really long unguessable random URLs so that
00:23:18.780 there's no need for user accounts in such a simple service and we store them in S3 because that's cheap and easy and
00:23:25.020 reliable and we don't have to convince anybody that that may not trust us that the data is
00:23:32.039 going to be there so when when the new guy pulls that down he's got he's got this blueprint stored
00:23:39.120 locally in his git repository and with blueprint apply Foo he's able to bring that new Dev box that's brand new up to
00:23:47.340 the same state as the one where the blueprint was created a minute ago so now with two commands the new hire is
00:23:53.340 up and running in writing code and that's great but uh it doesn't get you all the way to
00:23:58.620 production which is of course where everybody wants to be right uh you could use that same pattern of
00:24:05.159 pushing and then pulling and then applying on your production servers but that would mean that you have to have
00:24:11.460 blueprint installed which is not a big deal but for some folks that that may mean something and uh
00:24:18.059 and it's a little bit primitive and it's something that we can do better with so we do so the first and the easiest thing that
00:24:24.720 we can do better is uh is by using blueprint show and giving it the dash capital S argument this time and this
00:24:32.100 instead of printing a bunch of Json to the screen which is nearly useless we create a shell script that will apply
00:24:39.720 all of the resources identified in that blueprint one after the other following the algorithmic order and doing so in a
00:24:46.380 safe item potent way so you could take food.sh and immediately send it to production and run it or you could give
00:24:53.220 it to your your version control system and maybe your CI system to walk it
00:24:58.620 through the the staging process and make sure everything works fine there before pushing it to production
00:25:03.960 but now you don't have to worry about carting the whole of blueprint along with it blueprint can remain a
00:25:09.659 development only tool and the the code it generates looks
00:25:14.880 something like this and I mentioned that it follows the algorithm and that it does so in a nice item put in a way so
00:25:21.299 you can see that it's gating the installation of a package by checking
00:25:26.400 the version that's available so we don't waste time in apt if it's ultimately going to decide everything's fine
00:25:32.279 um and then when when that package is upgraded it notes to restart the SSH service which it does on the very last
00:25:38.640 line there if it needs to so that's cool right I didn't put in
00:25:43.740 Shell scripts without having to think about it this is the part where if you're a
00:25:49.260 puppet or Chef Shop already you can use puppet and Chef still in
00:25:55.200 production but uh but use blueprint and development and uh
00:26:01.080 the end result here is that you get the great workflow yet your operators are still happy that there's puppet and Chef
00:26:06.900 code so the puppet code looks something like this it's um
00:26:12.120 pretty much human readable it follows the coding standards and it provides a lot of hooks to be able to grab just the
00:26:18.720 subset of functionality you want should that become necessary and again it follows the algorithm for applying a
00:26:24.840 blueprint by declaring the order of dependencies along the way
00:26:32.279 here is the generated Chef code which is uh actually in Ruby a special API given
00:26:40.020 to Chef recipes and likewise applying those same three resources it follows the coding standards and it performs
00:26:45.960 them in the algorithmic order and if you gave this to your your operators
00:26:52.260 or your guy in charge of Chef or puppet he really wouldn't know the difference between a between this and human written
00:26:58.500 stuff yes
00:27:09.419 it's about the development workflow not about the particular text format or
00:27:14.880 anything like that so it's about the fact that you a developer don't have to ask operations or
00:27:22.200 deal with deal directly with puppet or Chef code to make changes to how the environment works
00:27:30.059 um so there is the generated Chef code it's awesome because it looks like a human wrote it and a machine did
00:27:38.520 um if you're not already using puppet or Chef though I'd encourage you to think about using blueprint through and
00:27:44.100 through and uh and one real cool thing we can do is since there's a cloud
00:27:50.760 service where you can push and pull blueprints we have an API endpoint there that can
00:27:56.760 generate Shell Code for the blueprint remotely which means you can download it analogous to a blueprint pool you can
00:28:03.120 download the Shell Code from the server directly and not have to install blueprint so you can do like this and I
00:28:09.480 apologize for the tiny font you can set a cron tab up to download the Shell Code
00:28:16.020 for your blueprint directly and pipe it to Shell so with one cron tab you've reinvented the the hourly compliance run
00:28:24.240 that puppet and Chef run demons for just right there and this means that when your definition
00:28:31.500 of what it is to be a web server or a database server or an app or whatever else changes everybody brings themselves
00:28:38.640 into compliance automatically as that definition is pushed
00:28:43.679 and the the last thing that we do uh that I think is especially cool is
00:28:48.980 interacting with ec2 and providing this sort of shim to allow straight ec2 users
00:28:56.340 or cloud formation users to immediately bootstrap new servers with a
00:29:01.919 given blueprint and so instead of the food.sh endpoint from a moment ago if you grab userdata.sh you get this little
00:29:08.940 shim that at first boot will launch download and launch the blueprint of
00:29:14.820 your choice to configure a server and make it ready to go yeah
00:29:25.140 it does not currently do that it's something I'm interested in adding
00:29:30.659 however um
00:29:40.679 yes the um in uh let's see in a system where you're
00:29:48.120 if you're uninstalling system packages or you're adding especially downgrading
00:29:53.760 a ruby gem since you can have multiple versions of rubygems those are the two cases where it could bite you
00:30:00.600 um in practice that doesn't tend to happen because people tend to upgrade rubygems
00:30:06.299 and people tend to just not rely if a system package should be uninstalled they tend
00:30:12.659 to just not rely on it anymore so it hasn't come up but it's something that I would like to make better
00:30:19.399 and that's one good piece of future work the others that I'm thinking about are
00:30:24.480 bringing some notion of modularity to the blueprints so that your common functionality across all your types of
00:30:30.960 servers can be in one blueprint and the extra special things that make each server unique can be in their own
00:30:37.500 Blueprints and in addition to that we're always adding new resource types new service providers new package providers
00:30:43.799 so if you've got features we're not supporting yet please submit bugs and tell us about it
00:30:49.020 and uh that's all so if there are more questions bring them on
00:30:54.539 yes
00:31:06.120 so um you caught me in a little bit of a lie it'll totally work on OSX and if we
00:31:11.820 built a package provider for Homebrew we would be able to extract that information very easily the problem is
00:31:18.120 nobody's deploying to OSX so there's not you know we'd have to maintain a mapping of what the Homebrew package name
00:31:24.480 actually means in Debian or red hat and that's that's the hard part and you know
00:31:29.580 it's just it's manual labor that we don't have the resources to do anybody else
00:31:35.880 yes
00:31:52.559 um there's some of that that's true and there's some of that that's not um
00:31:58.080 I think it needs to be a pretty collaborative process and the part that I think is is most of a
00:32:05.520 bottleneck is the evolution not the um and I guess this is the classic
00:32:11.279 argument The Operators want stability so nothing can change the developers want everything to change because they want new features and the middle ground is a
00:32:19.200 place where a developer can add a dependency in a in a real documented understandable way Say by pushing a new
00:32:25.740 version of a blueprint and an operator can understand and perhaps even reply with their own revisions to the
00:32:32.279 blueprint how that change should have been implemented can be implemented better or clean up the mess necessary to
00:32:39.000 actually make it work yes
00:33:11.460 so part of so part of the the process of provisioning one on a new server would
00:33:18.659 be knowing the host name or IP address and what blueprint should be applied and
00:33:25.440 um it's it's definitely outside of the scope of blueprint itself to be kind of an auto scaling uh feedback loop but
00:33:33.899 either through the pushing the git repository that blueprint keeps all of its data in to say your CI server or
00:33:41.700 your your build server or using the push and pull service directly one of those will give you the raw
00:33:48.360 material to configure a new box and then it's a matter of using either the shell download or blueprint show dash s
00:33:55.440 to generate the code that can bring up a new server with the correct configuration
00:34:00.480 but the feedback loop is something people Implement hundreds of different ways outside the scope of config
00:34:06.600 management yep puppets
00:34:29.399 um well with the exception of getting puppet on those nodes that's pretty much correct they're they're headed to a
00:34:35.159 world where uh you have a pool of unallocated servers and puppet can pick
00:34:40.919 one with the appropriate specs to fill a role as needed and that's that's been
00:34:45.960 the real long-term goal of puppet and they're just now getting to the point where that's feasible
00:34:51.960 and even then only on ec2 so far anyone else
00:35:01.020 looks like a no thanks guys
00:35:19.920 foreign
RC
Richard Crowley
2 talks
Explore all talks recorded at RubyConf 2011
+55