Chef – berkshelf lesson for dummies like me Ermahgerd!

I feel like some of the explanations on berkshelf on the internet are confusing.
So i felt like doing a small write up myself

berkshelf is pretty much a replacement for the “knife cookbook” command.
The big win with berkshelf is that it also resolves dependencies of a cookbook like apt or yum.
It reads a file called “Berksfile” for other cookbooks the current cookbook needs and what repositories to fetch them from and pulls them to your local system.

I will use the logstash cookbook at as an example
If you read the Berksfile at
it will show you what other cookbooks the logstash cookbook needs

So in order to get going

gem install berkshelf
git clone
cd logstash
berks install
berks upload

That installed berkshelf, cloned the logstash cookbook, resolved dependencies for the logstash cookbook and uploaded logstash cookbook and its dependencies to your chef-server

Additionally berkshelf installs its configuration file at : ~/.berkshelf/config.json
You may need to edit some stuff there to match your ~/.chef/knife.rb file


Chef – Nagios Server quickstart

Clone the opscode cookbook

$ git clone

Create some berkshelf dependency stuff to make your life easier
( I’m going to assume you have berkshelf installed, if not

gem install berkself

and read this )

$ cd nagios
$ cat>Berksfile<<EOF
cookbook 'bluepill'
cookbook 'perl'
cookbook 'rsyslog'
cookbook 'nginx'
cookbook 'nginx_simplecgi'

group :test do
#  cookbook 'minitest-handler', git: "git://"


Pull in your dependencies using Berkshelf and upload it to your chef-server

$ berks install
$ berks upload

Create your data bag for your nagios admin user

$ knife data bag create users
$ openssl passwd -1 -salt '78hJASHDGuywelhfsdkiukshdkfusdhgfu' 'nagiosadmin'
$ cat>nagiosadmin.json<<EOF
  "id": "nagiosadmin",
  "groups": "sysadmin",
  "htpasswd": "$1$78hJASHD$KlWqNTM0UXf/iM6imQ.9F1",
  "nagios": {
    "pager": "",
    "email": ""

Upload your nagiosadmin user to data bag on your chef-server

$ knife data bag from file users nagiosadmin.json

Create a chef role for “monitoring”

$ cat>monitoring.rb<<EOF
name "monitoring"
run_list %w[

  :nagios => {
    :server => {
      ### START Install Verison and Method
      :install_method => "package",
      ### END Install Version and Method
      :service_name => "nagios3",
      :home => "/usr/lib/nagios3",
      :conf_dir => "/etc/nagios3",
      :config_dir => "/etc/nagios3/conf.d",
      :cache_dir => "/var/cache/nagios3",
      :state_dir => "/var/lib/nagios3",
      :run_dir => "/var/run/nagios3",
      :docroot => "/usr/share/nagios3/htdocs",
      :server_name => "nagios",
      :web_server => "apache"
    :client => {
      :install_method => "package"
    :server_auth_method => "htauth",
    :url => ""

Upload the “monitoring” role to chef-server and then apply the role and run chef-client

$ knife role from file monitoring.rb

$ knife node run_list add -r "role[monitoring]"
$ knife ssh -a ipaddress "chef-client"

Edit your local system’s host file to point the domain to the ip of your server if you don’t have DNS

login at
username/password = nagiosadmin

Add the nrpe configurations on your clients

Create the application cookbook for your custom nrpe service checks

$ knife cookbook create mydomain_nrpe
$ cd mydomain_nrpe/recipes
$ cat>default.rb<<EOF
# Cookbook Name:: monitoring
# Recipe:: base_monitoring
# Copyright 2013, Example Company, Inc.
# This recipe defines the necessary NRPE commands for base system monitoring
# in Example Company Inc's Chef environment.

include_recipe 'nagios::client'

# Check for high load.  This check defines warning levels and attributes
nagios_nrpecheck "check_load" do
  command "#{node['nagios']['plugin_dir']}/check_load"
  warning_condition "6"
  critical_condition "10"
  action :add

# Check all non-NFS/tmp-fs disks.
nagios_nrpecheck "check_all_disks" do
  command "#{node['nagios']['plugin_dir']}/check_disk"
  warning_condition "8%"
  critical_condition "5%"
  parameters "-A -x /dev/shm -X nfs -i /boot"
  action :add

# Check for excessive users.  This command relies on the service definition to
# define what the warning/critical levels and attributes are
nagios_nrpecheck "check_users" do
  command "#{node['nagios']['plugin_dir']}/check_users"
  action :add

Upload the cookbook

$ knife cookbook upload mydomain_nrpe

Add the recipe to the run list of a node you want the nrpe services installed to or just assign it to a role

$ knife node run_list add james.mydomain "recipe[mydomain_nrpe]"
$ knife ssh -a ipaddress -x root name:james.mydomain "chef-client"

Add services to your nagios server using data bag entires in “nagios_services” data bag

$ knife data bag create nagios_services
$ mkdir nagios_services
$ cd nagios_services
$ cat>ssh.json<<EOF
  "id": "ssh",
  "hostgroup_name": "linux",
  "command_line": "$USER1$/check_ssh $HOSTADDRESS$"
$ cat>pingme.json<EOF
"id": "pingme",
 "hostgroup_name": "linux",
 "use_existing_command": "check-host-alive"
$ wget
$ wget
$ wget

Ingest all the nagios json service files to chef-server and run chef-client on the nagios server

$ ls |while read i ; do knife data bag from file nagios_services $i ; done
$ knife ssh -a ipaddress -x root "chef-client"

Install a system that’s not managed by chef

$ knife data bag create nagios_unmanagedhosts
$ cat >my host.json<EOF
  "address": "",
  "hostgroups": ["linux"],
  "id": "myhost",
  "notifications": 0
$ knife data bag from file nagios_unmanagedhosts host.json
$ knife ssh -x root -a ipaddress "chef-client"

Adding EBS Volumes with Opscode’s AWS cookbook

1. Download opscode’s aws cookbook and put it into your own cookbook repo

$ git clone opscode-cookbooks
$ cp -r opscode-cookbooks/aws my-cookbooks/
$ cd my-cookbooks 

2. Create a new cookbook that will utilize the aws cookbook

$ knife cookbook create aws-tests

3. Set the cookbook to have the dependency of the opscode aws cookbook

$ vi my-cookbooks/aws-tests/metadata.rb
maintainer       "YOUR_COMPANY_NAME"
maintainer_email "YOUR_EMAIL"
license          "All rights reserved"
description      "Installs/Configures aws_tests"
long_description, ''))
version          "0.0.1"

depends "aws"

4. Create your recipe to create and attach a new EBS volume to your ec2 instance

$ vi vi my-cookbooks/aws-tests/recipes/default.rb
# Create and attach your new EBS volume
aws_ebs_volume "new_ebs_volume" do
  aws_access_key "MYAPIKEY"
  aws_secret_access_key "MYAPIKEYSECRET"
  size 1
  device "/dev/xvdi"
  action [ :create, :attach ]

5. Create a filesystem and mount your new volume

# Create your partition and filesystem for ext4
bash "create_filesystem" do
  user "root"
  code <<-EOH
    parted /dev/xvdi mklabel gpt
    parted /dev/xvdi mkpart logical ext4 1 -1
    parted /dev/xvdi set 1 lvm on
    yes | parted /dev/xvdi mkpart logical ext4 1 -- "-1"
    mkfs.ext4 /dev/xvdi1
  not_if "parted /dev/xvdi1 |grep ext4"

directory "/mnt/test" do
  owner "root"
  group "root"
  mode "0755"
  recursive true

mount "/mnt/test" do
  device "/dev/xvdi1"
  options "rw noatime"
  fstype "ext4"
  action [ :enable, :mount ]
  not_if "cat /proc/mounts |grep /mnt/test"

6. Add aws and aws_tests recipes to your node

$ knife node edit i-fff4f8c
  "chef_environment": "_default",
  "name": "i-fff4f8",
  "run_list": [
  "normal": {
    "tags": [


7. Run chef-client on your node

$ chef-client

Debugging knife configure -i

If you ever see something like this

root@chefserver01:/etc/chef# knife configure -i -V
Overwrite /root/.chef/knife.rb? (Y/N) y
Please enter the chef server URL: [http://chefserver01:4000]
Please enter a clientname for the new client: [root] jtran7
Please enter the existing admin clientname: [chef-webui]
Please enter the location of the existing admin client's private key: [/etc/chef/webui.pem]
Please enter the validation clientname: [chef-validator]
Please enter the location of the validation key: [/etc/chef/validation.pem]
Please enter the path to a chef repository (or leave blank):
Creating initial API user...
INFO: HTTP Request Returned 500 Internal Server Error: Connection reset by peer
ERROR: Server returned error for http://chefserver01:4000/clients, retrying 1/5 in 4s
INFO: HTTP Request Returned 409 Conflict: Client already exists
INFO: HTTP Request Returned 500 Internal Server Error: Connection reset by peer
ERROR: Server returned error for http://chefserver01:4000/clients/jtran7, retrying 1/5 in 4s
INFO: HTTP Request Returned 500 Internal Server Error: Connection reset by peer
ERROR: Server returned error for http://chefserver01:4000/clients/jtran7, retrying 2/5 in 5s

it’s likely your ampq password is incorrect in /etc/chef/server.rb

# amqp_pass sets the password for the AMQP virtual host in rabbitmq-server.
amqp_pass "testing"

If you had deleted your rabbitmq mnesia table you had to do something like this.
The word “testing” is your ampq password

sudo rabbitmqctl add_vhost /chef
sudo rabbitmqctl add_user chef testing
sudo rabbitmqctl set_permissions -p /chef chef ".*" ".*" ".*"

Clustered AMQP Rabbit-MQ

RabbitMQ already has excellent documentation at:

But this will be a bit more chef specific
server01 = existing chef-server
server02 = new rabbitmq server to be added to cluster

On Server01

On your existing chef-server
*I assume you’re using the latest rabbitmq and that server02 will also install matching version
Check what the cluster output looks like

root@server01:~# rabbitmqctl cluster_status
Cluster status of node rabbit@server01 ..

Copy your rabbitmq cookie to server02

root@server01:~# scp /var/lib/rabbitmq/.erlang.cookie root@server02:/var/lib/rabbitmq/
Are you sure you want to continue connecting (yes/no)? yes
root@server02's password: 
.erlang.cookie                                                                                        100%   20     0.0KB/s   00:00    

On Server02

Install rabbitmq

root@server02:~# echo "deb testing main" |tee -a /etc/apt/sources.list
root@server02:~# wget
root@server02:~# apt-key add rabbitmq-signing-key-public.asc
root@server02:~# apt-get update
root@server02:/var/chef/cache# apt-get -y install rabbitmq-server 

Delete existing mnesia database and start rabbitmq

root@server02:~# service rabbitmq-server stop
root@server02:~# rm -fr /var/lib/rabbitmq/mnesia 
root@server02:~# service rabbitmq-server start
 * Starting message broker rabbitmq-server

Join Server02 to Server01

root@server02:~# rabbitmqctl cluster_status
Cluster status of node rabbit@server02 ...
root@server02:~# rabbitmqctl stop_app
Stopping node rabbit@server02 ...
root@server02~# rabbitmqctl reset   
Resetting node rabbit@server02 ...
root@server02:~# rabbitmqctl cluster rabbit@server01 rabbit@server02
Clustering node rabbit@server02 with [rabbit@server01,rabbit@server02] ...
root@server02:~# rabbitmqctl start
root@server02:~# rabbitmqctl cluster_status
Cluster status of node rabbit@server01 ...

Verify on server01 that cluster shows up as expected

root@server01:~# rabbitmqctl cluster_status
Cluster status of node rabbit@server01 ...

Make sure chef still works

root@server01:~# knife node list

root@server01:~# knife client list
root@server01:~# knife bootstrap -x root server02 --template-file ~/ubuntu12.04.rb 
Bootstrapping Chef on server02
Failed to authenticate root - trying password auth
Enter your password: 
server02 Updating installed gems

root@server01:~# knife node list

Essential Knife Plugins for chef

This plugin helps you control versioning of your cookbooks and also prevents you from accidently commiting things you don’t want to.

$ gem install knife-spork

Obviously this is a connector to ec2. I use Amazon ec2 so i am biased

$ gem install knife-ec2

A preflight plugin for Chef::Knife which lets you see which nodes and roles use a particular cookbook before you upload it.

$ gem install knife-preflight

A plugin for Chef::Knife which will diff the cookbook versions of two or more environments.

$ gem install knife-env-diff

Other mentionables

Threaded bootstrapping of aws nodes

Cluster Deployments

Search for more knife plugins:

$ gem search -r knife-

Multiple AWS Accounts with Knife Admin

I Recently stumbled across a predicament of multiple aws accounts.
This is a minor predicament but a predicament nonethless.
I have a situation where i have

1. A personal AWS account

2. A work AWS account

3. A vendor AWS account

These three AWS accounts all use the same chef-server. So to make my life easier i decided to organize them.
I created the following structure:

$ mkdir -p ~/chef-aws/{personal,work,thirdparty}/.chef

I copied my knife.rb from ~/.chef/knife.rb into each of these folders.

$ cp -p ~/.chef/knife.rb ~/chef-aws/personal
$ cp -p ~/.chef/knife.rb ~/chef-aws/work
$ cp -p ~/.chef/knife.rb ~/chef-aws/thirdparty

Here’s an example of the knife.rb file
You can find details on setting up knife with ec2 here : Knife-EC2 Configuration

current_dir = File.dirname(__FILE__)
log_level                :info
log_location             STDOUT
node_name                "neosirex"
client_key               "/home/James/.chef/myuser.pem"
validation_client_name   "neosirex-validator"
validation_key           "/home/James/.chef/random-validator.pem"
chef_server_url          ""
cache_type               'BasicFile'
cache_options( :path => "#{ENV['HOME']}/.chef/checksums" )
cookbook_path            ["#{current_dir}/../cookbooks"]

Here’s the snippet that’s added to each AWS specific knife.rb

knife[:aws_access_key_id] ='< AWS ACCESS KEY GOES HERE >'
knife[:aws_secret_access_key] ='< AWS SECRET KEY GOES HERE >'

So now in order to use different AWS accounts what i do is change into each of those aws directories and run knife commands from there.
Each of the following commands would give me the output only of the relevant AWS server

$ cd ~/chef-aws/personal && knife ec2 server list
$ cd ~/chef-aws/work && knife ec2 server list
$ cd ~/chef-aws/thirdparty && knife ec2 server list

I Leave my default ~/.chef/knife.rb file without AWS credentials in it.
This is because i don’t want to accidently deploy to the wrong AWS account.
There’s still room for human error but i suppose it’s better than nothing
If someone has a better approach to this i’d like to know about it.

Hash Keys and Values in General Ruby

Create a new Hash

node =

Create a new Hash of Hash

node[:one] =

Insert Values into your Hash of Hash

node[:one][:object] = "number1"
node[:one][:block] = "number2"

Print Key and Value

node[:one].each_pair do |k,v|
  puts k
  puts v

Print Keys only

node[:one].keys.each do |key|
  puts key

Print Values Only

node[:one].values.each do |value|
  puts value

Keys and Values from Hashes in Ruby Templates

ERB recognizes certain tags in the provided template and converts them based on the rules below:

<% Ruby code -- inline with output %>
<%= Ruby expression -- replace with result %>
<%# comment -- ignored -- useful in testing %>
% a line of Ruby code -- treated as <% line %> (optional -- see
%% replaced with % if first thing on a line and % processing is used
<%% or %%> -- replace with <% or %> respectively

Create a new “motd” cookbook

$ knife cookbook create motd

Example: ~/cookbooks/motd/recipes/default.rb

# Cookbook Name:: motd
# Recipe:: default
# Copyright 2012, YOUR_COMPANY_NAME
# All rights reserved - Do Not Redistribute

template "/home/motd" do
  source "motd.erb"
  owner "root"

Example: ~/cookbooks/motd/template/default/motd.erb
* You can find out what hashes are defined on a system by running “ohai”

Hostname : <%= node["hostname"] %>
Platform : <%= node["platform"] %>

Memory Usage
<% node["memory"].each_pair do |k,v| %>
<%= k %>     : <%= v%>
<% end %>

Block Devices
<% node["block_device"].each_pair do |k,v| %>
Key: <%= k %>   -   <%= v%>
<% end %>

Network Info
<% node["network"]["interfaces"].keys.each do |k| %>
Key: <%= k %>
<% end %>

<% node["network"]["interfaces"].values.each do |v| %>
Value: <%= v %>
<% end %>

Run chef-client with the “motd” cookbook installed and look at the output at

$ cat /home/motd
Hostname: ubuntu01
Platform: ubuntu

Memory Usage
vmalloc_total     : 34359738367kB
anon_pages     : 129892kB
writeback     : 0kB
dirty     : 0kB
vmalloc_used     : 266104kB
vmalloc_chunk     : 34359469948kB
active     : 186264kB
buffers     : 31252kB
commit_limit     : 773540kB
nfs_unstable     : 0kB
slab_unreclaim     : 11684kB
bounce     : 0kB
slab_reclaimable     : 17644kB
mapped     : 11580kB
cached     : 190268kB
slab     : 29328kB
inactive     : 165136kB
free     : 105708kB
total     : 502612kB
committed_as     : 926460kB
page_tables     : 4324kB
swap     : cached0kBfree522236kBtotal522236kB

Block Devices
Key: sda   -   timeout30modelVMware Virtual Sremovable0vendorVMware,rev1.0size41943040staterunning
Key: sr0   -   timeout30modelVMware IDE CDR10removable1vendorNECVMWarrev1.00size1401432staterunning
Key: fd0   -   removable1size0
Key: loop7   -   removable0size0
Key: loop6   -   removable0size0
Key: loop5   -   removable0size0
Key: loop4   -   removable0size0
Key: loop3   -   removable0size0
Key: loop2   -   removable0size0
Key: loop1   -   removable0size0
Key: loop0   -   removable0size0
Key: ram15   -   removable0size131072
Key: ram14   -   removable0size131072
Key: ram13   -   removable0size131072
Key: ram12   -   removable0size131072
Key: ram11   -   removable0size131072
Key: ram10   -   removable0size131072
Key: ram9   -   removable0size131072
Key: ram8   -   removable0size131072
Key: ram7   -   removable0size131072
Key: ram6   -   removable0size131072
Key: ram5   -   removable0size131072
Key: ram4   -   removable0size131072
Key: ram3   -   removable0size131072
Key: ram2   -   removable0size131072
Key: ram1   -   removable0size131072
Key: ram0   -   removable0size131072

Network Info
Key: eth0
Key: lo

Value: mtu16436encapsulationLoopbackflagsLOOPBACKUPLOWER_U....< i truncated output >
Value: mtu1500typeethencapsulationEthernetflagsBROADCASTM....< i truncated output >

Also for shits and giggles
Example Ohai output:

root@ubuntu01:~# ohai
  "idletime": "35 minutes 35 seconds",
  "uptime": "36 minutes 47 seconds",
  "dmi": {
    "base_board": {
      "chassis_handle": "0x0000",
      "location_in_chassis": "Not Specified",
      "product_name": "440BX Desktop Reference Platform",
      "serial_number": "None",
      "manufacturer": "Intel Corporation",
      "version": "None",
      "type": "Unknown",
      "features": "None",
      "contained_object_handles": "0",
      "all_records": [
  .. < i truncated output >

Knife EC2 Extension – Install and Use


$ apt-get install -y libxslt-dev libxml2-dev
$ gem install knife-ec2
$ gem install net-ssh-multi

Get Your AWS Keys

Login to your AWS account at
Go to My Account/Console -> Security Credentials

Scroll Down to The Certificates and Secret Keys Menu and generate your new access keys as needed

Generate the Keypair associated with your new ec2 instances ( for ssh )

Create a new keypair. This should result in a pem file output to you. If you lose this file you will not be able to access any ec2 instances associated with it unless you have alternate accounts you can login with.


$ cd ~/.chef
$ vi knife.rb

Append the following to your knife.rb

### AWS Configuration ###

## The below lines allow you to use the ec2 api
knife[:aws_access_key_id] ='< AWS ACCESS KEY GOES HERE >'
knife[:aws_secret_access_key] ='< AWS SECRET KEY GOES HERE >'

## The below allow you to ssh into new ec2 instance that are associated with the keypair below
## You can alternately choose to specify the username and key location on the knife command line
# knife[:aws_ssh_key_id] ='james-aws'
# knife[:identity_file] ="/home/james/.ssh/james-aws.pem"

Test knife-ec2 command

bootstrap file squeeze.rb can be grabbed from here:

$ knife ec2 server list
$ knife ec2 server create -I ami-e00df089 -f t1.micro -Z us-east-1a -G "default_security" -k james-aws --ssh-key /home/james/.ssh/james-aws.pem --template-file /home/james/bootstrap/squeeze.rb

knife-ec2 command reference list

knife ec2 server create --help
knife ec2 server create (options)
    -Z, --availability-zone ZONE     The Availability Zone
    -A, --aws-access-key-id KEY      Your AWS Access Key ID
    -K SECRET,                       Your AWS API Secret Access Key
        --user-data USER_DATA_FILE   The EC2 User Data file to provision the instance with
        --bootstrap-version VERSION  The version of Chef to install
    -N, --node-name NAME             The Chef node name for your new node
        --server-url URL             Chef Server URL
    -k, --key KEY                    API Client Key
        --color                      Use colored output
    -c, --config CONFIG              The configuration file to use
        --defaults                   Accept default values for all questions
    -d, --distro DISTRO              Bootstrap a distro using a template
        --ebs-no-delete-on-term      Do not delete EBS volumn on instance termination
        --ebs-size SIZE              The size of the EBS volume in GB, for EBS-backed instances
    -e, --editor EDITOR              Set the editor to use for interactive commands
    -E, --environment ENVIRONMENT    Set the Chef environment
    -f, --flavor FLAVOR              The flavor of server (m1.small, m1.medium, etc)
    -F, --format FORMAT              Which format to use for output
    -i IDENTITY_FILE,                The SSH identity file used for authentication
    -I, --image IMAGE                The AMI for the server
        --no-color                   Don't use colors in the output
    -n, --no-editor                  Do not open EDITOR, just accept the data as is
        --no-host-key-verify         Disable host key verification
    -u, --user USER                  API Client Username
        --prerelease                 Install the pre-release chef gems
        --print-after                Show the data after a destructive operation
        --region REGION              Your AWS region
    -r, --run-list RUN_LIST          Comma separated list of roles/recipes to apply
    -G, --groups X,Y,Z               The security groups for this server
    -S, --ssh-key KEY                The AWS SSH key id
    -P, --ssh-password PASSWORD      The ssh password
    -x, --ssh-user USERNAME          The ssh username
    -s, --subnet SUBNET-ID           create node in this Virtual Private Cloud Subnet ID (implies VPC mode)
        --template-file TEMPLATE     Full path to location of template to use
    -V, --verbose                    More verbose output. Use twice for max verbosity
    -v, --version                    Show chef version
    -y, --yes                        Say yes to all prompts for confirmation
    -h, --help                       Show this message