Ruby – Using RVM to create your ruby jail

* This has only been tested with ubuntu 12.04 – you also already need gcc and ruby of some sort installed
These instructions allow you to run your own version of ruby and rubygems from your home folder

Download and install rvm
Set a couple of environment variables

bash -s stable < <(curl -s https://raw.github.com/wayneeseguin/rvm/master/binscripts/rvm-installer) 

echo '[[ -s "$HOME/.rvm/scripts/rvm" ]] && source "$HOME/.rvm/scripts/rvm"' >> ~/.bash_profile 
echo 'PATH=$PATH:$HOME/.rvm/usr/bin # Add RVM to PATH for scripting' >> ~/.bash_profile
. ~/.bash_profile

Install Ruby 1.9.3

rvm install 1.9.3
rvm use 1.9.3 --default

Install some gnu tools you need to install gems

wget ftp://ftp.gnu.org/gnu/m4/m4-1.4.16.tar.gz 
tar xzvf m4-1.4.16.tar.gz && cd m4-1.4.16/
./configure --prefix=$HOME/.rvm/usr
make && make install

wget ftp://ftp.gnu.org/gnu/gperf/gperf-3.0.4.tar.gz
tar xzvf gperf-3.0.4.tar.gz
cd gperf-3.0.4/
./configure --prefix=$HOME/.rvm/usr
make && make install

wget ftp://invisible-island.net/byacc/byacc.tar.gz
tar xzvf byacc.tar.gz
cd byacc-20121003/
./configure --prefix=$HOME/.rvm/usr
make && make install

wget ftp://ftp.gnu.org/gnu/termcap/termcap-1.3.1.tar.gz
tar xzvf termcap-1.3.1.tar.gz
cd termcap-1.3.1/
./configure --prefix=$HOME/.rvm/usr
make && make install

wget ftp://ftp.gnu.org/gnu/ncurses/ncurses-5.9.tar.gz
tar xzvf ncurses-5.9.tar.gz
cd ncurses-5.9/
./configure --prefix=$HOME/.rvm/usr CFLAGS=-fPIC
make && make install

wget ftp://ftp.gnu.org/gnu/texinfo/texinfo-4.13a.tar.gz
tar xzvf texinfo-4.13a.tar.gz
cd texinfo-4.13/
./configure --prefix=$HOME/.rvm/usr LDFLAGS=-L$HOME/.rvm/usr/lib CPPFLAGS=-I$HOME/.rvm/usr/include/ncurses
make && make install

Install some more tools you need to install gems
This time just use the ones that rvm has packaged
# ORDER MATTERS !!!

for i in curl zlib readline openssl iconv pkgconfig autoconf libxml2 libxslt libyaml ; do rvm pkg install $i --verify-downloads 1 --with-opt-dir=$HOME/.rvm/usr ; done

Reinstall ruby 1.9.3 with the new path of your tools compiled in

rvm reinstall 1.9.3 --with-opt-dir=$HOME/.rvm/usr

Install the ‘fog’ gem

gem install fog

Your home folder will now be 1.4GB large but you’ll have a self contained ruby and rubygems installation with the fog library available

Advertisements

Adding EBS Volumes with Opscode’s AWS cookbook

1. Download opscode’s aws cookbook and put it into your own cookbook repo

$ git clone https://github.com/opscode/cookbooks.git opscode-cookbooks
$ cp -r opscode-cookbooks/aws my-cookbooks/
$ cd my-cookbooks 

2. Create a new cookbook that will utilize the aws cookbook

$ knife cookbook create aws-tests

3. Set the cookbook to have the dependency of the opscode aws cookbook

$ vi my-cookbooks/aws-tests/metadata.rb
maintainer       "YOUR_COMPANY_NAME"
maintainer_email "YOUR_EMAIL"
license          "All rights reserved"
description      "Installs/Configures aws_tests"
long_description IO.read(File.join(File.dirname(__FILE__), 'README.md'))
version          "0.0.1"

depends "aws"

4. Create your recipe to create and attach a new EBS volume to your ec2 instance

$ vi vi my-cookbooks/aws-tests/recipes/default.rb
# Create and attach your new EBS volume
aws_ebs_volume "new_ebs_volume" do
  aws_access_key "MYAPIKEY"
  aws_secret_access_key "MYAPIKEYSECRET"
  size 1
  device "/dev/xvdi"
  action [ :create, :attach ]
end

5. Create a filesystem and mount your new volume

# Create your partition and filesystem for ext4
bash "create_filesystem" do
  user "root"
  code <<-EOH
    parted /dev/xvdi mklabel gpt
    parted /dev/xvdi mkpart logical ext4 1 -1
    parted /dev/xvdi set 1 lvm on
    yes | parted /dev/xvdi mkpart logical ext4 1 -- "-1"
    mkfs.ext4 /dev/xvdi1
  EOH
  not_if "parted /dev/xvdi1 |grep ext4"
end


directory "/mnt/test" do
  owner "root"
  group "root"
  mode "0755"
  recursive true
end

mount "/mnt/test" do
  device "/dev/xvdi1"
  options "rw noatime"
  fstype "ext4"
  action [ :enable, :mount ]
  not_if "cat /proc/mounts |grep /mnt/test"
end

6. Add aws and aws_tests recipes to your node

$ knife node edit i-fff4f8c
{
  "chef_environment": "_default",
  "name": "i-fff4f8",
  "run_list": [
    "recipe[aws]",
    "recipe[aws_tests]"
  ],
  "normal": {
    "tags": [

    ],
  }
}

7. Run chef-client on your node

$ chef-client

Multiple AWS Accounts with Knife Admin

I Recently stumbled across a predicament of multiple aws accounts.
This is a minor predicament but a predicament nonethless.
I have a situation where i have

1. A personal AWS account

2. A work AWS account

3. A vendor AWS account

These three AWS accounts all use the same chef-server. So to make my life easier i decided to organize them.
I created the following structure:

$ mkdir -p ~/chef-aws/{personal,work,thirdparty}/.chef

I copied my knife.rb from ~/.chef/knife.rb into each of these folders.

$ cp -p ~/.chef/knife.rb ~/chef-aws/personal
$ cp -p ~/.chef/knife.rb ~/chef-aws/work
$ cp -p ~/.chef/knife.rb ~/chef-aws/thirdparty

Here’s an example of the knife.rb file
You can find details on setting up knife with ec2 here : Knife-EC2 Configuration


current_dir = File.dirname(__FILE__)
log_level                :info
log_location             STDOUT
node_name                "neosirex"
client_key               "/home/James/.chef/myuser.pem"
validation_client_name   "neosirex-validator"
validation_key           "/home/James/.chef/random-validator.pem"
chef_server_url          "https://api.opscode.com/organizations/somemakebelieveaccount"
cache_type               'BasicFile'
cache_options( :path => "#{ENV['HOME']}/.chef/checksums" )
cookbook_path            ["#{current_dir}/../cookbooks"]

Here’s the snippet that’s added to each AWS specific knife.rb

knife[:aws_access_key_id] ='< AWS ACCESS KEY GOES HERE >'
knife[:aws_secret_access_key] ='< AWS SECRET KEY GOES HERE >'

So now in order to use different AWS accounts what i do is change into each of those aws directories and run knife commands from there.
Each of the following commands would give me the output only of the relevant AWS server

$ cd ~/chef-aws/personal && knife ec2 server list
$ cd ~/chef-aws/work && knife ec2 server list
$ cd ~/chef-aws/thirdparty && knife ec2 server list

I Leave my default ~/.chef/knife.rb file without AWS credentials in it.
This is because i don’t want to accidently deploy to the wrong AWS account.
There’s still room for human error but i suppose it’s better than nothing
If someone has a better approach to this i’d like to know about it.

Knife EC2 Extension – Install and Use

Install

$ apt-get install -y libxslt-dev libxml2-dev
$ gem install knife-ec2
$ gem install net-ssh-multi

Get Your AWS Keys

Login to your AWS account at aws.amazon.com
Go to My Account/Console -> Security Credentials

Scroll Down to The Certificates and Secret Keys Menu and generate your new access keys as needed

Generate the Keypair associated with your new ec2 instances ( for ssh )

Create a new keypair. This should result in a pem file output to you. If you lose this file you will not be able to access any ec2 instances associated with it unless you have alternate accounts you can login with.

Configure

$ cd ~/.chef
$ vi knife.rb

Append the following to your knife.rb

### AWS Configuration ###

## The below lines allow you to use the ec2 api
knife[:aws_access_key_id] ='< AWS ACCESS KEY GOES HERE >'
knife[:aws_secret_access_key] ='< AWS SECRET KEY GOES HERE >'

## The below allow you to ssh into new ec2 instance that are associated with the keypair below
## You can alternately choose to specify the username and key location on the knife command line
# knife[:aws_ssh_key_id] ='james-aws'
# knife[:identity_file] ="/home/james/.ssh/james-aws.pem"

Test knife-ec2 command

bootstrap file squeeze.rb can be grabbed from here:
https://github.com/cookingclouds/bootstrap/

$ knife ec2 server list
$ knife ec2 server create -I ami-e00df089 -f t1.micro -Z us-east-1a -G "default_security" -k james-aws --ssh-key /home/james/.ssh/james-aws.pem --template-file /home/james/bootstrap/squeeze.rb

knife-ec2 command reference list

knife ec2 server create --help
knife ec2 server create (options)
    -Z, --availability-zone ZONE     The Availability Zone
    -A, --aws-access-key-id KEY      Your AWS Access Key ID
    -K SECRET,                       Your AWS API Secret Access Key
        --aws-secret-access-key
        --user-data USER_DATA_FILE   The EC2 User Data file to provision the instance with
        --bootstrap-version VERSION  The version of Chef to install
    -N, --node-name NAME             The Chef node name for your new node
        --server-url URL             Chef Server URL
    -k, --key KEY                    API Client Key
        --color                      Use colored output
    -c, --config CONFIG              The configuration file to use
        --defaults                   Accept default values for all questions
    -d, --distro DISTRO              Bootstrap a distro using a template
        --ebs-no-delete-on-term      Do not delete EBS volumn on instance termination
        --ebs-size SIZE              The size of the EBS volume in GB, for EBS-backed instances
    -e, --editor EDITOR              Set the editor to use for interactive commands
    -E, --environment ENVIRONMENT    Set the Chef environment
    -f, --flavor FLAVOR              The flavor of server (m1.small, m1.medium, etc)
    -F, --format FORMAT              Which format to use for output
    -i IDENTITY_FILE,                The SSH identity file used for authentication
        --identity-file
    -I, --image IMAGE                The AMI for the server
        --no-color                   Don't use colors in the output
    -n, --no-editor                  Do not open EDITOR, just accept the data as is
        --no-host-key-verify         Disable host key verification
    -u, --user USER                  API Client Username
        --prerelease                 Install the pre-release chef gems
        --print-after                Show the data after a destructive operation
        --region REGION              Your AWS region
    -r, --run-list RUN_LIST          Comma separated list of roles/recipes to apply
    -G, --groups X,Y,Z               The security groups for this server
    -S, --ssh-key KEY                The AWS SSH key id
    -P, --ssh-password PASSWORD      The ssh password
    -x, --ssh-user USERNAME          The ssh username
    -s, --subnet SUBNET-ID           create node in this Virtual Private Cloud Subnet ID (implies VPC mode)
        --template-file TEMPLATE     Full path to location of template to use
    -V, --verbose                    More verbose output. Use twice for max verbosity
    -v, --version                    Show chef version
    -y, --yes                        Say yes to all prompts for confirmation
    -h, --help                       Show this message