How-To: Install Mcollective & Puppet on EC2

by nathanmc@

=======

EC2 Prep

Create security groups with the following rules:

jump - tcp/22 - 0.0.0.0/0
puppet-cli
mcoll-cli
rabbit
mcoll-queue - tcp/6163 - mcoll-cli
puppet-master - tcp/8140 - puppet-cli

Set Up RabbitMQ

Instantiate a 64-bit m1.large of Ubuntu 12.04 LTS

Security Groups:
* rabbit
* mcoll-queue
* puppet-master
* jump

SSH to the rabbitmq host and sudo su -

Execute:

apt-get install -y erlang-base erlang-nox
wget http://www.rabbitmq.com/releases/rabbitmq-server/v2.8.1/rabbitmq-server_2.8.1-1_all.deb
dpkg -i rabbitmq-server_2.8.1-1_all.deb
rabbitmq-plugins enable amqp_client
rabbitmq-plugins enable rabbitmq_stomp

Edit /etc/rabbitmq/rabbitmq.config:

[
    {rabbitmq_stomp, [{tcp_listeners, [{"0.0.0.0", 6163},
                                       {"::1",       6163}]}]}
].

Execute:

rabbitmqctl add_user mcollective PASSWORD
rabbitmqctl set_user_tags mcollective administrator
rabbitmqctl set_permissions -p / mcollective ".*" ".*" ".*"
/etc/init.d/rabbitmq-server restart

Sanity check - Ensure it's now listening on 6163:
```
netstat -anp | grep -v unix
    ...
        tcp    0  0 0.0.0.0:6163       0.0.0.0:*           LISTEN      1240/beam 
        tcp6   0  0 ::1:6163           :::*                LISTEN      1240/beam
```
If you don't, check your history to ensure you enabled the rabbitmq_stomp plugin

Also look at /var/log/rabbitmq/startup*_log files for clues as to why it may not have started
Leave your terminal open and run
```
tail -f /var/log/rabbitmq/rabbit@*hostname*.log
```
This will show you the connections as they come in from the clients

Set Up Mcollective
Instantiate a 64-bit client host of either Ubuntu 12.04 LTS or Amazon Linux AMI 2012.03
```
Security Groups:
* jump
* puppet-cli
* mcoll-cli
```

SSH in and apply all updates

Amazon:
    sudo su -
    yum update
    shutdown -r now

Ubuntu
    sudo su -
    apt-get update
    apt-get upgrade
    shutdown -r now

SSH in and execute:

Amazon:
    sudo su -
    edit /etc/yum.repos.d/epel.repo
        set all to enabled=0
    yum install rubygem-stomp
    rpm -ivh http://yum.puppetlabs.com/el/6/products/i386/puppetlabs-release-6-5.noarch.rpm
    yum install mcollective mcollective-client

Make the following edits:

/etc/hosts - add the line:
        <internal ip of your rabbit host>   puppet

/etc/mcollective/server.cfg:
    plugin.psk = 01234    
        (NOTE:  Pre-Shared Key - Can be any value consistent across all)
    plugin.stomp.host = puppet
    plugin.stomp.port = 6163
    plugin.stomp.user = mcollective
    plugin.stomp.password = PASSWORD

/etc/mcollective/client.cfg:
    plugin.psk = 01234    
        (NOTE:  Pre-Shared Key - Can be any value consistent across all)
    plugin.stomp.host = puppet
    plugin.stomp.port = 6163
    plugin.stomp.user = mcollective
    plugin.stomp.password = PASSWORD

Restart mcollective
```
/etc/init.d/mcollective start
```
Test mcollective via the following command:
```
mco find hosts
```
Should see your localhost's name

If not, double-check that all the passwords and psk's are the name in client and server cfg's

Logs are written to /var/log/mcollective.log

You should see connections listed in the log on rabbitmq

Setup Puppet

NOTE: This guide works for puppet 2.6.x CLIENTS. Be careful about what version you're installing because most repos have more than one.

Back on the rabbitmq server, execute:

apt-cache policy puppetmaster

You should see output similar to the following

puppetmaster:
  Installed: (none)
  Candidate: 2.7.11-1ubuntu2.1
  Version table:
     2.7.11-1ubuntu2.1 0
        500 http://us-west-2.ec2.archive.ubuntu.com/ubuntu/ precise-updates/main amd64 Packages
        500 http://security.ubuntu.com/ubuntu/ precise-security/main amd64 Packages
     2.7.11-1ubuntu2 0
        500 http://us-west-2.ec2.archive.ubuntu.com/ubuntu/ precise/main amd64 Packages

For our purposes, a 2.7 server will work fine, so install it with:

apt-get install puppetmaster
cd /etc/puppet
mkdir files
cd manifests

Edit site.pp

    #/etc/puppet/manifests/site.pp
    import "nodes"
    filebucket { main: server => "<resolvable host name of self>" }

    #defaults
    File { backup => main }
    Exec { path => "/usr/bin:/usr/sbin:/bin:/sbin" }

Edit nodes.pp

    # /etc/puppet/manifests/nodes.pp

    # We're just going to put a simple example
    node default {
      exec { "touch_file":
      command => "touch /tmp/stamped.txt",
      path    => "/usr/local/bin/:/bin/",        }
    }

Edit /etc/puppet/autosign.conf (just an asterisk)

Restart the puppet daemon:

/etc/init.d/puppetmaster restart

On each puppet client:

Amazon Linux:

    yum info puppet
        ...
            Available Packages
            Name        : puppet
            Arch        : x86_64
            Version     : 2.6.16
            Release     : 1.6.amzn1
            Size        : 843 k
            Repo        : amzn-updates
            Summary     : A network tool for managing many disparate systems
            URL         : http://puppetlabs.com
            License     : GPLv2
    yum -y install puppet
    wget https://s3.amazonaws.com/trnsfr/nsmc-mco-puppetd-1.0.0-1.x86_64.rpm
    rpm -ivh nsmc-mco-puppetd-1.0.0-1.x86_64.rpm
    wget https://s3.amazonaws.com/trnsfr/nsmc-mco-facter-facts-1.0.0-1.x86_64.rpm
    rpm -ivh nsmc-mco-facter-facts-1.0.0-1.x86_64.rpm

Edit the mcollective configs:

/etc/mcollective/client.cfg:
    # Facts
    factsource = facter

/etc/mcollective/server.cfg:
    # Facts
    factsource = facter

Restart mcollective:

/etc/init.d/mcollective restart

Better Living Through Unix

Tuesday, November 27, 2012

Puppet & Mcollective

How-To: Install Mcollective & Puppet on EC2

by nathanmc@

EC2 Prep

Set Up RabbitMQ

Set Up Mcollective

Setup Puppet

About Me

Blog Archive