From Mitch Garnaat's post on the subject:
http://www.elastician.com/2010/04/subscribing-sqs-queue-to-sns-topic.html
User Chris Moyer points out:
One very important thing to note here is that the messages from SNS do NOT come through as base64 encoded, so you'll have to set the message class to "RawMessage" first:
from boto.sqs.message import RawMessage
q.set_message_class(RawMessage)
msg = q.read()
Also, the message "payload" is actually stored under "Message" in the provided JSON:
msg_data = json.loads(msg.get_body())['Message']
Showing posts with label aws. Show all posts
Showing posts with label aws. Show all posts
Tuesday, July 15, 2014
Tuesday, November 27, 2012
Puppet & Mcollective
How-To: Install Mcollective & Puppet on EC2
by nathanmc@
=======
EC2 Prep
Create security groups with the following rules:
jump - tcp/22 - 0.0.0.0/0 puppet-cli mcoll-cli rabbit mcoll-queue - tcp/6163 - mcoll-cli puppet-master - tcp/8140 - puppet-cliSet Up RabbitMQ
Instantiate a 64-bit m1.large of Ubuntu 12.04 LTS
Security Groups: * rabbit * mcoll-queue * puppet-master * jump- SSH to the rabbitmq host and
sudo su - Execute:
apt-get install -y erlang-base erlang-nox wget http://www.rabbitmq.com/releases/rabbitmq-server/v2.8.1/rabbitmq-server_2.8.1-1_all.deb dpkg -i rabbitmq-server_2.8.1-1_all.deb rabbitmq-plugins enable amqp_client rabbitmq-plugins enable rabbitmq_stompEdit /etc/rabbitmq/rabbitmq.config:
[ {rabbitmq_stomp, [{tcp_listeners, [{"0.0.0.0", 6163}, {"::1", 6163}]}]} ].Execute:
rabbitmqctl add_user mcollective PASSWORD rabbitmqctl set_user_tags mcollective administrator rabbitmqctl set_permissions -p / mcollective ".*" ".*" ".*" /etc/init.d/rabbitmq-server restartSanity check - Ensure it's now listening on 6163:
netstat -anp | grep -v unix ... tcp 0 0 0.0.0.0:6163 0.0.0.0:* LISTEN 1240/beam tcp6 0 0 ::1:6163 :::* LISTEN 1240/beamIf you don't, check your history to ensure you enabled the rabbitmq_stomp plugin
Also look at /var/log/rabbitmq/startup*_log files for clues as to why it may not have started
Leave your terminal open and run
tail -f /var/log/rabbitmq/rabbit@*hostname*.logThis will show you the connections as they come in from the clients
Set Up Mcollective
Instantiate a 64-bit client host of either Ubuntu 12.04 LTS or Amazon Linux AMI 2012.03
Security Groups: * jump * puppet-cli * mcoll-cliSSH in and apply all updates
Amazon: sudo su - yum update shutdown -r now Ubuntu sudo su - apt-get update apt-get upgrade shutdown -r nowSSH in and execute:
Amazon: sudo su - edit /etc/yum.repos.d/epel.repo set all to enabled=0 yum install rubygem-stomp rpm -ivh http://yum.puppetlabs.com/el/6/products/i386/puppetlabs-release-6-5.noarch.rpm yum install mcollective mcollective-clientMake the following edits:
/etc/hosts - add the line: <internal ip of your rabbit host> puppet /etc/mcollective/server.cfg: plugin.psk = 01234 (NOTE: Pre-Shared Key - Can be any value consistent across all) plugin.stomp.host = puppet plugin.stomp.port = 6163 plugin.stomp.user = mcollective plugin.stomp.password = PASSWORD /etc/mcollective/client.cfg: plugin.psk = 01234 (NOTE: Pre-Shared Key - Can be any value consistent across all) plugin.stomp.host = puppet plugin.stomp.port = 6163 plugin.stomp.user = mcollective plugin.stomp.password = PASSWORDRestart mcollective
/etc/init.d/mcollective startTest mcollective via the following command:
mco find hostsShould see your localhost's name
If not, double-check that all the passwords and psk's are the name in client and server cfg's
Logs are written to /var/log/mcollective.log
You should see connections listed in the log on rabbitmq
Setup Puppet
NOTE: This guide works for puppet 2.6.x CLIENTS. Be careful about what version you're installing because most repos have more than one.
Back on the rabbitmq server, execute:
apt-cache policy puppetmasterYou should see output similar to the following
puppetmaster: Installed: (none) Candidate: 2.7.11-1ubuntu2.1 Version table: 2.7.11-1ubuntu2.1 0 500 http://us-west-2.ec2.archive.ubuntu.com/ubuntu/ precise-updates/main amd64 Packages 500 http://security.ubuntu.com/ubuntu/ precise-security/main amd64 Packages 2.7.11-1ubuntu2 0 500 http://us-west-2.ec2.archive.ubuntu.com/ubuntu/ precise/main amd64 PackagesFor our purposes, a 2.7 server will work fine, so install it with:
apt-get install puppetmaster cd /etc/puppet mkdir files cd manifestsEdit site.pp
#/etc/puppet/manifests/site.pp import "nodes" filebucket { main: server => "<resolvable host name of self>" } #defaults File { backup => main } Exec { path => "/usr/bin:/usr/sbin:/bin:/sbin" }Edit nodes.pp
# /etc/puppet/manifests/nodes.pp # We're just going to put a simple example node default { exec { "touch_file": command => "touch /tmp/stamped.txt", path => "/usr/local/bin/:/bin/", } }Edit /etc/puppet/autosign.conf (just an asterisk)
*Restart the puppet daemon:
/etc/init.d/puppetmaster restartOn each puppet client:
Amazon Linux:
yum info puppet ... Available Packages Name : puppet Arch : x86_64 Version : 2.6.16 Release : 1.6.amzn1 Size : 843 k Repo : amzn-updates Summary : A network tool for managing many disparate systems URL : http://puppetlabs.com License : GPLv2 yum -y install puppet wget https://s3.amazonaws.com/trnsfr/nsmc-mco-puppetd-1.0.0-1.x86_64.rpm rpm -ivh nsmc-mco-puppetd-1.0.0-1.x86_64.rpm wget https://s3.amazonaws.com/trnsfr/nsmc-mco-facter-facts-1.0.0-1.x86_64.rpm rpm -ivh nsmc-mco-facter-facts-1.0.0-1.x86_64.rpmEdit the mcollective configs:
/etc/mcollective/client.cfg: # Facts factsource = facter /etc/mcollective/server.cfg: # Facts factsource = facterRestart mcollective:
/etc/init.d/mcollective restart
Subscribe to:
Posts (Atom)