Introducing NYMMS

I haven’t posted for a while (5 months, ugh), but that doesn’t mean I’ve been idle this whole time. I’ve been working on a new monitoring system, and I think it’s at the point where I want to share it with the world. It can do the basics - monitor services and send alerts when they fail. I’d love it if other people started to play with it so I can get some feedback/bugs.

Anyway, check it out:

• NYMMS docs
• NYMMS github

Also, be sure to checkout the Demo AMI page if you want to bring up an all-in-one NYMMS system with very basic configuration to play with.

(more) securely storing my AWS credentials

This is an update to my previous post, AWS/Boto Credentials and my prompt.

In that post I started storing my AWS credentials in my .bash_profile. It’s worked great for a while, but there’s been one thing that’s bothered me about it: my credentials were stored in plain text in my .bash_profile. Not awesome.

So I set out to fix it. The first step was to remove all the aws credential functions and instead move them to their own file. I named it .bash_aws_profiles. I then encrypted it with openssl using the aes-256 cipher with the following command:

openssl aes-256-cbc -a -salt -in .bash_aws_profiles -out .bash_aws_profiles.enc

It will ask for a password - pick a secure one, and don’t share it.

Next you need a way to get those functions back into your shell. I tried to do this by having source or eval read in the output from openssl unencrypting the file, but neither of them worked for me for various reasons. In the end I had to use a temporary file, which I’m not super excited about, but it only exists for a very short period of time, which is better than what we had before.

As my buddy Tim would say: “Don’t let best be the enemy of good!”

So I added these functions to my .bash_profile:

function load_aws () {
    AWSKEYFILE=$HOME/.bash_aws_profiles.enc
    TEMPFILE=`mktemp $HOME/.bash_aws_profiles.XXXXXXXXXXX`
    [ ! -f $AWSKEYFILE ] && echo "Error: Amazon key file ($AWSKEYFILE) not found." && return
    openssl aes-256-cbc -d -a -in $AWSKEYFILE -out $TEMPFILE
    source $TEMPFILE
    rm $TEMPFILE
    echo "! AWS keys loaded."
}

function unload_aws () {
    KEYFUNCS=`set | awk '/aws_[^ ]+ \(\)/ {print $1}'`
    for k in $KEYFUNCS
    do
        echo "! Unloading $k"
        unset $k
    done
}

Now whenever I want to use my AWS credentials I first have to type load_aws. It asks me for my password, then loads them into the environment. When I’m finished I can run unload_aws to delete them.

I’d really like it if there was a sort of boto-agent that would store the keys instead of the shell, and then expire them after a given amount of time. I’ll work on that someday.

Let me know if you have any questions, or any ideas for improvements. Thanks!

AWS/Boto Credentials and my prompt

I do a lot of stuff with Amazon AWS/EC2 everyday. I use many different amazon accounts with different credentials. For example I post this site to S3 in my personal AWS account. I also have multiple work accounts for various things. Most of my interactions with AWS are done through python & boto, sometimes through scripts and sometimes through the python shell. I used to keep all of my various credentials in my ~/.boto file, with only the one I was using uncommented.

This meant that I had to be extra careful and doublecheck my .boto file to make sure I was using the right account when doing anything with Amazon. After a couple of times where I ran commands (non-destructive) in the wrong account I complained to my coworker. He forwarded me a trick he’d been using, and I expanded on it.

One of boto’s features is that it can gather your credentials from a few places. One of those places is via the shell variables AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY. So first I created functions in my ~/.bash_profile that allowed me to reset those variables easily. As well, I had those functions set another variable AWS_KEY_NAME. That isn’t used by boto at all, but it’s useful for me. For example, say I have two accounts: my personal account and my work account. I’d setup the following in my profile:

function aws_personal () {
    export AWS_KEY_NAME="personal"
    export AWS_ACCESS_KEY_ID=<my personal access key id>
    export AWS_SECRET_ACCESS_KEY=<my personal secret access key>
}

function aws_work () {
    export AWS_KEY_NAME="work"
    export AWS_ACCESS_KEY_ID=<my work access key id>
    export AWS_SECRET_ACCESS_KEY=<my work secret access key>
}

With that in place (and after resourcing my .bash_profile) I can now call either of those functions from my shell and I’m using those credentials in that shell until I choose to change them again.

That’s useful, but now it’d be really good to know what credentials I’m currently using. There are two places I really need to know that:

• My shell
• My python interactive interpreter

So lets start with the first. In bash it’s easy enough to change your prompt. Since I set the AWS_KEY_NAME variable I can use that in my prompt. That said, my prompt is already pretty long. I finally gave in and decided I needed to have a two line prompt.

Since I was going to do a little surgery on my prompt, and I don’t do that very often, I decided to add a couple of features to it. Those were:

• multiline
• unobtrusive color unless a previous command failed, then make sure I know
• adding my current AWS credentials if they are set

The first is easy. The second I used a variable I set in my PROMPT_COMMAND that I discussed in a previous article Keeping bash history forever. The third I used the new AWS_KEY_NAME variable. In the end, this is what I ended up with for my new prompt:

COLOR_RESET="\e[00m"
COLOR_GRAY="\e[01;30m"
COLOR_RED="\e[22;31m"

PROMPT_COMMAND="ECODE=\$?; TODAY=\`date +%Y%m%d\`;[ -d $HOME/.history ] || mkdir -p $HOME/.history; echo : [\$(date)] $USER \$ECODE \$PWD\; \$(history 1 | sed -E 's/^[[:space:]]+[0-9]*[[:space:]]+//g') >> $HOME/.history/bash_history-\$TODAY"

PS1="${COLOR_GRAY}[\$([ \$ECODE -ne 0 ] && echo \"${COLOR_RED}\$ECODE${COLOR_GRAY} \")\u@\h \$([ ! -z \$AWS_KEY_NAME ] && echo \"aws:\$AWS_KEY_NAME \")\w]${COLOR_RESET}\n> "

With that set I have a two line, dark grey prompt that will display which of my Amazon credentials I am currently using and will also display the exit code of my last command in red only if the last command exited with a non-zero status (thx to my buddy John for that idea). Here’s an example:

Now my shell is displaying things, but I also play around in the python interactive interpreter a lot as well, so it’d be good to see what credentials I was using in there as well. Fortunately you can modify that prompt by setting sys.ps1 in the ~/.pythonrc.py file. The trick is that we want the prompt to be dyamic - fortunately sys.ps1 allows you to do that by calling str() on any non-string object you give it. This means you can pass an object with a dynamic __str__ method. Here’s the excerpt from my .pythonrc.py that does this:

import os
import sys

class AWSCredentialsPrompt(object):
    def __init__(self, base_prompt):
        self.base = base_prompt

    def __str__(self):
        try:
            return "(aws:%s) %s " % (os.environ['AWS_KEY_NAME'], self.base)
        except KeyError:
            return "%s " % (self.base)

sys.ps1 = AWSCredentialsPrompt('>>>')
sys.ps2 = AWSCredentialsPrompt('...')

That’s it! If I were using the work credentials and I enter my python interpreter, here’s what you’d see:

So that’s all there is to it. I’m working on a few functions in python to act the same as the shell functions to switch credentials. It shouldn’t be too difficult.

Let me know if you have any questions or something doesn’t work for you. Oh, and if you have any ideas for enhancements leave them in the comments!

shortcut your email on iOS

Ok, this is probably something people have already thought to do, but I only today figured it out. What’s screwed up is that I already knew about the shortcut feature in iOS (I hated the fact it defaulted ‘omw’ to ‘On my way!’).

Basically I end up having to type my email address fairly often, and typing email addresses on the iPhone isn’t the easiest. Soooo, today I setup a couple of shortcuts. You can set these up by going into Settings > General > Keyboard and then clicking on Add New Shortcut.

I setup the following shortcuts:

@@

My personal email address

@w

My work email address

@p

My phone number

Now I can just type those whenever I need to give my email address.

I’m trying to think of other shortcuts - if you have any ideas, share them in the comments!

disabling aliases in pyYaml

Lately I’ve been working with a python script to act as an external node classifier for Puppet. Puppet expects the output of the ENC to be in YAML, so naturally I reached for PyYAML. Everything was going great, but then I tried to put the same bit of information in both the classes AND parameters section. See this script for example:

import yaml

services = {
        'enabled': ['nginx', 'webapp'],
        'disabled': ['postgres'],
}

data = {'classes': {'services': services}, 'parameters': services}

print yaml.safe_dump(data, default_flow_style=False)

When you run this script you end up with the following output:

classes:
  services: &id001
    disabled:
    - postgres
    enabled:
    - nginx
    - webapp
parameters: \*id001

The issue with this is the use of aliases. Basically since the data I had in the ‘services’ dictionary was the same as what I had in parameters pyYAML recognized that and created the alias &id001 pointing at that data, then referenced it rather than copying that data into parameters.

Unfortunately the version of puppet that I am running (2.7.11, the default in Ubuntu 12.04) can’t handle aliases, even though they’re a definitely a part of the YAML syntax. Honestly in most cases I’d be super happy that PyYAML was smart enough to do this for me automatically. But in this case it was causing issues.

After digging around, I found out a way to work around it. Unfortunately it’s not as simple as an argument to safe_dump. Instead you need to provide your own instance of a Dumper object, and then override ignore_aliases so that it always returns True.

Here’s the updated script:

import yaml

services = {
        'enabled': ['nginx', 'webapp'],
        'disabled': ['postgres'],
}

data = {'classes': {'services': services}, 'parameters': services}

noalias_dumper = yaml.dumper.SafeDumper
noalias_dumper.ignore_aliases = lambda self, data: True
print yaml.dump(data, default_flow_style=False, Dumper=noalias_dumper)

All the changes are in the last three lines. First I create an object which is a copy of the SafeDumper class. Then I patch the ignore_aliases method on that object to always return True. Finally I use dump instead of safe_dump (safe_dump just calls dump, but specifies the SafeDumper) and pass along the newly created dumper which is based on SafeDumper. In the end, this is the new output:

classes:
  services:
    disabled:
    - postgres
    enabled:
    - nginx
    - webapp
parameters:
  disabled:
  - postgres
  enabled:
  - nginx
  - webapp

And now puppet can read it just fine.