Consul for Service Discovery

Why Service Discovery ?

Service Discovery effectively replacing having to manually assign or automate your own DNS entries for nodes on your network. Service Discovery aims to move even further away from treating VM’s like pets to cattle, by getting rid of the age old practice Hostname & FQDN having contextual value. Instead when using services discovery nodes are automatically registered by an agent and automatically are configured in DNS for both nodes and services running on the machine.


Consul by Hashicorp is becoming the de-facto standard for Service Discovery. Consul’s full features & simplistic deployment model make it an optimal choice for organizations looking to quickly deploy Service Discovery capabilities in their environment.

Components of Consul

  1. The Consul Agent
  2. An optional JSON config file for each service located under /etc/consul.d/<service>.json
    1. If you do not specific a JSON file, consul can still start and will provide discovery for the nodes (they will have DNS as well)

A Quick Example of Consul

How easy is it to deploy console ?

  1. Download / Decompress and install the Consul agent –
  2. Define services in a JSON file (if you want) –
  3. Start the agent on the nodes –
  4.  Make 1 node join 1 other node (does not matter which node) to join the cluster, which gets you access to all cluster metadata

Steps 1 and 2 Above

  1. After downloading the Consul binary to each machine and decompressing it, copy it to /usr/local/bin/ so it’s in your path.
  2. Create the directory
  3. Optionally, run the following to create a JSON file defining a fake service running

Step 3 Above

Run the agent on each node, changing IP accordingly.

Step 4 Above

Wow, simple…ok now for the examples….

Show cluster members

Look up DNS for a node

Lookup DNS for a service

Query the REST API for Nodes

Query the REST API for Services

Fun with Python, Tabular & AWS IP ranges

I have been spending a lot of time designing a Hybrid Cloud that consists of Openstack and public cloud platforms. In particular I have been spending a lot of time designing the AWS portion of the Hybrid Cloud Platform. Today I found myself continually needing to look up AWS public address space and then parsing out regions & services. Then I remembered something a mentor of mine told me…

if you are going to do something more than once, there is probably value in automating it.

I love writing command line tools and thus a short Python script was born. Since I rarely share Python code, even though I didn’t spend a lot time on this, and I certainly didn’t optimize it for DRY etc. I am sharing it anyway for others to use, enjoy and hack on,

but mainly to learn, which is…The entire purpose of the Tuxlabs site

I should mention I have strong views about when to use Python vs. Go a language I find myself writing in more and more and this tool falls under my rules for things that I should write in Go. So later as a follow up I will likely re-code this in Go and post the code for review & learning. For now here’s the Python code, enjoy !

Listing all IP Ranges


The code

How To: curl the Openstack API’s (v3 Keystone Auth)

While Openstack provides a python client(s) for interactions….

I frequently, finding myself needing to get data out of it without the pain of awk/sed’ing out the ASCII art.

Thus to quickly access the raw data, we can directly query the API’s using curl & parsing JSON instead, which is much better 🙂


Before we can interact with the other Openstack API’s we need to authenticate to Keystone openstack’s identity service. After authenticating we receive a token to use with our subequent API requests. So step 1 we are going to create a JSON object with the required authentication details.

Create a file called ‘token-request.json’ with an object that looks like this.

Btw, if you followed my tutorial on how to install Openstack Kilo, your authentication details for ‘admin’ is in your keystonerc_admin file.

Now we can use this file to authenticate like so:

The token is actually returned in the header of the HTTP response, so this is why we need ‘-i’ when curling. Notice we are parsing out the token and returning the value to an environment variable $TOKEN.

Now we can include this $TOKEN and run whatever API commands we want (assuming admin privileges for the tenant/project)

Curl Commands (Numerous Examples!)

I sometimes pipe the output to python -m json.tool, which provides formatting for JSON. Lets take a closer look at an example.

Listing servers (vm’s)

I only have 1 VM currently called spin1, but for the tutorials sake, if I had ten’s or hundred’s of VM’s and all I cared about was the VM name or ID, I would still need to parse this JSON object to avoid getting all this other meta-data.

My favorite command line way to do that without going full Python is using the handy JQ tool.

Here is how to use it !

The first command just takes whatever the STDOUT from curl is and indent’s and color’s the JSON making it pretty (colors gives it +1 vs. python -m json.tool).

The second example we actually parse what were after. As you can see it is pretty simple, but jq’s query language may not be 100% intuitive at first, but I promise it is pretty easy to understand if you have ever parsed JSON before. Read up more on JQ @ & check out the Openstack docs for more API commands

Hope you enjoyed this post ! Until next time.



Installing Openstack Kilo on Centos 7

In a previous article I wrote about how to install Openstack Icehouse on CentOS 6.5 in great detail. In this article, I am going to keep verbosity to a minimum and just give you the commands ! I am hoping this will be refreshing for my audience. If you are curious however, about the what, when and why please read my previous article.


  1. You need a machine with x86_64 architecture with at least 4 GB of memory & 2 NIC’s.
  2. On this machine you need to install CentOS 7 as a minimal install
  3. You should create a user with admin privileges (i.e. wheel, in my case ‘tuxninja’ was created)
  4. Disable SELinux
    1. vi /etc/sysconfig/selinux
    2. SELINUX=disabled
    3. save changes

Jumping Right In

Here are the commands you need to run.

  1. sudo yum update -y
  2. sudo yum install -y
  3. sudo yum install epel-release
  4. sudo yum install -y openstack-packstack

Now at this point if you ran ‘packstack’ you would run into a bug with this message

The workaround for this bug is as follows

  1. sudo rpm -e puppet
  2. sudo rpm rpm -e hiera
  3. curl -O
  4. sudo rpm -ivh hiera-1.3.4-1.el7.noarch.rpm
  5. vi /etc/yum.repos.d/epel.repo
    1. At the bottom of the [epel] section, after the gpgkey add a newline with: exclude=hiera*
    2. Save the file
  6. sud0 yum install -y puppet-3.6.2-3.el7.noarch
  7. reboot
  8. sudo rm /etc/puppet/hiera.yaml
  9. sudo packstack –allinone

This should successfully install. Godspeed.


Now that Openstack is setup, we still have to setup our network with private & public routed networks, so we can turn this into a real multi-node setup and ssh to our hosts and let them reach the internet etc. To do this, much like my previous post you need to modify your /etc/sysconfig/network-scripts/ files to reflect this.

Note: I deleted all the IPV6 crap, I think it messes some stuff up. When your done making the changes with your favorite editor, restart networking : sudo /etc/init.d/network restart or sudo systemctl restart network

Next go into in the Horizon Dashboard GUI and delete the demo project. See my previous article for details on how.

Back On the All-In-One Node Console

Next ‘reboot’ or restart all openstack services :

Note: it appears the –full-restart flag is gone, used to work !

When logging into your dashboard located at at some point you might hit a bug that prevent you from logging into the Horizon dashboard see : … the work-around for this is to clear your browser cookies.

You’re Done

That’s it. Next steps would be to create a project & new admin user, re-create the required network mappings in openstack using the above commands (modify the names to make them unique) and create your ssh key, import it, download some images, import them using glance, and create some VM’s. Also I like to delete the demo project (you can also prevent this from being created with a flag on the packstack command). Make sure you delete all default security rules and add back ICMP, TCP, and UDP allow ingress / egress rules for aka any/any, again you can see my article on CentOS 6.5 with more specifics on how to do this. Additionally, I have an article on how to add additional compute nodes as well.

As always I can be reached for assistance @ tuxninja [at]

Happying Stacking !

Creating a bootable USB for Centos on Mac OS X

I’m a huge Ubuntu fan. However, most of my ‘day job’ work requires CentOS or RHEL, thus I commonly have to re-image my on premise Cloud with the latest and greatest CentOS. My servers are 3 Rackables by SGI, two with more CPU & Memory and one (the controller node) with tons of Disk (12x1TB RAID 10) and then for off-premise I use Digital Ocean who has a fantastic product. Most modern servers do not have a CDROM and neither do my on premise systems. Therefore, I need to place the CentOS image on a USB drive so I can re-image my lab. Here are the steps do that on Mac OS X.

List the current Disks & Partitions

My USB drive is the 2GB drive at the bottom, we need to unmount that

Next we copy the CentOS image onto the unmounted USB disk.

When that’s gets done your Mac will pop up a window asking you to initialize the drive, ignore that. Remove it, and your ready to boot off this USB!