{"id":436,"date":"2016-12-10T15:05:51","date_gmt":"2016-12-10T15:05:51","guid":{"rendered":"http:\/\/tuxlabs.com\/?p=436"},"modified":"2016-12-10T15:15:31","modified_gmt":"2016-12-10T15:15:31","slug":"setting-up-netflixs-edda-cmdb-in-aws-on-ubuntu","status":"publish","type":"post","link":"https:\/\/tuxlabs.com\/?p=436","title":{"rendered":"Setting up Netflix’s Edda (CMDB) in AWS on Ubuntu"},"content":{"rendered":"

If you are running any kind of environment with greater than 10 servers, than you need a CMDB<\/a> (Configuration Management DataBase). CMDB’s are the brain of your fleet & it’s environment. You can store anything in a CMDB, but commonly the metadata in CMDB’s consists of any of the following physical & digital asset inventory, software licenses, software configuration data, policy information, relationships (I.E. This VM—> Compute –> Rack –> Availability Zone –> Datacenter), automation metadata, and more… they also commonly provide change history for changes in your environment.<\/p>\n

In the world of infrastructure as code, CMDB is king.<\/p><\/blockquote>\n

CMDB’s\u00a0enable endless automation possibilities, without them you are stuck gathering and collecting ‘current’ configuration state\u00a0about your infrastructure every time you want perform an automated change or run an audit\/report . In my career I have\u00a0built or been a part of CMDB efforts at nearly every company I have worked for. They are simply necessary, and by their nature they tend to require the choice of ‘built by us’ vs ‘buy or run’.<\/p>\n

However, if you have the luxury of only running in AWS, you are in luck, because Netflix (The AWS poster child) \u00a0open sourced\u00a0Edda<\/a>\u00a0in 2012<\/a> for this purpose!<\/p>\n

Rather than talk about the specific features of Edda refer to the blog post<\/a> or documentation<\/a>, I want to keep this article short and jump right into setting up Edda, which is a bit tricky, because the documentation is out of date!<\/p>\n

Setting Up Edda (2016)<\/h2>\n

First, in AWS you need setup an EC2 VM that has at least.. 6G for OS + dependencies including Mongo, and then however much disk you need to store the metadata for your environment (keep in mind it keeps change history). Personally I just created a root partition with 100G to keep things simple. For instance type I used ‘m4.xlarge’ and\u00a0the Ubuntu version is 14.04.<\/p>\n

After booting the VM, SSH to it and create a directory\u00a0wherever your storage is allocated partition wise to store Edda & it’s dependencies. I will be using \/cmdb\/ in my example.<\/p>\n

Initial Install Steps<\/h3>\n
mkdir \/cmdb\r\ncd \/cmdb\r\nexport JAVA_OPTS=\"-Xmx1g -XX:MaxPermSize=256M\"\r\ngit clone https:\/\/github.com\/Netflix\/edda.git\r\nsudo add-apt-repository -y ppa:webupd8team\/java &> \/dev\/null\r\nsudo apt-get update\r\nsudo debconf-set-selections <<< 'oracle-java8-installer shared\/accepted-oracle-license-v1-1 boolean true'\r\nsudo apt-get install -y oracle-java8-installer\r\nsudo apt-get install -y scala\r\nsudo apt-get install make\r\n\r\ncd \/cmdb\/edda\r\nmake build<\/pre>\n
For the record, the Edda Wiki<\/a> has the build steps wrong, it appears they no long are using Gradle, but have switch to SBT<\/a>… which reminds me be aware Edda is written in Scala, which isn’t as popular as Java, Python etc… in addition it’s functional programming, which I don’t personally know a lot about, but I hear it’s got quite the learning curve..so beware if you need to make custom code changes, I would not recommend it, unless you know Scala ! \ud83d\ude42<\/p>\n
After the build of Edda succeeds, install Mongo<\/p>\n
apt-get install -y mongodb\r\n<\/pre>\n
That’s it for dependencies<\/p>\n
Configuring Mongo<\/h3>\n
For Edda to use Mongo all we need to do is ‘use’\u00a0the database we want to use for Edda & create an associated user. (Mongo will auto-create DB’s upon insert).<\/p>\n
mongo\r\n\r\n> use edda\r\n> db.addUser({user:'edda',pwd:'t00t0ri4l',Roles: { edda: ['readWrite']}, roles: []})<\/pre>\n
You can test the user is working by doing…\u00a0<\/strong><\/p>\n
$ mongo edda -u edda -p\r\nMongoDB shell version: 2.4.9\r\nEnter password:\r\nconnecting to: edda\r\nServer has startup warnings:\r\nSat Dec 10 00:53:21.093 [initandlisten]\r\nSat Dec 10 00:53:21.094 [initandlisten] ** WARNING: You are running on a NUMA machine.\r\nSat Dec 10 00:53:21.094 [initandlisten] **          We suggest launching mongod like this to avoid performance problems:\r\nSat Dec 10 00:53:21.094 [initandlisten] **              numactl --interleave=all mongod [other options]\r\nSat Dec 10 00:53:21.094 [initandlisten]\r\n><\/pre>\n
Configuring Edda<\/h3>\n
Under \/cmdb\/edda\/src\/main\/resources we need to modify ‘edda.properties’ with valid config values for accounts, regions & mongo access.<\/p>\n
Relevant Mongo Values<\/strong><\/p>\n
edda.mongo.address=127.0.0.1:27017\r\nedda.mongo.database=edda\r\nedda.mongo.user=edda\r\nedda.mongo.password=t00t0ri4l<\/pre>\n
Account & Region Values\u00a0<\/strong><\/p>\n
edda.accounts=dev.us-east-1\r\nedda.dev.us-east-1.region=us-east-1\r\nedda.dev.us-east-1.aws.accessKey=fakeaccesskey\r\nedda.dev.us-east-1.aws.secretKey=fakesecret<\/pre>\n
The above example is using one account and only one region. The Edda configuration uses generic labels, they are very flexible, but when using them you might be confused by the name of the label as it’s intent. Don’t fall into that trap, I did, and then I found this post<\/a> on Google Groups… Check it out to gain more insight on how the configuration works and can be tweaked for \u00a0your needs. There is also the standard documentation<\/a>, but it’s a little light IMO.<\/p>\n
Running Edda<\/h3>\n
Congrats you made it, time to run Edda ! Again the documentation has this wrong (listed as gradle & Jetty)…instead were using SBT + Jetty…<\/p>\n
$ cd \/cmdb\/edda\/\r\n$ .\/project\/sbt\r\n> jetty:start<\/pre>\n
If everything goes smoothly you will start to see logs about crawling\u00a0AWS API’s spewing to your screen \ud83d\ude42 After about 2 minutes you should see data. You can check by doing a curl.<\/p>\n
curl http:\/\/127.0.0.1:8080\/api\/v2\/view\/instances<\/pre>\n
This API URL should return a JSON object with instance ID’s for the account & region specified.<\/p>\n
Additionally, Edda is listening on whatever private IP address you have setup, you will just need to modify the default security group to allow 8080 on your machine.<\/p>\n
I get a bit frustrated with out of date documentation..so I hope this helps ! Happy automating !<\/p>\n","protected":false},"excerpt":{"rendered":"
If you are running any kind of environment with greater than 10 servers, than you need a CMDB (Configuration Management DataBase). CMDB’s are the brain of your fleet & it’s environment. You can store anything in a CMDB, but commonly the metadata in CMDB’s consists of any of the following physical & digital asset inventory, […]<\/p>\n<\/a>","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[131,130,1],"tags":[23,155,154,153,104],"class_list":{"0":"post-436","1":"post","2":"type-post","3":"status-publish","4":"format-standard","6":"category-aws","7":"category-cloud","8":"category-howtos","9":"tag-aws","10":"tag-cmdb","11":"tag-edda","12":"tag-netflix","13":"tag-ubuntu","14":"h-entry","15":"hentry"},"_links":{"self":[{"href":"https:\/\/tuxlabs.com\/index.php?rest_route=\/wp\/v2\/posts\/436","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/tuxlabs.com\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/tuxlabs.com\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/tuxlabs.com\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/tuxlabs.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=436"}],"version-history":[{"count":9,"href":"https:\/\/tuxlabs.com\/index.php?rest_route=\/wp\/v2\/posts\/436\/revisions"}],"predecessor-version":[{"id":445,"href":"https:\/\/tuxlabs.com\/index.php?rest_route=\/wp\/v2\/posts\/436\/revisions\/445"}],"wp:attachment":[{"href":"https:\/\/tuxlabs.com\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=436"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/tuxlabs.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=436"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/tuxlabs.com\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=436"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}