{"id":357,"date":"2016-05-14T20:14:33","date_gmt":"2016-05-14T20:14:33","guid":{"rendered":"http:\/\/tuxlabs.com\/?p=357"},"modified":"2017-11-11T08:54:41","modified_gmt":"2017-11-11T08:54:41","slug":"consul-for-service-discovery","status":"publish","type":"post","link":"https:\/\/tuxlabs.com\/?p=357","title":{"rendered":"Consul for Service Discovery"},"content":{"rendered":"

Why Service Discovery ?<\/h2>\n

Service Discovery effectively replaces the process of having to manually assign or automate your own DNS entries for nodes on your network. Service Discovery aims\u00a0to move even further away from treating VM’s like pets to cattle, by getting rid of the age old practice of Hostname & FQDN having contextual value. Instead when using services discovery nodes are automatically registered by an agent and automatically are configured in DNS for both nodes and services running on the machine.<\/p>\n

Consul<\/h2>\n

Consul by Hashicorp<\/a> is becoming the de-facto standard for Service Discovery. Consul’s full features & simplistic deployment model make it an optimal choice for organizations looking to quickly deploy Service Discovery capabilities in their environment.<\/p>\n

Components of Consul<\/h2>\n
    \n
  1. The Consul Agent<\/li>\n
  2. An optional JSON config file for each service located under \/etc\/consul.d\/<service>.json\n
      \n
    1. If you do not specific a JSON file, consul can still start and will provide discovery for the nodes (they will have DNS as well)<\/li>\n<\/ol>\n<\/li>\n<\/ol>\n

      A Quick Example of Consul<\/h2>\n

      How easy is it to deploy console ?<\/h3>\n
        \n
      1. Download \/ Decompress and install the Consul agent –\u00a0https:\/\/www.consul.io\/downloads.html<\/a><\/li>\n
      2. Define services in a JSON file (if you want) –\u00a0https:\/\/www.consul.io\/intro\/getting-started\/services.html<\/a><\/span><\/li>\n
      3. Start the agent on the nodes –\u00a0https:\/\/www.consul.io\/intro\/getting-started\/join.html<\/a><\/span><\/li>\n
      4. \u00a0<\/span>Make 1 node join 1 other node (does not matter which node) to join the cluster, which gets you access to all cluster metadata<\/span><\/li>\n<\/ol>\n

        Steps 1 and 2 Above<\/h4>\n
          \n
        1. After downloading the Consul binary to each machine and decompressing it, copy it to \/usr\/local\/bin\/ so it’s in your path.<\/li>\n
        2. Create the directory
          \n\n\n\n
          \n
          sudo mkdir \/etc\/consul.d<\/pre>\n<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/li>\n
        3. Optionally, run the following to create a JSON file defining a fake service running<\/li>\n<\/ol>\n\n\n\n
          \n
          echo '{\"service\": {\"name\": \"web\", \"tags\": [\"rails\"], \"port\": 80}}' \\\r\n    >\/etc\/consul.d\/web.json<\/pre>\n<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n
          Step 3 Above<\/h4>\n
          Run the agent on each node, changing IP accordingly.<\/p>\n\n\n\n
          \n
          tuxninja@consul-d415:~$ consul agent -server -bootstrap-expect 1 -data-dir \/tmp\/consul -node=agent-one -bind=10.73.172.110 -config-dir \/etc\/consul.d<\/pre>\n<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n
          Step 4 Above<\/h4>\n\n\n\n
          \n
          tuxninja@consul-d415:~$ consul join 10.73.172.108\r\nSuccessfully joined cluster by contacting 1 nodes.<\/pre>\n<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n
          Wow, simple…ok now for the examples….<\/p>\n
          Show cluster members<\/h3>\n\n\n\n
          \n
          tuxninja@consul-dcb3:~$ consul join 10.73.172.110\r\nSuccessfully joined cluster by contacting 1 nodes.<\/pre>\n<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n
          Look up DNS for a node<\/h3>\n\n\n\n
          \n
          tuxninja@consul-dcb3:~$ dig @127.0.0.1 -p 8600 agent-one.node.consul\r\n; <<>> DiG 9.9.5-3ubuntu0.8-Ubuntu <<>> @127.0.0.1 -p 8600 agent-one.node.consul\r\n; (1 server found)\r\n;; global options: +cmd\r\n;; Got answer:\r\n;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 2450\r\n;; flags: qr aa rd; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 0\r\n;; WARNING: recursion requested but not available\r\n;; QUESTION SECTION:\r\n;agent-one.node.consul.\t\tIN\tA\r\n;; ANSWER SECTION:\r\nagent-one.node.consul.\t0\tIN\tA\t10.73.172.110\r\n;; Query time: 1 msec\r\n;; SERVER: 127.0.0.1#8600(127.0.0.1)\r\n;; WHEN: Tue May 03 21:43:47 UTC 2016\r\n;; MSG SIZE  rcvd: 76\r\ntuxninja@consul-dcb3:~$<\/pre>\n<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n
          Lookup DNS for a service<\/h3>\n\n\n\n
          \n
          tuxninja@consul-dcb3:~$  dig @127.0.0.1 -p 8600 web.service.consul\r\n; <<>> DiG 9.9.5-3ubuntu0.8-Ubuntu <<>> @127.0.0.1 -p 8600 web.service.consul\r\n; (1 server found)\r\n;; global options: +cmd\r\n;; Got answer:\r\n;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 55798\r\n;; flags: qr aa rd; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 0\r\n;; WARNING: recursion requested but not available\r\n;; QUESTION SECTION:\r\n;web.service.consul.\t\tIN\tA\r\n;; ANSWER SECTION:\r\nweb.service.consul.\t0\tIN\tA\t10.73.172.110\r\n;; Query time: 2 msec\r\n;; SERVER: 127.0.0.1#8600(127.0.0.1)\r\n;; WHEN: Tue May 03 21:46:54 UTC 2016\r\n;; MSG SIZE  rcvd: 70\r\ntuxninja@consul-dcb3:~$<\/pre>\n<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n
          Query the REST API for Nodes<\/h3>\n\n\n\n
          \n
          tuxninja@consul-dcb3:~$ curl localhost:8500\/v1\/catalog\/nodes\r\n[{\"Node\":\"agent-one\",\"Address\":\"10.73.172.110\",\"TaggedAddresses\":{\"wan\":\"10.73.172.110\"},\"CreateIndex\":3,\"ModifyIndex\":1311},{\"Node\":\"agent-two\",\"Address\":\"10.73.172.108\",\"TaggedAddresses\":{\"wan\":\"10.73.172.108\"},\"CreateIndex\":1338,\"ModifyIndex\":1339}<\/pre>\n<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n
          Query the REST API for Services<\/h3>\n\n\n\n
          \n
          tuxninja@consul-dcb3:~$ curl http:\/\/localhost:8500\/v1\/catalog\/service\/web\r\n[{\"Node\":\"agent-one\",\"Address\":\"10.73.172.110\",\"ServiceID\":\"web\",\"ServiceName\":\"web\",\"ServiceTags\":[\"rails\"],\"ServiceAddress\":\"\",\"ServicePort\":80,\"ServiceEnableTagOverride\":false,\"CreateIndex\":5,\"ModifyIndex\":772}<\/pre>\n<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n","protected":false},"excerpt":{"rendered":"
          Why Service Discovery ? Service Discovery effectively replaces the process of having to manually assign or automate your own DNS entries for nodes on your network. Service Discovery aims\u00a0to move even further away from treating VM’s like pets to cattle, by getting rid of the age old practice of Hostname & FQDN having contextual value. […]<\/p>\n<\/a>","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[91,131,130,1,28,12],"tags":[132,135,113,137,136],"class_list":{"0":"post-357","1":"post","2":"type-post","3":"status-publish","4":"format-standard","6":"category-architecture","7":"category-aws","8":"category-cloud","9":"category-howtos","10":"category-openstack-howtos","11":"category-systems-administration","12":"tag-cloud","13":"tag-consul","14":"tag-dns","15":"tag-hashicorp","16":"tag-service-discovery","17":"h-entry","18":"hentry"},"_links":{"self":[{"href":"https:\/\/tuxlabs.com\/index.php?rest_route=\/wp\/v2\/posts\/357","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/tuxlabs.com\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/tuxlabs.com\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/tuxlabs.com\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/tuxlabs.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=357"}],"version-history":[{"count":8,"href":"https:\/\/tuxlabs.com\/index.php?rest_route=\/wp\/v2\/posts\/357\/revisions"}],"predecessor-version":[{"id":683,"href":"https:\/\/tuxlabs.com\/index.php?rest_route=\/wp\/v2\/posts\/357\/revisions\/683"}],"wp:attachment":[{"href":"https:\/\/tuxlabs.com\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=357"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/tuxlabs.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=357"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/tuxlabs.com\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=357"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}