In the video below I walk through how to parse JSON data from a REST API using Python and the Requests library and json standard library for Python. I also drop a little nugget on duck typing. Hope you enjoy!
Tag: JSON
Fun with Python, Tabular & AWS IP ranges
I have been spending a lot of time designing a Hybrid Cloud that consists of Openstack and public cloud platforms. In particular I have been spending a lot of time designing the AWS portion of the Hybrid Cloud Platform. Today I found myself continually needing to look up AWS public address space and then parsing out regions & services. Then I remembered something a mentor of mine told me…
if you are going to do something more than once, there is probably value in automating it.
I love writing command line tools and thus a short Python script was born. Since I rarely share Python code, even though I didn’t spend a lot time on this, and I certainly didn’t optimize it for DRY etc. I am sharing it anyway for others to use, enjoy and hack on,
but mainly to learn, which is…The entire purpose of the Tuxlabs site
I should mention I have strong views about when to use Python vs. Go a language I find myself writing in more and more and this tool falls under my rules for things that I should write in Go. So later as a follow up I will likely re-code this in Go and post the code for review & learning. For now here’s the Python code, enjoy !
Listing all IP Ranges
(env) ➜ aws python aws-ranges.py region ip_prefix service -------------- ----------------- -------------------- us-east-1 23.20.0.0/14 AMAZON ap-northeast-1 27.0.0.0/22 AMAZON ap-southeast-1 43.250.192.0/24 AMAZON ap-southeast-1 43.250.193.0/24 AMAZON eu-west-1 46.51.128.0/18 AMAZON eu-west-1 46.51.192.0/20 AMAZON ap-southeast-1 46.51.216.0/21 AMAZON ap-northeast-1 46.51.224.0/19 AMAZON eu-west-1 46.137.0.0/17 AMAZON eu-west-1 46.137.128.0/18 AMAZON ap-southeast-1 46.137.192.0/19 AMAZON ap-southeast-1 46.137.224.0/19 AMAZON us-east-1 50.16.0.0/15 AMAZON us-west-1 50.18.0.0/16 AMAZON us-east-1 50.19.0.0/16 AMAZON us-west-2 50.112.0.0/16 AMAZON us-east-1 52.0.0.0/15 AMAZON us-east-1 52.2.0.0/15 AMAZON us-east-1 52.4.0.0/14 AMAZON us-west-1 52.8.0.0/16 AMAZON us-west-1 52.9.0.0/16 AMAZON us-west-2 52.10.0.0/15 AMAZON us-west-2 52.12.0.0/15 AMAZON eu-west-1 52.16.0.0/15 AMAZON eu-west-1 52.18.0.0/15 AMAZON us-east-1 52.20.0.0/14 AMAZON us-west-2 52.24.0.0/14 AMAZON eu-central-1 52.28.0.0/16 AMAZON eu-central-1 52.29.0.0/16 AMAZON eu-west-1 52.30.0.0/15 AMAZON us-west-2 52.32.0.0/14 AMAZON us-west-2 52.36.0.0/14 AMAZON us-west-2 52.40.0.0/14 AMAZON eu-west-1 52.48.0.0/14 AMAZON us-west-1 52.52.0.0/15 AMAZON eu-central-1 52.58.0.0/15 AMAZON ap-southeast-2 52.62.0.0/15 AMAZON ap-southeast-2 52.64.0.0/17 AMAZON ap-southeast-2 52.64.128.0/17 AMAZON ap-southeast-2 52.65.0.0/16 AMAZON sa-east-1 52.67.0.0/16 AMAZON ap-northeast-1 52.68.0.0/15 AMAZON us-east-1 52.70.0.0/15 AMAZON us-east-1 52.72.0.0/15 AMAZON ap-southeast-1 52.74.0.0/16 AMAZON ap-southeast-1 52.76.0.0/17 AMAZON ap-southeast-1 52.76.128.0/17 AMAZON ap-southeast-1 52.77.0.0/16 AMAZON ap-northeast-2 52.79.0.0/16 AMAZON GLOBAL 52.84.0.0/15 AMAZON us-east-1 52.86.0.0/15 AMAZON us-west-2 52.88.0.0/15 AMAZON us-east-1 52.90.0.0/15 AMAZON ap-northeast-2 52.92.0.0/20 AMAZON us-east-1 52.92.16.0/20 AMAZON us-west-2 52.92.32.0/22 AMAZON sa-east-1 52.92.39.0/24 AMAZON eu-west-1 52.92.40.0/21 AMAZON us-west-1 52.92.48.0/22 AMAZON ap-southeast-2 52.92.52.0/22 AMAZON ap-southeast-1 52.92.56.0/22 AMAZON ap-northeast-1 52.92.60.0/22 AMAZON sa-east-1 52.92.64.0/22 AMAZON eu-central-1 52.92.68.0/22 AMAZON sa-east-1 52.92.72.0/22 AMAZON us-gov-west-1 52.92.252.0/22 AMAZON eu-west-1 52.93.0.0/24 AMAZON us-east-1 52.93.1.0/24 AMAZON eu-west-1 52.93.2.0/24 AMAZON us-east-1 52.93.3.0/24 AMAZON ap-southeast-1 52.93.8.0/22 AMAZON us-east-1 52.94.0.0/22 AMAZON eu-west-1 52.94.5.0/24 AMAZON ap-northeast-2 52.94.6.0/24 AMAZON sa-east-1 52.94.7.0/24 AMAZON ap-northeast-1 52.94.8.0/24 AMAZON us-gov-west-1 52.94.9.0/24 AMAZON us-west-2 52.94.10.0/24 AMAZON ap-southeast-1 52.94.11.0/24 AMAZON us-west-1 52.94.12.0/24 AMAZON ap-southeast-2 52.94.13.0/24 AMAZON us-west-1 52.94.14.0/24 AMAZON us-east-1 52.94.254.0/23 AMAZON ap-northeast-1 52.95.30.0/23 AMAZON ap-northeast-1 52.95.34.0/24 AMAZON ap-southeast-1 52.95.35.0/24 AMAZON ap-southeast-2 52.95.36.0/22 AMAZON us-east-1 52.95.48.0/22 AMAZON us-east-1 52.95.52.0/22 AMAZON ap-northeast-1 52.95.56.0/22 AMAZON eu-west-1 52.95.60.0/24 AMAZON eu-west-1 52.95.61.0/24 AMAZON us-east-1 52.95.62.0/24 AMAZON us-east-1 52.95.63.0/24 AMAZON ap-northeast-2 52.95.192.0/20 AMAZON ap-southeast-1 52.95.212.0/22 AMAZON sa-east-1 52.95.240.0/24 AMAZON ap-southeast-2 52.95.241.0/24 AMAZON ap-southeast-1 52.95.242.0/24 AMAZON ap-northeast-1 52.95.243.0/24 AMAZON eu-west-1 52.95.244.0/24 AMAZON us-east-1 52.95.245.0/24 AMAZON us-west-1 52.95.246.0/24 AMAZON us-west-2 52.95.247.0/24 AMAZON eu-central-1 52.95.248.0/24 AMAZON cn-north-1 52.95.249.0/24 AMAZON ap-northeast-2 52.95.252.0/24 AMAZON sa-east-1 52.95.255.0/28 AMAZON ap-southeast-2 52.95.255.16/28 AMAZON ap-southeast-1 52.95.255.32/28 AMAZON ap-northeast-1 52.95.255.48/28 AMAZON eu-west-1 52.95.255.64/28 AMAZON us-east-1 52.95.255.80/28 AMAZON us-west-1 52.95.255.96/28 AMAZON us-west-2 52.95.255.112/28 AMAZON eu-central-1 52.95.255.128/28 AMAZON cn-north-1 52.95.255.144/28 AMAZON ap-northeast-1 52.192.0.0/15 AMAZON ap-northeast-1 52.196.0.0/14 AMAZON us-east-1 52.200.0.0/13 AMAZON eu-west-1 52.208.0.0/13 AMAZON ap-southeast-1 52.220.0.0/15 AMAZON ap-northeast-1 54.64.0.0/15 AMAZON ap-southeast-2 54.66.0.0/16 AMAZON us-west-1 54.67.0.0/16 AMAZON us-west-2 54.68.0.0/14 AMAZON eu-west-1 54.72.0.0/15 AMAZON eu-west-1 54.74.0.0/15 AMAZON eu-west-1 54.76.0.0/15 AMAZON eu-west-1 54.78.0.0/16 AMAZON ap-southeast-2 54.79.0.0/16 AMAZON us-east-1 54.80.0.0/13 AMAZON us-east-1 54.88.0.0/14 AMAZON ap-northeast-1 54.92.0.0/17 AMAZON us-east-1 54.92.128.0/17 AMAZON eu-central-1 54.93.0.0/16 AMAZON sa-east-1 54.94.0.0/16 AMAZON ap-northeast-1 54.95.0.0/16 AMAZON us-east-1 54.144.0.0/14 AMAZON us-west-2 54.148.0.0/15 AMAZON ap-northeast-1 54.150.0.0/16 AMAZON us-west-1 54.151.0.0/17 AMAZON ap-southeast-1 54.151.128.0/17 AMAZON us-east-1 54.152.0.0/16 AMAZON us-west-1 54.153.0.0/17 AMAZON ap-southeast-2 54.153.128.0/17 AMAZON eu-west-1 54.154.0.0/16 AMAZON eu-west-1 54.155.0.0/16 AMAZON us-east-1 54.156.0.0/14 AMAZON us-east-1 54.160.0.0/13 AMAZON ap-northeast-1 54.168.0.0/16 AMAZON ap-southeast-1 54.169.0.0/16 AMAZON eu-west-1 54.170.0.0/15 AMAZON us-east-1 54.172.0.0/15 AMAZON us-east-1 54.174.0.0/15 AMAZON us-west-1 54.176.0.0/15 AMAZON ap-northeast-1 54.178.0.0/16 AMAZON ap-southeast-1 54.179.0.0/16 AMAZON GLOBAL 54.182.0.0/16 AMAZON us-west-1 54.183.0.0/16 AMAZON us-west-2 54.184.0.0/13 AMAZON GLOBAL 54.192.0.0/16 AMAZON us-west-1 54.193.0.0/16 AMAZON eu-west-1 54.194.0.0/15 AMAZON us-east-1 54.196.0.0/15 AMAZON us-east-1 54.198.0.0/16 AMAZON ap-northeast-1 54.199.0.0/16 AMAZON us-west-2 54.200.0.0/15 AMAZON us-west-2 54.202.0.0/15 AMAZON us-east-1 54.204.0.0/15 AMAZON ap-southeast-2 54.206.0.0/16 AMAZON sa-east-1 54.207.0.0/16 AMAZON us-east-1 54.208.0.0/15 AMAZON us-east-1 54.210.0.0/15 AMAZON us-west-2 54.212.0.0/15 AMAZON us-west-2 54.214.0.0/16 AMAZON us-west-1 54.215.0.0/16 AMAZON eu-west-1 54.216.0.0/15 AMAZON us-west-2 54.218.0.0/16 AMAZON us-west-1 54.219.0.0/16 AMAZON eu-west-1 54.220.0.0/16 AMAZON us-east-1 54.221.0.0/16 AMAZON cn-north-1 54.222.0.0/19 AMAZON cn-north-1 54.222.128.0/17 AMAZON cn-north-1 54.223.0.0/16 AMAZON us-east-1 54.224.0.0/15 AMAZON us-east-1 54.226.0.0/15 AMAZON eu-west-1 54.228.0.0/16 AMAZON eu-west-1 54.229.0.0/16 AMAZON GLOBAL 54.230.0.0/16 AMAZON us-east-1 54.231.0.0/17 AMAZON eu-west-1 54.231.128.0/19 AMAZON us-west-2 54.231.160.0/19 AMAZON eu-central-1 54.231.192.0/20 AMAZON cn-north-1 54.231.208.0/20 AMAZON ap-northeast-1 54.231.224.0/21 AMAZON us-west-1 54.231.232.0/21 AMAZON ap-southeast-1 54.231.240.0/22 AMAZON us-east-1 54.231.244.0/22 AMAZON ap-southeast-2 54.231.248.0/22 AMAZON ap-southeast-2 54.231.252.0/24 AMAZON sa-east-1 54.231.253.0/24 AMAZON us-gov-west-1 54.231.254.0/24 AMAZON sa-east-1 54.232.0.0/16 AMAZON sa-east-1 54.233.0.0/18 AMAZON sa-east-1 54.233.64.0/18 AMAZON sa-east-1 54.233.128.0/17 AMAZON us-east-1 54.234.0.0/15 AMAZON us-east-1 54.236.0.0/15 AMAZON ap-northeast-1 54.238.0.0/16 AMAZON us-west-2 54.239.2.0/23 AMAZON eu-central-1 54.239.4.0/22 AMAZON us-east-1 54.239.8.0/21 AMAZON us-east-1 54.239.16.0/20 AMAZON eu-west-1 54.239.32.0/21 AMAZON us-west-2 54.239.48.0/22 AMAZON ap-northeast-1 54.239.52.0/23 AMAZON eu-central-1 54.239.54.0/23 AMAZON eu-central-1 54.239.56.0/21 AMAZON ap-northeast-1 54.239.96.0/24 AMAZON us-east-1 54.239.98.0/24 AMAZON eu-west-1 54.239.99.0/24 AMAZON eu-west-1 54.239.100.0/23 AMAZON us-east-1 54.239.104.0/23 AMAZON us-east-1 54.239.108.0/22 AMAZON eu-west-1 54.239.114.0/24 AMAZON ap-northeast-2 54.239.116.0/22 AMAZON ap-northeast-2 54.239.120.0/21 AMAZON GLOBAL 54.239.128.0/18 AMAZON GLOBAL 54.239.192.0/19 AMAZON GLOBAL 54.240.128.0/18 AMAZON ap-southeast-2 54.240.192.0/22 AMAZON us-east-1 54.240.196.0/24 AMAZON eu-west-1 54.240.197.0/24 AMAZON us-west-1 54.240.198.0/24 AMAZON ap-southeast-1 54.240.199.0/24 AMAZON ap-northeast-1 54.240.200.0/24 AMAZON us-east-1 54.240.202.0/24 AMAZON ap-southeast-2 54.240.203.0/24 AMAZON ap-southeast-2 54.240.204.0/22 AMAZON us-east-1 54.240.208.0/22 AMAZON us-west-1 54.240.212.0/22 AMAZON us-east-1 54.240.216.0/22 AMAZON eu-west-1 54.240.220.0/22 AMAZON ap-northeast-1 54.240.225.0/24 AMAZON ap-southeast-1 54.240.226.0/24 AMAZON ap-southeast-1 54.240.227.0/24 AMAZON us-east-1 54.240.228.0/23 AMAZON us-west-2 54.240.230.0/23 AMAZON us-east-1 54.240.232.0/22 AMAZON ap-northeast-2 54.240.236.0/22 AMAZON eu-central-1 54.240.240.0/24 AMAZON sa-east-1 54.240.244.0/22 AMAZON us-west-2 54.240.248.0/21 AMAZON us-west-1 54.241.0.0/16 AMAZON us-east-1 54.242.0.0/15 AMAZON us-west-2 54.244.0.0/16 AMAZON us-west-2 54.245.0.0/16 AMAZON eu-west-1 54.246.0.0/16 AMAZON eu-west-1 54.247.0.0/16 AMAZON ap-northeast-1 54.248.0.0/15 AMAZON ap-northeast-1 54.250.0.0/16 AMAZON ap-southeast-1 54.251.0.0/16 AMAZON ap-southeast-2 54.252.0.0/16 AMAZON ap-southeast-2 54.253.0.0/16 AMAZON ap-southeast-1 54.254.0.0/16 AMAZON ap-southeast-1 54.255.0.0/16 AMAZON us-east-1 67.202.0.0/18 AMAZON us-east-1 72.21.192.0/19 AMAZON us-east-1 72.44.32.0/19 AMAZON us-east-1 75.101.128.0/17 AMAZON eu-west-1 79.125.0.0/17 AMAZON eu-west-1 87.238.80.0/21 AMAZON us-gov-west-1 96.127.0.0/17 AMAZON ap-northeast-1 103.4.8.0/21 AMAZON ap-southeast-1 103.246.148.0/23 AMAZON ap-northeast-1 103.246.150.0/23 AMAZON us-east-1 107.20.0.0/14 AMAZON ap-southeast-1 122.248.192.0/18 AMAZON us-east-1 172.96.97.0/24 AMAZON us-east-1 174.129.0.0/16 AMAZON ap-southeast-1 175.41.128.0/18 AMAZON ap-northeast-1 175.41.192.0/18 AMAZON ap-northeast-1 176.32.64.0/19 AMAZON us-east-1 176.32.96.0/21 AMAZON eu-west-1 176.32.104.0/21 AMAZON us-west-1 176.32.112.0/21 AMAZON us-east-1 176.32.120.0/22 AMAZON us-west-2 176.32.125.0/25 AMAZON ap-northeast-1 176.34.0.0/19 AMAZON ap-northeast-1 176.34.32.0/19 AMAZON eu-west-1 176.34.64.0/18 AMAZON eu-west-1 176.34.128.0/17 AMAZON sa-east-1 177.71.128.0/17 AMAZON sa-east-1 177.72.240.0/21 AMAZON eu-west-1 178.236.0.0/20 AMAZON us-west-1 184.72.0.0/18 AMAZON us-east-1 184.72.64.0/18 AMAZON us-east-1 184.72.128.0/17 AMAZON us-east-1 184.73.0.0/16 AMAZON us-west-1 184.169.128.0/17 AMAZON eu-west-1 185.48.120.0/22 AMAZON ap-southeast-1 203.83.220.0/22 AMAZON us-west-1 204.236.128.0/18 AMAZON us-east-1 204.236.192.0/18 AMAZON us-west-1 204.246.160.0/22 AMAZON GLOBAL 204.246.164.0/22 AMAZON GLOBAL 204.246.168.0/22 AMAZON GLOBAL 204.246.174.0/23 AMAZON GLOBAL 204.246.176.0/20 AMAZON GLOBAL 205.251.192.0/19 AMAZON us-east-1 205.251.224.0/22 AMAZON us-west-1 205.251.228.0/22 AMAZON us-west-2 205.251.232.0/22 AMAZON us-gov-west-1 205.251.236.0/22 AMAZON us-east-1 205.251.240.0/22 AMAZON us-east-1 205.251.244.0/23 AMAZON us-east-1 205.251.247.0/24 AMAZON us-east-1 205.251.248.0/24 AMAZON GLOBAL 205.251.249.0/24 AMAZON GLOBAL 205.251.250.0/23 AMAZON GLOBAL 205.251.252.0/23 AMAZON GLOBAL 205.251.254.0/24 AMAZON us-east-1 205.251.255.0/24 AMAZON us-east-1 207.171.160.0/20 AMAZON us-east-1 207.171.176.0/20 AMAZON GLOBAL 216.137.32.0/19 AMAZON us-east-1 216.182.224.0/20 AMAZON us-east-1 23.20.0.0/14 EC2 eu-west-1 46.51.128.0/18 EC2 eu-west-1 46.51.192.0/20 EC2 ap-southeast-1 46.51.216.0/21 EC2 ap-northeast-1 46.51.224.0/19 EC2 eu-west-1 46.137.0.0/17 EC2 eu-west-1 46.137.128.0/18 EC2 ap-southeast-1 46.137.192.0/19 EC2 ap-southeast-1 46.137.224.0/19 EC2 us-east-1 50.16.0.0/15 EC2 us-west-1 50.18.0.0/16 EC2 us-east-1 50.19.0.0/16 EC2 us-west-2 50.112.0.0/16 EC2 us-east-1 52.0.0.0/15 EC2 us-east-1 52.2.0.0/15 EC2 us-east-1 52.4.0.0/14 EC2 us-west-1 52.8.0.0/16 EC2 us-west-1 52.9.0.0/16 EC2 us-west-2 52.10.0.0/15 EC2 us-west-2 52.12.0.0/15 EC2 eu-west-1 52.16.0.0/15 EC2 eu-west-1 52.18.0.0/15 EC2 us-east-1 52.20.0.0/14 EC2 us-west-2 52.24.0.0/14 EC2 eu-central-1 52.28.0.0/16 EC2 eu-central-1 52.29.0.0/16 EC2 eu-west-1 52.30.0.0/15 EC2 us-west-2 52.32.0.0/14 EC2 us-west-2 52.36.0.0/14 EC2 us-west-2 52.40.0.0/14 EC2 eu-west-1 52.48.0.0/14 EC2 us-west-1 52.52.0.0/15 EC2 eu-central-1 52.58.0.0/15 EC2 ap-southeast-2 52.62.0.0/15 EC2 ap-southeast-2 52.64.0.0/17 EC2 ap-southeast-2 52.64.128.0/17 EC2 ap-southeast-2 52.65.0.0/16 EC2 sa-east-1 52.67.0.0/16 EC2 ap-northeast-1 52.68.0.0/15 EC2 us-east-1 52.70.0.0/15 EC2 us-east-1 52.72.0.0/15 EC2 ap-southeast-1 52.74.0.0/16 EC2 ap-southeast-1 52.76.0.0/17 EC2 ap-southeast-1 52.76.128.0/17 EC2 ap-southeast-1 52.77.0.0/16 EC2 ap-northeast-2 52.79.0.0/16 EC2 us-east-1 52.86.0.0/15 EC2 us-west-2 52.88.0.0/15 EC2 us-east-1 52.90.0.0/15 EC2 sa-east-1 52.95.240.0/24 EC2 ap-southeast-2 52.95.241.0/24 EC2 ap-southeast-1 52.95.242.0/24 EC2 ap-northeast-1 52.95.243.0/24 EC2 eu-west-1 52.95.244.0/24 EC2 us-east-1 52.95.245.0/24 EC2 us-west-1 52.95.246.0/24 EC2 us-west-2 52.95.247.0/24 EC2 eu-central-1 52.95.248.0/24 EC2 cn-north-1 52.95.249.0/24 EC2 ap-northeast-2 52.95.252.0/24 EC2 sa-east-1 52.95.255.0/28 EC2 ap-southeast-2 52.95.255.16/28 EC2 ap-southeast-1 52.95.255.32/28 EC2 ap-northeast-1 52.95.255.48/28 EC2 eu-west-1 52.95.255.64/28 EC2 us-east-1 52.95.255.80/28 EC2 us-west-1 52.95.255.96/28 EC2 us-west-2 52.95.255.112/28 EC2 eu-central-1 52.95.255.128/28 EC2 cn-north-1 52.95.255.144/28 EC2 ap-northeast-1 52.192.0.0/15 EC2 ap-northeast-1 52.196.0.0/14 EC2 us-east-1 52.200.0.0/13 EC2 eu-west-1 52.208.0.0/13 EC2 ap-southeast-1 52.220.0.0/15 EC2 ap-northeast-1 54.64.0.0/15 EC2 ap-southeast-2 54.66.0.0/16 EC2 us-west-1 54.67.0.0/16 EC2 us-west-2 54.68.0.0/14 EC2 eu-west-1 54.72.0.0/15 EC2 eu-west-1 54.74.0.0/15 EC2 eu-west-1 54.76.0.0/15 EC2 eu-west-1 54.78.0.0/16 EC2 ap-southeast-2 54.79.0.0/16 EC2 us-east-1 54.80.0.0/13 EC2 us-east-1 54.88.0.0/14 EC2 ap-northeast-1 54.92.0.0/17 EC2 us-east-1 54.92.128.0/17 EC2 eu-central-1 54.93.0.0/16 EC2 sa-east-1 54.94.0.0/16 EC2 ap-northeast-1 54.95.0.0/16 EC2 us-east-1 54.144.0.0/14 EC2 us-west-2 54.148.0.0/15 EC2 ap-northeast-1 54.150.0.0/16 EC2 us-west-1 54.151.0.0/17 EC2 ap-southeast-1 54.151.128.0/17 EC2 us-east-1 54.152.0.0/16 EC2 us-west-1 54.153.0.0/17 EC2 ap-southeast-2 54.153.128.0/17 EC2 eu-west-1 54.154.0.0/16 EC2 eu-west-1 54.155.0.0/16 EC2 us-east-1 54.156.0.0/14 EC2 us-east-1 54.160.0.0/13 EC2 ap-northeast-1 54.168.0.0/16 EC2 ap-southeast-1 54.169.0.0/16 EC2 eu-west-1 54.170.0.0/15 EC2 us-east-1 54.172.0.0/15 EC2 us-east-1 54.174.0.0/15 EC2 us-west-1 54.176.0.0/15 EC2 ap-northeast-1 54.178.0.0/16 EC2 ap-southeast-1 54.179.0.0/16 EC2 us-west-1 54.183.0.0/16 EC2 us-west-2 54.184.0.0/13 EC2 us-west-1 54.193.0.0/16 EC2 eu-west-1 54.194.0.0/15 EC2 us-east-1 54.196.0.0/15 EC2 us-east-1 54.198.0.0/16 EC2 ap-northeast-1 54.199.0.0/16 EC2 us-west-2 54.200.0.0/15 EC2 us-west-2 54.202.0.0/15 EC2 us-east-1 54.204.0.0/15 EC2 ap-southeast-2 54.206.0.0/16 EC2 sa-east-1 54.207.0.0/16 EC2 us-east-1 54.208.0.0/15 EC2 us-east-1 54.210.0.0/15 EC2 us-west-2 54.212.0.0/15 EC2 us-west-2 54.214.0.0/16 EC2 us-west-1 54.215.0.0/16 EC2 eu-west-1 54.216.0.0/15 EC2 us-west-2 54.218.0.0/16 EC2 us-west-1 54.219.0.0/16 EC2 eu-west-1 54.220.0.0/16 EC2 us-east-1 54.221.0.0/16 EC2 cn-north-1 54.222.128.0/17 EC2 cn-north-1 54.223.0.0/16 EC2 us-east-1 54.224.0.0/15 EC2 us-east-1 54.226.0.0/15 EC2 eu-west-1 54.228.0.0/16 EC2 eu-west-1 54.229.0.0/16 EC2 sa-east-1 54.232.0.0/16 EC2 sa-east-1 54.233.0.0/18 EC2 sa-east-1 54.233.64.0/18 EC2 sa-east-1 54.233.128.0/17 EC2 us-east-1 54.234.0.0/15 EC2 us-east-1 54.236.0.0/15 EC2 ap-northeast-1 54.238.0.0/16 EC2 us-west-1 54.241.0.0/16 EC2 us-east-1 54.242.0.0/15 EC2 us-west-2 54.244.0.0/16 EC2 us-west-2 54.245.0.0/16 EC2 eu-west-1 54.246.0.0/16 EC2 eu-west-1 54.247.0.0/16 EC2 ap-northeast-1 54.248.0.0/15 EC2 ap-northeast-1 54.250.0.0/16 EC2 ap-southeast-1 54.251.0.0/16 EC2 ap-southeast-2 54.252.0.0/16 EC2 ap-southeast-2 54.253.0.0/16 EC2 ap-southeast-1 54.254.0.0/16 EC2 ap-southeast-1 54.255.0.0/16 EC2 us-east-1 67.202.0.0/18 EC2 us-east-1 72.44.32.0/19 EC2 us-east-1 75.101.128.0/17 EC2 eu-west-1 79.125.0.0/17 EC2 us-gov-west-1 96.127.0.0/17 EC2 ap-northeast-1 103.4.8.0/21 EC2 us-east-1 107.20.0.0/14 EC2 ap-southeast-1 122.248.192.0/18 EC2 us-east-1 174.129.0.0/16 EC2 ap-southeast-1 175.41.128.0/18 EC2 ap-northeast-1 175.41.192.0/18 EC2 ap-northeast-1 176.32.64.0/19 EC2 ap-northeast-1 176.34.0.0/19 EC2 ap-northeast-1 176.34.32.0/19 EC2 eu-west-1 176.34.64.0/18 EC2 eu-west-1 176.34.128.0/17 EC2 sa-east-1 177.71.128.0/17 EC2 us-west-1 184.72.0.0/18 EC2 us-east-1 184.72.64.0/18 EC2 us-east-1 184.72.128.0/17 EC2 us-east-1 184.73.0.0/16 EC2 us-west-1 184.169.128.0/17 EC2 eu-west-1 185.48.120.0/22 EC2 us-west-1 204.236.128.0/18 EC2 us-east-1 204.236.192.0/18 EC2 us-east-1 216.182.224.0/20 EC2 GLOBAL 205.251.192.0/21 ROUTE53 us-west-1 54.183.255.128/26 ROUTE53_HEALTHCHECKS eu-west-1 54.228.16.0/26 ROUTE53_HEALTHCHECKS sa-east-1 54.232.40.64/26 ROUTE53_HEALTHCHECKS us-west-1 54.241.32.64/26 ROUTE53_HEALTHCHECKS us-east-1 54.243.31.192/26 ROUTE53_HEALTHCHECKS us-west-2 54.244.52.192/26 ROUTE53_HEALTHCHECKS us-west-2 54.245.168.0/26 ROUTE53_HEALTHCHECKS ap-northeast-1 54.248.220.0/26 ROUTE53_HEALTHCHECKS ap-northeast-1 54.250.253.192/26 ROUTE53_HEALTHCHECKS ap-southeast-1 54.251.31.128/26 ROUTE53_HEALTHCHECKS ap-southeast-2 54.252.79.128/26 ROUTE53_HEALTHCHECKS ap-southeast-2 54.252.254.192/26 ROUTE53_HEALTHCHECKS ap-southeast-1 54.255.254.192/26 ROUTE53_HEALTHCHECKS us-east-1 107.23.255.0/26 ROUTE53_HEALTHCHECKS eu-west-1 176.34.159.192/26 ROUTE53_HEALTHCHECKS sa-east-1 177.71.207.128/26 ROUTE53_HEALTHCHECKS GLOBAL 52.84.0.0/15 CLOUDFRONT GLOBAL 54.182.0.0/16 CLOUDFRONT GLOBAL 54.192.0.0/16 CLOUDFRONT GLOBAL 54.230.0.0/16 CLOUDFRONT GLOBAL 54.239.128.0/18 CLOUDFRONT GLOBAL 54.239.192.0/19 CLOUDFRONT GLOBAL 54.240.128.0/18 CLOUDFRONT GLOBAL 204.246.164.0/22 CLOUDFRONT GLOBAL 204.246.168.0/22 CLOUDFRONT GLOBAL 204.246.174.0/23 CLOUDFRONT GLOBAL 204.246.176.0/20 CLOUDFRONT GLOBAL 205.251.192.0/19 CLOUDFRONT GLOBAL 205.251.249.0/24 CLOUDFRONT GLOBAL 205.251.250.0/23 CLOUDFRONT GLOBAL 205.251.252.0/23 CLOUDFRONT GLOBAL 205.251.254.0/24 CLOUDFRONT GLOBAL 216.137.32.0/19 CLOUDFRONT (env) ➜ aws
Filtering
(env) ➜ aws python aws-ranges.py -r sa-east-1 region ip_prefix service --------- ----------------- -------------------- sa-east-1 52.67.0.0/16 AMAZON sa-east-1 52.92.39.0/24 AMAZON sa-east-1 52.92.64.0/22 AMAZON sa-east-1 52.92.72.0/22 AMAZON sa-east-1 52.94.7.0/24 AMAZON sa-east-1 52.95.240.0/24 AMAZON sa-east-1 52.95.255.0/28 AMAZON sa-east-1 54.94.0.0/16 AMAZON sa-east-1 54.207.0.0/16 AMAZON sa-east-1 54.231.253.0/24 AMAZON sa-east-1 54.232.0.0/16 AMAZON sa-east-1 54.233.0.0/18 AMAZON sa-east-1 54.233.64.0/18 AMAZON sa-east-1 54.233.128.0/17 AMAZON sa-east-1 54.240.244.0/22 AMAZON sa-east-1 177.71.128.0/17 AMAZON sa-east-1 177.72.240.0/21 AMAZON sa-east-1 52.67.0.0/16 EC2 sa-east-1 52.95.240.0/24 EC2 sa-east-1 52.95.255.0/28 EC2 sa-east-1 54.94.0.0/16 EC2 sa-east-1 54.207.0.0/16 EC2 sa-east-1 54.232.0.0/16 EC2 sa-east-1 54.233.0.0/18 EC2 sa-east-1 54.233.64.0/18 EC2 sa-east-1 54.233.128.0/17 EC2 sa-east-1 177.71.128.0/17 EC2 sa-east-1 54.232.40.64/26 ROUTE53_HEALTHCHECKS sa-east-1 177.71.207.128/26 ROUTE53_HEALTHCHECKS (env) ➜ aws python aws-ranges.py -r sa-east-1 -s ec2 region ip_prefix service --------- --------------- --------- sa-east-1 52.67.0.0/16 EC2 sa-east-1 52.95.240.0/24 EC2 sa-east-1 52.95.255.0/28 EC2 sa-east-1 54.94.0.0/16 EC2 sa-east-1 54.207.0.0/16 EC2 sa-east-1 54.232.0.0/16 EC2 sa-east-1 54.233.0.0/18 EC2 sa-east-1 54.233.64.0/18 EC2 sa-east-1 54.233.128.0/17 EC2 sa-east-1 177.71.128.0/17 EC2 (env) ➜ aws
The code
#!/usr/bin/env python
#March 8th 2016
__author__ = 'Jason Riedel'
__description__ = 'Grabs IP ranges from amazon'
__version__ = '1.0'
url = 'https://ip-ranges.amazonaws.com/ip-ranges.json'
import requests
import json
import argparse
from tabulate import tabulate
parser = argparse.ArgumentParser()
parser.add_argument('-r', '--region', action="store", dest="regionFilter", required=False, help="Region to filter/return data for: us-west, ap-southeast-1, us-east-1")
parser.add_argument('-s', '--service', action="store", dest="serviceFilter", required=False, help="Service to filter/return data for: AMAZON, CLOUDFRONT, EC2")
parser.add_argument('-rl', action="store_true", dest="listRegions", required=False, help="List known regions")
parser.add_argument('-sl', action="store_true", dest="listService", required=False, help="List known services")
args = parser.parse_args()
def get_json(url):
try:
r = requests.get(url)
jdata = r.json()
except Exception, e:
print "ERROR - %s failed: %s" % (url, e)
return jdata
def list_filter_json(jdata, filter_on):
filtered_data = []
for i in range(len(jdata['prefixes'])):
filtered_data.append(jdata['prefixes'][i][filter_on])
return filtered_data
def filter_json(jdata, filters):
filtered_data = []
for i in range(len(jdata['prefixes'])):
if 'region' in filters and 'service' in filters:
if filters['region'].lower() in jdata['prefixes'][i]['region'].lower() and filters['service'].lower() in jdata['prefixes'][i]['service'].lower():
filtered_data.append(jdata['prefixes'][i])
elif 'region' in filters:
if filters['region'].lower() in jdata['prefixes'][i]['region'].lower():
filtered_data.append(jdata['prefixes'][i])
elif 'service' in filters:
if filters['service'].lower() in jdata['prefixes'][i]['service'].lower():
filtered_data.append(jdata['prefixes'][i])
return filtered_data
def list_regions(jdata):
regions = set()
filtered_data = list_filter_json(jdata, 'region')
for region in filtered_data:
regions.add(region)
for region in sorted(regions):
print region
def list_services(jdata):
services = set()
filtered_data = list_filter_json(jdata, 'service')
for service in filtered_data:
services.add(service)
for service in sorted(services):
print service
def table_it(filtered_data):
print tabulate(filtered_data, headers="keys")
if __name__ == "__main__":
jdata = get_json(url)
if args.listRegions or args.listService:
if args.listRegions:
list_regions(jdata)
elif args.listService:
list_services(jdata)
else:
parser.print_help()
else:
if args.regionFilter or args.serviceFilter:
if args.regionFilter and args.serviceFilter:
filters = {'region': args.regionFilter, 'service': args.serviceFilter}
else:
if args.regionFilter:
filters = {'region': args.regionFilter}
if args.serviceFilter:
filters = {'service': args.serviceFilter}
filtered_data = filter_json(jdata, filters)
table_it(filtered_data)
else:
table_it(jdata['prefixes'])
How To: curl the Openstack API’s (v3 Keystone Auth)
While Openstack provides a python client(s) for interactions….
[root@diamond ~]# source keystonerc_tuxninja [root@diamond ~(keystone_tuxninja)]# openstack server list +--------------------------------------+-------+--------+----------------------------------------+ | ID | Name | Status | Networks | +--------------------------------------+-------+--------+----------------------------------------+ | e5b35d6a-a9ba-4714-a9e1-6361706bd047 | spin1 | ACTIVE | private_tuxlabs=10.0.0.8, 192.168.1.52 | +--------------------------------------+-------+--------+----------------------------------------+ [root@diamond ~(keystone_tuxninja)]#
I frequently, finding myself needing to get data out of it without the pain of awk/sed’ing out the ASCII art.
Thus to quickly access the raw data, we can directly query the API’s using curl & parsing JSON instead, which is much better 🙂
Authentication
Before we can interact with the other Openstack API’s we need to authenticate to Keystone openstack’s identity service. After authenticating we receive a token to use with our subequent API requests. So step 1 we are going to create a JSON object with the required authentication details.
Create a file called ‘token-request.json’ with an object that looks like this.
{
"auth": {
"identity": {
"methods": [
"password"
],
"password": {
"user": {
"domain": {
"id": "default"
},
"name": "tuxninja",
"password": "put_your_openstack_pass"
}
}
}
}
}
Btw, if you followed my tutorial on how to install Openstack Kilo, your authentication details for ‘admin’ is in your keystonerc_admin file.
Now we can use this file to authenticate like so:
export TOKEN=`curl -si -d @token-request.json -H "Content-type: application/json" http://localhost:35357/v3/auth/tokens | awk '/X-Subject-Token/ {print $2}'`
The token is actually returned in the header of the HTTP response, so this is why we need ‘-i’ when curling. Notice we are parsing out the token and returning the value to an environment variable $TOKEN.
Now we can include this $TOKEN and run whatever API commands we want (assuming admin privileges for the tenant/project)
Curl Commands (Numerous Examples!)
# list domains
curl -si -H"X-Auth-Token:$TOKEN" -H "Content-type: application/json" http://localhost:35357/v3/domains
# create a domain
curl -H"X-Auth-Token:$TOKEN" -H "Content-type: application/json" -d '{"domain": {"description": "--optional--", "enabled": true, "name": "dom1"}}' http://localhost:35357/v3/domains
# list users
curl -si -H"X-Auth-Token:$TOKEN" -H "Content-type: application/json" http://localhost:35357/v3/users
# To create a users, create file named create_user.json file like this:
{
"user": {
"default_project_id": "18ed894bb8b84a5b9144c129fc754722",
"description": "Description",
"domain_id": "default",
"email": "tuxninja@tuxlabs.com",
"enabled": true,
"name": "tuxninja",
"password": "changeme" }
}
# then run
curl -si -H"X-Auth-Token:$TOKEN" -H "Content-type: application/json" http://localhost:35357/v3/users -d @create_user.json
# list images in nova
<tenant_id>
curl -s -H"X-Auth-Token:$TOKEN" -H "Content-type: application/json" http://localhost:8774/v2/18ed894bb8b84a5b9144c129fc754722/images | python -m json.tool
# list servers (vms)
curl -s -H"X-Auth-Token:$TOKEN" -H "Content-type: application/json" http://localhost:8774/v2/18ed894bb8b84a5b9144c129fc754722/servers | python -m json.tool
# neutron networks
curl -s -H"X-Auth-Token:$TOKEN" -H "Content-type: application/json" http://localhost:9696/v2.0/networks | python -m json.tool
# neutron subnets
curl -s -H"X-Auth-Token:$TOKEN" -H "Content-type: application/json" http://localhost:9696/v2.0/networks | python -m json.tool
I sometimes pipe the output to python -m json.tool, which provides formatting for JSON. Lets take a closer look at an example.
Listing servers (vm’s)
[root@diamond ~(keystone_tuxninja)]# curl -s -H"X-Auth-Token:$TOKEN" -H "Content-type: application/json" http://localhost:8774/v2/18ed894bb8b84a5b9144c129fc754722/servers | python -m json.tool
{
"servers": [
{
"id": "e5b35d6a-a9ba-4714-a9e1-6361706bd047",
"links": [
{
"href": "http://localhost:8774/v2/18ed894bb8b84a5b9144c129fc754722/servers/e5b35d6a-a9ba-4714-a9e1-6361706bd047",
"rel": "self"
},
{
"href": "http://localhost:8774/18ed894bb8b84a5b9144c129fc754722/servers/e5b35d6a-a9ba-4714-a9e1-6361706bd047",
"rel": "bookmark"
}
],
"name": "spin1"
}
]
}
[root@diamond ~(keystone_tuxninja)]#
I only have 1 VM currently called spin1, but for the tutorials sake, if I had ten’s or hundred’s of VM’s and all I cared about was the VM name or ID, I would still need to parse this JSON object to avoid getting all this other meta-data.
My favorite command line way to do that without going full Python is using the handy JQ tool.
Here is how to use it !
[root@diamond ~(keystone_tuxninja)]# curl -s -H"X-Auth-Token:$TOKEN" -H "Content-type: application/json" http://localhost:8774/v2/18ed894bb8b84a5b9144c129fc754722/servers | jq .
{
"servers": [
{
"name": "spin1",
"links": [
{
"rel": "self",
"href": "http://localhost:8774/v2/18ed894bb8b84a5b9144c129fc754722/servers/e5b35d6a-a9ba-4714-a9e1-6361706bd047"
},
{
"rel": "bookmark",
"href": "http://localhost:8774/18ed894bb8b84a5b9144c129fc754722/servers/e5b35d6a-a9ba-4714-a9e1-6361706bd047"
}
],
"id": "e5b35d6a-a9ba-4714-a9e1-6361706bd047"
}
]
}
[root@diamond ~(keystone_tuxninja)]#
[root@diamond ~(keystone_tuxninja)]# curl -s -H"X-Auth-Token:$TOKEN" -H "Content-type: application/json" http://localhost:8774/v2/18ed894bb8b84a5b9144c129fc754722/servers | jq .servers[0].name -r
spin1
[root@diamond ~(keystone_tuxninja)]#
The first command just takes whatever the STDOUT from curl is and indent’s and color’s the JSON making it pretty (colors gives it +1 vs. python -m json.tool).
The second example we actually parse what were after. As you can see it is pretty simple, but jq’s query language may not be 100% intuitive at first, but I promise it is pretty easy to understand if you have ever parsed JSON before. Read up more on JQ @ https://stedolan.github.io/jq/ & check out the Openstack docs for more API commands http://developer.openstack.org/api-ref.html
Hope you enjoyed this post ! Until next time.