#awswishlist Series: More granular service IPs in AWS ip-ranges.json list

This will be the first in a series of posts describing Tweets I’ve sent with the hashtag #awswishlist. Tweets to this #awswishlist hashtag come from anyone who uses AWS and is frustrated with the AWS user experience in some way. These Tweets are often responded to by AWS Support staff on Twitter, indicating they’ll be passing the feedback on to the team responsible for the AWS service. But in my tenure using AWS, I’ve only personally seen one wishlist item get resolved within a reasonable time frame.

This blog series will demonstrate why these #awswishlist items are so disappointing, and the contortions that users are required to perform in order to work around these UX deficiencies.

The straw that broke the DevOps back

So I was working on a project to allow traffic through our CDN for SNS notifications. (I know what you’re thinking. Saying “SNS notifications” is redundant. But if I just said “SNS” the subject isn’t clear enough.) It would be great if I could just create a list of CIDRs for the SNS IP ranges. Wait a minute…it appears there’s no way to filter just that service from the full list of 3000 or more CIDRs.

With this realization I shot out the following Tweet:

#awswishlist Tweet about ip-ranges.json (image)

And this is the response from AWS Support:

Response Tweet from AWS Support (image)

What is ip-ranges.json?

The AWS ip-ranges.json file is a huge list of CIDR ranges that are used by AWS services. As far as I know, it lists every AWS IP address so that users can do things like adding these IPs to their firewall allow rule sets. This is just one use case, but the list is provided for users to validate that an IP address is owned by AWS.

Thankfully AWS provides details for the CIDR ranges such as AWS Region (i.e. us-east-1), and the AWS service that these ranges are used for. But the services covered in this list is only a very small subset of all AWS services. This is limiting because users cannot filter every service that may send traffic from the AWS network to the user’s network. For example AWS Simple Notification Service (SNS) sends requests from AWS’s network to the user’s network. Yet this service is not listed in the ip-ranges.json list, so you would have to allow every AWS IP address, not a subset containing only the SNS CIDRs which would be much more reasonable.

The AWS documentation for this ip-ranges.json syntax shows that the user can only perform filtering on the services below, because they are the only services that are broken out separately.

AWS IP ranges documentation (image)

This is a major problem.

Why so few services?

The “Valid Values” list contains 17 service specific items, and the overall AMAZON value is a superset that contains every AWS CIDR (including those in the service specific subsets).

AWS now has over 300 services, obviously not every service has outgoing traffic to a user’s network, but that’s beside the point. If they separate out some services that have outgoing traffic, they should do the same for every service with outgoing traffic.

Also, some of their most important services such as SNS (which by the way, is one of AWS’s first services created, in Apr 2010) are not on this list. This should be corrected immediately so that users can add these SNS CIDRs to their firewall allow lists (I’m not asking for a friend, I’m the customer in this case who’s really interested in doing this).

Mind the gap

While its great that AWS Support responds to these Tweets now (there was a time when they did not, and you didn’t know if your lone voice in the wilderness was reaching anyone,) it seems that these deficiencies are everywhere, meaning in every AWS service. I know there are UX problems, AWS Support knows there are UX problems, Werner Vogels knows there are UX problems, and Jeff Bezos better know there are UX problems. It’s a tall order to get everything right. But AWS keeps saying they have relentless focus on the customer. Fix these rough edges before more services are released.

Why I’m annoyed here is that they started breaking out service subsets in ip-ranges.json, but they didn’t take it to its natural conclusion, which would have been including every (outgoing) service as a valid filter value. It isn’t relentless focus on the customer if the user has to build a list of all 3000 some odd CIDRs from the list in order to allow just the SNS traffic, when it should only require hundreds or even dozens in such an allow list.