Wanted: a serverless solution to exchange IAM access for network access to VPC resources
What would make AWS even better: #8 in my countdown from 10
This is item #8 in my count-down of items on my 2022 re:invent wishlist. You can find item #9 here.
The Current State of CloudShell
I was very excited when AWS released CloudShell at the end of 2020. It seemed like a very obvious missing piece of the AWS puzzle. Why should I need to spin up an entire EC2 instance or Workspace when all I want is to run a command against my AWS environment? Not to mention that GCP and Azure both had cloud shell options at the time.
Since then, I’m disappointed to say, I’ve only used CloudShell a handful of times. Maybe, I need to get the scheduled scaling configuration for an ECS service, and don’t feel like logging in again. Maybe I’m fighting a fire from someone else’s computer. Having a little shell which I can start in a new tab right in my browser is useful. Having the AWS CLI preconfigured with my current logged in identity is even more useful.
With this said, CloudShell is somehow not useful enough for me to actually use frequently.
The lesser reason for this is that it (understandably) spins up more slowly than my local computer. The greater reason is that CloudShell doesn’t have access to my network resources. The vast majority of the time that I pull up a terminal outside of developing in my IDE, I need access to my network resources.
Perhaps, I want to telnet to a port and see if it’s listening. Maybe, I want to connect to a database and view my data. I might want to test DNS resolution within a VPC. Or maybe an API Gateway endpoint is configured to reject traffic that does not originate in my VPC. For these situations, and any other situation where I need access to my network resources, CloudShell does not help me.
How I access Network Resources
I usually interact with network resources by connect to my private network resources over a Client VPN connection from my macbook.
This workflow replaces the previous workflow I used before AWS Client VPN was released: an ec2 bastion instance and SSH tunnels to network resources. Back then, I remember entire operations processes built around rotating SSH keys, and authorizing keys to access the bastion instance. I knew even less what I was doing then than I do now, but I do not miss those days.
A desired user experience
In its most distilled form, I am looking for a simple mechanism to exchange my IAM credentials for an ephemeral, serverless, shell session with access to VPC resources. Ephemeral, because I don’t want to pay $$$ when I terminate the session. Serverless, because I don’t want to need to worry about OS configurations or managing servers.
A design based on existing AWS services
If you are familiar with SSM Session Manager, you might have noticed that a mechanism to attain a shell session with access to VPC resources already exists, but not in a serverless variety.
In case you’re not familiar with SSM Session Manager, here is a brief overview of how it works. Highlights below:
Ensure that the SSM agent is running on an EC2 instances. Most of the prebuilt Amazon Linux AMIs will take care of this for you.
Grant your EC2 instance role access to a set of api actions necessary to make Session Manager work.
Install the session manager plugin for the AWS CLI
You make a request to establish a shell session with a target instance.
If your principal (role, user) has the appropriate permissions, a bidirectional communication channel is established between you and the server via Session Manager.
Better than simply being able to gain shell access, Session Manager allows you to tunnel traffic between a port on a client computer and a port SSM managed instance. With this functionality, it becomes possible to run SSH over session manager and access remote resources over an SSH tunnel like you would with a bastion host.
With these primitives in place, extending CloudShell to be able a session to connect to your VPC resources should be achievable. An instinctual proposed design for this feature looks something like the following:
The AWS console should let you create a set of launch profiles, each specifying a set of subnets and security groups.
For each profile, a network interface for the launch profile should be deployed in the corresponding subnet, and placed in the specified security groups.
When a CloudShell session is created, you should be able to associate it with a launch profile.
The created shell session should route traffic via the network interface with the associated launch profile. This should be achieved in the same way that AWS Lambda / ECS Fargate gain VPC access.
You should be able to connect to the CloudShell session via a Session Manager-like interface, but instead of targeting an instance, you target a CloudShell session id. In this way, you can tunnel traffic from your local machine through CloudShell to gain access to your network resources without deploying an EC2 instance.
A More Powerful GCP Identity Aware Proxy
One last aside, if you will. CloudShell + VPC launch profiles, effectively results in a more powerful version of GCPs Identity Aware Proxy.
GCP’s IAP lets you access HTTPS resources in your VPCs via a proxy instead of via VPN. The IAP ensures that a user is authenticated and authorized to access the HTTPS resource before proxying traffic to endpoints in your private network.
Cloudshell + VPC launch profiles with the above implementation lets you do the same to all network resources. Not just HTTPS resources.
Takeaways
With VPC access and the ability to tunnel through CloudShell sessions in place, I imagine I would use CloudShell a whole lot, both via the CLI and the console. Being able to troubleshoot and connect to AWS VPC network resources without deploying a client VPN would save money and time in many instances.
More importantly, the ability to bind network sessions to IAM authentication and control access centrally through IAM policies would be secure, and convenient to enforce and administer.
And so, if you are an AWS genie, item #8 on my wishlist is the ability to attain ephemeral access to my AWS network resources over IAM. I don’t really care about the implementation details, but based on the AWS technology I have worked with, enabling CloudShell to interact with my network resources would be a great start. Providing an IAM authenticated mechanism to tunnel traffic originating on my macbook through these CloudShell sessions to my VPC resources would be game changing.