Reconsidering SSH in AWS: A DevSecOps Approach to EC2 Security Access
Written on
In the rapidly evolving landscape of cloud security, the reliance on SSH for accessing AWS EC2 instances raises significant concerns. As a security expert, I've frequently encountered security groups with port 22 exposed to wide ranges like 10.0.0.0/8 or even 0.0.0.0/0. It's perplexing that SSH remains a common practice in 2024, especially when superior alternatives exist. My role often involves advocating for improved security practices, but I frequently encounter resistance. Many users prefer straightforward and immediate solutions, and I can understand their perspective. Yet, given the escalating number of security breaches, we must adopt methods that inherently prioritize security.
Tech tutorials on platforms like YouTube often demonstrate public EC2 instances with SSH access configured for "My IP." While I don't fault creators for showcasing these methods, many developers mistakenly believe this is the sole effective approach. Consequently, when security professionals suggest altering their connection methods, pushback is common.
So, how can we securely access an EC2 instance without using key pairs, public IP addresses, or misconfigured security groups, all while enhancing logging and security? The solution lies in AWS Systems Manager and Session Manager.
What is AWS Systems Manager (SSM)?
AWS Systems Manager serves as a centralized operations hub for managing applications and resources in AWS, providing a secure management solution for hybrid and multi-cloud setups that supports operations at scale.
Key services provided by Systems Manager include: - Patch management - Software distribution - EC2 inventory management - Configuration management - Session Manager
What is Session Manager?
Session Manager enables users to connect to EC2 instances through a browser-based shell or the AWS Command Line Interface (CLI). It facilitates secure, auditable management of nodes without the need for open inbound ports, bastion hosts, or SSH key management. Session Manager helps organizations adhere to policies requiring controlled access to managed nodes, enforcing strict security practices while providing a streamlined user experience.
Why opt for Session Manager over SSH?
- Centralized access control via IAM policies
- No need to keep inbound ports 22 or 3389 open in security groups
- Elimination of bastion host and SSH key management
- Comprehensive logging and auditing of session activities.
Session activity logging operates on two levels: all sessions are recorded in AWS CloudTrail, allowing for consistent auditing across AWS services. This ensures accountability for actions taken during active sessions. Additionally, keystrokes can be captured and sent to centralized logging systems like CloudWatch or Amazon S3. If regulatory compliance requires logging all terminal actions on an EC2 instance, this capability is invaluable. It also significantly aids Security Operations teams during incident investigations by allowing alerts based on known suspicious behaviors, such as privilege escalation.
Solution Architectural Overview
Designing a solution for a multi-account environment can be complex, especially when encryption keys are involved. However, this complexity can be managed by automating the configuration during the account provisioning process. Below is a design outline that addresses the SSH challenge without hindering developers or platform engineers.
Logging Account (S3 and CloudWatch) The Logging Account is situated within the Security Organization Unit (OU). This account hosts the S3 bucket and CloudWatch log group, both configured with KMS encryption.
VPC Endpoints (VPCe) In each AWS account within the Workloads OU, VPC endpoints should be set up to keep traffic internal to the AWS cloud, allowing access to the S3 bucket and CloudWatch log group without internet egress.
EC2 Instances EC2 instances will utilize a custom instance profile that grants permissions for logging to CloudWatch and S3, ensuring they can encrypt logs with the KMS key. Furthermore, a managed IAM policy from AWS enables Systems Manager operation and is attached to the instance profile.
Systems Manager & Session Manager Session Manager preferences will store the configuration outlined above, ensuring consistent usage of the designated S3 bucket or CloudWatch group.
Ingesting Logs to SIEM (Optional) A Security Information and Event Management (SIEM) system can capture logs from various sources for analysis and alerting on potential malicious activities. Instead of configuring alerts in CloudWatch, security analysts might find it easier to set rules within the SIEM. This approach also allows for faster expiration of CloudWatch logs to save costs, as logs will be ingested into an external system.
Pre-requisites for Session Recordings
- Use an Amazon Machine Image (AMI) with the SSM agent pre-installed (e.g., Amazon Linux 2023).
- Refer to AWS guidelines for configuring Systems Manager (some settings are included in my Terraform setup).
- Ensure VPC endpoints for ec2messages, ssm, ssmmessages, kms, logs, and s3 are configured; a Gateway endpoint for S3 may also be necessary.
Setting Up and Using Session Manager Logs
Terraform Configuration
Below is the Terraform code I utilized to set up the core components of this demonstration. The complete Terraform code is available on my GitHub.
Final Thoughts
AWS continues to explore more secure methodologies for accessing its services, yet persuading users to adopt these newer approaches remains a challenge. Many individuals are accustomed to traditional practices, making it difficult to shift their mindset from using direct terminal connections or bastion hosts, which have been the norm for over two decades.
While third-party solutions like Teleport offer excellent alternatives, utilizing AWS's native tools often proves more straightforward. This allows for centralized auditing through tools like Wiz and consistent log storage in CloudWatch or S3.
One critical note: this discussion assumes that a high level of consistency and established guardrails exist within your environment. If developers can launch EC2 instances without the appropriate IAM roles, SSM will not function. Therefore, implementing controls such as permission boundaries or Service Control Policies is crucial to enforce compliance with necessary guardrails. Infrastructure as Code scans may also be beneficial to prevent improper configurations from being deployed.