In part 1 of this two-part series, we discussed how Cloudera Altus cloud services put you, the Cloudera Altus customer, in control of your cloud resources. In that blog post, we described the problems and solutions in a cloud provider independent manner and then showed the solutions with Azure. In this second blog post, we’ll look at the solutions using AWS.
We previously introduced two control problems: the infrastructure construction problem (how do Cloudera Altus cloud services allow you to have precise control over how they build and manage a cluster for you); the second being the data access problem (how do Cloudera Altus cloud services allow you to manage the access that your cluster has into your cloud environment).
Solving the Infrastructure Construction Problem
AWS
Establishing trust is accomplished in AWS using a role that indicates the Altus Customer’s AWS account trusts the Altus AWS account (trust, within AWS, is at the account level). This role is created and controlled by the Altus Customer’s AWS administrator and is often called a Cross Account role.
The Cross Account role has two policies:
- A “trust” or “assume-role” policy, says exactly which account (in this case, Altus) can assume this role. Within AWS, when a user assumes a role, the user’s current permissions are temporarily overridden by the permissions governed by those of the assumed role.
- A “permissions policy”says which permissions (actions and resources) come with this role
In summary, the Cross Account role establishes trust, specifies actions, and manages resource scope.
Managing which Altus user can access which cloud resource is then a matter of allowing or denying access to a specific Environment to a specific user. All of this is built and controlled by the Altus Customer Administrator through the Altus application.
The trust policy to be used to allow Altus to assume the Cross Account role is shown below:
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::062663104137:root"
},
"Action": "sts:AssumeRole",
"Condition": {
"StringEquals": {
"sts:ExternalId": "5286XXXX-XXXX-XXXX-XXXX-XXXX16115df6"
}
}
}
]
}
Note the reference to the Altus AWS account (via the ARN arn:aws:iam::062663104137:root), and the use of an ExternalId to prevent the Confused Deputy problem.
The Altus Cross Account Permissions Policy is shown below, listing the necessary and sufficient permissions Altus needs to manage AWS resources on the Altus Customer’s behalf:
{
"Statement": [
{
"Action": [
"ec2:CreateTags",
"ec2:DescribeAvailabilityZones",
"ec2:DescribeImages",
"ec2:DescribeInstanceStatus",
"ec2:DescribeInstances",
"ec2:DescribeKeyPairs",
"ec2:DescribePlacementGroups",
"ec2:DescribeRegions",
"ec2:DescribeSecurityGroups",
"ec2:DescribeNetworkAcls",
"ec2:DescribeSubnets",
"ec2:ImportKeyPair",
"ec2:RunInstances",
"ec2:TerminateInstances",
"ec2:RequestSpotInstances",
"ec2:CancelSpotInstanceRequests",
"ec2:DescribeSpotInstanceRequests",
"ec2:DescribeSpotPriceHistory"
],
"Effect": "Allow",
"Resource": "*",
"Sid": "clouderaAltusEc2"
},
{
"Action": [
"iam:GetInstanceProfile",
"iam:PassRole",
"iam:SimulatePrincipalPolicy"
],
"Effect": "Allow",
"Resource": "*",
"Sid": "clouderaAltusIam"
},
{
"Action": [
"sts:DecodeAuthorizationMessage"
],
"Effect": "Allow",
"Resource": "*",
"Sid": "clouderaAltusSts"
},
{
"Action": [
"s3:GetBucketLocation"
],
"Effect": "Allow",
"Resource": "*",
"Sid": "clouderaAltusS3"
},
{
"Action": [
"ec2:DeleteKeyPair"
],
"Effect": "Allow",
"Resource": "*",
"Sid": "clouderaAltusEc2Optional"
},
{
"Action": [
"ec2:DeleteKeyPair",
"ec2:AttachVolume",
"ec2:CreateVolume",
"ec2:DeleteVolume",
"ec2:DetachVolume",
"ec2:DescribeVolumes",
"ec2:DescribeSubnets",
"ec2:ModifyInstanceAttribute",
"kms:Encrypt",
"kms:Decrypt",
"kms:ReEncrypt*",
"kms:GenerateDataKey*",
"kms:Describe*",
"kms:CreateGrant"
],
"Effect": "Allow",
"Resource": "*",
"Sid": "clouderaAltusKmsEbsVolume"
}
],
"Version": "2012-10-17"
}
If you let Altus build all your AWS infrastructure for you using the Quickstart then these policies are built by Altus. You may choose to build it all yourself but again Altus offers assistance: there’s extensive help in the Environment Wizard giving you the relevant JSON templates and documentation links so you can both understand the steps and succeed.
Further Reading
- Tutorial: Delegate Access Across AWS Accounts Using IAM Roles
- How do I allow users from another account to access resources in my account through IAM?
Solving the Data Access Problem
AWS
This solution, in AWS, is based around roles and policies. Just like infrastructure access problem, the data access problem is solved with a role with trust and permissions policies attached. However, the role is wrapped in an instance profile, and this is attached by AWS to the EC2 instance. When a program on the EC2 instance needs access to AWS resources then the instance profile returns the role which the program can assume, thus obtaining the necessary permissions.
At the time of writing this blog post, the AWS GUI hides some of this complexity so that the user only sees the role, and not the instance policy object. The instance policy object is constructed with the same name prefix (so, for example, if one constructed a role ClouderaEC2InstanceRole then the GUI would create an instance policy referencing that role called ClouderaEC2InstancePolicy. However, if one constructs these objects programmatically then one has to be aware that all these objects are required.
One example of the two policies a user might create would limit AWS instances to have access to a log bucket and a data bucket. This is shown below. Note that this policy can constrain access to other resources; in this example, we’ve been very strict about what access is available from these EC2 instances, limiting it to exactly a single bucket.
The trust policy:
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"Service": "ec2.amazonaws.com"
},
"Action": "sts:AssumeRole"
}
]
}
The permissions policy:
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "VisualEditor0",
"Effect": "Allow",
"Action": [
"s3:PutObject",
"s3:GetObject",
"s3:DeleteObject"
],
"Resource": [
"arn:aws:s3:::archive_bucket.bucket/*",
"arn:aws:s3:::data_bucket.bucket/*"
]
},
{
"Sid": "VisualEditor1",
"Effect": "Allow",
"Action": [
"s3:ListBucket",
"s3:GetBucketLocation"
],
"Resource": [
"arn:aws:s3:::archive_bucket",
"arn:aws:s3:::data_bucket.bucket"
]
}
]
}
Further Reading
- Using an IAM Role to Grant Permissions to Applications Running on Amazon EC2 Instances
- Using Instance Profiles
Conclusion
In this two-part series we have shown how Cloudera Altus cloud services put you in control of your infrastructure, enabling Cloudera to manage clusters in your cloud provider’s account, and to control the access to your cloud storage. This information should make you confident that your security is taken seriously, and that Cloudera does not have access to your data.
Cloudera Altus cloud services provide a multi-cloud PaaS solution designed to automate massive-scale data engineering and data warehouse workloads in your public cloud, without the headache of managing the infrastructure yourself. It gives end users complete control over which cloud resources Altus clusters can use without giving access to data in your cloud account.
Editor's Choice
