At work we've begun exploring the use of IAM policy conditions to limit access to items in our DynamoDB Tables. This gives a much more specific level of control on who can retrieve what data. Before a lambda executes, a separate process generates an IAM policy that allows access to DynamoDB based on certain conditions. By using the dynamodb:LeadingKeys
condition you can restrict access based on the Partition Key of the item.
Since these policies are intended to be user, or tenant, specific they can lock down your data so even if the user made a more generic query they wouldn't have access to get another tenants data.
Frequently, this process is known as using a Token Vending Machine to do Tenant Isolation.
Something we came across that wasn't immediately clear in the https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/specifying-conditions.html (or google / stackoverflow) was whether or not using the condition key of dynamodb:LeadingKeys
would work on Global Secondary Indexes... or not.
In the docs defense, they do say this about the LeadingKeys condition:
Represents the first key attribute of a tableβin other words, the partition key.
TL;DR - Since a GSI has its own partition key using the LeadingKeys condition works as expected. Yay for accurate docs π
The remainder of this post will walkthrough how to do a basic implementation using the LeadingKeys condition.
Code: https://github.com/martzcodes/blog-isolation
I bootstrapped the project using projen so ultimately the only folder you really need to focus on is the /src folder.
The stack consists of:
- 1 DynamoDB table with a GSI (creatively named
gsi
) - 3 Lambdas (using two source files)
- 1 Role to be assumed by one of the lambdas
- 1 API Gateway (RestApi)
- 5 Methods (all GETs) pointing to the three Lambdas
The Lambdas
Bootstrap Lambda
One of the lambdas is simply used to bootstrap the database with a static set of data:
The /init
endpoint invokes the bootstrap lambda... if you run it multiple times the data will look the same.
The Reader Lambda
The other lambda is a reader lambda which uses path parameters and query parameters to make dynamodb queries with and without granted permissions, roles, and whether or not to use the GSI.
The handler method for the reader lambda makes use of middy which is a simple middleware engine. The middleware looks to see if the credentials
query parameter is set and if it is adds a credentials object to the incoming context. When the DynamoDB client is initialized, it uses this credentials object instead of the lambdas own role.
Token Vending Machine
In order to create credentials for DynamoDB to use, we have to attach policy statements to an already existing Role. This is "1 Role" we mentioned above in the stack list.
When the getCredentials method is called, it creates a policy statement that Allows access to DynamoDB based on what the query parameter was set to:
const Policy = JSON.stringify({
Statement: [
{
Action: 'dynamodb:Query',
Condition: {
'ForAllValues:StringLike': {
'dynamodb:LeadingKeys': [`Tenant#${who}`],
},
},
Effect: Effect.ALLOW,
Resource: `${process.env.TableArn}`,
},
// When using GSIs the below is required...
{
Action: 'dynamodb:Query',
Condition: {
'ForAllValues:StringLike': {
'dynamodb:LeadingKeys': [`Tenant#${who}`],
},
},
Effect: Effect.ALLOW,
Resource: `${process.env.TableArn}/*`,
},
],
});
With the policy statement ready, Secure Token Service(STS) assumeRole
is called to get the credentials that will be used.
This example is only granting access to the dynamodb:Query
action... you may need additional logic to restrict other actions.
The API
The basic endpoint structure is:
<apigw url>/prod/<grant or noGrant>/<tenant number to query>/<optional: whether to use the gsi or not>
The credentials
query parameter will indicate which tenant number the generated policy should use.
You should absolutely use an authorizer lamdba or some other way of securing your API.
The Experiments
Note: If you try these yourself it may cost you some money... probably not very much though. I ran it for a few hours in a few variations yesterday and my bill was $0 since most of this falls in the free tier. If you've already exceeded that I'd expect it to be a few dollars, at most.
npx projen
npx projen build
# npx cdk deploy # RUNNING THIS MAY COST YOU MONEY
# npx cdk destroy # don't forget to destroy afterwards!
Once the stack is deployed, grab the URL at the end
Outputs:
blogIsolationStack.blogIsolationApiEndpoint237C4FAE = https://12d6dsdjl0.execute-api.us-east-1.amazonaws.com/prod/
and init the database
$ curl https://12d6dsdjl0.execute-api.us-east-1.amazonaws.com/prod/init
Verify the Baseline
First, check that our APIs are working and the items are there.
$ curl https://12d6dsdjl0.execute-api.us-east-1.amazonaws.com/prod/grant/1
{"readRes":[{"SK1":"Item#1","SK":"Item#1","PK":"Tenant#1","PK1":"Tenant#1"}]}
$ curl https://12d6dsdjl0.execute-api.us-east-1.amazonaws.com/prod/grant/2
{"readRes":[{"SK1":"Item#2","SK":"Item#2","PK":"Tenant#2","PK1":"Tenant#2"},{"SK1":"Item#3","SK":"Item#3","PK":"Tenant#2","PK1":"Tenant#1"}]}
- β Tenant 1 only has 1 item on its primary Partition Key
- β Tenant 2 has 2 items on its PK
Verify the GSI Query
Adding a /<something thats truthy>
to the end makes DDB use the GSI index for the query.
$ curl https://12d6dsdjl0.execute-api.us-east-1.amazonaws.com/prod/grant/1/1
{"readRes":[{"SK":"Item#1","SK1":"Item#1","PK1":"Tenant#1","PK":"Tenant#1"},{"SK":"Item#3","SK1":"Item#3","PK1":"Tenant#1","PK":"Tenant#2"}]}
$ curl https://12d6dsdjl0.execute-api.us-east-1.amazonaws.com/prod/grant/2/1
{"readRes":[{"SK":"Item#2","SK1":"Item#2","PK1":"Tenant#2","PK":"Tenant#2"}]}
- β Tenant 1 only has 2 items using the GSI
- β Tenant 2 has 1 item using the GSI
Verify Grants are Needed
Not sure why I included this when I wrote the code, but as a sanity check make sure you actually need to grantRead access to the dynamodb table:
$ curl https://12d6dsdjl0.execute-api.us-east-1.amazonaws.com/prod/noGrant/1
{"message": "Internal server error"}
$ curl https://12d6dsdjl0.execute-api.us-east-1.amazonaws.com/prod/noGrant/1\?credentials\=1
{"message": "Internal server error"}
- β Can't read items on the primary key without a grant
- β Can't read items on the GSI without a grant
Verify Using the Primary Index with AssumeRole Credentials
$ curl https://12d6dsdjl0.execute-api.us-east-1.amazonaws.com/prod/grant/1\?credentials\=1
{"readRes":[{"SK1":"Item#1","SK":"Item#1","PK":"Tenant#1","PK1":"Tenant#1"}]}
$ curl https://12d6dsdjl0.execute-api.us-east-1.amazonaws.com/prod/grant/2\?credentials\=1
{"message": "Internal server error"}
$ curl https://12d6dsdjl0.execute-api.us-east-1.amazonaws.com/prod/grant/1\?credentials\=2
{"message": "Internal server error"}
$ curl https://12d6dsdjl0.execute-api.us-east-1.amazonaws.com/prod/grant/2\?credentials\=2
{"readRes":[{"SK1":"Item#2","SK":"Item#2","PK":"Tenant#2","PK1":"Tenant#2"},{"SK1":"Item#3","SK":"Item#3","PK":"Tenant#2","PK1":"Tenant#1"}]}
- β LeadingKeys Tenant#1 can see Tenant#1 PK items
- β LeadingKeys Tenant#1 can NOT see Tenant#2 PK items
- β LeadingKeys Tenant#2 can see Tenant#2 PK items
- β LeadingKeys Tenant#2 can NOT see Tenant#1 PK items
Verify Using the GSI with AssumeRole Credentials
$ curl https://12d6dsdjl0.execute-api.us-east-1.amazonaws.com/prod/grant/1/1\?credentials\=1
{"readRes":[{"SK":"Item#1","SK1":"Item#1","PK1":"Tenant#1","PK":"Tenant#1"},{"SK":"Item#3","SK1":"Item#3","PK1":"Tenant#1","PK":"Tenant#2"}]}
$ curl https://12d6dsdjl0.execute-api.us-east-1.amazonaws.com/prod/grant/2/1\?credentials\=1
{"message": "Internal server error"}
$ curl https://12d6dsdjl0.execute-api.us-east-1.amazonaws.com/prod/grant/1/1\?credentials\=2
{"message": "Internal server error"}
$ curl https://12d6dsdjl0.execute-api.us-east-1.amazonaws.com/prod/grant/2/1\?credentials\=2
{"readRes":[{"SK":"Item#2","SK1":"Item#2","PK1":"Tenant#2","PK":"Tenant#2"}]}
Now, using the GSI:
- β LeadingKeys Tenant#1 can see the 2 expected Tenant#1 PK1 items
- β LeadingKeys Tenant#1 can NOT see Tenant#2 PK1 items
- β LeadingKeys Tenant#2 can see the 1 expected Tenant#2 PK1 items
- β LeadingKeys Tenant#2 can NOT see Tenant#1 PK1 items
Conclusion
Using the LeadingKeys condition correctly uses the GSI's Partition Key!
Note: The Internal server error
thrown above is really an AccessDeniedException
because the user doesn't have the right access. Depending on your API design you may or may not want to return this... you might just want to return an empty list, for example.