Securing s3 Access for Warehouse Storage
In order to access Tabular tables securely, you must configure an AWS IAM role with appropriate permissions to the s3 bucket used to store the table’s data and grant Tabular’s authorization service the ability to assume the role.
IAM Policy
Tabular requires the following actions on an s3 bucket for warehouse storage:
- s3:ListBucket
- s3:GetBucketLocation
- s3:GetBucketNotification
- s3:PutBucketNotification
- s3:GetObject
- s3:PutObject
- s3:PutObjectAcl
- s3:DeleteObject
- s3:AbortMultipartUpload
Example IAM policy (substitute <s3-bucket-name> with your s3 bucket name)
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"s3:ListBucket",
"s3:GetBucketLocation",
"s3:GetBucketNotification",
"s3:PutBucketNotification"
],
"Resource": [
"arn:aws:s3:::<s3-bucket-name>"
]
},
{
"Effect": "Allow",
"Action": [
"s3:PutObject",
"s3:GetObject",
"s3:DeleteObject",
"s3:PutObjectAcl",
"s3:AbortMultipartUpload"
],
"Resource": [
"arn:aws:s3:::<s3-bucket-name>/*"
]
}
]
}
IAM Role
When creating the IAM role, use the following settings:
- Trusted Entity Type:
AWS Account - AWS Account ID:
237881912361 - External ID:
<user-defined-value>
When creating a warehouse in the Tabular app, a random UUID is generated as a suggested external id, but this can be any arbitrary string value. See AWS documentation for more details.
Permission Policies: See IAM Policy
Example Trust Policy (for the trusted entities section):
{
"Version": "2008-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::237881912361:root"
},
"Action": [
"sts:AssumeRole",
"sts:TagSession"
],
"Condition": {
"StringEquals": {
"sts:ExternalId": "<user-defined-value>"
},
"ArnLike": {
"aws:PrincipalArn": "arn:aws:iam::237881912361:role/TabularSignerServiceRole*"
}
}
}
]
}
Inventory Lists
When enabled, Tabular will automatically process raw s3 bucket inventory lists files provided by AWS
and load the data into the system.s3_inventory_list table in your Tabular warehouse
for convenient access.
Enabling this feature requires creating an s3 bucket inventory configuration and a corresponding event notification to notify Tabular when new inventory files become available.
Inventory Configuration
When creating the s3 bucket inventory configuration, use the following settings:
- Inventory scope:
Include all versions - Destination bucket:
This account - Destination:
s3://<s3-bucket-name>/inventory - Frequency:
Daily - Output format:
Apache Parquet - Status:
Enable - Additional metadata fields:
Select ALL
Substitute
<s3-bucket-name>with your s3 bucket name
Event Notification for Inventory Lists
When creating the s3 bucket event notification, use the following settings:
- Prefix:
inventory/ - Suffix:
checksum - Event Types:
All object create events - Destination:
SNS topic - SNS topic ARN:
arn:aws:sns:us-east-1:237881912361:warehouses-s3-inventory-list-events
Access Logs
When enabled, Tabular will automatically process raw s3 bucket access log files provided by AWS
and load the data into the system.s3_access_logs table in your Tabular warehouse
for convenient access.
Enabling this feature requires enabling the Server access logging on the bucket and a
corresponding event notification to notify Tabular when new access files become available.
Server Access Logging
Set the Server access logging property for the bucket to Enable and set the target bucket to s3://<your-bucket-name>/access_logs/.
Event Notification for Access Logs
When creating the s3 bucket event notification, use the following settings:
- Prefix:
access_logs/ - Event Types:
All object create events - Destination:
SQS topic - SQS topic ARN:
arn:aws:sqs:us-east-1:237881912361:warehouses-s3-access-logs