Bring Your Own Bucket

With Bring Your Own Bucket, Oodle keeps the metrics and traces of your Oodle instance in an Amazon S3 bucket in your own AWS account. You own the bucket, and you control access to it through an IAM role that Oodle assumes.

Which data goes to your bucket

After the setup, Oodle writes this data to your bucket:

• Metrics
• Traces

Oodle writes only new data to your bucket. Data that Oodle wrote before the setup stays in the Oodle bucket, and you can continue to query it. Queries read from the two buckets and show one result.

Requirements

• Bring Your Own Bucket enabled for your instance. To enable it, contact Oodle support.
• An Oodle Admin role. Only an admin can see and change the bucket settings.
• An AWS account with permissions to create CloudFormation stacks, S3 buckets, and IAM roles.

Setup

1. In Oodle, go to Settings and open Bring Your Own Bucket in the Admin section.
2. Click Launch CloudFormation stack. The AWS console opens the stack in the region of your Oodle deployment, with all the required parameters already set. The stack fails in other regions, because Oodle cannot use a bucket in another region.
3. Create the stack. The stack creates:
   • An S3 bucket with public access blocked, SSE-S3 (AES256) default encryption, and the Oodle lifecycle rules.
   • An IAM role named OodleS3Integration-BYOB-<suffix> that Oodle can assume with the external ID.
4. When the stack status is CREATE_COMPLETE, open its Outputs tab. Copy the BucketName and RoleARN values into the settings page.
5. Click Validate and save. Oodle assumes the role, then writes, reads, lists, and deletes a test object in the bucket. If a check fails, the page shows which check failed and why.

New metrics and traces go to your bucket within 30 minutes after the save.

The CloudFormation template is available at https://s3.us-west-2.amazonaws.com/oodle-configs/storage/s3/byob-v1.yaml. If you create the stack from the template directly, set the OodleRegion parameter to the region of your Oodle deployment.

Choose the bucket name

The stack always creates a new bucket. To choose its name, enter the name in the BucketName parameter of the stack. The name must not be in use in any AWS account. If you keep the parameter empty, AWS makes a unique name.

You cannot use a bucket that you already have. A bucket that the stack creates always has the settings that Oodle needs:

• The lifecycle rules below. Oodle writes temporary data to these prefixes. Without the rules, the temporary data stays in the bucket and your storage cost increases.
• SSE-S3 default encryption. Oodle writes each object with SSE-S3 (ServerSideEncryption: AES256), so a bucket policy that requires SSE-KMS rejects the writes. Do not add such a policy to the bucket.

If you delete the stack, AWS keeps the bucket and its data.

Lifecycle rules

The stack adds these rules to the bucket. The write-ahead log (WAL) prefixes of all signals use one expiry, which you can set with the WalExpirationDays parameter (default 3 days).

Prefix	Expire objects after
lambda_query_res/	1 day
wal/	WalExpirationDays
wal_late/	WalExpirationDays
wal_TRACE	WalExpirationDays
logs/wal	WalExpirationDays

A rule for the full bucket also deletes incomplete multipart uploads 1 day after they start.

IAM permissions

The trust policy of the role lets the Oodle AWS account assume the role only with your external ID. The role has these permissions, and only on your bucket:

Action	Resource
s3:GetObject, s3:PutObject, s3:DeleteObject, s3:AbortMultipartUpload	arn:aws:s3:::<bucket>/*
s3:ListBucket, s3:GetBucketLocation	arn:aws:s3:::<bucket>

Oodle accepts only roles whose name starts with OodleS3Integration-BYOB- at path /. Keep the role name that the stack creates.

Change the bucket or the role

The bucket cannot be changed once it is set. Check the bucket name before you save.

You can replace the role for the same bucket. Create a new role with the stack, then enter the same bucket name and the new role ARN on the page and click Validate and save.