Create a storage bucket

This feature requires HCP Boundary or Boundary Enterprise

As of Boundary 0.13.0, you can record and audit user sessions. A Boundary resource known as a storage bucket is used to store the recorded sessions. The storage bucket represents a bucket in an external store. Before you can enable session recording, you must create one or more storage buckets in Boundary and associate them with the external store.

A storage bucket can only belong to the global scope or an org scope. A storage bucket that is associated with the global scope can be associated with any target. However, a storage bucket in an Org scope can only be associated with targets in a project from the same org scope. Any storage buckets associated with an Org scope are deleted when the org itself is deleted.

For more information about using session recording to audit user sessions, refer to Auditing.

Requirements

Before you can create a storage bucket in Boundary, you must ensure that your environment meets certain requirements. Session recording requires specific configuration for both the external store and the Boundary worker. At this time, the only supported storage is AWS S3.

Storage requirements

A BSR (Boundary Session Recording) captures all the data transmitted between a user and a target during a single session. As a result, the size of a BSR is dependent on user activity. At a minimum, a BSR for a session with one connection requires 8KB of space for its files- this is the overhead for BSR keys, checksums, metadata, etc.

Determining how much storage you need to allocate on workers and AWS for recordings depends on user activity, but the following two examples are provided to help with storage estimates:

For a minute of simple shell activity, a BSR can be around 20KB in size. The storage requirements for 1,000 such sessions would be 20MB.
Sending 50MB of data results in a BSR around 50.1MB in size. The storage requirements for 1,000 such sessions would be 50.1 GB.

When you estimate worker storage requirements, consider the number of concurrent sessions that will be recorded on that worker. Boundary writes the BSR to the worker's local storage while the session is active, and then moves it to the remote storage bucket when the session is closed.

When you estimate AWS storage requirements, consider your storage policy and how long a BSR will be retained in the storage bucket.

AWS requirements

An AWS S3 storage bucket
You must associate the Boundary storage bucket with an AWS S3 storage bucket. An AWS S3 storage bucket contains the bucket name, region, and optional prefix, as well as any credentials needed to access the bucket.
The AWS S3 storage bucket can use static or dynamic credentials. You can configure static credentials using an access key and secret key or dynamic credentials using the AWS AssumeRole API.

An AWS IAM role policy with the following statement:

 {
   "Version": "2012-10-17",
   "Statement": [
      {
         "Action": [
            "s3:PutObject",
            "s3:GetObject",
            "s3:GetObjectAttributes",
            "s3:DeleteObject",
            "s3:ListBucket"
         ],
         "Effect": "Allow",
         "Resource": "arn:aws:s3:::session_recording_storage*",
         "Resource": "arn:aws:s3:::session_recording_storage/foo/bar/zoo/*"
      },
      {
         "Action": [
            "iam:DeleteAccessKey",
            "iam:GetUser",
            "iam:CreateAccessKey"
         ],
         "Effect": "Allow",
         "Resource": "arn:aws:iam::123456789012:user/JohnDoe"
      }
   ]
}

If you apply KMS encryption to the storage bucket, you must add these additional permissions to the role policy for the storage bucket's IAM user:

{
   "Action": [
   "kms:Decrypt",
   "kms:GenerateDataKey",
   "kms:DescribeKey"
   ],
   "Effect": "Allow",
   "Resource": "arn:aws:kms:us-east-1:1234567890:key/uuid"
}

The following is an example working policy with KMS encryption configured on the S3 bucket:

{
  "Version": "2012-10-17",
  "Statement": [
     {
        "Sid": "S3Permissions",
        "Effect": "Allow",
        "Action": [
           "s3:PutObject",
           "s3:GetObject",
           "s3:GetObjectAttributes",
           "s3:DeleteObject",
           "s3:ListBucket"
        ],
        "Resource": [
           "arn:aws:s3:::test-session-recording-bucket/*"
        ]
     },
     {
        "Sid": "UserPermissions",
        "Effect": "Allow",
        "Action": [
           "iam:DeleteAccessKey",
           "iam:GetUser",
           "iam:CreateAccessKey"
        ],
        "Resource": [
           "arn:aws:iam::1234567890:user/test-boundary"
        ]
     },
     {
        "Sid": "KMSPermissions",
        "Effect": "Allow",
        "Action": [
           "kms:Decrypt",
           "kms:GenerateDataKey",
           "kms:DescribeKey"
        ],
        "Resource": [
           "arn:aws:kms:us-east-2:1234567890:key/4b887395-c376-4936-8f37-80c592ea582c"
        ]
     }
  ]
}

Boundary workers requirements

Session recording requires that you configure at least one worker for local storage. The worker that you configure for storage must:

Note

You cannot use an HCP managed worker for the local storage. HCP Boundary users must configure a self-managed worker to enable session recording.

Have access to the AWS S3 storage bucket
Have an accessible directory defined by recording_storage_path for storing session recordings while they are in progress. On session closure, Boundary moves the local session recording to remote storage and deletes the local copy.
- For HCP Boundary, refer to the Self-managed worker configuration documentation.
- For Boundary Enterprise, refer to the refer to the worker configuration documentation.
Have available disk space defined by recording_storage_minimum_available_capacity. If you do not configure the minimum available storage capacity, Boundary uses the default value of 500MiB.
Run Darwin, Windows, or Linux. The following binaries are not supported for session recording: NetBSD, OpenBSD, Solaris.

Local storage

Workers require sufficient local disk space in the recording_storage_path to store session recordings while they are in progress. Local disk space is also required for session recording playback.

The recording_storage_minimum_available_capacity vaule defines the minimum available disk space that must be available on the worker for session recording and session recording playback. This threshold determines the local storage state of the worker. The possible storage states based on the recording_storage_minimum_available_capacity are:

Available: The worker is above the storage threshold and is available to proxy sessions that are enabled with session recording.
Low storage: The worker is below the storage threshold. Existing sessions can continue without interruption, but new sessions that are enabled with session recording cannot be proxied. The worker is not available to record new sessions or play back existing recordings.
Critically low storage: The worker is below half the storage threshold. Existing sessions that are enabled with session recording will be forcefully closed. The worker is not available to record new sessions or play back existing recordings.
Out of storage: The worker is out of local disk space. It is not available to record new sessions or play back existing recordings. The worker is in a unrecoverable state. An administrator must intervene to remedy the issue.
Not configured: The worker does not have a recording_storage_path configured.
Unknown: The default local storage state of a worker. This state indicates that the local storage state of a worker is not yet known. Older workers (< 0.16.0) will be in this state.

If a worker is in an unhealthy local storage state, Boundary does not allow new session recordings or session recording playback until the worker is in an available local storage state.

Refer to the following example configuration:

worker {
auth_storage_path="/boundary/demo-worker-1"
initial_upstreams = ["10.0.0.1"]
recording_storage_path="/local/storage/directory"
recording_storage_minimum_available_capacity="500MB"
}

Create a storage bucket

Complete the following steps to create a storage bucket in Boundary for session recording:

The required fields for creating a storage bucket vary depending on whether you configured the AWS S3 bucket with static or dynamic credentials:

Log in to Boundary.
Use the following command to create a storage bucket in Boundary:
```
boundary storage-buckets create \
   -bucket-name mybucket1 \
   -plugin-name aws \
   -secrets ‘{“access_key_id”: “123456789” , “secret_access_key” : “123/456789/12345678”}’ \
   -worker-filter ‘“dev” in “/tags/type”’ \
   -attributes ‘{“region”:”us-east-1”,”disable_credential_rotation”:true}’ \
   -scope-id o_1234567890
```
Replace the values above with the following required AWS secrets and any optional attributes you want to associate with the Boundary storage bucket:
- region: (Required) The AWS region to use.
- bucket-name: (Required) Name of the AWS bucket you want to associate with the Boundary storage bucket.
- access_key_id: (Required) The AWS access key to use.
- secret_access_key_id: (Required) The AWS secret access key to use. This attribute contains the secret access key for static credentials.
- worker-filter: (Required) A filter that indicates which Boundary workers have access to the storage. The filter must match an existing worker in order to create a Boundary storage bucket.
- shared_credentials_file: (Optional) The shared credentials file to use.
- shared_credentials_profile: (Optional) The profile name to use in the shared credentials file.
- disable_credential_rotation: (Optional) Prevents the AWS plugin from automatically rotating credentials.
  Although credentials are stored encrypted in Boundary, by default the AWS plugin attempts to rotate the credentials you provide. The given credentials are used to create a new credential, and then the original credential is revoked. After rotation, only Boundary knows the client secret the plugin uses.

Log in to Boundary.
Use the following command to create a storage bucket in Boundary:
```
boundary storage-buckets create \
   -bucket-name mybucket1 \
   -plugin-name aws \
   -worker-filter ‘“dev” in “/tags/type”’ \
   -attributes ‘{“region”:”us-east-1”,”disable_credential_rotation”:true,"role_arn":"arn:aws:iam::123456789012:role/S3Access"}’ \
   -scope-id o_1234567890
```
Replace the values above with the following required AWS secrets and any optional attributes you want to associate with the Boundary storage bucket:
- region: (Required) The AWS region to use.
- bucket-name: (Required) Name of the AWS bucket you want to associate with the Boundary storage bucket.
- role_arn: (Required) The ARN (Amazon Resource Name) role that is attached to the EC2 instance that the self-managed worker runs on.
- role_external_id: (Optional) A required value if you delegate third party access to your AWS resources. For more information, refer to the AWS documentation for How to use an external ID when granting access to your AWS resources to a third party.
- role_session_name: (Optional) A unique identifier for the AWS session. You can use this value to control how IAM principals and applications name their role sesions when they assume an IAM role. By providing a session name, you enable tracking session actions in AWS CloudTrail logs. For more information, refer to the AWS documentation for Logging IAM and AWS STS API calls with AWS CloudTrail.
- role_tags: (Optional) An object with key-value pair attributes that is passed when you assume an IAM role. For more information, refer to the AWS documentation for Passing session tags in AWS STS.
- worker-filter: (Required) A filter that indicates which Boundary workers have access to the storage. The filter must match an existing worker in order to create a Boundary storage bucket.
- shared_credentials_file: (Optional) The shared credentials file to use.
- shared_credentials_profile: (Optional) The profile name to use in the shared credentials file.
- disable_credential_rotation: (Required) Prevents the AWS plugin from automatically rotating credentials. This option must be set to true if you use dynamic credentials.

Log in to Boundary.
Select Storage Buckets in the navigation bar.
Select New Storage Bucket.
Complete the following fields to create the Boundary storage bucket:
- Name: (Optional) The name field is optional, but if you enter a name it must be unique.
- Description: (Optional) An optional description of the Boundary storage bucket for identification purposes.
- Scope: (Required) A storage bucket can belong to the Global scope or an Org scope. It can only associated with targets from the scope it belongs to.
- Bucket name: (Required) Name of the AWS bucket you want to associate with the Boundary storage bucket.
- Bucket prefix: (Optional) A base path where session recordings are stored.
- Region: (Required) The AWS region to use.
- Credential type: (Required) The type of credential you want to use to authenticate to the external storage. The required fields for creating a storage bucket vary depending on whether you configured the AWS S3 bucket with static or dynamic credentials:
  - Static: Authenticates to the storage bucket using an access key that AWS generates.
  - Dynamic: Authenticates to the storage bucket using credentials that were generated by AWS AssumeRole.
- Access key ID: (Required) The access key ID that AWS generates for the IAM user to use with the storage bucket.
- Secret access key: (Required) The secret access key that AWS generates for the IAM user to use with this storage bucket.
- Worker filter: (Required) A filter that indicates which Boundary workers have access to the storage. The filter must match an existing worker in order to create a Boundary storage bucket.
- Disable credential rotation: (Optional) Prevents the AWS plugin from automatically rotating credentials.
  Although credentials are stored encrypted in Boundary, by default the AWS plugin attempts to rotate the credentials you provide. The given credentials are used to create a new credential, and then the original credential is revoked. After rotation, only Boundary knows the client secret the plugin uses.
- Role ARN: (Required) The ARN (Amazon Resource Name) role that is attached to the EC2 instance that the self-managed worker runs on.
- Role external ID: (Optional) A required value if you delegate third party access to your AWS resources. For more information, refer to the AWS documentation for How to use an external ID when granting access to your AWS resources to a third party.
- Role session name: (Optional) A unique identifier for the AWS session. You can use this value to control how IAM principals and applications name their role sesions when they assume an IAM role. By providing a session name, you enable tracking session actions in AWS CloudTrail logs. For more information, refer to the AWS documentation for Logging IAM and AWS STS API calls with AWS CloudTrail.
- Role tags: An object with key-value pair attributes that is passed when you assume an IAM role. For more information, refer to the AWS documentation for Passing session tags in AWS STS.
- Worker filter: (Required) A filter that indicates which Boundary workers have access to the storage. The filter must match an existing worker in order to create a Boundary storage bucket.
- Disable credential rotation: (Required) Prevents the AWS plugin from automatically rotating credentials. This option is required if you use dynamic credentials.
Select Save.

Boundary creates the storage bucket resource and provides you with the bucket's ID.

Next steps

After the storage bucket is created in Boundary, you can use the bucket's ID to enable session recording on targets.