Skip to content

Use SSE-C in Object Storage with the Secrets Manager

Last updated on

First, create a secure random key and its corresponding MD5 hash, and then store both values in the Secrets Manager.
Open a terminal and run the following commands. The commands generate the values and write them directly to the Secrets Manager.

  1. Generate a 32-byte (256-bit) random key and base64-encode it

    Terminal window
    KEY_B64=$(openssl rand 32 | base64)
  2. Generate the MD5 hash of the RAW key and base64-encode that hash

    Terminal window
    KEY_MD5_B64=$(echo -n "$KEY_B64" | base64 --decode | openssl dgst -md5 -binary | base64)
  3. Display the keys

    Terminal window
    echo "Base64 Key: $KEY_B64"
    echo "Base64 MD5: $KEY_MD5_B64"

Install the requirements for the Python script

Section titled “Install the requirements for the Python script”

To install the required dependencies, create a file named requirements.txt in your project’s root directory and add the following lines:

Terminal window
boto3 hvac

Then, run the following command in your terminal:

Terminal window
pip install -r requirements.txt

Save the Python code provided in this tutorial to a file on your local machine. For example, name the file ssec_secrets_manager_example.py.

Next, configure the script to match your environment.

Open the ssec_secrets_manager_example.py file in a text editor and update the configuration variables at the top of the file. Make sure the following values are set correctly:

import os
import boto3
import hvac
from botocore.exceptions import ClientError
# --- CONFIGURATION ---
# Please change the following values to match your setup
S3_ENDPOINT_URL = "https://object.storage.eu01.onstackit.cloud"
# This is the name of your bucket.
S3_BUCKET_NAME = "YOUR-BUCKET-NAME"
# This is the name of the file in the object storage.
OBJECT_KEY = "my-secret-object.txt"
# This is the name of the local file that will be created and uploaded in this example.
FILE_TO_UPLOAD = "secret-upload.txt"
# This is the name of the local file where the downloaded content will be saved.
DOWNLOAD_FILE_PATH = "secret-download.txt"
# Secrets Manager configuration
# We are compatible with the Secrets Manager to HashiCorp Vault. That is why we use the official Hashicorp Python Library
# See the following Documentation for the Basics of the Secret Manager https://support.docs.stackit.cloud/display/STACKIT/Vault+CLI+Basics+of+Secrets+Manager
VAULT_ADDR = "https://prod.sm.eu01.stackit.cloud"
VAULT_MOUNT_POINT = "YOUR-SECRETS-MANAGER-MOUNT-POINT"
VAULT_SECRET_PATH = "YOUR-SECRETS-MANAGER-SECRET-PATH"
"""
Following Variables need to be set in env:
# For Secrets Manager
export VAULT_USERNAME="secret-manager-username"
export VAULT_PASSWORD="seceret-manager-password"
# We are compatible with the Object Storage to AWS S3. That is why we use the official AWS SDK for python.
# For Object Storage or save it in (e.g., ~/.aws/credentials)
export AWS_ACCESS_KEY_ID="YOUR_S3_ACCESS_KEY"
export AWS_SECRET_ACCESS_KEY="YOUR_S3_SECRET_KEY"
"""
def get_ssec_keys_from_secrets_manager(vault_addr):
"""
Connects to Secrets Manager using env variables, authenticates via user/pass,
and fetches the SSE-C keys.
"""
print("Attempting to fetch secrets from Secrets Manager")
try:
vault_user = os.environ["VAULT_USERNAME"]
vault_pass = os.environ["VAULT_PASSWORD"]
except KeyError as e:
print(f"Error: Environment variable {e} not set.")
return None, None
# Initialize the HVAC Client
client = hvac.Client(url=vault_addr)
try:
# Authenticate using User/Pass method
client.auth.userpass.login(username=vault_user, password=vault_pass)
if not client.is_authenticated():
print("Secrets Manager authentication failed.")
return None, None
print("Secrets Manager authentication successfull.")
# Read the secrets from the Secrets Manager
secret_response = client.secrets.kv.v2.read_secret_version(
path=VAULT_SECRET_PATH, mount_point=VAULT_MOUNT_POINT
)
# Extract the key Data
secret_data = secret_response["data"]["data"]
key_b64 = secret_data.get("KEY_B64")
key_md5_b64 = secret_data.get("KEY_MD5_B64")
print("Successfully fetched SSE-C keys from Secrets Manager.")
return key_b64, key_md5_b64
except Exception as e:
print(f"An error occurred while communicating with the Secrets Manager: {e}")
return None, None
def s3_put_object(s3_client, bucket, object_key, file_path, ssec_key, ssec_key_md5):
"""
Example: Uploads a file to Object Storage using SSE-C
"""
print(f"\nUploading '{file_path}' to '{bucket}/{object_key} using SSE-C...")
try:
with open(file_path, "rb") as f:
response = s3_client.put_object(
Bucket=bucket,
Key=object_key,
Body=f,
SSECustomerAlgorithm="AES256",
SSECustomerKey=ssec_key,
SSECustomerKeyMD5=ssec_key_md5,
)
print("Upload successful.")
print(f"ETag: {response.get('ETag')}")
print(f"SSE-C Algorithm: {response.get('SSECustomerAlgorithm')}")
except ClientError as e:
print(f"Error uploading object: {e}")
def s3_get_object(s3_client, bucket, object_key, download_path, ssec_key, ssec_key_md5):
"""
Example: Downloads and decrypts an object from Object Storage using SSE-C.
"""
print(f"\nDownloading '{bucket}/{object_key} to '{download_path}' using SSE-C...")
try:
response = s3_client.get_object(
Bucket=bucket,
Key=object_key,
SSECustomerAlgorithm="AES256",
SSECustomerKey=ssec_key,
SSECustomerKeyMD5=ssec_key_md5,
)
# Stream the decrypted content to a local file
with open(download_path, "wb") as f:
f.write(response["Body"].read())
print("Download successful.")
print(f"File saved to: '{download_path}'")
print(f"Content-Length: {response.get('ContentLength')} bytes")
except ClientError as e:
print(f"Error downloading object: {e}")
if __name__ == "__main__":
# 1. Fetch keys from Secret-Manager
key_b64, key_md5_b64 = get_ssec_keys_from_secrets_manager(VAULT_ADDR)
# 2. Check if we got both key values
if not key_b64 or not key_md5_b64:
print("\nCould not retrieve one or both keys from Secrets Manager. Exiting.")
else:
# 3. Create a dummy file to upload
with open(FILE_TO_UPLOAD, "w") as f:
f.write(
"This is a highly confidential message encrypted with a customer key."
)
# 4. Initialize Boto3 S3 client for Object-Storage
s3 = boto3.client("s3", endpoint_url=S3_ENDPOINT_URL)
# 5. Perform the PUT and GET operations
s3_put_object(
s3, S3_BUCKET_NAME, OBJECT_KEY, FILE_TO_UPLOAD, key_b64, key_md5_b64
)
s3_get_object(
s3, S3_BUCKET_NAME, OBJECT_KEY, DOWNLOAD_FILE_PATH, key_b64, key_md5_b64
)
# 6. Print the downloaded Data
print(f"\nVerifying content of '{DOWNLOAD_FILE_PATH}'")
with open(DOWNLOAD_FILE_PATH, "r") as f:
print(f"Content: '{f.read()}'")

Set environment variables

Next, export the following environment variables in your terminal. The script uses them to securely access your credentials without hard coding sensitive values. Replace the placeholders with your actual credentials.

Terminal window
## For Secrets Manager
export VAULT_USERNAME='secret-manager-username'
export VAULT_PASSWORD='secret-manager-password'
# For Object Storage or save it in (e.g., \~/.aws/credentials)
export AWS_ACCESS_KEY_ID='YOUR_S3_ACCESS_KEY'
export AWS_SECRET_ACCESS_KEY='YOUR_S3_SECRET_KEY'

With the configuration complete, run the script:

Terminal window
python ssec_secrets_manager_example.py

After the script finishes, verify its actions in two places:

  • Local folder: Two identical files should be present:
    • secret-upload.txt: The original file created and uploaded by the script.
    • secret-download.txt: The decrypted copy downloaded from Object Storage.
  • Object Storage bucket: The encrypted object (my-secret-object.txt) is now stored in your bucket. Confirm this with the AWS CLI:
Terminal window
aws s3 ls s3://<your-bucket-name>/ --endpoint-url <your-endpoint-url>

To prove that the object is encrypted and cannot be accessed without the key, try downloading it with standard AWS CLI commands.
These attempts fail, demonstrating the effectiveness of SSE-C.

  1. Attempt with AWS s3 cp (fails)

    The high-level s3 cp command does not support the required SSE-C headers.

    Terminal window
    ## This command will fail
    aws s3 cp s3://<your-bucket-name>/my-secret-object.txt ./download-fail-1.txt --endpoint-url <your-endpoint-url>

    Expected error:

    Terminal window
    An error occurred (400) when calling the HeadObject operation: Bad Request.
  2. Attempt with AWS s3api get-object without keys (fails)

    The low-level s3api command also fails if you don’t provide the encryption key.

    Terminal window
    ## This command will fail
    aws s3api get-object --bucket <your-bucket-name> --key my-secret-object.txt ./download-fail-2.txt --endpoint-url <your-endpoint-url>

    Expected error:

    Terminal window
    An error occurred (InvalidRequest) when calling the GetObject operation:
    The object was stored using a form of Server Side Encryption.
    The correct parameters must be provided to retrieve the object.
  3. Download with AWS s3api get-object and the encryption key (succeeds)

    To download the file successfully with the AWS CLI, you must provide the same key and MD5 hash that were used in the Python script.

    First, export the values as environment variables:

    Terminal window
    export KEY_B64="YOUR-B64-KEY"
    export KEY_MD5_B64="YOUR-MD5-B64-KEY"

    Next, run the get-object command with the required SSE-C parameters:

    Terminal window
    ## This command will succeed
    aws s3api get-object \
    --bucket \
    --key my-secret-object.txt \
    --sse-customer-algorithm AES256 \
    --sse-customer-key "$KEY_B64" \
    --sse-customer-key-md5 "$KEY_MD5_B64" \
    --endpoint-url \
    ./download-success.txt

This command downloads and decrypts the object, proving that access is only granted when the correct encryption key is provided.