Use SSE-C in Object Storage with the Secrets Manager
Last updated on
Prerequisites
Section titled “Prerequisites”- An active STACKIT project with Object Storage enabled and a bucket created. For instructions, read Enable Object Storage and Create and manage Object Storage buckets.
- Object Storage access credentials. For instructions, read Create Object Storage credentials.
- Access to STACKIT Secrets Manager with a configured mount point and an existing secret path.
- Python 3 and
pipinstalled on your system. opensslinstalled on your system.
Generate and store an encryption key
Section titled “Generate and store an encryption key”First, create a secure random key and its corresponding MD5 hash, and then store both values in the
Secrets Manager.
Open a terminal and run the following commands. The commands generate the values and write them
directly to the Secrets Manager.
-
Generate a 32-byte (256-bit) random key and base64-encode it
Terminal window KEY_B64=$(openssl rand 32 | base64) -
Generate the MD5 hash of the RAW key and base64-encode that hash
Terminal window KEY_MD5_B64=$(echo -n "$KEY_B64" | base64 --decode | openssl dgst -md5 -binary | base64) -
Display the keys
Terminal window echo "Base64 Key: $KEY_B64"echo "Base64 MD5: $KEY_MD5_B64"
Install the requirements for the Python script
Section titled “Install the requirements for the Python script”To install the required dependencies, create a file named requirements.txt in your project’s root directory and add the following lines:
boto3 hvacThen, run the following command in your terminal:
pip install -r requirements.txtSave the example Python script
Section titled “Save the example Python script”Save the Python code provided in this tutorial to a file on your local machine. For example, name the file ssec_secrets_manager_example.py.
Configure the example Python script
Section titled “Configure the example Python script”Next, configure the script to match your environment.
Open the ssec_secrets_manager_example.py file in a text editor and update the configuration variables at the top of the file. Make sure the following values are set correctly:
import osimport boto3import hvacfrom botocore.exceptions import ClientError
# --- CONFIGURATION ---# Please change the following values to match your setupS3_ENDPOINT_URL = "https://object.storage.eu01.onstackit.cloud"# This is the name of your bucket.S3_BUCKET_NAME = "YOUR-BUCKET-NAME"# This is the name of the file in the object storage.OBJECT_KEY = "my-secret-object.txt"# This is the name of the local file that will be created and uploaded in this example.FILE_TO_UPLOAD = "secret-upload.txt"# This is the name of the local file where the downloaded content will be saved.DOWNLOAD_FILE_PATH = "secret-download.txt"# Secrets Manager configuration# We are compatible with the Secrets Manager to HashiCorp Vault. That is why we use the official Hashicorp Python Library# See the following Documentation for the Basics of the Secret Manager https://support.docs.stackit.cloud/display/STACKIT/Vault+CLI+Basics+of+Secrets+ManagerVAULT_ADDR = "https://prod.sm.eu01.stackit.cloud"VAULT_MOUNT_POINT = "YOUR-SECRETS-MANAGER-MOUNT-POINT"VAULT_SECRET_PATH = "YOUR-SECRETS-MANAGER-SECRET-PATH"
"""Following Variables need to be set in env:# For Secrets Managerexport VAULT_USERNAME="secret-manager-username"export VAULT_PASSWORD="seceret-manager-password"
# We are compatible with the Object Storage to AWS S3. That is why we use the official AWS SDK for python.# For Object Storage or save it in (e.g., ~/.aws/credentials)export AWS_ACCESS_KEY_ID="YOUR_S3_ACCESS_KEY"export AWS_SECRET_ACCESS_KEY="YOUR_S3_SECRET_KEY""""
def get_ssec_keys_from_secrets_manager(vault_addr): """ Connects to Secrets Manager using env variables, authenticates via user/pass, and fetches the SSE-C keys. """ print("Attempting to fetch secrets from Secrets Manager") try: vault_user = os.environ["VAULT_USERNAME"] vault_pass = os.environ["VAULT_PASSWORD"] except KeyError as e: print(f"Error: Environment variable {e} not set.") return None, None
# Initialize the HVAC Client client = hvac.Client(url=vault_addr)
try: # Authenticate using User/Pass method client.auth.userpass.login(username=vault_user, password=vault_pass) if not client.is_authenticated(): print("Secrets Manager authentication failed.") return None, None
print("Secrets Manager authentication successfull.")
# Read the secrets from the Secrets Manager secret_response = client.secrets.kv.v2.read_secret_version( path=VAULT_SECRET_PATH, mount_point=VAULT_MOUNT_POINT )
# Extract the key Data secret_data = secret_response["data"]["data"] key_b64 = secret_data.get("KEY_B64") key_md5_b64 = secret_data.get("KEY_MD5_B64") print("Successfully fetched SSE-C keys from Secrets Manager.") return key_b64, key_md5_b64
except Exception as e: print(f"An error occurred while communicating with the Secrets Manager: {e}") return None, None
def s3_put_object(s3_client, bucket, object_key, file_path, ssec_key, ssec_key_md5): """ Example: Uploads a file to Object Storage using SSE-C """ print(f"\nUploading '{file_path}' to '{bucket}/{object_key} using SSE-C...") try: with open(file_path, "rb") as f: response = s3_client.put_object( Bucket=bucket, Key=object_key, Body=f, SSECustomerAlgorithm="AES256", SSECustomerKey=ssec_key, SSECustomerKeyMD5=ssec_key_md5, ) print("Upload successful.") print(f"ETag: {response.get('ETag')}") print(f"SSE-C Algorithm: {response.get('SSECustomerAlgorithm')}") except ClientError as e: print(f"Error uploading object: {e}")
def s3_get_object(s3_client, bucket, object_key, download_path, ssec_key, ssec_key_md5): """ Example: Downloads and decrypts an object from Object Storage using SSE-C. """ print(f"\nDownloading '{bucket}/{object_key} to '{download_path}' using SSE-C...")
try: response = s3_client.get_object( Bucket=bucket, Key=object_key, SSECustomerAlgorithm="AES256", SSECustomerKey=ssec_key, SSECustomerKeyMD5=ssec_key_md5, )
# Stream the decrypted content to a local file with open(download_path, "wb") as f: f.write(response["Body"].read())
print("Download successful.") print(f"File saved to: '{download_path}'") print(f"Content-Length: {response.get('ContentLength')} bytes") except ClientError as e: print(f"Error downloading object: {e}")
if __name__ == "__main__":
# 1. Fetch keys from Secret-Manager key_b64, key_md5_b64 = get_ssec_keys_from_secrets_manager(VAULT_ADDR)
# 2. Check if we got both key values if not key_b64 or not key_md5_b64: print("\nCould not retrieve one or both keys from Secrets Manager. Exiting.") else: # 3. Create a dummy file to upload with open(FILE_TO_UPLOAD, "w") as f: f.write( "This is a highly confidential message encrypted with a customer key." )
# 4. Initialize Boto3 S3 client for Object-Storage s3 = boto3.client("s3", endpoint_url=S3_ENDPOINT_URL)
# 5. Perform the PUT and GET operations s3_put_object( s3, S3_BUCKET_NAME, OBJECT_KEY, FILE_TO_UPLOAD, key_b64, key_md5_b64 ) s3_get_object( s3, S3_BUCKET_NAME, OBJECT_KEY, DOWNLOAD_FILE_PATH, key_b64, key_md5_b64 ) # 6. Print the downloaded Data print(f"\nVerifying content of '{DOWNLOAD_FILE_PATH}'") with open(DOWNLOAD_FILE_PATH, "r") as f: print(f"Content: '{f.read()}'")Set environment variables
Next, export the following environment variables in your terminal. The script uses them to securely access your credentials without hard coding sensitive values. Replace the placeholders with your actual credentials.
## For Secrets Managerexport VAULT_USERNAME='secret-manager-username'export VAULT_PASSWORD='secret-manager-password'# For Object Storage or save it in (e.g., \~/.aws/credentials)export AWS_ACCESS_KEY_ID='YOUR_S3_ACCESS_KEY'export AWS_SECRET_ACCESS_KEY='YOUR_S3_SECRET_KEY'Run the script
Section titled “Run the script”With the configuration complete, run the script:
python ssec_secrets_manager_example.pyVerify the results
Section titled “Verify the results”After the script finishes, verify its actions in two places:
- Local folder: Two identical files should be present:
- secret-upload.txt: The original file created and uploaded by the script.
- secret-download.txt: The decrypted copy downloaded from Object Storage.
- Object Storage bucket: The encrypted object (my-secret-object.txt) is now stored in your bucket. Confirm this with the AWS CLI:
aws s3 ls s3://<your-bucket-name>/ --endpoint-url <your-endpoint-url>Prove the encryption
Section titled “Prove the encryption”To prove that the object is encrypted and cannot be accessed without the key, try downloading it
with standard AWS CLI commands.
These attempts fail, demonstrating the effectiveness of SSE-C.
-
Attempt with AWS s3 cp (fails)
The high-level s3 cp command does not support the required SSE-C headers.
Terminal window ## This command will failaws s3 cp s3://<your-bucket-name>/my-secret-object.txt ./download-fail-1.txt --endpoint-url <your-endpoint-url>Expected error:
Terminal window An error occurred (400) when calling the HeadObject operation: Bad Request. -
Attempt with AWS s3api get-object without keys (fails)
The low-level s3api command also fails if you don’t provide the encryption key.
Terminal window ## This command will failaws s3api get-object --bucket <your-bucket-name> --key my-secret-object.txt ./download-fail-2.txt --endpoint-url <your-endpoint-url>Expected error:
Terminal window An error occurred (InvalidRequest) when calling the GetObject operation:The object was stored using a form of Server Side Encryption.The correct parameters must be provided to retrieve the object. -
Download with AWS s3api get-object and the encryption key (succeeds)
To download the file successfully with the AWS CLI, you must provide the same key and MD5 hash that were used in the Python script.
First, export the values as environment variables:
Terminal window export KEY_B64="YOUR-B64-KEY"export KEY_MD5_B64="YOUR-MD5-B64-KEY"Next, run the
get-objectcommand with the required SSE-C parameters:Terminal window ## This command will succeedaws s3api get-object \--bucket \--key my-secret-object.txt \--sse-customer-algorithm AES256 \--sse-customer-key "$KEY_B64" \--sse-customer-key-md5 "$KEY_MD5_B64" \--endpoint-url \./download-success.txt
This command downloads and decrypts the object, proving that access is only granted when the correct encryption key is provided.