In this post, we walk you through how to make your Amazon Timestream for InfluxDB deployments more secure by offering a mechanism to automatically rotate long-lived credentials. We use AWS Secrets Manager to store your tokens and user credentials as secrets and rotate the secrets using the included AWS Lambda functions. Timestream for InfluxDB only provides long-lived credentials without support for expiration or configurable time limits, which can pose a security risk if they become exposed. To learn more about Timestream for InfluxDB and secrets, see How Amazon Timestream for InfluxDB uses secrets.
Secrets Manager is a secret storage service that you can use to protect your Timestream for InfluxDB credentials. When you store credentials in a secret, you can replace hard-coded credentials with an API call to Secrets Manager. This reduces the risk of credentials accidentally being exposed. Secrets Manager supports rotating secrets by way of Lambda functions.
You can rotate Timestream for InfluxDB users and tokens by creating a new secret and using either the InfluxDB single- or multi-user secret rotation Lambda functions. Secret rotation for your database credentials is a security best practice defined in the Identity Management section of the AWS Well-Architected framework. When you rotate secrets with Secrets Manager, exposed credentials are only valid until another secret rotation occurs, limiting the credentials’ usability for malicious actors.
Follow along with this post to learn how to rotate long-lived credentials for your Timestream for InfluxDB instance. You can find the required Lambda function code on the Github repo to begin building your own automated secret rotation solution for your Timestream for InfluxDB users and tokens.
Solution overview
When you create a new Timestream for InfluxDB instance, an admin secret is automatically created with the admin user credentials in Secrets Manager. By default, this secret doesn’t have rotation enabled, and you can rotate the secret with the InfluxDB single-user secret rotation Lambda function.
The InfluxDB single-user secret rotation Lambda function rotates user credentials by authenticating the user defined in the secret and updating the password with a new random password.
The InfluxDB multi-user secret rotation Lambda function can rotate a token by creating a new permission identical token and automatically replacing the old token. The multi-user rotator can rotate an existing token in the Timestream for InfluxDB instance, or create a new token when the AUTHENTICATION_CREATION_ENABLED environment variable is set to true in the Lambda function.
The following diagram illustrates the architecture used for rotating secrets with Secrets Manager and the single- or multi-user secrets rotation Lambda functions.
Prerequisites
Before you deploy the solution, complete the following steps:
- Create a Timestream for InfluxDB instance. The secret rotation functions require a Timestream for InfluxDB instance to rotate credentials. Refer to Run and manage open source InfluxDB databases with Amazon Timestream to create a new instance if needed.
- When the Timestream for InfluxDB instance is available, choose your instance on the Amazon Timestream management console and take note of the authentication properties Secrets Manager Amazon Resource Name (ARN) and DB instance identifier; you will need these values to configure your secret rotation functions. To locate an admin secret for a Timestream for InfluxDB instance, use the authentication properties Secrets Manager ARN from the Timestream for InfluxDB instance summary page.
- To build the deployment, clone the secret rotation GitHub repository. Use the single-user function for rotating user credentials, and the multi-user function for rotating token credentials.
- Open your preferred terminal and navigate to the
SecretsManagerInfluxDBRotationMultiUser
directory for the multi-user function, or theSecretsManagerInfluxDBRotationSingleUser
directory for the single-user function. - For macOS or Linux, complete the following steps:
- Create a temporary directory for the application code and dependencies with the following command:
- Install the required dependencies into the temporary directory with the following command:
- Copy the Lambda code into the temporary directory with the following command:
- Zip the contents of the temporary directory for uploading to Lambda with the following command:
- Clean up by deleting the temporary directory with the following command:
- Make the deployment package executable with the following command:
The deployment is now ready with the name
influxdb-secret-rotation-lambda.zip
for uploading as a Lambda function. - For Windows, complete the following steps using Windows PowerShell prompt:
- Create a temporary directory for the application code and dependencies with the following command:
- Install the required dependencies into the temporary directory with the following command:
- Copy the Lambda code into the temporary directory with the following command:
- Zip the contents of the temporary directory for uploading to Lambda with the following command:
- Clean up by deleting the temporary directory with the following command:
The build script creates a file with the name influxdb-secret-rotation-lambda.zip
. Use this file to create a Lambda function in the following sections.
Deploy the Lambda function
Complete the steps in this section to deploy the Lambda function to rotate your secrets.
Create the Lambda function
To create your Lambda function, complete the following steps:
- On the Lambda console, choose Functions in the navigation pane.
- Choose Create function, and select Author from scratch.
- For Function name, enter a name for your function.
- For Runtime, choose Python 3.12.
- Choose Create function.
Upload the deployment package
Complete the following steps to upload the deployment package:
- On the function overview page of the Lambda function, choose Code.
- Choose Upload from and select .zip
- Choose Upload and choose the
influxdb-secret-rotation-lambda.zip
deployment package that was created by the build script in the previous section. - Choose Save.
Configure environment variables for the Lambda function
Complete the following steps:
- On the function overview page of the Lambda function, choose Configuration.
- Choose Environment Variables and choose Edit.
- Choose Add environment variable.
- Enter
SECRETS_MANAGER_ENDPOINT
for the key value. - Enter
https://secretsmanager.{region}.amazonaws.com
and replace{region}
with the same AWS Region where you created the Lambda function. - Choose Save.
If you are deploying the multi-user function, you can create non-operator tokens by leaving the token value blank in the secret. To enable this feature, add the following environment variable.
- Choose Add environment variable.
- Enter
AUTHENTICATION_CREATION_ENABLED
for the key and true for the value. - Choose Save.
Configure the Lambda function role
Complete the following steps to configure the AWS Identity and Access Management (IAM) role for the Lambda function:
- In the Configuration section of the function overview page, choose Permissions.
- Under Execution role, choose the link to the Lambda role name to view the role on the IAM console.
- Under Permissions policies, choose Add permissions and Create inline policy.
- For Policy editor, choose JSON.
- Replace the JSON in the text editor with either the multi-user secret rotation policy or the single-user secret rotation policy.
Use the following IAM policy as the role for the multi-user Lambda function. The policy gives the Lambda function the required permissions to perform a secret rotation for Timestream for InfluxDB tokens.
Replace the following items in the IAM policy with values from your AWS account:
- {rotating_secret_arn} – The ARN for the secret being rotated can be found in the Secrets Manager secret details
- {authentication_properties_admin_secret_arn} – The Timestream for InfluxDB admin secret ARN can be found on the Timestream for InfluxDB instance summary page
- {db_instance_arn} – The Timestream for InfluxDB instance ARN can be found on the Timestream for InfluxDB instance summary page
Use the following IAM policy as the role for the single-user Lambda function. The policy gives the Lambda function the required permissions to perform a secret rotation for Timestream for InfluxDB users.
Replace the following items in the IAM policy with values from your AWS account:
- {rotating_secret_arn} – The ARN for the secret being rotated can be found in the Secrets Manager secret details
- {db_instance_arn} – The Timestream for InfluxDB instance ARN can be found on the Timestream for InfluxDB instance summary page
- Choose Next.
- Enter a policy name and choose Create policy.
Configure invoke Lambda permissions for the Lambda function
Complete the following steps:
- Navigate back to the Lambda function Permissions
- Under Resource-based policy statements, choose Add permissions.
- Choose AWS service.
- Select Secrets Manager under Service.
- Enter a statement ID of your choosing.
- For Principal, select lambda:InvokeFunction under Action.
- Choose Save.
Configure Lambda function timeout
Complete the following steps:
- Navigate to the Lambda function General configuration
- Under General configuration, choose Edit.
- Set the Timeout to 1 minute.
- Choose Save.
This concludes the configuration for your InfluxDB secret rotation Lambda function.
Create a Secrets Manager secret
Complete the following steps to configure the Secrets Manager secret:
- On the Secrets Manager console, choose Secrets in the navigation pane.
- Choose Store a new secret.
- Select Other type of secret.
- Choose Plaintext under Key/value pairs.
- Enter one of the following options, depending on whether you’re storing token, user, or admin credentials in the secret.
When you create a Timestream for InfluxDB instance, an admin secret is automatically stored in Secrets Manager with credentials to be used with the multi-user Lambda function. Set the adminSecretArn to the authentication properties Secrets Manager ARN value found on the Timestream for InfluxDB instance summary page.
When you store Timestream for InfluxDB token credentials in the secret, use the following format:
When you store Timestream for InfluxDB user credentials in the secret, use the following format:
When you store Timestream for InfluxDB admin credentials in the secret, use the following format:
- Choose Next.
- Enter a name for the secret.
- Choose Next.
- Choose Next (we configure the rotation after we deploy the Lambda function).
- Choose Store.
Configure Secrets Manager secret rotation
Complete the following steps to configure your secrets rotation:
- On the Secrets Manager console, choose Secrets in the navigation pane.
- Choose the secret that you created.
- Choose Rotation and then Edit rotation.
- Choose Automatic rotation.
- Fill in the rotation schedule.
- Select Rotate immediately when the secret is stored.
- Under Rotation function, choose the Lambda function you previously deployed.
- Choose Save.
Verify the token
Complete the following steps to verify the token:
- On the Secrets Manager console, choose Secrets in the navigation pane.
- Choose the secret you created.
- In the Overview section, choose Retrieve secret value.
If everything went correctly, you should see a new token value in the secret if you are using the multi-user rotation Lambda function, or a new password if you are using the single-user rotation Lambda function.
If the value for the token or password has not changed, refer to the next section for troubleshooting using Amazon CloudWatch.
View CloudWatch logs
Complete the following steps to view the function’s CloudWatch logs:
- On the Lambda console, choose Functions in the navigation pane.
- Choose the function you created.
- Choose Monitor.
- Choose View CloudWatch logs.
- Choose the latest log stream under Log streams.
- Look for Error or Info logs that can help determine the reason for failure to rotate the secret.
Clean up
To avoid ongoing charges to your account, delete the resources you created in this walkthrough.
Delete the Lambda function and Lambda execution role
Complete the following steps to delete the Lambda function and the Lambda execution role:
- On the Lambda console, choose Functions in the navigation pane.
- Choose the function you created.
- Choose Configuration.
- Choose the role name for the Lambda execution role to open the role in a new tab.
- Choose Delete to delete the Lambda execution role.
- Enter the Lambda execution role name in the confirmation text box.
- Choose Delete.
- Return to the Lambda function tab, and on the Actions menu, choose Delete.
- Choose Delete.
Delete the Timestream for InfluxDB instance
Complete the following steps to delete your Timestream for InfluxDB instance:
- On the Timestream console, choose InfluxDB databases in the navigation pane.
- Choose the database you created.
- Choose Delete.
- Enter confirm in the confirmation text box and choose Delete.
Delete the Secrets Manager secret
Complete the following steps to delete your secret:
- On the Secrets Manager console, choose Secrets in the navigation pane.
- Choose the secret you created.
- On the Actions menu, choose Delete.
- Enter a number between 7–30 days for a waiting period and choose Schedule deletion.
Conclusion
In this post, we showed you how to use single- and multi-user secret rotation Lambda functions to rotate credentials for your Timestream for InfluxDB instance. You can use the secret rotation functions to rotate an authorization token or user password periodically to align with security best practices for credential rotation. With secret rotation Lambda functions, you can limit the threat vector of exposed database credentials to the window of time between the next secret rotation.
Check out How Amazon Timestream for InfluxDB uses secrets for more information on using secrets with Timestream for InfluxDB.
About the Author
Forest Vey is a Senior Software Developer at Improving. He is experienced working with time series data and serverless technologies. After 5 years of experience developing cloud infrastructure and embedded systems applications, he has gained proficiency in designing software solutions from application components to deployment architectures. When not immersed in learning about software development, he enjoys rock climbing and playing sports with his friends.
Source: Read More