AWS S3

  • 12 minute(s) read
Prev Next

Overview

This guide will walk you through the following steps:

  • Configuring Amazon Web Services (AWS) S3 as your storage provider

  • Creating a Deployment to publish config to AWS S3

  • Deploying an orchestrator to consume the config in AWS

Prerequisites

  • An active AWS account with permissions to create and manage S3 buckets, IAM roles, and policies.

  • Permissions in your host environment install the orchestrator service

Configure storage provider

You will utilize AWS’s Simple Storage Service (S3) to publish configurations from the Maverics Console, which your orchestrators will access. The Maverics Console must have permissions to both read and write to this bucket, while the orchestrators need read-only access.

Create an AWS S3 bucket

Refer to the Creating a bucket AWS documentation for more information.

  1. Sign in to the AWS Management Console

    • Navigate to AWS Console

    • In the search bar, type S3 and select Amazon S3.

    • Select the AWS Region closest to where your Orchestrators are deployed.

  2. Create a New S3 Bucket

    • Click on "Create bucket".

    • Enter a Bucket name (must be globally unique, e.g., maverics-development).

  3. Configure Object Ownership

    • Choose ACLs disabled (recommended)

  4. Configure Public Access Settings

    • Under Block Public Access settings for this bucket, check Block all public access.

  5. Create the Bucket

    • Click Create bucket.

Create policies for your S3 bucket

Following best practices for least privilege, Strata recommends establishing two policies:

  1. One policy granting the Maverics Console read and write access to publish configurations to your bucket.

  2. A second policy providing read-only access for your orchestrators.

  1. Go to IAM in the AWS console

  2. Under Access Management, go to Policies and click Create Policy

  3. In Step 1: Specify permissions, select Policy editor: JSON.

  4. Copy the code block below and paste it into the editor, replacing the bucket's name fields with the bucket name you've created:

If you created or uploaded a folder to your S3 bucket for Maverics config files in the AWS Console, enter the path for that folder in the field. For example config/.

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "Policy",
      "Effect": "Allow",
      "Action": [
        "s3:PutObject",
        "s3:GetObject",
        "s3:ListBucket",
        "s3:DeleteObject"
      ],
      "Resource": [
        "arn:aws:s3:::bucket's name",
        "arn:aws:s3:::bucket's name/*"
      ]
    }
  ]
}

  1. Click Next

  2. Finish by entering Policy name other optional details. Click Create Policy

  1. Go to IAM in the AWS console

  2. Under Access Management, go to Policies

  3. In Step 1 Specify permissions, select JSON

  4. Copy the code block below and paste it into the editor, replacing the bucket's name fields with the bucket name you've created:

    If you created or uploaded a folder to your S3 bucket for Maverics config files in the AWS Console, enter the path for that folder in the field; for example config/.

    {
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "Policy",
      "Effect": "Allow",
      "Action": [
        "s3:GetObject",
        "s3:ListBucket"
      ],
      "Resource": [
        "arn:aws:s3:::bucket's name",
        "arn:aws:s3:::bucket's name/*"
      ]
    }
  ]
}

Give access to the bucket

Maverics Console uses an IAM role to publish config to the bucket. Orchestrators live in their own host environments, and use secure keys.

Refer to the create a separate IAM user AWS documentation for more information.

Create an IAM principal and an access key for your orchestrator host environment.

  1. Go to Identity and Access Management (IAM) within your AWS console.

  2. Under Access management, go to Users.

  3. Click Add users.

  4. Give the user a name and click Next.

  5. On the Set permissions page, select Attach policies directly, and search for the read-only orchestrator policy you created in the above section.

  6. Select the policy checkbox and click Next.

  7. Click Create user.

  8. After the user has been created, you are redirected to the users list. Click the name of the user you have just created.

  9. Click Security credentials.

  10. Scroll down to Access keys and click Create access key.

  11. Select Application running outside AWS and click Next.

  12. You can set a description tag or leave it blank, and click Create access key.

  13. On the Retrieve access keys page, copy the Access key and Secret access key, and keep them in a safe place. Alternatively, you can download the .csv file. These will be used in Orchestrator Remote Config steps.

  14. Click Done.

Refer to the IAM Roles AWS documentation for more information.

Create an IAM role with a policy for publishing config from the Maverics Console.

  1. Go to IAM in the AWS Console.

  2. Under Access Management, go to Roles. Click Create role.

  3. On Step 1, for Trusted entity type, select AWS account.

    Selecting trusted entity type for AWS account role

    Click to enlarge

  4. Under An AWS account, select Another AWS account, and enter the Account ID under the Role ARN field based on your Maverics region:

    Region

    Strata Account ID

    USA (maverics.strata.io)

    322849791940

    Canada (ca.maverics.strata.io)

    983197537105

    United Kingdom (uk.maverics.strata.io)

    339713018853

  5. Check the box marked Require external ID, and enter an alphanumeric string of your choice. Make note of this ID as you will use it in the next section for the External ID field in the Maverics Console.

  6. Click Next.

  7. On Step 2 Add permissions page, select the policy you created you created for your bucket. Click Next

  8. On Step 3 Name, review, create Create role.

  9. When you're redirected to the Roles page, click the role you've just created.

    Click to enlarge

    • On the Role page, the ARN can be found in the center column.

    • Copy the entire ARN string and keep it in a safe place. These will be used the next step for Configuration storage settings.

Maverics Console Configuration Storage

  1. Go to Deployments.

  2. Create a new or select an existing deployment.

  3. Under Host Environment section and next to Configuration Storage, click Edit.

  4. From the dropdown menu, select Amazon S3 bucket.

    Click to enlarge

  5. You will need to enter the following information:

Field

Description

Bucket Name

The unique name of the S3 bucket.

Role ARN

The Role’s Amazon Resource Number (ARN)

External ID

The External ID of the role.

Region

The AWS region of the bucket.

Optional:
Configuration File Path

Path within the S3 bucket to which your Maverics configuration file will be published. Do not include the file name in this path.

  1. Click Save

  2. To verify that Maverics can publish to your AWS S3 bucket:

    • Click Publish, and from the slide out window click Publish.

    • In the AWS Console, browse the contents of your S3 bucket to find the configuration bundle maverics.tar.gz

Orchestrator Remote Config

Refer to the Windows (Silent Install) article for details on system requirements, installation procedures, updates, and uninstallation.

If you are using the silent install process and the Windows machine will connect directly to a Maverics cloud environment, use the following PowerShell script as a template and update the configuration values as required:

# Set the configuration values
# Download MAVERICS_BUNDLE_PUBLIC_KEY_FILE your Maverics deployment
ownlo$RegistryPath = 'HKLM:\SYSTEM\CurrentControlSet\Services\mavericsSvc'
$Name         = 'Environment'
$Value        = @('MAVERICS_HTTP_ADDRESS=127.0.0.1:8888',
                'MAVERICS_POLLING_INTERVAL_SECONDS=30',
                'MAVERICS_RELOAD_CONFIG=true',
                'MAVERICS_AWS_CONFIG={  "bucketName": "aws-bucket-name",  "accessKeyID": "aws-access-key-id",  "secretAccessKey": "aws-secret-access-key",  "region": "aws-region",  "configurationFilePath": "folder1/folder2"}',
                'MAVERICS_BUNDLE_PUBLIC_KEY_FILE=C:\config\public_key.pem',
                'MAVERICS_CONFIGURATION_TYPE=1',
                'MAVERICS_REMOTE_CONFIG_TYPE=AWS')

# Create the relevant registry path if the Orchestrator MSI has not yet been installed
If (-NOT (Test-Path $RegistryPath)) {
  New-Item -Path $RegistryPath -Force | Out-Null
}

# Update the Environment
New-ItemProperty -Path $RegistryPath -Name $Name -Value $Value -Type MultiString -Force

The Orchestrator is compatible with Red Hat Linux and Ubuntu. Refer to the Linux install article for details on system requirements, installation procedures, updates, and uninstallation.

  1. Create a maverics.env file using a text editor in the directory where you downloaded and unzipped the orchestrator binary.

  2. Replace the bracketed placeholders with files and values needed for your environment. 

# MAVERICS_DEBUG_MODE: Enables or disables debug mode for Maverics.
# When set to true, additional debug information will be logged.
MAVERICS_DEBUG_MODE=true

# MAVERICS_HTTP_ADDRESS: Specifies the HTTP address and port for the Maverics server to listen on.
# Example: :443 or :8443
MAVERICS_HTTP_ADDRESS=:443

# MAVERICS_TLS_SERVER_CERT_FILE: Path or secret key name to the TLS server certificate file.
# This is used to enable HTTPS for the Maverics server.
MAVERICS_TLS_SERVER_CERT_FILE=your-cert.pem

# MAVERICS_TLS_SERVER_KEY_FILE: Path to the TLS server private key file.
# This is used in conjunction with the server certificate to enable HTTPS.
MAVERICS_TLS_SERVER_KEY_FILE=your-private_key.pem

# MAVERICS_RELOAD_CONFIG: This is required when deploying config from Maverics to your storage provider.
# When set to true, the configuration file will be reloaded automatically when changes are detected.
MAVERICS_RELOAD_CONFIG=true

# MAVERICS_POLLING_INTERVAL_SECONDS: Specifies the interval, in seconds, at which the configuration file is polled for changes.
MAVERICS_POLLING_INTERVAL_SECONDS=30

# MAVERICS_BUNDLE_PUBLIC_KEY_FILE: Path to the public key file used for verifying signed bundles.
# Download this public key file from your Maverics deployment.
MAVERICS_BUNDLE_PUBLIC_KEY_FILE=./public_key.pem

# MAVERICS_AWS_CONFIG: JSON string containing AWS configuration details.
# Includes region, bucket name, access key ID, secret access key, and (optional) configuration file path.
MAVERICS_AWS_CONFIG='{"region":"aws-region", "bucketName":"bucket name", "accessKeyID":"aws-access-key-id", "secretAccessKey":"aws-secret-access-key", "configurationFilePath": "s3-file-path"}'

Refer to the Docker install article for details on system requirements, installation procedures, updates, and uninstallation.

  1. Create a maverics.env using a text editor and save it to a working directory.

  2. Replace the bracketed placeholders with files and values needed for your environment. 

# MAVERICS_DEBUG_MODE: Enables or disables debug mode for Maverics.
# When set to true, additional debug information will be logged.
export MAVERICS_DEBUG_MODE=true

# MAVERICS_HTTP_ADDRESS: Specifies the HTTP address and port for the Maverics server to listen on.
# Example: :443 or :8443
export MAVERICS_HTTP_ADDRESS=:443

# MAVERICS_TLS_SERVER_CERT_FILE: Path or secret key name to the TLS server certificate file.
# This is used to enable HTTPS for the Maverics server.
export MAVERICS_TLS_SERVER_CERT_FILE=your_cert.pem

# MAVERICS_TLS_SERVER_KEY_FILE: Path to the TLS server private key file.
# This is used in conjunction with the server certificate to enable HTTPS.

export MAVERICS_TLS_SERVER_KEY_FILE=your_private_key.pem

# MAVERICS_RELOAD_CONFIG: This is required when deploying config from Maverics to your storage provider.
# When set to true, the configuration file will be reloaded automatically when changes are detected.
export MAVERICS_RELOAD_CONFIG=true

# MAVERICS_POLLING_INTERVAL_SECONDS: Specifies the interval, in seconds, at which the configuration file is polled for changes.
export MAVERICS_POLLING_INTERVAL_SECONDS=30

# MAVERICS_BUNDLE_PUBLIC_KEY_FILE: Path to the public key file used for verifying signed bundles.
# Download this public key file from your Maverics deployment.
export MAVERICS_BUNDLE_PUBLIC_KEY_FILE=./public_key.pem

# MAVERICS_AWS_CONFIG: JSON string containing AWS configuration details.
# Includes region, bucket name, access key ID, secret access key, and (optional) configuration file path.
MAVERICS_AWS_CONFIG='{"region":"aws-region", "bucketName":"bucket name", "accessKeyID":"aws-access-key-id", "secretAccessKey":"aws-secret-access-key", "configurationFilePath": "s3-file-path"}'

The Mac OS Orchestrator should only be used for development purposes only.

Refer to the Mac OS install article for details on system requirements, installation procedures, updates, and uninstallation.

  1. Create a maverics.env using a text editor and save it to a working directory.

  2. Replace the bracketed placeholders with files and values needed for your environment. 

# MAVERICS_DEBUG_MODE: Enables or disables debug mode for Maverics.
# When set to true, additional debug information will be logged.
export MAVERICS_DEBUG_MODE=true

# MAVERICS_HTTP_ADDRESS: Specifies the HTTP address and port for the Maverics server to listen on.
# Example: :443 or :8443
export MAVERICS_HTTP_ADDRESS=:443

# MAVERICS_TLS_SERVER_CERT_FILE: Path or secret key name to the TLS server certificate file.
# This is used to enable HTTPS for the Maverics server.
export MAVERICS_TLS_SERVER_CERT_FILE=your-cert.pem

# MAVERICS_TLS_SERVER_KEY_FILE: Path to the TLS server private key file.
# This is used in conjunction with the server certificate to enable HTTPS.
export MAVERICS_TLS_SERVER_KEY_FILE=your-private_key.pem

# MAVERICS_RELOAD_CONFIG: This is required when deploying config from Maverics to your storage provider.
# When set to true, the configuration file will be reloaded automatically when changes are detected.
export MAVERICS_RELOAD_CONFIG=true

# MAVERICS_POLLING_INTERVAL_SECONDS: Specifies the interval, in seconds, at which the configuration file is polled for changes.
export MAVERICS_POLLING_INTERVAL_SECONDS=30

# MAVERICS_BUNDLE_PUBLIC_KEY_FILE: Path to the public key file used for verifying signed bundles.
# Download this public key file from your Maverics deployment.
export MAVERICS_BUNDLE_PUBLIC_KEY_FILE=./public_key.pem

# MAVERICS_AWS_CONFIG: JSON string containing AWS configuration details.
# Includes region, bucket name, access key ID, secret access key, and (optional) configuration file path.
export MAVERICS_AWS_CONFIG='{"region":"aws-region", "bucketName":"aws-s3-bucket-name", "accessKeyID":"aws-access-key-id", "secretAccessKey":"aws-secret-access-key", "configurationFilePath": "s3-file-path"}'

Refer to the Windows MSI Installer article for details on system requirements, installation procedures, updates, and uninstallation.

The Windows MSI installer prompts you for the environment details as part of the installation process:

  1. After starting the MSI installer, click next until you get to the Select a configuration source step.

  2. Accept the default Maverics Cloud, and then click Next.

  3. Select Amazon S3.

  4. Edit the JSON replacing the placeholder values with the actual values from previous steps. 

{
  "bucketName": "bucket's name",
  "accessKeyID": "aws-access-key-id",
  "secretAccessKey": "aws-secret-access-key",
  "region": "aws-region",
  "configurationFilePath": "folder1/folder2"
}

  1. When you are done click Next to proceed to the Bundle public key file selection.

    Configuration Bundle Signing

    Configuration bundles published by the Maverics platform are signed to ensure data integrity. The orchestrator cannot load a published bundle without validating the configuration file signatures against the public key for the specific configuration deployment.

  2. In Maverics Console go to Deployments and open the deployment you created before, scroll to the Host Environment section and download the public key.

  3. In the MSI installer, click Change… navigate to the downloaded public key and click open. Click Next

  4. In the next screen choose the IP address, port and TLS settings for the orchestrator. Click Next and Install. If there are no install errors the orchestrator is receiving configuration from AWS.

  5. To verify you can view Maverics logs in the Event Viewer

    • Start→Search for Event Viewer

    • Go to Windows logs → Application

    • Maverics Event ...level=info msg="loaded config 'maverics.tar.gz' from s3"

Start orchestrator on the host

The Windows installation configures the orchestrator service to auto-start after all other services have started.

  1. In Terminal navigate to the directory where you downloaded and unzipped the orchestrator binary.

  2. Run the following command replacing with paths to the orchestrator binary.
    sudo systemctl start maverics

To start the Orchestrator container, use the docker run command. For example:

docker run --publish 443:8443 \
  --volume /opt/maverics:/etc/maverics \
  --env-file /opt/maverics/maverics.env \
  --name orchestrator maverics_base:0.18.10

The example command above illustrates the following options:

  • --publish (or -p) maps port 8443 on the container to port 443 on the host

  • --volume (or -v) bind mounts the hosts /opt/maverics/ directory to /etc/maverics in the container

  • --env-file sources environment variables from a maverics.env file for the orchestrator process

  • --name (optional) provides an explicit name for the container instance

  • maverics_base:0.18.10 specifies the image name and release number of the container

Paths, port addresses, names, versions, and other settings will vary depending on your use case. Information on all options can be found in the docker run reference documentation.

  1. In Terminal navigate to the directory where you downloaded and unzipped the orchestrator binary.

  2. Run the following command replacing with paths to the orchestrator binary.

source ./maverics.env  && ./maverics_darwin_ARM64

Troubleshooting

  • What do I do if orchestrator fails to start with an error message “…level=error msg="failed to instantiate config provider: unable to read file 'public_key.pem' specified in 'MAVERICS_BUNDLE_PUBLIC_KEY_FILE': open public_key.pem: no such file or directory" ?

    • Configuration bundles published by the Maverics platform are signed to ensure data integrity. The orchestrator cannot load the public key associated with your deployment. In the host environment configuration check that the file path is correct.

  • What do I do if the orchestrator fails to start with the error message "level=error msg="failed to load configuration: unable to unbundle 'maverics.tar.gz' config from <storage provider>: bundle signature verification failed: signature verification failed: go-jose/go-jose: error in cryptographic primitive" ?

    • In this case, the signature verification failed. The configuration bundle signature did not match with the public key used to start the orchestrator. Get the Deployment, scroll down to the Host Environment section, download the Public Key, and update the path in your host environment to the new file.

Related articles