Skip to main content

S3-Compatible Object Storage Configuration

Overview​

This guide provides comprehensive instructions for configuring any S3-compatible object storage (such as MinIO, Wasabi, Backblaze B2, DigitalOcean Spaces, Cloudflare R2, etc.) as your object storage backend for the Appcircle server.

By default, the Appcircle chart includes an in-cluster MinIO deployment provided by bitnami/minio. If you are installing the Appcircle for testing purposes, you may use the built-in MinIO deployment.

For production environments, it is recommended to configure an external MinIO or S3-compatible object storage instance. If you prefer to use an external MinIO service, the recommended MinIO version is 2024-03-15 or later, with a disk size of at least 100GB.

info

The recommended disk size for the object storage may vary depending on your usage requirements. It can range from 100GB to 3-4TB.

caution

You need to use version 0.3.20 or later of the Appcircle server Helm chart before applying this configuration. Please check the Appcircle server version and update the Helm chart if necessary.

What This Guide Covers​

This guide will walk you through the process of configuring an S3-compatible object storage backend for the Appcircle server Helm chart. To use an S3-compatible provider with Appcircle server, you need to:

  • Set up your object storage provider: Buckets, users, and permissions
  • CORS setup: For direct browser uploads/downloads
  • Configure Appcircle server: Update Helm values to use your S3-compatible storage
info

This guide is for any S3-compatible provider. For AWS S3 or GCP Cloud Storage, see their dedicated guides.

caution

If your Appcircle server is installed and contains data, changing or migrating to a different object storage provider is not supported. Object storage configuration must be finalized during the initial installation process.

Prerequisites​

To complete this guide, you must have the following:

  • An S3-compatible object storage provider (MinIO, Wasabi, Backblaze B2, DigitalOcean Spaces, Cloudflare R2, etc.)
  • Access to the provider's management console or CLI
  • Basic understanding of object storage, access keys, and Kubernetes
caution

Keep in mind that if you will use an AWS S3-compatible provider, the resource.s3.clientProvider setting used for the AWS option has limited support for the S3 endpoint styles.

The Appcircle server does not support the path-style S3 endpoint model. So, your S3 endpoints should be virtual-hosted style, which can be accessible using relevant DNS subdomains.

Refer here to see the differences between two styles and configure your S3 endpoints if necessary.

Configuration Steps​

1. Create and Configure Buckets​

Create the required buckets to store the artifacts generated by the Appcircle server.

Appcircle server requires the following buckets for different purposes:

  • ${BUCKET_PREFIX}temp: Temporary files and uploads (requires CORS configuration for direct uploads/downloads from the client browsers)
  • ${BUCKET_PREFIX}build: Build artifacts and logs
  • ${BUCKET_PREFIX}distribution: Testing Distribution files
  • ${BUCKET_PREFIX}storesubmit: Appcircle Store Submit files
  • ${BUCKET_PREFIX}store: Enterprise App Store files
  • ${BUCKET_PREFIX}agent-cache: Appcircle Runner cache files
  • ${BUCKET_PREFIX}backup: Backup files
  • ${BUCKET_PREFIX}publish: Published mobile app binaries
tip

Bucket Naming: Choose a unique bucket prefix for your organization or environment (e.g., appcircle-spacetech-).

Create the required buckets using your S3-compatible provider's management console, CLI, or API. Refer to your provider's documentation for specific instructions on bucket creation.

info

You can use your provider's web console, CLI tools, or API to create the buckets. The exact method varies by provider.

2. IAM/User Setup and Access Keys​

Create a user (or access key pair) with permissions to manage objects in the above buckets. The process varies by provider:

  • MinIO: Use the MinIO Console or mc admin user add to create a user and assign policies.
  • Wasabi: Use the Wasabi Console to create an access key/secret key pair with full access to the relevant buckets.
  • Backblaze B2: Use the B2 Console to create an application key with access to the required buckets.
  • DigitalOcean Spaces: Use the DigitalOcean Console to create a Spaces access key with appropriate permissions.
  • Cloudflare R2: Use the Cloudflare Dashboard to create an API token with R2 permissions.
caution

Restrict permissions to only the required buckets for better security.

3. CORS Configuration​

Configure CORS settings for the temp bucket to allow cross-origin requests from your Appcircle server dashboard.

  • Here is a sample CORS configuration for the temp bucket that is compatible with AWS CLI:
caution

Make sure to replace the https://my.appcircle.spacetech.com with the dashboard URL that you will use to access the Appcircle server. For example, if you are using .appcircle.spacetech.com as the domain in the Helm values.yaml file, the dashboard URL will be https://my.appcircle.spacetech.com.

export CORS='{
  "CORSRules": [
    {
      "AllowedHeaders": ["*"],
      "AllowedMethods": ["GET", "PUT", "POST", "DELETE", "HEAD"],
      "AllowedOrigins": ["https://my.appcircle.spacetech.com"],
      "ExposeHeaders": [],
      "MaxAgeSeconds": 3600
    }
  ]
}'

Refer to your provider's documentation for how to apply CORS settings.

tip
  • The CORS configuration is only required for the temp bucket.
  • Make sure that the clients are allowed to access the temp bucket over network. The clients (Appcircle users with the dashboard) use the temp bucket to upload/download files.
  • Other buckets don't require CORS configuration, as they are accessed server-side.
  • If you will use the Appcircle server dashboard with HTTP instead of HTTPS, replace https:// with http:// in the AllowedOrigins.
info

If the S3-compatible provider supports CORS configuration with the AWS CLI, you can use the following steps:

  • Configure the AWS CLI with your access key, secret key, region, and endpoint URL for the S3-compatible provider.
export AWS_ACCESS_KEY_ID=<YOUR_ACCESS_KEY>
export AWS_SECRET_ACCESS_KEY=<YOUR_SECRET_KEY>
export AWS_DEFAULT_REGION=<YOUR_REGION>
export AWS_ENDPOINT_URL=<YOUR_S3_ENDPOINT_URL>
  • Apply the CORS configuration to the temp bucket:
aws s3api put-bucket-cors \
    --endpoint-url $AWS_ENDPOINT_URL \
    --bucket ${BUCKET_PREFIX}temp \
    --cors-configuration "$CORS"
  • Check the CORS configuration:
aws s3api get-bucket-cors \
    --endpoint-url $AWS_ENDPOINT_URL \
    --bucket ${BUCKET_PREFIX}temp

4. Create Kubernetes/OpenShift Secret with Access Keys​

  • Create the namespace that Appcircle server will be installed in if you haven't yet:
kubectl create namespace appcircle
  • Create a Kubernetes secret named <helm-release-name>-minio-connection with your S3-compatible access and secret keys:
kubectl create secret generic appcircle-server-minio-connection \
  -n appcircle \
  --from-literal=accessKey=<YOUR_ACCESS_KEY> \
  --from-literal=secretKey=<YOUR_SECRET_KEY>
caution
  • Replace appcircle with your actual namespace or project if different.
  • Replace <YOUR_ACCESS_KEY> and <YOUR_SECRET_KEY> with your actual access and secret keys.
  • Replace appcircle-server-minio-connection with <helm-release-name>-minio-connection. Appcircle documentation uses appcircle-server as the release name.

5. Configure Appcircle server to Use S3-Compatible Storage​

Configure your values.yaml file to use your S3-compatible object storage.

Add or update the following configuration to your values.yaml file:

global:
  minio:
    url: "https://your-minio-endpoint.com" # e.g., https://minio.spacetech.com:9000
    region: "local" # MinIO uses "local" as the region
    useHttp: "false" # Set to "true" if the MinIO endpoint uses HTTP instead of HTTPS
    bucketPrefix: "appcircle-spacetech-" # Replace with your actual bucket prefix
resource:
  s3:
    clientProvider: "MINIO" # Use "MINIO" for MinIO
minio:
  enabled: false # Disable the internal MinIO server if you are using an external MinIO server
info
  • Set useHttp to true only if your S3 or MinIO endpoint does not support HTTPS (not recommended for production).
  • Check your provider's documentation for the correct endpoint URL and region.
  • Replace appcircle-spacetech- with your actual bucket prefix.
caution

Keep in mind that, in order to use HTTP (useHttp: "true") for the S3 endpoint, the Appcircle server should also be configured as HTTP.

Otherwise the browser might block HTTP requests because of the mixed content.

caution

In order to use HTTP (useHttp: "true") for the S3 endpoint, the Appcircle server version should be at least 3.28.2 or later.

Please check the Appcircle server version and upgrade the Helm chart if necessary. See the Helm chart version history for releases to choose an appropriate one.

Next Steps​

After completing the S3-compatible storage configuration:

  1. Return to the main installation guide:

  2. Continue with the installation process using your configured values.yaml file

  3. Verify the configuration by checking that the Appcircle server can access the buckets after installation.

Need help?​

Get help from Appcircle's support team, or see how others are using Appcircle by joining our Slack Channel.

Slack
No description available
join.slack.com
Preview of Slack