ceph-logo

Object Storage @ eodc

eodc provides S3-compatible object storage backed by Ceph. Access is organised through Ceph IAM accounts: each customer can get a root account and administer its users, groups, and buckets themselves. No ticket to eodc is needed for day-to-day access management.

Endpoint: https://objects.eodc.eu

Getting started

To request an account, contact support@eodc.eu. Briefly describe your use case and the capacity you expect, and our accounting will follow up with an offer tailored to your project.

Please include two independent ways to reach the person who will hold the root user (for example e-mail and phone). We use two separate channels to hand over the initial access credentials securely.

You will then receive an IAM account, an account ID, and credentials for a root user that administers it.

This page is a working manual for account administrators. Commands are copy-paste ready, with notes on the pitfalls worth knowing.

All examples use the AWS CLI (backed by Boto3). Anything shown here works the same through any other S3 client or library — boto3, s3cmd, rclone, and so on.

All names in the examples are samples: a fictional copernicus account with the org prefix cop, a sentinel2-mosaic project, and users named after satellites (cop-aeolus, cop-cryosat). Substitute your own account, prefix, and identities throughout.

Prerequisites

  • The AWS CLI installed.

  • A profile configured for the root user that administers your account. This is the identity the administration commands run as; it is for provisioning only, never day-to-day data-plane work.

# Set up the admin profile once. Use the root user's real keys.
ADMIN=iam-copernicus-root-mission-control
ORG_PREFIX=cop          # org label baked into every group name

aws configure set aws_access_key_id     '<ACCESS_KEY>'           --profile "$ADMIN"
aws configure set aws_secret_access_key '<SECRET_KEY>'           --profile "$ADMIN"
aws configure set region                default                  --profile "$ADMIN"
aws configure set endpoint_url          https://objects.eodc.eu  --profile "$ADMIN"
aws configure set s3.addressing_style   path                     --profile "$ADMIN"
aws configure set output                json                     --profile "$ADMIN"

s3.addressing_style path is required: the gateway serves every bucket from the single hostname above, so the client must not fall back to the virtual-hosted bucket.objects.eodc.eu form. A working (data-plane) user configures a profile the same way, with their own keys.

Every administration example below assumes $ADMIN and $ORG_PREFIX are set in your shell.

Basic usage: push and pull

Data lives in buckets and moves with the ordinary aws s3 commands. These run as a working user with its own profile — configured exactly like the admin profile above, with that user’s access key — not as the root account.

USER_PROFILE=cop-aeolus          # a working user's profile, set up as in Prerequisites
BUCKET=cop-sentinel2-mosaic

# list what's in a bucket
aws --profile "$USER_PROFILE" s3 ls s3://$BUCKET/

# push: one file, then a whole directory
aws --profile "$USER_PROFILE" s3 cp ./mosaic.tif s3://$BUCKET/
aws --profile "$USER_PROFILE" s3 cp ./results/ s3://$BUCKET/results/ --recursive

# pull: one object, then sync a prefix down to a local folder
aws --profile "$USER_PROFILE" s3 cp s3://$BUCKET/mosaic.tif ./mosaic.tif
aws --profile "$USER_PROFILE" s3 sync s3://$BUCKET/results/ ./results/

aws s3 ls with no bucket name returns nothing for a project-scoped user: the project policies grant access to their bucket, not ListAllMyBuckets. List inside the bucket you were granted, as above.

How access works

There is no direct “attach a user to a bucket” operation; access is layered:

account (copernicus)
  └─ root user       -> administers the account (users, groups, policies)
      └─ group       -> one per project and tier; carries a policy
          ├─ policy  -> which bucket(s) the group may touch + which actions
          └─ members -> users in the group inherit the policy

You want to…

You actually…

Give a project a place to store

Create a bucket

Define who-can-do-what

Create a group with a policy scoped to that bucket

Give a person access

Add the user to the group

Give a person full S3 access

Attach the managed AmazonS3FullAccess policy

A user can be in several groups at once. Someone working on multiple projects is simply a member of each project group.

Two kinds of policy

  • Managed policies are attached by ARN with attach-user-policy / attach-group-policy (e.g. arn:aws:iam::aws:policy/AmazonS3FullAccess). List them with list-attached-*-policies.

  • Inline policies are custom JSON attached with put-user-policy / put-group-policy (your project bucket policies). List them with list-*-policies, then read them with get-*-policy.

put-group-policy rejects a --policy-name containing an underscore (PolicyName contains invalid characters), so every name in this guide uses hyphens. The group name then doubles as its own policy name (--policy-name $GROUP, no rewriting). See S3 limitations.

Conventions and scope

Nothing in this section is enforced by the platform. It is a sample scheme that has worked well in practice. Pick one scheme, adapt it to your organisation, and use it consistently; that keeps every project in the account uniform and access easy to reason about later.

Scope one account to one organisation. An IAM account belongs to a single organisation, and projects live inside it as a bucket plus its groups. Do not create one account per project: account membership is permanent and users cannot cross accounts, so a per-project account would trap every user inside it. The only per-org knobs are the admin profile, the ORG_PREFIX, and the org label. Another organisation stands up its own account by copying this tooling with its own values; there is no cross-tenant reach between accounts.

The prefix on a name says what kind of identity it is:

Prefix

Who

Notes

cop-<id>

A person

Created once, reused across every project.

svc-<project>

A service or machine login

Project-bound, its own key.

ext-<project>-<who>

An external partner

Long-lived scoped key, easy to revoke.

The group a user sits in says what they can do:

Tier

Access

rw

Read and write.

ro

Read only.

wo

Drop-box: put only, no read and no list.

Type and tier are independent: any identity can sit in any tier. Suggested rule: externals go in ro or wo, never rw.

A group is named <org_prefix>-<project>-<tier>, for example cop-sentinel2-mosaic-rw, and carries its access as a bucket-scoped inline policy. Re-run the tooling per project rather than looking for a shared policy object; there are no customer-managed policies on this Ceph release.

Every key is long-lived, so scope service and external keys tightly and rotate or revoke them by hand. Each identity is keyed once and a reused identity is never re-keyed. The root account is for management and provisioning only, never day-to-day data-plane work.

Bucket names must be lowercase and DNS-compliant: letters, digits, hyphens, no underscores or uppercase. That is an S3 rule. The Swift API on the same gateway accepts mixed-case and underscored container names, but S3 does not, so a Swift-created name like that can’t be reached as an S3 bucket. This guide uses lowercase hyphens for every name — persons, groups, policies, buckets — so one rule covers them all. A non-compliant name is flagged and must be confirmed with an explicit phrase before creation; use this guard instead of a bare s3 mb:

# Lowercase DNS-compliant bucket names only. A non-compliant name must be
# confirmed by typing the exact override phrase.
make_bucket() {
  local name="$1"
  if [[ "$name" =~ ^[a-z0-9]([a-z0-9.-]{1,61}[a-z0-9])?$ && "$name" != *..* ]]; then
    aws --profile "$ADMIN" s3 mb "s3://$name"
  else
    echo "WARNING: '$name' is not a lowercase DNS-compliant bucket name."
    echo "To override, type exactly:  yes, create $name"
    read -r reply
    [[ "$reply" == "yes, create $name" ]] || { echo "Aborted."; return 1; }
    aws --profile "$ADMIN" s3 mb "s3://$name"
  fi
}

Set up a project

Group names follow <org_prefix>-<project>-<tier> and bucket names are lowercase DNS-compliant, per the Conventions section.

Create one project

PROJECT=sentinel2-mosaic
GROUP=${ORG_PREFIX}-${PROJECT}-rw
BUCKET=${ORG_PREFIX}-${PROJECT}

# bucket (owned by the account); make_bucket enforces the naming rule
make_bucket $BUCKET

# group
aws --profile "$ADMIN" iam create-group --group-name $GROUP

# read-write policy scoped to just this bucket
cat > /tmp/$GROUP-policy.json <<EOF
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "s3:GetObject", "s3:PutObject", "s3:DeleteObject",
        "s3:ListBucket", "s3:GetBucketLocation",
        "s3:ListBucketMultipartUploads", "s3:AbortMultipartUpload"
      ],
      "Resource": [
        "arn:aws:s3:::$BUCKET",
        "arn:aws:s3:::$BUCKET/*"
      ]
    }
  ]
}
EOF

aws --profile "$ADMIN" iam put-group-policy \
  --group-name $GROUP \
  --policy-name $GROUP \
  --policy-document file:///tmp/$GROUP-policy.json

The actions are full read/write, but the Resource block pins them to this one bucket, so the grant is not account-wide. IAM is default-deny, so a member of this group can act only on $BUCKET; every action on any other bucket is denied. ListAllMyBuckets is not granted either, so aws s3 ls with no bucket returns nothing for these users; they list their own bucket with aws s3 ls s3://$BUCKET. To grant more than one bucket, add its ARN pair to the Resource array (see Multiple buckets per project).

Add users to a project

People are created once with the <org_prefix>-<id> prefix and reused across projects. A user who already exists is added to the new group without re-keying.

for U in cop-aeolus cop-cryosat cop-swarm; do
  aws --profile "$ADMIN" iam create-user --user-name $U 2>/dev/null || true
  aws --profile "$ADMIN" iam create-access-key --user-name $U   # capture output: shown once!
  aws --profile "$ADMIN" iam add-user-to-group --group-name $GROUP --user-name $U
done

create-access-key prints the secret only once, so capture the output right away. That credential pair is what the user needs to reach their bucket. There is no way to recover it later; if it is lost you have to issue a new key.

If the user already exists and has a key, skip create-access-key — reused identities are never re-keyed. Run it only the first time you create the person.

iam create-user --user-name $U works as long as the resulting RGW display-name is also $U, which RGW normally sets for you. Bucket policies match principals by display-name, so any user whose display-name differs from $U cannot be named in a policy. See the sharing rule in “Sharing a bucket beyond its owning group”.

Read-only tier

For members who should only read, make the ro group for the project and drop the write actions:

PROJECT=sentinel2-mosaic
GROUP=${ORG_PREFIX}-${PROJECT}-ro
BUCKET=${ORG_PREFIX}-${PROJECT}

aws --profile "$ADMIN" iam create-group --group-name $GROUP

cat > /tmp/$GROUP-policy.json <<EOF
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "s3:GetObject", "s3:ListBucket",
        "s3:ListBucketMultipartUploads", "s3:GetBucketLocation"
      ],
      "Resource": [
        "arn:aws:s3:::$BUCKET",
        "arn:aws:s3:::$BUCKET/*"
      ]
    }
  ]
}
EOF

aws --profile "$ADMIN" iam put-group-policy \
  --group-name $GROUP --policy-name $GROUP \
  --policy-document file:///tmp/$GROUP-policy.json

Put each person in whichever group fits.

Write-only tier (wo, drop-box)

For an identity that should be able to drop files but not read or list the bucket, make the wo group. This is the tier to use for an external partner who only needs to deliver data. Scope the writes to a drop prefix rather than the whole bucket, so a partner can’t overwrite or scatter objects elsewhere in it. The object actions are pinned to $DROP/* and GetObject / ListBucket are left out entirely:

PROJECT=sentinel2-mosaic
GROUP=${ORG_PREFIX}-${PROJECT}-wo
BUCKET=${ORG_PREFIX}-${PROJECT}
DROP=incoming            # drop prefix; use incoming/<partner> to isolate each one

aws --profile "$ADMIN" iam create-group --group-name $GROUP

cat > /tmp/$GROUP-policy.json <<EOF
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "s3:PutObject",
        "s3:AbortMultipartUpload", "s3:ListMultipartUploadParts"
      ],
      "Resource": "arn:aws:s3:::$BUCKET/$DROP/*"
    },
    {
      "Effect": "Allow",
      "Action": "s3:GetBucketLocation",
      "Resource": "arn:aws:s3:::$BUCKET"
    }
  ]
}
EOF

aws --profile "$ADMIN" iam put-group-policy \
  --group-name $GROUP --policy-name $GROUP \
  --policy-document file:///tmp/$GROUP-policy.json

The partner uploads to s3://$BUCKET/$DROP/...; a PutObject anywhere else in the bucket is denied. GetBucketLocation stays bucket-scoped because it has no object form. If you ever need the drop-box to accept objects at the bucket root, widen the object resource back to arn:aws:s3:::$BUCKET/*, but that gives up the containment and is not recommended for externals.

Multiple buckets per project

Add more ARN pairs to the policy’s Resource array:

"Resource": [
  "arn:aws:s3:::cop-sentinel2-mosaic",  "arn:aws:s3:::cop-sentinel2-mosaic/*",
  "arn:aws:s3:::cop-shared",            "arn:aws:s3:::cop-shared/*"
]

Sample: create several projects at once

for PROJECT in sentinel2-mosaic; do   # add more project slugs to this list
  GROUP=${ORG_PREFIX}-${PROJECT}-rw
  BUCKET=${ORG_PREFIX}-${PROJECT}

  make_bucket $BUCKET
  aws --profile "$ADMIN" iam create-group --group-name $GROUP 2>/dev/null || true

  cat > /tmp/$GROUP-policy.json <<EOF
{ "Version":"2012-10-17","Statement":[{"Effect":"Allow",
  "Action":["s3:GetObject","s3:PutObject","s3:DeleteObject","s3:ListBucket","s3:GetBucketLocation","s3:ListBucketMultipartUploads","s3:AbortMultipartUpload"],
  "Resource":["arn:aws:s3:::$BUCKET","arn:aws:s3:::$BUCKET/*"]}]}
EOF

  aws --profile "$ADMIN" iam put-group-policy \
    --group-name $GROUP --policy-name $GROUP \
    --policy-document file:///tmp/$GROUP-policy.json
done

Sample: a full project setup (Sentinel-2 mosaic)

A complete sample setup, end to end. One bucket, a read-write group and a read-only group, and four people. Three get read-write, one gets read-only. Everyone here is a person, so they use the cop-<id> prefix and are reused across any other project they join.

Piece

Name

Bucket

cop-sentinel2-mosaic

Read-write group

cop-sentinel2-mosaic-rw

Read-only group

cop-sentinel2-mosaic-ro

Read-write users

cop-aeolus, cop-cryosat, cop-swarm

Read-only user

cop-envisat

The bucket name is lowercase and DNS-compliant, so make_bucket creates it without prompting. The group names follow <org_prefix>-<project>-<tier> off the lowercased project.

1. Bucket and both groups.

BUCKET=cop-sentinel2-mosaic
RWGROUP=cop-sentinel2-mosaic-rw
ROGROUP=cop-sentinel2-mosaic-ro

make_bucket "$BUCKET"
aws --profile "$ADMIN" iam create-group --group-name $RWGROUP
aws --profile "$ADMIN" iam create-group --group-name $ROGROUP

2. Read-write policy on the rw group.

cat > /tmp/$RWGROUP-policy.json <<EOF
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "s3:GetObject", "s3:PutObject", "s3:DeleteObject",
        "s3:ListBucket", "s3:GetBucketLocation",
        "s3:ListBucketMultipartUploads", "s3:AbortMultipartUpload"
      ],
      "Resource": [
        "arn:aws:s3:::$BUCKET",
        "arn:aws:s3:::$BUCKET/*"
      ]
    }
  ]
}
EOF

aws --profile "$ADMIN" iam put-group-policy \
  --group-name $RWGROUP --policy-name $RWGROUP \
  --policy-document file:///tmp/$RWGROUP-policy.json

3. Read-only policy on the ro group. Same bucket, read actions only.

cat > /tmp/$ROGROUP-policy.json <<EOF
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "s3:GetObject", "s3:ListBucket",
        "s3:ListBucketMultipartUploads", "s3:GetBucketLocation"
      ],
      "Resource": [
        "arn:aws:s3:::$BUCKET",
        "arn:aws:s3:::$BUCKET/*"
      ]
    }
  ]
}
EOF

aws --profile "$ADMIN" iam put-group-policy \
  --group-name $ROGROUP --policy-name $ROGROUP \
  --policy-document file:///tmp/$ROGROUP-policy.json

4. Users into their groups. Capture each create-access-key block before moving on (the secret prints once). If any of these people already exist from another project, skip their create-access-key and just add them to the group.

# read-write members
for U in cop-aeolus cop-cryosat cop-swarm; do
  aws --profile "$ADMIN" iam create-user       --user-name $U 2>/dev/null || true
  aws --profile "$ADMIN" iam create-access-key --user-name $U   # secret shown ONCE
  aws --profile "$ADMIN" iam add-user-to-group --group-name $RWGROUP --user-name $U
done

# read-only member
aws --profile "$ADMIN" iam create-user       --user-name cop-envisat 2>/dev/null || true
aws --profile "$ADMIN" iam create-access-key --user-name cop-envisat   # secret shown ONCE
aws --profile "$ADMIN" iam add-user-to-group --group-name $ROGROUP --user-name cop-envisat

5. Verify.

aws --profile "$ADMIN" iam get-group --group-name $RWGROUP --query 'Users[].UserName' --output text
aws --profile "$ADMIN" iam get-group --group-name $ROGROUP --query 'Users[].UserName' --output text

The first should list the three read-write users, the second just cop-envisat. If someone later needs to move from read-only to read-write, remove them from the ro group and add them to the rw group. Do not stack both.

Inspect and audit

List all users

aws --profile "$ADMIN" iam list-users --query 'Users[].UserName' --output text

List all groups

aws --profile "$ADMIN" iam list-groups --query 'Groups[].GroupName' --output text

Members of a group / a user’s groups

aws --profile "$ADMIN" iam get-group --group-name cop-sentinel2-mosaic-rw
aws --profile "$ADMIN" iam list-groups-for-user --user-name cop-aeolus

List all policies on every user and group

This discovers the names itself — nothing is hard-coded — so it works the same on a brand-new account or a fully-populated one.

echo "######## USERS ########"
for U in $(aws --profile "$ADMIN" iam list-users --query 'Users[].UserName' --output text); do
  echo "=== user: $U ==="
  echo "  managed:"
  aws --profile "$ADMIN" iam list-attached-user-policies \
    --user-name "$U" --query 'AttachedPolicies[].PolicyArn' --output text
  echo "  inline:"
  aws --profile "$ADMIN" iam list-user-policies \
    --user-name "$U" --query 'PolicyNames[]' --output text
done

echo "######## GROUPS ########"
for G in $(aws --profile "$ADMIN" iam list-groups --query 'Groups[].GroupName' --output text); do
  echo "=== group: $G ==="
  echo "  managed:"
  aws --profile "$ADMIN" iam list-attached-group-policies \
    --group-name "$G" --query 'AttachedPolicies[].PolicyArn' --output text
  echo "  inline:"
  aws --profile "$ADMIN" iam list-group-policies \
    --group-name "$G" --query 'PolicyNames[]' --output text
done

If both managed and inline come back empty for a user, that user has no permissions yet. That is the normal default state right after create-user.

Read the JSON of an inline policy

# The policy name is just the group name (see S3 limitations).
aws --profile "$ADMIN" iam get-group-policy \
  --group-name cop-sentinel2-mosaic-rw --policy-name cop-sentinel2-mosaic-rw \
  --query 'PolicyDocument' --output json | jq .

List buckets and bucket policies

All buckets:

aws --profile "$ADMIN" s3api list-buckets --query 'Buckets[].Name' --output text

Bucket policy for each bucket. Note that get-bucket-policy throws a NoSuchBucketPolicy error when a bucket has none, so we swallow that and print a clean line instead:

for B in $(aws --profile "$ADMIN" s3api list-buckets --query 'Buckets[].Name' --output text); do
  echo "=== $B ==="
  aws --profile "$ADMIN" s3api get-bucket-policy --bucket "$B" \
    --query Policy --output text 2>/dev/null | jq . 2>/dev/null \
    || echo "  (no bucket policy)"
done

In an account run on the group model, most (probably all) buckets come back with (no bucket policy): access is driven by IAM group policies, not by anything attached to the buckets. When someone asks who can reach a bucket, the answer is in the group inline policies above, not here.

Grant a single user full S3 access

When someone needs unrestricted access (not scoped to one project’s buckets), attach the managed policy directly:

aws --profile "$ADMIN" iam attach-user-policy \
  --user-name cop-mission-control \
  --policy-arn arn:aws:iam::aws:policy/AmazonS3FullAccess

Verify. The output should now list the policy instead of an empty array:

aws --profile "$ADMIN" iam list-attached-user-policies --user-name cop-mission-control

Note on root vs working users. iam-<account>-root-<name> users are --account-root identities that administer the account, and they are for provisioning only. A plain user like cop-mission-control is a normal working identity. Give people full S3 access on the working user, not the root identity.

Delegating administration with managed policies

attach-*-policy is not limited to AmazonS3FullAccess. Ceph ships these managed policies, usable on users or groups:

ARN

Grants

arn:aws:iam::aws:policy/AmazonS3FullAccess

All S3 actions on all account buckets

arn:aws:iam::aws:policy/AmazonS3ReadOnlyAccess

Read-only S3 across the account

arn:aws:iam::aws:policy/IAMFullAccess

Full IAM management within the account

arn:aws:iam::aws:policy/IAMReadOnlyAccess

Read-only IAM (inspect users/policies)

arn:aws:iam::aws:policy/AmazonSNSFullAccess

Manage notification topics

arn:aws:iam::aws:policy/AmazonSNSReadOnlyAccess

Read-only notification topics

Sample: a sub-admin for a team

A sub-admin lets a team or department manage its own users and groups:

aws --profile "$ADMIN" iam create-user --user-name cop-team-admin
aws --profile "$ADMIN" iam create-access-key --user-name cop-team-admin
aws --profile "$ADMIN" iam attach-user-policy \
  --user-name cop-team-admin \
  --policy-arn arn:aws:iam::aws:policy/IAMFullAccess

That sub-admin can now run the same iam create-user / create-group / put-group-policy commands within the account. Cluster-level operations (radosgw-admin, quotas across accounts) stay with eodc.

Sample: read-only access for auditors

aws --profile "$ADMIN" iam attach-user-policy \
  --user-name cop-auditor \
  --policy-arn arn:aws:iam::aws:policy/IAMReadOnlyAccess

They can run all the commands in “Inspect and audit” but change nothing.

Revoking a managed policy

Attaching has a matching detach. This is the clean way to remove access:

aws --profile "$ADMIN" iam detach-user-policy \
  --user-name cop-mission-control \
  --policy-arn arn:aws:iam::aws:policy/AmazonS3FullAccess

(For inline policies, the equivalent is delete-user-policy / delete-group-policy by --policy-name.)

Resource-based bucket policies

Everything so far has used identity policies: JSON attached to a user or group saying “this identity may touch these buckets”. Ceph also supports bucket policies: JSON attached to the bucket itself, saying “these principals may do these things to me”. They are managed with S3 calls (s3api put-bucket-policy), not radosgw-admin or iam.

You do not need bucket policies for normal project access. The group model covers that. Use them for things identity policies cannot do cleanly: security hardening, conditional access, and sharing a bucket with a principal outside the owning group.

Read / remove a bucket policy

aws --profile "$ADMIN" s3api get-bucket-policy --bucket "$BUCKET" --query Policy --output text | jq .
aws --profile "$ADMIN" s3api delete-bucket-policy --bucket "$BUCKET"

Limitations to know. Ceph’s bucket policies are a subset of AWS. There is no string interpolation, and only a limited set of condition keys is supported: aws:CurrentTime, aws:EpochTime, aws:PrincipalType, aws:Referer, aws:SourceIp, aws:SecureTransport. Keep bucket policies simple and put the fine-grained per-project logic in the group policies.

Sharing a bucket beyond its owning group

Sometimes a bucket needs to be readable by someone outside its project group, for example a shared reference dataset or cross-team collaboration. The tool for this is a bucket policy that names the principal directly. The account ID (RGW...) is part of your account details from eodc.

BUCKET=cop-shared-dem
cat > /tmp/$BUCKET-share.json <<EOF
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "AllowSharedDemRead",
      "Effect": "Allow",
      "Principal": { "AWS": "arn:aws:iam::RGW00000000000000001:user/cop-envisat" },
      "Action": ["s3:GetObject", "s3:ListBucket"],
      "Resource": [
        "arn:aws:s3:::$BUCKET",
        "arn:aws:s3:::$BUCKET/*"
      ]
    }
  ]
}
EOF

aws --profile "$ADMIN" s3api put-bucket-policy \
  --bucket "$BUCKET" --policy file:///tmp/$BUCKET-share.json

Within an account, the principal is arn:aws:iam::<ACCOUNT_ID>:user/<name>, where <name> is the user’s --display-name. For a genuinely separate account, name that account’s ID as the principal. Note that granting an entire account access grants every user in it, so prefer naming specific users.

Name the principal by the exact user-name. iam create-user --user-name cop-aeolus sets the display-name equal to the login (cop-aeolus), so the login is the principal name. Create users through the IAM API and write the Principal as arn:aws:iam::<ACCOUNT_ID>:user/cop-aeolus, that same string. A principal that does not exactly match an existing user’s name silently never matches — access is denied and nothing in the policy flags it.

Avoid ACLs for sharing. On a shared cluster, legacy S3 ACL grants like authenticated-read can mean “any identity that can authenticate to this endpoint”, not “just my team”. That is a real cross-tenant exposure risk. Use bucket policies instead of ACLs.

Reference

S3 limitations

These are limits of the S3 interface (and its IAM and bucket-policy surface), not of this specific cluster; they hold on any Ceph RGW of this release. The commands in this guide work as written except for the following:

Limitation

Effect

What to do

S3 naming is strict: lowercase, hyphens, no underscores

Bucket names must be lowercase and DNS-compliant, and RGW applies the same rule to IAM policy names: an underscore in --policy-name fails with PolicyName contains invalid characters. Group and user names do accept underscores, but mixing separators is what forces a workaround.

Use lowercase hyphenated names for everything. The group name then works as the policy name (--policy-name $GROUP).

Swift allows mixed case, S3 doesn’t

The same RGW serves both APIs over the same buckets. Swift accepts mixed-case and underscored container names, but S3 requires lowercase, so a container created through Swift with uppercase or underscores can’t be created or addressed as an S3 bucket.

Create and name everything through S3 with lowercase hyphenated names.

get-account-summary not implemented

Returns An error occurred (Unknown).

Don’t rely on it; use list-users / list-groups / s3api list-buckets for inventory.

Bucket-policy condition keys are a subset

Only aws:CurrentTime, aws:EpochTime, aws:PrincipalType, aws:Referer, aws:SourceIp, aws:SecureTransport; no string interpolation.

Keep bucket policies simple; put per-project logic in group policies. See “Resource-based bucket policies”.

No temporary credentials

Every credential is long-lived.

Scope keys tightly, rotate and revoke them by hand.

Command cheat sheet

Task

Command

List users

iam list-users

List groups

iam list-groups

List buckets

s3api list-buckets

Members of a group

iam get-group --group-name G

A user’s groups

iam list-groups-for-user --user-name U

Managed policies on a user

iam list-attached-user-policies --user-name U

Managed policies on a group

iam list-attached-group-policies --group-name G

Inline policy names on a user

iam list-user-policies --user-name U

Inline policy names on a group

iam list-group-policies --group-name G

Read an inline group policy

iam get-group-policy --group-name G --policy-name G †

Bucket policy

s3api get-bucket-policy --bucket B

Create user (name == display-name)

iam create-user --user-name U

Create user credentials

iam create-access-key --user-name U

Create group

iam create-group --group-name G

Attach managed policy

iam attach-user-policy --user-name U --policy-arn ARN

Detach managed policy

iam detach-user-policy --user-name U --policy-arn ARN

Attach inline policy

iam put-group-policy --group-name G --policy-name G ... †

Delete inline policy

iam delete-group-policy --group-name G --policy-name G †

Add user to group

iam add-user-to-group --group-name G --user-name U

Remove user from group

iam remove-user-from-group --group-name G --user-name U

Create bucket

s3 mb s3://B

Set a bucket policy

s3api put-bucket-policy --bucket B --policy file://...

Remove a bucket policy

s3api delete-bucket-policy --bucket B

† The inline policy name is just the group name G — no separate name to manage. See S3 limitations.

Example naming scheme

This scheme is not enforced by anything; it is the sample used throughout this guide. Adapt it to your organisation.

Thing

Convention

Example

Root user

iam-<account>-root-<id>

iam-copernicus-root-mission-control

Person

<org_prefix>-<id>

cop-envisat

Service login

svc-<project>

svc-sentinel2-mosaic

External partner

ext-<project>-<who>

ext-sentinel2-mosaic-partner

Group

<org_prefix>-<project>-<tier>

cop-sentinel2-mosaic-rw

Inline policy

same as the group name

cop-sentinel2-mosaic-rw

Access tier

rw / ro / wo

...-ro, ...-wo

Bucket

lowercase, DNS-compliant

cop-sentinel2-mosaic

Every name in this scheme is lowercase and hyphenated, so one rule covers persons, groups, policies, and buckets alike. Bucket names share one flat namespace for the whole account, so keep them distinctive; an org or project prefix helps.