rust-s3-server

KO | EN

gitlite — search

#python #ai #typescript #llm #golang #nodejs #macos #windows #kubernetes #vue #security #mcp #openai #open-source

(★ 18)

An S3 compatible Object Storage written in Rust. Good performance, good for home use!

rust-s3-server Latest Version Download

Download Latest Version (.zip)

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# rusts3 — single-node S3-compatible server

A lightweight, single-node S3-compatible server in Rust, with a built-in
management console. The core AWS S3 API — authentication, bucket/object CRUD,
listings, multipart, copy, presigning, browser POST, and both path-style and
virtual-hosted-style addressing — is verified continuously against the MinIO
Mint suite and real AWS SDKs.

Good fit for local development, integration tests, CI, and small trusted
production deployments. Features that only make sense at AWS scale (versioning,
ACLs, replication, S3 Select) are [deliberately out of scope](#intentional-limits).

## Quick start

```bash
cargo build --release
target/release/rusts3 init      # writes config.yaml with every option at its default
target/release/rusts3 run
```

Or with Docker — the image runs with no configuration at all:

```bash
docker build -t rusts3:latest .
docker run -d --name rusts3 \
  -p 8002:8002 -p 8003:8003 \
  -v rusts3-data:/data \
  -e RUSTS3_ADMIN_PASSWORD=change-me \
  -e RUSTS3_ACCESS_KEY=myaccesskey -e RUSTS3_SECRET_KEY=mysecretkey \
  rusts3:latest
```

- **S3 API** → `http://127.0.0.1:8002` (point the AWS CLI/SDKs, `mc`, `s3cmd`, rclone here)
- **Console** → `http://127.0.0.1:8003` (username/password login)

The image also ships `rs3`, the bundled high-performance CLI client
(see [`client/README.md`](client/README.md)), so a running container can be
driven without installing anything else:

```bash
docker exec -e MC_HOST_LOCAL=http://myaccesskey:mysecretkey@127.0.0.1:8002 \
  rusts3 rs3 ls --recursive local/
```

## The two surfaces

rusts3 exposes two independent HTTP surfaces from one binary:

| | S3 API (`8002`) | Management console (`8003`) |
|---|---|---|
| Speaks | AWS S3 wire protocol | Operator web UI |
| Auth | Access keys (SigV4/SigV2, presigned, browser POST) | Username/password sessions |
| For | Any S3 client or SDK | Day-to-day operation and IAM admin |

Credentials never mix: a leaked console password grants no S3 API access, and
vice versa. Each surface can be firewalled or proxied independently.

## Highlights

- **Fully streaming** single-part and multipart uploads/downloads — object
  bodies are never buffered in memory, on the API or in the console.
- **SigV4 and SigV2** authentication, presigned URLs, `aws-chunked` streaming,
  and signed browser POST uploads.
- **Path-style and virtual-hosted-style addressing**, with adaptive SigV4
  verification for both shapes.
- **AWS-style IAM policy evaluation**: explicit deny, wildcards, prefix/delimiter
  conditions, user and group policies — managed from the console.
- **Management console**: object browser with multi-select and bulk delete,
  folder drag-and-drop upload, transparent multipart for large files (to the
  5 TiB S3 ceiling), in-flight multipart visibility with one-click abort, IAM
  administration, share links, and a live task monitor.
- **Crash-safe storage**: RocksDB-primary indexes over immutable blob
  directories, write-ahead intents, atomic publication, automatic index
  rebuilds, and configurable durability.
- Request/auth/authz/operation audit logs, live throughput, and
  MinIO-compatible health and metrics paths.

<details>
<summary>Screenshots</summary>

### Login

### Object browser

### IAM and policies

### Policy builder

### Live tasks

### Object scanner

</details>

## Compatibility

### MinIO Mint

The latest MinIO Mint core run exercised all 15 suites: **362 passes out of 417
checks**, 25 failures, 30 not applicable. Core bucket/object CRUD, ranged reads,
ListObjects v1/v2, multipart, copy/compose, presigned GET/PUT, browser POST,
health checks, and metrics passed broadly across clients. Unimplemented features
now report `NotImplemented` instead of being silently accepted, which is why the
not-applicable count is higher and the failure count lower than in earlier runs.

| Client/suite | Result |
|---|---|
| `s3cmd` | 8 pass, 0 fail |
| AWS SDK for Ruby | 13 pass, 0 fail |
| Health checks | 6 pass, 0 fail |
| AWS CLI | 16 pass, 0 fail |
| MinIO Client (`mc`) | 26 pass, 1 feature-gap failure |
| MinIO JavaScript | 234 pass, 3 feature-gap failures |

The dated [MinIO Mint compatibility report](test_report.md) is the source of
truth — the summary above is not a claim of complete S3 or MinIO compatibility.

### The Rust data stack (`object_store`)

rusts3 ships a dedicated harness that drives the
[`object_store`](https://crates.io/crates/object_store) crate (0.14, `aws`
feature) against a live server — the S3 client underneath **Delta Lake,
DataFusion, Iceberg, Lance**, and much of the Rust data ecosystem. If it passes,
those applications' storage layer works on rusts3.

Fifteen tests cover put/get roundtrip, overwrite, empty objects, `head` (size +
ETag), `get_range` / `get_ranges`, `NotFound` mapping, recursive `list`,
`list_with_delimiter`, `list_with_offset`, `copy`, `rename`, multipart upload
and abort, and **conditional PUT** (`PutMode::Create` via `If-None-Match: *` —
the atomic-commit primitive Delta Lake and Iceberg depend on). No known gaps.

```sh
cd compat/object_store && ./run.sh
```

The launcher is self-contained: it builds rusts3, starts a throwaway instance on
a temp data dir and free ports, creates the bucket, runs the tests, and tears
everything down. See [`compat/object_store/README.md`](compat/object_store/README.md)
for filtering, port overrides, and running against an existing server.

## Intentional limits

Not implemented:

- bucket versioning and delete markers;
- bucket policies via the S3 API (use console IAM policies instead), ACLs,
  websites, lifecycle rules, tagging;
- object lock, retention/legal hold, replication, notifications, S3 Select;
- server-side encryption and storage-tier behavior (storage class is metadata only);
- MinIO admin APIs and Snowball archive extraction;
- **TLS termination** — put rusts3 behind a reverse proxy (see below).

`GET /{bucket}?versions` is a compatibility view that reports the current object
of each key as its single `null` version, exactly as S3 answers for an
unversioned bucket. Overwritten and deleted blobs are retained in trash for the
retention window, but they are an operator recovery buffer, not versions: there
is no `?versionId` API to fetch or delete one, so they are not listed.

## Configuration

Every field is optional and falls back to the default below. `rusts3 init`
writes a complete `config.yaml` by serializing the built-in defaults, so it can
never drift from the parser — the trade-off is no inline comments, which is why
field documentation lives here and in `src/server/config.rs` /
`src/storage/config.rs`.

A minimal authenticated setup:

```yaml
server:
  bind_port: 8002
  base_dir: ./rusts3-data

auth:
  enabled: true
  public_hostname: "s3.example.test"   # no scheme; enables host-style + share links
  public_scheme: https
  users:
    - user: admin
      password: "$2b$12$..."           # generate with: rusts3 genpassword
      api_keys:
        - ak: "ROOTKEY001"
          secret: "replace-this-secret"

ui:
  enabled: true
  bind_port: 8003
```

Built-in users live in configuration, cannot be edited at runtime, and bypass
policy checks. Runtime users, groups, access keys, and policies live in
`<base_dir>/admin.rocksdb` and are **default-deny** unless an attached
user/group policy allows the request; a matching explicit deny always wins.

Bcrypt is recommended for console passwords (cleartext is still accepted). S3
secrets must stay recoverable, because request authentication needs the original
HMAC key.

<details>
<summary>Full field reference</summary>

### Server and UI

| Field | Default | Description |
|---|---:|---|
| `server.bind_address` | `0.0.0.0` | S3 API listen address. |
| `server.bind_port` | `8002` | S3 API listen port. |
| `server.base_dir` | `./rusts3-data` | Object, index, and IAM data root. |
| `ui.enabled` | `true` | Enable the management console. |
| `ui.bind_address` | S3 bind address | Optional separate console listen address. |
| `ui.bind_port` | `8003` | Console listen port. |
| `ui.public_hostname` | absent | Public console hostname and optional port; its exact origin is allowed to PUT through the S3 CORS layer. Required for direct console uploads. |
| `ui.public_scheme` | `http` | Console origin scheme (`http` or `https`). |

### Storage

| Field | Default | Description |
|---|---:|---|
| `storage.meta_cache_capacity` | `200000` | Maximum cached object metadata entries. |
| `storage.durability` | `full` | `full` fsyncs blobs and syncs the RocksDB index WAL on every write; `relaxed` skips per-PUT blob fsync and leaves the WAL to the OS. |
| `storage.rebuild_reader_threads` | `0` | Parallel index-rebuild workers; `0` selects one per CPU core. |
| `storage.rebuild_queue_bound` | `1000` | Bounded rebuild pipeline queue. |
| `storage.rebuild_batch_size` | `1000` | Index rows written per rebuild batch. |

`full` is the safe default for power-loss durability. `relaxed` improves write
throughput but can lose the last acknowledged writes after power loss. A normal
process crash preserves committed writes in either mode.

### Authentication and IAM

| Field | Default | Description |
|---|---:|---|
| `auth.enabled` | `false` | Require signatures on the S3 API. Health and metrics endpoints remain public. |
| `auth.credentials` | `[]` | Legacy unrestricted `{access_key, secret_key}` pairs. |
| `auth.users` | `[]` | Built-in, unrestricted bootstrap administrators. |
| `auth.public_hostname` | absent | Public hostname and optional port used to verify proxy-safe signatures, generate share links, and enable virtual-hosted-style addressing (`<bucket>.<public_hostname>`). No scheme. |
| `auth.public_scheme` | `http` | `http` or `https`, used for generated share links. |

### Logging

| Field | Default | Description |
|---|---:|---|
| `logging.level` | `info` | `trace`, `debug`, `info`, `warn`, or `error`. |
| `logging.enable_bandwidth_report` | `true` | Log aggregate bytes, rates, request totals, and QPS every 10 seconds. |
| `logging.dir` | absent | Split logs into `auth.log`, `authz.log`, `audit.log`, `server.log`. Takes precedence over `file`. |
| `logging.file` | absent | Combined rolling log file. With neither `dir` nor `file`, logs go to stdout only. |
| `logging.rotation_size_mb` | `100` | Rotate a file at this size. |
| `logging.keep_files` | `5` | Number of rotated archives to keep. |
| `logging.compress` | `false` | Gzip rotated archives. |

File logging also echoes to stdout. Every request receives a correlation ID,
returned in `x-amz-request-id` and included in logs.

### Background maintenance

| Field | Default | Description |
|---|---:|---|
| `sweeper.interval_secs` | `300` | Schedule interval for intent resolution, staging/trash cleanup, and legacy-layout migration. |
| `sweeper.intent_batch_size` | `100` | Stale intents processed per batch; a run drains all eligible batches. |
| `sweeper.intent_grace_period_secs` | `3600` | Minimum intent age during normal operation. Startup recovery bypasses the grace period. |
| `sweeper.staging_expiry_secs` | `86400` | Idle age before abandoned single-PUT staging is removed. |
| `sweeper.multipart_upload_expiry_secs` | `2592000` | Idle age before an incomplete multipart upload is removed; `0` keeps them forever (S3 behavior). |
| `sweeper.trash_expiry_secs` | `86400` | Idle age before retired blobs are removed; values below 10800 (3 hours) are rejected. |
| `sweeper.reclaim_interval_secs` | `300` | Interval for reclaiming empty fanout directories. |

Older visibility-repair setting names are accepted as aliases for the intent
batch/grace settings.

</details>

## CLI

```text
rusts3 run [-c FILE]                   Start the server (default: config.yaml)
rusts3 validate [-c FILE]              Validate configuration and exit
rusts3 genpassword [--cost N]          Generate a bcrypt console password
rusts3 verifypassword [HASH]           Verify a bcrypt console password
rusts3 init                            Write a config.yaml with every option at its default
rusts3 healthcheck [-c FILE]           Probe a running server; exit non-zero if unhealthy
```

Running `rusts3` with no subcommand is still supported (built-in defaults, or
the file given with the legacy `-c`). Only one process may own a data directory;
this is enforced with `<base_dir>/.rusts3.lock`.

## Using the AWS CLI

```bash
aws configure --profile local          # region: us-east-1
export AWS_PROFILE=local
export AWS_ENDPOINT_URL=http://127.0.0.1:8002

aws s3 mb s3://my-bucket
aws s3 cp ./file.bin s3://my-bucket/path/file.bin
aws s3 ls s3://my-bucket/
aws s3 cp s3://my-bucket/path/file.bin s3://my-bucket/copy.bin
aws s3 rm s3://my-bucket/path/ --recursive
aws s3 presign s3://my-bucket/copy.bin --expires-in 3600
```

Multipart is selected automatically for large files; low-level multipart is
available via `aws s3api`.

## Reverse proxy and TLS

rusts3 serves plain HTTP and delegates TLS to a fronting proxy. A production
setup typically publishes three names:

| Public name | Backend | Purpose |
|---|---|---|
| `rusts3.example.com` | `127.0.0.1:8003` | Management console. |
| `s3.example.com` | `127.0.0.1:8002` | S3 API, path-style. Set as `auth.public_hostname`. |
| `*.s3.example.com` | `127.0.0.1:8002` | S3 API, virtual-hosted-style (bucket subdomains). |

Whatever proxy you use, it **must**:

1. **preserve the `Host` header** — SigV4 verification and host-style bucket
   detection both depend on it;
2. stream request/response bodies without size limits or buffering (console
   multipart parts are 256 MiB);
3. pass WebSocket upgrades for the console's `/api/tasks/ws` task monitor.

Then set `auth.public_hostname: s3.example.com` and `auth.public_scheme: https`
so presigned links and host-style detection agree with the public names.

<details>
<summary>Proxy choices and virtual-hosted-style details</summary>

### Recommended: tlsproxy_rs

[`tlsproxy_rs`](https://github.com/wushilin/tlsproxy_rs) is a hostname-routed
proxy with **integrated ACME certificate management** — it obtains and renews
Let's Encrypt certificates automatically via TLS-ALPN-01, issuing per-hostname
certificates on demand as routes are hit (renewals start 15 days before expiry).
That makes it a natural fit: point the three names above at their backend ports
through its admin console, and certificates for the console, the API endpoint,
and each bucket subdomain are provisioned and rotated with no manual handling.
It also provides an audited admin UI with configuration revisions and rollback,
hot reload, and load-balanced backend pools.

### Alternative: Caddy

[Caddy](https://caddyserver.com/) works equally well; the wildcard certificate
for bucket subdomains requires the DNS-01 challenge (a DNS provider plugin):

```caddyfile
rusts3.example.com {
    reverse_proxy 127.0.0.1:8003
}
s3.example.com, *.s3.example.com {
    reverse_proxy 127.0.0.1:8002
}
```

### Virtual-hosted-style addressing

Setting `auth.public_hostname` enables host-style requests: any request whose
`Host` is `<bucket>.<public_hostname>` is served as an access to that bucket,
while requests to the bare hostname stay path-style — the same dual behavior as
AWS S3. SigV4 verification is adaptive: each request is verified against exactly
the path and host the client signed, in either style. The proxy must provide
wildcard DNS and TLS for `*.<public_hostname>` and preserve `Host`.

</details>

## Management console

Open `http://127.0.0.1:8003`. Configure at least one `auth.users` entry with a
password to bootstrap administration. The console supports:

- creating/deleting buckets; browsing, uploading, downloading, deleting objects;
- per-bucket CORS settings, with the configured console origin implicitly allowed;
- folder upload (button or drag-and-drop), recreating the tree under the current prefix;
- transparent multipart: files over 256 MiB are split into streamed parts (no
  browser or server buffering) up to 5 TiB, with progress and cancel-with-abort;
- per-bucket view of in-flight multipart uploads with one-click abort — covering
  both console uploads and sessions started by external S3 clients;
- multi-select and bulk delete (recursive for folders), plus prefix-scoped bulk
  delete with a typed prefix, warning dialog, and live progress;
- presigned share links (requires `auth.public_hostname`);
- runtime users and groups, password resets, access-key rotation, policy editing
  via both a rule builder and a JSON editor;
- IAM export, and **staged import**: a read-only preview shows per-family row
  counts and sample names (never secrets) before anything is written;
- bucket statistics and operator-triggered index rebuilds;
- a WebSocket task monitor for active/recent requests and jobs, with throughput
  and cancellation for safely cancellable work.

Share links carry the creator's current authority — deleting the user or
narrowing their policy revokes or narrows existing links.

## Health and metrics

Unauthenticated compatibility endpoints on the S3 port:

```text
GET /minio/health/live
GET /minio/health/ready
GET /minio/prometheus/metrics
GET /minio/v2/metrics/{cluster,node,bucket,resource}
```

The metrics endpoints currently expose a minimal `rusts3_up 1` gauge. Detailed
request/byte totals and rates are in the periodic logs and the console task view.

<details>
<summary>Supported S3 operations (full table)</summary>

### Buckets

| Method | Resource | Operation |
|---|---|---|
| `GET` | `/` | List buckets. |
| `PUT` / `HEAD` / `DELETE` | `/{bucket}` | Create, inspect, or delete an empty bucket. |
| `GET` | `/{bucket}` | ListObjects v1/v2 with prefix, delimiter, marker/continuation token, encoding, and pagination. |
| `GET` | `/{bucket}?location` | Get bucket location. |
| `GET` | `/{bucket}?uploads` | List multipart uploads. |
| `GET/PUT/DELETE` | `/{bucket}?cors` | Read, replace, or remove bucket CORS rules. |
| `GET` | `/{bucket}?versions` | Compatibility listing: one `null` version per live key, paginated by `key-marker`; not S3 versioning. |
| `POST` | `/{bucket}?delete` | Multi-object delete, including quiet mode. |
| `POST` | `/{bucket}?rebuildIndex` | Start an index rebuild (`202`; `409` if already running). |
| `POST` | `/{bucket}` | SigV4 browser form upload with policy validation. |

### Objects

| Method | Resource | Operation |
|---|---|---|
| `PUT` | `/{bucket}/{key}` | Streaming upload, including `aws-chunked`, content hashes, metadata, and storage class. |
| `GET` / `HEAD` | `/{bucket}/{key}` | Streaming read, single byte ranges, conditional reads, response-header overrides, and metadata. |
| `DELETE` | `/{bucket}/{key}` | Idempotent delete. `forceDelete=true` / `x-minio-force-delete` deletes a prefix for MinIO compatibility. |
| `PUT` | object + `x-amz-copy-source` | Server-side copy with metadata directive and source preconditions. |
| `POST` | object + `?uploads` | Initiate multipart upload. |
| `PUT` | object + `uploadId`, `partNumber` | Upload a part, or copy a source/range into a part. |
| `GET` | object + `uploadId` | List uploaded parts. |
| `POST` | object + `uploadId` | Complete multipart upload. |
| `DELETE` | object + `uploadId` | Abort multipart upload. |

Authentication supports header and query-string SigV4/SigV2. Presigned SigV4
URLs enforce AWS's seven-day maximum expiry. Browser POST policies validate
expiry and form conditions before accepting the object.

</details>

<details>
<summary>Docker: config templating, storage permissions, health</summary>

### Config templating

The image ships `config.docker.yaml` at `/etc/rusts3/config.yaml`, in which
every value is a placeholder:

```yaml
server:
  bind_port: {{RUSTS3_PORT:8002}}
  base_dir: "{{RUSTS3_DATA_DIR:/data}}"
```

`{{NAME:default}}` expands to `$NAME` when set and non-empty, else `default`. So
the image runs with no configuration, and any single field can be overridden
with `-e` — there is no separate list of environment bindings to keep in step
with the schema, because the config file *is* the list.

That short form is one *provider* — `env` — with its name implicit. The general
shape is `{{provider<sep>param<sep>param…}}`, where the character immediately
after the provider name is the separator, so a value containing colons can
choose something else:

```text
{{RUSTS3_PORT:8002}}            shorthand: env lookup with a default
{{env:RUSTS3_PORT:8002}}        the same thing, said explicitly
{{env|DSN|postgres://h:5432}}   separator '|', so the colons are data
{{upper:env:RUSTS3_REGION}}     transforms compose with lookups
{{lower$env$RUSTS3_ADMIN_USER}} any single character works as the separator
```

The first segment is a provider only if it *both* matches `[A-Za-z_]+` (letters
and underscores, no digits) and names a registered provider. Failing either, the
placeholder falls back to the `env` shorthand — so `{{a:b}}` is `$a` defaulting
to `b` unless `a` is registered, and `RUSTS3_PORT` cannot be read as a call at
all, because the digit disqualifies it first. Everything after the first
separator is the last parameter, so in `{{a:b:c:d}}` the default is `b:c:d`.

Providers today are `env`, `upper`, and `lower`. Extra parameters are accepted
and ignored, so the calling convention can grow without breaking existing files.
Adding a provider means implementing `Resolver` and registering it; a resolver
receives the whole call including the registry, so it can resolve nested calls
the way `upper` does. If you have an environment variable named after a
provider, disambiguate explicitly: `{{env:ENV}}` reads `$ENV`.

`{{NAME}}` without a default is **required**: if unset, the server refuses to
start and names the variable. Delete the defaults from the `auth` entries in
`config.docker.yaml` if you would rather the container fail than come up with
known credentials.

Substitution is textual and happens before YAML parsing, so quote a placeholder
whose value might contain spaces or YAML punctuation. To go further than
individual overrides, mount your own file over `/etc/rusts3/config.yaml`.

### Unfilled slots

Templating cannot express "if this, then not that", so a container config is
*flat*: it lists every slot a deployment might use and lets unused ones expand
to nothing. An unfilled slot is treated as an absent one.

```yaml
auth:
  users:
    - user: "{{RUSTS3_ADMIN_USER:admin}}"
      password: "{{RUSTS3_ADMIN_PASSWORD:changeme}}"
      api_keys:
        - ak: "{{RUSTS3_ACCESS_KEY:rusts3admin}}"
          secret: "{{RUSTS3_SECRET_KEY:rusts3admin}}"
        - ak: "{{RUSTS3_ACCESS_KEY_2:}}"      # unset → the whole entry is dropped
          secret: "{{RUSTS3_SECRET_KEY_2:}}"
    - user: "{{RUSTS3_USER_2:}}"              # unset → this user does not exist
      password: "{{RUSTS3_PASSWORD_2:}}"
```

**Empty is the marker**, rather than a reserved word like `none`. An unset
variable already produces empty, so there is no convention to remember and no
escaping rule for the day someone genuinely wants a value of `none`; and empty
can never be a valid username, access key, or secret, so nothing legitimate is
discarded by accident. The rule applies to optional scalars too — an empty
`public_hostname` reads as "not configured". Anything dropped is named on
stderr at startup:

```text
config: ignoring 3 unfilled entries (an unfilled api_key of user "admin",
an unfilled auth.users entry, an unfilled auth.users entry)
```

One case is a safety fix rather than a convenience. A built-in user's password
is compared against the candidate, so a user left with an *empty* password would
authenticate against the empty string. An unfilled password is therefore turned
into no password at all, so that user simply cannot log in. A credential missing
either half is discarded for the same reason.

### Storage permissions

`/data` holds everything durable: buckets, the IAM database, scan history, logs.
A named volume is the simplest thing that works. A **host directory** needs one
extra thought, because the image runs as a non-root user (uid 10001) — either
give that uid ownership:

```bash
sudo chown -R 10001:10001 /srv/rusts3
docker run -d -v /srv/rusts3:/data ... rusts3:latest
```

or run the container as a user that already owns it:

```bash
docker run -d -v /srv/rusts3:/data --user "$(id -u):$(id -g)" ... rusts3:latest
```

Getting this wrong is not mysterious — the server names the directory, the uid
it was running as, and both remedies.

### Health and shutdown

`HEALTHCHECK` runs `rusts3 healthcheck`, which reads the same config the server
did (so it probes whatever port is configured) and requests the unauthenticated
`/minio/health/live`. It needs no `curl` in the image. Podman ignores
`HEALTHCHECK` unless the image is built with `--format docker`.

The server handles `SIGTERM` as well as `SIGINT`, which is what makes
`docker stop` clean rather than a ten-second wait followed by `SIGKILL` — in a
container the entrypoint is PID 1, and PID 1 gets no default signal handling
from the kernel. In-flight connections are given a few seconds to drain and then
the process exits regardless, so a client holding a keep-alive socket cannot
delay a stop.

### Testing an image

`./docker-test.sh` starts the image and drives it with the AWS CLI: bucket
lifecycle, single and multipart round-trips compared byte for byte, ranged
reads, server-side copy, prefix and delimiter listing, presigned download, a
restart to prove the volume is durable, and deletion. Works with docker or
podman.

</details>

<details>
<summary>Storage and recovery model</summary>

Each bucket has an authoritative RocksDB index. Object data lives in an
immutable, self-describing blob directory, and the index records that
directory's path. A mutation follows this protocol:

1. Stream a complete blob into staging.
2. Commit a publish intent.
3. Atomically rename the blob into the live tree.
4. Atomically switch the index row and record retirement of any old blob.
5. Move the old blob to trash and clear the retirement intent.

After a crash, startup drains the small intent table instead of scanning every
object. Reads and listings use the RocksDB index as the source of truth.
Per-key locking, monotonic modification timestamps, and immutable snapshots keep
concurrent overwrites/deletes consistent with in-flight downloads.

The current layout uses a single 16-bit fanout directory:

```text
<base_dir>/
 .rusts3.lock
 admin.rocksdb/
 buckets/<bucket>/
 bucket.json
 index.rocksdb/
 objects/<4-hex>/V1<6-hex>_<unique>/
 meta.json
 part.1, part.2, ...
 staging/put/<id>/
 staging/multipart/<upload-id>/
 trash/<id>/
```

Older four-level fanout objects remain readable and are migrated in the
background. Empty directories are reclaimed without deleting non-empty object
directories.

If an index is missing or uses an old schema, rusts3 starts a parallel rebuild
from `meta.json`. The affected bucket returns `503 SlowDown` while rebuilding,
so normal S3 retry behavior applies. An operator can trigger the same tracked
job from the console or with:

```bash
curl -X POST 'http://127.0.0.1:8002/my-bucket?rebuildIndex'
```

**Graceful shutdown.** `SIGINT` (Ctrl-C) stops accepting new connections, drains
the S3 server, cancels background work through cooperative cancellation,
releases the data-root lock, and exits cleanly.

</details>

rust-s3-server

rust-s3-server Latest Version Download

Today's picks

Was this content helpful?

Recent Feedback

rust-s3-server

rust-s3-server Latest Version Download

Today's picks

Was this content helpful?

Recent Feedback

Download README