Join the AlphaWave competition
Developer tools

Installation & setup

Use existing S3 tooling alongside the Recall S3 adapter.


The recall_s3 binary is a simple adapter that allows you to use the Recall object storage system with S3 APIs. Let's walk through how to set it up, and subsequent pages walk through how to use it with ancillary tools like MinIO.

Setup

Install

First, clone the repo:

git clone https://github.com/recallnet/recall-s3

Then, cd into recall-s3 and install the adapter binary:

make install

This should make recall_s3 available in your path, which you can verify with recall_s3 --version. Alternatively, if you don't want to install it, you can run it directly with cargo run in the subsequent step.

You MUST own tokens and credits to create a bucket and store data, respectively. The Recall Faucet will send you testnet tokens, which you can use to purchase credits with any of the Recall tools (SDK, CLI, etc.). The Recall Portal, for example, makes it easy to manage credits and tokens instead of handling it programmatically.

Run the adapter

Starting the Recall S3 adapter uses port 8014 by default. There are three required flags if you're doing write operations:

FlagRequiredDescription
--private-keyYesHex encoded, Ethereum-style private key (ECDSA, secp256k1) for sending transactions (environment variable: PRIVATE_KEY).
--access-keyYesAn S3 access key (e.g,. AKEXAMPLES3S)
--secret-keyYesAn S3 secret key (e.g., SKEXAMPLES3S)
--networkNoThe network to connect to (defaults to testnet; environment variable: NETWORK).
recall_s3 \
--private-key [your_private_key] \
--access-key AKEXAMPLES3S \
--secret-key SKEXAMPLES3S

The access key and secret aren't used for anything other than matching the MinIO client configuration. If you want to do read-only queries, simply omit the private key.

recall_s3 \
--access-key AKEXAMPLES3S \
--secret-key SKEXAMPLES3S

Note that if you didn't install the binary in the previous step, you can run it directly with cargo run --features binary -- ... and the arguments above.

If you do not already have a private key, you can generate one with the Recall CLI. You also must make sure your account exists on the subnet, which requires you to have deposited funds into your subnet account. Neglecting this step will lead to errors like failed to init sequence; actor ... cannot be found.

Create an aliased bucket

Bucket usage slightly deviates from how you typically would in S3. The S3 adapter uses the alias key stored in the bucket's metadata to identify the bucket name and its relationship with other onchain buckets owned by the creator.

Under the hood, the general naming convention is <0x_address>.<bucket_name>. For example, if you create a bucket called foo, the bucket name interpreted by the adapter will be 0x90f79bf6eb2c4f870365e785982e1f101e93b906.foo.

There are two common patterns when you specify a bucket name for usage with the S3 adapter:

  1. Expanded: Use the aliased name prefixed by the creator's public key (i.e., EVM hex address), delimited by a period—e.g., 0x90f79bf6eb2c4f870365e785982e1f101e93b906.foo.
  2. Aliased: Use only the aliased name as the bucket name—e.g., foo.

The S3 API prohibits bucket names from being longer than 63 characters: here. Since a hex address is 42 characters, it leaves 20 characters for the aliased bucket name (since actual buckets are defined as <hex_address>.<name> with a period in between).

Read or write to S3

If you're reading/writing to buckets you've created, you'll typically just use the aliased name as the bucket name. You can opt for the expanded format, if desired, but the adapter will try to use the address of the provided private key as the owner of the bucket—unless explicitly overridden. However, if you're interacting with buckets you don't own, you must use the expanded format with the aliased name prefixed by the creator's public key.

For example, let's say account 0x90f79bf6eb2c4f870365e785982e1f101e93b906 creates a bucket called foo:

  • If 0x90f7... wants to read or write data, they can simply use foo.
  • But, if some other account wants to read or write to foo, they must use the expanded format 0x90f79bf6eb2c4f870365e785982e1f101e93b906.foo.

That is, this pattern can be used for multiwriter buckets, too. If 0x90f7... makes a credit approval to account 0x1234..., the "owner" of the bucket is still 0x90f7.... If 0x1234... wants to write to foo, they must use the expanded format.

More generally, this is required because the bucket alias is not in a global namespace but only as part of the bucket's metadata. The adapter needs to know who the bucket's owner is in order to query the metadata and retrieve the correct bucket.

Next steps

You can now use Recall with any S3-compatible tooling! There are a few walkthroughs that show how to use it with ancillary tools like MinIO and Python boto3.

On this page