eosvc 1.1.0

EOSVC (Ersilia Version Control) is a Python-based tool designed to unify code versioning with data and result management by combining Git and S3 workflows around the main branch.

Ersilia Version Control

EOSVC is a small CLI for syncing large artifacts to S3, while your code remains in Git.

EOSVC supports two repo types (detected from access.json):

• Standard repos: manage data/ and output/
• Model repos: manage model/checkpoints/ and model/framework/fit/

EOSVC does not manage Git operations anymore (no clone/pull/push). Use git directly for code workflows.

---

Quick Start

Installation

Clone the repository and install the package in editable mode:

git clone https://github.com/ersilia-os/eosvc.git
cd eosvc
pip install -e .

Verify that the CLI is available:

eosvc --help

AWS Credentials Setup

1. Create an AWS Access Key

Create an access key in AWS (IAM) for a user or role with permissions to access the target S3 bucket.

You will need:

• Access Key ID
• Secret Access Key
• Session Token (only if using temporary credentials)
• AWS Region (example: eu-central-2)

2. Configure EOSVC

Run the following command and replace the placeholders with your credentials:

eosvc config \
  --access-key-id "..." \
  --secret-access-key "..." \
  --session-token "..." \
  --region "eu-central-2"

Access Rules Configuration (access.json)

Create or edit an access.json file in your working directory to define which folders are uploaded to S3 and their access level.

Example

{
  "data": "public",
  "output": "private"
}

• Files inside the data/ folder will be uploaded with public access.
• Files inside the output/ folder will be uploaded with private access

Git Ignore (Recommended)

If you are working inside a git repository, prevent data folders from being committed by adding them to .gitignore:

data/
output/

Upload Data

Upload data from a local directory to S3:

eosvc upload --path <path-to-data-to-upload>

Download Data

Download data from S3 into a local directory:

eosvc download --path <path-to-data-to-download>

Technical Details

What EOSVC stores where

EOSVC syncs artifacts under an S3 prefix equal to the repo name.

By default, the repo name is the local folder name (repo directory basename).
If your folder name differs from the remote repo/S3 prefix, set:

export EVC_REPO_NAME="my-actual-repo-name"

Standard repos

Managed roots:

• data/
• output/

S3 mapping for repo ersilia-repo:

• s3://<bucket>/ersilia-repo/data/...
• s3://<bucket>/ersilia-repo/output/...

Model repos

Managed roots:

• model/checkpoints/
• model/framework/fit/

Accepted path aliases for convenience:

• checkpoints/... → model/checkpoints/...
• fit/... → model/framework/fit/...

S3 mapping for repo my-model-repo:

• s3://<bucket>/my-model-repo/model/checkpoints/...
• s3://<bucket>/my-model-repo/model/framework/fit/...

In model repos, EOSVC refuses operations on data/ and output/.

---

Buckets and access

Buckets:

Bucket	Used by	Access
eosvc-public	Standard repos (data, output)	Public
eosvc-private	Standard repos (data, output)	Private
eosvc-models-public	Model repos (checkpoints, fit)	Public
eosvc-models-private	Model repos (checkpoints, fit)	Private

Rules:

• Read from eosvc-public or eosvc-models-public may work without AWS credentials (unsigned S3 client).
• Read from eosvc-private or eosvc-models-private requires AWS credentials.
• Any upload requires AWS credentials, regardless of bucket.

Note: For unauthenticated reads to work, the public bucket policy must allow s3:GetObject. For unauthenticated view to work, it must also allow s3:ListBucket constrained to the relevant prefixes.

---

Installation

pip install -e .

eosvc --help

---

Credentials

EOSVC resolves credentials in this order:

1. .env files (loaded with python-dotenv) from:

   • <repo>/.config/.env and <repo>/.config/eosvc/.env
   • ./.config/.env and ./.config/eosvc/.env
   • ~/.eosvc/.config (written by eosvc config)
   • <repo>/.env and ./.env

2. AWS default credential chain (environment variables and/or ~/.aws/* if present)

3. Falls back to anonymous — only valid for reads from public buckets

Option A: environment variables (standard AWS)

export AWS_ACCESS_KEY_ID="..."
export AWS_SECRET_ACCESS_KEY="..."
export AWS_SESSION_TOKEN="..."   # optional
export AWS_REGION="eu-central-2"     # optional

Option B: EOSVC config (writes ~/.eosvc/.config)

EOSVC provides a config command to store credentials in:

• ~/.eosvc/.config (permissions set to 600 when possible)

eosvc config \
  --access-key-id "..." \
  --secret-access-key "..." \
  --session-token "..." \
  --region "eu-central-2"

This is similar in spirit to aws configure, but EOSVC writes a .env file and loads it alongside other sources.

Option C: local .env files

Create .env in the repo (or current directory):

AWS_ACCESS_KEY_ID="..."
AWS_SECRET_ACCESS_KEY="..."
AWS_SESSION_TOKEN="..."   # optional
AWS_REGION="eu-central-2"    # optional

---

access.json (required)

EOSVC requires an access.json at the repo root. EOSVC identifies the repo root by searching upward for access.json starting from the current directory.

Standard repo access.json

{
  "data": "public",
  "output": "private"
}

Model repo access.json

{
  "checkpoints": "public",
  "fit": "public"
}

Valid values are: "public" or "private".

---

Commands

config

Write AWS credentials to ~/.eosvc/.config:

eosvc config --access-key-id "..." --secret-access-key "..."

Optional flags:

eosvc config \
  --access-key-id "..." \
  --secret-access-key "..." \
  --session-token "..." \
  --region "eu-central-2" \
  --default-region "eu-central-2"

view (S3 tree)

View the artifact layout in S3:

eosvc view
eosvc view --path data
eosvc view --path output
eosvc view --path model/checkpoints
eosvc view --path model/framework/fit
eosvc view --path checkpoints
eosvc view --path fit

download

Download a file or folder from S3 into your repo:

eosvc download --path data/processed/file.csv
eosvc download --path output/
eosvc download --path model/checkpoints/
eosvc download --path model/framework/fit/
eosvc download --path checkpoints/
eosvc download --path fit/

upload

Upload a file or folder to S3 (requires credentials):

eosvc upload --path output/some_folder
eosvc upload --path data/test
eosvc upload --path model/checkpoints/test-run
eosvc upload --path model/framework/fit/test-fit
eosvc upload --path checkpoints/test-run
eosvc upload --path fit/test-fit

---

Quick Test

From the project root:

cd tests
chmod +x test.sh
./test.sh

The test script:

• uses git clone to obtain test repos
• runs view, upload, and download for both model and standard repos

---

Access lock (no public/private migration)

EOSVC creates a local lock file:

• .eosvc/access.lock.json

If you later change access.json (e.g., public → private), EOSVC will refuse to run.

To override (not recommended), delete the lock file manually:

rm .eosvc/access.lock.json

---

Common troubleshooting

“AccessDenied” when reading a public bucket without creds

Your bucket policy probably does not allow anonymous access for the prefixes EOSVC uses.

For standard repos (eosvc-public), if you want unauthenticated download to work:

• allow s3:GetObject on arn:aws:s3:::eosvc-public/*

If you want unauthenticated view to work:

• also allow s3:ListBucket on arn:aws:s3:::eosvc-public
• restrict with s3:prefix conditions for your repo prefixes

For model repos (eosvc-models-public), apply the same policy to arn:aws:s3:::eosvc-models-public.

“AWS credentials are missing or invalid”

Provide credentials via:

• env vars, or
• eosvc config (writes ~/.eosvc/.config), or
• a .env file in the repo/current directory or its .config/ subdirectory

---

About the Ersilia Open Source Initiative

The Ersilia Open Source Initiative is a tech-nonprofit organization fueling sustainable research in the Global South. Ersilia's main asset is the Ersilia Model Hub, an open-source repository of AI/ML models for antimicrobial drug discovery.