arroba 0.4

Python implementation of Bluesky PDS and AT Protocol, including repo, MST, and sync methods

arroba

Python implementation of Bluesky PDS and AT Protocol, including data repository, Merkle search tree, and com.atproto.sync XRPC methods.

Arroba is the Spanish word for the @ character ("at sign").

Install from PyPI with pip install arroba.

License: This project is placed into the public domain.

• Usage
• Changelog
• Release instructions

Usage

See app.py for the minimal wrapper code needed to run a fully functional PDS based on arroba, for testing with the ATProto federation sandbox.

Environment variables:

• APPVIEW_HOST, default api.bsky-sandbox.dev
• BGS_HOST, default bgs.bsky-sandbox.dev
• PLC_HOST, default plc.bsky-sandbox.dev
• PDS_HOST, where you're running your PDS
• REPO_DID, repo user's DID, defaults to contents of repo_did file
• REPO_HANDLE, repo user's domain handle, defaults to did:plc:*.json file
• REPO_PASSWORD, repo user's password, defaults to contents of repo_password file
• REPO_PRIVKEY, repo user's private key in PEM format, defaults to contents of privkey.pem file
• REPO_TOKEN, static token to use as both accessJwt and refreshJwt, defaults to contents of repo_token file. Not required to be an actual JWT.

More docs to come!

Changelog

0.4 - 2023-09-19

• Migrate to ATProto repo v3. Specifically, the existing subscribeRepos sequence number is reused as the new rev field in commits. (Discussion.).
• Add new did module with utilities to create and resolve did:plcs and resolve did:webs.
• Add new util.service_jwt function that generates ATProto inter-service JWTs.
• Repo:
  • Add new signing_key/rotation_key attributes. Generate store, and load both in datastore_storage.
  • Remove format_init_commit, migrate existing calls to format_commit.
• Storage:
  • Rename read_from_seq => read_blocks_by_seq (and in MemoryStorage and DatastoreStorage), add new read_commits_by_seq method.
  • Merge load_repo did/handle kwargs into did_or_handle.
• XRPCs:
  • Make subscribeRepos check storage for all new commits every time it wakes up.
    • As part of this, replace xrpc_sync.enqueue_commit with new send_new_commits function that takes no parameters.
  • Drop bundled app.bsky/com.atproto lexicons, use lexrpc's instead.

0.3 - 2023-08-29

Big milestone: arroba is successfully federating with the ATProto sandbox! See app.py for the minimal demo code needed to wrap arroba in a fully functional PDS.

• Add Google Cloud Datastore implementation of repo storage.
• Implement com.atproto XRPC methods needed to federate with sandbox, including most of repo and sync.
  • Notably, includes subscribeRepos server side over websocket.
• ...and much more.

0.2 - 2023-05-18

Implement repo and commit chain in new Repo class, including pluggable storage. This completes the first pass at all PDS data structures. Next release will include initial implementations of the com.atproto.sync.* XRPC methods.

0.1 - 2023-04-30

Initial release! Still very in progress. MST, Walker, and Diff classes are mostly complete and working. Repo, commits, and sync XRPC methods are still in progress.

Release instructions

Here's how to package, test, and ship a new release.

1. Run the unit tests.

   source local/bin/activate.csh
   python3 -m unittest discover
   

2. Bump the version number in pyproject.toml and docs/conf.py. git grep the old version number to make sure it only appears in the changelog. Change the current changelog entry in README.md for this new version from unreleased to the current date.

3. Build the docs. If you added any new modules, add them to the appropriate file(s) in docs/source/. Then run ./docs/build.sh. Check that the generated HTML looks fine by opening docs/_build/html/index.html and looking around.

4. setenv ver X.Y
   git commit -am "release v$ver"
   

5. Upload to test.pypi.org for testing.

   python3 -m build
   twine upload -r pypitest dist/arroba-$ver*
   

6. Install from test.pypi.org.

   cd /tmp
   python3 -m venv local
   source local/bin/activate.csh
   # make sure we force pip to use the uploaded version
   pip3 uninstall arroba
   pip3 install --upgrade pip
   pip3 install -i https://test.pypi.org/simple --extra-index-url https://pypi.org/simple arroba==$ver
   deactivate
   

7. Smoke test that the code trivially loads and runs.

   source local/bin/activate.csh
   python3
   # TODO: test code
   deactivate
   

8. Tag the release in git. In the tag message editor, delete the generated comments at bottom, leave the first line blank (to omit the release "title" in github), put ### Notable changes on the second line, then copy and paste this version's changelog contents below it.

   git tag -a v$ver --cleanup=verbatim
   git push && git push --tags
   

9. Click here to draft a new release on GitHub. Enter vX.Y in the Tag version box. Leave Release title empty. Copy ### Notable changes and the changelog contents into the description text box.

10. Upload to pypi.org!

    twine upload dist/arroba-$ver*
    

11. Wait for the docs to build on Read the Docs, then check that they look ok.

12. On the Versions page, check that the new version is active, If it's not, activate it in the Activate a Version section.