What is this?

A basic Content Addressable aRchive (CAR) v1 reader and writer. Enough to get you reading from the ATProto firehose and emitting blocks from your own PDS. More details on ATProto's CAR usage here.

Installation

pip install carbox

Basic Usage

from datetime import datetime
from carbox.car import Block, read_car, write_car
from carbox.message import read_event_pair
import dag_cbor

# Where websocket_msg is a message comes from the firehose.
header, event = read_event_pair(websocket_msg)
roots, blocks = read_car(event['blocks'])
records = [dag_cbor.decode(block) for block in blocks]

record = {
  '$type': 'app.bsky.feed.post',
  'text': 'Hello, world!',
  'createdAt': datetime.now().isoformat(),
}
block = Block(decoded=record)
car_bytes = write_car(block.cid, [block])

Changelog

0.3 - 2024-03-16

• Relax dependencies to allow minor version upgrades.

0.2 - 2023-09-07

Add write support via car.write_car. See usage example above for details.

0.0.1 - 2023-05-07

Initial release.

Release instructions

Here's how to package, test, and ship a new release.

1. Run the unit tests.

   source [path_to_virtualenv]/bin/activate.csh
   pytest
   

2. Bump the version number in pyproject.toml. git grep the old version number to make sure it only appears in the changelog. Change the current changelog entry in README.md for this new version from unreleased to the current date.

3. git commit -am 'release vX.Y'

4. Upload to test.pypi.org for testing.

   poetry build
   setenv ver X.Y
   twine upload -r pypitest dist/carbox-$ver*
   

5. Install from test.pypi.org.

   cd /tmp
   python3 -m venv local
   source local/bin/activate.csh
   # make sure we force pip to use the uploaded version
   pip3 uninstall carbox
   pip3 install --upgrade pip
   pip3 install -i https://test.pypi.org/simple --extra-index-url https://pypi.org/simple carbox==$ver
   deactivate
   

6. Smoke test that the code trivially loads and runs:

   from carbox.car import Block, read_car, write_car
   
   block = Block(decoded={'foo': ['bar', 2, 3.14]})
   car_bytes = write_car([block.cid], [block])
   assert read_car(car_bytes) == ([block.cid], [block])
   

7. Tag the release in git. In the tag message editor, delete the generated comments at bottom, leave the first line blank (to omit the release "title" in github), put ### Notable changes on the second line, then copy and paste this version's changelog contents below it.

   git tag -a v$ver --cleanup=verbatim
   git push && git push --tags
   

8. Click here to draft a new release on GitHub. Enter vX.Y in the Tag version box. Leave Release title empty. Copy ### Notable changes and the changelog contents into the description text box.

9. Upload to pypi.org!

   twine upload dist/carbox-$ver*