Skip to main content

ArcticDB DataFrame Database

Project description


Three minute ArcticDB demo from PyQuantNews

Options data in ArcticDB

ArcticDB Website | 📘 ArcticDB Docs | 📰: ArcticDB Blog | Press Release | Press Release | Community


ArcticDB is a high performance, serverless DataFrame database built for the Python Data Science ecosystem. Launched in March 2023, it is the successor to Arctic.

Use of ArcticDB in production (including business or commercial environments) or for a Database Service requires a paid for license from ArcticDB Limited . Please contact info@arcticdb.io for further details.

ArcticDB offers an intuitive Python-centric API enabling you to read and write Pandas DataFrames to S3 or LMDB utilising a fast C++ data-processing and compression engine.

ArcticDB allows you to:

  • Pandas in, Pandas out: Read and write Pandas DataFrames, NumPy arrays and native types to S3 and LMDB without leaving Python.
  • Built for time-series data: Efficiently index and query time-series data across billions of rows
  • Time travel: Travel back in time to see previous versions of your data and create customizable snapshots of the database
  • Schemaless Database: Append, update and modify data without being constrained by the existing schema
  • Optimised for streaming data: Built in support for efficient sparse data storage
  • Powerful processing: Filter, aggregate and create new columns on-the-fly with a Pandas-like syntax
  • C++ efficiency: Accelerate analytics though concurrency in the C++ data-processing engine

ArcticDB handles data that is big in both row count and column count, so a 20-year history of more than 400,000 unique securities can be stored in a single symbol. Each symbol is maintained as a separate entity with no shared data which means ArcticDB can scale horizontally across symbols, maximising the performance potential of your compute, storage and network.

ArcticDB is designed from the outset to be resilient; there is no single point of failure, and persistent data structures in the storage mean that once a version of a symbol has been written, it can never be corrupted by subsequent updates. Pulling compressed data directly from storage to the client means that there is no server to overload, so your data is always available when you need it.

Quickstart

Prebuilt binary availability (for the current version)

PyPI (Python 3.9 - 3.14) conda-forge (Python 3.10 - 3.14)
Linux x86_64 ✔️ ✔️
Linux arm64 ✔️
Windows x86_64 ✔️ ✔️
MacOS x86_64 ✔️
MacOS arm64 ✔️ ✔️

Storage compatibility

Linux Windows Mac
S3 ✔️ ✔️ ✔️
LMDB ✔️ ✔️ ✔️
Azure Blob Storage ✔️ ✔️ ✔️

We have tested against the following S3 backends:

  • AWS S3
  • Ceph
  • MinIO on Linux
  • Pure Storage S3
  • Scality S3
  • VAST Data S3

Installation

Install ArcticDB:

$ pip install arcticdb

or using conda-forge

$ conda install -c conda-forge arcticdb

Import ArcticDB:

>>> import arcticdb as adb

Create an instance on your S3 storage (with or without explicit credentials):

# Leave AWS to derive credential information
>>> ac = adb.Arctic('s3://MY_ENDPOINT:MY_BUCKET?aws_auth=true')

# Manually specify creds
>>> ac = adb.Arctic('s3://MY_ENDPOINT:MY_BUCKET?region=YOUR_REGION&access=ABCD&secret=DCBA')

Or create an instance on your local disk:

>>> ac = adb.Arctic("lmdb:///<path>")

Create your first library and list the libraries in the instance:

>>> ac.create_library('travel_data')
>>> ac.list_libraries()

Create a test dataframe:

>>> import numpy as np
>>> import pandas as pd
>>> NUM_COLUMNS=10
>>> NUM_ROWS=100_000
>>> df = pd.DataFrame(np.random.randint(0,100,size=(NUM_ROWS, NUM_COLUMNS)), columns=[f"COL_{i}" for i in range(NUM_COLUMNS)], index=pd.date_range('2000', periods=NUM_ROWS, freq='h'))

Get the library, write some data to it, and read it back:

>>> lib = ac['travel_data']
>>> lib.write("my_data", df)
>>> data = lib.read("my_data")

To find out more about working with data, visit our docs


Documentation

The source code for the ArcticDB docs are located in the docs folder, and are hosted at docs.arcticdb.io.

License

ArcticDB is released under a Business Source License 1.1 (BSL)

BSL features are free to use and the source code is available, but users may not use ArcticDB for production use or for a Database Service, without agreement with Man Group Operations Limited.

Use of ArcticDB in production or for a Database Service requires a paid for license from ArcticDB Limited and is licensed under the ArcticDB Software License Agreement. For more information please contact info@arcticdb.io.

The BSL is not certified as an open-source license, but most of the Open Source Initiative (OSI) criteria are met. Please see version conversion dates in the below table:

ArcticDB Version License Converts to Apache 2.0
1.0 Business Source License 1.1 Mar 16, 2025
1.2 Business Source License 1.1 May 22, 2025
1.3 Business Source License 1.1 Jun 9, 2025
1.4 Business Source License 1.1 Jun 23, 2025
1.5 Business Source License 1.1 Jul 11, 2025
1.6 Business Source License 1.1 Jul 25, 2025
2.0 Business Source License 1.1 Aug 29, 2025
3.0 Business Source License 1.1 Sep 13, 2025
4.0 Business Source License 1.1 Sep 27, 2025
4.1 Business Source License 1.1 Nov 1, 2025
4.2 Business Source License 1.1 Nov 12, 2025
4.3 Business Source License 1.1 Feb 7, 2026
4.4 Business Source License 1.1 Apr 5, 2026
4.5 Business Source License 1.1 Aug 14, 2026
5.0 Business Source License 1.1 Oct 31, 2026
5.1 Business Source License 1.1 Nov 15, 2026
5.2 Business Source License 1.1 Jan 27, 2027
5.3 Business Source License 1.1 Mar 24, 2027
5.4 Business Source License 1.1 Apr 28, 2027
5.5 Business Source License 1.1 May 13, 2027
5.6 Business Source License 1.1 May 26, 2027
5.7 Business Source License 1.1 Jun 2, 2027
5.8 Business Source License 1.1 Jun 9, 2027
5.9 Business Source License 1.1 Jul 7, 2027
5.10 Business Source License 1.1 Jul 29, 2027
6.1 Business Source License 1.1 Aug 20, 2027
6.2 Business Source License 1.1 Sep 8, 2027
6.3 Business Source License 1.1 Nov 3, 2027
6.5 Business Source License 1.1 Dec 11, 2027
6.6 Business Source License 1.1 Jan 6, 2028

Code of Conduct

Code of Conduct

This project has adopted a Code of Conduct. If you have any concerns about the Code, or behaviour that you have experienced in the project, please contact us at info@arcticdb.io.

Contributing/Building From Source

We welcome your contributions to help us improve and extend this project!

Please refer to the Contributing page and feel free to open issues on GitHub.

We are also always looking for feedback from our dedicated community! If you have used ArcticDB please let us know, we would love to hear about your experience!

Our release process is documented here.

Community

We would love to hear how your ArcticDB journey evolves, email us at info@arcticdb.io or come chat to us on Twitter!

Interested in learning more about ArcticDB? Head over to our blog!

Do you have any questions or issues? Chat to us and other users through our dedicated Slack Workspace - sign up for Slack access on our website.

Project details


Release history Release notifications | RSS feed

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distributions

No source distribution files available for this release.See tutorial on generating distribution archives.

Built Distributions

If you're not sure about the file name format, learn more about wheel file names.

arcticdb-6.11.0-cp314-cp314-win_amd64.whl (9.4 MB view details)

Uploaded CPython 3.14Windows x86-64

arcticdb-6.11.0-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (19.4 MB view details)

Uploaded CPython 3.14manylinux: glibc 2.17+ x86-64

arcticdb-6.11.0-cp314-cp314-macosx_15_0_arm64.whl (18.0 MB view details)

Uploaded CPython 3.14macOS 15.0+ ARM64

arcticdb-6.11.0-cp313-cp313-win_amd64.whl (9.2 MB view details)

Uploaded CPython 3.13Windows x86-64

arcticdb-6.11.0-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (19.4 MB view details)

Uploaded CPython 3.13manylinux: glibc 2.17+ x86-64

arcticdb-6.11.0-cp313-cp313-macosx_15_0_arm64.whl (18.0 MB view details)

Uploaded CPython 3.13macOS 15.0+ ARM64

arcticdb-6.11.0-cp312-cp312-win_amd64.whl (9.2 MB view details)

Uploaded CPython 3.12Windows x86-64

arcticdb-6.11.0-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (19.5 MB view details)

Uploaded CPython 3.12manylinux: glibc 2.17+ x86-64

arcticdb-6.11.0-cp312-cp312-macosx_15_0_arm64.whl (18.0 MB view details)

Uploaded CPython 3.12macOS 15.0+ ARM64

arcticdb-6.11.0-cp311-cp311-win_amd64.whl (9.2 MB view details)

Uploaded CPython 3.11Windows x86-64

arcticdb-6.11.0-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (19.5 MB view details)

Uploaded CPython 3.11manylinux: glibc 2.17+ x86-64

arcticdb-6.11.0-cp311-cp311-macosx_15_0_arm64.whl (18.0 MB view details)

Uploaded CPython 3.11macOS 15.0+ ARM64

arcticdb-6.11.0-cp310-cp310-win_amd64.whl (9.2 MB view details)

Uploaded CPython 3.10Windows x86-64

arcticdb-6.11.0-cp310-cp310-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (19.5 MB view details)

Uploaded CPython 3.10manylinux: glibc 2.17+ x86-64

arcticdb-6.11.0-cp310-cp310-macosx_15_0_arm64.whl (18.0 MB view details)

Uploaded CPython 3.10macOS 15.0+ ARM64

arcticdb-6.11.0-cp39-cp39-win_amd64.whl (9.2 MB view details)

Uploaded CPython 3.9Windows x86-64

arcticdb-6.11.0-cp39-cp39-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (19.5 MB view details)

Uploaded CPython 3.9manylinux: glibc 2.17+ x86-64

arcticdb-6.11.0-cp39-cp39-macosx_15_0_arm64.whl (18.0 MB view details)

Uploaded CPython 3.9macOS 15.0+ ARM64

File details

Details for the file arcticdb-6.11.0-cp314-cp314-win_amd64.whl.

File metadata

  • Download URL: arcticdb-6.11.0-cp314-cp314-win_amd64.whl
  • Upload date:
  • Size: 9.4 MB
  • Tags: CPython 3.14, Windows x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.2.0 CPython/3.10.12

File hashes

Hashes for arcticdb-6.11.0-cp314-cp314-win_amd64.whl
Algorithm Hash digest
SHA256 3ab119252744ad696714279d7252f5da8c69e4f3a11cb6d0bc3cee06b161f4b1
MD5 0c008e33f81ef1d58790e034d5ddd60a
BLAKE2b-256 870fc45a96ffa472db890e3c30b8f8cd277e9da623ad9760542c4ec80d198c9d

See more details on using hashes here.

File details

Details for the file arcticdb-6.11.0-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.whl.

File metadata

File hashes

Hashes for arcticdb-6.11.0-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.whl
Algorithm Hash digest
SHA256 4cfe5b3d08dc1437ea7dfb668269b8a9e05642f8008eca820299db33dab9f46f
MD5 fdc01612f83ef124db35744da6c25dba
BLAKE2b-256 2855e717414e492918a1c041b74f0eecba84300acd83f4e4204081c976e71d5b

See more details on using hashes here.

File details

Details for the file arcticdb-6.11.0-cp314-cp314-macosx_15_0_arm64.whl.

File metadata

File hashes

Hashes for arcticdb-6.11.0-cp314-cp314-macosx_15_0_arm64.whl
Algorithm Hash digest
SHA256 b0aafe369b093d9c52583ea30705ccf16bd656920c1270f34727c5e06c9b0aab
MD5 c349f64c4e15314679fdcf773bf0268a
BLAKE2b-256 527823a40786e2203eef77946fc016a3a6fc130633154efdec604e8b46ba4ea4

See more details on using hashes here.

File details

Details for the file arcticdb-6.11.0-cp313-cp313-win_amd64.whl.

File metadata

  • Download URL: arcticdb-6.11.0-cp313-cp313-win_amd64.whl
  • Upload date:
  • Size: 9.2 MB
  • Tags: CPython 3.13, Windows x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.2.0 CPython/3.10.12

File hashes

Hashes for arcticdb-6.11.0-cp313-cp313-win_amd64.whl
Algorithm Hash digest
SHA256 8986a5b7d28b480fc99eca94dfcc810b6ffc24f07a5a3e637f728c639fd08818
MD5 5cc15a8f5b39862ca8b48f72996571b0
BLAKE2b-256 1cf2f9c55ecd0af649779403ec2e0b9cac3b2ef647516987681b2d6bca95888e

See more details on using hashes here.

File details

Details for the file arcticdb-6.11.0-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.whl.

File metadata

File hashes

Hashes for arcticdb-6.11.0-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.whl
Algorithm Hash digest
SHA256 ce6fdf045c403fa5ecfc877cc312f117b8f73b6a451ccfea81ef290222fd4f49
MD5 9c4411873f9411f93be8ea64b56aa08f
BLAKE2b-256 ce5ed67b8223c8dbcb8a0550c5c86e2cf698f169f21bed4b1571900356fe7ffa

See more details on using hashes here.

File details

Details for the file arcticdb-6.11.0-cp313-cp313-macosx_15_0_arm64.whl.

File metadata

File hashes

Hashes for arcticdb-6.11.0-cp313-cp313-macosx_15_0_arm64.whl
Algorithm Hash digest
SHA256 f33beefb9fc5f443e2e24daaddb24a63d579424afc6b10d1f1e055ed8a3e4876
MD5 15843abfd6003638e1e94ee0b5ae3cb5
BLAKE2b-256 51edbfc3cff9b312f64802633316bb80481bbef45fea02c3861bc6eae322985f

See more details on using hashes here.

File details

Details for the file arcticdb-6.11.0-cp312-cp312-win_amd64.whl.

File metadata

  • Download URL: arcticdb-6.11.0-cp312-cp312-win_amd64.whl
  • Upload date:
  • Size: 9.2 MB
  • Tags: CPython 3.12, Windows x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.2.0 CPython/3.10.12

File hashes

Hashes for arcticdb-6.11.0-cp312-cp312-win_amd64.whl
Algorithm Hash digest
SHA256 959550e5ad916e1747290f86a9485d7d72523241bbaf89bea3376e5e6f4c7cca
MD5 369097f0285c79c911be35c1c3dd3199
BLAKE2b-256 5ef13a9dece04bcf102e36a67d342e39730731c6825fb88a81bef9f48a772cdd

See more details on using hashes here.

File details

Details for the file arcticdb-6.11.0-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.whl.

File metadata

File hashes

Hashes for arcticdb-6.11.0-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.whl
Algorithm Hash digest
SHA256 8195dd69ac00dbe6d52da7ef31c476c4a62e066a37888af3a45c51ea59b52da2
MD5 0f76ebfeb0ce62da375674e94143bfc7
BLAKE2b-256 bdb1b1dc94164421aa0cbbf03c95a5175c5d1a338b0ec916b25510443fc4d676

See more details on using hashes here.

File details

Details for the file arcticdb-6.11.0-cp312-cp312-macosx_15_0_arm64.whl.

File metadata

File hashes

Hashes for arcticdb-6.11.0-cp312-cp312-macosx_15_0_arm64.whl
Algorithm Hash digest
SHA256 5ffee0e82ee8d8d0b0ff21e4ce6ba2f8d471b10e233c38d89dd36f5e1e982d02
MD5 e38838331b17c9f8d438c1fa5423515c
BLAKE2b-256 ff8574a720e472fd5782c7a3f6c0e72aabcd22ecbc682f02664a53c6967110ad

See more details on using hashes here.

File details

Details for the file arcticdb-6.11.0-cp311-cp311-win_amd64.whl.

File metadata

  • Download URL: arcticdb-6.11.0-cp311-cp311-win_amd64.whl
  • Upload date:
  • Size: 9.2 MB
  • Tags: CPython 3.11, Windows x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.2.0 CPython/3.10.12

File hashes

Hashes for arcticdb-6.11.0-cp311-cp311-win_amd64.whl
Algorithm Hash digest
SHA256 2a2da6e9ffc3486be90fc357d06c3f4222711e3ab96d8f578bf5bedb4c6d5bb8
MD5 86358a9e74f2d02f45ade4de76598d29
BLAKE2b-256 7c05292d67a575cf4de199a70336a65e1913fc81d3de1fb171b9b3179ab30da0

See more details on using hashes here.

File details

Details for the file arcticdb-6.11.0-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.whl.

File metadata

File hashes

Hashes for arcticdb-6.11.0-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.whl
Algorithm Hash digest
SHA256 582812596218ec10d28e95545242215b07d7632420a31cc47ac7ff56e6cb81a0
MD5 5707501efa61fd6619df4e0d903a2937
BLAKE2b-256 2e3e19e5fd9db741d785b0819708a8207a34899950d74f235de4ef356d24751e

See more details on using hashes here.

File details

Details for the file arcticdb-6.11.0-cp311-cp311-macosx_15_0_arm64.whl.

File metadata

File hashes

Hashes for arcticdb-6.11.0-cp311-cp311-macosx_15_0_arm64.whl
Algorithm Hash digest
SHA256 a665c7520141b5c1084aada5f1a98c46bb0b47cb308714d5c18ed89db3515ceb
MD5 cbba91c14e1b25c57fc351d67dcacc10
BLAKE2b-256 3cc6f497734dea675b925f90009d84bfa3e5c0fa222eb68e1b56f029f4274ac7

See more details on using hashes here.

File details

Details for the file arcticdb-6.11.0-cp310-cp310-win_amd64.whl.

File metadata

  • Download URL: arcticdb-6.11.0-cp310-cp310-win_amd64.whl
  • Upload date:
  • Size: 9.2 MB
  • Tags: CPython 3.10, Windows x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.2.0 CPython/3.10.12

File hashes

Hashes for arcticdb-6.11.0-cp310-cp310-win_amd64.whl
Algorithm Hash digest
SHA256 a3a48b8e15f8fb71baff7c59a7b20b11eb0da6199858549252c91a087c6c40d3
MD5 f468e872e43dc9f8f90b372e0ac4b8c5
BLAKE2b-256 deeb2f9aff3afadf49d800711d611db244b0e38c7cfa8b4c8c84b07cf5836787

See more details on using hashes here.

File details

Details for the file arcticdb-6.11.0-cp310-cp310-manylinux2014_x86_64.manylinux_2_17_x86_64.whl.

File metadata

File hashes

Hashes for arcticdb-6.11.0-cp310-cp310-manylinux2014_x86_64.manylinux_2_17_x86_64.whl
Algorithm Hash digest
SHA256 04b031936d3a00685d0bc1df410e10640d45c6657f0d2064edb97b7e94468b57
MD5 e8a077e65ee896bc34da01fa56007e92
BLAKE2b-256 0f0c46bdbc2a18d16327ce50394bb049c30b4b396f58a342d45aa65f6baeaa3e

See more details on using hashes here.

File details

Details for the file arcticdb-6.11.0-cp310-cp310-macosx_15_0_arm64.whl.

File metadata

File hashes

Hashes for arcticdb-6.11.0-cp310-cp310-macosx_15_0_arm64.whl
Algorithm Hash digest
SHA256 b66f6ff43484653ebbe7c0d2ab18951306cc851ca24b676d355e76e61f2ecca3
MD5 b27ca0098c39d2b0646c51b2525063e1
BLAKE2b-256 3e67f75edb63802fd65ad6b94818edd7df708b4ab1c7b955b465d0a67d22346d

See more details on using hashes here.

File details

Details for the file arcticdb-6.11.0-cp39-cp39-win_amd64.whl.

File metadata

  • Download URL: arcticdb-6.11.0-cp39-cp39-win_amd64.whl
  • Upload date:
  • Size: 9.2 MB
  • Tags: CPython 3.9, Windows x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.2.0 CPython/3.10.12

File hashes

Hashes for arcticdb-6.11.0-cp39-cp39-win_amd64.whl
Algorithm Hash digest
SHA256 a4bee14cc258bf0923f5c52e02ec9ff7ecd5ea48f9ca636e507f918617616003
MD5 a272917f447c2f10f45cb9f3773fe6e8
BLAKE2b-256 218b990e6561460b89dbaa6725416ce6ccffadc439611b885b483052a91a1a0c

See more details on using hashes here.

File details

Details for the file arcticdb-6.11.0-cp39-cp39-manylinux2014_x86_64.manylinux_2_17_x86_64.whl.

File metadata

File hashes

Hashes for arcticdb-6.11.0-cp39-cp39-manylinux2014_x86_64.manylinux_2_17_x86_64.whl
Algorithm Hash digest
SHA256 d3d1356aaff8f93e1f0c4cc548e9ac42a90702fe2ba9a7e37a348e95e27d54e0
MD5 b993ed9c00645ec269359d64304b360d
BLAKE2b-256 d822b7fef85c1a0eb97cffa8d628e606942d36d12b767f678129dd5076b2c9fe

See more details on using hashes here.

File details

Details for the file arcticdb-6.11.0-cp39-cp39-macosx_15_0_arm64.whl.

File metadata

File hashes

Hashes for arcticdb-6.11.0-cp39-cp39-macosx_15_0_arm64.whl
Algorithm Hash digest
SHA256 30ecabfcda8e4758b4671496bc4607ad92e77826aec2601c4cc137bb11e459d4
MD5 167af5edfa9fad30ed774e9ce55a2c1e
BLAKE2b-256 26db62b36fe183c9ca519caee98b2fbcb9283164e4e9619954bd34ff594a460e

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page