Skip to main content

ArcticDB DataFrame Database

Project description


Three minute ArcticDB demo from PyQuantNews

Options data in ArcticDB

ArcticDB Website | 📘 ArcticDB Docs | 📰: ArcticDB Blog | Press Release | Press Release | Community


ArcticDB is a high performance, serverless DataFrame database built for the Python Data Science ecosystem. Launched in March 2023, it is the successor to Arctic.

Use of ArcticDB in production (including business or commercial environments) or for a Database Service requires a paid for license from ArcticDB Limited . Please contact info@arcticdb.io for further details.

ArcticDB offers an intuitive Python-centric API enabling you to read and write Pandas DataFrames to S3 or LMDB utilising a fast C++ data-processing and compression engine.

ArcticDB allows you to:

  • Pandas in, Pandas out: Read and write Pandas DataFrames, NumPy arrays and native types to S3 and LMDB without leaving Python.
  • Built for time-series data: Efficiently index and query time-series data across billions of rows
  • Time travel: Travel back in time to see previous versions of your data and create customizable snapshots of the database
  • Schemaless Database: Append, update and modify data without being constrained by the existing schema
  • Optimised for streaming data: Built in support for efficient sparse data storage
  • Powerful processing: Filter, aggregate and create new columns on-the-fly with a Pandas-like syntax
  • C++ efficiency: Accelerate analytics though concurrency in the C++ data-processing engine

ArcticDB handles data that is big in both row count and column count, so a 20-year history of more than 400,000 unique securities can be stored in a single symbol. Each symbol is maintained as a separate entity with no shared data which means ArcticDB can scale horizontally across symbols, maximising the performance potential of your compute, storage and network.

ArcticDB is designed from the outset to be resilient; there is no single point of failure, and persistent data structures in the storage mean that once a version of a symbol has been written, it can never be corrupted by subsequent updates. Pulling compressed data directly from storage to the client means that there is no server to overload, so your data is always available when you need it.

Quickstart

Prebuilt binary availability (for the current version)

PyPI (Python 3.9 - 3.14) conda-forge (Python 3.10 - 3.14)
Linux x86_64 ✔️ ✔️
Linux arm64 ✔️
Windows x86_64 ✔️ ✔️
MacOS x86_64 ✔️
MacOS arm64 ✔️ ✔️

Storage compatibility

Linux Windows Mac
S3 ✔️ ✔️ ✔️
LMDB ✔️ ✔️ ✔️
Azure Blob Storage ✔️ ✔️ ✔️

We have tested against the following S3 backends:

  • AWS S3
  • Ceph
  • MinIO on Linux
  • Pure Storage S3
  • Scality S3
  • VAST Data S3

Installation

Install ArcticDB:

$ pip install arcticdb

or using conda-forge

$ conda install -c conda-forge arcticdb

Import ArcticDB:

>>> import arcticdb as adb

Create an instance on your S3 storage (with or without explicit credentials):

# Leave AWS to derive credential information
>>> ac = adb.Arctic('s3://MY_ENDPOINT:MY_BUCKET?aws_auth=true')

# Manually specify creds
>>> ac = adb.Arctic('s3://MY_ENDPOINT:MY_BUCKET?region=YOUR_REGION&access=ABCD&secret=DCBA')

Or create an instance on your local disk:

>>> ac = adb.Arctic("lmdb:///<path>")

Create your first library and list the libraries in the instance:

>>> ac.create_library('travel_data')
>>> ac.list_libraries()

Create a test dataframe:

>>> import numpy as np
>>> import pandas as pd
>>> NUM_COLUMNS=10
>>> NUM_ROWS=100_000
>>> df = pd.DataFrame(np.random.randint(0,100,size=(NUM_ROWS, NUM_COLUMNS)), columns=[f"COL_{i}" for i in range(NUM_COLUMNS)], index=pd.date_range('2000', periods=NUM_ROWS, freq='h'))

Get the library, write some data to it, and read it back:

>>> lib = ac['travel_data']
>>> lib.write("my_data", df)
>>> data = lib.read("my_data")

To find out more about working with data, visit our docs


Documentation

The source code for the ArcticDB docs are located in the docs folder, and are hosted at docs.arcticdb.io.

License

ArcticDB is released under a Business Source License 1.1 (BSL)

BSL features are free to use and the source code is available, but users may not use ArcticDB for production use or for a Database Service, without agreement with Man Group Operations Limited.

Use of ArcticDB in production or for a Database Service requires a paid for license from ArcticDB Limited and is licensed under the ArcticDB Software License Agreement. For more information please contact info@arcticdb.io.

The BSL is not certified as an open-source license, but most of the Open Source Initiative (OSI) criteria are met. Please see version conversion dates in the below table:

ArcticDB Version License Converts to Apache 2.0
1.0 Business Source License 1.1 Mar 16, 2025
1.2 Business Source License 1.1 May 22, 2025
1.3 Business Source License 1.1 Jun 9, 2025
1.4 Business Source License 1.1 Jun 23, 2025
1.5 Business Source License 1.1 Jul 11, 2025
1.6 Business Source License 1.1 Jul 25, 2025
2.0 Business Source License 1.1 Aug 29, 2025
3.0 Business Source License 1.1 Sep 13, 2025
4.0 Business Source License 1.1 Sep 27, 2025
4.1 Business Source License 1.1 Nov 1, 2025
4.2 Business Source License 1.1 Nov 12, 2025
4.3 Business Source License 1.1 Feb 7, 2026
4.4 Business Source License 1.1 Apr 5, 2026
4.5 Business Source License 1.1 Aug 14, 2026
5.0 Business Source License 1.1 Oct 31, 2026
5.1 Business Source License 1.1 Nov 15, 2026
5.2 Business Source License 1.1 Jan 27, 2027
5.3 Business Source License 1.1 Mar 24, 2027
5.4 Business Source License 1.1 Apr 28, 2027
5.5 Business Source License 1.1 May 13, 2027
5.6 Business Source License 1.1 May 26, 2027
5.7 Business Source License 1.1 Jun 2, 2027
5.8 Business Source License 1.1 Jun 9, 2027
5.9 Business Source License 1.1 Jul 7, 2027
5.10 Business Source License 1.1 Jul 29, 2027
6.1 Business Source License 1.1 Aug 20, 2027
6.2 Business Source License 1.1 Sep 8, 2027
6.3 Business Source License 1.1 Nov 3, 2027
6.5 Business Source License 1.1 Dec 11, 2027
6.6 Business Source License 1.1 Jan 6, 2028

Code of Conduct

Code of Conduct

This project has adopted a Code of Conduct. If you have any concerns about the Code, or behaviour that you have experienced in the project, please contact us at info@arcticdb.io.

Contributing/Building From Source

We welcome your contributions to help us improve and extend this project!

Please refer to the Contributing page and feel free to open issues on GitHub.

We are also always looking for feedback from our dedicated community! If you have used ArcticDB please let us know, we would love to hear about your experience!

Our release process is documented here.

Community

We would love to hear how your ArcticDB journey evolves, email us at info@arcticdb.io or come chat to us on Twitter!

Interested in learning more about ArcticDB? Head over to our blog!

Do you have any questions or issues? Chat to us and other users through our dedicated Slack Workspace - sign up for Slack access on our website.

Project details


Release history Release notifications | RSS feed

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distributions

No source distribution files available for this release.See tutorial on generating distribution archives.

Built Distributions

If you're not sure about the file name format, learn more about wheel file names.

arcticdb-6.18.1-cp314-cp314-win_amd64.whl (9.6 MB view details)

Uploaded CPython 3.14Windows x86-64

arcticdb-6.18.1-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (20.7 MB view details)

Uploaded CPython 3.14manylinux: glibc 2.17+ x86-64

arcticdb-6.18.1-cp314-cp314-macosx_15_0_arm64.whl (19.0 MB view details)

Uploaded CPython 3.14macOS 15.0+ ARM64

arcticdb-6.18.1-cp313-cp313-win_amd64.whl (9.4 MB view details)

Uploaded CPython 3.13Windows x86-64

arcticdb-6.18.1-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (20.7 MB view details)

Uploaded CPython 3.13manylinux: glibc 2.17+ x86-64

arcticdb-6.18.1-cp313-cp313-macosx_15_0_arm64.whl (19.1 MB view details)

Uploaded CPython 3.13macOS 15.0+ ARM64

arcticdb-6.18.1-cp312-cp312-win_amd64.whl (9.4 MB view details)

Uploaded CPython 3.12Windows x86-64

arcticdb-6.18.1-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (20.7 MB view details)

Uploaded CPython 3.12manylinux: glibc 2.17+ x86-64

arcticdb-6.18.1-cp312-cp312-macosx_15_0_arm64.whl (19.1 MB view details)

Uploaded CPython 3.12macOS 15.0+ ARM64

arcticdb-6.18.1-cp311-cp311-win_amd64.whl (9.4 MB view details)

Uploaded CPython 3.11Windows x86-64

arcticdb-6.18.1-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (20.7 MB view details)

Uploaded CPython 3.11manylinux: glibc 2.17+ x86-64

arcticdb-6.18.1-cp311-cp311-macosx_15_0_arm64.whl (19.1 MB view details)

Uploaded CPython 3.11macOS 15.0+ ARM64

arcticdb-6.18.1-cp310-cp310-win_amd64.whl (9.4 MB view details)

Uploaded CPython 3.10Windows x86-64

arcticdb-6.18.1-cp310-cp310-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (20.8 MB view details)

Uploaded CPython 3.10manylinux: glibc 2.17+ x86-64

arcticdb-6.18.1-cp310-cp310-macosx_15_0_arm64.whl (19.1 MB view details)

Uploaded CPython 3.10macOS 15.0+ ARM64

arcticdb-6.18.1-cp39-cp39-win_amd64.whl (9.4 MB view details)

Uploaded CPython 3.9Windows x86-64

arcticdb-6.18.1-cp39-cp39-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (20.8 MB view details)

Uploaded CPython 3.9manylinux: glibc 2.17+ x86-64

arcticdb-6.18.1-cp39-cp39-macosx_15_0_arm64.whl (19.1 MB view details)

Uploaded CPython 3.9macOS 15.0+ ARM64

File details

Details for the file arcticdb-6.18.1-cp314-cp314-win_amd64.whl.

File metadata

  • Download URL: arcticdb-6.18.1-cp314-cp314-win_amd64.whl
  • Upload date:
  • Size: 9.6 MB
  • Tags: CPython 3.14, Windows x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.2.0 CPython/3.10.12

File hashes

Hashes for arcticdb-6.18.1-cp314-cp314-win_amd64.whl
Algorithm Hash digest
SHA256 6f08d813b2fceaecb3b855a57f8ad551a21e48590c0b419e8c42330253830bbe
MD5 c8b59288e24dcc10a16a0ca600632341
BLAKE2b-256 507d8c64aafd238d840cac34126d15a7662b110c99df9c38ad4f31d1cb1ead25

See more details on using hashes here.

File details

Details for the file arcticdb-6.18.1-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.whl.

File metadata

File hashes

Hashes for arcticdb-6.18.1-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.whl
Algorithm Hash digest
SHA256 27888ddc108f615b79b1f9de3fd059cdcc6625034cd655cdcdd58ef784cd5adf
MD5 e7a509dbbb78885585a9538fc2a05cb1
BLAKE2b-256 30d64dc01973fa657c56baa3886e073f1d77ee7f9154388e11c8167d4609ba52

See more details on using hashes here.

File details

Details for the file arcticdb-6.18.1-cp314-cp314-macosx_15_0_arm64.whl.

File metadata

File hashes

Hashes for arcticdb-6.18.1-cp314-cp314-macosx_15_0_arm64.whl
Algorithm Hash digest
SHA256 ffcdde95d5870d0f15a833ab716936c4c06ae7250f5b51d60beacd5432455137
MD5 50f67030381b3549316c1f8f9d2fe28a
BLAKE2b-256 223c4cfb623870a15f480fa0af0c6b38d16817dd76f44a692fb8b93adc4ffef6

See more details on using hashes here.

File details

Details for the file arcticdb-6.18.1-cp313-cp313-win_amd64.whl.

File metadata

  • Download URL: arcticdb-6.18.1-cp313-cp313-win_amd64.whl
  • Upload date:
  • Size: 9.4 MB
  • Tags: CPython 3.13, Windows x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.2.0 CPython/3.10.12

File hashes

Hashes for arcticdb-6.18.1-cp313-cp313-win_amd64.whl
Algorithm Hash digest
SHA256 ff28f96a1e6e0037d0f9781b24c703280bb1904327bd30ac40b22f5449a78a40
MD5 fa101b4707279c68114075534a299859
BLAKE2b-256 da4064b7e524c2cacf336e05ebfaa1e8642730fd7578f644c576156f6a30efa3

See more details on using hashes here.

File details

Details for the file arcticdb-6.18.1-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.whl.

File metadata

File hashes

Hashes for arcticdb-6.18.1-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.whl
Algorithm Hash digest
SHA256 bd798e59b6a0a1fe71e9006f3eb0ab054866cc043b688ad462ede03ce4b7c5e8
MD5 e343d7d038c25ce509d830779a680410
BLAKE2b-256 0c7e91d0791a4beb040d2411e8a633f8de59b3e323eeb40282e0d769857ca87c

See more details on using hashes here.

File details

Details for the file arcticdb-6.18.1-cp313-cp313-macosx_15_0_arm64.whl.

File metadata

File hashes

Hashes for arcticdb-6.18.1-cp313-cp313-macosx_15_0_arm64.whl
Algorithm Hash digest
SHA256 c4e130d0ea8fb0fd972fe39f02e520998ff16f9efadd771c8291b092da697a05
MD5 f1c06b111862dca4c9dd60e724d9516f
BLAKE2b-256 ce70aa31474c3cb81df793c2ebaf8474f2f4a7101a198b0f5c57d6b22b4874f1

See more details on using hashes here.

File details

Details for the file arcticdb-6.18.1-cp312-cp312-win_amd64.whl.

File metadata

  • Download URL: arcticdb-6.18.1-cp312-cp312-win_amd64.whl
  • Upload date:
  • Size: 9.4 MB
  • Tags: CPython 3.12, Windows x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.2.0 CPython/3.10.12

File hashes

Hashes for arcticdb-6.18.1-cp312-cp312-win_amd64.whl
Algorithm Hash digest
SHA256 d30dcf34e2f8554a5c4e251bb9f29d013a0c55d902be5ccc9489419fb0ae853e
MD5 f5cfa4e35d6bfebdd15b38de91bdfc65
BLAKE2b-256 1e50062ebd1eee6b10117bf4f6cb57123cd2240b7f1e2466f57052bf0cf792ac

See more details on using hashes here.

File details

Details for the file arcticdb-6.18.1-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.whl.

File metadata

File hashes

Hashes for arcticdb-6.18.1-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.whl
Algorithm Hash digest
SHA256 98367010c34e4f178b842b7c99ef8e576f04da7238b7ba102636ef4963e41069
MD5 6aecfc4299b0047dd7a96a47a77821bb
BLAKE2b-256 978cb42f0b65fb57c0df660e00676cd6c4ba5cc39bdc9e7c18522a536d3471ef

See more details on using hashes here.

File details

Details for the file arcticdb-6.18.1-cp312-cp312-macosx_15_0_arm64.whl.

File metadata

File hashes

Hashes for arcticdb-6.18.1-cp312-cp312-macosx_15_0_arm64.whl
Algorithm Hash digest
SHA256 6290160de48a71c3c9d07c921973f81408ce2ddab9874315e1661522a3857467
MD5 df65fe03386b2432fce36a6b9ced245d
BLAKE2b-256 8fc1b2c9b8250fbf797d4ae89a01a0c7d0927f92638143b2702a95f22be7d80c

See more details on using hashes here.

File details

Details for the file arcticdb-6.18.1-cp311-cp311-win_amd64.whl.

File metadata

  • Download URL: arcticdb-6.18.1-cp311-cp311-win_amd64.whl
  • Upload date:
  • Size: 9.4 MB
  • Tags: CPython 3.11, Windows x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.2.0 CPython/3.10.12

File hashes

Hashes for arcticdb-6.18.1-cp311-cp311-win_amd64.whl
Algorithm Hash digest
SHA256 0010e979c46de6e945491a70af42f42327f2a44d2dfa34c88173049f3e295ac6
MD5 bdb6c399dd44fc5a2686ff88132a6f69
BLAKE2b-256 8e5f4caa254765509631f3abf2e72c588672bacefa57031eeafb7a72d8f9da5c

See more details on using hashes here.

File details

Details for the file arcticdb-6.18.1-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.whl.

File metadata

File hashes

Hashes for arcticdb-6.18.1-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.whl
Algorithm Hash digest
SHA256 75553d97bce0e13bed3b2f7473135b00309f1159af032e6dd36ba8497d1c476c
MD5 534bbee0968fd59835572cdafd16f4f7
BLAKE2b-256 1cb05ccd77d1daa267bab8042742e6c7f4d204bc6495c25a9472d4545f73bb73

See more details on using hashes here.

File details

Details for the file arcticdb-6.18.1-cp311-cp311-macosx_15_0_arm64.whl.

File metadata

File hashes

Hashes for arcticdb-6.18.1-cp311-cp311-macosx_15_0_arm64.whl
Algorithm Hash digest
SHA256 ab0e90493b803f12db0886c8ef65cf40ec64dbe0597b17a90f1efb114a7c5f79
MD5 6346d043af28e570d43f5fa56dab66b3
BLAKE2b-256 eac04a65cb936258ddda3abb68b8a371320cf0736a13e5baafb8c52daf8c57e2

See more details on using hashes here.

File details

Details for the file arcticdb-6.18.1-cp310-cp310-win_amd64.whl.

File metadata

  • Download URL: arcticdb-6.18.1-cp310-cp310-win_amd64.whl
  • Upload date:
  • Size: 9.4 MB
  • Tags: CPython 3.10, Windows x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.2.0 CPython/3.10.12

File hashes

Hashes for arcticdb-6.18.1-cp310-cp310-win_amd64.whl
Algorithm Hash digest
SHA256 c357ab85e8d52aa3bcd42a7356eb6c0eb3ddf575f864620f747d005c7d2d24a1
MD5 14e26b68f06b4fb7bf9076959f26d81c
BLAKE2b-256 93cb36817ec0a153226270b22493db341b9c30e432c1dcf4e526dda7f4248cf6

See more details on using hashes here.

File details

Details for the file arcticdb-6.18.1-cp310-cp310-manylinux2014_x86_64.manylinux_2_17_x86_64.whl.

File metadata

File hashes

Hashes for arcticdb-6.18.1-cp310-cp310-manylinux2014_x86_64.manylinux_2_17_x86_64.whl
Algorithm Hash digest
SHA256 b93e3c168832507b4b763b2d2a78be6ac8015753936a070749749078316cecb0
MD5 c83f8456ebf40f7272c9eae79693a5b1
BLAKE2b-256 f2ab97d2cbfe1690fd5575d3826511f87ac6af3ac861cdd8b1513366c33d96ce

See more details on using hashes here.

File details

Details for the file arcticdb-6.18.1-cp310-cp310-macosx_15_0_arm64.whl.

File metadata

File hashes

Hashes for arcticdb-6.18.1-cp310-cp310-macosx_15_0_arm64.whl
Algorithm Hash digest
SHA256 9086e5c4f7bfddbcfe4f9a5298a54c953f46d083ee55e0676552165e323409c1
MD5 70d5f64e78a1ad770e85fdad67939476
BLAKE2b-256 ade109a974243c5075126ae5e64c147ae854651c16c0ef56eae889a0ae71b783

See more details on using hashes here.

File details

Details for the file arcticdb-6.18.1-cp39-cp39-win_amd64.whl.

File metadata

  • Download URL: arcticdb-6.18.1-cp39-cp39-win_amd64.whl
  • Upload date:
  • Size: 9.4 MB
  • Tags: CPython 3.9, Windows x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.2.0 CPython/3.10.12

File hashes

Hashes for arcticdb-6.18.1-cp39-cp39-win_amd64.whl
Algorithm Hash digest
SHA256 ae8e14987dd1e052e076ff3679e224bbbfd8efc525f1f26ca5225e006d291f99
MD5 1c7ce1746abaa2b2e5d9c52ccceca859
BLAKE2b-256 34751097cab8190aa8f5f3b035d1f8f4d5aa7073f0ba534ae6eacf5a15d77f4a

See more details on using hashes here.

File details

Details for the file arcticdb-6.18.1-cp39-cp39-manylinux2014_x86_64.manylinux_2_17_x86_64.whl.

File metadata

File hashes

Hashes for arcticdb-6.18.1-cp39-cp39-manylinux2014_x86_64.manylinux_2_17_x86_64.whl
Algorithm Hash digest
SHA256 0306604e6ad69c5195eb1adfa6e8fe2c610d5a8d72ed52f8d9df41513de4f731
MD5 f30603e2aed9146d24286955af85da25
BLAKE2b-256 b339bdb433240f54154f30294ab2e5d54bdb466a991aeddad66b5ff122d5bd48

See more details on using hashes here.

File details

Details for the file arcticdb-6.18.1-cp39-cp39-macosx_15_0_arm64.whl.

File metadata

File hashes

Hashes for arcticdb-6.18.1-cp39-cp39-macosx_15_0_arm64.whl
Algorithm Hash digest
SHA256 d55c386ab12cd9445187db1f0d67bc48e4245ba8efb79b670688404411711f20
MD5 3dc0a066ece04368a1f1fcb808ce2a96
BLAKE2b-256 dda9d315ffb5170d3c3709eb9c667de1119ff1856ce0580cd3e91e42ae95cc0c

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page