Skip to main content

ArcticDB DataFrame Database

Project description


Three minute ArcticDB demo from PyQuantNews

Options data in ArcticDB

ArcticDB Website | 📘 ArcticDB Docs | 📰: ArcticDB Blog | Press Release | Press Release | Community


ArcticDB is a high performance, serverless DataFrame database built for the Python Data Science ecosystem. Launched in March 2023, it is the successor to Arctic.

Use of ArcticDB in production (including business or commercial environments) or for a Database Service requires a paid for license from ArcticDB Limited . Please contact info@arcticdb.io for further details.

ArcticDB offers an intuitive Python-centric API enabling you to read and write Pandas DataFrames to S3 or LMDB utilising a fast C++ data-processing and compression engine.

ArcticDB allows you to:

  • Pandas in, Pandas out: Read and write Pandas DataFrames, NumPy arrays and native types to S3 and LMDB without leaving Python.
  • Built for time-series data: Efficiently index and query time-series data across billions of rows
  • Time travel: Travel back in time to see previous versions of your data and create customizable snapshots of the database
  • Schemaless Database: Append, update and modify data without being constrained by the existing schema
  • Optimised for streaming data: Built in support for efficient sparse data storage
  • Powerful processing: Filter, aggregate and create new columns on-the-fly with a Pandas-like syntax
  • C++ efficiency: Accelerate analytics though concurrency in the C++ data-processing engine

ArcticDB handles data that is big in both row count and column count, so a 20-year history of more than 400,000 unique securities can be stored in a single symbol. Each symbol is maintained as a separate entity with no shared data which means ArcticDB can scale horizontally across symbols, maximising the performance potential of your compute, storage and network.

ArcticDB is designed from the outset to be resilient; there is no single point of failure, and persistent data structures in the storage mean that once a version of a symbol has been written, it can never be corrupted by subsequent updates. Pulling compressed data directly from storage to the client means that there is no server to overload, so your data is always available when you need it.

Quickstart

Prebuilt binary availability (for the current version)

PyPI (Python 3.9 - 3.14) conda-forge (Python 3.10 - 3.14)
Linux x86_64 ✔️ ✔️
Linux arm64 ✔️
Windows x86_64 ✔️ ✔️
MacOS x86_64 ✔️
MacOS arm64 ✔️ ✔️

Storage compatibility

Linux Windows Mac
S3 ✔️ ✔️ ✔️
LMDB ✔️ ✔️ ✔️
Azure Blob Storage ✔️ ✔️ ✔️

We have tested against the following S3 backends:

  • AWS S3
  • Ceph
  • MinIO on Linux
  • Pure Storage S3
  • Scality S3
  • VAST Data S3

Installation

Install ArcticDB:

$ pip install arcticdb

or using conda-forge

$ conda install -c conda-forge arcticdb

Import ArcticDB:

>>> import arcticdb as adb

Create an instance on your S3 storage (with or without explicit credentials):

# Leave AWS to derive credential information
>>> ac = adb.Arctic('s3://MY_ENDPOINT:MY_BUCKET?aws_auth=true')

# Manually specify creds
>>> ac = adb.Arctic('s3://MY_ENDPOINT:MY_BUCKET?region=YOUR_REGION&access=ABCD&secret=DCBA')

Or create an instance on your local disk:

>>> ac = adb.Arctic("lmdb:///<path>")

Create your first library and list the libraries in the instance:

>>> ac.create_library('travel_data')
>>> ac.list_libraries()

Create a test dataframe:

>>> import numpy as np
>>> import pandas as pd
>>> NUM_COLUMNS=10
>>> NUM_ROWS=100_000
>>> df = pd.DataFrame(np.random.randint(0,100,size=(NUM_ROWS, NUM_COLUMNS)), columns=[f"COL_{i}" for i in range(NUM_COLUMNS)], index=pd.date_range('2000', periods=NUM_ROWS, freq='h'))

Get the library, write some data to it, and read it back:

>>> lib = ac['travel_data']
>>> lib.write("my_data", df)
>>> data = lib.read("my_data")

To find out more about working with data, visit our docs


Documentation

The source code for the ArcticDB docs are located in the docs folder, and are hosted at docs.arcticdb.io.

License

ArcticDB is released under a Business Source License 1.1 (BSL)

BSL features are free to use and the source code is available, but users may not use ArcticDB for production use or for a Database Service, without agreement with Man Group Operations Limited.

Use of ArcticDB in production or for a Database Service requires a paid for license from ArcticDB Limited and is licensed under the ArcticDB Software License Agreement. For more information please contact info@arcticdb.io.

The BSL is not certified as an open-source license, but most of the Open Source Initiative (OSI) criteria are met. Please see version conversion dates in the below table:

ArcticDB Version License Converts to Apache 2.0
1.0 Business Source License 1.1 Mar 16, 2025
1.2 Business Source License 1.1 May 22, 2025
1.3 Business Source License 1.1 Jun 9, 2025
1.4 Business Source License 1.1 Jun 23, 2025
1.5 Business Source License 1.1 Jul 11, 2025
1.6 Business Source License 1.1 Jul 25, 2025
2.0 Business Source License 1.1 Aug 29, 2025
3.0 Business Source License 1.1 Sep 13, 2025
4.0 Business Source License 1.1 Sep 27, 2025
4.1 Business Source License 1.1 Nov 1, 2025
4.2 Business Source License 1.1 Nov 12, 2025
4.3 Business Source License 1.1 Feb 7, 2026
4.4 Business Source License 1.1 Apr 5, 2026
4.5 Business Source License 1.1 Aug 14, 2026
5.0 Business Source License 1.1 Oct 31, 2026
5.1 Business Source License 1.1 Nov 15, 2026
5.2 Business Source License 1.1 Jan 27, 2027
5.3 Business Source License 1.1 Mar 24, 2027
5.4 Business Source License 1.1 Apr 28, 2027
5.5 Business Source License 1.1 May 13, 2027
5.6 Business Source License 1.1 May 26, 2027
5.7 Business Source License 1.1 Jun 2, 2027
5.8 Business Source License 1.1 Jun 9, 2027
5.9 Business Source License 1.1 Jul 7, 2027
5.10 Business Source License 1.1 Jul 29, 2027
6.1 Business Source License 1.1 Aug 20, 2027
6.2 Business Source License 1.1 Sep 8, 2027
6.3 Business Source License 1.1 Nov 3, 2027
6.5 Business Source License 1.1 Dec 11, 2027
6.6 Business Source License 1.1 Jan 6, 2028

Code of Conduct

Code of Conduct

This project has adopted a Code of Conduct. If you have any concerns about the Code, or behaviour that you have experienced in the project, please contact us at info@arcticdb.io.

Contributing/Building From Source

We welcome your contributions to help us improve and extend this project!

Please refer to the Contributing page and feel free to open issues on GitHub.

We are also always looking for feedback from our dedicated community! If you have used ArcticDB please let us know, we would love to hear about your experience!

Our release process is documented here.

Community

We would love to hear how your ArcticDB journey evolves, email us at info@arcticdb.io or come chat to us on Twitter!

Interested in learning more about ArcticDB? Head over to our blog!

Do you have any questions or issues? Chat to us and other users through our dedicated Slack Workspace - sign up for Slack access on our website.

Project details


Release history Release notifications | RSS feed

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distributions

No source distribution files available for this release.See tutorial on generating distribution archives.

Built Distributions

If you're not sure about the file name format, learn more about wheel file names.

arcticdb-6.17.0-cp314-cp314-win_amd64.whl (9.7 MB view details)

Uploaded CPython 3.14Windows x86-64

arcticdb-6.17.0-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (20.1 MB view details)

Uploaded CPython 3.14manylinux: glibc 2.17+ x86-64

arcticdb-6.17.0-cp314-cp314-macosx_15_0_arm64.whl (18.7 MB view details)

Uploaded CPython 3.14macOS 15.0+ ARM64

arcticdb-6.17.0-cp313-cp313-win_amd64.whl (9.5 MB view details)

Uploaded CPython 3.13Windows x86-64

arcticdb-6.17.0-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (20.1 MB view details)

Uploaded CPython 3.13manylinux: glibc 2.17+ x86-64

arcticdb-6.17.0-cp313-cp313-macosx_15_0_arm64.whl (18.7 MB view details)

Uploaded CPython 3.13macOS 15.0+ ARM64

arcticdb-6.17.0-cp312-cp312-win_amd64.whl (9.5 MB view details)

Uploaded CPython 3.12Windows x86-64

arcticdb-6.17.0-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (20.1 MB view details)

Uploaded CPython 3.12manylinux: glibc 2.17+ x86-64

arcticdb-6.17.0-cp312-cp312-macosx_15_0_arm64.whl (18.7 MB view details)

Uploaded CPython 3.12macOS 15.0+ ARM64

arcticdb-6.17.0-cp311-cp311-win_amd64.whl (9.5 MB view details)

Uploaded CPython 3.11Windows x86-64

arcticdb-6.17.0-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (20.2 MB view details)

Uploaded CPython 3.11manylinux: glibc 2.17+ x86-64

arcticdb-6.17.0-cp311-cp311-macosx_15_0_arm64.whl (18.7 MB view details)

Uploaded CPython 3.11macOS 15.0+ ARM64

arcticdb-6.17.0-cp310-cp310-win_amd64.whl (9.5 MB view details)

Uploaded CPython 3.10Windows x86-64

arcticdb-6.17.0-cp310-cp310-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (20.2 MB view details)

Uploaded CPython 3.10manylinux: glibc 2.17+ x86-64

arcticdb-6.17.0-cp310-cp310-macosx_15_0_arm64.whl (18.7 MB view details)

Uploaded CPython 3.10macOS 15.0+ ARM64

arcticdb-6.17.0-cp39-cp39-win_amd64.whl (9.5 MB view details)

Uploaded CPython 3.9Windows x86-64

arcticdb-6.17.0-cp39-cp39-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (20.1 MB view details)

Uploaded CPython 3.9manylinux: glibc 2.17+ x86-64

arcticdb-6.17.0-cp39-cp39-macosx_15_0_arm64.whl (18.7 MB view details)

Uploaded CPython 3.9macOS 15.0+ ARM64

File details

Details for the file arcticdb-6.17.0-cp314-cp314-win_amd64.whl.

File metadata

  • Download URL: arcticdb-6.17.0-cp314-cp314-win_amd64.whl
  • Upload date:
  • Size: 9.7 MB
  • Tags: CPython 3.14, Windows x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.2.0 CPython/3.10.12

File hashes

Hashes for arcticdb-6.17.0-cp314-cp314-win_amd64.whl
Algorithm Hash digest
SHA256 fa225ad3422b0b2406b70d2018073236d318a71968babb5c27298477934de837
MD5 6f81190a1f6a6e550026b803fe5be6e2
BLAKE2b-256 de6f879bb9c2280596ec1182949ce241c6f8eff5d1620306a9ae14aee58c2e8b

See more details on using hashes here.

File details

Details for the file arcticdb-6.17.0-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.whl.

File metadata

File hashes

Hashes for arcticdb-6.17.0-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.whl
Algorithm Hash digest
SHA256 e614287b570652562e82f318759e8e6ba9e77eb9e519722e7f671cf5a3ede509
MD5 423b6bd1be56d4fed5d90715b2fc6f42
BLAKE2b-256 4f90a2a5b63836783fd069541ceea53721254c6954188eadc7d913e97052cf11

See more details on using hashes here.

File details

Details for the file arcticdb-6.17.0-cp314-cp314-macosx_15_0_arm64.whl.

File metadata

File hashes

Hashes for arcticdb-6.17.0-cp314-cp314-macosx_15_0_arm64.whl
Algorithm Hash digest
SHA256 105c836a2ec87db644a65c72371656f6b1437339ddc23f2f247ac964ce16ee41
MD5 080b6b614f06264dbdc9509113948a65
BLAKE2b-256 fef42df7dae8b79f104d9693c09f65258a03493b8842b31da285e839661ea58e

See more details on using hashes here.

File details

Details for the file arcticdb-6.17.0-cp313-cp313-win_amd64.whl.

File metadata

  • Download URL: arcticdb-6.17.0-cp313-cp313-win_amd64.whl
  • Upload date:
  • Size: 9.5 MB
  • Tags: CPython 3.13, Windows x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.2.0 CPython/3.10.12

File hashes

Hashes for arcticdb-6.17.0-cp313-cp313-win_amd64.whl
Algorithm Hash digest
SHA256 4a1acfccae1214a9f3db78cb3a357ef01c480587627d8ff5f3f1541687f748ed
MD5 afed3116caa851884a240c646a9ce632
BLAKE2b-256 bce426a8dba5c7f08860dec4a58be447a4993a6def6db55d00d65568239d0a92

See more details on using hashes here.

File details

Details for the file arcticdb-6.17.0-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.whl.

File metadata

File hashes

Hashes for arcticdb-6.17.0-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.whl
Algorithm Hash digest
SHA256 732ffa0a35e48a0839fef98aad0f8a94d60e00c4c6bb7737c9b315973f2645dc
MD5 f453ea79461823e51760312a74392eb9
BLAKE2b-256 dd43db688f8ed946863c0923f65530333a2114da267ef272604a685c0205f946

See more details on using hashes here.

File details

Details for the file arcticdb-6.17.0-cp313-cp313-macosx_15_0_arm64.whl.

File metadata

File hashes

Hashes for arcticdb-6.17.0-cp313-cp313-macosx_15_0_arm64.whl
Algorithm Hash digest
SHA256 701e1b64c198e90ccf42e9dec66f3bcad7e1ae1e9a15ccf917326aff7e1a645e
MD5 404fb642796c4fc467093fdfe378745f
BLAKE2b-256 60b2e305add4fb85867095bb3221ef951e350d672ebe725bb6ff3e7a00415747

See more details on using hashes here.

File details

Details for the file arcticdb-6.17.0-cp312-cp312-win_amd64.whl.

File metadata

  • Download URL: arcticdb-6.17.0-cp312-cp312-win_amd64.whl
  • Upload date:
  • Size: 9.5 MB
  • Tags: CPython 3.12, Windows x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.2.0 CPython/3.10.12

File hashes

Hashes for arcticdb-6.17.0-cp312-cp312-win_amd64.whl
Algorithm Hash digest
SHA256 22bf1e23dc1a86d20021d0a88836df122ba4e05281a86cfb0aa9b52d9ab7d340
MD5 fed079d0676dc77804a489a197b62b1c
BLAKE2b-256 68e56c48aae14c340107237415754ed6f843ce8401b91d6c19edfbef88b6f2ae

See more details on using hashes here.

File details

Details for the file arcticdb-6.17.0-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.whl.

File metadata

File hashes

Hashes for arcticdb-6.17.0-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.whl
Algorithm Hash digest
SHA256 59031d2990f91eca1680d17e69bf07b276d48ada5aa7da326aaf1ee0781062f6
MD5 8b7f9321acc734b4ded233301a915ed5
BLAKE2b-256 a2d1ae1bc26f460515e19508a9c875eaf7954975802546876bff01f327b546b3

See more details on using hashes here.

File details

Details for the file arcticdb-6.17.0-cp312-cp312-macosx_15_0_arm64.whl.

File metadata

File hashes

Hashes for arcticdb-6.17.0-cp312-cp312-macosx_15_0_arm64.whl
Algorithm Hash digest
SHA256 d69a6ec14dd0eb8ba99828909af04f2d9dd519ec37079a763824bb62dd6c45cf
MD5 fe2d2270e15f862a0a4d0240e1e06ffd
BLAKE2b-256 3abd4971b99b752b55f9aa5dabc4bd2bf91047efbde00fd1c4b9730c257decde

See more details on using hashes here.

File details

Details for the file arcticdb-6.17.0-cp311-cp311-win_amd64.whl.

File metadata

  • Download URL: arcticdb-6.17.0-cp311-cp311-win_amd64.whl
  • Upload date:
  • Size: 9.5 MB
  • Tags: CPython 3.11, Windows x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.2.0 CPython/3.10.12

File hashes

Hashes for arcticdb-6.17.0-cp311-cp311-win_amd64.whl
Algorithm Hash digest
SHA256 2fc5f4b79f2f0be49a0f5d6e25761d10dff2c75a4016bb443e6f406580e364d9
MD5 a4d9b3c26bc56f1c55b15d0b92939206
BLAKE2b-256 8d86b60b879e150338ee096a6af4cb03340d6a1cb636670cdbe50842bee8fd32

See more details on using hashes here.

File details

Details for the file arcticdb-6.17.0-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.whl.

File metadata

File hashes

Hashes for arcticdb-6.17.0-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.whl
Algorithm Hash digest
SHA256 5216171e93e304bfd808fd7301309055ac6ea8efdf57cd3836ec139d31c3525a
MD5 6bb6960e6b2cdad327ae1ab7bd3e2438
BLAKE2b-256 73d37b604e00ea8c84089e5d32f010caa58b4384b23435e4f7b28946691030ee

See more details on using hashes here.

File details

Details for the file arcticdb-6.17.0-cp311-cp311-macosx_15_0_arm64.whl.

File metadata

File hashes

Hashes for arcticdb-6.17.0-cp311-cp311-macosx_15_0_arm64.whl
Algorithm Hash digest
SHA256 14be8d36b911b7c4ce8b6c770aae4cbb052eec6c6ed955a4ea28b062f158965e
MD5 1e8a5edbc77c1e0feaeedc596cfd76e0
BLAKE2b-256 a64ee086530f2e2dabcc76a11c75746846b0b42e1b62482877f2a7a3c53c033f

See more details on using hashes here.

File details

Details for the file arcticdb-6.17.0-cp310-cp310-win_amd64.whl.

File metadata

  • Download URL: arcticdb-6.17.0-cp310-cp310-win_amd64.whl
  • Upload date:
  • Size: 9.5 MB
  • Tags: CPython 3.10, Windows x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.2.0 CPython/3.10.12

File hashes

Hashes for arcticdb-6.17.0-cp310-cp310-win_amd64.whl
Algorithm Hash digest
SHA256 307dbaec0bd290c2d5eecde2b2a0d38bbecb5565eadca3a52018ad2d72e43f43
MD5 38a19b8ad79ea41c682d7746b96d7ac2
BLAKE2b-256 7bf590d52d57ee93a1362d9bb6fadc6fa95bced3ba2d47317b5c854681a96c6c

See more details on using hashes here.

File details

Details for the file arcticdb-6.17.0-cp310-cp310-manylinux2014_x86_64.manylinux_2_17_x86_64.whl.

File metadata

File hashes

Hashes for arcticdb-6.17.0-cp310-cp310-manylinux2014_x86_64.manylinux_2_17_x86_64.whl
Algorithm Hash digest
SHA256 f410f529995d41882429444b8f9d2972b6dabfc3c28798b144b5536fa8f1a92b
MD5 74fd382b765e1a09729768814d78c41e
BLAKE2b-256 4affa268734eeb2d53821a6e8f1083af212a74cb8e1951fcd6b500cbfb0cd52e

See more details on using hashes here.

File details

Details for the file arcticdb-6.17.0-cp310-cp310-macosx_15_0_arm64.whl.

File metadata

File hashes

Hashes for arcticdb-6.17.0-cp310-cp310-macosx_15_0_arm64.whl
Algorithm Hash digest
SHA256 37ff70be5774d7941bdca1aba35e00b5e90b42c2f04444c34b60bf70e2edd46f
MD5 a20de742abde3524962c0ad8bea4d93b
BLAKE2b-256 b119940eb1c64fa75851eeb5426ddc14f4f94557394f0ce76304d071452e21af

See more details on using hashes here.

File details

Details for the file arcticdb-6.17.0-cp39-cp39-win_amd64.whl.

File metadata

  • Download URL: arcticdb-6.17.0-cp39-cp39-win_amd64.whl
  • Upload date:
  • Size: 9.5 MB
  • Tags: CPython 3.9, Windows x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.2.0 CPython/3.10.12

File hashes

Hashes for arcticdb-6.17.0-cp39-cp39-win_amd64.whl
Algorithm Hash digest
SHA256 c144b286592ad5446a45d788b34b6ce321d627a2ada3cf14774f0a4d54fe9dbe
MD5 22a4a002d27ebc786f58a21b471aca31
BLAKE2b-256 882265862d8cb5030480c5d3fdb1fc8e218e792b01feddf53a727b1796b10ad8

See more details on using hashes here.

File details

Details for the file arcticdb-6.17.0-cp39-cp39-manylinux2014_x86_64.manylinux_2_17_x86_64.whl.

File metadata

File hashes

Hashes for arcticdb-6.17.0-cp39-cp39-manylinux2014_x86_64.manylinux_2_17_x86_64.whl
Algorithm Hash digest
SHA256 e0d7212a2198646351aa4d4ee081195e97e47f20cde55d1e985586e8a8865aa8
MD5 88e735cfc3db93bb2d05fdd4a29ad043
BLAKE2b-256 250f4c1e2dfb96e4f81e310c3b14969525c02916c9ef0befb0900980d40ddf57

See more details on using hashes here.

File details

Details for the file arcticdb-6.17.0-cp39-cp39-macosx_15_0_arm64.whl.

File metadata

File hashes

Hashes for arcticdb-6.17.0-cp39-cp39-macosx_15_0_arm64.whl
Algorithm Hash digest
SHA256 9da353e84f7a67d8b84f93604bd831ccbae5c37ba95397ab5629a43656cb3bdb
MD5 a55cecdcfcca54108fda10c66c639457
BLAKE2b-256 b7bd39317ed56ac2f725cd84ac77a26e4196ce403ab9c59b68703b7e0b3b3b28

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page