Skip to main content

ArcticDB DataFrame Database

Project description


Three minute ArcticDB demo from PyQuantNews

Options data in ArcticDB

ArcticDB Website | 📘 ArcticDB Docs | 📰: ArcticDB Blog | Press Release | Press Release | Community


ArcticDB is a high performance, serverless DataFrame database built for the Python Data Science ecosystem. Launched in March 2023, it is the successor to Arctic.

Use of ArcticDB in production (including business or commercial environments) or for a Database Service requires a paid for license from ArcticDB Limited . Please contact info@arcticdb.io for further details.

ArcticDB offers an intuitive Python-centric API enabling you to read and write Pandas DataFrames to S3 or LMDB utilising a fast C++ data-processing and compression engine.

ArcticDB allows you to:

  • Pandas in, Pandas out: Read and write Pandas DataFrames, NumPy arrays and native types to S3 and LMDB without leaving Python.
  • Built for time-series data: Efficiently index and query time-series data across billions of rows
  • Time travel: Travel back in time to see previous versions of your data and create customizable snapshots of the database
  • Schemaless Database: Append, update and modify data without being constrained by the existing schema
  • Optimised for streaming data: Built in support for efficient sparse data storage
  • Powerful processing: Filter, aggregate and create new columns on-the-fly with a Pandas-like syntax
  • C++ efficiency: Accelerate analytics though concurrency in the C++ data-processing engine

ArcticDB handles data that is big in both row count and column count, so a 20-year history of more than 400,000 unique securities can be stored in a single symbol. Each symbol is maintained as a separate entity with no shared data which means ArcticDB can scale horizontally across symbols, maximising the performance potential of your compute, storage and network.

ArcticDB is designed from the outset to be resilient; there is no single point of failure, and persistent data structures in the storage mean that once a version of a symbol has been written, it can never be corrupted by subsequent updates. Pulling compressed data directly from storage to the client means that there is no server to overload, so your data is always available when you need it.

Quickstart

Prebuilt binary availability (for the current version)

PyPI (Python 3.9 - 3.14) conda-forge (Python 3.10 - 3.14)
Linux x86_64 ✔️ ✔️
Linux arm64 ✔️
Windows x86_64 ✔️ ✔️
MacOS x86_64 ✔️
MacOS arm64 ✔️ ✔️

Storage compatibility

Linux Windows Mac
S3 ✔️ ✔️ ✔️
LMDB ✔️ ✔️ ✔️
Azure Blob Storage ✔️ ✔️ ✔️

We have tested against the following S3 backends:

  • AWS S3
  • Ceph
  • MinIO on Linux
  • Pure Storage S3
  • Scality S3
  • VAST Data S3

Installation

Install ArcticDB:

$ pip install arcticdb

or using conda-forge

$ conda install -c conda-forge arcticdb

Import ArcticDB:

>>> import arcticdb as adb

Create an instance on your S3 storage (with or without explicit credentials):

# Leave AWS to derive credential information
>>> ac = adb.Arctic('s3://MY_ENDPOINT:MY_BUCKET?aws_auth=true')

# Manually specify creds
>>> ac = adb.Arctic('s3://MY_ENDPOINT:MY_BUCKET?region=YOUR_REGION&access=ABCD&secret=DCBA')

Or create an instance on your local disk:

>>> ac = adb.Arctic("lmdb:///<path>")

Create your first library and list the libraries in the instance:

>>> ac.create_library('travel_data')
>>> ac.list_libraries()

Create a test dataframe:

>>> import numpy as np
>>> import pandas as pd
>>> NUM_COLUMNS=10
>>> NUM_ROWS=100_000
>>> df = pd.DataFrame(np.random.randint(0,100,size=(NUM_ROWS, NUM_COLUMNS)), columns=[f"COL_{i}" for i in range(NUM_COLUMNS)], index=pd.date_range('2000', periods=NUM_ROWS, freq='h'))

Get the library, write some data to it, and read it back:

>>> lib = ac['travel_data']
>>> lib.write("my_data", df)
>>> data = lib.read("my_data")

To find out more about working with data, visit our docs


Documentation

The source code for the ArcticDB docs are located in the docs folder, and are hosted at docs.arcticdb.io.

License

ArcticDB is released under a Business Source License 1.1 (BSL)

BSL features are free to use and the source code is available, but users may not use ArcticDB for production use or for a Database Service, without agreement with Man Group Operations Limited.

Use of ArcticDB in production or for a Database Service requires a paid for license from ArcticDB Limited and is licensed under the ArcticDB Software License Agreement. For more information please contact info@arcticdb.io.

The BSL is not certified as an open-source license, but most of the Open Source Initiative (OSI) criteria are met. Please see version conversion dates in the below table:

ArcticDB Version License Converts to Apache 2.0
1.0 Business Source License 1.1 Mar 16, 2025
1.2 Business Source License 1.1 May 22, 2025
1.3 Business Source License 1.1 Jun 9, 2025
1.4 Business Source License 1.1 Jun 23, 2025
1.5 Business Source License 1.1 Jul 11, 2025
1.6 Business Source License 1.1 Jul 25, 2025
2.0 Business Source License 1.1 Aug 29, 2025
3.0 Business Source License 1.1 Sep 13, 2025
4.0 Business Source License 1.1 Sep 27, 2025
4.1 Business Source License 1.1 Nov 1, 2025
4.2 Business Source License 1.1 Nov 12, 2025
4.3 Business Source License 1.1 Feb 7, 2026
4.4 Business Source License 1.1 Apr 5, 2026
4.5 Business Source License 1.1 Aug 14, 2026
5.0 Business Source License 1.1 Oct 31, 2026
5.1 Business Source License 1.1 Nov 15, 2026
5.2 Business Source License 1.1 Jan 27, 2027
5.3 Business Source License 1.1 Mar 24, 2027
5.4 Business Source License 1.1 Apr 28, 2027
5.5 Business Source License 1.1 May 13, 2027
5.6 Business Source License 1.1 May 26, 2027
5.7 Business Source License 1.1 Jun 2, 2027
5.8 Business Source License 1.1 Jun 9, 2027
5.9 Business Source License 1.1 Jul 7, 2027
5.10 Business Source License 1.1 Jul 29, 2027
6.1 Business Source License 1.1 Aug 20, 2027
6.2 Business Source License 1.1 Sep 8, 2027
6.3 Business Source License 1.1 Nov 3, 2027
6.5 Business Source License 1.1 Dec 11, 2027
6.6 Business Source License 1.1 Jan 6, 2028

Code of Conduct

Code of Conduct

This project has adopted a Code of Conduct. If you have any concerns about the Code, or behaviour that you have experienced in the project, please contact us at info@arcticdb.io.

Contributing/Building From Source

We welcome your contributions to help us improve and extend this project!

Please refer to the Contributing page and feel free to open issues on GitHub.

We are also always looking for feedback from our dedicated community! If you have used ArcticDB please let us know, we would love to hear about your experience!

Our release process is documented here.

Community

We would love to hear how your ArcticDB journey evolves, email us at info@arcticdb.io or come chat to us on Twitter!

Interested in learning more about ArcticDB? Head over to our blog!

Do you have any questions or issues? Chat to us and other users through our dedicated Slack Workspace - sign up for Slack access on our website.

Project details


Release history Release notifications | RSS feed

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distributions

No source distribution files available for this release.See tutorial on generating distribution archives.

Built Distributions

If you're not sure about the file name format, learn more about wheel file names.

arcticdb-6.12.1-cp314-cp314-win_amd64.whl (9.4 MB view details)

Uploaded CPython 3.14Windows x86-64

arcticdb-6.12.1-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (19.4 MB view details)

Uploaded CPython 3.14manylinux: glibc 2.17+ x86-64

arcticdb-6.12.1-cp314-cp314-macosx_15_0_arm64.whl (18.0 MB view details)

Uploaded CPython 3.14macOS 15.0+ ARM64

arcticdb-6.12.1-cp313-cp313-win_amd64.whl (9.2 MB view details)

Uploaded CPython 3.13Windows x86-64

arcticdb-6.12.1-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (19.5 MB view details)

Uploaded CPython 3.13manylinux: glibc 2.17+ x86-64

arcticdb-6.12.1-cp313-cp313-macosx_15_0_arm64.whl (18.0 MB view details)

Uploaded CPython 3.13macOS 15.0+ ARM64

arcticdb-6.12.1-cp312-cp312-win_amd64.whl (9.2 MB view details)

Uploaded CPython 3.12Windows x86-64

arcticdb-6.12.1-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (19.5 MB view details)

Uploaded CPython 3.12manylinux: glibc 2.17+ x86-64

arcticdb-6.12.1-cp312-cp312-macosx_15_0_arm64.whl (18.0 MB view details)

Uploaded CPython 3.12macOS 15.0+ ARM64

arcticdb-6.12.1-cp311-cp311-win_amd64.whl (9.2 MB view details)

Uploaded CPython 3.11Windows x86-64

arcticdb-6.12.1-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (19.5 MB view details)

Uploaded CPython 3.11manylinux: glibc 2.17+ x86-64

arcticdb-6.12.1-cp311-cp311-macosx_15_0_arm64.whl (18.0 MB view details)

Uploaded CPython 3.11macOS 15.0+ ARM64

arcticdb-6.12.1-cp310-cp310-win_amd64.whl (9.2 MB view details)

Uploaded CPython 3.10Windows x86-64

arcticdb-6.12.1-cp310-cp310-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (19.5 MB view details)

Uploaded CPython 3.10manylinux: glibc 2.17+ x86-64

arcticdb-6.12.1-cp310-cp310-macosx_15_0_arm64.whl (18.1 MB view details)

Uploaded CPython 3.10macOS 15.0+ ARM64

arcticdb-6.12.1-cp39-cp39-win_amd64.whl (9.2 MB view details)

Uploaded CPython 3.9Windows x86-64

arcticdb-6.12.1-cp39-cp39-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (19.5 MB view details)

Uploaded CPython 3.9manylinux: glibc 2.17+ x86-64

arcticdb-6.12.1-cp39-cp39-macosx_15_0_arm64.whl (18.1 MB view details)

Uploaded CPython 3.9macOS 15.0+ ARM64

File details

Details for the file arcticdb-6.12.1-cp314-cp314-win_amd64.whl.

File metadata

  • Download URL: arcticdb-6.12.1-cp314-cp314-win_amd64.whl
  • Upload date:
  • Size: 9.4 MB
  • Tags: CPython 3.14, Windows x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.2.0 CPython/3.10.12

File hashes

Hashes for arcticdb-6.12.1-cp314-cp314-win_amd64.whl
Algorithm Hash digest
SHA256 062c71d66dbf550fb49ea05f5c4f002685bb356698f074db871596dbef02c9a1
MD5 6433c815cedcf13a27338f8686f699de
BLAKE2b-256 65aa818f0de4aa8192e3a45a7d6c331e4093dc4c436ba26b99efe2839395f3f7

See more details on using hashes here.

File details

Details for the file arcticdb-6.12.1-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.whl.

File metadata

File hashes

Hashes for arcticdb-6.12.1-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.whl
Algorithm Hash digest
SHA256 4978678373c797b332548f4f7ad2ac04573e412f75c15340e15492f0ef4ceea1
MD5 ba7df57fd887c40275dcd0e943193218
BLAKE2b-256 d67ec312dab3b8d0b3092d69884af5ae0c47e6a57ee749d7d6b53d91411ac39c

See more details on using hashes here.

File details

Details for the file arcticdb-6.12.1-cp314-cp314-macosx_15_0_arm64.whl.

File metadata

File hashes

Hashes for arcticdb-6.12.1-cp314-cp314-macosx_15_0_arm64.whl
Algorithm Hash digest
SHA256 91bb390ac76c15c17a35c931fec5d75212cd6c9c8a939c22e0f94120a52bf3cc
MD5 65a5e4c32daad79d9676563c21256085
BLAKE2b-256 51b6c1baf2c11419b4ecfa01d12fe2ef0144ce19ea77d7af537e141760a83f91

See more details on using hashes here.

File details

Details for the file arcticdb-6.12.1-cp313-cp313-win_amd64.whl.

File metadata

  • Download URL: arcticdb-6.12.1-cp313-cp313-win_amd64.whl
  • Upload date:
  • Size: 9.2 MB
  • Tags: CPython 3.13, Windows x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.2.0 CPython/3.10.12

File hashes

Hashes for arcticdb-6.12.1-cp313-cp313-win_amd64.whl
Algorithm Hash digest
SHA256 3eb455fdff1eb8e6b40d50cefa5037395ab808978df634363ab0e643270626b4
MD5 b8b4de143b05fd63f455a60642f1aa75
BLAKE2b-256 138195c1c08061ab0b080b1867518cc2a1c883101bf167c8721dd90dd29711b7

See more details on using hashes here.

File details

Details for the file arcticdb-6.12.1-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.whl.

File metadata

File hashes

Hashes for arcticdb-6.12.1-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.whl
Algorithm Hash digest
SHA256 e7ea356109b3f836180aaee245c1f41b62efac0e7c43f1952b969690d8734b41
MD5 e9b4965efa23f509dd354211bb5fb5b6
BLAKE2b-256 8c84171fac50bbc07ef66c33cbf5a59ff2f983a54655dab132ef0f339430099b

See more details on using hashes here.

File details

Details for the file arcticdb-6.12.1-cp313-cp313-macosx_15_0_arm64.whl.

File metadata

File hashes

Hashes for arcticdb-6.12.1-cp313-cp313-macosx_15_0_arm64.whl
Algorithm Hash digest
SHA256 73d25d9d532e017327b1763f6a269b38c950f89a3ed4be3412ffb52fadc22b8c
MD5 4541c641ad15d2236019e533e9799fef
BLAKE2b-256 154b6efbe9ad5a6a223afd203bf8730b90d5fcf85f33d1e82e4bcc6e4caceac3

See more details on using hashes here.

File details

Details for the file arcticdb-6.12.1-cp312-cp312-win_amd64.whl.

File metadata

  • Download URL: arcticdb-6.12.1-cp312-cp312-win_amd64.whl
  • Upload date:
  • Size: 9.2 MB
  • Tags: CPython 3.12, Windows x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.2.0 CPython/3.10.12

File hashes

Hashes for arcticdb-6.12.1-cp312-cp312-win_amd64.whl
Algorithm Hash digest
SHA256 bd68db6f42f6f97bdaf57ec6850380595df32f62770201f71ee413e03f4dcd56
MD5 cd69547b18c59e612c832227ccb0cffc
BLAKE2b-256 9c66280dd5b0c97646844f9279e216c12cd9f1548ab547afe1e54f5663ea2bdd

See more details on using hashes here.

File details

Details for the file arcticdb-6.12.1-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.whl.

File metadata

File hashes

Hashes for arcticdb-6.12.1-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.whl
Algorithm Hash digest
SHA256 468cc39af4293f6c42179f5a3206fb90aadc6c3e94b10c3d4593254118460ff4
MD5 81583180dcd64ae6d1d420cd3a6755e5
BLAKE2b-256 64224b7873b4ce6daa5dd19cc92c18951a49ac3f171065a768f3e7a9aa4838f4

See more details on using hashes here.

File details

Details for the file arcticdb-6.12.1-cp312-cp312-macosx_15_0_arm64.whl.

File metadata

File hashes

Hashes for arcticdb-6.12.1-cp312-cp312-macosx_15_0_arm64.whl
Algorithm Hash digest
SHA256 03ece548322f88284f191cfe36ebcd2e9951810dd6f5c1f7252340096fd30360
MD5 e21691456811f6887eb673e8b56d1db0
BLAKE2b-256 480d08d219001333407e9e56edf3884dd737b273cb75a01546eae35a2214580b

See more details on using hashes here.

File details

Details for the file arcticdb-6.12.1-cp311-cp311-win_amd64.whl.

File metadata

  • Download URL: arcticdb-6.12.1-cp311-cp311-win_amd64.whl
  • Upload date:
  • Size: 9.2 MB
  • Tags: CPython 3.11, Windows x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.2.0 CPython/3.10.12

File hashes

Hashes for arcticdb-6.12.1-cp311-cp311-win_amd64.whl
Algorithm Hash digest
SHA256 7c8fa5e5bfb727cab1b5e93c50946b6aafba71acb4f3c0426e0aa9a0ab0f3f41
MD5 f54af1191b78f3b57dc7dabbbaaa31de
BLAKE2b-256 b707b21572ec4adee6b0dcd856bf1861a418a15e6b665ab7d98ae1710417f7ca

See more details on using hashes here.

File details

Details for the file arcticdb-6.12.1-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.whl.

File metadata

File hashes

Hashes for arcticdb-6.12.1-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.whl
Algorithm Hash digest
SHA256 da234139558108ab48074fa0ae801e3cc20f38c5a99d98885667654310a0e098
MD5 ac9b62ebd46cedf9ecacdfcc6a798e2a
BLAKE2b-256 4d7f7ae6838ce94672d48906432de143eafc7930c31378e706cf3277bc51d5e3

See more details on using hashes here.

File details

Details for the file arcticdb-6.12.1-cp311-cp311-macosx_15_0_arm64.whl.

File metadata

File hashes

Hashes for arcticdb-6.12.1-cp311-cp311-macosx_15_0_arm64.whl
Algorithm Hash digest
SHA256 3702911895d23bc3dffb2c4b777166449fa8cd4f8d6d4ad4a121142818815434
MD5 5e83b203c3eddd116cc60a7531932b3c
BLAKE2b-256 7b003f6c39339d0c572ce8d7c51ad6d251f72c699bb1c9c785486030c20396d5

See more details on using hashes here.

File details

Details for the file arcticdb-6.12.1-cp310-cp310-win_amd64.whl.

File metadata

  • Download URL: arcticdb-6.12.1-cp310-cp310-win_amd64.whl
  • Upload date:
  • Size: 9.2 MB
  • Tags: CPython 3.10, Windows x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.2.0 CPython/3.10.12

File hashes

Hashes for arcticdb-6.12.1-cp310-cp310-win_amd64.whl
Algorithm Hash digest
SHA256 63c50671185b8d937847db4aa1b201c234c86e17ffffe9b17353566d98fbf4bb
MD5 c525710ad303bb29609cecc920590b4a
BLAKE2b-256 70bf87617700ee8bbdf13edd306cdcf0dfdc33d889d4995c44be58340222d4d2

See more details on using hashes here.

File details

Details for the file arcticdb-6.12.1-cp310-cp310-manylinux2014_x86_64.manylinux_2_17_x86_64.whl.

File metadata

File hashes

Hashes for arcticdb-6.12.1-cp310-cp310-manylinux2014_x86_64.manylinux_2_17_x86_64.whl
Algorithm Hash digest
SHA256 b0547292e707f29e211e9d45775f31f8ebb355e10f9d8d186331d2116ae2bd5c
MD5 835ea909508bea63130868d79ea0501d
BLAKE2b-256 d88d2094aabb9d0615f56501d98143932b90d5eca912d2db73c763cd0ec8296b

See more details on using hashes here.

File details

Details for the file arcticdb-6.12.1-cp310-cp310-macosx_15_0_arm64.whl.

File metadata

File hashes

Hashes for arcticdb-6.12.1-cp310-cp310-macosx_15_0_arm64.whl
Algorithm Hash digest
SHA256 9472eb479e21c6e44ec93d7bd5401c37dff3e1b6deca78dc01f61b393fc54310
MD5 b9516299fad8157cdd015321125ca306
BLAKE2b-256 194a7cdfe7cb40e5f79472804d83e51363b0ccd698e8ebceb162ec2711d9827f

See more details on using hashes here.

File details

Details for the file arcticdb-6.12.1-cp39-cp39-win_amd64.whl.

File metadata

  • Download URL: arcticdb-6.12.1-cp39-cp39-win_amd64.whl
  • Upload date:
  • Size: 9.2 MB
  • Tags: CPython 3.9, Windows x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.2.0 CPython/3.10.12

File hashes

Hashes for arcticdb-6.12.1-cp39-cp39-win_amd64.whl
Algorithm Hash digest
SHA256 316a8c5dbe964dfdb3b9a021114a1e8e3ed3f3bc2a1763c80e554fe554edcc6a
MD5 243a6e8397f775c7fbd81cfee86e2f14
BLAKE2b-256 082d691a878e32b972b4372661049ffadbc1a25a69b27c21f87910762f4c6024

See more details on using hashes here.

File details

Details for the file arcticdb-6.12.1-cp39-cp39-manylinux2014_x86_64.manylinux_2_17_x86_64.whl.

File metadata

File hashes

Hashes for arcticdb-6.12.1-cp39-cp39-manylinux2014_x86_64.manylinux_2_17_x86_64.whl
Algorithm Hash digest
SHA256 537c9521a61090335f883b46e197f0df2f7b1e91ff0a0c0d23723195c6a2d61d
MD5 79692f572270aa60775b625f3b8c65c0
BLAKE2b-256 7f72640808d3019c1ade6c6e5bd5405b892e8a880a8e246d6dc17a8eda70bf7d

See more details on using hashes here.

File details

Details for the file arcticdb-6.12.1-cp39-cp39-macosx_15_0_arm64.whl.

File metadata

File hashes

Hashes for arcticdb-6.12.1-cp39-cp39-macosx_15_0_arm64.whl
Algorithm Hash digest
SHA256 2e4a3366523da3c152f37742f253d27aad38ec6ac58e4f2f8b5c0fb6e5d38ca3
MD5 28587a843376aa2be380183fff477fc8
BLAKE2b-256 4a06765bf77734992de3920b24f3707efa5108b2c1ad2310d5bbee6f28a1f4f4

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page