Skip to main content

ArcticDB DataFrame Database

Project description


Three minute ArcticDB demo from PyQuantNews

Options data in ArcticDB

ArcticDB Website | 📘 ArcticDB Docs | 📰: ArcticDB Blog | Press Release | Press Release | Community


ArcticDB is a high performance, serverless DataFrame database built for the Python Data Science ecosystem. Launched in March 2023, it is the successor to Arctic.

Use of ArcticDB in production (including business or commercial environments) or for a Database Service requires a paid for license from ArcticDB Limited . Please contact info@arcticdb.io for further details.

ArcticDB offers an intuitive Python-centric API enabling you to read and write Pandas DataFrames to S3 or LMDB utilising a fast C++ data-processing and compression engine.

ArcticDB allows you to:

  • Pandas in, Pandas out: Read and write Pandas DataFrames, NumPy arrays and native types to S3 and LMDB without leaving Python.
  • Built for time-series data: Efficiently index and query time-series data across billions of rows
  • Time travel: Travel back in time to see previous versions of your data and create customizable snapshots of the database
  • Schemaless Database: Append, update and modify data without being constrained by the existing schema
  • Optimised for streaming data: Built in support for efficient sparse data storage
  • Powerful processing: Filter, aggregate and create new columns on-the-fly with a Pandas-like syntax
  • C++ efficiency: Accelerate analytics though concurrency in the C++ data-processing engine

ArcticDB handles data that is big in both row count and column count, so a 20-year history of more than 400,000 unique securities can be stored in a single symbol. Each symbol is maintained as a separate entity with no shared data which means ArcticDB can scale horizontally across symbols, maximising the performance potential of your compute, storage and network.

ArcticDB is designed from the outset to be resilient; there is no single point of failure, and persistent data structures in the storage mean that once a version of a symbol has been written, it can never be corrupted by subsequent updates. Pulling compressed data directly from storage to the client means that there is no server to overload, so your data is always available when you need it.

Quickstart

Prebuilt binary availability (for the current version)

PyPI (Python 3.9 - 3.14) conda-forge (Python 3.10 - 3.14)
Linux x86_64 ✔️ ✔️
Linux arm64 ✔️
Windows x86_64 ✔️ ✔️
MacOS x86_64 ✔️
MacOS arm64 ✔️ ✔️

Storage compatibility

Linux Windows Mac
S3 ✔️ ✔️ ✔️
LMDB ✔️ ✔️ ✔️
Azure Blob Storage ✔️ ✔️ ✔️

We have tested against the following S3 backends:

  • AWS S3
  • Ceph
  • MinIO on Linux
  • Pure Storage S3
  • Scality S3
  • VAST Data S3

Installation

Install ArcticDB:

$ pip install arcticdb

or using conda-forge

$ conda install -c conda-forge arcticdb

Import ArcticDB:

>>> import arcticdb as adb

Create an instance on your S3 storage (with or without explicit credentials):

# Leave AWS to derive credential information
>>> ac = adb.Arctic('s3://MY_ENDPOINT:MY_BUCKET?aws_auth=true')

# Manually specify creds
>>> ac = adb.Arctic('s3://MY_ENDPOINT:MY_BUCKET?region=YOUR_REGION&access=ABCD&secret=DCBA')

Or create an instance on your local disk:

>>> ac = adb.Arctic("lmdb:///<path>")

Create your first library and list the libraries in the instance:

>>> ac.create_library('travel_data')
>>> ac.list_libraries()

Create a test dataframe:

>>> import numpy as np
>>> import pandas as pd
>>> NUM_COLUMNS=10
>>> NUM_ROWS=100_000
>>> df = pd.DataFrame(np.random.randint(0,100,size=(NUM_ROWS, NUM_COLUMNS)), columns=[f"COL_{i}" for i in range(NUM_COLUMNS)], index=pd.date_range('2000', periods=NUM_ROWS, freq='h'))

Get the library, write some data to it, and read it back:

>>> lib = ac['travel_data']
>>> lib.write("my_data", df)
>>> data = lib.read("my_data")

To find out more about working with data, visit our docs


Documentation

The source code for the ArcticDB docs are located in the docs folder, and are hosted at docs.arcticdb.io.

License

ArcticDB is released under a Business Source License 1.1 (BSL)

BSL features are free to use and the source code is available, but users may not use ArcticDB for production use or for a Database Service, without agreement with Man Group Operations Limited.

Use of ArcticDB in production or for a Database Service requires a paid for license from ArcticDB Limited and is licensed under the ArcticDB Software License Agreement. For more information please contact info@arcticdb.io.

The BSL is not certified as an open-source license, but most of the Open Source Initiative (OSI) criteria are met. Please see version conversion dates in the below table:

ArcticDB Version License Converts to Apache 2.0
1.0 Business Source License 1.1 Mar 16, 2025
1.2 Business Source License 1.1 May 22, 2025
1.3 Business Source License 1.1 Jun 9, 2025
1.4 Business Source License 1.1 Jun 23, 2025
1.5 Business Source License 1.1 Jul 11, 2025
1.6 Business Source License 1.1 Jul 25, 2025
2.0 Business Source License 1.1 Aug 29, 2025
3.0 Business Source License 1.1 Sep 13, 2025
4.0 Business Source License 1.1 Sep 27, 2025
4.1 Business Source License 1.1 Nov 1, 2025
4.2 Business Source License 1.1 Nov 12, 2025
4.3 Business Source License 1.1 Feb 7, 2026
4.4 Business Source License 1.1 Apr 5, 2026
4.5 Business Source License 1.1 Aug 14, 2026
5.0 Business Source License 1.1 Oct 31, 2026
5.1 Business Source License 1.1 Nov 15, 2026
5.2 Business Source License 1.1 Jan 27, 2027
5.3 Business Source License 1.1 Mar 24, 2027
5.4 Business Source License 1.1 Apr 28, 2027
5.5 Business Source License 1.1 May 13, 2027
5.6 Business Source License 1.1 May 26, 2027
5.7 Business Source License 1.1 Jun 2, 2027
5.8 Business Source License 1.1 Jun 9, 2027
5.9 Business Source License 1.1 Jul 7, 2027
5.10 Business Source License 1.1 Jul 29, 2027
6.1 Business Source License 1.1 Aug 20, 2027
6.2 Business Source License 1.1 Sep 8, 2027
6.3 Business Source License 1.1 Nov 3, 2027
6.5 Business Source License 1.1 Dec 11, 2027
6.6 Business Source License 1.1 Jan 6, 2028

Code of Conduct

Code of Conduct

This project has adopted a Code of Conduct. If you have any concerns about the Code, or behaviour that you have experienced in the project, please contact us at info@arcticdb.io.

Contributing/Building From Source

We welcome your contributions to help us improve and extend this project!

Please refer to the Contributing page and feel free to open issues on GitHub.

We are also always looking for feedback from our dedicated community! If you have used ArcticDB please let us know, we would love to hear about your experience!

Our release process is documented here.

Community

We would love to hear how your ArcticDB journey evolves, email us at info@arcticdb.io or come chat to us on Twitter!

Interested in learning more about ArcticDB? Head over to our blog!

Do you have any questions or issues? Chat to us and other users through our dedicated Slack Workspace - sign up for Slack access on our website.

Project details


Release history Release notifications | RSS feed

This version

6.9.0

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distributions

No source distribution files available for this release.See tutorial on generating distribution archives.

Built Distributions

If you're not sure about the file name format, learn more about wheel file names.

arcticdb-6.9.0-cp314-cp314-win_amd64.whl (9.3 MB view details)

Uploaded CPython 3.14Windows x86-64

arcticdb-6.9.0-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (18.9 MB view details)

Uploaded CPython 3.14manylinux: glibc 2.17+ x86-64

arcticdb-6.9.0-cp314-cp314-macosx_15_0_arm64.whl (17.5 MB view details)

Uploaded CPython 3.14macOS 15.0+ ARM64

arcticdb-6.9.0-cp313-cp313-win_amd64.whl (9.0 MB view details)

Uploaded CPython 3.13Windows x86-64

arcticdb-6.9.0-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (18.9 MB view details)

Uploaded CPython 3.13manylinux: glibc 2.17+ x86-64

arcticdb-6.9.0-cp313-cp313-macosx_15_0_arm64.whl (17.5 MB view details)

Uploaded CPython 3.13macOS 15.0+ ARM64

arcticdb-6.9.0-cp312-cp312-win_amd64.whl (9.1 MB view details)

Uploaded CPython 3.12Windows x86-64

arcticdb-6.9.0-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (18.9 MB view details)

Uploaded CPython 3.12manylinux: glibc 2.17+ x86-64

arcticdb-6.9.0-cp312-cp312-macosx_15_0_arm64.whl (17.5 MB view details)

Uploaded CPython 3.12macOS 15.0+ ARM64

arcticdb-6.9.0-cp311-cp311-win_amd64.whl (9.1 MB view details)

Uploaded CPython 3.11Windows x86-64

arcticdb-6.9.0-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (18.9 MB view details)

Uploaded CPython 3.11manylinux: glibc 2.17+ x86-64

arcticdb-6.9.0-cp311-cp311-macosx_15_0_arm64.whl (17.5 MB view details)

Uploaded CPython 3.11macOS 15.0+ ARM64

arcticdb-6.9.0-cp310-cp310-win_amd64.whl (9.1 MB view details)

Uploaded CPython 3.10Windows x86-64

arcticdb-6.9.0-cp310-cp310-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (18.9 MB view details)

Uploaded CPython 3.10manylinux: glibc 2.17+ x86-64

arcticdb-6.9.0-cp310-cp310-macosx_15_0_arm64.whl (17.5 MB view details)

Uploaded CPython 3.10macOS 15.0+ ARM64

arcticdb-6.9.0-cp39-cp39-win_amd64.whl (9.1 MB view details)

Uploaded CPython 3.9Windows x86-64

arcticdb-6.9.0-cp39-cp39-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (18.9 MB view details)

Uploaded CPython 3.9manylinux: glibc 2.17+ x86-64

arcticdb-6.9.0-cp39-cp39-macosx_15_0_arm64.whl (17.5 MB view details)

Uploaded CPython 3.9macOS 15.0+ ARM64

File details

Details for the file arcticdb-6.9.0-cp314-cp314-win_amd64.whl.

File metadata

  • Download URL: arcticdb-6.9.0-cp314-cp314-win_amd64.whl
  • Upload date:
  • Size: 9.3 MB
  • Tags: CPython 3.14, Windows x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.2.0 CPython/3.10.12

File hashes

Hashes for arcticdb-6.9.0-cp314-cp314-win_amd64.whl
Algorithm Hash digest
SHA256 04780ef10b7b56f2fd3f186973367bda21f79d8239c5f12eb01a7402bd60a01c
MD5 0f79bd625d1347b91abfaf410dba54f5
BLAKE2b-256 89337a698399f1e7a93af8fca6776b5bcff0de43146f680aca42cce4bef10159

See more details on using hashes here.

File details

Details for the file arcticdb-6.9.0-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.whl.

File metadata

File hashes

Hashes for arcticdb-6.9.0-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.whl
Algorithm Hash digest
SHA256 0935ed36127c440cb14a8ce80e1816054922ef1debcee8ca562f7442dc9896e2
MD5 86691160c25c969272d602fe65cfe7c7
BLAKE2b-256 b016f46118731fd6538e1dd25e46ee7e37968ce4b9d64e8d17e1a7b3fe6a6a6f

See more details on using hashes here.

File details

Details for the file arcticdb-6.9.0-cp314-cp314-macosx_15_0_arm64.whl.

File metadata

File hashes

Hashes for arcticdb-6.9.0-cp314-cp314-macosx_15_0_arm64.whl
Algorithm Hash digest
SHA256 9cf5f10d7f0cef0adeae4e96861638e555affc79e0c6fefa4e1a59a2d85fa921
MD5 a41cbbd84de7bbebfe046c62b044358d
BLAKE2b-256 440686cd4972137b4389ed79ff714eefe1f47782877c5937af045e0c77ae95ec

See more details on using hashes here.

File details

Details for the file arcticdb-6.9.0-cp313-cp313-win_amd64.whl.

File metadata

  • Download URL: arcticdb-6.9.0-cp313-cp313-win_amd64.whl
  • Upload date:
  • Size: 9.0 MB
  • Tags: CPython 3.13, Windows x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.2.0 CPython/3.10.12

File hashes

Hashes for arcticdb-6.9.0-cp313-cp313-win_amd64.whl
Algorithm Hash digest
SHA256 bf79fb8ea11cd3282903526c995c83ee4ff1167246637b52fb8603a50bb974f2
MD5 2b65dcd822fb338b57d05a1ef45c67b0
BLAKE2b-256 bf7cc316f9521f7078763d80e561e4899c6e808e2223bc60c70ab888e76ae047

See more details on using hashes here.

File details

Details for the file arcticdb-6.9.0-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.whl.

File metadata

File hashes

Hashes for arcticdb-6.9.0-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.whl
Algorithm Hash digest
SHA256 d297326895190f7a144c57729a7936533c3a1a54eb0de3fe8dce5b70f6c855fe
MD5 eb75bcee75e0fd267de2c44be05c6747
BLAKE2b-256 4c2d5f2487d1600aa402c657f49e70a9df0a5d83f7fb6e6b81d78c4fb8e37199

See more details on using hashes here.

File details

Details for the file arcticdb-6.9.0-cp313-cp313-macosx_15_0_arm64.whl.

File metadata

File hashes

Hashes for arcticdb-6.9.0-cp313-cp313-macosx_15_0_arm64.whl
Algorithm Hash digest
SHA256 fffae2ef7e5588b773fa19572814b006682c5a26b42824abdc5c02ecc2a89fc4
MD5 a50bc739b22bb49615464b5b705b2815
BLAKE2b-256 e23e2a6260f9f1a3ac1cc9088b16e5f129d7a8b3805c76ee35aad45439ecb2b2

See more details on using hashes here.

File details

Details for the file arcticdb-6.9.0-cp312-cp312-win_amd64.whl.

File metadata

  • Download URL: arcticdb-6.9.0-cp312-cp312-win_amd64.whl
  • Upload date:
  • Size: 9.1 MB
  • Tags: CPython 3.12, Windows x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.2.0 CPython/3.10.12

File hashes

Hashes for arcticdb-6.9.0-cp312-cp312-win_amd64.whl
Algorithm Hash digest
SHA256 d3e2de0eb36ee8a6b272a9fb07aa83bb5936d2718f332e320dbfd611bd4cebbd
MD5 1379642844c08bd94abbdc77f9e363b0
BLAKE2b-256 ec851d9c3c3f51f83ff2b0e0b4919089137eac53b4ade87a873a97646e15837a

See more details on using hashes here.

File details

Details for the file arcticdb-6.9.0-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.whl.

File metadata

File hashes

Hashes for arcticdb-6.9.0-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.whl
Algorithm Hash digest
SHA256 fa95e2a490dd0491e2674462220b4e1945c359a3d6238031b5a34d74115e1d34
MD5 1f6a171bdb3dc6338fdbdb0546325ca7
BLAKE2b-256 2e49da7a4feccfe0bb815d35467dcc7e4cec44fce39dd5b37b4462febabecbc9

See more details on using hashes here.

File details

Details for the file arcticdb-6.9.0-cp312-cp312-macosx_15_0_arm64.whl.

File metadata

File hashes

Hashes for arcticdb-6.9.0-cp312-cp312-macosx_15_0_arm64.whl
Algorithm Hash digest
SHA256 87d9c18ba86885609255fc0de775aebfd892f0e458f94c230e6cfe922f36acc5
MD5 fd8b322de1ae46c3085063c4110a691d
BLAKE2b-256 be37ac1ea8cd64bc07d6b3ac940779afb81aed81ce21c12d9c2e2c884b5a0096

See more details on using hashes here.

File details

Details for the file arcticdb-6.9.0-cp311-cp311-win_amd64.whl.

File metadata

  • Download URL: arcticdb-6.9.0-cp311-cp311-win_amd64.whl
  • Upload date:
  • Size: 9.1 MB
  • Tags: CPython 3.11, Windows x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.2.0 CPython/3.10.12

File hashes

Hashes for arcticdb-6.9.0-cp311-cp311-win_amd64.whl
Algorithm Hash digest
SHA256 c178400b5eb8f111a412cb79487b8ace7a2c80359fa285c2fcc13eb10fa5d975
MD5 be3c70d2ee4c1dd764a63512023c3a8c
BLAKE2b-256 7bfabb4920116fe4377fbc1ccc6248635fb0031dd83c14b5eaedbe8d1a57ffb9

See more details on using hashes here.

File details

Details for the file arcticdb-6.9.0-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.whl.

File metadata

File hashes

Hashes for arcticdb-6.9.0-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.whl
Algorithm Hash digest
SHA256 046b6fb1d73313e9f62e2d777b3efc171a1d5766d759d5fa558196d093feb832
MD5 9e4eeb778470eb76bccc15bf7957b4fc
BLAKE2b-256 61fad72ada991a0313740c45013b8cf9829d01f8fad9d07e7cd62e4f47e29a1c

See more details on using hashes here.

File details

Details for the file arcticdb-6.9.0-cp311-cp311-macosx_15_0_arm64.whl.

File metadata

File hashes

Hashes for arcticdb-6.9.0-cp311-cp311-macosx_15_0_arm64.whl
Algorithm Hash digest
SHA256 4c0206a087165067bba2405fbf9ed96d9945637b0739722b652a394510d4f465
MD5 88f2a8a3fd26028b3e98820437bebd13
BLAKE2b-256 c11f48e16d96dca1eac3c2bc2ed0cddf39b7b17f54b86ec6b1767cf71c42c8be

See more details on using hashes here.

File details

Details for the file arcticdb-6.9.0-cp310-cp310-win_amd64.whl.

File metadata

  • Download URL: arcticdb-6.9.0-cp310-cp310-win_amd64.whl
  • Upload date:
  • Size: 9.1 MB
  • Tags: CPython 3.10, Windows x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.2.0 CPython/3.10.12

File hashes

Hashes for arcticdb-6.9.0-cp310-cp310-win_amd64.whl
Algorithm Hash digest
SHA256 cc38b818d4da50ce9a5078308dd7cb278491ab554d05242c77b1be4f695b6a9c
MD5 00a704cc6fe40ec4f7c162081dec8db1
BLAKE2b-256 4e2aac93416b0bbecf4bb1e0e5a7a9874a30a0390f796527d912b14a61f83d48

See more details on using hashes here.

File details

Details for the file arcticdb-6.9.0-cp310-cp310-manylinux2014_x86_64.manylinux_2_17_x86_64.whl.

File metadata

File hashes

Hashes for arcticdb-6.9.0-cp310-cp310-manylinux2014_x86_64.manylinux_2_17_x86_64.whl
Algorithm Hash digest
SHA256 a7544e460e0aa1618596c6c297ba2eca9f409476ccda16f9b30d8536ea6d26a9
MD5 31efdbca6aa35a7e2a318f53a887cce6
BLAKE2b-256 fadc022bbde1493c863ff1c4bc2f59e2a8ffbd91cd275b2e32bd5af421129424

See more details on using hashes here.

File details

Details for the file arcticdb-6.9.0-cp310-cp310-macosx_15_0_arm64.whl.

File metadata

File hashes

Hashes for arcticdb-6.9.0-cp310-cp310-macosx_15_0_arm64.whl
Algorithm Hash digest
SHA256 fde8081a97472b054fa1a87241d34c3db50191cf11b830905423f38a6ea0c27e
MD5 29d0680f1ec11d5a08fed649b9e45488
BLAKE2b-256 bd64eb45b2510321c307d7ac9c1303fe3a2df630829951ad4b416bca20aa6e20

See more details on using hashes here.

File details

Details for the file arcticdb-6.9.0-cp39-cp39-win_amd64.whl.

File metadata

  • Download URL: arcticdb-6.9.0-cp39-cp39-win_amd64.whl
  • Upload date:
  • Size: 9.1 MB
  • Tags: CPython 3.9, Windows x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.2.0 CPython/3.10.12

File hashes

Hashes for arcticdb-6.9.0-cp39-cp39-win_amd64.whl
Algorithm Hash digest
SHA256 af5968c8cd25d6c4375cd1a272d8df809ec3916cb4043056cfa9cdbcfeed56dc
MD5 71378acbe2f51929b1ae27baf013dfe6
BLAKE2b-256 7184eb7878922d427d570f3df94addf0ef4a1f041ee5f72c5eca7572ec4594cf

See more details on using hashes here.

File details

Details for the file arcticdb-6.9.0-cp39-cp39-manylinux2014_x86_64.manylinux_2_17_x86_64.whl.

File metadata

File hashes

Hashes for arcticdb-6.9.0-cp39-cp39-manylinux2014_x86_64.manylinux_2_17_x86_64.whl
Algorithm Hash digest
SHA256 dd699bac6d760a076d6663329c7ac436b07c196c66dd5dccb34f6d72bc7c094d
MD5 efdd1d92a87d3c772f64624d4467e358
BLAKE2b-256 6cd3a76962a2a0b8f5d1bb8efac10c29e2834b204760687194c67c94448ec49a

See more details on using hashes here.

File details

Details for the file arcticdb-6.9.0-cp39-cp39-macosx_15_0_arm64.whl.

File metadata

File hashes

Hashes for arcticdb-6.9.0-cp39-cp39-macosx_15_0_arm64.whl
Algorithm Hash digest
SHA256 50c0e74079943f62ad5db4b446c3d5072014ee6af239fe1cf43157b74e53a03c
MD5 a7e92c08711639051175935b973e88ec
BLAKE2b-256 8374b7b1921cec24b8bd335c0907c265117bbdef828a4b08e6450b2ba51b011d

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page