Skip to main content

ArcticDB DataFrame Database

Project description


Three minute ArcticDB demo from PyQuantNews

Options data in ArcticDB

ArcticDB Website | 📘 ArcticDB Docs | 📰: ArcticDB Blog | Press Release | Press Release | Community


ArcticDB is a high performance, serverless DataFrame database built for the Python Data Science ecosystem. Launched in March 2023, it is the successor to Arctic.

Use of ArcticDB in production (including business or commercial environments) or for a Database Service requires a paid for license from ArcticDB Limited . Please contact info@arcticdb.io for further details.

ArcticDB offers an intuitive Python-centric API enabling you to read and write Pandas DataFrames to S3 or LMDB utilising a fast C++ data-processing and compression engine.

ArcticDB allows you to:

  • Pandas in, Pandas out: Read and write Pandas DataFrames, NumPy arrays and native types to S3 and LMDB without leaving Python.
  • Built for time-series data: Efficiently index and query time-series data across billions of rows
  • Time travel: Travel back in time to see previous versions of your data and create customizable snapshots of the database
  • Schemaless Database: Append, update and modify data without being constrained by the existing schema
  • Optimised for streaming data: Built in support for efficient sparse data storage
  • Powerful processing: Filter, aggregate and create new columns on-the-fly with a Pandas-like syntax
  • C++ efficiency: Accelerate analytics though concurrency in the C++ data-processing engine

ArcticDB handles data that is big in both row count and column count, so a 20-year history of more than 400,000 unique securities can be stored in a single symbol. Each symbol is maintained as a separate entity with no shared data which means ArcticDB can scale horizontally across symbols, maximising the performance potential of your compute, storage and network.

ArcticDB is designed from the outset to be resilient; there is no single point of failure, and persistent data structures in the storage mean that once a version of a symbol has been written, it can never be corrupted by subsequent updates. Pulling compressed data directly from storage to the client means that there is no server to overload, so your data is always available when you need it.

Quickstart

Prebuilt binary availability (for the current version)

PyPI (Python 3.9 - 3.14) conda-forge (Python 3.10 - 3.14)
Linux x86_64 ✔️ ✔️
Linux arm64 ✔️
Windows x86_64 ✔️ ✔️
MacOS x86_64 ✔️
MacOS arm64 ✔️ ✔️

Storage compatibility

Linux Windows Mac
S3 ✔️ ✔️ ✔️
LMDB ✔️ ✔️ ✔️
Azure Blob Storage ✔️ ✔️ ✔️

We have tested against the following S3 backends:

  • AWS S3
  • Ceph
  • MinIO on Linux
  • Pure Storage S3
  • Scality S3
  • VAST Data S3

Installation

Install ArcticDB:

$ pip install arcticdb

or using conda-forge

$ conda install -c conda-forge arcticdb

Import ArcticDB:

>>> import arcticdb as adb

Create an instance on your S3 storage (with or without explicit credentials):

# Leave AWS to derive credential information
>>> ac = adb.Arctic('s3://MY_ENDPOINT:MY_BUCKET?aws_auth=true')

# Manually specify creds
>>> ac = adb.Arctic('s3://MY_ENDPOINT:MY_BUCKET?region=YOUR_REGION&access=ABCD&secret=DCBA')

Or create an instance on your local disk:

>>> ac = adb.Arctic("lmdb:///<path>")

Create your first library and list the libraries in the instance:

>>> ac.create_library('travel_data')
>>> ac.list_libraries()

Create a test dataframe:

>>> import numpy as np
>>> import pandas as pd
>>> NUM_COLUMNS=10
>>> NUM_ROWS=100_000
>>> df = pd.DataFrame(np.random.randint(0,100,size=(NUM_ROWS, NUM_COLUMNS)), columns=[f"COL_{i}" for i in range(NUM_COLUMNS)], index=pd.date_range('2000', periods=NUM_ROWS, freq='h'))

Get the library, write some data to it, and read it back:

>>> lib = ac['travel_data']
>>> lib.write("my_data", df)
>>> data = lib.read("my_data")

To find out more about working with data, visit our docs


Documentation

The source code for the ArcticDB docs are located in the docs folder, and are hosted at docs.arcticdb.io.

License

ArcticDB is released under a Business Source License 1.1 (BSL)

BSL features are free to use and the source code is available, but users may not use ArcticDB for production use or for a Database Service, without agreement with Man Group Operations Limited.

Use of ArcticDB in production or for a Database Service requires a paid for license from ArcticDB Limited and is licensed under the ArcticDB Software License Agreement. For more information please contact info@arcticdb.io.

The BSL is not certified as an open-source license, but most of the Open Source Initiative (OSI) criteria are met. Please see version conversion dates in the below table:

ArcticDB Version License Converts to Apache 2.0
1.0 Business Source License 1.1 Mar 16, 2025
1.2 Business Source License 1.1 May 22, 2025
1.3 Business Source License 1.1 Jun 9, 2025
1.4 Business Source License 1.1 Jun 23, 2025
1.5 Business Source License 1.1 Jul 11, 2025
1.6 Business Source License 1.1 Jul 25, 2025
2.0 Business Source License 1.1 Aug 29, 2025
3.0 Business Source License 1.1 Sep 13, 2025
4.0 Business Source License 1.1 Sep 27, 2025
4.1 Business Source License 1.1 Nov 1, 2025
4.2 Business Source License 1.1 Nov 12, 2025
4.3 Business Source License 1.1 Feb 7, 2026
4.4 Business Source License 1.1 Apr 5, 2026
4.5 Business Source License 1.1 Aug 14, 2026
5.0 Business Source License 1.1 Oct 31, 2026
5.1 Business Source License 1.1 Nov 15, 2026
5.2 Business Source License 1.1 Jan 27, 2027
5.3 Business Source License 1.1 Mar 24, 2027
5.4 Business Source License 1.1 Apr 28, 2027
5.5 Business Source License 1.1 May 13, 2027
5.6 Business Source License 1.1 May 26, 2027
5.7 Business Source License 1.1 Jun 2, 2027
5.8 Business Source License 1.1 Jun 9, 2027
5.9 Business Source License 1.1 Jul 7, 2027
5.10 Business Source License 1.1 Jul 29, 2027
6.1 Business Source License 1.1 Aug 20, 2027
6.2 Business Source License 1.1 Sep 8, 2027
6.3 Business Source License 1.1 Nov 3, 2027
6.5 Business Source License 1.1 Dec 11, 2027
6.6 Business Source License 1.1 Jan 6, 2028

Code of Conduct

Code of Conduct

This project has adopted a Code of Conduct. If you have any concerns about the Code, or behaviour that you have experienced in the project, please contact us at info@arcticdb.io.

Contributing/Building From Source

We welcome your contributions to help us improve and extend this project!

Please refer to the Contributing page and feel free to open issues on GitHub.

We are also always looking for feedback from our dedicated community! If you have used ArcticDB please let us know, we would love to hear about your experience!

Our release process is documented here.

Community

We would love to hear how your ArcticDB journey evolves, email us at info@arcticdb.io or come chat to us on Twitter!

Interested in learning more about ArcticDB? Head over to our blog!

Do you have any questions or issues? Chat to us and other users through our dedicated Slack Workspace - sign up for Slack access on our website.

Project details


Release history Release notifications | RSS feed

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distributions

No source distribution files available for this release.See tutorial on generating distribution archives.

Built Distributions

If you're not sure about the file name format, learn more about wheel file names.

arcticdb-6.11.1-cp314-cp314-win_amd64.whl (9.5 MB view details)

Uploaded CPython 3.14Windows x86-64

arcticdb-6.11.1-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (19.5 MB view details)

Uploaded CPython 3.14manylinux: glibc 2.17+ x86-64

arcticdb-6.11.1-cp314-cp314-macosx_15_0_arm64.whl (18.0 MB view details)

Uploaded CPython 3.14macOS 15.0+ ARM64

arcticdb-6.11.1-cp313-cp313-win_amd64.whl (9.2 MB view details)

Uploaded CPython 3.13Windows x86-64

arcticdb-6.11.1-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (19.5 MB view details)

Uploaded CPython 3.13manylinux: glibc 2.17+ x86-64

arcticdb-6.11.1-cp313-cp313-macosx_15_0_arm64.whl (18.1 MB view details)

Uploaded CPython 3.13macOS 15.0+ ARM64

arcticdb-6.11.1-cp312-cp312-win_amd64.whl (9.2 MB view details)

Uploaded CPython 3.12Windows x86-64

arcticdb-6.11.1-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (19.5 MB view details)

Uploaded CPython 3.12manylinux: glibc 2.17+ x86-64

arcticdb-6.11.1-cp312-cp312-macosx_15_0_arm64.whl (18.1 MB view details)

Uploaded CPython 3.12macOS 15.0+ ARM64

arcticdb-6.11.1-cp311-cp311-win_amd64.whl (9.2 MB view details)

Uploaded CPython 3.11Windows x86-64

arcticdb-6.11.1-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (19.5 MB view details)

Uploaded CPython 3.11manylinux: glibc 2.17+ x86-64

arcticdb-6.11.1-cp311-cp311-macosx_15_0_arm64.whl (18.1 MB view details)

Uploaded CPython 3.11macOS 15.0+ ARM64

arcticdb-6.11.1-cp310-cp310-win_amd64.whl (9.3 MB view details)

Uploaded CPython 3.10Windows x86-64

arcticdb-6.11.1-cp310-cp310-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (19.5 MB view details)

Uploaded CPython 3.10manylinux: glibc 2.17+ x86-64

arcticdb-6.11.1-cp310-cp310-macosx_15_0_arm64.whl (18.1 MB view details)

Uploaded CPython 3.10macOS 15.0+ ARM64

arcticdb-6.11.1-cp39-cp39-win_amd64.whl (9.2 MB view details)

Uploaded CPython 3.9Windows x86-64

arcticdb-6.11.1-cp39-cp39-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (19.5 MB view details)

Uploaded CPython 3.9manylinux: glibc 2.17+ x86-64

arcticdb-6.11.1-cp39-cp39-macosx_15_0_arm64.whl (18.1 MB view details)

Uploaded CPython 3.9macOS 15.0+ ARM64

File details

Details for the file arcticdb-6.11.1-cp314-cp314-win_amd64.whl.

File metadata

  • Download URL: arcticdb-6.11.1-cp314-cp314-win_amd64.whl
  • Upload date:
  • Size: 9.5 MB
  • Tags: CPython 3.14, Windows x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.2.0 CPython/3.10.12

File hashes

Hashes for arcticdb-6.11.1-cp314-cp314-win_amd64.whl
Algorithm Hash digest
SHA256 4cfb7c5bc5bb1fa03db8f274893340d04739ddd6c11c5f733a069e33f3388d31
MD5 d857b1d6ebdeb97181a380aab40b6f4e
BLAKE2b-256 03e2e81681307c1d3f155b4ef2164c01cfc5794622fd3d895037dbd2fee3f724

See more details on using hashes here.

File details

Details for the file arcticdb-6.11.1-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.whl.

File metadata

File hashes

Hashes for arcticdb-6.11.1-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.whl
Algorithm Hash digest
SHA256 78fbc1c85e42d838cd7ea86ae47b43742486c54a79953c1055761b0e0b432b11
MD5 b028563a8a6a3af4331f0c5e50d8339f
BLAKE2b-256 d367ae844e873e2a81ccf1587964cfc5f85a1b79470169adc30291692f06c8e2

See more details on using hashes here.

File details

Details for the file arcticdb-6.11.1-cp314-cp314-macosx_15_0_arm64.whl.

File metadata

File hashes

Hashes for arcticdb-6.11.1-cp314-cp314-macosx_15_0_arm64.whl
Algorithm Hash digest
SHA256 b79b1d6f7dbc3ba5248c96934befd4b38f67361f00f7f058e7309d6672c93bea
MD5 fb038f39d571ad65ef9cac96175a4e05
BLAKE2b-256 a7b703444fc1e2827dd53f15af7814c73d1e7304c789a95ab5a1342dfda4effa

See more details on using hashes here.

File details

Details for the file arcticdb-6.11.1-cp313-cp313-win_amd64.whl.

File metadata

  • Download URL: arcticdb-6.11.1-cp313-cp313-win_amd64.whl
  • Upload date:
  • Size: 9.2 MB
  • Tags: CPython 3.13, Windows x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.2.0 CPython/3.10.12

File hashes

Hashes for arcticdb-6.11.1-cp313-cp313-win_amd64.whl
Algorithm Hash digest
SHA256 67b9956adaad0802a60b84c0d306b3f64bd59e08a6a1da912af9d25313150758
MD5 aae8f2cd1a3782b042da50bed404ec76
BLAKE2b-256 9c7abc2d2c99e86166086ef76efc3485fb0bcaa9cfdfa8b24c3a339e927e7e89

See more details on using hashes here.

File details

Details for the file arcticdb-6.11.1-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.whl.

File metadata

File hashes

Hashes for arcticdb-6.11.1-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.whl
Algorithm Hash digest
SHA256 4ab241310fb49e8d5ae91344a2b07f2e16e78effaf93fb4553027f362fc13c61
MD5 71fab3815f68c3fae0b56cd14756900c
BLAKE2b-256 43bea94247ff66326f4a9f128c1704275b76afe72c8a54e006aeb16a82b28c92

See more details on using hashes here.

File details

Details for the file arcticdb-6.11.1-cp313-cp313-macosx_15_0_arm64.whl.

File metadata

File hashes

Hashes for arcticdb-6.11.1-cp313-cp313-macosx_15_0_arm64.whl
Algorithm Hash digest
SHA256 0ffd7ef2e027e1ca1e6c7e0bd21b5df74c97dec693e7b6cb1fa5c1043defd7e7
MD5 d07c4e5e3e7a74805733892b29510129
BLAKE2b-256 ad476ebfb7b9448e362edd3ace9db60c61f4db2f8ec29e1aca8aa47a54502bec

See more details on using hashes here.

File details

Details for the file arcticdb-6.11.1-cp312-cp312-win_amd64.whl.

File metadata

  • Download URL: arcticdb-6.11.1-cp312-cp312-win_amd64.whl
  • Upload date:
  • Size: 9.2 MB
  • Tags: CPython 3.12, Windows x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.2.0 CPython/3.10.12

File hashes

Hashes for arcticdb-6.11.1-cp312-cp312-win_amd64.whl
Algorithm Hash digest
SHA256 7695219c6bb002c68f49db8b2cc7af63740b7561dbd61dde55f868a9e7889555
MD5 1ba31dce0825b29761093dd768a556ac
BLAKE2b-256 a39a69b88fd54247ee760b60bb1b06804d4eeed41b1a3e9e31e2b9cef8452253

See more details on using hashes here.

File details

Details for the file arcticdb-6.11.1-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.whl.

File metadata

File hashes

Hashes for arcticdb-6.11.1-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.whl
Algorithm Hash digest
SHA256 3fc894f8dae4fafe22fd5d7e79b78a5ccaeb40971114575b6dbe4cbf6a65084c
MD5 358ab21123f3bc21370595fa47cca46d
BLAKE2b-256 dc846f1e432b71de5689eb7f6b577180a5707a1092cb58a2e7e56382837b01bf

See more details on using hashes here.

File details

Details for the file arcticdb-6.11.1-cp312-cp312-macosx_15_0_arm64.whl.

File metadata

File hashes

Hashes for arcticdb-6.11.1-cp312-cp312-macosx_15_0_arm64.whl
Algorithm Hash digest
SHA256 cf5ed068f796ffd1c3f02c6faea9697263523316dfa22505fe92b5ca217e9f0f
MD5 cd737e75655a205cb09f637d4da90260
BLAKE2b-256 1bee640a8e395a786901ecb49f468e4512d5aae001c3299f22edd28177d7fbc6

See more details on using hashes here.

File details

Details for the file arcticdb-6.11.1-cp311-cp311-win_amd64.whl.

File metadata

  • Download URL: arcticdb-6.11.1-cp311-cp311-win_amd64.whl
  • Upload date:
  • Size: 9.2 MB
  • Tags: CPython 3.11, Windows x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.2.0 CPython/3.10.12

File hashes

Hashes for arcticdb-6.11.1-cp311-cp311-win_amd64.whl
Algorithm Hash digest
SHA256 bf854f095b923b68efb644935b37ac196bbc36f17be64242107d4032764b42e4
MD5 834498bb1833ffe3dbfd11876807b863
BLAKE2b-256 2376bbaa59948a0b51face196c38ac2a8b0ea34d4be3a9c4e1adae2f5ba90785

See more details on using hashes here.

File details

Details for the file arcticdb-6.11.1-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.whl.

File metadata

File hashes

Hashes for arcticdb-6.11.1-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.whl
Algorithm Hash digest
SHA256 2f05ebaea8b0f46f49ca46f422187f09e49467727fb95e05b160ccb53e56f7be
MD5 5724d48b8a966ecad99fda9212644373
BLAKE2b-256 05cae648d613706d2bad99a101b9b67c1e6d1e86c81e90d32f6f8020eebd3c2e

See more details on using hashes here.

File details

Details for the file arcticdb-6.11.1-cp311-cp311-macosx_15_0_arm64.whl.

File metadata

File hashes

Hashes for arcticdb-6.11.1-cp311-cp311-macosx_15_0_arm64.whl
Algorithm Hash digest
SHA256 71adfd7fcbaee80d38aa03685736f1dccc35d3de19573fcd58d9536bd8102e76
MD5 77c6951d4eb0dfe22bc573cadf898750
BLAKE2b-256 bdab5e41e7212ce666158c8c07e24ced669928f404559d7082290c9dd4d77787

See more details on using hashes here.

File details

Details for the file arcticdb-6.11.1-cp310-cp310-win_amd64.whl.

File metadata

  • Download URL: arcticdb-6.11.1-cp310-cp310-win_amd64.whl
  • Upload date:
  • Size: 9.3 MB
  • Tags: CPython 3.10, Windows x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.2.0 CPython/3.10.12

File hashes

Hashes for arcticdb-6.11.1-cp310-cp310-win_amd64.whl
Algorithm Hash digest
SHA256 0c97e3766e0b2c2edde6a1af61b36629fca29377166068dbf3fa3dffaec5a21e
MD5 762cabd65a9a66122099c8eff3f34e3e
BLAKE2b-256 959a53d5f77e3e6a02c51a41fb11deb040fecababbb4de812f2f7807cffe0be9

See more details on using hashes here.

File details

Details for the file arcticdb-6.11.1-cp310-cp310-manylinux2014_x86_64.manylinux_2_17_x86_64.whl.

File metadata

File hashes

Hashes for arcticdb-6.11.1-cp310-cp310-manylinux2014_x86_64.manylinux_2_17_x86_64.whl
Algorithm Hash digest
SHA256 81331a68e427a60ffef134a96a9ee27b21c847f60f376c9c1807c70b2c3b9277
MD5 d078c05e46508b5875dbae06dcc1b4ce
BLAKE2b-256 e2da346467f71d649d08c27b3c9eda4f5d27cf3776cb8adc883518332a089f5d

See more details on using hashes here.

File details

Details for the file arcticdb-6.11.1-cp310-cp310-macosx_15_0_arm64.whl.

File metadata

File hashes

Hashes for arcticdb-6.11.1-cp310-cp310-macosx_15_0_arm64.whl
Algorithm Hash digest
SHA256 c083daf9d6f31655cdf2c5346ccdae0a78333b34aa347fedbd12b5ae44d4c720
MD5 ecd9d02b909b29c042a18a78d6ce6142
BLAKE2b-256 0e57bc50a70d92df5ac65625e3a0b912aa9bd31a992f873decf60f2c8b5e1480

See more details on using hashes here.

File details

Details for the file arcticdb-6.11.1-cp39-cp39-win_amd64.whl.

File metadata

  • Download URL: arcticdb-6.11.1-cp39-cp39-win_amd64.whl
  • Upload date:
  • Size: 9.2 MB
  • Tags: CPython 3.9, Windows x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.2.0 CPython/3.10.12

File hashes

Hashes for arcticdb-6.11.1-cp39-cp39-win_amd64.whl
Algorithm Hash digest
SHA256 68fcabf8b1a15e993c7127da1941469e191da8e9f00476ff494580792c5285ae
MD5 3d94a1d382228e918397466170cd01c4
BLAKE2b-256 4308722d46b51acd8777a1d47ac7a049e70f174f8a5a4248b7ab6970be72807d

See more details on using hashes here.

File details

Details for the file arcticdb-6.11.1-cp39-cp39-manylinux2014_x86_64.manylinux_2_17_x86_64.whl.

File metadata

File hashes

Hashes for arcticdb-6.11.1-cp39-cp39-manylinux2014_x86_64.manylinux_2_17_x86_64.whl
Algorithm Hash digest
SHA256 fac4e778a9abc1ead903129fef1f5945a69f789fb30f33389521ec51f9b3fd59
MD5 ce79801a396354a0f77d5993d4007d28
BLAKE2b-256 e90de4369c449173dee216bdfd109bbedf60e582f83e0a1219667d154050151b

See more details on using hashes here.

File details

Details for the file arcticdb-6.11.1-cp39-cp39-macosx_15_0_arm64.whl.

File metadata

File hashes

Hashes for arcticdb-6.11.1-cp39-cp39-macosx_15_0_arm64.whl
Algorithm Hash digest
SHA256 eb734f1f0fdbb9d095b2454add30184ab16507870e553013c167af1496d0008f
MD5 56daae0e6882bc5b5ce389dda70d5a2f
BLAKE2b-256 9f269bf5ec341ebe00e889fb6a7846568f3c21a896382a1dbc541ea66fe5c7c2

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page