Skip to main content

ArcticDB DataFrame Database

Project description


Three minute ArcticDB demo from PyQuantNews

Options data in ArcticDB

ArcticDB Website | 📘 ArcticDB Docs | 📰: ArcticDB Blog | Press Release | Press Release | Community


ArcticDB is a high performance, serverless DataFrame database built for the Python Data Science ecosystem. Launched in March 2023, it is the successor to Arctic.

Use of ArcticDB in production (including business or commercial environments) or for a Database Service requires a paid for license from ArcticDB Limited . Please contact info@arcticdb.io for further details.

ArcticDB offers an intuitive Python-centric API enabling you to read and write Pandas DataFrames to S3 or LMDB utilising a fast C++ data-processing and compression engine.

ArcticDB allows you to:

  • Pandas in, Pandas out: Read and write Pandas DataFrames, NumPy arrays and native types to S3 and LMDB without leaving Python.
  • Built for time-series data: Efficiently index and query time-series data across billions of rows
  • Time travel: Travel back in time to see previous versions of your data and create customizable snapshots of the database
  • Schemaless Database: Append, update and modify data without being constrained by the existing schema
  • Optimised for streaming data: Built in support for efficient sparse data storage
  • Powerful processing: Filter, aggregate and create new columns on-the-fly with a Pandas-like syntax
  • C++ efficiency: Accelerate analytics though concurrency in the C++ data-processing engine

ArcticDB handles data that is big in both row count and column count, so a 20-year history of more than 400,000 unique securities can be stored in a single symbol. Each symbol is maintained as a separate entity with no shared data which means ArcticDB can scale horizontally across symbols, maximising the performance potential of your compute, storage and network.

ArcticDB is designed from the outset to be resilient; there is no single point of failure, and persistent data structures in the storage mean that once a version of a symbol has been written, it can never be corrupted by subsequent updates. Pulling compressed data directly from storage to the client means that there is no server to overload, so your data is always available when you need it.

Quickstart

Prebuilt binary availability (for the current version)

PyPI (Python 3.9 - 3.14) conda-forge (Python 3.10 - 3.14)
Linux x86_64 ✔️ ✔️
Linux arm64 ✔️
Windows x86_64 ✔️ ✔️
MacOS x86_64 ✔️
MacOS arm64 ✔️ ✔️

Storage compatibility

Linux Windows Mac
S3 ✔️ ✔️ ✔️
LMDB ✔️ ✔️ ✔️
Azure Blob Storage ✔️ ✔️ ✔️

We have tested against the following S3 backends:

  • AWS S3
  • Ceph
  • MinIO on Linux
  • Pure Storage S3
  • Scality S3
  • VAST Data S3

Installation

Install ArcticDB:

$ pip install arcticdb

or using conda-forge

$ conda install -c conda-forge arcticdb

Import ArcticDB:

>>> import arcticdb as adb

Create an instance on your S3 storage (with or without explicit credentials):

# Leave AWS to derive credential information
>>> ac = adb.Arctic('s3://MY_ENDPOINT:MY_BUCKET?aws_auth=true')

# Manually specify creds
>>> ac = adb.Arctic('s3://MY_ENDPOINT:MY_BUCKET?region=YOUR_REGION&access=ABCD&secret=DCBA')

Or create an instance on your local disk:

>>> ac = adb.Arctic("lmdb:///<path>")

Create your first library and list the libraries in the instance:

>>> ac.create_library('travel_data')
>>> ac.list_libraries()

Create a test dataframe:

>>> import numpy as np
>>> import pandas as pd
>>> NUM_COLUMNS=10
>>> NUM_ROWS=100_000
>>> df = pd.DataFrame(np.random.randint(0,100,size=(NUM_ROWS, NUM_COLUMNS)), columns=[f"COL_{i}" for i in range(NUM_COLUMNS)], index=pd.date_range('2000', periods=NUM_ROWS, freq='h'))

Get the library, write some data to it, and read it back:

>>> lib = ac['travel_data']
>>> lib.write("my_data", df)
>>> data = lib.read("my_data")

To find out more about working with data, visit our docs


Documentation

The source code for the ArcticDB docs are located in the docs folder, and are hosted at docs.arcticdb.io.

License

ArcticDB is released under a Business Source License 1.1 (BSL)

BSL features are free to use and the source code is available, but users may not use ArcticDB for production use or for a Database Service, without agreement with Man Group Operations Limited.

Use of ArcticDB in production or for a Database Service requires a paid for license from ArcticDB Limited and is licensed under the ArcticDB Software License Agreement. For more information please contact info@arcticdb.io.

The BSL is not certified as an open-source license, but most of the Open Source Initiative (OSI) criteria are met. Please see version conversion dates in the below table:

ArcticDB Version License Converts to Apache 2.0
1.0 Business Source License 1.1 Mar 16, 2025
1.2 Business Source License 1.1 May 22, 2025
1.3 Business Source License 1.1 Jun 9, 2025
1.4 Business Source License 1.1 Jun 23, 2025
1.5 Business Source License 1.1 Jul 11, 2025
1.6 Business Source License 1.1 Jul 25, 2025
2.0 Business Source License 1.1 Aug 29, 2025
3.0 Business Source License 1.1 Sep 13, 2025
4.0 Business Source License 1.1 Sep 27, 2025
4.1 Business Source License 1.1 Nov 1, 2025
4.2 Business Source License 1.1 Nov 12, 2025
4.3 Business Source License 1.1 Feb 7, 2026
4.4 Business Source License 1.1 Apr 5, 2026
4.5 Business Source License 1.1 Aug 14, 2026
5.0 Business Source License 1.1 Oct 31, 2026
5.1 Business Source License 1.1 Nov 15, 2026
5.2 Business Source License 1.1 Jan 27, 2027
5.3 Business Source License 1.1 Mar 24, 2027
5.4 Business Source License 1.1 Apr 28, 2027
5.5 Business Source License 1.1 May 13, 2027
5.6 Business Source License 1.1 May 26, 2027
5.7 Business Source License 1.1 Jun 2, 2027
5.8 Business Source License 1.1 Jun 9, 2027
5.9 Business Source License 1.1 Jul 7, 2027
5.10 Business Source License 1.1 Jul 29, 2027
6.1 Business Source License 1.1 Aug 20, 2027
6.2 Business Source License 1.1 Sep 8, 2027
6.3 Business Source License 1.1 Nov 3, 2027
6.5 Business Source License 1.1 Dec 11, 2027
6.6 Business Source License 1.1 Jan 6, 2028

Code of Conduct

Code of Conduct

This project has adopted a Code of Conduct. If you have any concerns about the Code, or behaviour that you have experienced in the project, please contact us at info@arcticdb.io.

Contributing/Building From Source

We welcome your contributions to help us improve and extend this project!

Please refer to the Contributing page and feel free to open issues on GitHub.

We are also always looking for feedback from our dedicated community! If you have used ArcticDB please let us know, we would love to hear about your experience!

Our release process is documented here.

Community

We would love to hear how your ArcticDB journey evolves, email us at info@arcticdb.io or come chat to us on Twitter!

Interested in learning more about ArcticDB? Head over to our blog!

Do you have any questions or issues? Chat to us and other users through our dedicated Slack Workspace - sign up for Slack access on our website.

Project details


Release history Release notifications | RSS feed

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distributions

No source distribution files available for this release.See tutorial on generating distribution archives.

Built Distributions

If you're not sure about the file name format, learn more about wheel file names.

arcticdb-6.18.0-cp314-cp314-win_amd64.whl (9.6 MB view details)

Uploaded CPython 3.14Windows x86-64

arcticdb-6.18.0-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (20.7 MB view details)

Uploaded CPython 3.14manylinux: glibc 2.17+ x86-64

arcticdb-6.18.0-cp314-cp314-macosx_15_0_arm64.whl (19.0 MB view details)

Uploaded CPython 3.14macOS 15.0+ ARM64

arcticdb-6.18.0-cp313-cp313-win_amd64.whl (9.4 MB view details)

Uploaded CPython 3.13Windows x86-64

arcticdb-6.18.0-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (20.7 MB view details)

Uploaded CPython 3.13manylinux: glibc 2.17+ x86-64

arcticdb-6.18.0-cp313-cp313-macosx_15_0_arm64.whl (19.1 MB view details)

Uploaded CPython 3.13macOS 15.0+ ARM64

arcticdb-6.18.0-cp312-cp312-win_amd64.whl (9.4 MB view details)

Uploaded CPython 3.12Windows x86-64

arcticdb-6.18.0-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (20.7 MB view details)

Uploaded CPython 3.12manylinux: glibc 2.17+ x86-64

arcticdb-6.18.0-cp312-cp312-macosx_15_0_arm64.whl (19.1 MB view details)

Uploaded CPython 3.12macOS 15.0+ ARM64

arcticdb-6.18.0-cp311-cp311-win_amd64.whl (9.4 MB view details)

Uploaded CPython 3.11Windows x86-64

arcticdb-6.18.0-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (20.7 MB view details)

Uploaded CPython 3.11manylinux: glibc 2.17+ x86-64

arcticdb-6.18.0-cp311-cp311-macosx_15_0_arm64.whl (19.1 MB view details)

Uploaded CPython 3.11macOS 15.0+ ARM64

arcticdb-6.18.0-cp310-cp310-win_amd64.whl (9.4 MB view details)

Uploaded CPython 3.10Windows x86-64

arcticdb-6.18.0-cp310-cp310-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (20.8 MB view details)

Uploaded CPython 3.10manylinux: glibc 2.17+ x86-64

arcticdb-6.18.0-cp310-cp310-macosx_15_0_arm64.whl (19.1 MB view details)

Uploaded CPython 3.10macOS 15.0+ ARM64

arcticdb-6.18.0-cp39-cp39-win_amd64.whl (9.4 MB view details)

Uploaded CPython 3.9Windows x86-64

arcticdb-6.18.0-cp39-cp39-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (20.8 MB view details)

Uploaded CPython 3.9manylinux: glibc 2.17+ x86-64

arcticdb-6.18.0-cp39-cp39-macosx_15_0_arm64.whl (19.1 MB view details)

Uploaded CPython 3.9macOS 15.0+ ARM64

File details

Details for the file arcticdb-6.18.0-cp314-cp314-win_amd64.whl.

File metadata

  • Download URL: arcticdb-6.18.0-cp314-cp314-win_amd64.whl
  • Upload date:
  • Size: 9.6 MB
  • Tags: CPython 3.14, Windows x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.2.0 CPython/3.10.12

File hashes

Hashes for arcticdb-6.18.0-cp314-cp314-win_amd64.whl
Algorithm Hash digest
SHA256 ffcb006df361d0d6a64964f7c5537cdaf7fd66cac65355c4c6a1c359e79d9b6e
MD5 3a131024a34f08b5ffb9ce73aee2c421
BLAKE2b-256 dd73dd85cdfa4ab520e33084d6fabbb554165cc723990002ed4040e384ff89a7

See more details on using hashes here.

File details

Details for the file arcticdb-6.18.0-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.whl.

File metadata

File hashes

Hashes for arcticdb-6.18.0-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.whl
Algorithm Hash digest
SHA256 7890bfe6014cd8999618f99c7f0b6538766b107233fbb0c6fe5461bd7aa55ced
MD5 b5a87b013b7e1aff5a28b9523eecadc6
BLAKE2b-256 52b7a6f0e1fe002d9a14b429d3b48c5bcd7847d6e91e14db92314871cd58776d

See more details on using hashes here.

File details

Details for the file arcticdb-6.18.0-cp314-cp314-macosx_15_0_arm64.whl.

File metadata

File hashes

Hashes for arcticdb-6.18.0-cp314-cp314-macosx_15_0_arm64.whl
Algorithm Hash digest
SHA256 73f11011dd130ff1ed4e73bcedd72f6cf82b1187076c7bb0ce4973e6b455cffa
MD5 f50a8ac42845f054dc553d1de1422d70
BLAKE2b-256 a901f43a092ed9679edd9a9e54baf41d5608870a2cc6ae242109672cf80b4a52

See more details on using hashes here.

File details

Details for the file arcticdb-6.18.0-cp313-cp313-win_amd64.whl.

File metadata

  • Download URL: arcticdb-6.18.0-cp313-cp313-win_amd64.whl
  • Upload date:
  • Size: 9.4 MB
  • Tags: CPython 3.13, Windows x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.2.0 CPython/3.10.12

File hashes

Hashes for arcticdb-6.18.0-cp313-cp313-win_amd64.whl
Algorithm Hash digest
SHA256 18387c8fa47e3d520858910dc27295fa73aeffe8dd29398581534f229d3fff6a
MD5 8f0152f672e5157a6b029d304fdc3fd0
BLAKE2b-256 057c4a8c583a983cb94529a1ea8e1ab2d2fa5514520842394e65dd2eab7b6d77

See more details on using hashes here.

File details

Details for the file arcticdb-6.18.0-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.whl.

File metadata

File hashes

Hashes for arcticdb-6.18.0-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.whl
Algorithm Hash digest
SHA256 756a452055f768eb78ecad946cc2f786578fe070b064ef6047c8b8b3748e024a
MD5 f9513224da0b8df9b6c5a0817a468c0c
BLAKE2b-256 797c9419443d4fe9afb1e6155857ccd448f03835d07db750a490d30bbf8fa498

See more details on using hashes here.

File details

Details for the file arcticdb-6.18.0-cp313-cp313-macosx_15_0_arm64.whl.

File metadata

File hashes

Hashes for arcticdb-6.18.0-cp313-cp313-macosx_15_0_arm64.whl
Algorithm Hash digest
SHA256 acf471053794a351b6903d8c9652e748cd3bd0341c9b34adfbfd283851d675e1
MD5 e9975d4e88975dc0879041626a9f8d16
BLAKE2b-256 779a63e8dcc35ef978d445f5a5f854fc4db7363792acf9c8f39f429793fecc83

See more details on using hashes here.

File details

Details for the file arcticdb-6.18.0-cp312-cp312-win_amd64.whl.

File metadata

  • Download URL: arcticdb-6.18.0-cp312-cp312-win_amd64.whl
  • Upload date:
  • Size: 9.4 MB
  • Tags: CPython 3.12, Windows x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.2.0 CPython/3.10.12

File hashes

Hashes for arcticdb-6.18.0-cp312-cp312-win_amd64.whl
Algorithm Hash digest
SHA256 a4f924adf940400abe730ce24e224aee65030fd88ea6a5ce1a7352a1c69e8611
MD5 85aef3c319a82718fce5fdc69094ea10
BLAKE2b-256 8893bb9e5774850b08f039f35a770f7e07dc35acac4f8371d9fead2621dd16d1

See more details on using hashes here.

File details

Details for the file arcticdb-6.18.0-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.whl.

File metadata

File hashes

Hashes for arcticdb-6.18.0-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.whl
Algorithm Hash digest
SHA256 004a54e926d940aca5ab356843a39a2a05937732dc9555ed2b2567701cff96d6
MD5 7cdb7018c4c1246cc43117f4d44fb8b3
BLAKE2b-256 4a2dcbffc4f2e5ce2ddda547ded8dc5b0e7a7f9ce583b22252f438887d2aed6f

See more details on using hashes here.

File details

Details for the file arcticdb-6.18.0-cp312-cp312-macosx_15_0_arm64.whl.

File metadata

File hashes

Hashes for arcticdb-6.18.0-cp312-cp312-macosx_15_0_arm64.whl
Algorithm Hash digest
SHA256 59ccff7d77e84056ade62063377751eca008b0ea750e96b64b5da0635d2987b6
MD5 7d146f895880bdbd456714b8bac2ced6
BLAKE2b-256 e569d5d116bb7a8b1f6c8209ca9c3b99defaa67bdea3a06a20f0dfa1bc097111

See more details on using hashes here.

File details

Details for the file arcticdb-6.18.0-cp311-cp311-win_amd64.whl.

File metadata

  • Download URL: arcticdb-6.18.0-cp311-cp311-win_amd64.whl
  • Upload date:
  • Size: 9.4 MB
  • Tags: CPython 3.11, Windows x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.2.0 CPython/3.10.12

File hashes

Hashes for arcticdb-6.18.0-cp311-cp311-win_amd64.whl
Algorithm Hash digest
SHA256 14c7c2487baee48c3d65888d1bc21eb63e14df8456c0dadd3db179da893d9960
MD5 8c17c40a1a97bd4daed3f616e1582ad3
BLAKE2b-256 76ab9de9d7370558e0e7b550f684385522c988b82216f193892645d0a065b653

See more details on using hashes here.

File details

Details for the file arcticdb-6.18.0-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.whl.

File metadata

File hashes

Hashes for arcticdb-6.18.0-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.whl
Algorithm Hash digest
SHA256 d809066a729fd839f18cf872d2b9cd1b67dcb5861a6c4b3725438147ab507c5f
MD5 4bc2bb3b531b823c0f03f8a7351d3ae9
BLAKE2b-256 4bfc659fab0908b333bc04ef1b01a429efd3ebb2b1652bd26d66317614c261c4

See more details on using hashes here.

File details

Details for the file arcticdb-6.18.0-cp311-cp311-macosx_15_0_arm64.whl.

File metadata

File hashes

Hashes for arcticdb-6.18.0-cp311-cp311-macosx_15_0_arm64.whl
Algorithm Hash digest
SHA256 afee6069622e471da1c24a8f233024094b78723f0e1a2d2e68bd470e42cbb927
MD5 90e6337245de767aed4465010d7a3167
BLAKE2b-256 481721181d46f044f35640bfc3742f912eaab2fd00218ed30d144556967ceaa9

See more details on using hashes here.

File details

Details for the file arcticdb-6.18.0-cp310-cp310-win_amd64.whl.

File metadata

  • Download URL: arcticdb-6.18.0-cp310-cp310-win_amd64.whl
  • Upload date:
  • Size: 9.4 MB
  • Tags: CPython 3.10, Windows x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.2.0 CPython/3.10.12

File hashes

Hashes for arcticdb-6.18.0-cp310-cp310-win_amd64.whl
Algorithm Hash digest
SHA256 98ced566d55474d7990c37b07a2c2ff634afdc90ff691c167cb56206c26c07b5
MD5 8a1084d8772bd78f63ce90e5d312b9bd
BLAKE2b-256 e46b150b58dac36410f090607de8030200cbd6045855c3af0f5a989b07b6865b

See more details on using hashes here.

File details

Details for the file arcticdb-6.18.0-cp310-cp310-manylinux2014_x86_64.manylinux_2_17_x86_64.whl.

File metadata

File hashes

Hashes for arcticdb-6.18.0-cp310-cp310-manylinux2014_x86_64.manylinux_2_17_x86_64.whl
Algorithm Hash digest
SHA256 5247ba8bfc2e4755c1adf90417298a48abcf024745ac469dc3020dea28e7f958
MD5 0c911db4ab813687b605f6a02daf2ca8
BLAKE2b-256 d60f84643a3ee7759955d4975ac29c09442e07c40ae7762e8d60f075a6ab4138

See more details on using hashes here.

File details

Details for the file arcticdb-6.18.0-cp310-cp310-macosx_15_0_arm64.whl.

File metadata

File hashes

Hashes for arcticdb-6.18.0-cp310-cp310-macosx_15_0_arm64.whl
Algorithm Hash digest
SHA256 a77bfc02f9445d9f193fa8e0354bc3a2550971d02964906ea5d1100fb34d72ee
MD5 35f4300d8faada816ef445c5135e9d6a
BLAKE2b-256 feae173bcb3597ccbc55ce816f5d2df9403d870a055a99a9926994ce89d5c473

See more details on using hashes here.

File details

Details for the file arcticdb-6.18.0-cp39-cp39-win_amd64.whl.

File metadata

  • Download URL: arcticdb-6.18.0-cp39-cp39-win_amd64.whl
  • Upload date:
  • Size: 9.4 MB
  • Tags: CPython 3.9, Windows x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.2.0 CPython/3.10.12

File hashes

Hashes for arcticdb-6.18.0-cp39-cp39-win_amd64.whl
Algorithm Hash digest
SHA256 4a7466d6b4fd925a00c985b54f0ffbfa12fd7d8dfe480aa7cd8dc2f0c485b37e
MD5 fed78099978eb011d91b5a41fc45523f
BLAKE2b-256 cd9c9db94fcb2ced37bd8d8ad81f06e50acd217ac24800472f665ab1e2bc3dbe

See more details on using hashes here.

File details

Details for the file arcticdb-6.18.0-cp39-cp39-manylinux2014_x86_64.manylinux_2_17_x86_64.whl.

File metadata

File hashes

Hashes for arcticdb-6.18.0-cp39-cp39-manylinux2014_x86_64.manylinux_2_17_x86_64.whl
Algorithm Hash digest
SHA256 cb4217f3c4064cd2b4215678441d09497bd3343e83aabefe09c27a9fbbec9fc5
MD5 a2796255e3513ff6dc5fac38d9bb7b40
BLAKE2b-256 a2c5f824d525421c9fa36ecf83885cc06ea598c54fd0a6df3524fb90224452fa

See more details on using hashes here.

File details

Details for the file arcticdb-6.18.0-cp39-cp39-macosx_15_0_arm64.whl.

File metadata

File hashes

Hashes for arcticdb-6.18.0-cp39-cp39-macosx_15_0_arm64.whl
Algorithm Hash digest
SHA256 441b4dade613065fd81d7a05470cf102062f7953e48797c51c4642af27222448
MD5 b362273d086c0dd1ef63f08347080c86
BLAKE2b-256 e4c3baf5c0d6dc330d457e5857d173666a4a07591f525897e8cdc54b08585282

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page