Skip to main content

ArcticDB DataFrame Database

Project description


Three minute ArcticDB demo from PyQuantNews

Options data in ArcticDB

ArcticDB Website | 📘 ArcticDB Docs | 📰: ArcticDB Blog | Press Release | Press Release | Community


ArcticDB is a high performance, serverless DataFrame database built for the Python Data Science ecosystem. Launched in March 2023, it is the successor to Arctic.

Use of ArcticDB in production (including business or commercial environments) or for a Database Service requires a paid for license from ArcticDB Limited . Please contact info@arcticdb.io for further details.

ArcticDB offers an intuitive Python-centric API enabling you to read and write Pandas DataFrames to S3 or LMDB utilising a fast C++ data-processing and compression engine.

ArcticDB allows you to:

  • Pandas in, Pandas out: Read and write Pandas DataFrames, NumPy arrays and native types to S3 and LMDB without leaving Python.
  • Built for time-series data: Efficiently index and query time-series data across billions of rows
  • Time travel: Travel back in time to see previous versions of your data and create customizable snapshots of the database
  • Schemaless Database: Append, update and modify data without being constrained by the existing schema
  • Optimised for streaming data: Built in support for efficient sparse data storage
  • Powerful processing: Filter, aggregate and create new columns on-the-fly with a Pandas-like syntax
  • C++ efficiency: Accelerate analytics though concurrency in the C++ data-processing engine

ArcticDB handles data that is big in both row count and column count, so a 20-year history of more than 400,000 unique securities can be stored in a single symbol. Each symbol is maintained as a separate entity with no shared data which means ArcticDB can scale horizontally across symbols, maximising the performance potential of your compute, storage and network.

ArcticDB is designed from the outset to be resilient; there is no single point of failure, and persistent data structures in the storage mean that once a version of a symbol has been written, it can never be corrupted by subsequent updates. Pulling compressed data directly from storage to the client means that there is no server to overload, so your data is always available when you need it.

Quickstart

Prebuilt binary availability (for the current version)

PyPI (Python 3.9 - 3.14) conda-forge (Python 3.10 - 3.14)
Linux x86_64 ✔️ ✔️
Linux arm64 ✔️
Windows x86_64 ✔️ ✔️
MacOS x86_64 ✔️
MacOS arm64 ✔️ ✔️

Storage compatibility

Linux Windows Mac
S3 ✔️ ✔️ ✔️
LMDB ✔️ ✔️ ✔️
Azure Blob Storage ✔️ ✔️ ✔️

We have tested against the following S3 backends:

  • AWS S3
  • Ceph
  • MinIO on Linux
  • Pure Storage S3
  • Scality S3
  • VAST Data S3

Installation

Install ArcticDB:

$ pip install arcticdb

or using conda-forge

$ conda install -c conda-forge arcticdb

Import ArcticDB:

>>> import arcticdb as adb

Create an instance on your S3 storage (with or without explicit credentials):

# Leave AWS to derive credential information
>>> ac = adb.Arctic('s3://MY_ENDPOINT:MY_BUCKET?aws_auth=true')

# Manually specify creds
>>> ac = adb.Arctic('s3://MY_ENDPOINT:MY_BUCKET?region=YOUR_REGION&access=ABCD&secret=DCBA')

Or create an instance on your local disk:

>>> ac = adb.Arctic("lmdb:///<path>")

Create your first library and list the libraries in the instance:

>>> ac.create_library('travel_data')
>>> ac.list_libraries()

Create a test dataframe:

>>> import numpy as np
>>> import pandas as pd
>>> NUM_COLUMNS=10
>>> NUM_ROWS=100_000
>>> df = pd.DataFrame(np.random.randint(0,100,size=(NUM_ROWS, NUM_COLUMNS)), columns=[f"COL_{i}" for i in range(NUM_COLUMNS)], index=pd.date_range('2000', periods=NUM_ROWS, freq='h'))

Get the library, write some data to it, and read it back:

>>> lib = ac['travel_data']
>>> lib.write("my_data", df)
>>> data = lib.read("my_data")

To find out more about working with data, visit our docs


Documentation

The source code for the ArcticDB docs are located in the docs folder, and are hosted at docs.arcticdb.io.

License

ArcticDB is released under a Business Source License 1.1 (BSL)

BSL features are free to use and the source code is available, but users may not use ArcticDB for production use or for a Database Service, without agreement with Man Group Operations Limited.

Use of ArcticDB in production or for a Database Service requires a paid for license from ArcticDB Limited and is licensed under the ArcticDB Software License Agreement. For more information please contact info@arcticdb.io.

The BSL is not certified as an open-source license, but most of the Open Source Initiative (OSI) criteria are met. Please see version conversion dates in the below table:

ArcticDB Version License Converts to Apache 2.0
1.0 Business Source License 1.1 Mar 16, 2025
1.2 Business Source License 1.1 May 22, 2025
1.3 Business Source License 1.1 Jun 9, 2025
1.4 Business Source License 1.1 Jun 23, 2025
1.5 Business Source License 1.1 Jul 11, 2025
1.6 Business Source License 1.1 Jul 25, 2025
2.0 Business Source License 1.1 Aug 29, 2025
3.0 Business Source License 1.1 Sep 13, 2025
4.0 Business Source License 1.1 Sep 27, 2025
4.1 Business Source License 1.1 Nov 1, 2025
4.2 Business Source License 1.1 Nov 12, 2025
4.3 Business Source License 1.1 Feb 7, 2026
4.4 Business Source License 1.1 Apr 5, 2026
4.5 Business Source License 1.1 Aug 14, 2026
5.0 Business Source License 1.1 Oct 31, 2026
5.1 Business Source License 1.1 Nov 15, 2026
5.2 Business Source License 1.1 Jan 27, 2027
5.3 Business Source License 1.1 Mar 24, 2027
5.4 Business Source License 1.1 Apr 28, 2027
5.5 Business Source License 1.1 May 13, 2027
5.6 Business Source License 1.1 May 26, 2027
5.7 Business Source License 1.1 Jun 2, 2027
5.8 Business Source License 1.1 Jun 9, 2027
5.9 Business Source License 1.1 Jul 7, 2027
5.10 Business Source License 1.1 Jul 29, 2027
6.1 Business Source License 1.1 Aug 20, 2027
6.2 Business Source License 1.1 Sep 8, 2027
6.3 Business Source License 1.1 Nov 3, 2027
6.5 Business Source License 1.1 Dec 11, 2027
6.6 Business Source License 1.1 Jan 6, 2028

Code of Conduct

Code of Conduct

This project has adopted a Code of Conduct. If you have any concerns about the Code, or behaviour that you have experienced in the project, please contact us at info@arcticdb.io.

Contributing/Building From Source

We welcome your contributions to help us improve and extend this project!

Please refer to the Contributing page and feel free to open issues on GitHub.

We are also always looking for feedback from our dedicated community! If you have used ArcticDB please let us know, we would love to hear about your experience!

Our release process is documented here.

Community

We would love to hear how your ArcticDB journey evolves, email us at info@arcticdb.io or come chat to us on Twitter!

Interested in learning more about ArcticDB? Head over to our blog!

Do you have any questions or issues? Chat to us and other users through our dedicated Slack Workspace - sign up for Slack access on our website.

Project details


Release history Release notifications | RSS feed

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distributions

No source distribution files available for this release.See tutorial on generating distribution archives.

Built Distributions

If you're not sure about the file name format, learn more about wheel file names.

arcticdb-6.15.0-cp314-cp314-win_amd64.whl (9.7 MB view details)

Uploaded CPython 3.14Windows x86-64

arcticdb-6.15.0-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (19.8 MB view details)

Uploaded CPython 3.14manylinux: glibc 2.17+ x86-64

arcticdb-6.15.0-cp314-cp314-macosx_15_0_arm64.whl (18.3 MB view details)

Uploaded CPython 3.14macOS 15.0+ ARM64

arcticdb-6.15.0-cp313-cp313-win_amd64.whl (9.4 MB view details)

Uploaded CPython 3.13Windows x86-64

arcticdb-6.15.0-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (19.9 MB view details)

Uploaded CPython 3.13manylinux: glibc 2.17+ x86-64

arcticdb-6.15.0-cp313-cp313-macosx_15_0_arm64.whl (18.3 MB view details)

Uploaded CPython 3.13macOS 15.0+ ARM64

arcticdb-6.15.0-cp312-cp312-win_amd64.whl (9.4 MB view details)

Uploaded CPython 3.12Windows x86-64

arcticdb-6.15.0-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (19.9 MB view details)

Uploaded CPython 3.12manylinux: glibc 2.17+ x86-64

arcticdb-6.15.0-cp312-cp312-macosx_15_0_arm64.whl (18.4 MB view details)

Uploaded CPython 3.12macOS 15.0+ ARM64

arcticdb-6.15.0-cp311-cp311-win_amd64.whl (9.4 MB view details)

Uploaded CPython 3.11Windows x86-64

arcticdb-6.15.0-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (19.9 MB view details)

Uploaded CPython 3.11manylinux: glibc 2.17+ x86-64

arcticdb-6.15.0-cp311-cp311-macosx_15_0_arm64.whl (18.4 MB view details)

Uploaded CPython 3.11macOS 15.0+ ARM64

arcticdb-6.15.0-cp310-cp310-win_amd64.whl (9.5 MB view details)

Uploaded CPython 3.10Windows x86-64

arcticdb-6.15.0-cp310-cp310-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (19.9 MB view details)

Uploaded CPython 3.10manylinux: glibc 2.17+ x86-64

arcticdb-6.15.0-cp310-cp310-macosx_15_0_arm64.whl (18.4 MB view details)

Uploaded CPython 3.10macOS 15.0+ ARM64

arcticdb-6.15.0-cp39-cp39-win_amd64.whl (9.4 MB view details)

Uploaded CPython 3.9Windows x86-64

arcticdb-6.15.0-cp39-cp39-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (19.9 MB view details)

Uploaded CPython 3.9manylinux: glibc 2.17+ x86-64

arcticdb-6.15.0-cp39-cp39-macosx_15_0_arm64.whl (18.4 MB view details)

Uploaded CPython 3.9macOS 15.0+ ARM64

File details

Details for the file arcticdb-6.15.0-cp314-cp314-win_amd64.whl.

File metadata

  • Download URL: arcticdb-6.15.0-cp314-cp314-win_amd64.whl
  • Upload date:
  • Size: 9.7 MB
  • Tags: CPython 3.14, Windows x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.2.0 CPython/3.10.12

File hashes

Hashes for arcticdb-6.15.0-cp314-cp314-win_amd64.whl
Algorithm Hash digest
SHA256 47d1f9bdc664892eac4c38fe7995b1bd031fd9dc9c3f8097567c7fd96168ef33
MD5 9a223e411115f5b0eb620beaa2c9eb24
BLAKE2b-256 cb9800fff057507639d13f03a7b28df1f472036c59da77ecc86610d94c14f23d

See more details on using hashes here.

File details

Details for the file arcticdb-6.15.0-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.whl.

File metadata

File hashes

Hashes for arcticdb-6.15.0-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.whl
Algorithm Hash digest
SHA256 159863c22907cdec0b851d114eb648dadbb4ad8849e71dba1768105e626ea1ea
MD5 281817febf589a46ec128bbeac3794ff
BLAKE2b-256 40fc53f6b2279bde571479ecc34df43f5233856382b035e476d0a3e8875ff1e4

See more details on using hashes here.

File details

Details for the file arcticdb-6.15.0-cp314-cp314-macosx_15_0_arm64.whl.

File metadata

File hashes

Hashes for arcticdb-6.15.0-cp314-cp314-macosx_15_0_arm64.whl
Algorithm Hash digest
SHA256 545af9d50ec9db685dabe84d6ab834ef4814a072553f5f54ecc2749548dfdb77
MD5 f067c9c7c65c0e825d1c5b632f7efec3
BLAKE2b-256 dcd3d546b838dea29e5b0f1cfcfa35ac29f01ba92b974edae999619e9b5dc066

See more details on using hashes here.

File details

Details for the file arcticdb-6.15.0-cp313-cp313-win_amd64.whl.

File metadata

  • Download URL: arcticdb-6.15.0-cp313-cp313-win_amd64.whl
  • Upload date:
  • Size: 9.4 MB
  • Tags: CPython 3.13, Windows x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.2.0 CPython/3.10.12

File hashes

Hashes for arcticdb-6.15.0-cp313-cp313-win_amd64.whl
Algorithm Hash digest
SHA256 1677052d8eb20cf889340c16b03996142536a802bb92635fe4d7d57f5f51914a
MD5 d85ecd1182783743aea5720568cdb6fe
BLAKE2b-256 769643dafa262de1334585304284d63e2eb39f71a2166ea9830efe52d385b3a3

See more details on using hashes here.

File details

Details for the file arcticdb-6.15.0-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.whl.

File metadata

File hashes

Hashes for arcticdb-6.15.0-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.whl
Algorithm Hash digest
SHA256 cec478cbe8b9e8aebe23ddd3ee5182f1f652dfe0615d3af9a03e3107800020f6
MD5 0ae3b2d2e694245ecbad13a323319dc0
BLAKE2b-256 6ed5ffa9bc73c6c8fc46d7f845f8e19752e4effc0a0dfbd6d6f80ab1e1f324c4

See more details on using hashes here.

File details

Details for the file arcticdb-6.15.0-cp313-cp313-macosx_15_0_arm64.whl.

File metadata

File hashes

Hashes for arcticdb-6.15.0-cp313-cp313-macosx_15_0_arm64.whl
Algorithm Hash digest
SHA256 ae9a909a14617d046a4afed702ada348ef3b287cd077ffb157c4adec39b759ee
MD5 008fddcbbc8f572d3fb0f96766777362
BLAKE2b-256 ec41a7888c8191c80d80989cea68a7b83eccfe6ca49e5e62d5a57cbe75923fcd

See more details on using hashes here.

File details

Details for the file arcticdb-6.15.0-cp312-cp312-win_amd64.whl.

File metadata

  • Download URL: arcticdb-6.15.0-cp312-cp312-win_amd64.whl
  • Upload date:
  • Size: 9.4 MB
  • Tags: CPython 3.12, Windows x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.2.0 CPython/3.10.12

File hashes

Hashes for arcticdb-6.15.0-cp312-cp312-win_amd64.whl
Algorithm Hash digest
SHA256 012604f2e41432ac417bed1407bfb560c52ab46fa334e79c524e9ccb79232cb6
MD5 3e4b106f35382853a50669a5e8c13ec7
BLAKE2b-256 09853dc7c898765e0256ee4562d29342fed655c492d2c31cb8e22a78473c2a3f

See more details on using hashes here.

File details

Details for the file arcticdb-6.15.0-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.whl.

File metadata

File hashes

Hashes for arcticdb-6.15.0-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.whl
Algorithm Hash digest
SHA256 3b324b3cab4d6643a4f2a4d378044878e9e71a1757e2f2a753ed7febb17c15d1
MD5 422f844acfb5eacef3a2064be9ae65b1
BLAKE2b-256 8a796fd6a707c84d8288a2bfc0a75008cba389325183425316d3f29e510c0a18

See more details on using hashes here.

File details

Details for the file arcticdb-6.15.0-cp312-cp312-macosx_15_0_arm64.whl.

File metadata

File hashes

Hashes for arcticdb-6.15.0-cp312-cp312-macosx_15_0_arm64.whl
Algorithm Hash digest
SHA256 0a5a8182e3c5be6a65f57b4a49b7ee1c072eee043cbf620a8170b82be7a77838
MD5 69df49b672a401dead19e131c429488a
BLAKE2b-256 02d018abc2c5fc8f25867b67593bb25a63edfd0cd9ccc8b9a29cfbc54f64d88d

See more details on using hashes here.

File details

Details for the file arcticdb-6.15.0-cp311-cp311-win_amd64.whl.

File metadata

  • Download URL: arcticdb-6.15.0-cp311-cp311-win_amd64.whl
  • Upload date:
  • Size: 9.4 MB
  • Tags: CPython 3.11, Windows x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.2.0 CPython/3.10.12

File hashes

Hashes for arcticdb-6.15.0-cp311-cp311-win_amd64.whl
Algorithm Hash digest
SHA256 ff1891257d8ee39e932dbd863d64dee06eb4cfda0b2b56eb0205f17f0b2966d1
MD5 9cdfa0aad8e77fcdde61b4afc31f2b65
BLAKE2b-256 b96db9061b3cf2349b7ea2709dd9067d72b6081899f1595278b4352ab0fee8b1

See more details on using hashes here.

File details

Details for the file arcticdb-6.15.0-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.whl.

File metadata

File hashes

Hashes for arcticdb-6.15.0-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.whl
Algorithm Hash digest
SHA256 c15b3eb8f3b00f52cd250831d960e7b1efa34850673ad43c61aa5fdec6ecef61
MD5 01277c5ce9a73a72cd839439bc275f66
BLAKE2b-256 4668d9ada3b2b1bd79db8955f88df58146abcc14873765326b94b0b58632335c

See more details on using hashes here.

File details

Details for the file arcticdb-6.15.0-cp311-cp311-macosx_15_0_arm64.whl.

File metadata

File hashes

Hashes for arcticdb-6.15.0-cp311-cp311-macosx_15_0_arm64.whl
Algorithm Hash digest
SHA256 98b0a891d5443e3df7352011183ad12844385d6e9169b4e8dfdd565d078c7d9c
MD5 a91859ddf426ba9616abd96e791d6e88
BLAKE2b-256 b2f37f6a662ae7f73605ce546865998a71fffae0117cdd8da0cd4e4cc44c1443

See more details on using hashes here.

File details

Details for the file arcticdb-6.15.0-cp310-cp310-win_amd64.whl.

File metadata

  • Download URL: arcticdb-6.15.0-cp310-cp310-win_amd64.whl
  • Upload date:
  • Size: 9.5 MB
  • Tags: CPython 3.10, Windows x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.2.0 CPython/3.10.12

File hashes

Hashes for arcticdb-6.15.0-cp310-cp310-win_amd64.whl
Algorithm Hash digest
SHA256 cb982d43bf4323385bd30d3194e25ca23230618583ff56d51fe1401f90aa702a
MD5 9bba989f92359ccc53d0bcf416ae4325
BLAKE2b-256 0862fd6592e3f2856661330ae9415b19f3d4e9de1e980ec06ba4e8f1d5b951b3

See more details on using hashes here.

File details

Details for the file arcticdb-6.15.0-cp310-cp310-manylinux2014_x86_64.manylinux_2_17_x86_64.whl.

File metadata

File hashes

Hashes for arcticdb-6.15.0-cp310-cp310-manylinux2014_x86_64.manylinux_2_17_x86_64.whl
Algorithm Hash digest
SHA256 5fe762eb28199ca9dca49866e0ccb178733a83e3c2b7f3dca19aebfca165c4ca
MD5 6fe4ccd5ca4f6924bfec72ecd1df2a59
BLAKE2b-256 53567313fddf740126def24ee66b85b9caa4f561bb14aeee8e47ca75c1dbaf68

See more details on using hashes here.

File details

Details for the file arcticdb-6.15.0-cp310-cp310-macosx_15_0_arm64.whl.

File metadata

File hashes

Hashes for arcticdb-6.15.0-cp310-cp310-macosx_15_0_arm64.whl
Algorithm Hash digest
SHA256 5ebf39802d0110d0eb96bcd15bcfb2ea59a4aabd29e24302d4789b3bb942358e
MD5 5e676b49014beed07907c5784d2a5995
BLAKE2b-256 0a54bee469dd727dab1624f0d50c432be5ff5382b48a4b9a70943681bb1cb0a4

See more details on using hashes here.

File details

Details for the file arcticdb-6.15.0-cp39-cp39-win_amd64.whl.

File metadata

  • Download URL: arcticdb-6.15.0-cp39-cp39-win_amd64.whl
  • Upload date:
  • Size: 9.4 MB
  • Tags: CPython 3.9, Windows x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.2.0 CPython/3.10.12

File hashes

Hashes for arcticdb-6.15.0-cp39-cp39-win_amd64.whl
Algorithm Hash digest
SHA256 5d14a96da419bb132406674d8dbca1d30bd5534f0162f6f52f9cd66033194165
MD5 6fee78d8e825668eb56f1e5b9576c0eb
BLAKE2b-256 5c089280f79aa161a8411740aa7bcbc04a2f1fac0b20d16d3f8799c8aa50c22e

See more details on using hashes here.

File details

Details for the file arcticdb-6.15.0-cp39-cp39-manylinux2014_x86_64.manylinux_2_17_x86_64.whl.

File metadata

File hashes

Hashes for arcticdb-6.15.0-cp39-cp39-manylinux2014_x86_64.manylinux_2_17_x86_64.whl
Algorithm Hash digest
SHA256 8aa18c39c544d00ac37e6a24a75363f38b97c43408cc0b0071aa68c1ceb1e56b
MD5 e66645fc4b138c370fc0082727490702
BLAKE2b-256 5ebc7b1adff652c608260825e6100a0a1b388f581cf69663a4d334bffae5ee2e

See more details on using hashes here.

File details

Details for the file arcticdb-6.15.0-cp39-cp39-macosx_15_0_arm64.whl.

File metadata

File hashes

Hashes for arcticdb-6.15.0-cp39-cp39-macosx_15_0_arm64.whl
Algorithm Hash digest
SHA256 5eeb303bd12ad68ae9ae4d38382ea8621da090d3a672ca8f3cf8fd82e3ec831e
MD5 57a6e0ffa934fab47bf8d53ba48d0711
BLAKE2b-256 1f9396274d3eaed7fae240cf805d0713cca38bb8ec96ba9c1ff66e8a1227db89

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page