Skip to main content

ArcticDB DataFrame Database

Project description


Three minute ArcticDB demo from PyQuantNews

Options data in ArcticDB

ArcticDB Website | 📘 ArcticDB Docs | 📰: ArcticDB Blog | Press Release | Press Release | Community


ArcticDB is a high performance, serverless DataFrame database built for the Python Data Science ecosystem. Launched in March 2023, it is the successor to Arctic.

Use of ArcticDB in production (including business or commercial environments) or for a Database Service requires a paid for license from ArcticDB Limited . Please contact info@arcticdb.io for further details.

ArcticDB offers an intuitive Python-centric API enabling you to read and write Pandas DataFrames to S3 or LMDB utilising a fast C++ data-processing and compression engine.

ArcticDB allows you to:

  • Pandas in, Pandas out: Read and write Pandas DataFrames, NumPy arrays and native types to S3 and LMDB without leaving Python.
  • Built for time-series data: Efficiently index and query time-series data across billions of rows
  • Time travel: Travel back in time to see previous versions of your data and create customizable snapshots of the database
  • Schemaless Database: Append, update and modify data without being constrained by the existing schema
  • Optimised for streaming data: Built in support for efficient sparse data storage
  • Powerful processing: Filter, aggregate and create new columns on-the-fly with a Pandas-like syntax
  • C++ efficiency: Accelerate analytics though concurrency in the C++ data-processing engine

ArcticDB handles data that is big in both row count and column count, so a 20-year history of more than 400,000 unique securities can be stored in a single symbol. Each symbol is maintained as a separate entity with no shared data which means ArcticDB can scale horizontally across symbols, maximising the performance potential of your compute, storage and network.

ArcticDB is designed from the outset to be resilient; there is no single point of failure, and persistent data structures in the storage mean that once a version of a symbol has been written, it can never be corrupted by subsequent updates. Pulling compressed data directly from storage to the client means that there is no server to overload, so your data is always available when you need it.

Quickstart

Prebuilt binary availability (for the current version)

PyPI (Python 3.9 - 3.14) conda-forge (Python 3.10 - 3.14)
Linux x86_64 ✔️ ✔️
Linux arm64 ✔️
Windows x86_64 ✔️ ✔️
MacOS x86_64 ✔️
MacOS arm64 ✔️ ✔️

Storage compatibility

Linux Windows Mac
S3 ✔️ ✔️ ✔️
LMDB ✔️ ✔️ ✔️
Azure Blob Storage ✔️ ✔️ ✔️

We have tested against the following S3 backends:

  • AWS S3
  • Ceph
  • MinIO on Linux
  • Pure Storage S3
  • Scality S3
  • VAST Data S3

Installation

Install ArcticDB:

$ pip install arcticdb

or using conda-forge

$ conda install -c conda-forge arcticdb

Import ArcticDB:

>>> import arcticdb as adb

Create an instance on your S3 storage (with or without explicit credentials):

# Leave AWS to derive credential information
>>> ac = adb.Arctic('s3://MY_ENDPOINT:MY_BUCKET?aws_auth=true')

# Manually specify creds
>>> ac = adb.Arctic('s3://MY_ENDPOINT:MY_BUCKET?region=YOUR_REGION&access=ABCD&secret=DCBA')

Or create an instance on your local disk:

>>> ac = adb.Arctic("lmdb:///<path>")

Create your first library and list the libraries in the instance:

>>> ac.create_library('travel_data')
>>> ac.list_libraries()

Create a test dataframe:

>>> import numpy as np
>>> import pandas as pd
>>> NUM_COLUMNS=10
>>> NUM_ROWS=100_000
>>> df = pd.DataFrame(np.random.randint(0,100,size=(NUM_ROWS, NUM_COLUMNS)), columns=[f"COL_{i}" for i in range(NUM_COLUMNS)], index=pd.date_range('2000', periods=NUM_ROWS, freq='h'))

Get the library, write some data to it, and read it back:

>>> lib = ac['travel_data']
>>> lib.write("my_data", df)
>>> data = lib.read("my_data")

To find out more about working with data, visit our docs


Documentation

The source code for the ArcticDB docs are located in the docs folder, and are hosted at docs.arcticdb.io.

License

ArcticDB is released under a Business Source License 1.1 (BSL)

BSL features are free to use and the source code is available, but users may not use ArcticDB for production use or for a Database Service, without agreement with Man Group Operations Limited.

Use of ArcticDB in production or for a Database Service requires a paid for license from ArcticDB Limited and is licensed under the ArcticDB Software License Agreement. For more information please contact info@arcticdb.io.

The BSL is not certified as an open-source license, but most of the Open Source Initiative (OSI) criteria are met. Please see version conversion dates in the below table:

ArcticDB Version License Converts to Apache 2.0
1.0 Business Source License 1.1 Mar 16, 2025
1.2 Business Source License 1.1 May 22, 2025
1.3 Business Source License 1.1 Jun 9, 2025
1.4 Business Source License 1.1 Jun 23, 2025
1.5 Business Source License 1.1 Jul 11, 2025
1.6 Business Source License 1.1 Jul 25, 2025
2.0 Business Source License 1.1 Aug 29, 2025
3.0 Business Source License 1.1 Sep 13, 2025
4.0 Business Source License 1.1 Sep 27, 2025
4.1 Business Source License 1.1 Nov 1, 2025
4.2 Business Source License 1.1 Nov 12, 2025
4.3 Business Source License 1.1 Feb 7, 2026
4.4 Business Source License 1.1 Apr 5, 2026
4.5 Business Source License 1.1 Aug 14, 2026
5.0 Business Source License 1.1 Oct 31, 2026
5.1 Business Source License 1.1 Nov 15, 2026
5.2 Business Source License 1.1 Jan 27, 2027
5.3 Business Source License 1.1 Mar 24, 2027
5.4 Business Source License 1.1 Apr 28, 2027
5.5 Business Source License 1.1 May 13, 2027
5.6 Business Source License 1.1 May 26, 2027
5.7 Business Source License 1.1 Jun 2, 2027
5.8 Business Source License 1.1 Jun 9, 2027
5.9 Business Source License 1.1 Jul 7, 2027
5.10 Business Source License 1.1 Jul 29, 2027
6.1 Business Source License 1.1 Aug 20, 2027
6.2 Business Source License 1.1 Sep 8, 2027
6.3 Business Source License 1.1 Nov 3, 2027
6.5 Business Source License 1.1 Dec 11, 2027
6.6 Business Source License 1.1 Jan 6, 2028

Code of Conduct

Code of Conduct

This project has adopted a Code of Conduct. If you have any concerns about the Code, or behaviour that you have experienced in the project, please contact us at info@arcticdb.io.

Contributing/Building From Source

We welcome your contributions to help us improve and extend this project!

Please refer to the Contributing page and feel free to open issues on GitHub.

We are also always looking for feedback from our dedicated community! If you have used ArcticDB please let us know, we would love to hear about your experience!

Our release process is documented here.

Community

We would love to hear how your ArcticDB journey evolves, email us at info@arcticdb.io or come chat to us on Twitter!

Interested in learning more about ArcticDB? Head over to our blog!

Do you have any questions or issues? Chat to us and other users through our dedicated Slack Workspace - sign up for Slack access on our website.

Project details


Release history Release notifications | RSS feed

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distributions

No source distribution files available for this release.See tutorial on generating distribution archives.

Built Distributions

If you're not sure about the file name format, learn more about wheel file names.

arcticdb-6.14.0-cp314-cp314-win_amd64.whl (9.6 MB view details)

Uploaded CPython 3.14Windows x86-64

arcticdb-6.14.0-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (19.8 MB view details)

Uploaded CPython 3.14manylinux: glibc 2.17+ x86-64

arcticdb-6.14.0-cp314-cp314-macosx_15_0_arm64.whl (18.3 MB view details)

Uploaded CPython 3.14macOS 15.0+ ARM64

arcticdb-6.14.0-cp313-cp313-win_amd64.whl (9.4 MB view details)

Uploaded CPython 3.13Windows x86-64

arcticdb-6.14.0-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (19.8 MB view details)

Uploaded CPython 3.13manylinux: glibc 2.17+ x86-64

arcticdb-6.14.0-cp313-cp313-macosx_15_0_arm64.whl (18.3 MB view details)

Uploaded CPython 3.13macOS 15.0+ ARM64

arcticdb-6.14.0-cp312-cp312-win_amd64.whl (9.4 MB view details)

Uploaded CPython 3.12Windows x86-64

arcticdb-6.14.0-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (19.8 MB view details)

Uploaded CPython 3.12manylinux: glibc 2.17+ x86-64

arcticdb-6.14.0-cp312-cp312-macosx_15_0_arm64.whl (18.3 MB view details)

Uploaded CPython 3.12macOS 15.0+ ARM64

arcticdb-6.14.0-cp311-cp311-win_amd64.whl (9.4 MB view details)

Uploaded CPython 3.11Windows x86-64

arcticdb-6.14.0-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (19.8 MB view details)

Uploaded CPython 3.11manylinux: glibc 2.17+ x86-64

arcticdb-6.14.0-cp311-cp311-macosx_15_0_arm64.whl (18.3 MB view details)

Uploaded CPython 3.11macOS 15.0+ ARM64

arcticdb-6.14.0-cp310-cp310-win_amd64.whl (9.4 MB view details)

Uploaded CPython 3.10Windows x86-64

arcticdb-6.14.0-cp310-cp310-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (19.8 MB view details)

Uploaded CPython 3.10manylinux: glibc 2.17+ x86-64

arcticdb-6.14.0-cp310-cp310-macosx_15_0_arm64.whl (18.3 MB view details)

Uploaded CPython 3.10macOS 15.0+ ARM64

arcticdb-6.14.0-cp39-cp39-win_amd64.whl (9.4 MB view details)

Uploaded CPython 3.9Windows x86-64

arcticdb-6.14.0-cp39-cp39-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (19.8 MB view details)

Uploaded CPython 3.9manylinux: glibc 2.17+ x86-64

arcticdb-6.14.0-cp39-cp39-macosx_15_0_arm64.whl (18.3 MB view details)

Uploaded CPython 3.9macOS 15.0+ ARM64

File details

Details for the file arcticdb-6.14.0-cp314-cp314-win_amd64.whl.

File metadata

  • Download URL: arcticdb-6.14.0-cp314-cp314-win_amd64.whl
  • Upload date:
  • Size: 9.6 MB
  • Tags: CPython 3.14, Windows x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.2.0 CPython/3.10.12

File hashes

Hashes for arcticdb-6.14.0-cp314-cp314-win_amd64.whl
Algorithm Hash digest
SHA256 049b2a00fbee8584ca0e3d6771918a44646e07457c0e0e70856dc33317ba8b21
MD5 7eb925b17006dbaa2fc2aad015f44364
BLAKE2b-256 25d6d8ee2ddc5d36d2e45eec10a82698abfa20d014871a1f37e61272702b1683

See more details on using hashes here.

File details

Details for the file arcticdb-6.14.0-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.whl.

File metadata

File hashes

Hashes for arcticdb-6.14.0-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.whl
Algorithm Hash digest
SHA256 faa27444f9574106f20031b123dccf58863eaccbaf7ed4839c6ac61ce5444d6f
MD5 b9ee7ba56c8dfa32f5f28af51ec47ec0
BLAKE2b-256 560cccc1c7f55ed7557e41c3b2685eb76a7183993d9fbae76b886578cbc20eb1

See more details on using hashes here.

File details

Details for the file arcticdb-6.14.0-cp314-cp314-macosx_15_0_arm64.whl.

File metadata

File hashes

Hashes for arcticdb-6.14.0-cp314-cp314-macosx_15_0_arm64.whl
Algorithm Hash digest
SHA256 85c2652e7a5e8c55c1d5306700f5098038cbaa2173152019fcc8a44b48efdee1
MD5 d029d81b0159bbc82866002d0d22e9ab
BLAKE2b-256 68fcd6fb7e17d59ec24de7792f437a276712daa10f15950950cdd679ae99996a

See more details on using hashes here.

File details

Details for the file arcticdb-6.14.0-cp313-cp313-win_amd64.whl.

File metadata

  • Download URL: arcticdb-6.14.0-cp313-cp313-win_amd64.whl
  • Upload date:
  • Size: 9.4 MB
  • Tags: CPython 3.13, Windows x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.2.0 CPython/3.10.12

File hashes

Hashes for arcticdb-6.14.0-cp313-cp313-win_amd64.whl
Algorithm Hash digest
SHA256 76b40ad3f9a8057e442726f5a44ecec10e08b241631e697380fb5cbd6186eb2c
MD5 f10049abd4260f30557ef8ca306ef886
BLAKE2b-256 e1e2dfb54786fdd62ff2715d3c0cb221f6bbd40912b3300610e1590fe8bf63d2

See more details on using hashes here.

File details

Details for the file arcticdb-6.14.0-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.whl.

File metadata

File hashes

Hashes for arcticdb-6.14.0-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.whl
Algorithm Hash digest
SHA256 86691f3548a99eb0705fb4ca9dee362b5583abd55055c6d01a550d91ee255f81
MD5 fabd0cb87db6594bd343ae6ab416fd53
BLAKE2b-256 62c7419f9b1feaa4812dd3518d7c68ca3f8986efd1506adf8d0de7c49051a0fe

See more details on using hashes here.

File details

Details for the file arcticdb-6.14.0-cp313-cp313-macosx_15_0_arm64.whl.

File metadata

File hashes

Hashes for arcticdb-6.14.0-cp313-cp313-macosx_15_0_arm64.whl
Algorithm Hash digest
SHA256 359ad4832294913a8aa268b2d8ea22ce183e227af3b419191f5b4710fad8a650
MD5 925adb566def88e3204c9339ac0ecb84
BLAKE2b-256 606287b555e208d3bd5bf368cb363285fab06effec99e05af15763f6c318451d

See more details on using hashes here.

File details

Details for the file arcticdb-6.14.0-cp312-cp312-win_amd64.whl.

File metadata

  • Download URL: arcticdb-6.14.0-cp312-cp312-win_amd64.whl
  • Upload date:
  • Size: 9.4 MB
  • Tags: CPython 3.12, Windows x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.2.0 CPython/3.10.12

File hashes

Hashes for arcticdb-6.14.0-cp312-cp312-win_amd64.whl
Algorithm Hash digest
SHA256 dddf49f14cb948dd670bae8603d2ef48e6f1cf0b5ef059ae18dfa70c55053f2e
MD5 e167784b201eca39772c60f63b3aed73
BLAKE2b-256 754401287ca813c7c3d6d4bb995acbdc462ef40448d6be69ffc1f301f883a55a

See more details on using hashes here.

File details

Details for the file arcticdb-6.14.0-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.whl.

File metadata

File hashes

Hashes for arcticdb-6.14.0-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.whl
Algorithm Hash digest
SHA256 94a05454e32a876085446e84930f4782486efb48454f34c8fbb211a4933e2433
MD5 4656aea49f1773cfb3fd4cfacb9ec784
BLAKE2b-256 aaa8ad4005ed5350489b7f082290cc5ed6fe4a070be7319fc5cf075292838a85

See more details on using hashes here.

File details

Details for the file arcticdb-6.14.0-cp312-cp312-macosx_15_0_arm64.whl.

File metadata

File hashes

Hashes for arcticdb-6.14.0-cp312-cp312-macosx_15_0_arm64.whl
Algorithm Hash digest
SHA256 7b54d91b4f8a5eea08f41b63423087a045b987d1f49b44455e58787d3781200f
MD5 f26bb91e9bf1c4e3946ba53ba37ac143
BLAKE2b-256 75dd7ac09def67b18081cfedcb08a466dd0d81f9cc5ce3e2b638d96ac6aae8ea

See more details on using hashes here.

File details

Details for the file arcticdb-6.14.0-cp311-cp311-win_amd64.whl.

File metadata

  • Download URL: arcticdb-6.14.0-cp311-cp311-win_amd64.whl
  • Upload date:
  • Size: 9.4 MB
  • Tags: CPython 3.11, Windows x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.2.0 CPython/3.10.12

File hashes

Hashes for arcticdb-6.14.0-cp311-cp311-win_amd64.whl
Algorithm Hash digest
SHA256 4e22600b86fcaec0f0d70cb0347289ad1eba0d7563d17fbf666b9ff460cf708e
MD5 8374af63c1425e4859f5fdfdf3d569d5
BLAKE2b-256 04ac498c4f858952120e90c9b3f9f2f07489716a6c2ec055f7866453286987a1

See more details on using hashes here.

File details

Details for the file arcticdb-6.14.0-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.whl.

File metadata

File hashes

Hashes for arcticdb-6.14.0-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.whl
Algorithm Hash digest
SHA256 a268541e205f0e912f2a12a1998fe1122fa1b64314006e38ac9572130acab02b
MD5 7f3af65bf320f20970b9c28680dd33b4
BLAKE2b-256 4b23ef80175fa64c1010263ce9e8077956d75ef1ae7a4ddd84bd510a960d536f

See more details on using hashes here.

File details

Details for the file arcticdb-6.14.0-cp311-cp311-macosx_15_0_arm64.whl.

File metadata

File hashes

Hashes for arcticdb-6.14.0-cp311-cp311-macosx_15_0_arm64.whl
Algorithm Hash digest
SHA256 b2571c842566705c29a58a850658b7eab9d566046db00c00944bb41b6c270157
MD5 10c43540dbacf45fbcb4f617e81f96c0
BLAKE2b-256 ae97b4076cd83a8772d31c92804c2bda06bb55f506ebc354a19c5b9f3a17080d

See more details on using hashes here.

File details

Details for the file arcticdb-6.14.0-cp310-cp310-win_amd64.whl.

File metadata

  • Download URL: arcticdb-6.14.0-cp310-cp310-win_amd64.whl
  • Upload date:
  • Size: 9.4 MB
  • Tags: CPython 3.10, Windows x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.2.0 CPython/3.10.12

File hashes

Hashes for arcticdb-6.14.0-cp310-cp310-win_amd64.whl
Algorithm Hash digest
SHA256 312d096fdbdb903ee8936b172c47fd00d5df26d253a3b44dc41f4867fb9c18b2
MD5 24ab7581da572c0dd0bdac0b3d6d516c
BLAKE2b-256 7307a9cdf77276d06066148770645216c991822c24c63b9f76fbbd31e6f1760e

See more details on using hashes here.

File details

Details for the file arcticdb-6.14.0-cp310-cp310-manylinux2014_x86_64.manylinux_2_17_x86_64.whl.

File metadata

File hashes

Hashes for arcticdb-6.14.0-cp310-cp310-manylinux2014_x86_64.manylinux_2_17_x86_64.whl
Algorithm Hash digest
SHA256 4a119cce9fafcbcf459b503fd0f89fb885c3578111e70578750d2c1873cdc6b0
MD5 af04aaaf795b54f5e371e2f6128ee446
BLAKE2b-256 d96b4bbf1df67424d05f1aa5c6a76954a663a7fb4f6fe23b79f9490d0004508d

See more details on using hashes here.

File details

Details for the file arcticdb-6.14.0-cp310-cp310-macosx_15_0_arm64.whl.

File metadata

File hashes

Hashes for arcticdb-6.14.0-cp310-cp310-macosx_15_0_arm64.whl
Algorithm Hash digest
SHA256 6eaa3a147be94afb2279fb012b1edc8c5277169f086059c665cc46b18edb340d
MD5 8e6c78373f592033ef0f2909076db47f
BLAKE2b-256 c94d2bbb7e020103985e3abf722526e30b480e719fa870992532999385456c60

See more details on using hashes here.

File details

Details for the file arcticdb-6.14.0-cp39-cp39-win_amd64.whl.

File metadata

  • Download URL: arcticdb-6.14.0-cp39-cp39-win_amd64.whl
  • Upload date:
  • Size: 9.4 MB
  • Tags: CPython 3.9, Windows x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.2.0 CPython/3.10.12

File hashes

Hashes for arcticdb-6.14.0-cp39-cp39-win_amd64.whl
Algorithm Hash digest
SHA256 439df0980129149f03861aabc65c2401557591830d00b977d8b57f4b91655e7f
MD5 d8c3c18144ff8b6cf170056227822beb
BLAKE2b-256 173b6e290fce605f9e9e3d42a50a7d79ab61004844dae472734bd9eb5223c83b

See more details on using hashes here.

File details

Details for the file arcticdb-6.14.0-cp39-cp39-manylinux2014_x86_64.manylinux_2_17_x86_64.whl.

File metadata

File hashes

Hashes for arcticdb-6.14.0-cp39-cp39-manylinux2014_x86_64.manylinux_2_17_x86_64.whl
Algorithm Hash digest
SHA256 452858ee56112e2305c98f02bd00cbb15d8a08b16f03c15430fae0f0c851e25b
MD5 83c77b28045dc279978b6881abc025b5
BLAKE2b-256 281ca9912bebbc30027a6c59d28e791064e463baadad1b28623e7fb5c54e0f5f

See more details on using hashes here.

File details

Details for the file arcticdb-6.14.0-cp39-cp39-macosx_15_0_arm64.whl.

File metadata

File hashes

Hashes for arcticdb-6.14.0-cp39-cp39-macosx_15_0_arm64.whl
Algorithm Hash digest
SHA256 a3d6971bd5695698cf496f0e0c1c5077d2e0bc3e4b8bd8b02683e1a6374e7257
MD5 aece85ef97fed7873b5ee8ea03255254
BLAKE2b-256 cf8b6f8725f02ad70de28aa86464a4cc1c6b86a6047df468ac8ca2a071d61c42

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page