Skip to main content

ArcticDB DataFrame Database

Project description


Three minute ArcticDB demo from PyQuantNews

Options data in ArcticDB

ArcticDB Website | 📘 ArcticDB Docs | 📰: ArcticDB Blog | Press Release | Press Release | Community


ArcticDB is a high performance, serverless DataFrame database built for the Python Data Science ecosystem. Launched in March 2023, it is the successor to Arctic.

Use of ArcticDB in production (including business or commercial environments) or for a Database Service requires a paid for license from ArcticDB Limited . Please contact info@arcticdb.io for further details.

ArcticDB offers an intuitive Python-centric API enabling you to read and write Pandas DataFrames to S3 or LMDB utilising a fast C++ data-processing and compression engine.

ArcticDB allows you to:

  • Pandas in, Pandas out: Read and write Pandas DataFrames, NumPy arrays and native types to S3 and LMDB without leaving Python.
  • Built for time-series data: Efficiently index and query time-series data across billions of rows
  • Time travel: Travel back in time to see previous versions of your data and create customizable snapshots of the database
  • Schemaless Database: Append, update and modify data without being constrained by the existing schema
  • Optimised for streaming data: Built in support for efficient sparse data storage
  • Powerful processing: Filter, aggregate and create new columns on-the-fly with a Pandas-like syntax
  • C++ efficiency: Accelerate analytics though concurrency in the C++ data-processing engine

ArcticDB handles data that is big in both row count and column count, so a 20-year history of more than 400,000 unique securities can be stored in a single symbol. Each symbol is maintained as a separate entity with no shared data which means ArcticDB can scale horizontally across symbols, maximising the performance potential of your compute, storage and network.

ArcticDB is designed from the outset to be resilient; there is no single point of failure, and persistent data structures in the storage mean that once a version of a symbol has been written, it can never be corrupted by subsequent updates. Pulling compressed data directly from storage to the client means that there is no server to overload, so your data is always available when you need it.

Quickstart

Prebuilt binary availability (for the current version)

PyPI (Python 3.9 - 3.14) conda-forge (Python 3.10 - 3.14)
Linux x86_64 ✔️ ✔️
Linux arm64 ✔️
Windows x86_64 ✔️ ✔️
MacOS x86_64 ✔️
MacOS arm64 ✔️ ✔️

Storage compatibility

Linux Windows Mac
S3 ✔️ ✔️ ✔️
LMDB ✔️ ✔️ ✔️
Azure Blob Storage ✔️ ✔️ ✔️

We have tested against the following S3 backends:

  • AWS S3
  • Ceph
  • MinIO on Linux
  • Pure Storage S3
  • Scality S3
  • VAST Data S3

Installation

Install ArcticDB:

$ pip install arcticdb

or using conda-forge

$ conda install -c conda-forge arcticdb

Import ArcticDB:

>>> import arcticdb as adb

Create an instance on your S3 storage (with or without explicit credentials):

# Leave AWS to derive credential information
>>> ac = adb.Arctic('s3://MY_ENDPOINT:MY_BUCKET?aws_auth=true')

# Manually specify creds
>>> ac = adb.Arctic('s3://MY_ENDPOINT:MY_BUCKET?region=YOUR_REGION&access=ABCD&secret=DCBA')

Or create an instance on your local disk:

>>> ac = adb.Arctic("lmdb:///<path>")

Create your first library and list the libraries in the instance:

>>> ac.create_library('travel_data')
>>> ac.list_libraries()

Create a test dataframe:

>>> import numpy as np
>>> import pandas as pd
>>> NUM_COLUMNS=10
>>> NUM_ROWS=100_000
>>> df = pd.DataFrame(np.random.randint(0,100,size=(NUM_ROWS, NUM_COLUMNS)), columns=[f"COL_{i}" for i in range(NUM_COLUMNS)], index=pd.date_range('2000', periods=NUM_ROWS, freq='h'))

Get the library, write some data to it, and read it back:

>>> lib = ac['travel_data']
>>> lib.write("my_data", df)
>>> data = lib.read("my_data")

To find out more about working with data, visit our docs


Documentation

The source code for the ArcticDB docs are located in the docs folder, and are hosted at docs.arcticdb.io.

License

ArcticDB is released under a Business Source License 1.1 (BSL)

BSL features are free to use and the source code is available, but users may not use ArcticDB for production use or for a Database Service, without agreement with Man Group Operations Limited.

Use of ArcticDB in production or for a Database Service requires a paid for license from ArcticDB Limited and is licensed under the ArcticDB Software License Agreement. For more information please contact info@arcticdb.io.

The BSL is not certified as an open-source license, but most of the Open Source Initiative (OSI) criteria are met. Please see version conversion dates in the below table:

ArcticDB Version License Converts to Apache 2.0
1.0 Business Source License 1.1 Mar 16, 2025
1.2 Business Source License 1.1 May 22, 2025
1.3 Business Source License 1.1 Jun 9, 2025
1.4 Business Source License 1.1 Jun 23, 2025
1.5 Business Source License 1.1 Jul 11, 2025
1.6 Business Source License 1.1 Jul 25, 2025
2.0 Business Source License 1.1 Aug 29, 2025
3.0 Business Source License 1.1 Sep 13, 2025
4.0 Business Source License 1.1 Sep 27, 2025
4.1 Business Source License 1.1 Nov 1, 2025
4.2 Business Source License 1.1 Nov 12, 2025
4.3 Business Source License 1.1 Feb 7, 2026
4.4 Business Source License 1.1 Apr 5, 2026
4.5 Business Source License 1.1 Aug 14, 2026
5.0 Business Source License 1.1 Oct 31, 2026
5.1 Business Source License 1.1 Nov 15, 2026
5.2 Business Source License 1.1 Jan 27, 2027
5.3 Business Source License 1.1 Mar 24, 2027
5.4 Business Source License 1.1 Apr 28, 2027
5.5 Business Source License 1.1 May 13, 2027
5.6 Business Source License 1.1 May 26, 2027
5.7 Business Source License 1.1 Jun 2, 2027
5.8 Business Source License 1.1 Jun 9, 2027
5.9 Business Source License 1.1 Jul 7, 2027
5.10 Business Source License 1.1 Jul 29, 2027
6.1 Business Source License 1.1 Aug 20, 2027
6.2 Business Source License 1.1 Sep 8, 2027
6.3 Business Source License 1.1 Nov 3, 2027
6.5 Business Source License 1.1 Dec 11, 2027
6.6 Business Source License 1.1 Jan 6, 2028

Code of Conduct

Code of Conduct

This project has adopted a Code of Conduct. If you have any concerns about the Code, or behaviour that you have experienced in the project, please contact us at info@arcticdb.io.

Contributing/Building From Source

We welcome your contributions to help us improve and extend this project!

Please refer to the Contributing page and feel free to open issues on GitHub.

We are also always looking for feedback from our dedicated community! If you have used ArcticDB please let us know, we would love to hear about your experience!

Our release process is documented here.

Community

We would love to hear how your ArcticDB journey evolves, email us at info@arcticdb.io or come chat to us on Twitter!

Interested in learning more about ArcticDB? Head over to our blog!

Do you have any questions or issues? Chat to us and other users through our dedicated Slack Workspace - sign up for Slack access on our website.

Project details


Release history Release notifications | RSS feed

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distributions

No source distribution files available for this release.See tutorial on generating distribution archives.

Built Distributions

If you're not sure about the file name format, learn more about wheel file names.

arcticdb-6.13.0-cp314-cp314-win_amd64.whl (9.5 MB view details)

Uploaded CPython 3.14Windows x86-64

arcticdb-6.13.0-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (19.6 MB view details)

Uploaded CPython 3.14manylinux: glibc 2.17+ x86-64

arcticdb-6.13.0-cp314-cp314-macosx_15_0_arm64.whl (18.1 MB view details)

Uploaded CPython 3.14macOS 15.0+ ARM64

arcticdb-6.13.0-cp313-cp313-win_amd64.whl (9.3 MB view details)

Uploaded CPython 3.13Windows x86-64

arcticdb-6.13.0-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (19.6 MB view details)

Uploaded CPython 3.13manylinux: glibc 2.17+ x86-64

arcticdb-6.13.0-cp313-cp313-macosx_15_0_arm64.whl (18.2 MB view details)

Uploaded CPython 3.13macOS 15.0+ ARM64

arcticdb-6.13.0-cp312-cp312-win_amd64.whl (9.3 MB view details)

Uploaded CPython 3.12Windows x86-64

arcticdb-6.13.0-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (19.6 MB view details)

Uploaded CPython 3.12manylinux: glibc 2.17+ x86-64

arcticdb-6.13.0-cp312-cp312-macosx_15_0_arm64.whl (18.2 MB view details)

Uploaded CPython 3.12macOS 15.0+ ARM64

arcticdb-6.13.0-cp311-cp311-win_amd64.whl (9.3 MB view details)

Uploaded CPython 3.11Windows x86-64

arcticdb-6.13.0-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (19.6 MB view details)

Uploaded CPython 3.11manylinux: glibc 2.17+ x86-64

arcticdb-6.13.0-cp311-cp311-macosx_15_0_arm64.whl (18.2 MB view details)

Uploaded CPython 3.11macOS 15.0+ ARM64

arcticdb-6.13.0-cp310-cp310-win_amd64.whl (9.3 MB view details)

Uploaded CPython 3.10Windows x86-64

arcticdb-6.13.0-cp310-cp310-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (19.6 MB view details)

Uploaded CPython 3.10manylinux: glibc 2.17+ x86-64

arcticdb-6.13.0-cp310-cp310-macosx_15_0_arm64.whl (18.2 MB view details)

Uploaded CPython 3.10macOS 15.0+ ARM64

arcticdb-6.13.0-cp39-cp39-win_amd64.whl (9.3 MB view details)

Uploaded CPython 3.9Windows x86-64

arcticdb-6.13.0-cp39-cp39-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (19.6 MB view details)

Uploaded CPython 3.9manylinux: glibc 2.17+ x86-64

arcticdb-6.13.0-cp39-cp39-macosx_15_0_arm64.whl (18.2 MB view details)

Uploaded CPython 3.9macOS 15.0+ ARM64

File details

Details for the file arcticdb-6.13.0-cp314-cp314-win_amd64.whl.

File metadata

  • Download URL: arcticdb-6.13.0-cp314-cp314-win_amd64.whl
  • Upload date:
  • Size: 9.5 MB
  • Tags: CPython 3.14, Windows x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.2.0 CPython/3.10.12

File hashes

Hashes for arcticdb-6.13.0-cp314-cp314-win_amd64.whl
Algorithm Hash digest
SHA256 c1d58b990b3b30ab06a6a1969524f3b8915219fc1c08e5acea7658bd4ceef3c8
MD5 08b51c83f652f88029f7c6c1956cbd82
BLAKE2b-256 a232d2fb48615d3f2adb85d623e756c57000daae54e738ed3d9a5566397cd73f

See more details on using hashes here.

File details

Details for the file arcticdb-6.13.0-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.whl.

File metadata

File hashes

Hashes for arcticdb-6.13.0-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.whl
Algorithm Hash digest
SHA256 83e89bce41c8cf6644f89689607fc09118e920e78aecda4f676959285f803062
MD5 c2b14a6ac6b55e7ff0be7bd2db0f4b17
BLAKE2b-256 733b35c7ad784d522aa6eacb41df57194189e8c83bb08bbe9947c3dd01d8e413

See more details on using hashes here.

File details

Details for the file arcticdb-6.13.0-cp314-cp314-macosx_15_0_arm64.whl.

File metadata

File hashes

Hashes for arcticdb-6.13.0-cp314-cp314-macosx_15_0_arm64.whl
Algorithm Hash digest
SHA256 c7aa07f75798ea6cf6d6dae9bffc65c3364df4eecf52d5da1cf477897f77486a
MD5 a9a5a80a72467d49bc647fbc5bf4de24
BLAKE2b-256 522e6146c67cc868c554c7f9e1ae771d2ce9192de8f1dd88eabe39469e3cb961

See more details on using hashes here.

File details

Details for the file arcticdb-6.13.0-cp313-cp313-win_amd64.whl.

File metadata

  • Download URL: arcticdb-6.13.0-cp313-cp313-win_amd64.whl
  • Upload date:
  • Size: 9.3 MB
  • Tags: CPython 3.13, Windows x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.2.0 CPython/3.10.12

File hashes

Hashes for arcticdb-6.13.0-cp313-cp313-win_amd64.whl
Algorithm Hash digest
SHA256 bb6d4563d12df39cea3e3bfa791d0f7ad22887956fe180792f1709ce1a25d2d3
MD5 032d69490ebe110d8084da1892feb77d
BLAKE2b-256 3daa1dc1197defffb5e2e1d67ee97c79a1ff2d532627cdce9c5fcf897cda05ad

See more details on using hashes here.

File details

Details for the file arcticdb-6.13.0-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.whl.

File metadata

File hashes

Hashes for arcticdb-6.13.0-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.whl
Algorithm Hash digest
SHA256 4b2dfcbb3ef45c3a6d53ad54249f05c6be5436d541f006b6fbf235152ea75d26
MD5 2bf3d86c92d53faef22c47349499581a
BLAKE2b-256 c0104972075500f715c919ff1c55ba70336deb3b3bdf2c9d91bc2f42ebc8bd91

See more details on using hashes here.

File details

Details for the file arcticdb-6.13.0-cp313-cp313-macosx_15_0_arm64.whl.

File metadata

File hashes

Hashes for arcticdb-6.13.0-cp313-cp313-macosx_15_0_arm64.whl
Algorithm Hash digest
SHA256 da23ab039e5733b5972afbf9ca59d859832b64950c5257ecac827685cab6c361
MD5 ccba82deae715974ae8493ca9cf56093
BLAKE2b-256 d269ba9a4d538885f4dab129fbea79badf5f8e46269fc11ba6ea84ec215dc22c

See more details on using hashes here.

File details

Details for the file arcticdb-6.13.0-cp312-cp312-win_amd64.whl.

File metadata

  • Download URL: arcticdb-6.13.0-cp312-cp312-win_amd64.whl
  • Upload date:
  • Size: 9.3 MB
  • Tags: CPython 3.12, Windows x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.2.0 CPython/3.10.12

File hashes

Hashes for arcticdb-6.13.0-cp312-cp312-win_amd64.whl
Algorithm Hash digest
SHA256 d21295f6080a733a6356041abf40245ec7ad120d0fe877c66cb6c21e67adf20f
MD5 f8938334f29c2ca42e1628389272547b
BLAKE2b-256 856aa27d497ed2352969860032f232033a3ae978cdfabb97a1b97bacbca56690

See more details on using hashes here.

File details

Details for the file arcticdb-6.13.0-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.whl.

File metadata

File hashes

Hashes for arcticdb-6.13.0-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.whl
Algorithm Hash digest
SHA256 0bd2fbf8d955c663fb2d26357b0f035ea4138e6713200ab2b47d47f442038fbf
MD5 756529ed13db8ee69efaa58c530de386
BLAKE2b-256 666aa5f1a7c8f4a2dae166655846bacebe90aa1b48a5952165862a53f35f33a3

See more details on using hashes here.

File details

Details for the file arcticdb-6.13.0-cp312-cp312-macosx_15_0_arm64.whl.

File metadata

File hashes

Hashes for arcticdb-6.13.0-cp312-cp312-macosx_15_0_arm64.whl
Algorithm Hash digest
SHA256 bd610b73b4db33e662805026742d657235195c3f01f806f90e077d5e98107688
MD5 af1a2aa565c0e6450eaa63091b9b643f
BLAKE2b-256 9e7185ba2479195d75db4956d8e6a1f249a1b3deeee1d8ee4015d01b07821b20

See more details on using hashes here.

File details

Details for the file arcticdb-6.13.0-cp311-cp311-win_amd64.whl.

File metadata

  • Download URL: arcticdb-6.13.0-cp311-cp311-win_amd64.whl
  • Upload date:
  • Size: 9.3 MB
  • Tags: CPython 3.11, Windows x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.2.0 CPython/3.10.12

File hashes

Hashes for arcticdb-6.13.0-cp311-cp311-win_amd64.whl
Algorithm Hash digest
SHA256 8e87c8dafa4b0eca51acbe7186175a8623c2d136c85924372678fdecbfa9644e
MD5 8bc72709b7c7bb32f7d1fb79206f302a
BLAKE2b-256 f0ce3383c813e68f1cc78321e4cddeed58379f63983a7aa53521bae223cd6ad9

See more details on using hashes here.

File details

Details for the file arcticdb-6.13.0-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.whl.

File metadata

File hashes

Hashes for arcticdb-6.13.0-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.whl
Algorithm Hash digest
SHA256 9f2ee05cfcae89f380c527a5b8769792bd464162f07d1283004140311aafd0c3
MD5 49001757f5f06c1e90d306fc4460f104
BLAKE2b-256 eaf81d248da15b7120475503ce2370ef7f1a4b719287ac190e1a83d60ccf63e9

See more details on using hashes here.

File details

Details for the file arcticdb-6.13.0-cp311-cp311-macosx_15_0_arm64.whl.

File metadata

File hashes

Hashes for arcticdb-6.13.0-cp311-cp311-macosx_15_0_arm64.whl
Algorithm Hash digest
SHA256 78de7594e46ca98424d893b8c4124de6d0a5a90df9724bc69d59a7e3d90b81c7
MD5 b058bcabe4968d98dd611c8161af82a6
BLAKE2b-256 a685f2fa46a2e028a4ff6931caa855856e3a032048c7e543291c9ee10006a216

See more details on using hashes here.

File details

Details for the file arcticdb-6.13.0-cp310-cp310-win_amd64.whl.

File metadata

  • Download URL: arcticdb-6.13.0-cp310-cp310-win_amd64.whl
  • Upload date:
  • Size: 9.3 MB
  • Tags: CPython 3.10, Windows x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.2.0 CPython/3.10.12

File hashes

Hashes for arcticdb-6.13.0-cp310-cp310-win_amd64.whl
Algorithm Hash digest
SHA256 8f3befe15635c3b92e0d80a28749586c2f608b8362ce408f462f2133c30852e1
MD5 ed82fbe45a2a67b6faa070c293b7af6c
BLAKE2b-256 37235c5c9edc1fa58fea6112488ff69e0c5e5faff1b6eeca997ba8981de9777c

See more details on using hashes here.

File details

Details for the file arcticdb-6.13.0-cp310-cp310-manylinux2014_x86_64.manylinux_2_17_x86_64.whl.

File metadata

File hashes

Hashes for arcticdb-6.13.0-cp310-cp310-manylinux2014_x86_64.manylinux_2_17_x86_64.whl
Algorithm Hash digest
SHA256 749aabbf73e6a25605f4a85bba4a8ccc247ac73c7c8994c428ae2b0f22c8c319
MD5 a6bd3c72fb196364f64d5ef1c189d786
BLAKE2b-256 a48c99b546fed0a4b82b137866f72561d281e0e0ae4fac9ea4910894a339fe9a

See more details on using hashes here.

File details

Details for the file arcticdb-6.13.0-cp310-cp310-macosx_15_0_arm64.whl.

File metadata

File hashes

Hashes for arcticdb-6.13.0-cp310-cp310-macosx_15_0_arm64.whl
Algorithm Hash digest
SHA256 a39bca7bbf0ac45002eb7234dc723ed7f0587810da5b3a6f991c66e9b3cd6c86
MD5 69faca971a8d8b7ac08e8a7ef7416488
BLAKE2b-256 4823b457739f07b11298cd6f21229d2cbbecc3f72fb593db3122e8290a47eeb1

See more details on using hashes here.

File details

Details for the file arcticdb-6.13.0-cp39-cp39-win_amd64.whl.

File metadata

  • Download URL: arcticdb-6.13.0-cp39-cp39-win_amd64.whl
  • Upload date:
  • Size: 9.3 MB
  • Tags: CPython 3.9, Windows x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.2.0 CPython/3.10.12

File hashes

Hashes for arcticdb-6.13.0-cp39-cp39-win_amd64.whl
Algorithm Hash digest
SHA256 1d18160f7a4e18e4603bd93fb3e165b8754d3384983e000ea52932db3a89a580
MD5 39b103b3230e71cb26f7389355ada3e9
BLAKE2b-256 9dcc6ee4c0adb3c24c41c06ac4c02dde06a77239deac8dc030798269d4e87358

See more details on using hashes here.

File details

Details for the file arcticdb-6.13.0-cp39-cp39-manylinux2014_x86_64.manylinux_2_17_x86_64.whl.

File metadata

File hashes

Hashes for arcticdb-6.13.0-cp39-cp39-manylinux2014_x86_64.manylinux_2_17_x86_64.whl
Algorithm Hash digest
SHA256 30308dd2ebb40ae465ba4c3c524a1b77a4b7311a3de469e5f85aee51dffa9b3b
MD5 4b160952a8fb4c81185943aad7f7702b
BLAKE2b-256 626acf151b32f2c2c07493a94c973242d10caa9951f52dad5d19e1b0410548a4

See more details on using hashes here.

File details

Details for the file arcticdb-6.13.0-cp39-cp39-macosx_15_0_arm64.whl.

File metadata

File hashes

Hashes for arcticdb-6.13.0-cp39-cp39-macosx_15_0_arm64.whl
Algorithm Hash digest
SHA256 27cdfac305ec9069c677b2537f99e9b20d79645363ee34b8d29b9a76031ea3ca
MD5 d8c700a18f37a05e399a14ea85748363
BLAKE2b-256 c59bd63cdb83475bf227e540ab7e614a9eb40cb491f9360c18201e41e36b9100

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page