Scale your pandas workflows by changing one line of code
To use Modin, replace the pandas import:
# import pandas as pd
import modin.pandas as pd
Installation
Modin can be installed from PyPI:
pip install modin
Full Documentation
Visit the complete documentation on readthedocs: http://modin.readthedocs.io
Scale your pandas workflow by changing a single line of code.
Modin uses Ray to provide an effortless way to speed up your pandas notebooks, scripts, and libraries. Unlike other distributed DataFrame libraries, Modin provides seamless integration and compatibility with existing pandas code. Even using the DataFrame constructor is identical.
import modin.pandas as pd
import numpy as np
frame_data = np.random.randint(0, 100, size=(2**10, 2**8))
df = pd.DataFrame(frame_data)
To use Modin, you do not need to know how many cores your system has and you do not need to specify how to distribute the data. In fact, you can continue using your previous pandas notebooks while experiencing a considerable speedup from Modin, even on a single machine. Once you’ve changed your import statement, you’re ready to use Modin just like you would pandas.
Faster pandas, even on your laptop
The modin.pandas DataFrame is an extremely light-weight parallel DataFrame. Modin
transparently distributes the data and computation so that all you need to do is
continue using the pandas API as you were before installing Modin. Unlike other parallel
DataFrame systems, Modin is an extremely light-weight, robust DataFrame. Because it is
so light-weight, Modin provides speed-ups of up to 4x on a laptop with 4 physical cores.
In pandas, you are only able to use one core at a time when you are doing computation of
any kind. With Modin, you are able to use all of the CPU cores on your machine. Even in
read_csv, we see large gains by efficiently distributing the work across your entire
machine.
import modin.pandas as pd
df = pd.read_csv("my_dataset.csv")
Modin is a DataFrame designed for datasets from 1KB to 1TB+
We have focused heavily on bridging the solutions between DataFrames for small data (e.g. pandas) and large data. Often data scientists require different tools for doing the same thing on different sizes of data. The DataFrame solutions that exist for 1KB do not scale to 1TB+, and the overheads of the solutions for 1TB+ are too costly for datasets in the 1KB range. With Modin, because of its light-weight, robust, and scalable nature, you get a fast DataFrame at small and large data. With preliminary cluster and out of core support, Modin is a DataFrame library with great single-node performance and high scalability in a cluster.
modin.pandas is currently under active development. Requests and contributions are welcome!
More information and Getting Involved
- Documentation
- Ask questions on our mailing list modin-dev@googlegroups.com.
- Submit bug reports to our GitHub Issues Page.
- Contributions are welcome! Open a pull request.
7 maintainers
Metadata
7 maintainers
Release files for modin 0.3.1
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| modin-0.3.1.tar.gz | 137.4 kB | Details |
Built distributions (wheels)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| modin-0.3.1-py3-none-any.whl | Python 3 | none | any | Details |
| modin-0.3.1-py2-none-any.whl | Python 2 | none | any | Details |
Total release size: 466.0 kB
Release files / modin-0.3.1.tar.gz
| Download URL | modin-0.3.1.tar.gz |
|---|---|
| Size | 137.4 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
47a1ec52afa2fd719db463616534ea4d42da224caaf3f2b4ff0358ba79cb2ff0
|
|
BLAKE2b-256 checksum How to use checksums |
fa8eda5381e215a576fc290c772a60c16a9570570f3eb65e41b7458a43b70dbe
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/1.12.1 pkginfo/1.4.2 requests/2.19.1 setuptools/40.4.3 requests-toolbelt/0.8.0 tqdm/4.28.1 CPython/3.6.6
|
Release files / modin-0.3.1-py3-none-any.whl
| Download URL | modin-0.3.1-py3-none-any.whl |
|---|---|
| Size | 165.0 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
d0f59dc84e790c67d45c1ca3d3c55e48afe3e85975ec4cc14f757ea2d36299e8
|
|
BLAKE2b-256 checksum How to use checksums |
4a98ddf887885750e57d9395ebedbb8771eb5a419fc1c029d694ad0520670198
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/1.12.1 pkginfo/1.4.2 requests/2.19.1 setuptools/40.4.3 requests-toolbelt/0.8.0 tqdm/4.28.1 CPython/3.6.6
|
Release files / modin-0.3.1-py2-none-any.whl
| Download URL | modin-0.3.1-py2-none-any.whl |
|---|---|
| Size | 163.6 kB |
| Tags | Python 2 |
|
SHA-256 checksum How to use checksums |
832cba72ad0866b2acbefa5f4963ce168e38f2e94d733816eb0e2f8bedbdaad8
|
|
BLAKE2b-256 checksum How to use checksums |
5fe64d7f58e3f84b457fa38db4cc7d2719df967483d2cae0384aa2baa4358893
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/1.12.1 pkginfo/1.4.2 requests/2.19.1 setuptools/40.4.3 requests-toolbelt/0.8.0 tqdm/4.28.1 CPython/3.6.6
|
