ODPS Python SDK and data analysis framework
Project description
Elegent way to access ODPS API. Documentation
Installation
The quick way:
pip install pyodps[full]
To integrate with Mars:
pip install pyodps#egg=pyodps[mars]
If you don’t need to use Jupyter, just type
pip install pyodps
The dependencies will be installed automatically.
Or from source code:
$ virtualenv pyodps_env
$ source pyodps_env/bin/activate
$ git clone <git clone URL> pyodps
$ cd pyodps
$ python setup.py install
Dependencies
Python (>=2.6), including Python 3+, pypy, Python 2.7 recommended
setuptools (>=3.0)
requests (>=2.4.0)
Run Unittest
copy conf/test.conf.template to odps/tests/test.conf, and fill it with your account
run python -m unittest discover
Usage
>>> from odps import ODPS
>>> o = ODPS('**your-access-id**', '**your-secret-access-key**',
... project='**your-project**', endpoint='**your-end-point**')
>>> dual = o.get_table('dual')
>>> dual.name
'dual'
>>> dual.schema
odps.Schema {
c_int_a bigint
c_int_b bigint
c_double_a double
c_double_b double
c_string_a string
c_string_b string
c_bool_a boolean
c_bool_b boolean
c_datetime_a datetime
c_datetime_b datetime
}
>>> dual.creation_time
datetime.datetime(2014, 6, 6, 13, 28, 24)
>>> dual.is_virtual_view
False
>>> dual.size
448
>>> dual.schema.columns
[<column c_int_a, type bigint>,
<column c_int_b, type bigint>,
<column c_double_a, type double>,
<column c_double_b, type double>,
<column c_string_a, type string>,
<column c_string_b, type string>,
<column c_bool_a, type boolean>,
<column c_bool_b, type boolean>,
<column c_datetime_a, type datetime>,
<column c_datetime_b, type datetime>]
DataFrame API
>>> from odps.df import DataFrame
>>> df = DataFrame(o.get_table('pyodps_iris'))
>>> df.dtypes
odps.Schema {
sepallength float64
sepalwidth float64
petallength float64
petalwidth float64
name string
}
>>> df.head(5)
|==========================================| 1 / 1 (100.00%) 0s
sepallength sepalwidth petallength petalwidth name
0 5.1 3.5 1.4 0.2 Iris-setosa
1 4.9 3.0 1.4 0.2 Iris-setosa
2 4.7 3.2 1.3 0.2 Iris-setosa
3 4.6 3.1 1.5 0.2 Iris-setosa
4 5.0 3.6 1.4 0.2 Iris-setosa
>>> df[df.sepalwidth > 3]['name', 'sepalwidth'].head(5)
|==========================================| 1 / 1 (100.00%) 12s
name sepalwidth
0 Iris-setosa 3.5
1 Iris-setosa 3.2
2 Iris-setosa 3.1
3 Iris-setosa 3.6
4 Iris-setosa 3.9
Command-line and IPython enhancement
In [1]: %load_ext odps In [2]: %enter Out[2]: <odps.inter.Room at 0x10fe0e450> In [3]: %sql select * from pyodps_iris limit 5 |==========================================| 1 / 1 (100.00%) 2s Out[3]: sepallength sepalwidth petallength petalwidth name 0 5.1 3.5 1.4 0.2 Iris-setosa 1 4.9 3.0 1.4 0.2 Iris-setosa 2 4.7 3.2 1.3 0.2 Iris-setosa 3 4.6 3.1 1.5 0.2 Iris-setosa 4 5.0 3.6 1.4 0.2 Iris-setosa
Python UDF Debugging Tool
#file: plus.py
from odps.udf import annotate
@annotate('bigint,bigint->bigint')
class Plus(object):
def evaluate(self, a, b):
return a + b
$ cat plus.input 1,1 3,2 $ pyou plus.Plus < plus.input 2 5
Contributing
For a development install, clone the repository and then install from source:
git clone https://github.com/aliyun/aliyun-odps-python-sdk cd pyodps pip install -r requirements.txt -e .
If you need to modify the frontend code, you need to install nodejs/npm. To build and install your frontend code, use
python setup.py build_js python setup.py install_js
License
Licensed under the Apache License 2.0
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distributions
Hashes for pyodps-0.11.2.1-cp310-cp310-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | f67cefb8beaef9ad4c3346d6bc2530885c5a9473c8d0c524c7ba3b8929bef79f |
|
MD5 | 8a6617b53e972846408803af984e11d8 |
|
BLAKE2b-256 | 0f64750bd052c4f7852d2a9d0948cf3513bb0e051db9efc460f2c0c2193b0349 |
Hashes for pyodps-0.11.2.1-cp310-cp310-win32.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | c0de6a719ab5243337d612a0a7794aabbeea1e4ac0c9bfbc6b63fae3436b8812 |
|
MD5 | 68f8ebf674c6b81928855877bdbcb4a1 |
|
BLAKE2b-256 | 619b16fbfaf85044a7b2932e1588d12bc806bc73f59f806ddb311422554d07f1 |
Hashes for pyodps-0.11.2.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 41644fcae28a1c05d6f71a2dfaee1713ea31e1e4dcb09d62c934020753e52b01 |
|
MD5 | b3f1534fcd8ae818b55c0d9c309a94d6 |
|
BLAKE2b-256 | 99f89903f41165d7d3ba621ff32b20b8c5e8e82abc6a2837d648f2a8938f7ce7 |
Hashes for pyodps-0.11.2.1-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | e55894115b9c1403ddda4dc063ddb1833c727ed31d2705379d5614737dd9a2bc |
|
MD5 | b8af74dbe70bf4d20c762e02f7edc1cc |
|
BLAKE2b-256 | 37f3ac2a8a6336b7b56e9f23d7374d60449942f278341ff42176ad251f11db0c |
Hashes for pyodps-0.11.2.1-cp310-cp310-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 0522608c4b8d74fdf5e95856585222501b0930eadbaaa851dc942dda6a4c7161 |
|
MD5 | 94d98737a2ed80399f856c741634ee96 |
|
BLAKE2b-256 | 3e1916399512331afb764f21c09fc92b99120b14839dc3ebca66dfe8a467fc56 |
Hashes for pyodps-0.11.2.1-cp310-cp310-macosx_10_9_universal2.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 55801d44367f5111c7b99a24000adc9a6b79194031338699762acdd6b8958a9d |
|
MD5 | 8a35b3327f1220bfc6b196886e71452e |
|
BLAKE2b-256 | 1b7ab82ecc93726293369efe80e7cd5e39a0efe766715fd567504571458d698c |
Hashes for pyodps-0.11.2.1-cp39-cp39-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 2aa62799aa5c2534ac8ebb7eded15fd3761de8d16a820ba2656513f0b7b1af6b |
|
MD5 | 072cecfd7292a7511e45ece27732b8c0 |
|
BLAKE2b-256 | a6cd8a2df9c97747b3a97575116540d7eb05df42de7b4b146a3a9afcb118b175 |
Hashes for pyodps-0.11.2.1-cp39-cp39-win32.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | b1957c3cf3cea35601334c6d1b48339f949333502539679a925e1d13bcce0630 |
|
MD5 | 07ddeab09e0aaf7349c626fe56e3fd58 |
|
BLAKE2b-256 | 9eb7ed727fb8ebb15ee07a00dc3f2657c7ea942b7a553fb205e94fd4fb0edc34 |
Hashes for pyodps-0.11.2.1-cp39-cp39-manylinux2014_aarch64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 29e7ccbc6a961ba95e71f21b01e5291438912bb5e6b4a5e8f73c2a806f674f36 |
|
MD5 | 056acaad8125c16447037d961dcde2e2 |
|
BLAKE2b-256 | 4b5153b108d932b41ca99b4a63f91670b03fb16a7162f9540c585d945fc27658 |
Hashes for pyodps-0.11.2.1-cp39-cp39-manylinux1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | bc0553c1d6572a9030fe55779db43d2974a5e5dc5c48ce3f9960510b54811a21 |
|
MD5 | 7d5c6b32621991110b4c1106f5d7f2c1 |
|
BLAKE2b-256 | 2fee09ee4fcee155943f0aabda6fb6f457e0dae4c74d93f8f6897aac53d66726 |
Hashes for pyodps-0.11.2.1-cp39-cp39-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 766a845495388cd646050b1748d823c7d39c84dcde11cdbcac72c7867a1483c8 |
|
MD5 | bb477985c43c8e81841a2b271137430b |
|
BLAKE2b-256 | 6482abf2ae2df55e08879791615e55672efe5075658dc90cc38ce95c3d0be3d6 |
Hashes for pyodps-0.11.2.1-cp39-cp39-macosx_10_9_universal2.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 19bab8d2fcfc067580575e4edf15ac97845b84b3fe70670dd9ef8b2f517921c3 |
|
MD5 | d6edc4a01a530cffa8fd96dc5cd38c70 |
|
BLAKE2b-256 | 128117dbd81a1bf8647c751067d59f6f7739570f9e88ac4b042f5b6d6c57b616 |
Hashes for pyodps-0.11.2.1-cp38-cp38-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 97c9f56ea2316b48c98f4a2729fadf03c06a718ad35d3615d3d875c0fc68d13e |
|
MD5 | 1403dc900d3a55fd0e98a2c6e2f5ed9a |
|
BLAKE2b-256 | 94d9d1b0c207af766ee40ba60a8eaa657e48810de65a34deccf86dcd01701eab |
Hashes for pyodps-0.11.2.1-cp38-cp38-win32.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | bb1cfdb2e0f48e606fbfe2124375dc6b49e07d9753e95161b8cfdb17b889d119 |
|
MD5 | 9d9ac1543ad30eb37b4f071355db1d31 |
|
BLAKE2b-256 | e9654d129fb134ac6f2f74e717387ea0bf03bcd719f9cb5e12ae623377481502 |
Hashes for pyodps-0.11.2.1-cp38-cp38-manylinux2014_aarch64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | e962b09384ecfad694ff58f6857d7fa14d35bce4d0659748a5a4ec9ee92cf917 |
|
MD5 | 39717fdb896e9f0959e0e90a12636f0f |
|
BLAKE2b-256 | 9811d574bf5b91f220361707571ca3cca8ac6f8ebc2b192c220169db40659b3f |
Hashes for pyodps-0.11.2.1-cp38-cp38-manylinux1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 0d8ff3c6ce3b2a572551719bafbe217877367f3177e1ad0eecde3e7a91bfe3a9 |
|
MD5 | d6b0aa8cb1a03db86faf5b53e56ad8fd |
|
BLAKE2b-256 | 7381870ab4fc2509b32cfde40179a3507f877cd7e0f552750a0b28d8ed072a54 |
Hashes for pyodps-0.11.2.1-cp38-cp38-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 90edaa4e43a9518bc5279e0e8033c8f75fe92c50a40a5a63dde576b58300f37f |
|
MD5 | af48060060a4df3f59fd33e218e5b3a2 |
|
BLAKE2b-256 | 2cca11966642575eb54141f9b33b11d77334e0e72636a2006470cee9bf8730d0 |
Hashes for pyodps-0.11.2.1-cp37-cp37m-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 2ea816e7924a154103685ad58716bb2a7fb354383ae843e27aad885bf41c214e |
|
MD5 | 9fb286e91119aef59129c13061764611 |
|
BLAKE2b-256 | 11510de9aa05be5c0937c6b93c0d25bf2168eab03b13b70326624415511e1f2a |
Hashes for pyodps-0.11.2.1-cp37-cp37m-win32.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | a9001ce08748e85d2b81e61cab996cba343b178b96e659087fd1dcc5433d3706 |
|
MD5 | b3394e1a4593d91a4e4e79103c90fa7b |
|
BLAKE2b-256 | f4d514038ff667e75763af0a2d47acc7763f1550b4b8181e592427164fb4664d |
Hashes for pyodps-0.11.2.1-cp37-cp37m-manylinux2014_aarch64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 60031c6ba3faeb977fb03a6ad14e05c29a19305226b06476b5e0bb4039add92a |
|
MD5 | 16b0aee008b866674b054abaf9d0cd99 |
|
BLAKE2b-256 | 89142f0158d10b5d2af51e6dcd7d27df98ac616f7d33e16d838d8a8a6d141a1b |
Hashes for pyodps-0.11.2.1-cp37-cp37m-manylinux1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | fb519632ac128cc983c68bbd61b1b5a9507a274e25d6e76979a43ff06098cff5 |
|
MD5 | 61bc4dfc3204885ccff2a8542fafd698 |
|
BLAKE2b-256 | 3422c56d7353c1d1de0409f0226afc2c684c57abc80676a5561e61a05c37c255 |
Hashes for pyodps-0.11.2.1-cp37-cp37m-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 8043b041f9b8beba64917538bf4b3075427e5a92cec1fa812988d5c645c749de |
|
MD5 | 1caec174c79caf2290d9b057de30829a |
|
BLAKE2b-256 | c6fbf467a48c6e4256bc88dce95bd932265f843ab1f31a6dca65bbb274a9d69e |
Hashes for pyodps-0.11.2.1-cp36-cp36m-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | de9dba5a5df1ae682ab9ac7afeec13a25fe7916837cdc0c7faa84e318fa78f5d |
|
MD5 | 3564bbcc624036900868a3687a6748f3 |
|
BLAKE2b-256 | 471b06c32eb8b7aca4519366fd1bdcbe513727c0b97cbbfec5b1c67863c1d763 |
Hashes for pyodps-0.11.2.1-cp36-cp36m-win32.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 7fd6d7700d4dde5fb11eb5870a824e68a6dec110c3555fa4ae788b7080411bf5 |
|
MD5 | c1965d117ce6d8b1c557ef7490560720 |
|
BLAKE2b-256 | 013d0b01ec2a23fa51b4a4e5c8d192542c13aedb9bb117fa50565bfbe9e1fe61 |
Hashes for pyodps-0.11.2.1-cp36-cp36m-manylinux1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | cfabea39695daff0be8fa7ba24973926c14f442f190b83fd3947ab80a6ae8d24 |
|
MD5 | 6ae0b7f070538b91c3bc9e3e5e995ff1 |
|
BLAKE2b-256 | 964b43e4509b9be93f1d47b133da7445b4df2666da76268a14d80a8d9755f51f |
Hashes for pyodps-0.11.2.1-cp36-cp36m-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 5c923585606df135f6fda38de5e0d94185c0b99705087434c85f6721877c3171 |
|
MD5 | 45042a4fc9248629c0bc5e0dcc035d80 |
|
BLAKE2b-256 | 73c3c048ed1d8d1c56007d46fd4115af16eea47d5b0b44285cffe30fc0fa11f7 |
Hashes for pyodps-0.11.2.1-cp35-cp35m-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 632f17d743c6b7e83ecae779930479dc99f5c0874833e6cad913cd985fca7135 |
|
MD5 | 3c6d9a7eda74d73e2740d36d49472e32 |
|
BLAKE2b-256 | b8156b211bc6485932a30cb431d4706a93d73d3c8536065f5eebf147f7ac2d1b |
Hashes for pyodps-0.11.2.1-cp35-cp35m-win32.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | cbb3c5920d125505cf20e5f818911c49832c407828dfb159a43370e86999d3a4 |
|
MD5 | d78e6f3752567422ee7ddb938e8abb61 |
|
BLAKE2b-256 | dbf28541e54e907dde8c89d9c8d6b26844ff6bb9c14af97c09aa5f01852a8f4d |
Hashes for pyodps-0.11.2.1-cp35-cp35m-manylinux1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 31cb8542fd0dd2edef1f92ef8a8a330c0e8025cd1c603bfabf05b14f4a853851 |
|
MD5 | b8f8778de64ca82b1b9f40eae1d00aaa |
|
BLAKE2b-256 | 30e971655d86d8850757430f2b167f34b2b5602560e85fe34dbbe1c9224bd857 |
Hashes for pyodps-0.11.2.1-cp27-cp27mu-manylinux1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 4fd53d78d23a03fb82903ebb7c74b089643f72ae670c4690d8ce7b0d0e2cbbf7 |
|
MD5 | f287b3bc76441e366f9426ab24ed5d8c |
|
BLAKE2b-256 | 2921cd5290064a7a403282c6f8ddb922dad61e100f828064dacea1bec6ea40f5 |
Hashes for pyodps-0.11.2.1-cp27-cp27m-manylinux1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 484cf95ef80340563112f94d81ffbc43eebb7c2fbcb1f6fad5ee867a431bc332 |
|
MD5 | d7b3ef4ac572142487afc100be3ed97b |
|
BLAKE2b-256 | 9f1a076cd00b5c7596c1a79a83b5ee5b45790e1edb5373a44f20d3c26c8f27dd |
Hashes for pyodps-0.11.2.1-cp27-cp27m-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 6cd2b180a4dd9c0c1ec45423cfc1e08e2527b21328e9e43054a57f1772476637 |
|
MD5 | 24cb51f671c3c7ce026e8fdee135e3ff |
|
BLAKE2b-256 | 7571c829f37bb320d983e411fc11bf5b4f1b87b34459b5a62d81a0b5a44ac4e7 |