ODPS Python SDK and data analysis framework
Project description
Elegent way to access ODPS API. Documentation
Installation
The quick way:
pip install pyodps[full]
If you don’t need to use Jupyter, just type
pip install pyodps
The dependencies will be installed automatically.
Or from source code:
$ virtualenv pyodps_env
$ source pyodps_env/bin/activate
$ pip install git+https://github.com/aliyun/aliyun-odps-python-sdk.git
Dependencies
Python (>=2.7), including Python 3+, pypy, Python 3.7 recommended
setuptools (>=3.0)
Run Tests
install pytest
copy conf/test.conf.template to odps/tests/test.conf, and fill it with your account
run pytest odps
Usage
>>> from odps import ODPS
>>> o = ODPS('**your-access-id**', '**your-secret-access-key**',
... project='**your-project**', endpoint='**your-end-point**')
>>> dual = o.get_table('dual')
>>> dual.name
'dual'
>>> dual.table_schema
odps.Schema {
c_int_a bigint
c_int_b bigint
c_double_a double
c_double_b double
c_string_a string
c_string_b string
c_bool_a boolean
c_bool_b boolean
c_datetime_a datetime
c_datetime_b datetime
}
>>> dual.creation_time
datetime.datetime(2014, 6, 6, 13, 28, 24)
>>> dual.is_virtual_view
False
>>> dual.size
448
>>> dual.table_schema.columns
[<column c_int_a, type bigint>,
<column c_int_b, type bigint>,
<column c_double_a, type double>,
<column c_double_b, type double>,
<column c_string_a, type string>,
<column c_string_b, type string>,
<column c_bool_a, type boolean>,
<column c_bool_b, type boolean>,
<column c_datetime_a, type datetime>,
<column c_datetime_b, type datetime>]
DataFrame API
>>> from odps.df import DataFrame
>>> df = DataFrame(o.get_table('pyodps_iris'))
>>> df.dtypes
odps.Schema {
sepallength float64
sepalwidth float64
petallength float64
petalwidth float64
name string
}
>>> df.head(5)
|==========================================| 1 / 1 (100.00%) 0s
sepallength sepalwidth petallength petalwidth name
0 5.1 3.5 1.4 0.2 Iris-setosa
1 4.9 3.0 1.4 0.2 Iris-setosa
2 4.7 3.2 1.3 0.2 Iris-setosa
3 4.6 3.1 1.5 0.2 Iris-setosa
4 5.0 3.6 1.4 0.2 Iris-setosa
>>> df[df.sepalwidth > 3]['name', 'sepalwidth'].head(5)
|==========================================| 1 / 1 (100.00%) 12s
name sepalwidth
0 Iris-setosa 3.5
1 Iris-setosa 3.2
2 Iris-setosa 3.1
3 Iris-setosa 3.6
4 Iris-setosa 3.9
Command-line and IPython enhancement
In [1]: %load_ext odps In [2]: %enter Out[2]: <odps.inter.Room at 0x10fe0e450> In [3]: %sql select * from pyodps_iris limit 5 |==========================================| 1 / 1 (100.00%) 2s Out[3]: sepallength sepalwidth petallength petalwidth name 0 5.1 3.5 1.4 0.2 Iris-setosa 1 4.9 3.0 1.4 0.2 Iris-setosa 2 4.7 3.2 1.3 0.2 Iris-setosa 3 4.6 3.1 1.5 0.2 Iris-setosa 4 5.0 3.6 1.4 0.2 Iris-setosa
Python UDF Debugging Tool
#file: plus.py
from odps.udf import annotate
@annotate('bigint,bigint->bigint')
class Plus(object):
def evaluate(self, a, b):
return a + b
$ cat plus.input 1,1 3,2 $ pyou plus.Plus < plus.input 2 5
Contributing
For a development install, clone the repository and then install from source:
git clone https://github.com/aliyun/aliyun-odps-python-sdk.git cd pyodps pip install -r requirements.txt -e .
If you need to modify the frontend code, you need to install nodejs/npm. To build and install your frontend code, use
python setup.py build_js python setup.py install_js
License
Licensed under the Apache License 2.0
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distributions
Hashes for pyodps-0.11.4-cp310-cp310-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 86fdb444ca51fa9172d2c23804e218d26e2df191842cf31253b57e7744e2e26f |
|
MD5 | 8801d43012e9f7113b4159b3dcebf12e |
|
BLAKE2b-256 | 012af1f1d9af28b44221e99f9e3133377d1f46f03b1442cb5dff378908561407 |
Hashes for pyodps-0.11.4-cp310-cp310-win32.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 963225cc44a664ea99b3010c66e935b40493d45b862a738a3320cd3f6beb0ed7 |
|
MD5 | 9a96add6e63b036c60527903fe098792 |
|
BLAKE2b-256 | e1991ed183a557c8e8a0b7d28403117156fc1435c83643d7499ebd76ab52871a |
Hashes for pyodps-0.11.4-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 9ac2978efcfeff64a82346bedc23929f91dd5a7fc7a486f99bfc39351cb2ad36 |
|
MD5 | 47d2b92cbe57e103aa66f15ec871c84f |
|
BLAKE2b-256 | 168c6be5f89ef1737e7d6fd18e76268946f59cd37f342b4720f43c4f4cbd5ed9 |
Hashes for pyodps-0.11.4-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | a794e138adf0c777f0d14ee391de8434232202e3b8e2eceb9e6bc4b0f029f433 |
|
MD5 | c2a157273d429bb4370fbb4d29b88b82 |
|
BLAKE2b-256 | b18da583af88c5bf1b72f569877020d0e2d88f2aaddf71d11405149ce9c3cefe |
Hashes for pyodps-0.11.4-cp310-cp310-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | d4ec4c3199daaccb57c898c72b9884edf5092c52bf1140722ed1f44347c5e800 |
|
MD5 | 1dc33540219534380d5a861fa4406807 |
|
BLAKE2b-256 | 362c9cece5a2cb8f5e0d07418cada6d0e74d528096ebd86c46f6b00544801aa7 |
Hashes for pyodps-0.11.4-cp310-cp310-macosx_10_9_universal2.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 05ff44a041ed84dc5cbfc1dd8c585cee34ff9eb6b1d954415c5f8a58335a3b6d |
|
MD5 | db4d5329a921dcb2b648e50e75a4f06f |
|
BLAKE2b-256 | c495dc2a15baf5baf80bcd62771b220638621bee16016646ce7314d9286e1e74 |
Hashes for pyodps-0.11.4-cp39-cp39-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 35a35d58f67bf02309b916beeed1c8ee59465e391b8aac242aade7b7fbeb8258 |
|
MD5 | a9c9dfd3ade56666ff7bfa41e8e75b16 |
|
BLAKE2b-256 | 24205b72ecb30a85ddc137e4d2af898755c452e110e378ea59d6da7f9d0177de |
Hashes for pyodps-0.11.4-cp39-cp39-win32.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 39cc1d76fb90c1c0d65e8f7076039435781f0f6591e41cb829a455b0671811a7 |
|
MD5 | 0349de9b369d31fdc26f85589b75bf2d |
|
BLAKE2b-256 | 84c9ad298d8c6ecacc2c61471606c7c394fb8cfb6eee2b32e1301139e85782fc |
Hashes for pyodps-0.11.4-cp39-cp39-manylinux2014_aarch64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 27d3c5f7227af1f5fe721328fb9421b6e2b944e8f0ef610d667c1640b5653b55 |
|
MD5 | b3e3d0869515c87e7237908017ba98cc |
|
BLAKE2b-256 | 2206b235a355e19bcd94e670d3646a1250d6318432f1b6b8734d157f070bf6ea |
Hashes for pyodps-0.11.4-cp39-cp39-manylinux1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | a5f586d405039a2ea8b6b816e871b5903f719675f2c567586a2bed899e16a9ba |
|
MD5 | 6848ffe0929c9895a0ec67a43e34b748 |
|
BLAKE2b-256 | c9485e76c453a4f9f0032aef80f59d16fa2ebf87f1585d964d4f20847a2d128b |
Hashes for pyodps-0.11.4-cp39-cp39-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 391d496bc758726326250756b63c9edb1a2612559c3db32bd43746abd9aa3e08 |
|
MD5 | f42d519b6ab6463394c750d2bf94520b |
|
BLAKE2b-256 | 4fc106e37c896ecc9641611c954a2eeef473e8fdb9b5b42650b9feb68ec9c224 |
Hashes for pyodps-0.11.4-cp39-cp39-macosx_10_9_universal2.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | d97c06a0a7f20074e4daf2be6dcfdbc7d7dc1e0bf96887edda07220c41ed6482 |
|
MD5 | d4c3455b78f52356d5cc992d0e71df46 |
|
BLAKE2b-256 | 710d897760a0b7f519e9aa5ed0c3aaac07be03127df51b947ecad2ea645bfadc |
Hashes for pyodps-0.11.4-cp38-cp38-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | e3c438242652eddb49f8e84588bd865b08557fc79eff4ce4bf2696f941eaf611 |
|
MD5 | 3aad770790ed11bb446efd8da3131b19 |
|
BLAKE2b-256 | 19abee7b74a8a08c3e4ceb6a9c64df851b772e96d8663cfe516f216f5c7ac1d6 |
Hashes for pyodps-0.11.4-cp38-cp38-win32.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 4688c29e77a824c01c4156c06071611a3b0d431e9d4fe43ad7269f2b16d04d58 |
|
MD5 | 21d03662a88965e6ac27c8a3c3d3a08c |
|
BLAKE2b-256 | 7c50fd5f41e53ac44dd9bc023162f0f246c025f883e9b9271378bab5c2c9a9a6 |
Hashes for pyodps-0.11.4-cp38-cp38-manylinux2014_aarch64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 944521ab788be421ef57ca55595aee4de60757407feee9a6bdd7d81bcbc7973d |
|
MD5 | 268d3aae14291ac221bdc6b3a93f8b20 |
|
BLAKE2b-256 | 609409f00455feb51c3d19a638b5565ba8eefaf8a33e664d38d4f84f8b7cdeb2 |
Hashes for pyodps-0.11.4-cp38-cp38-manylinux1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 87d9508fb9f194a79fffb2c546e4eefee23e636d7194902580df6156537b15d2 |
|
MD5 | de843ad26a0cf38368cd400ce319b6c4 |
|
BLAKE2b-256 | 2fa185fe97c38ec656a69137ce8f197791fbbfa3edbd78ed9c8c50faa4abedf0 |
Hashes for pyodps-0.11.4-cp38-cp38-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | cb551e33e05d6666ff5f308236747bff1c05c145fecbdd90c9115975d8a5ccad |
|
MD5 | d527e80c68835fa7e2b64252a0b77489 |
|
BLAKE2b-256 | f3dc6a18b0f8e6492a6e642cc80b2f005391bf91ef7bedb25077fc858924ec77 |
Hashes for pyodps-0.11.4-cp37-cp37m-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 3b4e37136e06e0a3e7044c5606cc02b79db119490f41277a5031f50d031bf519 |
|
MD5 | cd2d68738760136898453126f40050e7 |
|
BLAKE2b-256 | ef661c8b2ef54a04a5d3bd4761c48f2a74a301695692eb99b7ba891da84afffe |
Hashes for pyodps-0.11.4-cp37-cp37m-win32.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 8545f1c8a573bcbb8744b17fea1813cf550270e0f3cf33fe7758b3703d0e3d72 |
|
MD5 | 24dd36a474b163126a024c1b0ed0fd11 |
|
BLAKE2b-256 | 648de25a9fc71d51f87e1585927c61b57c362c61a0979996da8cff737442f45c |
Hashes for pyodps-0.11.4-cp37-cp37m-manylinux2014_aarch64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | c9b7a9a265cc20440454b49ee47291b806521b2632ff63fb76ca882e23776ad0 |
|
MD5 | 82e93617f424f5ea416c74709ce77308 |
|
BLAKE2b-256 | f26813c20ed47b0b3c2a4e49b7217dd3b7ded6636cdede78b5ff83d53a5c3835 |
Hashes for pyodps-0.11.4-cp37-cp37m-manylinux1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | b989f072747884bacac6a5d10067bd6bce91d2f5abcee5bf4ec3a782b0b00d80 |
|
MD5 | d3de21ae0fc26ca7d16150c7efbeff94 |
|
BLAKE2b-256 | a22c5ebf0206068adefca13f3b3c46edfb7d620563af6d5974f32fced36b9fc6 |
Hashes for pyodps-0.11.4-cp37-cp37m-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | eede25c3df87b35b1fac98b4dea980630fe51fbf80ee66600bfd6aba9c016a43 |
|
MD5 | 7799827b5bd87165a33a7270ce255a71 |
|
BLAKE2b-256 | 6a748ffecb9017174772b1c58e6dced95fe333cadcf0760b4ca123ff6b7813c3 |
Hashes for pyodps-0.11.4-cp36-cp36m-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 9ba33d3b34b14ee855a931fb254fc440b8e262ab8e0765078c719475b9cbc6fa |
|
MD5 | ba29739ccd8f147f568b1e0c28c5e5be |
|
BLAKE2b-256 | 9c53b25801427b05aa04d05b8b2f246d63ce6a2c2c0b2ad0d131953bd3421517 |
Hashes for pyodps-0.11.4-cp36-cp36m-win32.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 956459d8798dc50e20586b2d2b5af177d237e7c3b7dede5c38de6d5e6f99ffe1 |
|
MD5 | 7266799a43b9c110ddaba7f6eff9ffce |
|
BLAKE2b-256 | b37672dc90820f7815f42298b5c4dc7df6fee4907b2227ca8a86d6d21a7227da |
Hashes for pyodps-0.11.4-cp36-cp36m-manylinux1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 98367b7b53caaee6aceba41c4a4c6e08909d7753cf02f05ddb0791114dd473f0 |
|
MD5 | 2631bd35eec953b2317cc78b7fab44da |
|
BLAKE2b-256 | 5bc4d05fb73664c537191a2c0fce577f18396cd363a82951b893d9689bb8909d |
Hashes for pyodps-0.11.4-cp36-cp36m-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | bc68e2a030f6382808af8ac7e966ce31537ac490aff73b3c127ea5efdd3ddf90 |
|
MD5 | 50e6dd0bd4213d9f77268bfab9ee55d7 |
|
BLAKE2b-256 | 34b044ead9074f773ecf048c679f9c4d65bf75938f91b8f71f438cfc12937843 |
Hashes for pyodps-0.11.4-cp35-cp35m-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | fb2610d1acf291ef7f41e21acb82589606c71cee9e3979723f8867655ef925f3 |
|
MD5 | 11ef50b324351c2c1e4e6571a9ae941a |
|
BLAKE2b-256 | 1b6f5b259484dd7e9c64cba6c07be9cd68fda45e879dbea4948c3c886684fbde |
Hashes for pyodps-0.11.4-cp35-cp35m-win32.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | d2caa1bc72d888c4cb9962e7c9c1a6b7f940fbd390218684f0a12cc9a898c3ea |
|
MD5 | 535069bf077a37c7faf07a201551af80 |
|
BLAKE2b-256 | e58792608e7f4ca1047cb33ef2664e88458dc86fecfdd1c7554a13c6d46f48f6 |
Hashes for pyodps-0.11.4-cp35-cp35m-manylinux1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 61bdfdee1bedeb5ee088b567bee78a07b6dc4324976f2785eb5e7d606812a3ac |
|
MD5 | d4ccdddffe2c51c693e4518b2105e154 |
|
BLAKE2b-256 | 0fb7db240637a20a3eb5e60e5b46ca17182e214ea5bfdea7ba422cdc71f32df5 |
Hashes for pyodps-0.11.4-cp27-cp27mu-manylinux1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 67837fd4c1fcebfb4287447de2edf5d7c3392becfdf07c13aeaa946ee67d4b6a |
|
MD5 | 072fa90ae1b4a3732d8d29a66831c34b |
|
BLAKE2b-256 | 4c83529fbc2a9ac2bc860c06a129836b10dc708d6e5a51b72835a58ac33007a8 |
Hashes for pyodps-0.11.4-cp27-cp27m-manylinux1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | ffd668feda559353a3b0ac052572cc1af75522a3fe44c6d008b9c0aca96784b8 |
|
MD5 | 2d8667812e6911bfa7cce96e530ac660 |
|
BLAKE2b-256 | 2cb8e7606a0eb282d759eafa80fd2d759ba6dd37e5d3c7574f21775ce1f9f603 |
Hashes for pyodps-0.11.4-cp27-cp27m-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | d51a41181fd46420d9f5b0a3b52359e3a1e49c47a1b51b1b47e0b1ca63fe459b |
|
MD5 | 48218f230eef488242282f5a223a73bd |
|
BLAKE2b-256 | 811fbd1a3480fb9c7bec5f75d6dd5759a543af0a4bb49b2be2a194a49c4c746c |