ODPS Python SDK and data analysis framework
Project description
Elegent way to access ODPS API. Documentation
Installation
The quick way:
pip install pyodps[full]
To integrate with Mars:
pip install pyodps#egg=pyodps[mars]
If you don’t need to use Jupyter, just type
pip install pyodps
The dependencies will be installed automatically.
Or from source code:
$ virtualenv pyodps_env
$ source pyodps_env/bin/activate
$ git clone <git clone URL> pyodps
$ cd pyodps
$ python setup.py install
Dependencies
Python (>=2.6), including Python 3+, pypy, Python 2.7 recommended
setuptools (>=3.0)
requests (>=2.4.0)
Run Unittest
copy conf/test.conf.template to odps/tests/test.conf, and fill it with your account
run python -m unittest discover
Usage
>>> from odps import ODPS
>>> o = ODPS('**your-access-id**', '**your-secret-access-key**',
... project='**your-project**', endpoint='**your-end-point**')
>>> dual = o.get_table('dual')
>>> dual.name
'dual'
>>> dual.schema
odps.Schema {
c_int_a bigint
c_int_b bigint
c_double_a double
c_double_b double
c_string_a string
c_string_b string
c_bool_a boolean
c_bool_b boolean
c_datetime_a datetime
c_datetime_b datetime
}
>>> dual.creation_time
datetime.datetime(2014, 6, 6, 13, 28, 24)
>>> dual.is_virtual_view
False
>>> dual.size
448
>>> dual.schema.columns
[<column c_int_a, type bigint>,
<column c_int_b, type bigint>,
<column c_double_a, type double>,
<column c_double_b, type double>,
<column c_string_a, type string>,
<column c_string_b, type string>,
<column c_bool_a, type boolean>,
<column c_bool_b, type boolean>,
<column c_datetime_a, type datetime>,
<column c_datetime_b, type datetime>]
DataFrame API
>>> from odps.df import DataFrame
>>> df = DataFrame(o.get_table('pyodps_iris'))
>>> df.dtypes
odps.Schema {
sepallength float64
sepalwidth float64
petallength float64
petalwidth float64
name string
}
>>> df.head(5)
|==========================================| 1 / 1 (100.00%) 0s
sepallength sepalwidth petallength petalwidth name
0 5.1 3.5 1.4 0.2 Iris-setosa
1 4.9 3.0 1.4 0.2 Iris-setosa
2 4.7 3.2 1.3 0.2 Iris-setosa
3 4.6 3.1 1.5 0.2 Iris-setosa
4 5.0 3.6 1.4 0.2 Iris-setosa
>>> df[df.sepalwidth > 3]['name', 'sepalwidth'].head(5)
|==========================================| 1 / 1 (100.00%) 12s
name sepalwidth
0 Iris-setosa 3.5
1 Iris-setosa 3.2
2 Iris-setosa 3.1
3 Iris-setosa 3.6
4 Iris-setosa 3.9
Command-line and IPython enhancement
In [1]: %load_ext odps In [2]: %enter Out[2]: <odps.inter.Room at 0x10fe0e450> In [3]: %sql select * from pyodps_iris limit 5 |==========================================| 1 / 1 (100.00%) 2s Out[3]: sepallength sepalwidth petallength petalwidth name 0 5.1 3.5 1.4 0.2 Iris-setosa 1 4.9 3.0 1.4 0.2 Iris-setosa 2 4.7 3.2 1.3 0.2 Iris-setosa 3 4.6 3.1 1.5 0.2 Iris-setosa 4 5.0 3.6 1.4 0.2 Iris-setosa
Python UDF Debugging Tool
#file: plus.py
from odps.udf import annotate
@annotate('bigint,bigint->bigint')
class Plus(object):
def evaluate(self, a, b):
return a + b
$ cat plus.input 1,1 3,2 $ pyou plus.Plus < plus.input 2 5
Contributing
For a development install, clone the repository and then install from source:
git clone https://github.com/aliyun/aliyun-odps-python-sdk cd pyodps pip install -r requirements.txt -e .
If you need to modify the frontend code, you need to install nodejs/npm. To build and install your frontend code, use
python setup.py build_js python setup.py install_js
License
Licensed under the Apache License 2.0
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distributions
Hashes for pyodps-0.11.2.3-cp310-cp310-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | c6031e754d762ae37edec480f264e9b6b325ffcb53b6e43f5eac031f50205bb0 |
|
MD5 | 82ddc6c3ef9ff1d2423b4eb50a20b7c2 |
|
BLAKE2b-256 | 94732a5490ded6b583470ef75df074926a107565eb6b5a04a70d94f62cf12415 |
Hashes for pyodps-0.11.2.3-cp310-cp310-win32.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | f61893b4a2e7bc6f9c913efaab2dba41c70cfc5c0dd0228e8d769492ab742269 |
|
MD5 | a9b86b5c65c17f91ca0571c1ce177b3c |
|
BLAKE2b-256 | 7f374a6d6c928dbe384e1464cb0492b7820eae986cb15a462e2088947bdd508a |
Hashes for pyodps-0.11.2.3-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | eef2f60eda36dbf1e42bf9f37349edaf48735f1a51a4d672959c27c524fec9c4 |
|
MD5 | b25b0c09588a3c8b62f6864645a2f30f |
|
BLAKE2b-256 | a5eb4c43d40b91ebb7a23089798d9e76c26db9e2d85b79ae3d21982cf72b8d92 |
Hashes for pyodps-0.11.2.3-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | afd673fe240c7f2d6ef2c774f57ec59ad01d351f45a6a6c85b09d31b4735f269 |
|
MD5 | 450a311dca969aacf8e0a6730f24cf7e |
|
BLAKE2b-256 | 69188ee014491015dc4dfba9ceb2f652660223e8190f846aeaaef52b8754c59f |
Hashes for pyodps-0.11.2.3-cp310-cp310-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | e21cbb0be4a7452994b9ebe23ab3ff5b2f31885b0c8001da4e98e9f3fd25f031 |
|
MD5 | 8e4b3f38d16729e11ca8e80af182b381 |
|
BLAKE2b-256 | 12781764fa30eabf1092c84590bd1455f830029693ff94c424ca942fa2e01081 |
Hashes for pyodps-0.11.2.3-cp310-cp310-macosx_10_9_universal2.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 271edc053e1eab72ad6034701e1aa5f26055c123927a1095d863d15f3f6c54ce |
|
MD5 | b3254f5c3937c68879b82bc859e96ad2 |
|
BLAKE2b-256 | 8015e59834137ef9af958532e551b3a9fd63ba0b2afca0a1f43176088e17d7b1 |
Hashes for pyodps-0.11.2.3-cp39-cp39-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 490ca6eed47e6c545378b756ed61ed8ece6e93d739dde5b1de659b21cc258c48 |
|
MD5 | 168c8d4552a876f9dea43762439adce3 |
|
BLAKE2b-256 | e882b68c18062f5ba1161014c275a6d4b75ecab085ae8643095a9f5aebb9a948 |
Hashes for pyodps-0.11.2.3-cp39-cp39-win32.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 75c907fe8334f48624b860e9844acaa766f3679ef6cb391abe645c37c19cd572 |
|
MD5 | eff338ffb9865600bdbb942816a69f91 |
|
BLAKE2b-256 | d2ad499decd8890e602dd26ea228657c4fed2e880ae6b1581b64761ed327c4f7 |
Hashes for pyodps-0.11.2.3-cp39-cp39-manylinux2014_aarch64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 603a676edb846bfaf8bf77184263519458de7517a6c8584fd66aa7747be82b4b |
|
MD5 | 42959671f85268fbb593efc31cd064ab |
|
BLAKE2b-256 | 107bc08727cb677b387794f37662a3160270cfe124704d4dc4cba92117c06f00 |
Hashes for pyodps-0.11.2.3-cp39-cp39-manylinux1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 69634ccfd427c89eaba06726f3af515bac61b4ed2cebbca6d25611e0125e3ba6 |
|
MD5 | 31752f0399301febd607cee4d3e3849c |
|
BLAKE2b-256 | 23728a96fe9700f2f93b4aeb954b2b3f890416d84e7c155ed40cd9c90bc89961 |
Hashes for pyodps-0.11.2.3-cp39-cp39-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 54fe6475c259dd7d2bae141a9c130154ad988a50fe577cf0566086c89a32d26f |
|
MD5 | 12abb71625bc43f2f926d3436f0fa1ed |
|
BLAKE2b-256 | e5c5a6aea4a7df40faee6ba6e223afba5ee6bc49015d005b7342c8bc81127d0d |
Hashes for pyodps-0.11.2.3-cp39-cp39-macosx_10_9_universal2.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | ffa7d14e8ddab9dd736c623fd4a6af9304c6a955b72c981f237cfdfc120a8b1c |
|
MD5 | 9f0f67b8c9eb30411f582551a9a41e02 |
|
BLAKE2b-256 | 24cc240689a306ad2c52d3724af8be8e8650dde69c44253a342ce854cad10da7 |
Hashes for pyodps-0.11.2.3-cp38-cp38-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 9dc396e4567dda62256836c4ec1b309b0b83f6f1811042b5054c9831fc2780bf |
|
MD5 | 0a163654d63e1f38f86d51599a3e7017 |
|
BLAKE2b-256 | 7a62267cd0dbf4ccff271231d588a03e94c02a133b3d4bf7ff4ac85612762f56 |
Hashes for pyodps-0.11.2.3-cp38-cp38-win32.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 1a83d71feeeaa376df9adc0e3120c473b356682f5609f7cf14451580fd264b03 |
|
MD5 | 2a1bcd186e4e7dfebe5ba424b9542070 |
|
BLAKE2b-256 | 2094d43282ab2a026f3580fadecf47c243009e0acffac9db7f0cffb96304186e |
Hashes for pyodps-0.11.2.3-cp38-cp38-manylinux2014_aarch64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 27de24f5611959f784147978f6b5d2bc923edfd7e8912843100298408dc12826 |
|
MD5 | 7cd46952d7bf17108c99cf510a4b0690 |
|
BLAKE2b-256 | e76857fc48a3fc2e748d69114d85cc4667b867341c5d883f49c3fa7181bc39ba |
Hashes for pyodps-0.11.2.3-cp38-cp38-manylinux1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | e6fbfa56f2383599d37faa7e8a2f28476dc7d381f5f342ccd466ddd4901509be |
|
MD5 | f3c453942bce536ace7227ee5032a336 |
|
BLAKE2b-256 | 3430488a40d159c65538fe334af8d109b6806a3866b80ae5ba844cbe4bd729a7 |
Hashes for pyodps-0.11.2.3-cp38-cp38-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 6eccec13f6e0e7fa83653c7533594e56b225de538213c9582b113ddc17134a21 |
|
MD5 | f38c5da50245b39ed776e88499f9f277 |
|
BLAKE2b-256 | 6b99b146692c85ef9db910589a14deea93509b6d7bbe79ec455906a5fc40a7e6 |
Hashes for pyodps-0.11.2.3-cp37-cp37m-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | dac013c2a357a12c063b2be321dc991a77342532620fd338f48509c1babd9cd9 |
|
MD5 | 3fab559730db7d1bdadafbbaa2fe1c50 |
|
BLAKE2b-256 | 43737411ea95f0f61a5cc96768a94a03118d55de550b6eefc3a4f122e281367f |
Hashes for pyodps-0.11.2.3-cp37-cp37m-win32.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 25d2559deae2da61be3eed11f4523fbbbc8aa4f5b82b4f9b44a9bec53b3ae436 |
|
MD5 | 2ff42a8f2dce189d76b1134489c1ecbe |
|
BLAKE2b-256 | f1429271b4f20114f44b3f989bdcb31c63862e67fc57b2c22a46b46f3ea8ff04 |
Hashes for pyodps-0.11.2.3-cp37-cp37m-manylinux2014_aarch64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | d0d2490ad360a1b2158d962efb8251c8063387fc40a49c9b0e9dcff25619aede |
|
MD5 | 84e7034e99be2a117e437ff6ebb5fb96 |
|
BLAKE2b-256 | e40c4bb3ee9b39acb6e13717383977beed448364ac3e9f7f29b4b59167012314 |
Hashes for pyodps-0.11.2.3-cp37-cp37m-manylinux1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 9cffc182daae67885de6aa451aab9aec129689ce3688e0e3ace3ebdfbc990077 |
|
MD5 | 190b5f9babb0b504ed05792d3634f38a |
|
BLAKE2b-256 | 6f5db83d31a877d47406ae3cf95db8ae0fa1bae9e45f60da426e9574d98d08d2 |
Hashes for pyodps-0.11.2.3-cp37-cp37m-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | d2b29cb36113ba279141200948fb5afec1fcd4504200892a9d7d02d9bd9f4850 |
|
MD5 | bcb78605c18549a1ec883dd32700901d |
|
BLAKE2b-256 | 5a0969318eb1ea3f69f9abd9c1e9979163e73e7a350985db9fd423fa1eb2353c |
Hashes for pyodps-0.11.2.3-cp36-cp36m-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 171036561a3b3a860a8ce8a0e4c635635b4ac196a28f04e38a86d14268d56ba3 |
|
MD5 | 3057687af66e33b3f9bf531d2529af6e |
|
BLAKE2b-256 | b1a7b11e69e05935fda3132b6d6410f3da0d06c86910e5f83945a23a4fa74265 |
Hashes for pyodps-0.11.2.3-cp36-cp36m-win32.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 4386b5ccbd921f03c64d8cea266407d7179a4b8af2db235eb83ebbe145cbd329 |
|
MD5 | e6ce713947e099dcc57c2fdc48bfa750 |
|
BLAKE2b-256 | 34bd54ecdc8c2ea56193f32f01243906825cc3ccbd89b1f9b9d46176ff738a15 |
Hashes for pyodps-0.11.2.3-cp36-cp36m-manylinux1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 45c7e1dc204342fbda9e23e1f4289068075b3551b9fd27d3e7c989096e1d2fdf |
|
MD5 | 21e94442c4d4755a5ecac0185e65971c |
|
BLAKE2b-256 | a08ae7a4044f17bae86bc94a5164ba016f511ac7837fe92a6a8f65bd62875ca4 |
Hashes for pyodps-0.11.2.3-cp36-cp36m-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 0de4143487801738079cf705bc0db0594a318d0d28df7df55a0010eded1ea584 |
|
MD5 | 92654e3de599df2cdff050bccec076c5 |
|
BLAKE2b-256 | 0d00a2fb6e52d41fbf268754094cd75d4cf531afd51bf4a75479d90e1226f446 |
Hashes for pyodps-0.11.2.3-cp35-cp35m-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 89e7bd1498d499e95374312582c0c5a23ad850e99e7f34f6f08b137355539116 |
|
MD5 | 762d46cd3638e35a3f202c02ecf62a92 |
|
BLAKE2b-256 | c681ccffe678a354965114b1d299a5b39c1af302189c269f7a89ba387257220f |
Hashes for pyodps-0.11.2.3-cp35-cp35m-win32.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 65d3905d37a1604456469058c2202fdd1f02468fc74dbe0390742f56219c39dc |
|
MD5 | 1435a359329b7336dc38bbb17c51672e |
|
BLAKE2b-256 | 57a4cc3cbbfcbb5cc4bff208409f0c96ca2e158c4f0f1fc49c7f2abe89b32d8d |
Hashes for pyodps-0.11.2.3-cp35-cp35m-manylinux1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | a5018b91c126489a1ab6c28797454bf423d4ca7a083069b1c0bd71103589f7aa |
|
MD5 | 880f743ee62c6e9688ce11257de49425 |
|
BLAKE2b-256 | 62c44a00e6ccaa48b207b30b2b534ef0e365eae130f4c7cf27de8dc669df0ac0 |
Hashes for pyodps-0.11.2.3-cp27-cp27mu-manylinux1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 8c8a4dbe9830073bbf9abfefc8b6cec5b008a6ef63c9362550165a044e7a329e |
|
MD5 | 3cf0998d5ac7449e2a418b1add6ee8fe |
|
BLAKE2b-256 | c8084a6eba1111064bcf04679c156fa460d2687b8464d9fd31632af2c983993c |
Hashes for pyodps-0.11.2.3-cp27-cp27m-manylinux1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 7e317723bd21a39fb6f5aa9cf41627437e43a5bd01f380f95b9dd8f7200a644b |
|
MD5 | b8e8d17106500f0a227c1770b08cf6ab |
|
BLAKE2b-256 | f07e3954c6b4d03e303263e7bcfadfa778938b8abc9d1fae31b6228375d44fea |
Hashes for pyodps-0.11.2.3-cp27-cp27m-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | c3e3f42ad838921039cd4768bc05c407ef63d3cfc94871f8c2ffc8b5c4b97b76 |
|
MD5 | 99e12a9cb3012ced104f680ad7703d2f |
|
BLAKE2b-256 | 26dc49f9bb401f5687f82b5832512273733095638efd81d1ad109cee8fd09a4c |