ODPS Python SDK and data analysis framework
Project description
Elegent way to access ODPS API. Documentation
Installation
The quick way:
pip install pyodps[full]
To integrate with Mars:
pip install pyodps#egg=pyodps[mars]
If you don’t need to use Jupyter, just type
pip install pyodps
The dependencies will be installed automatically.
Or from source code:
$ virtualenv pyodps_env
$ source pyodps_env/bin/activate
$ git clone <git clone URL> pyodps
$ cd pyodps
$ python setup.py install
Dependencies
Python (>=2.6), including Python 3+, pypy, Python 2.7 recommended
setuptools (>=3.0)
requests (>=2.4.0)
Run Unittest
copy conf/test.conf.template to odps/tests/test.conf, and fill it with your account
run python -m unittest discover
Usage
>>> from odps import ODPS
>>> o = ODPS('**your-access-id**', '**your-secret-access-key**',
... project='**your-project**', endpoint='**your-end-point**')
>>> dual = o.get_table('dual')
>>> dual.name
'dual'
>>> dual.schema
odps.Schema {
c_int_a bigint
c_int_b bigint
c_double_a double
c_double_b double
c_string_a string
c_string_b string
c_bool_a boolean
c_bool_b boolean
c_datetime_a datetime
c_datetime_b datetime
}
>>> dual.creation_time
datetime.datetime(2014, 6, 6, 13, 28, 24)
>>> dual.is_virtual_view
False
>>> dual.size
448
>>> dual.schema.columns
[<column c_int_a, type bigint>,
<column c_int_b, type bigint>,
<column c_double_a, type double>,
<column c_double_b, type double>,
<column c_string_a, type string>,
<column c_string_b, type string>,
<column c_bool_a, type boolean>,
<column c_bool_b, type boolean>,
<column c_datetime_a, type datetime>,
<column c_datetime_b, type datetime>]
DataFrame API
>>> from odps.df import DataFrame
>>> df = DataFrame(o.get_table('pyodps_iris'))
>>> df.dtypes
odps.Schema {
sepallength float64
sepalwidth float64
petallength float64
petalwidth float64
name string
}
>>> df.head(5)
|==========================================| 1 / 1 (100.00%) 0s
sepallength sepalwidth petallength petalwidth name
0 5.1 3.5 1.4 0.2 Iris-setosa
1 4.9 3.0 1.4 0.2 Iris-setosa
2 4.7 3.2 1.3 0.2 Iris-setosa
3 4.6 3.1 1.5 0.2 Iris-setosa
4 5.0 3.6 1.4 0.2 Iris-setosa
>>> df[df.sepalwidth > 3]['name', 'sepalwidth'].head(5)
|==========================================| 1 / 1 (100.00%) 12s
name sepalwidth
0 Iris-setosa 3.5
1 Iris-setosa 3.2
2 Iris-setosa 3.1
3 Iris-setosa 3.6
4 Iris-setosa 3.9
Command-line and IPython enhancement
In [1]: %load_ext odps In [2]: %enter Out[2]: <odps.inter.Room at 0x10fe0e450> In [3]: %sql select * from pyodps_iris limit 5 |==========================================| 1 / 1 (100.00%) 2s Out[3]: sepallength sepalwidth petallength petalwidth name 0 5.1 3.5 1.4 0.2 Iris-setosa 1 4.9 3.0 1.4 0.2 Iris-setosa 2 4.7 3.2 1.3 0.2 Iris-setosa 3 4.6 3.1 1.5 0.2 Iris-setosa 4 5.0 3.6 1.4 0.2 Iris-setosa
Python UDF Debugging Tool
#file: plus.py
from odps.udf import annotate
@annotate('bigint,bigint->bigint')
class Plus(object):
def evaluate(self, a, b):
return a + b
$ cat plus.input 1,1 3,2 $ pyou plus.Plus < plus.input 2 5
Contributing
For a development install, clone the repository and then install from source:
git clone https://github.com/aliyun/aliyun-odps-python-sdk cd pyodps pip install -r requirements.txt -e .
If you need to modify the frontend code, you need to install nodejs/npm. To build and install your frontend code, use
python setup.py build_js python setup.py install_js
License
Licensed under the Apache License 2.0
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distributions
Hashes for pyodps-0.11.2-cp310-cp310-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 7968b50a15a9410d7798ad01635a9d45c403760af46a39116f94307bb3cbe6e4 |
|
MD5 | ae20a4bde4b54f38390700336e2e27d7 |
|
BLAKE2b-256 | 0327770bdf4484807d7625c330dd1150db8507e907a703f479d6130fc8327f1d |
Hashes for pyodps-0.11.2-cp310-cp310-win32.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | e0dfcaa0c293b4d8c053b30c3ba93a68101d93356c592268f641aaf45ac0aa24 |
|
MD5 | 680aceaac13013bb3ea8965bf79042e9 |
|
BLAKE2b-256 | 0d1e31106a26839e0cee44e9a5772225e8eb73ff38e4ad29a1c7550595878a1e |
Hashes for pyodps-0.11.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 4926e2f650f9daca99bb65b8fd29ee950794694fee26d1476195a4338bb1aade |
|
MD5 | 94f70f5199151d224313d22804d89294 |
|
BLAKE2b-256 | b5636637d16b5d1b2485b5a961eb0d94aceb547a210bbfab9e281d00b780776d |
Hashes for pyodps-0.11.2-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | bd99d6d5c44bee6d5354ddb9075f75623ced450eacd73f3d9de588b874c7b712 |
|
MD5 | e03a8fa300aad6985de0867db41a3814 |
|
BLAKE2b-256 | ab0c00de16b6a85847194c6adad91f635b13bcc3b854c4614be9fb7cef004216 |
Hashes for pyodps-0.11.2-cp310-cp310-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 8dc8f651c21c629136908a72221b7320597828651a6a6b1565803f29200ad2f0 |
|
MD5 | ff1a3e1c2b13a56a1d8247a84cca283b |
|
BLAKE2b-256 | 87cccf5a3d3d4e88021273d61c993fe97d9eb00fc47c5ee3482820c2cdac346d |
Hashes for pyodps-0.11.2-cp310-cp310-macosx_10_9_universal2.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | cdc3ac8cd0c4986826631b2f0c969f0faf482f6999b0ea1b205d6f0d09fb1af5 |
|
MD5 | fb56ced964b2fddab9901b7d87dac6ba |
|
BLAKE2b-256 | 22927b726f8225a60f604c676e747978ca2da631b218c440ef4fdec920a5b8f2 |
Hashes for pyodps-0.11.2-cp39-cp39-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 833c0fe91a5c033cb592cd2e81090528e01444dc8723a8704bef4830b41d3a8d |
|
MD5 | c12e4752eb3f8d6a6c3298639392a957 |
|
BLAKE2b-256 | 47f1593771e97ab942929c49518245393431a5b017bd4cbc920ee6d4f51428f9 |
Hashes for pyodps-0.11.2-cp39-cp39-win32.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 1dc1c8a0b978b61d571ee61940bf209158a78e581e9357e085182e1a382a6349 |
|
MD5 | 037fd0cfc7a8fcd528df29084c5af068 |
|
BLAKE2b-256 | ebeb1ecd041d94ca0958ed582e762a567e846a5bc7fc6fd439984a43db28c8c5 |
Hashes for pyodps-0.11.2-cp39-cp39-manylinux2014_aarch64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | ccd9b3ea5a5d7a7998bb07c9b3857e0eef30059d2cd67e37769eff05713575c3 |
|
MD5 | 77a89e03c988b31a7540ac5eb2e0915b |
|
BLAKE2b-256 | 7088a6f65d1c172ad80199edd4e6f43ec47224c9cf47e7a60837442effb81131 |
Hashes for pyodps-0.11.2-cp39-cp39-manylinux1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 3212e8600cd2d90fe2dd5a7569dc6e4453ff1b47d52f99866a7300ad90ab8531 |
|
MD5 | 2f68e906878a1aec3f21e8451c908e38 |
|
BLAKE2b-256 | 7551a6334c3398e501e78b1deee44f9a268a3781f90e72b5f31b32523bdfd4a2 |
Hashes for pyodps-0.11.2-cp39-cp39-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 9c4d6d775de93b4ee842e5c278e5cce4f456714498735467a333139893b88cba |
|
MD5 | b7a351969d9518e2cdd3202cbd7ecffd |
|
BLAKE2b-256 | 5b599e5a984cc81119de9172516c3bc392fe01b99a7f9775ab8bf54c1b354acc |
Hashes for pyodps-0.11.2-cp39-cp39-macosx_10_9_universal2.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 7575dc322f7fdd8687edd9548b00541da19b695f0c38af9d1b8d80d69a454478 |
|
MD5 | d088f9aa991c6609ca787321aaef0a26 |
|
BLAKE2b-256 | 586a5423bf15cdf74911393f896e2cb8a15c63d7aa13d27b5c100623edd7e0fc |
Hashes for pyodps-0.11.2-cp38-cp38-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 8115880e5dec37c1fd4af4200b20b02dd41977525bd4f8017791f56801943197 |
|
MD5 | 5d717722630f04ced20399e7ca7192df |
|
BLAKE2b-256 | f1a12aec59aeac9ab433f1bbed93fe242e8b52716b2f804b3da3910d442d3b02 |
Hashes for pyodps-0.11.2-cp38-cp38-win32.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | b36c5517aee6d206b1b9273fb06311c5e69797ec35b159b8156a74b6c1503571 |
|
MD5 | 4a624bdff475b7e692056b13185aa0b4 |
|
BLAKE2b-256 | 5e97761593d52cdab5b2ba758d8d268832b464bc47ccbfc92b5f997fc5ecb0e9 |
Hashes for pyodps-0.11.2-cp38-cp38-manylinux2014_aarch64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 47f099309c1a329468330aba03e645116176b9213e2965a89da8cfd5648bfa91 |
|
MD5 | 19931c4e979aa1b729d5067bb7da523d |
|
BLAKE2b-256 | 6804cb3d41d21b68d31470939ee98d8d4ade5413544fd9ca40358faff3c14241 |
Hashes for pyodps-0.11.2-cp38-cp38-manylinux1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 60101b1dd47c07f5684519ee7d7a4f2185c3aa69e0d06a01415b1682a1da8413 |
|
MD5 | 9de3c6fe6b67a2398a68a3c82874676b |
|
BLAKE2b-256 | 2c3e052b239cba0588c7b1f82f8ab618faa3c6d9a4b58fb98baee9e1012c66cc |
Hashes for pyodps-0.11.2-cp38-cp38-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 1ac63ca21da2340c74643a3108a16e3d1e482a3da413504962522df759595338 |
|
MD5 | 1f78ecf6de3c5fdae6edc23177a49b31 |
|
BLAKE2b-256 | 8b182b3f52693a82dc4fd510ea315e4de168762c2e8f62fc1f7331b23bba4736 |
Hashes for pyodps-0.11.2-cp37-cp37m-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 9ce0be0ecc272a39cc1db86058d02742af1b1445c000653e59be865acd5f4eb8 |
|
MD5 | 09043bd3fbd5a60360e1691b8c549975 |
|
BLAKE2b-256 | 91cb5b0e30b27d502744e9d5b4dbe66894d520fdc339558b747062cbc7775384 |
Hashes for pyodps-0.11.2-cp37-cp37m-win32.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 32f239ce0b83eecbd83a8bbcadec8b80ceb8d4fc1afe6e9769ebed3c884c01a7 |
|
MD5 | 3204dc2f6dcf7b125361ec3e332be6b6 |
|
BLAKE2b-256 | 9f5f2cbf20ce014b5ddc3a7bbebe493e916c0310edce152b151faac691e1c4ae |
Hashes for pyodps-0.11.2-cp37-cp37m-manylinux2014_aarch64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 2d65e72f858508c7f1d295e7263b35f243ff0c7a27b5798a600643ecd32f314e |
|
MD5 | fb5cc8c89bb3affa0c86a7a7d146260c |
|
BLAKE2b-256 | 2af2826878eaf3b951f9e3289adc37ce9e8a3c1ca7ecc38b2610cf0aec8717a8 |
Hashes for pyodps-0.11.2-cp37-cp37m-manylinux1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | a657581f06533186748961e28c9e98a2fd56de1bbd21181e5f3fc63b8add5b76 |
|
MD5 | c0e15644862ec6dac1295de0da83c0df |
|
BLAKE2b-256 | 91a0fd372ab6097c29b3f005acb109b8913d4e10a3c5934e1d450036a7e75a4a |
Hashes for pyodps-0.11.2-cp37-cp37m-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | be81c5ac2720888bd3e77df9461712363c853ddd588d2d6416f927a1b43752ec |
|
MD5 | 519a1544f97c5c4de6c6efba148f322e |
|
BLAKE2b-256 | ba9608b985d71bd67552c95c212f214a9f206e3866f116d7b5ed04be54251ff3 |
Hashes for pyodps-0.11.2-cp36-cp36m-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 4b98db471a33d83e4ce12a2b247f963074d295cbcb89a90def69af9fe75d5649 |
|
MD5 | d70d5ad0e54110e3306131f177151fa1 |
|
BLAKE2b-256 | d18eb9c295a348850a165e03505c084b9e74e88146ae6a838c8389ec85594a5b |
Hashes for pyodps-0.11.2-cp36-cp36m-win32.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 630cd43dd6bdc13ad921aeec2c23bc149b685f0bb15cbc7f7d5f0adf996d380d |
|
MD5 | 0b8a75580787941e991dcb3869ea5d8b |
|
BLAKE2b-256 | 6c43046a6b891ffdd08f15ca3835d8353864072c4453125fbdf1e6a0621471cc |
Hashes for pyodps-0.11.2-cp36-cp36m-manylinux1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 0fb136a9cc9788dda99f3e069a7b683f3b9d2cd5b4bab481119eec97e9de623f |
|
MD5 | b5b07238368a6f872488c48711f626f4 |
|
BLAKE2b-256 | be65132579ade65a139eecd18dc51677f7da156eecbc82f985b62781afe0cecf |
Hashes for pyodps-0.11.2-cp36-cp36m-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | e4c648ad36564927c6cd7d8b1bdc59c9ef6252895e4edb2d9b198a9aaff3fd66 |
|
MD5 | 8e5135ea3cef6fca7a070ccc7d300d5f |
|
BLAKE2b-256 | 0b2b24db3e55934ab75390a2de700182490b50b9819bdbfc2223ee4c5a24fb97 |
Hashes for pyodps-0.11.2-cp35-cp35m-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | f25cb5836b758cf15f86cf17d46173f06483046c35b1c0e11811f70ed5215d98 |
|
MD5 | 58ad9c86502b7071f6e3cba4a84128ba |
|
BLAKE2b-256 | ba3a3ee344253e54a425d6fdd98b4fb0c6af8ca9927e76631427f751cde99e41 |
Hashes for pyodps-0.11.2-cp35-cp35m-win32.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 5219a8313cbe28edabd04339a0a2d78ef4c5a172e4112253c073c2f3c1e9c912 |
|
MD5 | b11d53ca11ac29cc4f64eb775fe30e01 |
|
BLAKE2b-256 | 14fceee3f4726257285fc43a3df478cfa262c1b96907239370c119cb684990ff |
Hashes for pyodps-0.11.2-cp35-cp35m-manylinux1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 9bffa859f914a5c50c6d1d48a32bcac6fbe7cfbc86f1cc7b4b74c6e06bc38bb6 |
|
MD5 | f82cfdc4ce3727857f5f01df214524f5 |
|
BLAKE2b-256 | 13349814df9d70f992ee0fbc2ad5768225ff0862c3be5b15d66d27191d8e89a4 |
Hashes for pyodps-0.11.2-cp27-cp27mu-manylinux1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 0346a905bdd32b7dd350d515a95c13a3d8b09b48e58f107382eb71315ffe8361 |
|
MD5 | 661d5e4b052577c46fb2f734c59b6e31 |
|
BLAKE2b-256 | 94b14ec435cc13770462c26447d5cdd0fe04388eb7ebe06c49d1e8b755f42f65 |
Hashes for pyodps-0.11.2-cp27-cp27m-manylinux1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 8a20afac5e71f10c4a9d3192aae3b1fd4b0d12415b3368c240d9d9fd229c354b |
|
MD5 | f1a907d7da2b2de804a93300288be976 |
|
BLAKE2b-256 | f9382070a32193df864502e9f26d54ebbb4dbd0c7f85fc6185bfc69063671788 |
Hashes for pyodps-0.11.2-cp27-cp27m-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | d89ca0aecfd9797780b66ff5d220b2c08231c709f955a982027dba2e4ccfdf63 |
|
MD5 | 0303415ad2a87734cc1031a81eeb87fb |
|
BLAKE2b-256 | f7870369ccbc5af7726fd2303a42798c77b2bcada901203de3b6bde2625aebf9 |