ODPS Python SDK and data analysis framework
Project description
Elegent way to access ODPS API. Documentation
Installation
The quick way:
pip install pyodps[full]
To integrate with Mars:
pip install pyodps#egg=pyodps[mars]
If you don’t need to use Jupyter, just type
pip install pyodps
The dependencies will be installed automatically.
Or from source code:
$ virtualenv pyodps_env
$ source pyodps_env/bin/activate
$ git clone <git clone URL> pyodps
$ cd pyodps
$ python setup.py install
Dependencies
Python (>=2.6), including Python 3+, pypy, Python 2.7 recommended
setuptools (>=3.0)
requests (>=2.4.0)
Run Unittest
copy conf/test.conf.template to odps/tests/test.conf, and fill it with your account
run python -m unittest discover
Usage
>>> from odps import ODPS
>>> o = ODPS('**your-access-id**', '**your-secret-access-key**',
... project='**your-project**', endpoint='**your-end-point**')
>>> dual = o.get_table('dual')
>>> dual.name
'dual'
>>> dual.table_schema
odps.Schema {
c_int_a bigint
c_int_b bigint
c_double_a double
c_double_b double
c_string_a string
c_string_b string
c_bool_a boolean
c_bool_b boolean
c_datetime_a datetime
c_datetime_b datetime
}
>>> dual.creation_time
datetime.datetime(2014, 6, 6, 13, 28, 24)
>>> dual.is_virtual_view
False
>>> dual.size
448
>>> dual.table_schema.columns
[<column c_int_a, type bigint>,
<column c_int_b, type bigint>,
<column c_double_a, type double>,
<column c_double_b, type double>,
<column c_string_a, type string>,
<column c_string_b, type string>,
<column c_bool_a, type boolean>,
<column c_bool_b, type boolean>,
<column c_datetime_a, type datetime>,
<column c_datetime_b, type datetime>]
DataFrame API
>>> from odps.df import DataFrame
>>> df = DataFrame(o.get_table('pyodps_iris'))
>>> df.dtypes
odps.Schema {
sepallength float64
sepalwidth float64
petallength float64
petalwidth float64
name string
}
>>> df.head(5)
|==========================================| 1 / 1 (100.00%) 0s
sepallength sepalwidth petallength petalwidth name
0 5.1 3.5 1.4 0.2 Iris-setosa
1 4.9 3.0 1.4 0.2 Iris-setosa
2 4.7 3.2 1.3 0.2 Iris-setosa
3 4.6 3.1 1.5 0.2 Iris-setosa
4 5.0 3.6 1.4 0.2 Iris-setosa
>>> df[df.sepalwidth > 3]['name', 'sepalwidth'].head(5)
|==========================================| 1 / 1 (100.00%) 12s
name sepalwidth
0 Iris-setosa 3.5
1 Iris-setosa 3.2
2 Iris-setosa 3.1
3 Iris-setosa 3.6
4 Iris-setosa 3.9
Command-line and IPython enhancement
In [1]: %load_ext odps In [2]: %enter Out[2]: <odps.inter.Room at 0x10fe0e450> In [3]: %sql select * from pyodps_iris limit 5 |==========================================| 1 / 1 (100.00%) 2s Out[3]: sepallength sepalwidth petallength petalwidth name 0 5.1 3.5 1.4 0.2 Iris-setosa 1 4.9 3.0 1.4 0.2 Iris-setosa 2 4.7 3.2 1.3 0.2 Iris-setosa 3 4.6 3.1 1.5 0.2 Iris-setosa 4 5.0 3.6 1.4 0.2 Iris-setosa
Python UDF Debugging Tool
#file: plus.py
from odps.udf import annotate
@annotate('bigint,bigint->bigint')
class Plus(object):
def evaluate(self, a, b):
return a + b
$ cat plus.input 1,1 3,2 $ pyou plus.Plus < plus.input 2 5
Contributing
For a development install, clone the repository and then install from source:
git clone https://github.com/aliyun/aliyun-odps-python-sdk cd pyodps pip install -r requirements.txt -e .
If you need to modify the frontend code, you need to install nodejs/npm. To build and install your frontend code, use
python setup.py build_js python setup.py install_js
License
Licensed under the Apache License 2.0
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distributions
Hashes for pyodps-0.11.3b1-cp310-cp310-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 1dd12609494d3f4573bd7656245bda0c08d4059036ce6d29294407b5d39784a1 |
|
MD5 | 7a1c1aeae6e6cc0cb68100ef5e281243 |
|
BLAKE2b-256 | f2eb33ded2989b0cabbdc189e0fc254bbd6fd42c3b1e039a5e8c63ab7e6237c1 |
Hashes for pyodps-0.11.3b1-cp310-cp310-win32.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 0184e0f401da09e3128afebc6a930d1e4a04edf50d0ede9b6b37a3c51e660a70 |
|
MD5 | 8a1f02138505a8a1fe926a4aed406f57 |
|
BLAKE2b-256 | 06650f6fc0ce154be917432099441e9c5d516320b9813cc2c9aa59b241a98fb0 |
Hashes for pyodps-0.11.3b1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 2ad1d5f3ef804d2063d5761bb2e279dc9b8ed026c4bfd41c4fe0a689a649dfcf |
|
MD5 | 426db5d9db0a97790f0fca1b94aba74b |
|
BLAKE2b-256 | b4ca86a57c14a8f99fc52270bf8d3f87d88a06407576aa71cd02e93567138f9b |
Hashes for pyodps-0.11.3b1-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | a166fa036d41d5423512f72ae5d5b78a0db452d6b78c8b714056c3ece56a2cbe |
|
MD5 | 2a77c85acd1b0dc6d9511bce25cf1685 |
|
BLAKE2b-256 | 64703e7ad9c1767dab21690267bbd9f76399522f233292145aa0e77b62861bbe |
Hashes for pyodps-0.11.3b1-cp310-cp310-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | d30e4f57a3857928039221932e47db7227ceed8c12279e9acaf358ffe33fc564 |
|
MD5 | 564fb5579ecc6cd4ea3271cc7df87bc3 |
|
BLAKE2b-256 | c3cd56d78f7305010882aa5ef2eea7ca208c56c2e1b8ceefefefdfec0f18fd72 |
Hashes for pyodps-0.11.3b1-cp310-cp310-macosx_10_9_universal2.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 92c28d57f97502945379d9afb86d56af704b675dc5090536aeb53f2ae05a16c3 |
|
MD5 | a4a820d192402b1800cda97530a5485d |
|
BLAKE2b-256 | f682bd9167fb9a7ac4f5d2a6c5b10271bf7e3503f53e166fb536f285765d645a |
Hashes for pyodps-0.11.3b1-cp39-cp39-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | b3e48d5ad389f53d805a6d4d6aa72c6886a499720a5fd30dcf32c26a3ab1ed7c |
|
MD5 | fbc7048dd80a8af04722b43156bde3e2 |
|
BLAKE2b-256 | fe1c70555ac7b790f7136331f51d8faeb23b9f57189250a87bc5d058aa4e9de9 |
Hashes for pyodps-0.11.3b1-cp39-cp39-win32.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 78c69ee20e35fa7fefa7085ebf70ce0eb7fa9efbbc30b6aa3b9764d57e3d655a |
|
MD5 | edc443a31a9aa3fe190f7013c310f985 |
|
BLAKE2b-256 | 3b094e76df92cd3e7d25c2c6d84f120f64d94619624b25a7ebf9f3ce84007ee7 |
Hashes for pyodps-0.11.3b1-cp39-cp39-manylinux2014_aarch64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | b4926706b90fbcb2e7495aac5c363f155219239090185298818cd5583bb58005 |
|
MD5 | ff7a43447ebde92fdaeed00913690997 |
|
BLAKE2b-256 | 995d29652fe21e5bb54cdffaa51e4d4c9c263c4e1e82b679d5e355d906357c21 |
Hashes for pyodps-0.11.3b1-cp39-cp39-manylinux1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 95897e04cb3be0bd3eeb95ad2d66c4e13e5ff98a3cf5d24de9738bc21e29e67d |
|
MD5 | e798a56e600a4638a9d660b41c9ec222 |
|
BLAKE2b-256 | c40cac604ede481f515e6b38df440184210096f4cf9cba0c9224df2c8be3f490 |
Hashes for pyodps-0.11.3b1-cp39-cp39-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | fba0feecd9115fe4bb8bb82a129c617ac67569d988fab4a13a277de5648b84c6 |
|
MD5 | ee4a9dd0e64cfa867e4489125049f96f |
|
BLAKE2b-256 | 4570802797b1ded1d0aeca6c288ff68ab508f4f674d8b49bfd19ed4613849078 |
Hashes for pyodps-0.11.3b1-cp39-cp39-macosx_10_9_universal2.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | ee29d305a8ddff415ea308bc37a445da2132ce667aecadef0a343ac42d6993b0 |
|
MD5 | 7568fee00a18d6d18c34b34d6f73dd88 |
|
BLAKE2b-256 | 405cd1d39847aa4327b2d6c43dda3022a8c3ecef6cec135547a527f31c852193 |
Hashes for pyodps-0.11.3b1-cp38-cp38-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | a4bb832692944813e46ffb29325f17b5c07d542f78b5a3f073acd08a293a0a26 |
|
MD5 | b893b60f9550fd74fe45f5fbf080e3da |
|
BLAKE2b-256 | 86b6748fd5b0a1c78ae52f6085d9f69c212e0bfa85b53a3ba69c56dcce1afbf6 |
Hashes for pyodps-0.11.3b1-cp38-cp38-win32.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 6335fa0d0751d21babcdee3c156cbbf7bbd746335faff6e0721521bdf4e8c837 |
|
MD5 | f2baf117914dc79d9da59cc0b8927200 |
|
BLAKE2b-256 | 8e56dc09b64ad8be2bf59a48da428e3c64ffe9d71af98ddc23c7915f6eaa1e18 |
Hashes for pyodps-0.11.3b1-cp38-cp38-manylinux2014_aarch64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 9d45c67d2e3d3b98e3da5d157a0776004b72e23e223dfd4da098f08ea76400d7 |
|
MD5 | 93fa625bfe366d1871b9e00e381cb8af |
|
BLAKE2b-256 | d427e4df435af5d1eddd1d936ddd203587b20f00b236b4ac6920fb2889c13b7e |
Hashes for pyodps-0.11.3b1-cp38-cp38-manylinux1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 57191687115749318e67e2b28b444ce6b5c0a4b1a2bb2ce41ea2b6c83bcd430f |
|
MD5 | 279d66637b3f1b97df2f8b35887a18db |
|
BLAKE2b-256 | 7411874f1ac4d518838c1c8e339f07fc9f6ffaaf5474b8af4f03834c43cb34f2 |
Hashes for pyodps-0.11.3b1-cp38-cp38-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 5ee956474dfa2fd8cf97f68f2177754bc52d096da55f6b9a06de70a3b36b5dc2 |
|
MD5 | 97e9b4bef2d21824883266393511b0c6 |
|
BLAKE2b-256 | 0fd1bfd937df6b79de2df6a2d6a42cf03e3d491e6c05bfeb187eb90cc47f32ca |
Hashes for pyodps-0.11.3b1-cp37-cp37m-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 9a67c29c6bead78617ec1efb20a044fa41b6dd43b17e786df4536b1f1fcb11f8 |
|
MD5 | 3c72c5a4a93e6b2703cac21f57e46d9b |
|
BLAKE2b-256 | 12dc3c2effa7ee1677113a8cdf9ff8a0f121f141dbba1b4563bb46da12f1e5bb |
Hashes for pyodps-0.11.3b1-cp37-cp37m-win32.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 9819380012c1d579381d904cf8ba63f54dccc1fe33c0cd3b65393404529348f6 |
|
MD5 | 78b1cc91c7cffc29599ae5fc5a221417 |
|
BLAKE2b-256 | a8cde4cbf3ef19b4253427d266213af8bfd3c1de4c0be3b83e43d550d1683ca8 |
Hashes for pyodps-0.11.3b1-cp37-cp37m-manylinux2014_aarch64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 546a1276b95e159aed942c7774c5ea7e77171663590a40d5b60ec9644884743e |
|
MD5 | d8acc18770bcc995ad9b35e567148ce1 |
|
BLAKE2b-256 | 217156aeacc4ca28654a61d20d7150bc0bddd7f14b1d39e1401843ba91cc8e86 |
Hashes for pyodps-0.11.3b1-cp37-cp37m-manylinux1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 7891db71196bb39830330eaeb53e844e482ae2da42572809696e97a125338eb2 |
|
MD5 | 10039c00c0512e3ec1016b6a85bd5b63 |
|
BLAKE2b-256 | 864fcf93355fd030f3694470193bfd95c57f4bf1c339aba86d7d52b206643634 |
Hashes for pyodps-0.11.3b1-cp37-cp37m-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 53323d575e1e641e90731e9bc8f0db2e4d3ff2651f63c8d93ab4b043a77ccd8d |
|
MD5 | 7982ad0abcc15f66641fca9f740240f9 |
|
BLAKE2b-256 | 953262286095ef15043525587f91620c52d5caed264568a6795d2d441b5286c7 |
Hashes for pyodps-0.11.3b1-cp36-cp36m-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | fe4f6a5e05bb94dfee838d0e0f30cd11acb2b2bc10cb2f42579ce3d61ff33ed2 |
|
MD5 | 2f5504839e852c82d15a4f9c30bba314 |
|
BLAKE2b-256 | c05dea4a519f739ce88865b61874e74cd3ee164ea15db8715ae0f75018fbd085 |
Hashes for pyodps-0.11.3b1-cp36-cp36m-win32.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 0afdd93ddd9a535ffa7091ebbac4df7bf59b2df4051568629936956b2fd8579d |
|
MD5 | 1ad1108df2d3337956647d0126231bd9 |
|
BLAKE2b-256 | ab2f6f588ca872a58965795e4b8e81af681f5fff37850342438f9e4780d6d7e9 |
Hashes for pyodps-0.11.3b1-cp36-cp36m-manylinux1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 4aaf2be5320b5257cca5ea16e91fdfedf3faef1d72cd40bfb2796b2eaa6f7e7f |
|
MD5 | 2068e07084cb0484274635a49ae3f01c |
|
BLAKE2b-256 | 3c6e3cacf93a9f209584762a83da11a45c196e6275e3f05dc5ddf6a862114abb |
Hashes for pyodps-0.11.3b1-cp36-cp36m-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 4e0f26fac80d4cb59b7ebe779dc18e998c763dfeebc1a206549d16d6e370b069 |
|
MD5 | 31578db9705e6c45f26f39010bc8abab |
|
BLAKE2b-256 | 1cc6c690d489f6e24335583c99a6d531ca2a5fa4da9ba5626aa4f6b0d4fda3ac |
Hashes for pyodps-0.11.3b1-cp35-cp35m-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 5b534364ba6effeab7b5761d9e2fdcdb538c7c61881e86ead2e7859e2a21b920 |
|
MD5 | bb5d3bec5f0db918cbc144017dcd71f8 |
|
BLAKE2b-256 | c7500239a325413c1a7d4a2d91b845a701ba271f31eb509f8fdaef6489033f12 |
Hashes for pyodps-0.11.3b1-cp35-cp35m-win32.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 7d460359bb523cb01091e7ba7d1132522e7fd5d3144512614aca4b275a158a33 |
|
MD5 | aaec512bfef0491c3111113eddcf594e |
|
BLAKE2b-256 | 669ca038b9eddae94ebe75836d297c41cbcb936e1258c4bea60a9953503d582a |
Hashes for pyodps-0.11.3b1-cp35-cp35m-manylinux1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | f8b7e23e19f9e807e5afee914db05ce9f232694601f11dce7be0c7d8a3ba188c |
|
MD5 | 52a9747345a561245c1620fa8dfde358 |
|
BLAKE2b-256 | a513a8c3321210875dc2305dd8e3a826b7d23a1a792664d1d2d6318e49fd2159 |
Hashes for pyodps-0.11.3b1-cp27-cp27mu-manylinux1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 7c46a5755117a78433e43b8a63f4e741d0c27470d335c3062fd933194af7e657 |
|
MD5 | bf726fe0610c19043cb4515c40211b84 |
|
BLAKE2b-256 | ff5f64292110be6b61b4e3ac041ce2410caeec7773434e6576f245784a47648b |
Hashes for pyodps-0.11.3b1-cp27-cp27m-manylinux1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 8143342a8b739b56699c3077e2f48d185ce371baab7853b15b7cbe83d13b5fc0 |
|
MD5 | 42d8d1da14cd453722ecb8eef874c64b |
|
BLAKE2b-256 | d705432c43c4fe3222e0437bf1b8d41d74995de128a9ea0e1d12d1dfa6291e80 |
Hashes for pyodps-0.11.3b1-cp27-cp27m-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 5663d1c33086e3550882145a249c2c136b53af485a461ea0e985894389ae4413 |
|
MD5 | 836cda237e28453a34139aae6a7bd4b6 |
|
BLAKE2b-256 | 4c9dcbfe1dfa79702e16db0e6d5d7d4fd5decb00536de84cc704ab7f0938bb5a |