ODPS Python SDK and data analysis framework
Project description
Elegent way to access ODPS API. Documentation
Installation
The quick way:
pip install pyodps[full]
If you don’t need to use Jupyter, just type
pip install pyodps
The dependencies will be installed automatically.
Or from source code:
$ virtualenv pyodps_env
$ source pyodps_env/bin/activate
$ pip install git+https://github.com/aliyun/aliyun-odps-python-sdk.git
Dependencies
Python (>=2.7), including Python 3+, pypy, Python 3.7 recommended
setuptools (>=3.0)
Run Tests
install pytest
copy conf/test.conf.template to odps/tests/test.conf, and fill it with your account
run pytest odps
Usage
>>> import os
>>> from odps import ODPS
>>> # Make sure environment variable ALIBABA_CLOUD_ACCESS_KEY_ID already set to Access Key ID of user
>>> # while environment variable ALIBABA_CLOUD_ACCESS_KEY_SECRET set to Access Key Secret of user.
>>> # Not recommended to hardcode Access Key ID or Access Key Secret in your code.
>>> o = ODPS(
>>> os.getenv('ALIBABA_CLOUD_ACCESS_KEY_ID'),
>>> os.getenv('ALIBABA_CLOUD_ACCESS_KEY_SECRET'),
>>> project='**your-project**',
>>> endpoint='**your-endpoint**',
>>> )
>>> dual = o.get_table('dual')
>>> dual.name
'dual'
>>> dual.table_schema
odps.Schema {
c_int_a bigint
c_int_b bigint
c_double_a double
c_double_b double
c_string_a string
c_string_b string
c_bool_a boolean
c_bool_b boolean
c_datetime_a datetime
c_datetime_b datetime
}
>>> dual.creation_time
datetime.datetime(2014, 6, 6, 13, 28, 24)
>>> dual.is_virtual_view
False
>>> dual.size
448
>>> dual.table_schema.columns
[<column c_int_a, type bigint>,
<column c_int_b, type bigint>,
<column c_double_a, type double>,
<column c_double_b, type double>,
<column c_string_a, type string>,
<column c_string_b, type string>,
<column c_bool_a, type boolean>,
<column c_bool_b, type boolean>,
<column c_datetime_a, type datetime>,
<column c_datetime_b, type datetime>]
DataFrame API
>>> from odps.df import DataFrame
>>> df = DataFrame(o.get_table('pyodps_iris'))
>>> df.dtypes
odps.Schema {
sepallength float64
sepalwidth float64
petallength float64
petalwidth float64
name string
}
>>> df.head(5)
|==========================================| 1 / 1 (100.00%) 0s
sepallength sepalwidth petallength petalwidth name
0 5.1 3.5 1.4 0.2 Iris-setosa
1 4.9 3.0 1.4 0.2 Iris-setosa
2 4.7 3.2 1.3 0.2 Iris-setosa
3 4.6 3.1 1.5 0.2 Iris-setosa
4 5.0 3.6 1.4 0.2 Iris-setosa
>>> df[df.sepalwidth > 3]['name', 'sepalwidth'].head(5)
|==========================================| 1 / 1 (100.00%) 12s
name sepalwidth
0 Iris-setosa 3.5
1 Iris-setosa 3.2
2 Iris-setosa 3.1
3 Iris-setosa 3.6
4 Iris-setosa 3.9
Command-line and IPython enhancement
In [1]: %load_ext odps In [2]: %enter Out[2]: <odps.inter.Room at 0x10fe0e450> In [3]: %sql select * from pyodps_iris limit 5 |==========================================| 1 / 1 (100.00%) 2s Out[3]: sepallength sepalwidth petallength petalwidth name 0 5.1 3.5 1.4 0.2 Iris-setosa 1 4.9 3.0 1.4 0.2 Iris-setosa 2 4.7 3.2 1.3 0.2 Iris-setosa 3 4.6 3.1 1.5 0.2 Iris-setosa 4 5.0 3.6 1.4 0.2 Iris-setosa
Python UDF Debugging Tool
#file: plus.py
from odps.udf import annotate
@annotate('bigint,bigint->bigint')
class Plus(object):
def evaluate(self, a, b):
return a + b
$ cat plus.input 1,1 3,2 $ pyou plus.Plus < plus.input 2 5
Contributing
For a development install, clone the repository and then install from source:
git clone https://github.com/aliyun/aliyun-odps-python-sdk.git cd pyodps pip install -r requirements.txt -e .
If you need to modify the frontend code, you need to install nodejs/npm. To build and install your frontend code, use
python setup.py build_js python setup.py install_js
License
Licensed under the Apache License 2.0
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distributions
Hashes for pyodps-0.11.5b2-cp311-cp311-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | a7f917ce37a73f32d5124e84208a4a911b8f9ccfe9baf583b73d70fcbf7566e2 |
|
MD5 | 468b9a0c98fd40abe510966fa81db834 |
|
BLAKE2b-256 | 229541c36651bf7802ecc21e792f0c729bacbb0e38aea55a64dd7f259bf572ac |
Hashes for pyodps-0.11.5b2-cp311-cp311-win32.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 6b50630a56610f4938d665364550686d3e4f2f1e8c5076fffcbdbb42ab4a32ff |
|
MD5 | 4ce0752e06210a535b42c1972b73603f |
|
BLAKE2b-256 | 63be6cd9f49ebff4b039c7d2a2b75f3f1ffe10aeacb64ba669021f4e9c96f2ba |
Hashes for pyodps-0.11.5b2-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 814a98f646403a58e06b883f9bca652b415412c816b4cde6f0f4e219c70e4d25 |
|
MD5 | 6b64f6f29d5366da2fa3f7ea8e249f14 |
|
BLAKE2b-256 | 15a81620526d8ced71e3a4cf5356d30f271aceb3232016dae17c792110b32533 |
Hashes for pyodps-0.11.5b2-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | b9389b97e558357fab415d1e3088b2b0b4cc375e89fc1de06e620b25175761e5 |
|
MD5 | 255c3c14a2c0eb10b5c7c61166c130b5 |
|
BLAKE2b-256 | 58cc182e528a041b26ee4d5e3d6154de96676f99a8cd60faaa9425307b1905b4 |
Hashes for pyodps-0.11.5b2-cp311-cp311-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 5e5fc824529a510816c4561a23ad99324134b0869ecb6fa1890eb678a3e1b835 |
|
MD5 | 262cf6d26e9fc11674d171bb84609260 |
|
BLAKE2b-256 | 99b9291f82b1092c6437e3431a7a3242ab0fc9fef04adf218d899434b63b79bc |
Hashes for pyodps-0.11.5b2-cp311-cp311-macosx_10_9_universal2.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 8dbc13db226d8e6ca0e84ded0efe55a844c841a870281f3a945ca6fda4be91ba |
|
MD5 | ba7cb05d7d29796abb34419c6c5d9994 |
|
BLAKE2b-256 | e9bf8c6c9d8098e8e96d650d186d6caba72370fde7bb8ec4785db9a763c7e10f |
Hashes for pyodps-0.11.5b2-cp310-cp310-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 8edbf07067cfc7bd5966d4496d13cf65bb1e34738888e6b2db26b01bbe729423 |
|
MD5 | 3153519644892def8e42ad410e6bba12 |
|
BLAKE2b-256 | 2a83afa7b31a999624bee75972a0edc34372489a5d6bd7279cfbcd6904ab8a40 |
Hashes for pyodps-0.11.5b2-cp310-cp310-win32.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 3114f90b9aa203afad450c0910f49526b1ee1c1fe5c3a74d086ba98ce514ef82 |
|
MD5 | 2175d074b223a50eaf4462f4048ece6f |
|
BLAKE2b-256 | 375efbaf519297acbc0a7ab4d14d8a22fe348d2c47b791d4f7285d842898676d |
Hashes for pyodps-0.11.5b2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | edc8edeb243f78add0c0494e4e06ec1e51bddf6b82eb5193d68155a3f89daf20 |
|
MD5 | 0dfcdb20dabf78801b178475dbf01457 |
|
BLAKE2b-256 | 32f43403b2b0a40e051de493bcf71185ca29d2a312976a4deadf9c3d07341186 |
Hashes for pyodps-0.11.5b2-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 99888f4dd2c3940f9dbcee9d147cdd0de17b7103badb9cfd384e6cae3aadf445 |
|
MD5 | f8f0f428e2e16782bb826524ebaa37aa |
|
BLAKE2b-256 | 768cd1a09464cf0e0031cd1da3f97112820cf6434f397ff112a24835a2627cb9 |
Hashes for pyodps-0.11.5b2-cp310-cp310-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | c73829e0abf5534390ce67bd8c98e82fa0c7f49df0617faed0254f42c4331863 |
|
MD5 | db7caed896dac8c4c099f1ba5560a8b2 |
|
BLAKE2b-256 | 248748d768bbf8385f5947571e06ddccfae7d5de7d8396796f3556c34ca1fce0 |
Hashes for pyodps-0.11.5b2-cp310-cp310-macosx_10_9_universal2.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 9270d2b9e1d33b81f75600de23c467340be24084c62f121d3ccbb5ea5a4b1ce8 |
|
MD5 | cf9e410e02557274d6d042f982b41a5d |
|
BLAKE2b-256 | ed1980e3f659163c08df45b1a3c00101ef2b2de0af56186cbf69988b40d2df89 |
Hashes for pyodps-0.11.5b2-cp39-cp39-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 5e1a7a7ad7f7923ecc2c4b45064523f5750d557796425fd6d54270259721cc05 |
|
MD5 | 3b85821cba5fa0c5eb6c4e74ece961f2 |
|
BLAKE2b-256 | 6518ae3231bc045f40e0849737a866f8ff87aafd564e5aa330944b076fbf33b6 |
Hashes for pyodps-0.11.5b2-cp39-cp39-win32.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | a807558ad7e227d8388b1cec2b9a4bd043e835b0b29747b70282eaa92c1c08ea |
|
MD5 | 18f99dcff75acd6fe884c2a04e1d62e6 |
|
BLAKE2b-256 | 21d7fa0f2f1151447440523ff15a67f78e10a0e0caa0a371c52b8b188867b12c |
Hashes for pyodps-0.11.5b2-cp39-cp39-manylinux2014_aarch64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 6d74f7dda16ddaf326129c49662d4849251b024a131b560cc62481d6f6f1a148 |
|
MD5 | 265fe79a78ab7c739374bee78ab960b2 |
|
BLAKE2b-256 | f718f37e18d8c07206c9d12e447cddc0bba6cbf79e3c705e10986ef2ff90e8fd |
Hashes for pyodps-0.11.5b2-cp39-cp39-manylinux1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | c82844e5b55b0144d19b8195cf1ec30bc9a199571f90340fc630d3baf843329f |
|
MD5 | c9e8d51b76b865da15771b25353a0003 |
|
BLAKE2b-256 | b64868661accd653a190678b4e90fdbfa7a6971586d776213b4bca3ad54e55c0 |
Hashes for pyodps-0.11.5b2-cp39-cp39-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | afe28ca8f47738f52cbfcff3fb675421a711887b9b92fa249d07aa80597c8b53 |
|
MD5 | 2b9ef06b3bfc8bce3f99a19b3b8e9b68 |
|
BLAKE2b-256 | 53382688bf42788b58502f172763981506402a19cb071b9c9271eeee1cbc1267 |
Hashes for pyodps-0.11.5b2-cp39-cp39-macosx_10_9_universal2.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 90db3e1c112a2b983f10854cd1017750652d33df4e987cdd22e57794d674f72d |
|
MD5 | b24015e6bb9431ac64ea0467823525ba |
|
BLAKE2b-256 | 5e59611bc76e08831c7d88778378e3d90f8715e95a29cb2e14e51801df14ce19 |
Hashes for pyodps-0.11.5b2-cp38-cp38-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | b1b45103f222a9d16bd0d079a8cbf1e56b8867dfb180fe68a78b029a5144456a |
|
MD5 | 3c775ad1b400cb61b092c1be621f75c1 |
|
BLAKE2b-256 | acbaf46463eb07af111bf2e74fd070d8f31297e9504f7894671323679ad6b933 |
Hashes for pyodps-0.11.5b2-cp38-cp38-win32.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 83ad3a96f4542b97dec1adbe3cba4b05ddcb177eff17dc30665f641e8412b72b |
|
MD5 | fe29f93f956f123f197282ef1beab757 |
|
BLAKE2b-256 | a227be5d520f488360d12fde9681a027aa0355714fdfbf4771c9276b6c7b5dde |
Hashes for pyodps-0.11.5b2-cp38-cp38-manylinux2014_aarch64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 7ef2f856d3c2bfe918f16c3c7dfc31e83f01c68599548eae1fd0494c718481d8 |
|
MD5 | a252050ba662b144a6c41111937a9227 |
|
BLAKE2b-256 | afe2dd2c8d3634930a8e3ea3a4aaacbdf04c67eb21d3f11874b73e7965ae9380 |
Hashes for pyodps-0.11.5b2-cp38-cp38-manylinux1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 59603452781cc5975f6a363beb25a615e2aa443fc0a5af7b5876ef69c4c667bb |
|
MD5 | dcb371736a2fb21fd5d6532410f74302 |
|
BLAKE2b-256 | 0bcd6f545d3db78e39926fdc46ec17f80e7c08e9cb751b60d946ac1c98f29ce6 |
Hashes for pyodps-0.11.5b2-cp38-cp38-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 73e05995c8ff76bf9a8dcd7a695108aa3af6fe6638aa21f6234b04c867a3c8f9 |
|
MD5 | 24cabf099d045d4158ad59ee6d2747ca |
|
BLAKE2b-256 | 917028578eae7124e54fe4bb39b2bbfc1903bcd355d6e618cf72ee3e107c8f55 |
Hashes for pyodps-0.11.5b2-cp37-cp37m-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 939709e28e64a8e8dede37c442c340c0776281762cf2fd24c1bb57818b886702 |
|
MD5 | e3760d301ef265565226d84becb39c26 |
|
BLAKE2b-256 | 09d07a9840fd5e9f5816108222d33d3799c99c26bbc7a6e212b471c236933bf1 |
Hashes for pyodps-0.11.5b2-cp37-cp37m-win32.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | f604907210d9c6ef51eb64f69fc407a9676ccbc822638db3bf395f3a102556ec |
|
MD5 | 8e11f34a44a223ab5f4d6e5e5bf74c9a |
|
BLAKE2b-256 | 686f85f361bea96d7d94870fb377cfd84013b3b24304e3c37b291b73ebd99970 |
Hashes for pyodps-0.11.5b2-cp37-cp37m-manylinux2014_aarch64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | ed8080f21f6b5f1451365852d3935f2aa4a05e7532f784aa4fb7d1f80364016e |
|
MD5 | e82eaf6cc8148f2d798bbc3d47c40b4c |
|
BLAKE2b-256 | efd8c8763d9580e606d19e617aef7553977d8cd9fa79bcea1efc803260d3e04d |
Hashes for pyodps-0.11.5b2-cp37-cp37m-manylinux1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 61cb4600bddb1f3a1c90a0527365068c97d7b5a5977d71a788a87cfd998e5071 |
|
MD5 | e44f965b1639a96d67523e69d0fb9a40 |
|
BLAKE2b-256 | 1304659e0d7880c62d6267a1cc34320f0c203e5be3472ff60a30e858285cbaec |
Hashes for pyodps-0.11.5b2-cp37-cp37m-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 99cfc952692e2681d96ed9505a3e3a0550500f45529edfd0b0990c0f9e7a73bc |
|
MD5 | cd10223794c707d17b1838f3833a451f |
|
BLAKE2b-256 | f2619afb5ce4b92a50255f66537e3a49a8ae7f5e14819bff13476788fe29c75e |
Hashes for pyodps-0.11.5b2-cp36-cp36m-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | c787e121b2eee9ff0d6954dcdbf4a4223f48c195871cda07ff598006488e645f |
|
MD5 | 4ab60bbdcfe54e7a76ecf9a5fb6e5513 |
|
BLAKE2b-256 | 66c10fe6ea404f8b6ee00e9181932a49e106df25487ded55d6b85fbabcf198b1 |
Hashes for pyodps-0.11.5b2-cp36-cp36m-win32.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | e5d06a25f46d39f7e2a0a969d816d2a4d4a3ff93c19bc7443c3fab24f201bbff |
|
MD5 | 5739b2abe681777d07767e6222105765 |
|
BLAKE2b-256 | 1d61c4291faff907c51a02f800a7c42bec9521afda20e92f3d92166dde27ac72 |
Hashes for pyodps-0.11.5b2-cp36-cp36m-manylinux1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 7a71914bba3c7eec0c4078cf4aa3cd71173d6d7d7faf834843027c4151517a48 |
|
MD5 | 45684ef5d567c6599e603f3cbf1cd8fd |
|
BLAKE2b-256 | d8d3cc974081c682b1117622bb1d09a87506a8ea19ef43bc1e925aba00cd9571 |
Hashes for pyodps-0.11.5b2-cp36-cp36m-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 3d5bbb494694de0963c0094e1790984264b122b89a4d0cab24b71c032d8a3798 |
|
MD5 | f45d975583de8c94d7232e29ebcc5c95 |
|
BLAKE2b-256 | e9d21930b800651e922d3d800262c2bec3e435d1c529b8c6901e93c7d32d737a |
Hashes for pyodps-0.11.5b2-cp35-cp35m-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | c1734f8fb919dc562954141d14c8f090a009a5e6731d0c1c39127d7e9b276fe0 |
|
MD5 | 61de6ec1542cefc951efc37b6df7d681 |
|
BLAKE2b-256 | 5b6d7f362ebb1db12147e888d0c7eb0d8f1b95f5a559aa92cd17cc0ace96de9b |
Hashes for pyodps-0.11.5b2-cp35-cp35m-win32.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | b91fa0051ec5c610c50a9b730bfd5bcebca944f78cc93c9562841fa71d13e329 |
|
MD5 | 8acac8b9194776b2fe4267f13bb34d7b |
|
BLAKE2b-256 | 51124b197a39d358bb952aac2c0d5a2e6be2f7807278925a068a4f9a2071d8d6 |
Hashes for pyodps-0.11.5b2-cp35-cp35m-manylinux1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | b25823b849e97194dc4dccd0d6044e54566e7520067a2574fb4c4010e8820f43 |
|
MD5 | c32efc9c350a21a96221b44abb175a76 |
|
BLAKE2b-256 | c95bf4525b50741037ecc092f951f88ec8099c3a114567970ef01f6b4b6a9254 |
Hashes for pyodps-0.11.5b2-cp27-cp27mu-manylinux1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | d2c4b6cc15e8c706c54bbc021c87f826f5b99ae5ef17a8ad122860475b96751e |
|
MD5 | 380ab8f241a590e7f5d81fbb9664e239 |
|
BLAKE2b-256 | 17203ec9b47f01c8ccda38d09c6b473994f776bfddb57b904dc94e3a39c3f73e |
Hashes for pyodps-0.11.5b2-cp27-cp27m-manylinux1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 02708899d907e9c53d10ffd2aebc412d87b5b7dc09e35d976ab592a459f66f98 |
|
MD5 | 675e2fe63c108dc48a9733b47642faca |
|
BLAKE2b-256 | 3e77061dc660cee1ee9403f4fa8b134540fd2e83d0fc382d90f510fd40995585 |
Hashes for pyodps-0.11.5b2-cp27-cp27m-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | bb0dc1ad0cbe6b51e2a4d27a46943343212620feb99ed8819c55ed207481c318 |
|
MD5 | a9de1d0718e9a05aeba27a4109a8faa9 |
|
BLAKE2b-256 | 347bbfd4ec028cd98d5d4ead203f1816a3737932f749caad62dd8a0c4d8978a6 |