ODPS Python SDK and data analysis framework
Project description
Elegent way to access ODPS API. Documentation
Installation
The quick way:
pip install pyodps[full]
If you don’t need to use Jupyter, just type
pip install pyodps
The dependencies will be installed automatically.
Or from source code:
$ virtualenv pyodps_env
$ source pyodps_env/bin/activate
$ pip install git+https://github.com/aliyun/aliyun-odps-python-sdk.git
Dependencies
Python (>=2.7), including Python 3+, pypy, Python 3.7 recommended
setuptools (>=3.0)
Run Tests
install pytest
copy conf/test.conf.template to odps/tests/test.conf, and fill it with your account
run pytest odps
Usage
>>> import os
>>> from odps import ODPS
>>> # Make sure environment variable ALIBABA_CLOUD_ACCESS_KEY_ID already set to Access Key ID of user
>>> # while environment variable ALIBABA_CLOUD_ACCESS_KEY_SECRET set to Access Key Secret of user.
>>> # Not recommended to hardcode Access Key ID or Access Key Secret in your code.
>>> o = ODPS(
>>> os.getenv('ALIBABA_CLOUD_ACCESS_KEY_ID'),
>>> os.getenv('ALIBABA_CLOUD_ACCESS_KEY_SECRET'),
>>> project='**your-project**',
>>> endpoint='**your-endpoint**',
>>> )
>>> dual = o.get_table('dual')
>>> dual.name
'dual'
>>> dual.table_schema
odps.Schema {
c_int_a bigint
c_int_b bigint
c_double_a double
c_double_b double
c_string_a string
c_string_b string
c_bool_a boolean
c_bool_b boolean
c_datetime_a datetime
c_datetime_b datetime
}
>>> dual.creation_time
datetime.datetime(2014, 6, 6, 13, 28, 24)
>>> dual.is_virtual_view
False
>>> dual.size
448
>>> dual.table_schema.columns
[<column c_int_a, type bigint>,
<column c_int_b, type bigint>,
<column c_double_a, type double>,
<column c_double_b, type double>,
<column c_string_a, type string>,
<column c_string_b, type string>,
<column c_bool_a, type boolean>,
<column c_bool_b, type boolean>,
<column c_datetime_a, type datetime>,
<column c_datetime_b, type datetime>]
DataFrame API
>>> from odps.df import DataFrame
>>> df = DataFrame(o.get_table('pyodps_iris'))
>>> df.dtypes
odps.Schema {
sepallength float64
sepalwidth float64
petallength float64
petalwidth float64
name string
}
>>> df.head(5)
|==========================================| 1 / 1 (100.00%) 0s
sepallength sepalwidth petallength petalwidth name
0 5.1 3.5 1.4 0.2 Iris-setosa
1 4.9 3.0 1.4 0.2 Iris-setosa
2 4.7 3.2 1.3 0.2 Iris-setosa
3 4.6 3.1 1.5 0.2 Iris-setosa
4 5.0 3.6 1.4 0.2 Iris-setosa
>>> df[df.sepalwidth > 3]['name', 'sepalwidth'].head(5)
|==========================================| 1 / 1 (100.00%) 12s
name sepalwidth
0 Iris-setosa 3.5
1 Iris-setosa 3.2
2 Iris-setosa 3.1
3 Iris-setosa 3.6
4 Iris-setosa 3.9
Command-line and IPython enhancement
In [1]: %load_ext odps In [2]: %enter Out[2]: <odps.inter.Room at 0x10fe0e450> In [3]: %sql select * from pyodps_iris limit 5 |==========================================| 1 / 1 (100.00%) 2s Out[3]: sepallength sepalwidth petallength petalwidth name 0 5.1 3.5 1.4 0.2 Iris-setosa 1 4.9 3.0 1.4 0.2 Iris-setosa 2 4.7 3.2 1.3 0.2 Iris-setosa 3 4.6 3.1 1.5 0.2 Iris-setosa 4 5.0 3.6 1.4 0.2 Iris-setosa
Python UDF Debugging Tool
#file: plus.py
from odps.udf import annotate
@annotate('bigint,bigint->bigint')
class Plus(object):
def evaluate(self, a, b):
return a + b
$ cat plus.input 1,1 3,2 $ pyou plus.Plus < plus.input 2 5
Contributing
For a development install, clone the repository and then install from source:
git clone https://github.com/aliyun/aliyun-odps-python-sdk.git cd pyodps pip install -r requirements.txt -e .
If you need to modify the frontend code, you need to install nodejs/npm. To build and install your frontend code, use
python setup.py build_js python setup.py install_js
License
Licensed under the Apache License 2.0
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distributions
Hashes for pyodps-0.11.5-cp311-cp311-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | ff86c0128f2137608d6d5b64ab8561154690cdbe62248a2c22ed2a6736769a57 |
|
MD5 | cc732e0028c2ed5fb9db2aa10d260ec7 |
|
BLAKE2b-256 | af2fe8f3dd2d85412af041b9a0cad6f388a63cb35a3d38b762140abe0bf01405 |
Hashes for pyodps-0.11.5-cp311-cp311-win32.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 64b643ebbacda61ab8a2d59e54308432f770c564ecb1cd18420c09594357eeaf |
|
MD5 | 02bd9fc84774ad6b6e09c3a6e5d1aecf |
|
BLAKE2b-256 | 99e08585aa3bdbf9a0880b7923352f7659f26cf9efb9010f82cad1720263c8f8 |
Hashes for pyodps-0.11.5-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | d6ebf66337e0e5643d04a7927378868a0353d9876055d4a9bbb43d1cdec8b857 |
|
MD5 | 83cc60d558dcb3eae3660490ed362407 |
|
BLAKE2b-256 | 7d3463bad3ff40b9c97c67911d57eb11cd6ee70925a3b7f60235a726ba8313bf |
Hashes for pyodps-0.11.5-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 73b30e91def6aa06f9919009c0a02b32c1da22c778496b5a1d94da5c0b8006de |
|
MD5 | 7ee4648ee7abd527302caba9832df47c |
|
BLAKE2b-256 | 5b71cc971f9a87bfad7b1471e0f30c09823c26dbb5145c09acd12b7098d02312 |
Hashes for pyodps-0.11.5-cp311-cp311-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 2e8f82ce8f7d4abdcdf8ef0618486fc22a46ca7adf82ab3a1bdec0dc88e88f56 |
|
MD5 | 52fb776af6c224adfdb739aaa2a0fa44 |
|
BLAKE2b-256 | 72591d842b7f04827718124851a178fa0f691acab4188655de79100b5e99b779 |
Hashes for pyodps-0.11.5-cp311-cp311-macosx_10_9_universal2.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | feaaac63b91f4d266f86a0ef7eb4c563b2a29c8017fe4541e2fcb22533b5758f |
|
MD5 | 567d599fe25c11ef8c3580cbbfafcf65 |
|
BLAKE2b-256 | 62d65493ec5fdeb92614e36f0803dff77adea2a0963faa8eaa45ccb1f3f41917 |
Hashes for pyodps-0.11.5-cp310-cp310-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | edcdb1ab38463dbe839c21dd1ec7805bc118c0e3e66051f157db7411e81169bf |
|
MD5 | 4f8766e2be2bda121d25fb3934920465 |
|
BLAKE2b-256 | bdd324a5afbde24f7c7141fc81e7690ae0dcbc54526b15e2e6af0741d393b2bc |
Hashes for pyodps-0.11.5-cp310-cp310-win32.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 895db29ade6ef2ca1c99d2682d8fd519b09cb12bbfc52e1a2fb1f404c262693a |
|
MD5 | dc79d8bcbbe1fb7e8b3dea8490acd013 |
|
BLAKE2b-256 | d668f29c27b7ede9fdd9faf540e974df7b4c72b452cd76529c3e5fdbb4d647b3 |
Hashes for pyodps-0.11.5-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 129e24ad924ec62b19e02895cf30b1380616b9341040393ee343fd0b66b5dfc4 |
|
MD5 | da06854cc5d1eae92e34593bce076d43 |
|
BLAKE2b-256 | f321c00389c2b26f92283170794ce81be1d2deb4404b488af8d1f154d6a024c5 |
Hashes for pyodps-0.11.5-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 9c60d3ced0d36f54f270a56600fcf286c23f7ed7da939e3bc1e1f743f2175fb2 |
|
MD5 | be19dde77b370d0747d8025867fb1616 |
|
BLAKE2b-256 | 2efedc9a12751857123af5b6a6051dd64e6de16f90e383851e9cc65c75e252e6 |
Hashes for pyodps-0.11.5-cp310-cp310-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 833d0c901b34525577b4fcbd84065317fd0fa5a59c96e112fac28bfa233f88a7 |
|
MD5 | 83788a19a0115e74800c76ede75a657c |
|
BLAKE2b-256 | 6fa90e0de219349010da64ffeb10ba70791135231aa25f2c8a358c441e2b2686 |
Hashes for pyodps-0.11.5-cp310-cp310-macosx_10_9_universal2.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | a6a5b5ad7d3545869f3f1fb85717bf67f10bdec79d9e79b0be530842c6db1ebf |
|
MD5 | 9bdb6de3b546df77a7541e0c1d688a1a |
|
BLAKE2b-256 | 4bce189192cf1ff6f4a92d12311ac81713ba876d6e5341d942e30fda6e11b4c1 |
Hashes for pyodps-0.11.5-cp39-cp39-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 1889f50fd657bf928073dfb73e80e124c008d73ddde5ffbc119b9957ab08b0e6 |
|
MD5 | a8a1430bb59422c7bc697ad18be7e64f |
|
BLAKE2b-256 | 9ce1f6d1727dd53e842ad835989f3b9d431f264ec852a11b29b9cf6fed2fe935 |
Hashes for pyodps-0.11.5-cp39-cp39-win32.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 82ab834cd6bb661ca36f0e46d34ba2081998b9c15d89baabcc0465903b511216 |
|
MD5 | 39f1a85f72ae479b593e2b1314089126 |
|
BLAKE2b-256 | af31c6bed0433673f3ddddca9f09342ed5f1e35d77b637eb8eeab4f25af69dab |
Hashes for pyodps-0.11.5-cp39-cp39-manylinux2014_aarch64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | ae230b34b01ba13afa1499944b89c0bee17ce4366dde8e680041e4c2e02590e1 |
|
MD5 | 18b0164314cd2b3165f5179578c0abd2 |
|
BLAKE2b-256 | f40079ee27a09cc95bcb0aab939c237714868d9fbf7766c5ff0b1bf490de6d39 |
Hashes for pyodps-0.11.5-cp39-cp39-manylinux1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | a65f141c70bd5be84b7b7b9dee68ea4135afc74c61f8b32fd347eb5963e5a604 |
|
MD5 | 1d79509987a0c911061d08e6c29e3bee |
|
BLAKE2b-256 | f7918f87322b14d205c37b31c5c0eb4bda1728d3cb2b92ff38a44102397bbf0e |
Hashes for pyodps-0.11.5-cp39-cp39-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 723ce79bcebb06241b285b7619be4ad408b0526c0c23f6bf85fbd84c1003b61f |
|
MD5 | 401c1f66b489ba60db8d88d0b5752c48 |
|
BLAKE2b-256 | 2648c815709403b0f00bec5d7bbe0593be4d26bf991016e533768ca3b1fd19fe |
Hashes for pyodps-0.11.5-cp39-cp39-macosx_10_9_universal2.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 62978f10f2a153844db9f07dc66eb51aa950ec8e8160bf84a46c066141bf1c98 |
|
MD5 | 93c3013e06754a4bea48adb2731eb44e |
|
BLAKE2b-256 | b423b7225cf554557452c62f8ee4cf5f7983707942227f6309b24494ba6a6b1f |
Hashes for pyodps-0.11.5-cp38-cp38-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 8283e194618a1c1deea02887f0ffcf133cd2e66edf02ebb03237a96d69898bd9 |
|
MD5 | 548ef5e7025ba45da5180b4178a3c5a0 |
|
BLAKE2b-256 | e2a439e1a8e18a09766861e7b6bed28912dac8c6670aa839eec90515f826675f |
Hashes for pyodps-0.11.5-cp38-cp38-win32.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | c69bdb32362e490c8f49fdad180c334c841d7f6ee7dc24e6b10b8f1317af52c8 |
|
MD5 | da07e71e1bf0cc49a16d774bbb8b5e3e |
|
BLAKE2b-256 | 603ffd8ddf08ae97046dcadca4e7d9a8fb60071cddc673a41611488cc9c1cf09 |
Hashes for pyodps-0.11.5-cp38-cp38-manylinux2014_aarch64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 15104d6d041c3f348600d05464668415a541e0c62769947e56e21ba0fbc3d9be |
|
MD5 | 57c341003170eff501987642f542fe8c |
|
BLAKE2b-256 | c7f5e1db56303e3e06fcc4784485e163f417570f46c04d48e19477fa9029b051 |
Hashes for pyodps-0.11.5-cp38-cp38-manylinux1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 16ed4c28dd2c4186a1f6a8f595401d735445f96adb2163c231a23f2942819660 |
|
MD5 | 71e2d30ffb288b76b16f424b5c49d1f4 |
|
BLAKE2b-256 | 954f585a55f63b166ac1cb83d621ac564311e448897f0bfaac3bc9d5547f53c9 |
Hashes for pyodps-0.11.5-cp38-cp38-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | dd18b4c2d7bb3818cfac9ee8619be4cafd972474e0ec06295df3acfc3266ba5a |
|
MD5 | e78848ae70976bdb9edb96e622579140 |
|
BLAKE2b-256 | 95bfc8aae406e4025d403821f5d3437275757a1735d14e8a28a1efecd3770a1e |
Hashes for pyodps-0.11.5-cp37-cp37m-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 525d94df91465721cf43a8f8ba000ba438df0be8bff2200d13bf7252a098d1b5 |
|
MD5 | 90a813959de14e363c162df2d04d6907 |
|
BLAKE2b-256 | e5d58ab53ab0710b1b422ae0d15fa2c975b674d3ed1a07bed8f6c109c49749dd |
Hashes for pyodps-0.11.5-cp37-cp37m-win32.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 781ba7ec0819adecc5133cd5d38959f29ecff0be0a151aea737010b198de0d74 |
|
MD5 | 357f82bf992ed30d6802a1e0f8796b90 |
|
BLAKE2b-256 | dc0f97cfabb448b1a6242984904a3b6fe590dd524565170d7c7c402c0accb967 |
Hashes for pyodps-0.11.5-cp37-cp37m-manylinux2014_aarch64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 912e285b3d9df396e030a1ce79a0fe4ff8923022c64734112e7a2dda7717b65f |
|
MD5 | fe010f9fa2961fbf9540d08f5d876f0b |
|
BLAKE2b-256 | 6a6be86d36a18fd6e0cc06a0c34d70a98d90c6a125ea4dfc919a472ca39cf483 |
Hashes for pyodps-0.11.5-cp37-cp37m-manylinux1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 9b5a12a20bbfa129eee4dbb9414bc4ce42e93bb7822231649051385ebf8daa5e |
|
MD5 | 5e80eb50155c1dc8d91e683f944b2131 |
|
BLAKE2b-256 | 04a26c2896c1aead34661833577228cc7e8b89e9d99e5d5b3f31e44c372b835c |
Hashes for pyodps-0.11.5-cp37-cp37m-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 6ad569fccfdb64d63998af46162ca06e4e508ae0ef130417c2eba2c0281a5094 |
|
MD5 | 7cccc475a8d46761c85fdc8ddeef54d3 |
|
BLAKE2b-256 | 66a3fb6e6d195f28618b768bbdac037955dd776fafc2db8cdeefd73a427ffa42 |
Hashes for pyodps-0.11.5-cp36-cp36m-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 8f1b552d59de7bdc04df16c635991dcfd36adbb67410824d3f826aaaad6aed56 |
|
MD5 | aaa0fba52ce58d18374055a2a09f5f29 |
|
BLAKE2b-256 | 431b76f1586934bde22269d85c50e77229faef5224211b6ff450df655b5d128c |
Hashes for pyodps-0.11.5-cp36-cp36m-win32.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | f15a3b5c318f9bd123aa51e350a164e5c670f21dfe6aa56228edac974904dc88 |
|
MD5 | de8f7bcf14c97a9ab8e034f7f6bf1985 |
|
BLAKE2b-256 | c795b231c0afa1836e23f81c71c33c583b228c7731eeb39762f65615684aa2ea |
Hashes for pyodps-0.11.5-cp36-cp36m-manylinux1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 9af491835f5e151e2a002dabca79e44761107207bed9e6711f0e30ccd0e99b7b |
|
MD5 | aae2b3818111fb90cfcf4b75994362f8 |
|
BLAKE2b-256 | c27f75325a028123462f3543d30486cfec349f04b9d0dbc365e22d125046a35b |
Hashes for pyodps-0.11.5-cp36-cp36m-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 318c5b7b6a22d6b3bae879d4ffcaac4baba12d6fd23a937c89d5293b7bb48464 |
|
MD5 | df26fae818bea04b5cba6b725dd36057 |
|
BLAKE2b-256 | 1a4dcb26b8c7b9f214e5d4bf4465132929e167af0e3df86493191a9bbda3f9b0 |
Hashes for pyodps-0.11.5-cp35-cp35m-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 479647ac1c3697f6a96a6a391ba7e063067229fe33f9fc7931f5ce4a89d64209 |
|
MD5 | 63c2033bcd8c896ddec62e065966b366 |
|
BLAKE2b-256 | 0be41ee5bda80189cdde2d0c0fd8ade6929937128d162f7dcac4d40493e3d3b1 |
Hashes for pyodps-0.11.5-cp35-cp35m-win32.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 24789ee40ed573bb33d4ef740310a20daf8af6c095252857d021425d97e73826 |
|
MD5 | ad59775c05ab7f332089e72b54b41bdd |
|
BLAKE2b-256 | 55304cecaef152de89e5ce365b8af79655f2bac0104c0b5b00055af737d82a6b |
Hashes for pyodps-0.11.5-cp35-cp35m-manylinux1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | c47048b1f426753efc350b2d71d57022be58f49b0e58a749a4101ba93008900f |
|
MD5 | 8bdef58f8828d6207ec9198e3bebffe9 |
|
BLAKE2b-256 | 07d581b6acff96f3c11e8928a2395523a4ef0d870f1859751c541d2311b0cad3 |
Hashes for pyodps-0.11.5-cp27-cp27mu-manylinux1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 8af2a14d3cbc356f490a8e394bbf50f3fb89e2ccc4450a4b359589ee767ed249 |
|
MD5 | b69157aabed11a73ae91a3e374f0f2a6 |
|
BLAKE2b-256 | 683a849bd2bc3034c4489e290891fbf8b8476c609969daf30567346b0f65a7ef |
Hashes for pyodps-0.11.5-cp27-cp27m-manylinux1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | eb90097550308cfd93ac1549749ed08a11b74e8bb8124637ca6b1d76bef54063 |
|
MD5 | f4c953006ddaa84b10b83a344aed1e40 |
|
BLAKE2b-256 | 9143786f37ed1bf1fc3acb96a93e962c65cd39a5022f6aeb6402090547b7d047 |
Hashes for pyodps-0.11.5-cp27-cp27m-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 8095ac0313492c189765e90b5df43b45211edfda6af7462e7208253d582141a7 |
|
MD5 | 698ffeb5be9af0f4ca69fb8879642163 |
|
BLAKE2b-256 | 7b176152c7b00394b555cf7821267867e38d1c0c1e93d8eefe9e6550ab45a2e6 |