ODPS Python SDK and data analysis framework
Project description
Elegent way to access ODPS API. Documentation
Installation
The quick way:
pip install pyodps[full]
If you don’t need to use Jupyter, just type
pip install pyodps
The dependencies will be installed automatically.
Or from source code:
$ virtualenv pyodps_env
$ source pyodps_env/bin/activate
$ pip install git+https://github.com/aliyun/aliyun-odps-python-sdk.git
Dependencies
Python (>=2.7), including Python 3+, pypy, Python 3.7 recommended
setuptools (>=3.0)
Run Tests
install pytest
copy conf/test.conf.template to odps/tests/test.conf, and fill it with your account
run pytest odps
Usage
>>> from odps import ODPS
>>> o = ODPS('**your-access-id**', '**your-secret-access-key**',
... project='**your-project**', endpoint='**your-end-point**')
>>> dual = o.get_table('dual')
>>> dual.name
'dual'
>>> dual.table_schema
odps.Schema {
c_int_a bigint
c_int_b bigint
c_double_a double
c_double_b double
c_string_a string
c_string_b string
c_bool_a boolean
c_bool_b boolean
c_datetime_a datetime
c_datetime_b datetime
}
>>> dual.creation_time
datetime.datetime(2014, 6, 6, 13, 28, 24)
>>> dual.is_virtual_view
False
>>> dual.size
448
>>> dual.table_schema.columns
[<column c_int_a, type bigint>,
<column c_int_b, type bigint>,
<column c_double_a, type double>,
<column c_double_b, type double>,
<column c_string_a, type string>,
<column c_string_b, type string>,
<column c_bool_a, type boolean>,
<column c_bool_b, type boolean>,
<column c_datetime_a, type datetime>,
<column c_datetime_b, type datetime>]
DataFrame API
>>> from odps.df import DataFrame
>>> df = DataFrame(o.get_table('pyodps_iris'))
>>> df.dtypes
odps.Schema {
sepallength float64
sepalwidth float64
petallength float64
petalwidth float64
name string
}
>>> df.head(5)
|==========================================| 1 / 1 (100.00%) 0s
sepallength sepalwidth petallength petalwidth name
0 5.1 3.5 1.4 0.2 Iris-setosa
1 4.9 3.0 1.4 0.2 Iris-setosa
2 4.7 3.2 1.3 0.2 Iris-setosa
3 4.6 3.1 1.5 0.2 Iris-setosa
4 5.0 3.6 1.4 0.2 Iris-setosa
>>> df[df.sepalwidth > 3]['name', 'sepalwidth'].head(5)
|==========================================| 1 / 1 (100.00%) 12s
name sepalwidth
0 Iris-setosa 3.5
1 Iris-setosa 3.2
2 Iris-setosa 3.1
3 Iris-setosa 3.6
4 Iris-setosa 3.9
Command-line and IPython enhancement
In [1]: %load_ext odps In [2]: %enter Out[2]: <odps.inter.Room at 0x10fe0e450> In [3]: %sql select * from pyodps_iris limit 5 |==========================================| 1 / 1 (100.00%) 2s Out[3]: sepallength sepalwidth petallength petalwidth name 0 5.1 3.5 1.4 0.2 Iris-setosa 1 4.9 3.0 1.4 0.2 Iris-setosa 2 4.7 3.2 1.3 0.2 Iris-setosa 3 4.6 3.1 1.5 0.2 Iris-setosa 4 5.0 3.6 1.4 0.2 Iris-setosa
Python UDF Debugging Tool
#file: plus.py
from odps.udf import annotate
@annotate('bigint,bigint->bigint')
class Plus(object):
def evaluate(self, a, b):
return a + b
$ cat plus.input 1,1 3,2 $ pyou plus.Plus < plus.input 2 5
Contributing
For a development install, clone the repository and then install from source:
git clone https://github.com/aliyun/aliyun-odps-python-sdk.git cd pyodps pip install -r requirements.txt -e .
If you need to modify the frontend code, you need to install nodejs/npm. To build and install your frontend code, use
python setup.py build_js python setup.py install_js
License
Licensed under the Apache License 2.0
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distributions
Hashes for pyodps-0.11.4.1-cp310-cp310-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 355660ddfe782bdac126062fa550bfc5ab063b369b22dc574768dc63313028f5 |
|
MD5 | 0820d560fd0c7bbcc54c832e2f5b631f |
|
BLAKE2b-256 | f682ce313305b6a02c946fc98016b4d79df12626cbec08a9119676b63b9c49f1 |
Hashes for pyodps-0.11.4.1-cp310-cp310-win32.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 1dcde289dfc75f6da9e683e85b11b4c638d6a5c85830f136b34a6e2e26b22b19 |
|
MD5 | 354687ea5ba86c590834f6b61bbe2554 |
|
BLAKE2b-256 | c2d85b58c29b1af5432e6936385bbbe71167aa7937e678c4b87183cf38d6a36f |
Hashes for pyodps-0.11.4.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 55844ed6811df48e673f8b4886ecee8a0b2bb1e8de67a2b9095619bccccc03e2 |
|
MD5 | 54e27917e8dd3b54f15075fecde2ae31 |
|
BLAKE2b-256 | 96611e702f7871f81849b3aa0503ba6bc31b6dbb96ac1e2e8aa16519fb8db70f |
Hashes for pyodps-0.11.4.1-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 598e96439b62ce1774947af273ac41d5d23252899b83c13438dea448cdae9817 |
|
MD5 | cf731623371472c3448b60f683919f01 |
|
BLAKE2b-256 | ad233fb226604d0ed813219128f17c72dafd50073e03835c7cee9b067f09d527 |
Hashes for pyodps-0.11.4.1-cp310-cp310-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | b94db6231fb655b6f243bf7406a12adee191a11dc187134413b0e3813cfa0ec0 |
|
MD5 | 4b9748fb142c4a31a9663a51ae07267b |
|
BLAKE2b-256 | 5a45d105c70c633a8ac42040d1e9f80561f4353afe21d9ff929d2b788028a9ee |
Hashes for pyodps-0.11.4.1-cp310-cp310-macosx_10_9_universal2.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | bb218714f5aaa2a2114cd88368fd70f8450593c3095938635bd960a9c9f65090 |
|
MD5 | 4a9615747232ab20be59ff4cdf7ab37c |
|
BLAKE2b-256 | c8a9e169e0d9934cbef5440b86de1e925f5f34702a20e7dec36c350da2398315 |
Hashes for pyodps-0.11.4.1-cp39-cp39-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 3d692be36a35a230487e63bf212b700bea8e7d09acf99eb4d04d3f9396248692 |
|
MD5 | c480af2966a6ec5f1e9eca4e95868191 |
|
BLAKE2b-256 | fd728ad770e1b6f61b7f09b17ec4f273d7ab6caa534c328c0147c36583a571c3 |
Hashes for pyodps-0.11.4.1-cp39-cp39-win32.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 3852a0acbeb2fcd31593ce7b855f201bb5a81492c221e196caf00d720d6c6297 |
|
MD5 | a45d6fa2e722899345b882db31db5176 |
|
BLAKE2b-256 | cd0d8783ea871f12ec7fef1c4b8009fe2958f48dacdd7f284f24f765c6d6e961 |
Hashes for pyodps-0.11.4.1-cp39-cp39-manylinux2014_aarch64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | f48472812ba4ea19dc88dca6b8c9d252fcd736c874a65e1a17091b20ccfe7220 |
|
MD5 | ea9e8955130d6e98459586b71510019e |
|
BLAKE2b-256 | dbdd8ef610c1610ccbfd96dac47665c4811dd4315e8a1125ebbe7a8f78ff10cb |
Hashes for pyodps-0.11.4.1-cp39-cp39-manylinux1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | c2c79e4928ed11b48feaf5253c1959b0dbd2edabfe8ee217b23aa866fd01cd95 |
|
MD5 | d5a46a4c5ab874236d221ac27c0de3e3 |
|
BLAKE2b-256 | 1d62172c38258c7d548df196cce80515fc71758f95059586aa787bf96ab0b533 |
Hashes for pyodps-0.11.4.1-cp39-cp39-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | be2e92bfb8df172c7b3db6bbdab3ed8c041dafcb5ad93580f87ae94f8433d0d5 |
|
MD5 | d52ef3ae21c7e89f919af1fdcc3b4d7c |
|
BLAKE2b-256 | 690c9b9ea201131e02d761122359face4e0ccfe4312045c23fa249d14b2b6cf0 |
Hashes for pyodps-0.11.4.1-cp39-cp39-macosx_10_9_universal2.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 10deea20ba4a0d9a5af5c0b748e614977027e245f26ec86bd68b550b5a811b71 |
|
MD5 | ffca3a70eba188ee6bd5ae8c9047b41b |
|
BLAKE2b-256 | a6112200fdf467a897b6695e73789f8e1e4424d6e3ff22a6061445df190e1a6b |
Hashes for pyodps-0.11.4.1-cp38-cp38-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 13158f5988a130e04389e63c50a1327716dea9bc82cfa73da19e6156f7de7565 |
|
MD5 | 8e2d26a5bf0379d9ec4d7bba51df163f |
|
BLAKE2b-256 | 8286895d5cef3f0baec2e6950e52df70c6817f6c048bc2fde5341e79ce7a6d22 |
Hashes for pyodps-0.11.4.1-cp38-cp38-win32.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | eefd3837c5189e96003ce250f7960981ba309880c6df6de919fe6afa300cfcf3 |
|
MD5 | df15e20ec28304f068dda2830e7b7973 |
|
BLAKE2b-256 | 6f1ce8f578a3d580f6c9b5a0bdafda218c984af48f12cb1fc516c53b74e40639 |
Hashes for pyodps-0.11.4.1-cp38-cp38-manylinux2014_aarch64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 2e563714e223238dc27610be89e97413643ade646a07fd19f42c2d7acfa8ef91 |
|
MD5 | d3f52d72baf15eb4edf7d824a6e2644b |
|
BLAKE2b-256 | 7ddeb33aa739779bed8596c53d0a7927166f8f3114cbb9102b6aa7af250b6497 |
Hashes for pyodps-0.11.4.1-cp38-cp38-manylinux1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | a666b5ff74eb7c54b1bcd7c13effa1e00efcb0de74e3779fef0bae4d7a214bf9 |
|
MD5 | 14fd89c2e76711ac7b3e20ba0245c330 |
|
BLAKE2b-256 | 15bb82e5cda2f232c335e44c5545f327616c72de02c92ea2cf361514de683811 |
Hashes for pyodps-0.11.4.1-cp38-cp38-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | a59d3cb526178f756a794ee6be71488a62de8be38c4ba3f6235cef9213144f65 |
|
MD5 | 7bbc95cd9e30228da8d78a031b7bfe19 |
|
BLAKE2b-256 | 0eb75291ff2ab8e8f5640a31d07acb23ef3fc5a05b94dc9fb9e949188cab363e |
Hashes for pyodps-0.11.4.1-cp37-cp37m-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 37622b2a8603ae67cd09b7f075b67d7fab1c046c73f95f3b31dfc2a8447a0127 |
|
MD5 | 7d8c0ba56cb59d88d2432646ed429224 |
|
BLAKE2b-256 | 7af42152cc6724cd8396b368075889ec90da0706e0b27874b5a149e3312d4a8e |
Hashes for pyodps-0.11.4.1-cp37-cp37m-win32.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 02f0fd82d8fef380bdd2e389044425afcd4c442da82dce71e01305603951fabe |
|
MD5 | cdcf661c5c9ca7ba2f641b2c5d7a5764 |
|
BLAKE2b-256 | 40d9879ee412bc49b87392617a313443dc4f7820ab2dde7dd7f072465c0be8be |
Hashes for pyodps-0.11.4.1-cp37-cp37m-manylinux2014_aarch64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 58d50b32e44eb3a05a718bc2a47c86c35f60efcdb40e32c794604ea16b0fb7cf |
|
MD5 | ed291578f2665de8f97fd74eeb27e63f |
|
BLAKE2b-256 | ac3300feca08795073fcc90b60c57822b66f549d80ec0a407932069f4f17e6e9 |
Hashes for pyodps-0.11.4.1-cp37-cp37m-manylinux1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 654001a123262de73055567c8b7308888c230a71b8a98897946c839b48755ce2 |
|
MD5 | 492ea54919feec95d6a7d85ac4526096 |
|
BLAKE2b-256 | 89bd8aa04b30356d98e7d2398929475268576ec8c4e9b2fe70ef09b4c1f31fa9 |
Hashes for pyodps-0.11.4.1-cp37-cp37m-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 85bd1b147dbc771f75443cdbc281e2e6cef3113cf5d1e0fae59acfe1cbfea8af |
|
MD5 | f9478956624ed97eabc80c4aa29c20aa |
|
BLAKE2b-256 | e668f5babe52187dd0cec61ac0a1a99e223c3f5082fdbff98baabe85899d7d82 |
Hashes for pyodps-0.11.4.1-cp36-cp36m-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 20a43d9be6ff15d77d421fefeb15ee8b84ea43ba55a45f062807ecdf2e3d8f4a |
|
MD5 | e608d71abb84c6525466cd6174ec6148 |
|
BLAKE2b-256 | 11153ae2376e0141a6fde0917e03e7ba49f3d61f3c596dd5598e9db25951103b |
Hashes for pyodps-0.11.4.1-cp36-cp36m-win32.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | fa78700b2810218be8ab601e6a3d29ca290c681f432fdf9d15e2dcd33b4c594f |
|
MD5 | 767f77765c2327db135b9019c20bfbfd |
|
BLAKE2b-256 | 679a0a70e9cce05d25d87eac1adf0523794d28a19bbc584e899cf48eb119773d |
Hashes for pyodps-0.11.4.1-cp36-cp36m-manylinux1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 2e9bb240c7836e2829e0bc401950270b96f8d5b7d45f8498e787c24d55e062eb |
|
MD5 | ee84ed1a31b33f703291e0c5e82c7b16 |
|
BLAKE2b-256 | 591791b770b1c6fea7f9df75aafc04ace3e36c2f5c31b2ccddd05c031093654f |
Hashes for pyodps-0.11.4.1-cp36-cp36m-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 9418ac7f44b2a5c68a85791a07c9f369ffa278ef7f7a48b4dc62b1d226f87ef7 |
|
MD5 | 20b352fae0f57ca9b32fc295742936f7 |
|
BLAKE2b-256 | fc87720f119f7d656fb0898e566975b22ff7ae34d7b11045b188309b2c18a6b9 |
Hashes for pyodps-0.11.4.1-cp35-cp35m-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | a9ef1d53548b0da948988c05a02a93f3cd7d7f7687b54f086b9f36793babfa81 |
|
MD5 | 132c02603fcd4a6f902e1c9f8f189fe5 |
|
BLAKE2b-256 | 528d120bfca4b9bb518d2c35761caf4378bc850f316d99fe0a9f55cedd99e694 |
Hashes for pyodps-0.11.4.1-cp35-cp35m-win32.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 53b4e6f86a3c9cc4cf8554311e1c1c711f248f0cb22a1229f8183e0b2df4c51e |
|
MD5 | 7b5b1361c8187b5d28819ab9f197e38d |
|
BLAKE2b-256 | 135a1a119ada4aa817e89cee413800651df48770832a8d06b3ae4ad54dc5f118 |
Hashes for pyodps-0.11.4.1-cp35-cp35m-manylinux1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | c326f5d3f8075bac592649ac03fed6e205d18a1974e3b140b7cb55a9aa15e2d7 |
|
MD5 | 251cdca076ace865208ab290192e734e |
|
BLAKE2b-256 | 468b47d8c69d73748a9e42d61c0a1c7e45f8ef1cce149b8046a12ed4cad0a72d |
Hashes for pyodps-0.11.4.1-cp27-cp27mu-manylinux1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 053974bb2ab6fe5fa34c67f71aa31252a61bd95c47c4ec18e31039fa7a2cb896 |
|
MD5 | 353169d47b9bed7f40b78cafb72dd0cb |
|
BLAKE2b-256 | 59ebbb5e5dd29c43c96af67474b3623bb472ee03fcd946eec17ed7519c41296c |
Hashes for pyodps-0.11.4.1-cp27-cp27m-manylinux1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 95c5b37baeef1124e62f648900a30b41fff914a94faf2363b433497959637494 |
|
MD5 | fc26b6397dd7da5c7c01d4f03d1692cf |
|
BLAKE2b-256 | 9e2eeb704f82e16ef3df95690d59b403554c242be011c88b43c45ba3a913ac11 |
Hashes for pyodps-0.11.4.1-cp27-cp27m-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 64f66707ea3cb3436d2730630144b10e012b7d078a5854df141df395fd3f8f77 |
|
MD5 | 04f0cdfabdf62ffe928b194605b15232 |
|
BLAKE2b-256 | 5b0df86991c8e83926e1cb735c474b53324b1d77bd80e64a130e1c54f9d2547b |