ODPS Python SDK and data analysis framework
Project description
Elegent way to access ODPS API. Documentation
Installation
The quick way:
pip install 'pyodps[full]'
If you don’t need to use Jupyter, just type
pip install pyodps
The dependencies will be installed automatically.
Or from source code:
$ virtualenv pyodps_env
$ source pyodps_env/bin/activate
$ git clone <git clone URL> pyodps
$ cd pyodps
$ python setup.py install
Dependencies
Python (>=2.6), including Python 3+, pypy, Python 2.7 recommended
setuptools (>=3.0)
requests (>=2.4.0)
Run Unittest
copy conf/test.conf.template to odps/tests/test.conf, and fill it with your account
run python -m unittest discover
Usage
>>> from odps import ODPS
>>> o = ODPS('**your-access-id**', '**your-secret-access-key**',
... project='**your-project**', endpoint='**your-end-point**')
>>> dual = o.get_table('dual')
>>> dual.name
'dual'
>>> dual.schema
odps.Schema {
c_int_a bigint
c_int_b bigint
c_double_a double
c_double_b double
c_string_a string
c_string_b string
c_bool_a boolean
c_bool_b boolean
c_datetime_a datetime
c_datetime_b datetime
}
>>> dual.creation_time
datetime.datetime(2014, 6, 6, 13, 28, 24)
>>> dual.is_virtual_view
False
>>> dual.size
448
>>> dual.schema.columns
[<column c_int_a, type bigint>,
<column c_int_b, type bigint>,
<column c_double_a, type double>,
<column c_double_b, type double>,
<column c_string_a, type string>,
<column c_string_b, type string>,
<column c_bool_a, type boolean>,
<column c_bool_b, type boolean>,
<column c_datetime_a, type datetime>,
<column c_datetime_b, type datetime>]
DataFrame API
>>> from odps.df import DataFrame
>>> df = DataFrame(o.get_table('pyodps_iris'))
>>> df.dtypes
odps.Schema {
sepallength float64
sepalwidth float64
petallength float64
petalwidth float64
name string
}
>>> df.head(5)
|==========================================| 1 / 1 (100.00%) 0s
sepallength sepalwidth petallength petalwidth name
0 5.1 3.5 1.4 0.2 Iris-setosa
1 4.9 3.0 1.4 0.2 Iris-setosa
2 4.7 3.2 1.3 0.2 Iris-setosa
3 4.6 3.1 1.5 0.2 Iris-setosa
4 5.0 3.6 1.4 0.2 Iris-setosa
>>> df[df.sepalwidth > 3]['name', 'sepalwidth'].head(5)
|==========================================| 1 / 1 (100.00%) 12s
name sepalwidth
0 Iris-setosa 3.5
1 Iris-setosa 3.2
2 Iris-setosa 3.1
3 Iris-setosa 3.6
4 Iris-setosa 3.9
Command-line and IPython enhancement
In [1]: %load_ext odps In [2]: %enter Out[2]: <odps.inter.Room at 0x10fe0e450> In [3]: %sql select * from pyodps_iris limit 5 |==========================================| 1 / 1 (100.00%) 2s Out[3]: sepallength sepalwidth petallength petalwidth name 0 5.1 3.5 1.4 0.2 Iris-setosa 1 4.9 3.0 1.4 0.2 Iris-setosa 2 4.7 3.2 1.3 0.2 Iris-setosa 3 4.6 3.1 1.5 0.2 Iris-setosa 4 5.0 3.6 1.4 0.2 Iris-setosa
Python UDF Debugging Tool
#file: plus.py
from odps.udf import annotate
@annotate('bigint,bigint->bigint')
class Plus(object):
def evaluate(self, a, b):
return a + b
$ cat plus.input 1,1 3,2 $ pyou plus.Plus < plus.input 2 5
Contributing
For a development install, clone the repository and then install from source:
git clone https://github.com/aliyun/aliyun-odps-python-sdk cd pyodps pip install -r requirements.txt -e .
If you need to modify the frontend code, you need to install nodejs/npm. To build and install your frontend code, use
python setup.py build_js python setup.py install_js
License
Licensed under the Apache License 2.0
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distributions
Hashes for pyodps-0.7.18-cp37-cp37m-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 3d7908cfba1b55cde5d0469001a87b596159c6e40011662243a19bc513b040f6 |
|
MD5 | cb5e1eaf53b5d399be9db7a8ebc5c500 |
|
BLAKE2b-256 | 51586b2598fefd5125e35962216aa27cc1ed2bd5c4889e32ea9ff59a35b3c561 |
Hashes for pyodps-0.7.18-cp37-cp37m-win32.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | f28cc5abf19b9b9c18e79db08bcea801781516ae50d69d9414cc88e879bc0425 |
|
MD5 | 7743f8115f31040ddfa515bc836bebf0 |
|
BLAKE2b-256 | 709cfc3692bb246736455d62caa2da1f18cf2405df8d645b83d1bdc801986d3b |
Hashes for pyodps-0.7.18-cp37-cp37m-manylinux1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 126b9a7040364bd6eb61a6753154da63d35f37f662d0e95d1f355014d6c581a9 |
|
MD5 | 61908bddda17ff6c302fb5e2e835fb89 |
|
BLAKE2b-256 | 95a92cb340bbbeb401a14ae641e585f6cb884ebcfbcabc9683b6754f7a68576b |
Hashes for pyodps-0.7.18-cp37-cp37m-manylinux1_i686.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 8a60b551bc3bb3b6d0c6311c4022120c85af48ed5d42f73d6d2a2df7fa0e0d64 |
|
MD5 | 1c2fb59d93661e919c0e506a61e6f55f |
|
BLAKE2b-256 | 1df5a85a6226df04ecef1d8124b800f0ea17af32908ddaf063f1abe64eadbf15 |
Hashes for pyodps-0.7.18-cp37-cp37m-macosx_10_12_x86_64.macosx_10_9_intel.macosx_10_9_x86_64.macosx_10_10_intel.macosx_10_10_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 53284247b620e914856bb9ba5e4f1c9b0b79faf1306a0b1d09f5d2f00d338482 |
|
MD5 | 777f27afde7b95301bee1d7d6f42c5f2 |
|
BLAKE2b-256 | cfd6f4e15d0dc1a53b9903c50a04dd4a3473c83da28ecde52736a68b32ee0f91 |
Hashes for pyodps-0.7.18-cp36-cp36m-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 3107f4741e12c1b3cecdba28768cb69543c0f8edf7553021f86dd7f50f23c0e9 |
|
MD5 | 1620ceb87c10289e70afd730b7a0904e |
|
BLAKE2b-256 | 1859738eb71e73ceb763c9bccc475423c49b6fb5851fd61514d4406ce924bcce |
Hashes for pyodps-0.7.18-cp36-cp36m-win32.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | bdb3574a079af35906b8a353ccf5a7313328cb4d5ccb634129360c7f60c24581 |
|
MD5 | 4196c06ea3dcc883d235fb59d8a18c11 |
|
BLAKE2b-256 | 18d24117d700d5a08e03fe447cc07b21a75e8eed6f01b2a9cc3c29dc087ced3f |
Hashes for pyodps-0.7.18-cp36-cp36m-manylinux1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | aa9c51e20c1c3e94badecc40734ea77666ba6b426a10d5d3f3d466331781ee96 |
|
MD5 | 3c66e762fc877d8308299e59b6dd61d8 |
|
BLAKE2b-256 | c8cecc3be555955d0a0750c9e2537ef7f1c069e0f00d0120c0c151d7c4fb401a |
Hashes for pyodps-0.7.18-cp36-cp36m-manylinux1_i686.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 2db91ba0135d3eac9013076777a45bc0547c7dd46fd31dbc582ae35e2fd4f131 |
|
MD5 | d0655f6e5895c2d2005867fb3af21f70 |
|
BLAKE2b-256 | 8e53b0ab3f9e47735f7004ba47e438328ae12de87c9db2a66cb8f976beaf2171 |
Hashes for pyodps-0.7.18-cp36-cp36m-macosx_10_12_x86_64.macosx_10_9_intel.macosx_10_9_x86_64.macosx_10_10_intel.macosx_10_10_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | c3ae55cf8aeb352028971c3ea2ddf3c3a39a71a66a9e59a6085361330652600f |
|
MD5 | dcea179c4abef9bb5e5f127dff335d84 |
|
BLAKE2b-256 | ba90246a4b9e15dfbae93835b7d6e0c1aaaa6062f8cb938d763c4c5f02397018 |
Hashes for pyodps-0.7.18-cp35-cp35m-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | f6b1fc5c9db48d74c81c2b7d9a51b29e76f4892fb21fe166f4f4cab085a9b264 |
|
MD5 | c92b548f86ec1d5ca759f98ecb44503b |
|
BLAKE2b-256 | c39188fb53d29b28a158c8cb2280270e1f6bbdbead4d224a28e0d46f714790f0 |
Hashes for pyodps-0.7.18-cp35-cp35m-win32.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 758053709f778cd98994004b404a5f74f96f31b87eb5a870135361a14947b228 |
|
MD5 | 740e0b81987ac62c2f9cebf857e2c0a4 |
|
BLAKE2b-256 | cfa366cdb98a0392536e7e3fa1a7f533688c6ce95e44f02345b3959caaec0302 |
Hashes for pyodps-0.7.18-cp35-cp35m-manylinux1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 1708da881760e01fc875c45e2d3ddc7254133a3848a47201a3dc58adea34b5f8 |
|
MD5 | 6d2c609ea3f315aa29d36e0ed1b0b939 |
|
BLAKE2b-256 | a681aca2cc2673f0d8d429931f9285a8f6629523e4cb3c154d8b53802f93712e |
Hashes for pyodps-0.7.18-cp35-cp35m-manylinux1_i686.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 4f3589cafd7209d34aa7c738a61dd359614afb5559ba66acda84f0de35d90028 |
|
MD5 | 65deb9ce621bba8ebb615bf01d12deec |
|
BLAKE2b-256 | ea7cc42f7809f42050b7e613d24001fcb18838e7f498691ad5cbbf4a037ebfe1 |
Hashes for pyodps-0.7.18-cp35-cp35m-macosx_10_12_x86_64.macosx_10_9_intel.macosx_10_9_x86_64.macosx_10_10_intel.macosx_10_10_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | e0de668e7049ff956159d1f3e149b172e9ddcea676fe247c241eca48da8b1e8f |
|
MD5 | 6f232c0af18741acd6dff3c1d777b79e |
|
BLAKE2b-256 | 3ed175bf72823c60174ac1ffd05122c9a21bdc3aaf37ecd89c3cc02178dc6502 |
Hashes for pyodps-0.7.18-cp27-cp27mu-manylinux1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 12174e8da6ff7376a655bed3b38cbb9bd6c26941d9c980a2c20b9e7a4b5b5e23 |
|
MD5 | a00ae300c004cfb6f27b8ecf198513b2 |
|
BLAKE2b-256 | 1c819e9a79bc8628f685352f077225b0337474c857487895047063e1fb6f893c |
Hashes for pyodps-0.7.18-cp27-cp27mu-manylinux1_i686.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | d9967dd2c1ca3d33674bb1134956ee990fc13f571c8e476bee5296913184bc5e |
|
MD5 | 05b14737817e4e35a6e1384e8b99f0bc |
|
BLAKE2b-256 | 0e528c9b9230d50180fcf1e5294baa6ee532a2ab86634d507175c093831e3a05 |
Hashes for pyodps-0.7.18-cp27-cp27m-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | d4f2b4be6370f5292c9b492197345a96ea26937929c831712d86a7264267c9f6 |
|
MD5 | 82a9f15978bfd4565b72931d04150b21 |
|
BLAKE2b-256 | cdea98296dceadcc4c38d23519f52da798f44b5e203aeb0fc607b07abd1c03c9 |
Hashes for pyodps-0.7.18-cp27-cp27m-win32.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | e1c4f7e3656625837271d0f934beb98c8057e96b7eb8fffee38bf20e1bdf361d |
|
MD5 | 67544ec99b6f1d10c1e03579eddff03a |
|
BLAKE2b-256 | 4939876881debb8ee898a88cb0753502ba831d2ea7068740fc58ed43d84478ef |
Hashes for pyodps-0.7.18-cp27-cp27m-manylinux1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 205eb481d95ac5906043aa779f00283275ca585fa1661a3a7dc3f6c8cf4583af |
|
MD5 | 5349dc362181dd290cd8d4026dee9770 |
|
BLAKE2b-256 | f7d0726dbaf842c96a4af7198e898164eea516a2f64d2de899e3719635c77864 |
Hashes for pyodps-0.7.18-cp27-cp27m-manylinux1_i686.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 474762eec8fd4843bd3b086eb0bbe76a8c3a7d14b245f98d8ba2f13f61272d92 |
|
MD5 | 3169444e82d928b58108b1131f17c37e |
|
BLAKE2b-256 | e597fb1b03152f76ce52a56bfb39e858f0ccb1455299c893db0188895b1538a5 |
Hashes for pyodps-0.7.18-cp27-cp27m-macosx_10_12_x86_64.macosx_10_9_intel.macosx_10_9_x86_64.macosx_10_10_intel.macosx_10_10_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 335bd2ed6bbcfca8aa56ade9939c62faf0bdf6e3122bdd5c14bbb9ca1f3317dc |
|
MD5 | 63d4472da918e772e5348088b1f936b5 |
|
BLAKE2b-256 | 8167efe5394b16458e0d1b37ab6f2d912a4607704357f24842293ca07b309f51 |