ODPS Python SDK and data analysis framework
Project description
Elegent way to access ODPS API. Documentation
Installation
The quick way:
pip install 'pyodps[full]'
If you don’t need to use Jupyter, just type
pip install pyodps
The dependencies will be installed automatically.
Or from source code:
$ virtualenv pyodps_env
$ source pyodps_env/bin/activate
$ git clone <git clone URL> pyodps
$ cd pyodps
$ python setup.py install
Dependencies
Python (>=2.6), including Python 3+, pypy, Python 2.7 recommended
setuptools (>=3.0)
requests (>=2.4.0)
Run Unittest
copy conf/test.conf.template to odps/tests/test.conf, and fill it with your account
run python -m unittest discover
Usage
>>> from odps import ODPS
>>> o = ODPS('**your-access-id**', '**your-secret-access-key**',
... project='**your-project**', endpoint='**your-end-point**')
>>> dual = o.get_table('dual')
>>> dual.name
'dual'
>>> dual.schema
odps.Schema {
c_int_a bigint
c_int_b bigint
c_double_a double
c_double_b double
c_string_a string
c_string_b string
c_bool_a boolean
c_bool_b boolean
c_datetime_a datetime
c_datetime_b datetime
}
>>> dual.creation_time
datetime.datetime(2014, 6, 6, 13, 28, 24)
>>> dual.is_virtual_view
False
>>> dual.size
448
>>> dual.schema.columns
[<column c_int_a, type bigint>,
<column c_int_b, type bigint>,
<column c_double_a, type double>,
<column c_double_b, type double>,
<column c_string_a, type string>,
<column c_string_b, type string>,
<column c_bool_a, type boolean>,
<column c_bool_b, type boolean>,
<column c_datetime_a, type datetime>,
<column c_datetime_b, type datetime>]
DataFrame API
>>> from odps.df import DataFrame
>>> df = DataFrame(o.get_table('pyodps_iris'))
>>> df.dtypes
odps.Schema {
sepallength float64
sepalwidth float64
petallength float64
petalwidth float64
name string
}
>>> df.head(5)
|==========================================| 1 / 1 (100.00%) 0s
sepallength sepalwidth petallength petalwidth name
0 5.1 3.5 1.4 0.2 Iris-setosa
1 4.9 3.0 1.4 0.2 Iris-setosa
2 4.7 3.2 1.3 0.2 Iris-setosa
3 4.6 3.1 1.5 0.2 Iris-setosa
4 5.0 3.6 1.4 0.2 Iris-setosa
>>> df[df.sepalwidth > 3]['name', 'sepalwidth'].head(5)
|==========================================| 1 / 1 (100.00%) 12s
name sepalwidth
0 Iris-setosa 3.5
1 Iris-setosa 3.2
2 Iris-setosa 3.1
3 Iris-setosa 3.6
4 Iris-setosa 3.9
Command-line and IPython enhancement
In [1]: %load_ext odps In [2]: %enter Out[2]: <odps.inter.Room at 0x10fe0e450> In [3]: %sql select * from pyodps_iris limit 5 |==========================================| 1 / 1 (100.00%) 2s Out[3]: sepallength sepalwidth petallength petalwidth name 0 5.1 3.5 1.4 0.2 Iris-setosa 1 4.9 3.0 1.4 0.2 Iris-setosa 2 4.7 3.2 1.3 0.2 Iris-setosa 3 4.6 3.1 1.5 0.2 Iris-setosa 4 5.0 3.6 1.4 0.2 Iris-setosa
Python UDF Debugging Tool
#file: plus.py
from odps.udf import annotate
@annotate('bigint,bigint->bigint')
class Plus(object):
def evaluate(self, a, b):
return a + b
$ cat plus.input 1,1 3,2 $ pyou plus.Plus < plus.input 2 5
Contributing
For a development install, clone the repository and then install from source:
git clone https://github.com/aliyun/aliyun-odps-python-sdk cd pyodps pip install -r requirements.txt -e .
If you need to modify the frontend code, you need to install nodejs/npm. To build and install your frontend code, use
python setup.py build_js python setup.py install_js
License
Licensed under the Apache License 2.0
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distributions
Hashes for pyodps-0.8.1-cp37-cp37m-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 9c9788cf6b9cd3c8827e43635b627881c1cd517d23ce9405de8947c9b8f390d9 |
|
MD5 | 77f15e71f9db2d914b3c8e804e614b4b |
|
BLAKE2b-256 | 4bf7c8a95b29cd729bda2d069e7743ec51aa3918daa8271ecf5269ce0a2632c6 |
Hashes for pyodps-0.8.1-cp37-cp37m-win32.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 23d6bcda4958e7fb104a66955582c22befbd8488b4d4b57e54d1402c64c7cc5c |
|
MD5 | ef3d399437e2099278adac8613b94679 |
|
BLAKE2b-256 | 0c3833df7ac7a86ec673293a9528a0679d3b3672383de8a23b41766f75dd6658 |
Hashes for pyodps-0.8.1-cp37-cp37m-manylinux1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | cba7dd99d822b0caaa13b96dff2dab102a1725ada2e2e383ec62a0ab16ff2bd6 |
|
MD5 | 3055ec30dc4c1779fd3e8f7feb4c93aa |
|
BLAKE2b-256 | 858da930a179ac6d119aa4e7d6dd4bbee352557bc37ca0f2a2e1bac204661667 |
Hashes for pyodps-0.8.1-cp37-cp37m-manylinux1_i686.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 264922cae35f8cf3892a86f3442a8acbcdc34a8c496df8b7adffa47e29e19ff8 |
|
MD5 | 67451556498ac71f7bc51a5593d31d60 |
|
BLAKE2b-256 | 93e1afdae966e68bfe9a6a5a85a639ba9af90683c83cc418e4aed4c10c4ecfb9 |
Hashes for pyodps-0.8.1-cp37-cp37m-macosx_10_13_x86_64.macosx_10_9_intel.macosx_10_9_x86_64.macosx_10_10_intel.macosx_10_10_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 5c408d136abf16c83410e2687ac544b2bf033b38c05d9e962766785665479e1b |
|
MD5 | 80b7060b710940d26575e48b46ef1aa6 |
|
BLAKE2b-256 | f5eddd31c01c6a4fa99263c5676cc5a54eec7665bbbe72e713207e053f6d75ea |
Hashes for pyodps-0.8.1-cp36-cp36m-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | f417eeb361fc0f17ddd21086cbaae82ecfcfd8cd7fecf322808c9de6782a1aab |
|
MD5 | 2f1ff9caf8b62f09558ddf322fe30c9e |
|
BLAKE2b-256 | 5a5f3781518dc9cf659cc31f546b56bdea9469ac7a4ff8cbdfd0437e58181e77 |
Hashes for pyodps-0.8.1-cp36-cp36m-win32.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 2f46d073e5e2faa599aee494ec9226f6d6e776545e535ab04a62a59f5f5dbb3e |
|
MD5 | 3ccdbecc2b08dfde87c023a87f0ba5b9 |
|
BLAKE2b-256 | 62370dc2ee5699368176307bb2929971982f2d2e50643e1cfadc2e42a00ab526 |
Hashes for pyodps-0.8.1-cp36-cp36m-manylinux1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 2d1a9a60c87580fb4017035a6eebaeeb3cbcbbe85d3ac4e372dffba2ad3c572b |
|
MD5 | c3983c377b3ed53cf286c0b6f26bbdc3 |
|
BLAKE2b-256 | ec119e0c531994a9040cb12b04dc204c1f63e09f99375a165f224369fadde3cd |
Hashes for pyodps-0.8.1-cp36-cp36m-manylinux1_i686.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 1020f517f76d7ad6a6e188e0c7a74a644fd6a519d0d50a56a895c9d758fcb0c9 |
|
MD5 | 4db52822df545a02350c2f299f705ccc |
|
BLAKE2b-256 | 572c754c3a30dec95a38d6ca8ae4cacbe89d723782a27bdda000ea2c22fde037 |
Hashes for pyodps-0.8.1-cp36-cp36m-macosx_10_13_x86_64.macosx_10_9_intel.macosx_10_9_x86_64.macosx_10_10_intel.macosx_10_10_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 6e963075e85b92a245330c45f0e426f53cdeaa6741b53510f9549ed7fbe6e147 |
|
MD5 | 27aa8515b2a93d5b6862ec51e5d71dd0 |
|
BLAKE2b-256 | 456809212d683b383af47c762c68bb6fd02bcef97d6e9612a641cd02fcce9c64 |
Hashes for pyodps-0.8.1-cp35-cp35m-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 4504d7d7d0b5161f9e7c3830e5086b3fded009b2d2d8daec8a06968f0b034ae9 |
|
MD5 | fe7481d148a02c8713f22cfa29ad063a |
|
BLAKE2b-256 | 9f4772dcb397b3f57f4a37377d19cf72e520bd9a4c356f4e7b5994aca9e2197a |
Hashes for pyodps-0.8.1-cp35-cp35m-win32.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | a72d161860adb140960a2e8b16a05271a21f4793c717c4deb318ec7e880ab103 |
|
MD5 | b995367c3bdc2d3ccd407dda331c049f |
|
BLAKE2b-256 | 4f0469baaf4233cc2e45422dbc5e9a3762cefc0c22bbd5cc45662fb7b4218cb0 |
Hashes for pyodps-0.8.1-cp35-cp35m-manylinux1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | ab1a971c760b4a5a4839c33661a49a1b0f3e9861eb4472300cd33aa0ee32edc4 |
|
MD5 | 7622b009f08d9b8c85c5d24ab2f3839c |
|
BLAKE2b-256 | 32a0791d23a7f05f5d5ad13a81a6f2343f39e3cc1c03d0a13a1034106aad3e29 |
Hashes for pyodps-0.8.1-cp35-cp35m-manylinux1_i686.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | c65432d2cb8fc5d27681abde5d4bbead3e8dacaf5e3a4628ab32d50cdd59cf51 |
|
MD5 | 8f9d66b0c4aa1a2a5a40d7317dca7469 |
|
BLAKE2b-256 | d7fa203a816d2ae41defdc1d2d0dfa96775cb40204581446ae4654790f543c5d |
Hashes for pyodps-0.8.1-cp35-cp35m-macosx_10_13_x86_64.macosx_10_9_intel.macosx_10_9_x86_64.macosx_10_10_intel.macosx_10_10_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 593a7e0bc9cf62e5463a83157b384c8a39cd2b0ad55a016fe9307d13ef52a0d1 |
|
MD5 | 55a9704d848d7c0364e2ea8f5042cc9d |
|
BLAKE2b-256 | e0db204a5f99ac6f84ce324f988336d5f5e554078f31ae22cb194506829cc713 |
Hashes for pyodps-0.8.1-cp27-cp27mu-manylinux1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 15025b17749f4bd49e54ace7d78099628bae3b877a554444f3bc9ddaa76c2d34 |
|
MD5 | 95efe0fd13fa7322e7341021829e24c7 |
|
BLAKE2b-256 | 28fc345e80490e4a890bf70d33f6dbaecb121a81a8e99f9c920607c295601de6 |
Hashes for pyodps-0.8.1-cp27-cp27mu-manylinux1_i686.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 3816d84d6e6713542ece751349a3283c9a890a1cb532b025e277e9434af3619c |
|
MD5 | 6387f4051d401cdebb3f84347d136cdf |
|
BLAKE2b-256 | 4d257a511310a5c51825189f765eb4189544979252f05b0ccd97334132d288b7 |
Hashes for pyodps-0.8.1-cp27-cp27m-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 5694655e5ef34ba56865b0711183fcc9b6b50debbbc52b501426647f3042e8c0 |
|
MD5 | 31f9595195fa83248815179618d49b5f |
|
BLAKE2b-256 | ccc3d925facd3ddd954c0041fd2583b08c3bec3b0f7e16241a08047226633f5c |
Hashes for pyodps-0.8.1-cp27-cp27m-win32.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | a74ee0a871aca953ada9360b6119039f821e118e96ec838c25b864d2fa9f69b7 |
|
MD5 | 93277a55c768286ea50e7de3b5f7af9d |
|
BLAKE2b-256 | 68550011b4a89901453778d3246f7b5d8473e45e07816cd7f6cb30ddf088618d |
Hashes for pyodps-0.8.1-cp27-cp27m-manylinux1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 388e628e6de4cbe7a7e7cc0fecbfdd3eb7d36315d359118540a180e0769a0a85 |
|
MD5 | d544fd9896c3d76502f7782ed222b1f9 |
|
BLAKE2b-256 | 8fccffbd9882ec8a61b5f39c63d38ad1a2e27472ff4a51388654cd2baec77d51 |
Hashes for pyodps-0.8.1-cp27-cp27m-manylinux1_i686.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | dd72e4be86e22a2bf82273121b9dcfc9179f9c265b8f395acd9eec6e663b30b1 |
|
MD5 | 6f14a2a3c642bd2a5c085abba702f6a4 |
|
BLAKE2b-256 | db3d81dd3314451327fee19696d287260a22547a6e63179093cca203f25b784d |
Hashes for pyodps-0.8.1-cp27-cp27m-macosx_10_13_x86_64.macosx_10_9_intel.macosx_10_9_x86_64.macosx_10_10_intel.macosx_10_10_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | a93ac1c91a632bdecbf7d669ad01845d2a7183be6abe37dfe81c8b4179690c9e |
|
MD5 | 12b2eed2b7adaf1d4a0cf2581834f7a5 |
|
BLAKE2b-256 | 48149949e05e0a5dc1da1c3cf0c8f3e590f03b02d97c725705155fb3a55f05cc |