ODPS Python SDK and data analysis framework
Project description
Elegent way to access ODPS API. Documentation
Installation
The quick way:
pip install pyodps[full]
If you don’t need to use Jupyter, just type
pip install pyodps
The dependencies will be installed automatically.
Or from source code:
$ virtualenv pyodps_env
$ source pyodps_env/bin/activate
$ git clone <git clone URL> pyodps
$ cd pyodps
$ python setup.py install
Dependencies
Python (>=2.6), including Python 3+, pypy, Python 2.7 recommended
setuptools (>=3.0)
requests (>=2.4.0)
Run Unittest
copy conf/test.conf.template to odps/tests/test.conf, and fill it with your account
run python -m unittest discover
Usage
>>> from odps import ODPS
>>> o = ODPS('**your-access-id**', '**your-secret-access-key**',
... project='**your-project**', endpoint='**your-end-point**')
>>> dual = o.get_table('dual')
>>> dual.name
'dual'
>>> dual.schema
odps.Schema {
c_int_a bigint
c_int_b bigint
c_double_a double
c_double_b double
c_string_a string
c_string_b string
c_bool_a boolean
c_bool_b boolean
c_datetime_a datetime
c_datetime_b datetime
}
>>> dual.creation_time
datetime.datetime(2014, 6, 6, 13, 28, 24)
>>> dual.is_virtual_view
False
>>> dual.size
448
>>> dual.schema.columns
[<column c_int_a, type bigint>,
<column c_int_b, type bigint>,
<column c_double_a, type double>,
<column c_double_b, type double>,
<column c_string_a, type string>,
<column c_string_b, type string>,
<column c_bool_a, type boolean>,
<column c_bool_b, type boolean>,
<column c_datetime_a, type datetime>,
<column c_datetime_b, type datetime>]
DataFrame API
>>> from odps.df import DataFrame
>>> df = DataFrame(o.get_table('pyodps_iris'))
>>> df.dtypes
odps.Schema {
sepallength float64
sepalwidth float64
petallength float64
petalwidth float64
name string
}
>>> df.head(5)
|==========================================| 1 / 1 (100.00%) 0s
sepallength sepalwidth petallength petalwidth name
0 5.1 3.5 1.4 0.2 Iris-setosa
1 4.9 3.0 1.4 0.2 Iris-setosa
2 4.7 3.2 1.3 0.2 Iris-setosa
3 4.6 3.1 1.5 0.2 Iris-setosa
4 5.0 3.6 1.4 0.2 Iris-setosa
>>> df[df.sepalwidth > 3]['name', 'sepalwidth'].head(5)
|==========================================| 1 / 1 (100.00%) 12s
name sepalwidth
0 Iris-setosa 3.5
1 Iris-setosa 3.2
2 Iris-setosa 3.1
3 Iris-setosa 3.6
4 Iris-setosa 3.9
Command-line and IPython enhancement
In [1]: %load_ext odps In [2]: %enter Out[2]: <odps.inter.Room at 0x10fe0e450> In [3]: %sql select * from pyodps_iris limit 5 |==========================================| 1 / 1 (100.00%) 2s Out[3]: sepallength sepalwidth petallength petalwidth name 0 5.1 3.5 1.4 0.2 Iris-setosa 1 4.9 3.0 1.4 0.2 Iris-setosa 2 4.7 3.2 1.3 0.2 Iris-setosa 3 4.6 3.1 1.5 0.2 Iris-setosa 4 5.0 3.6 1.4 0.2 Iris-setosa
Python UDF Debugging Tool
#file: plus.py
from odps.udf import annotate
@annotate('bigint,bigint->bigint')
class Plus(object):
def evaluate(self, a, b):
return a + b
$ cat plus.input 1,1 3,2 $ pyou plus.Plus < plus.input 2 5
Contributing
For a development install, clone the repository and then install from source:
git clone https://github.com/aliyun/aliyun-odps-python-sdk cd pyodps pip install -r requirements.txt -e .
If you need to modify the frontend code, you need to install nodejs/npm. To build and install your frontend code, use
python setup.py build_js python setup.py install_js
License
Licensed under the Apache License 2.0
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distributions
Hashes for pyodps-0.9.1-cp37-cp37m-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 9c584218d3735f26387e74f1ba81cc71fdea6038f5bb2be9054737fc488344e3 |
|
MD5 | d3640f76f540eb039cea5e1ab5c647fa |
|
BLAKE2b-256 | 1254834b54dec4c4e46add9c896715cd9dd13bbd6f7e9840043056d3442e820b |
Hashes for pyodps-0.9.1-cp37-cp37m-win32.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 48ebc5488c6f32d8a7d82c738872b3727ae46235edf20d8838987c8a501b5964 |
|
MD5 | 3518a1d2344ce9f6fb8cf574915b41be |
|
BLAKE2b-256 | a242e9669aa801a5d5e0d91700de7fa8e029e0ef7f35a152035520e19106e393 |
Hashes for pyodps-0.9.1-cp37-cp37m-manylinux1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 88f3c263b0bc404824b00e598881e4619fdb15366d16c705fcc8f64a02b7f0a2 |
|
MD5 | cbf16a4e46e712636549d07fcd6ff6cd |
|
BLAKE2b-256 | 558969fa9f5d06374b5ddf207145f21bcd4a60379acdbe6af66b741a4c734a97 |
Hashes for pyodps-0.9.1-cp37-cp37m-manylinux1_i686.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 9fa07479db7a0bad4f2fbde44838829d3c7707b222bded935237bec6ac56620d |
|
MD5 | fb23ca79b24fd99bf5cdb3e9dbc651c4 |
|
BLAKE2b-256 | 2fcb319b857702ebba97a2b1dcc659006de26dc15f819ce46a35f5b4a8e90b7a |
Hashes for pyodps-0.9.1-cp37-cp37m-macosx_10_13_x86_64.macosx_10_9_intel.macosx_10_9_x86_64.macosx_10_10_intel.macosx_10_10_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | c2a13e40c28a387cefa402625f73322491e1895512184dff8ec8f182e0d389da |
|
MD5 | 13eadc580b0f0ef084980d4fb3c04b33 |
|
BLAKE2b-256 | 7293d48c7f9523bb233732e0ee24316fc7a07676b58e92cc980bb26cac23c370 |
Hashes for pyodps-0.9.1-cp36-cp36m-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 1ff30aae16a2e5e37e009c844b3eb2740957fecdabeec9214aa09944d7c873e1 |
|
MD5 | 65f74940ac0eb827d9e1a9d60268f3d5 |
|
BLAKE2b-256 | fb58f3068198d358afa3f0c9697768ddd53e63c0686247f2eee811640e02b77f |
Hashes for pyodps-0.9.1-cp36-cp36m-win32.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 2c70a5a19e0a73d2b8fe045ed24a1785edd1eafaa2ecd0aedeed2e8c0e7253f4 |
|
MD5 | 0e92cd273e03622824b45bf150209bf2 |
|
BLAKE2b-256 | 9f08551ecd1085b3b08ae65fd83fd399939bc1fbc2ebe1b43c883fb7a3df9a46 |
Hashes for pyodps-0.9.1-cp36-cp36m-manylinux1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | aef240b9fc10134b830f031518a6703626823da4da20a630d53f11929a29c80a |
|
MD5 | 189af4b89e4d4f04909c3754c9b17796 |
|
BLAKE2b-256 | fb7b291f9946550a0172a8cecc3361bac15874604e60fecec69c1fcd8caaf1be |
Hashes for pyodps-0.9.1-cp36-cp36m-manylinux1_i686.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 6331d304a3f5f148b0bab6f15eb8ab013b6012dbecc0f0eba5cfc7aef0b9001c |
|
MD5 | 8fb1dbea16abb1f74cb5f477d65b89d8 |
|
BLAKE2b-256 | 961d3d6030db9077abf997d526c8d705bf4d3a8f858996ea82538e946d3ad359 |
Hashes for pyodps-0.9.1-cp36-cp36m-macosx_10_13_x86_64.macosx_10_9_intel.macosx_10_9_x86_64.macosx_10_10_intel.macosx_10_10_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 8f7c3ad9d748ef5a4b0f84ed7d1a721a474bc803845415ddf44a9c89182337aa |
|
MD5 | b949585a2b56bb64aad186506a17ac51 |
|
BLAKE2b-256 | c3990faa0c8ef0ad418d952e1a2a0d8dafaec3df14e330afafb538009cda2ad6 |
Hashes for pyodps-0.9.1-cp35-cp35m-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 10affe2dcd985b7dcb64932532391581921853932f994e8c9db89be057e900a2 |
|
MD5 | 67be1a74fb66cc08af639dae360b4d29 |
|
BLAKE2b-256 | 9abe7b83d2f9586f488c50ef4214bfb0f5f0fa3637ae41f0ed1283a91dfbb0d2 |
Hashes for pyodps-0.9.1-cp35-cp35m-win32.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 73b9050bb4df53428f17f1f9b762918b181b6183f3144d33a000217f5f39fb09 |
|
MD5 | dfb29f23ec2b881acd923c5f8c382659 |
|
BLAKE2b-256 | f0fefd9fcd34370b29def1b91610328151b2a3f0449cc78a4c631e0e42818a5b |
Hashes for pyodps-0.9.1-cp35-cp35m-manylinux1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | e1731d8405c61e2bce64b741788770635d4c494ea22eee440447c787ee00adb1 |
|
MD5 | 856f8c78f61e03e8b22fbc27efee4377 |
|
BLAKE2b-256 | 7c7f14adbf5f21268e8dbd05aec4b81a6e372cefeaf2b0e0e155732e02d832b5 |
Hashes for pyodps-0.9.1-cp35-cp35m-manylinux1_i686.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | b591b1cbbd682cfc0ce1e6e83b7fc0203dbbf6cdb9442d2a3ea918d85c9bb941 |
|
MD5 | 993a01306fdbd348e38de9ea1159d7b7 |
|
BLAKE2b-256 | e996407ab338d624ea7ce8c9c2dd489ca4831a61e75bacfc57863f8c211bfca1 |
Hashes for pyodps-0.9.1-cp35-cp35m-macosx_10_13_x86_64.macosx_10_9_intel.macosx_10_9_x86_64.macosx_10_10_intel.macosx_10_10_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 2d3de13e77e325dd093b0a7beea30141db71dbceea4a258efc3b6d6a64755a33 |
|
MD5 | c0bf7183815e23def35a6160ebef5892 |
|
BLAKE2b-256 | aec3c69213a490cf05ea5a31ddf4a85933f7ee44dbca68a7317d94214b8e56ef |
Hashes for pyodps-0.9.1-cp27-cp27mu-manylinux1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 1d0cb08d30c7f9cb96edb11868b811cc3322f0e6dfc9eab92e02b4b1c8a9e442 |
|
MD5 | 8e29c8c17655dcc11950ee06079ef6e3 |
|
BLAKE2b-256 | 9055c3cc100e5283d737182dec93b1d8f1a6721b848c1980d47d81c4bfcf03de |
Hashes for pyodps-0.9.1-cp27-cp27mu-manylinux1_i686.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | e63f84bd8080aac792c73e376e39f612a439fb1c7ccb104c9902150ddc162185 |
|
MD5 | 01cf9443b4f161e7fe445031d74e88ea |
|
BLAKE2b-256 | 07673894546c87cc33d6bc8f758bb232d173b325f0e2838a2d0abc879344a36d |
Hashes for pyodps-0.9.1-cp27-cp27m-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 32282b65800528a9dcf1f61cabbdb9a4f9efe74d3f62df017b88648513ef7f26 |
|
MD5 | 66daa91809d13765733d030ca535104e |
|
BLAKE2b-256 | afc409a03216551f9fb1a814444fa3a93fd726900395cf39d8655c52462a6a44 |
Hashes for pyodps-0.9.1-cp27-cp27m-win32.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 1971a9905603e8163acb5874083e6215e6b9db18502e0a7031570681bec3f98b |
|
MD5 | 2630335493da6774e405e539b0cfc23c |
|
BLAKE2b-256 | 6b8c0e41823f528981851d34e2ff4fa8316e0d4f830faf01f1c72e472cc29a5b |
Hashes for pyodps-0.9.1-cp27-cp27m-manylinux1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | f882c2d972255c0ebb02c97f34a4adddeee6568113739c1cf50643c55560f752 |
|
MD5 | 01a7ab124a002069d162dd41f94d9c67 |
|
BLAKE2b-256 | f9962238b98d5248e98633ed372f8a2960b5fc10501fc9db7cc3a264d6acae00 |
Hashes for pyodps-0.9.1-cp27-cp27m-manylinux1_i686.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | d05766ba1821f9901a15aca439c1eed505c925b3bea016e0f6ada1f7bb7c32eb |
|
MD5 | 898ebbd9da3bf917255e1799f3760305 |
|
BLAKE2b-256 | 4fd2118ee42bb45d4e623878a34be3433249c134f3330ecb4936f80b87307f7e |