ODPS Python SDK and data analysis framework
Project description
Elegent way to access ODPS API. Documentation
Installation
The quick way:
pip install 'pyodps[full]'
If you don’t need to use Jupyter, just type
pip install pyodps
The dependencies will be installed automatically.
Or from source code:
$ virtualenv pyodps_env
$ source pyodps_env/bin/activate
$ git clone <git clone URL> pyodps
$ cd pyodps
$ python setup.py install
Dependencies
Python (>=2.6), including Python 3+, pypy, Python 2.7 recommended
setuptools (>=3.0)
requests (>=2.4.0)
Run Unittest
copy conf/test.conf.template to odps/tests/test.conf, and fill it with your account
run python -m unittest discover
Usage
>>> from odps import ODPS
>>> o = ODPS('**your-access-id**', '**your-secret-access-key**',
... project='**your-project**', endpoint='**your-end-point**')
>>> dual = o.get_table('dual')
>>> dual.name
'dual'
>>> dual.schema
odps.Schema {
c_int_a bigint
c_int_b bigint
c_double_a double
c_double_b double
c_string_a string
c_string_b string
c_bool_a boolean
c_bool_b boolean
c_datetime_a datetime
c_datetime_b datetime
}
>>> dual.creation_time
datetime.datetime(2014, 6, 6, 13, 28, 24)
>>> dual.is_virtual_view
False
>>> dual.size
448
>>> dual.schema.columns
[<column c_int_a, type bigint>,
<column c_int_b, type bigint>,
<column c_double_a, type double>,
<column c_double_b, type double>,
<column c_string_a, type string>,
<column c_string_b, type string>,
<column c_bool_a, type boolean>,
<column c_bool_b, type boolean>,
<column c_datetime_a, type datetime>,
<column c_datetime_b, type datetime>]
DataFrame API
>>> from odps.df import DataFrame
>>> df = DataFrame(o.get_table('pyodps_iris'))
>>> df.dtypes
odps.Schema {
sepallength float64
sepalwidth float64
petallength float64
petalwidth float64
name string
}
>>> df.head(5)
|==========================================| 1 / 1 (100.00%) 0s
sepallength sepalwidth petallength petalwidth name
0 5.1 3.5 1.4 0.2 Iris-setosa
1 4.9 3.0 1.4 0.2 Iris-setosa
2 4.7 3.2 1.3 0.2 Iris-setosa
3 4.6 3.1 1.5 0.2 Iris-setosa
4 5.0 3.6 1.4 0.2 Iris-setosa
>>> df[df.sepalwidth > 3]['name', 'sepalwidth'].head(5)
|==========================================| 1 / 1 (100.00%) 12s
name sepalwidth
0 Iris-setosa 3.5
1 Iris-setosa 3.2
2 Iris-setosa 3.1
3 Iris-setosa 3.6
4 Iris-setosa 3.9
Command-line and IPython enhancement
In [1]: %load_ext odps In [2]: %enter Out[2]: <odps.inter.Room at 0x10fe0e450> In [3]: %sql select * from pyodps_iris limit 5 |==========================================| 1 / 1 (100.00%) 2s Out[3]: sepallength sepalwidth petallength petalwidth name 0 5.1 3.5 1.4 0.2 Iris-setosa 1 4.9 3.0 1.4 0.2 Iris-setosa 2 4.7 3.2 1.3 0.2 Iris-setosa 3 4.6 3.1 1.5 0.2 Iris-setosa 4 5.0 3.6 1.4 0.2 Iris-setosa
Python UDF Debugging Tool
#file: plus.py
from odps.udf import annotate
@annotate('bigint,bigint->bigint')
class Plus(object):
def evaluate(self, a, b):
return a + b
$ cat plus.input 1,1 3,2 $ pyou plus.Plus < plus.input 2 5
Contributing
For a development install, clone the repository and then install from source:
git clone https://github.com/aliyun/aliyun-odps-python-sdk cd pyodps pip install -r requirements.txt -e .
If you need to modify the frontend code, you need to install nodejs/npm. To build and install your frontend code, use
python setup.py build_js python setup.py install_js
License
Licensed under the Apache License 2.0
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distributions
Hashes for pyodps-0.7.20-cp37-cp37m-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | dcda4c315fa1a7ca337d243303762e6095392d0ddd60f5340dee0261d005cb13 |
|
MD5 | 6a85592bf5152f850c43d4a4a39dac8c |
|
BLAKE2b-256 | 30ad3831f547efa694ab22c1e5b7481a4b45372a8bbb7e7161cea535d82dab5d |
Hashes for pyodps-0.7.20-cp37-cp37m-win32.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 92f3bbcd0b3105b19c15f6b302bf36be735a0dd711d0b92a738d2bb68fd1f066 |
|
MD5 | f1763ca916001e3f6c1836f269020851 |
|
BLAKE2b-256 | d045befea0eb663c142327b5cb198aee41502922c3167c57b28c51cea8357b15 |
Hashes for pyodps-0.7.20-cp37-cp37m-manylinux1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 9b2a760addc66fb1165cfc6f35aa2cd9d667a5bb51fa5cc3835a2602476a9363 |
|
MD5 | 2cdbb379e64902245f3094d6878b9673 |
|
BLAKE2b-256 | 82a8774ffd65487779f5cc8b9c21d681a1587f627486981b606fe55ad879ebd1 |
Hashes for pyodps-0.7.20-cp37-cp37m-manylinux1_i686.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 8e1684d53d6bda3dd2949fab4194562b7dc5c2db73f2745ffe6cbff8700346b0 |
|
MD5 | 5325cb4ca08b36a202bb018818125589 |
|
BLAKE2b-256 | e22a76ecd680da7be63d72a03ca58061a9b6f0f3071f539609ad60951c550fab |
Hashes for pyodps-0.7.20-cp37-cp37m-macosx_10_13_x86_64.macosx_10_9_intel.macosx_10_9_x86_64.macosx_10_10_intel.macosx_10_10_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 629f5b56f8a78896a689b094eda977daa20438c04e15a11ffd453022bb2db0e2 |
|
MD5 | 72a3ac7e0210d6e95b10cf286532fa36 |
|
BLAKE2b-256 | 2b6c7912bff86a50c93580543b369c5235b63c3375b6ef670c149d8e484b606d |
Hashes for pyodps-0.7.20-cp36-cp36m-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 64c052fa279e4c1ac45482bd6ab28cf8291cd389ee367762e3d82329750e9f55 |
|
MD5 | a0c26b8645d716c573ff9e1fcb6512a0 |
|
BLAKE2b-256 | 952c9efe4b7daf57a1cab1920f72f57e51d1fcb4f7f797ef9460d1c731ce7013 |
Hashes for pyodps-0.7.20-cp36-cp36m-win32.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | e68e2698fce6e3a6f464d4564f7ae2a0da007bb1619e186c858815f7b1b7a9d1 |
|
MD5 | 04018fd4fcfa34006a6b69ff75447322 |
|
BLAKE2b-256 | c717e21c4edffd5d38dd24e1e597cf9004c9094ad7a125f4a9fdd7c66b29267f |
Hashes for pyodps-0.7.20-cp36-cp36m-manylinux1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 3de515946fd6a5d83ccb84294736f17d43aff6988260bfd436b287063acbaaff |
|
MD5 | 5d95c94d5ee70a083ab22d05cafa21e5 |
|
BLAKE2b-256 | 7b5187369b217dbc88bbc2c4a2eb8361f099e25a93329854d1f5499664c13e7a |
Hashes for pyodps-0.7.20-cp36-cp36m-manylinux1_i686.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 4fae3145309f81e9655f7b50fa4e944287cd59968db665d18dc7caaf20115d97 |
|
MD5 | 33b325dab8f5034d13d2950d3effc8c3 |
|
BLAKE2b-256 | 571433d81cd2622136ec822b79ba038f67ce325fe4fba565ac95fd89e16081c4 |
Hashes for pyodps-0.7.20-cp36-cp36m-macosx_10_13_x86_64.macosx_10_9_intel.macosx_10_9_x86_64.macosx_10_10_intel.macosx_10_10_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | ff5d71a5358a3fdbd9606bc375449a1106b347ae5ba26dbaf4feb5806ef01ac4 |
|
MD5 | 126a121e286ddabb78b84750b549797a |
|
BLAKE2b-256 | 9a14fecdb6868879ded672304ce170cab62000e265d46db967221a4899ad64dc |
Hashes for pyodps-0.7.20-cp35-cp35m-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 06e55b41fa9f14de6a922717a3b3482f25dc83d87776caecbfa1e036e41a8f40 |
|
MD5 | 7c65331d38ddaefa14b0be787d32d66f |
|
BLAKE2b-256 | c735797023837966c7d600a639dcfbf658701ffaf9992183a2343615e4772b29 |
Hashes for pyodps-0.7.20-cp35-cp35m-win32.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 52b42bf3b282c7d462027f39a95eb878d090d9ff57ca0713155b7f1107c5f90f |
|
MD5 | 7293c37450324938f69419bec22aaf99 |
|
BLAKE2b-256 | 0c46647a50ae1d3a96c24cb19783164f75f782aca42c062c104e5d9bde5b9a6d |
Hashes for pyodps-0.7.20-cp35-cp35m-manylinux1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | e861f261a37231e1ff0a61b2c83e8ea87082dba3c0cd94db5a4550e9520378c5 |
|
MD5 | 218b9b9370101427016dab8863bda082 |
|
BLAKE2b-256 | e6641ccede998c9bf80db8255375193c767af5cbeaa5d1480168ef44b479c4bb |
Hashes for pyodps-0.7.20-cp35-cp35m-manylinux1_i686.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 0cf48267c0fdc229db451dac24fb8440f18c926566de9695333421eba6c849a2 |
|
MD5 | fc69852bcacbf4c32eafb92b8ba46052 |
|
BLAKE2b-256 | f48146240296a1f6313fbbc30b5c251538995d8bed258bf6d70a9f7bf05a136d |
Hashes for pyodps-0.7.20-cp35-cp35m-macosx_10_13_x86_64.macosx_10_9_intel.macosx_10_9_x86_64.macosx_10_10_intel.macosx_10_10_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 61173e17ccd8b0f30229ed1aceb1e5ac132953ea590d88a1a24471886ef56662 |
|
MD5 | d849f30bf8ede2a839018e27349dc2d3 |
|
BLAKE2b-256 | 317f3c04ead83386a058a394c36b91afac293a5a806c77ad77464cd2c1b19d56 |
Hashes for pyodps-0.7.20-cp27-cp27mu-manylinux1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | be29f4c528db9a62575c3b9dd8a4a75691a349b5f860eb9793214ed1fb7b3562 |
|
MD5 | c17da997bc285c3f6e40524e3636a784 |
|
BLAKE2b-256 | bdb091ad4cfec5a4775bcdc97e176ffa6601ddb3ff88d55418e96da3321f7a1f |
Hashes for pyodps-0.7.20-cp27-cp27mu-manylinux1_i686.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 2a802f52d6a4cb5c5ff3f835654c32e4860ffa7c21449aa8cea639ed9647e96f |
|
MD5 | d82184a413bca8ad4e7b6dd50de606c1 |
|
BLAKE2b-256 | f9723128de285c6bf5e027e7fd3921eff6dbc572b7042ea4953f2b53656e3841 |
Hashes for pyodps-0.7.20-cp27-cp27m-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 62c3a807b6963a8c2eaed514b2c42e1b11512e0e19a8acfb90adbcdf132937db |
|
MD5 | 35ec708ac198fd6887f44acf7fa2b07f |
|
BLAKE2b-256 | 3b1b99a7abf707a3a9ba980013ecb13e1bb5f70b56bc4f32da8354a924b56a58 |
Hashes for pyodps-0.7.20-cp27-cp27m-win32.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 3f1d7beb4f53564ad8267c0126e922e19f668e9e8bafcddb81970eb0f1058729 |
|
MD5 | 9849e434ecb92dc970a835da29220492 |
|
BLAKE2b-256 | 8cc60069b288dcc9e83d1d548a38ba373e8d5a586245824ea06feeca1bb8709b |
Hashes for pyodps-0.7.20-cp27-cp27m-manylinux1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 8f9e079f2fbefb37577718c2172b46ee2db51a40b3ceb3ac6f03f1d7790f6776 |
|
MD5 | 9e937e44cf827ab31f341158c953ccca |
|
BLAKE2b-256 | 49ff308c44909d9179b2fbee5eda3feef301519919e5c98d4a2d510c2d174530 |
Hashes for pyodps-0.7.20-cp27-cp27m-manylinux1_i686.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 96b2d08608b40bc0565fee717dc948fb287c6af10f2895f0e7c4ebfd8c8d9944 |
|
MD5 | 06dfdd952f68450956f85f74a90b7c23 |
|
BLAKE2b-256 | ddc60a8e155696d3792d7e7feea0c0d298dd5724cf54a0f6f5c79325d07e09f9 |
Hashes for pyodps-0.7.20-cp27-cp27m-macosx_10_13_x86_64.macosx_10_9_intel.macosx_10_9_x86_64.macosx_10_10_intel.macosx_10_10_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | dad94797ecf7dbde44f39b77496cedb0beafaac3f3b202ab0b8a13d4644ce059 |
|
MD5 | 76687edba99d3573ad016b250b768a27 |
|
BLAKE2b-256 | 95bfb166b79235847182c7ee266612d7606eab88e20ba6d29fe9a51f5d2d0984 |