ODPS Python SDK and data analysis framework
Project description
Elegent way to access ODPS API. Documentation
Installation
The quick way:
pip install pyodps[full]
If you don’t need to use Jupyter, just type
pip install pyodps
The dependencies will be installed automatically.
Or from source code:
$ virtualenv pyodps_env
$ source pyodps_env/bin/activate
$ git clone <git clone URL> pyodps
$ cd pyodps
$ python setup.py install
Dependencies
Python (>=2.6), including Python 3+, pypy, Python 2.7 recommended
setuptools (>=3.0)
requests (>=2.4.0)
Run Unittest
copy conf/test.conf.template to odps/tests/test.conf, and fill it with your account
run python -m unittest discover
Usage
>>> from odps import ODPS
>>> o = ODPS('**your-access-id**', '**your-secret-access-key**',
... project='**your-project**', endpoint='**your-end-point**')
>>> dual = o.get_table('dual')
>>> dual.name
'dual'
>>> dual.schema
odps.Schema {
c_int_a bigint
c_int_b bigint
c_double_a double
c_double_b double
c_string_a string
c_string_b string
c_bool_a boolean
c_bool_b boolean
c_datetime_a datetime
c_datetime_b datetime
}
>>> dual.creation_time
datetime.datetime(2014, 6, 6, 13, 28, 24)
>>> dual.is_virtual_view
False
>>> dual.size
448
>>> dual.schema.columns
[<column c_int_a, type bigint>,
<column c_int_b, type bigint>,
<column c_double_a, type double>,
<column c_double_b, type double>,
<column c_string_a, type string>,
<column c_string_b, type string>,
<column c_bool_a, type boolean>,
<column c_bool_b, type boolean>,
<column c_datetime_a, type datetime>,
<column c_datetime_b, type datetime>]
DataFrame API
>>> from odps.df import DataFrame
>>> df = DataFrame(o.get_table('pyodps_iris'))
>>> df.dtypes
odps.Schema {
sepallength float64
sepalwidth float64
petallength float64
petalwidth float64
name string
}
>>> df.head(5)
|==========================================| 1 / 1 (100.00%) 0s
sepallength sepalwidth petallength petalwidth name
0 5.1 3.5 1.4 0.2 Iris-setosa
1 4.9 3.0 1.4 0.2 Iris-setosa
2 4.7 3.2 1.3 0.2 Iris-setosa
3 4.6 3.1 1.5 0.2 Iris-setosa
4 5.0 3.6 1.4 0.2 Iris-setosa
>>> df[df.sepalwidth > 3]['name', 'sepalwidth'].head(5)
|==========================================| 1 / 1 (100.00%) 12s
name sepalwidth
0 Iris-setosa 3.5
1 Iris-setosa 3.2
2 Iris-setosa 3.1
3 Iris-setosa 3.6
4 Iris-setosa 3.9
Command-line and IPython enhancement
In [1]: %load_ext odps In [2]: %enter Out[2]: <odps.inter.Room at 0x10fe0e450> In [3]: %sql select * from pyodps_iris limit 5 |==========================================| 1 / 1 (100.00%) 2s Out[3]: sepallength sepalwidth petallength petalwidth name 0 5.1 3.5 1.4 0.2 Iris-setosa 1 4.9 3.0 1.4 0.2 Iris-setosa 2 4.7 3.2 1.3 0.2 Iris-setosa 3 4.6 3.1 1.5 0.2 Iris-setosa 4 5.0 3.6 1.4 0.2 Iris-setosa
Python UDF Debugging Tool
#file: plus.py
from odps.udf import annotate
@annotate('bigint,bigint->bigint')
class Plus(object):
def evaluate(self, a, b):
return a + b
$ cat plus.input 1,1 3,2 $ pyou plus.Plus < plus.input 2 5
Contributing
For a development install, clone the repository and then install from source:
git clone https://github.com/aliyun/aliyun-odps-python-sdk cd pyodps pip install -r requirements.txt -e .
If you need to modify the frontend code, you need to install nodejs/npm. To build and install your frontend code, use
python setup.py build_js python setup.py install_js
License
Licensed under the Apache License 2.0
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distributions
Hashes for pyodps-0.9.5-cp37-cp37m-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 99842d3e19aa2c6eec62cf2855f49aa97a9c607c2c94eea70bd8c19fd5f7d0b6 |
|
MD5 | 104e04ffbb369a9cab2609c47afa22d1 |
|
BLAKE2b-256 | e9fdd45f877bed6751d80b0d6e2b6cc3d236378bdb1267d9c6020e8ece95b12a |
Hashes for pyodps-0.9.5-cp37-cp37m-win32.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 764064c48358368c054a8e4647282e332213bae7c023034dab1dbb9e368bbd76 |
|
MD5 | 7b4ff8725f4dd528f3e6717bf1b7031c |
|
BLAKE2b-256 | ed743d4b2735f950015f170a91fa02b64f8cfdb05b62aa2bae42b658e5bb5b8f |
Hashes for pyodps-0.9.5-cp37-cp37m-manylinux1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | b7a8dc5bc53bfc49c1fa4f6425d141fb098e1e1930c34e20da59a976d2353f20 |
|
MD5 | 865eaf60c538653b11ceb96fa35ba797 |
|
BLAKE2b-256 | ce3e6112259a3d3ae4bfca6eec6f6ca2a279be8d49b68446f23050c305c35829 |
Hashes for pyodps-0.9.5-cp37-cp37m-manylinux1_i686.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 57ca9a167dc2f6aea07db9d135c2e964394e36c1fee8c0ae462fa4cffa6a256a |
|
MD5 | 7b19d8dc74df38bb7aec831e85029f72 |
|
BLAKE2b-256 | 6305f70c2c7ec6d3b13b955389c7dcfa4faa305498f2d65d96e3e34588368d94 |
Hashes for pyodps-0.9.5-cp37-cp37m-macosx_10_13_x86_64.macosx_10_9_intel.macosx_10_9_x86_64.macosx_10_10_intel.macosx_10_10_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 1f349d6d42258ff4b87630d287e5ac4fade7c8da7e62aec233faa6960792828f |
|
MD5 | cc72e7273bc12272f0d351f423907043 |
|
BLAKE2b-256 | 747cddd4ee59040357784047d18d63dc4f31db4a8bb87a329bb503b5ec6979d5 |
Hashes for pyodps-0.9.5-cp36-cp36m-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 7ff95b9f655521f8b4918418eb6b8fcf9a3cd3d39a1d684de1fbe126adf24baa |
|
MD5 | 58f84ef522143e785767112a1c75df15 |
|
BLAKE2b-256 | 4bcc64626bbdcd895ab7f5b481f605302bf64f8fe2684326ea1da6b57ab8cd9c |
Hashes for pyodps-0.9.5-cp36-cp36m-win32.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 976baca87de592ec4653e528a164e5afb109ca0c88ad9c2bcf4101e87b20e922 |
|
MD5 | 7e8990b59a6e741c7e9208021b11c7ff |
|
BLAKE2b-256 | 3124cda345444d823721875dbf24986a374893fe4357d5f32077344a3dbdcbe8 |
Hashes for pyodps-0.9.5-cp36-cp36m-manylinux1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | d3678ca3df0d88c4c03aed6cdf17d68586a7c8e5e9f6b18bc091be7e8164e7e3 |
|
MD5 | 7e484e6a8612d7672d613f6b4fab4c0a |
|
BLAKE2b-256 | d814da27f2982d14b177458f45597d29b3aa08ba13b976ecb9b6b8ecee938ad0 |
Hashes for pyodps-0.9.5-cp36-cp36m-manylinux1_i686.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | d7a6fef10ba1d31fbc3546d93274ac22039f735191006ab5b0dab8d3ce09f2d7 |
|
MD5 | 712d185bc12bc68942aa59df7271cf48 |
|
BLAKE2b-256 | bac2f946ea667080441af34b0455522505baeee37bf6251ea2d37b2c4af226fb |
Hashes for pyodps-0.9.5-cp36-cp36m-macosx_10_13_x86_64.macosx_10_9_intel.macosx_10_9_x86_64.macosx_10_10_intel.macosx_10_10_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | e156ecace4191d059dbd09bf40c9ecee049c4bb4b64d3331cdf599f51f42df05 |
|
MD5 | 684f20843efc9e76afc7c276fc48675f |
|
BLAKE2b-256 | 412ccbd3e50a35dcccf68255aeef365258b86df2656d0286f63678137b138383 |
Hashes for pyodps-0.9.5-cp35-cp35m-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | d52634a6f061b2aa2c2e222f7888dffe30493c9c60d22f1fe3e1655bf03b8e41 |
|
MD5 | e076418d43c0482c2b8d54c637788687 |
|
BLAKE2b-256 | deed2323b3f18b1da5d33d30b8d710be2c92bc51c744a04a695ae85d7ce76438 |
Hashes for pyodps-0.9.5-cp35-cp35m-win32.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 8eb99c6e4077db34e0e1d40c121b5efd3dba316abf7373db9ddb4bd985d34a36 |
|
MD5 | 597e0b76462f123c04e96c0ed9cffc56 |
|
BLAKE2b-256 | ab9af693e2dbefa2c857585c84995b0aac4d87bc384be7c360971ec9cf7c7358 |
Hashes for pyodps-0.9.5-cp35-cp35m-manylinux1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | fa540bd8ddd8ae4ff991ee4544cb1f6c8f2c1669c6ed20d3cf677f4dcda3fa3a |
|
MD5 | e3268f4bcd5c9401ccf9a280a3582879 |
|
BLAKE2b-256 | 684263c34f4165048b0fcdd9659bf6a957f8768846288a217d1a6eebcd5aa19b |
Hashes for pyodps-0.9.5-cp35-cp35m-manylinux1_i686.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 7e1f0b72f27f2f6134684e22c5856e281142147384ab90a4edf5650792633c23 |
|
MD5 | e30e074aec27c597d8d7625ec3d3803e |
|
BLAKE2b-256 | afaafea651dba28d963d59ea47dd4d683e1df0945c9d96295d4dd8ea8b784464 |
Hashes for pyodps-0.9.5-cp35-cp35m-macosx_10_13_x86_64.macosx_10_9_intel.macosx_10_9_x86_64.macosx_10_10_intel.macosx_10_10_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 8db5cd7f2da333a8de6e56091e45bf06f1dbdcb6ce0e1a494fbb893fce277590 |
|
MD5 | aec16aeeeecace1082f36dde9fae8c13 |
|
BLAKE2b-256 | 0132ea9cb45d60d9ac44e78920148956e17682ae6e21531a9b82201fa0fe498a |
Hashes for pyodps-0.9.5-cp27-cp27mu-manylinux1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 1520fc28f0edfa258710aa5b7374bbb959994ff06190779a79c7bfa0fc515024 |
|
MD5 | df0221ee998fac434cd7195a43464504 |
|
BLAKE2b-256 | 365aa3ba672d17c9c9f8ee9fdf841c2c59fc30a931c687d9ab68a50155821eb8 |
Hashes for pyodps-0.9.5-cp27-cp27mu-manylinux1_i686.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | c629532ed7c76a800cbb0b46de2d3a337b0cb1c8b9ba1d45195e786f062062e7 |
|
MD5 | 28713894ca7312228e138e01be66ab3c |
|
BLAKE2b-256 | e37a21ff20d660355feca135fd84597246c4bbe2542ccc1c1dc3ff56137210f4 |
Hashes for pyodps-0.9.5-cp27-cp27m-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | d0ba5da07524127b7050b8ad56fbab94c4f4886b89862d0810d5cb6f3b8f7a3b |
|
MD5 | 153f142ef63d80719fb67b455f939cdf |
|
BLAKE2b-256 | 0d19020267a2553f44643b1b5cac31b175fd3079503c8242443822828fefbc68 |
Hashes for pyodps-0.9.5-cp27-cp27m-win32.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 5e640d3a012acfba6860f54870e5a57a143865963fca3f8fe75af462a50ea6a1 |
|
MD5 | 702c5501d3bc14095ea2ad18daf87c28 |
|
BLAKE2b-256 | e0fc953fb2dcd075fc6cf2c92c5fc9f03a60d3c74868b0bcffaa6a8e2d97f310 |
Hashes for pyodps-0.9.5-cp27-cp27m-manylinux1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 7e27b1c8b73b1a199019a3726f93be1001f9a72f642844b33ad08c6796d10f4e |
|
MD5 | d5ce43c89926fab8a95d24900e3bf017 |
|
BLAKE2b-256 | baec1b88a83aba22055a9daa3b23194bbc8634afa3f496aec9d4a3cef6b15aff |
Hashes for pyodps-0.9.5-cp27-cp27m-manylinux1_i686.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | e0844fc8943c41249b1fca0211e7b84034134b2a25928a4ed07ff44995f5f7a8 |
|
MD5 | 77303056e9f5b5522bee3b215549dd05 |
|
BLAKE2b-256 | ab2e5ce470cd7a7c047a664ca96352ef36999702e5aec7043312bcf5f0117a72 |