ODPS Python SDK and data analysis framework
Project description
Elegent way to access ODPS API. Documentation
Installation
The quick way:
pip install pyodps[full]
To integrate with Mars:
pip install pyodps#egg=pyodps[mars]
If you don’t need to use Jupyter, just type
pip install pyodps
The dependencies will be installed automatically.
Or from source code:
$ virtualenv pyodps_env
$ source pyodps_env/bin/activate
$ git clone <git clone URL> pyodps
$ cd pyodps
$ python setup.py install
Dependencies
Python (>=2.6), including Python 3+, pypy, Python 2.7 recommended
setuptools (>=3.0)
requests (>=2.4.0)
Run Unittest
copy conf/test.conf.template to odps/tests/test.conf, and fill it with your account
run python -m unittest discover
Usage
>>> from odps import ODPS
>>> o = ODPS('**your-access-id**', '**your-secret-access-key**',
... project='**your-project**', endpoint='**your-end-point**')
>>> dual = o.get_table('dual')
>>> dual.name
'dual'
>>> dual.schema
odps.Schema {
c_int_a bigint
c_int_b bigint
c_double_a double
c_double_b double
c_string_a string
c_string_b string
c_bool_a boolean
c_bool_b boolean
c_datetime_a datetime
c_datetime_b datetime
}
>>> dual.creation_time
datetime.datetime(2014, 6, 6, 13, 28, 24)
>>> dual.is_virtual_view
False
>>> dual.size
448
>>> dual.schema.columns
[<column c_int_a, type bigint>,
<column c_int_b, type bigint>,
<column c_double_a, type double>,
<column c_double_b, type double>,
<column c_string_a, type string>,
<column c_string_b, type string>,
<column c_bool_a, type boolean>,
<column c_bool_b, type boolean>,
<column c_datetime_a, type datetime>,
<column c_datetime_b, type datetime>]
DataFrame API
>>> from odps.df import DataFrame
>>> df = DataFrame(o.get_table('pyodps_iris'))
>>> df.dtypes
odps.Schema {
sepallength float64
sepalwidth float64
petallength float64
petalwidth float64
name string
}
>>> df.head(5)
|==========================================| 1 / 1 (100.00%) 0s
sepallength sepalwidth petallength petalwidth name
0 5.1 3.5 1.4 0.2 Iris-setosa
1 4.9 3.0 1.4 0.2 Iris-setosa
2 4.7 3.2 1.3 0.2 Iris-setosa
3 4.6 3.1 1.5 0.2 Iris-setosa
4 5.0 3.6 1.4 0.2 Iris-setosa
>>> df[df.sepalwidth > 3]['name', 'sepalwidth'].head(5)
|==========================================| 1 / 1 (100.00%) 12s
name sepalwidth
0 Iris-setosa 3.5
1 Iris-setosa 3.2
2 Iris-setosa 3.1
3 Iris-setosa 3.6
4 Iris-setosa 3.9
Command-line and IPython enhancement
In [1]: %load_ext odps In [2]: %enter Out[2]: <odps.inter.Room at 0x10fe0e450> In [3]: %sql select * from pyodps_iris limit 5 |==========================================| 1 / 1 (100.00%) 2s Out[3]: sepallength sepalwidth petallength petalwidth name 0 5.1 3.5 1.4 0.2 Iris-setosa 1 4.9 3.0 1.4 0.2 Iris-setosa 2 4.7 3.2 1.3 0.2 Iris-setosa 3 4.6 3.1 1.5 0.2 Iris-setosa 4 5.0 3.6 1.4 0.2 Iris-setosa
Python UDF Debugging Tool
#file: plus.py
from odps.udf import annotate
@annotate('bigint,bigint->bigint')
class Plus(object):
def evaluate(self, a, b):
return a + b
$ cat plus.input 1,1 3,2 $ pyou plus.Plus < plus.input 2 5
Contributing
For a development install, clone the repository and then install from source:
git clone https://github.com/aliyun/aliyun-odps-python-sdk cd pyodps pip install -r requirements.txt -e .
If you need to modify the frontend code, you need to install nodejs/npm. To build and install your frontend code, use
python setup.py build_js python setup.py install_js
License
Licensed under the Apache License 2.0
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distributions
Hashes for pyodps-0.11.2.2-cp310-cp310-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | ec23d5b5f46efbed23b3ca9d5e6223555743f3edaf5d2a70c2a167fc6c9a3415 |
|
MD5 | 0a10eb062587e71c49aceda64612d00d |
|
BLAKE2b-256 | d157aa30c3b8031197fd6bd343e3ed28b8571b313fac6258f6e20f04b4572bb2 |
Hashes for pyodps-0.11.2.2-cp310-cp310-win32.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 577ff0025fd7c7e10af924f91d162fd3178d5e9755fe036ca1d22175cf915011 |
|
MD5 | 2eec848db1999c7edaccf2946bf98c57 |
|
BLAKE2b-256 | 02ce5eeadb2b5758cd779745e7949c4ca825cc8f5a03de19600b74d3b873245b |
Hashes for pyodps-0.11.2.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | a80fb5bbc184ed439c80c6ca4029c7e19395c43c0cb4576b6b2a151d38479801 |
|
MD5 | f93a107bb12f186c83e91288a8fbf0c3 |
|
BLAKE2b-256 | 24e58cab6a3d465cea93b658210dc8417066c6a4b6dca75cf995a78204e9f0c2 |
Hashes for pyodps-0.11.2.2-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 44650daa6383d1fe966cf3ad9e8ffba25697d67cb857c4ebbfefeffb600eca12 |
|
MD5 | 8e6b79719650ee8d5534d787655c9f63 |
|
BLAKE2b-256 | 88edcc3d2d7d5297cce3b3df40765d51f33fadc082886348dedb65b36f2558dc |
Hashes for pyodps-0.11.2.2-cp310-cp310-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | f7a9e90394ec9a4c7a809ea7603a23afd7b641de6769f73d0c11e8606b552895 |
|
MD5 | e36a38b1b9a747b830eb72f75fa1a5ff |
|
BLAKE2b-256 | acd311c690649d3807b5b8f8ab3c007f9f7c2f5f5a848fad517c229522244eff |
Hashes for pyodps-0.11.2.2-cp310-cp310-macosx_10_9_universal2.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 996d384f90fc013cbe0280fe593098b08b6de2639b64d47b8c4aefdaaca6b3a7 |
|
MD5 | 6311aaf8f565345748916e882fbd3d50 |
|
BLAKE2b-256 | 935a3a87a9dfc7167a022b3367f09cb2145183521e172c8920cd02fe744e205e |
Hashes for pyodps-0.11.2.2-cp39-cp39-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | b19bb6052b0c38b479e7a388ca295e601dad42c226816d531b225c16860e9ce7 |
|
MD5 | 0d90f3d0a07cf81584ad05a225a23abc |
|
BLAKE2b-256 | e776113d738b102440c14dcc3fc271433a67efc5a8f913314b1dbad1db87e66e |
Hashes for pyodps-0.11.2.2-cp39-cp39-win32.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | a64350075f181e86eb01d8a7e04b940416b16f9c6b778b3c1861c70c0f5a0c55 |
|
MD5 | ed6a65d6ae06329ea47ec8a527449479 |
|
BLAKE2b-256 | b8e29a236a18e1b950a3d8959e36f8ab712d11a6124579f800c151d392ca8eed |
Hashes for pyodps-0.11.2.2-cp39-cp39-manylinux2014_aarch64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | d66e8941640a5d0e8cc1d920c30742878c269412f9fa56ab27676cb4969ba927 |
|
MD5 | 776ce8d2861e823f7bfccf97dcd9cbd2 |
|
BLAKE2b-256 | ef6cd09a14c75e0fd7a781c04a08a981c3c7a4d7e92f512bdbe28f81cc9662a9 |
Hashes for pyodps-0.11.2.2-cp39-cp39-manylinux1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 6655a3bfbcd63e888e5ac483480f52e77c00601ae4580aa7d2dd0c37ee5d2828 |
|
MD5 | 28549dba27ffecd59e69f748458cbdc2 |
|
BLAKE2b-256 | 7e521c87d5c5430cb03652e2030406e9405be08e92af57cb965e9f02a8792d32 |
Hashes for pyodps-0.11.2.2-cp39-cp39-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 991acd36129993b356fab70f8718cf63a0980bb9616287a1844f022265028826 |
|
MD5 | 4e2cdcdb96a185b6378c0c2687a57cda |
|
BLAKE2b-256 | 8090f8e14e8e345dadb9eded0414f2e2897f9aec1dda6174cfe883df2d4616f2 |
Hashes for pyodps-0.11.2.2-cp39-cp39-macosx_10_9_universal2.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 6c0620cf6ee15ecd613d5224b9efeeae9f7fe6d51ceb278b1dfdc610235b8dea |
|
MD5 | 904075fc52e3eb06afef825a3536b282 |
|
BLAKE2b-256 | 644c537d63045c96e42a200c2c5e11325f7bef1d1ca2ba81bcc8fc31d15c97fc |
Hashes for pyodps-0.11.2.2-cp38-cp38-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 5fc3c930f03d288bb333edd2708a6e807b46508f9feb640493677faed07f4094 |
|
MD5 | a7c2f1d864a6733b8765dafe1fa56273 |
|
BLAKE2b-256 | 779cce685869b86a985ff157e14b5d4bf756f14c5aac6ff0afb0c79bc0f5bd1e |
Hashes for pyodps-0.11.2.2-cp38-cp38-win32.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 817b8ac4f7532b510781f54b3c1bbafa285496ab73b80cb5b1d5b70f983b6be1 |
|
MD5 | cc7c230c9a11731d49c840f2a0fb00d2 |
|
BLAKE2b-256 | c922e0b41d04795314825e9aeef9e42837150c17323320f4b1137db683e6408e |
Hashes for pyodps-0.11.2.2-cp38-cp38-manylinux2014_aarch64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 776760ed0959a121ab612657c30471d3ef4d0637d22f9c1df8da3c8edfffacc4 |
|
MD5 | 95305afa43b5060fcd7843f0fcf95262 |
|
BLAKE2b-256 | bffe501a156f4090f2ba144fb9e8ac006348b835e0f71d460d61459bbf5910f6 |
Hashes for pyodps-0.11.2.2-cp38-cp38-manylinux1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 1ab7218139aea8eefee77ebfe6908c633fcd38e80976e35d4d803c467bf85269 |
|
MD5 | efb9bf8a5a712e65185e849ed9dba26b |
|
BLAKE2b-256 | 894db6b28173cb30730ff8e29f235ce17e34848507c6255107368c054eda47f8 |
Hashes for pyodps-0.11.2.2-cp38-cp38-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 472ae7117f962809022edc3c0ea90f98bceb49b869b5a95ee2370638fa5c2b94 |
|
MD5 | a3509645c706d1a7900164fc870858a8 |
|
BLAKE2b-256 | 5daa23b7e78904b7bfab6871090bfbc133e7a22ec926cb912d4500ae04df9b06 |
Hashes for pyodps-0.11.2.2-cp37-cp37m-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 677375f910df4ffa3bff6ca40fd80c89f28329267fd1a8da5abdaf7be5596019 |
|
MD5 | a9d484201f7da3a095b4b3da0a30b8f2 |
|
BLAKE2b-256 | 73e107332c783762ee6b963b03ede2021afb3ee7e971d70ba39acb6faec95c9e |
Hashes for pyodps-0.11.2.2-cp37-cp37m-win32.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 66116a524335b8064e598a43b1b2c4f40eb950cbf7a4c5ad2e5b335ea9f8fcee |
|
MD5 | 2d66ac43fde346b2eb6e1367056213d8 |
|
BLAKE2b-256 | 0be38510b12df889fed0b9971f9bc81cedc555eaae4da4b59f48377753a0b404 |
Hashes for pyodps-0.11.2.2-cp37-cp37m-manylinux2014_aarch64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 1d974bf35b4068d64bf36fafe4e5475739fb584e8cace0a1734c4456fd25f591 |
|
MD5 | 31782e382c4483ebf1db935ef122f234 |
|
BLAKE2b-256 | e6bb48f3d60c88cec12819af2f6258d94b835567a65ba701a2f21a760c323f1b |
Hashes for pyodps-0.11.2.2-cp37-cp37m-manylinux1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 55b52b3f45fc95e973a86aca623ee878df3bda84eb61dfcaf1f724629a577736 |
|
MD5 | fc575d15e9aff5aff490f534e8252ca2 |
|
BLAKE2b-256 | 6b12b1c329df8a83a11ef7187503ec9443b25c55313973649c6bedff64f7b064 |
Hashes for pyodps-0.11.2.2-cp37-cp37m-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 75e522fc771f7efbb4ccc4faba423c09131f31148f612913f165eecee4a3bc5d |
|
MD5 | fea97c7993fec9a23c84d479479b3548 |
|
BLAKE2b-256 | d9ee7843cfa81bf9dc9955ad7327453594a2df860c069bf4699e07c421b697d8 |
Hashes for pyodps-0.11.2.2-cp36-cp36m-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | c371cfb91dc174e8232d079520d55bfd8fbc4ea0a392c3e600f70de7598df5d4 |
|
MD5 | 976f39930fe1cd12c0b0765f69e748f3 |
|
BLAKE2b-256 | c0a0efa4ac7a743903a181a7cb2d6128eded76c7e7ff3df5535074ba3c34a117 |
Hashes for pyodps-0.11.2.2-cp36-cp36m-win32.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 7e1dd624569f696c536dcb92b352dfd42043855ea2575096f22d0f8a502765ee |
|
MD5 | 294e9e966ab3a26e1affe1656aaf994f |
|
BLAKE2b-256 | 90fb704e74f45d456d9c79ef45be88baed800e7a5fe756d36c939d6e24553b3a |
Hashes for pyodps-0.11.2.2-cp36-cp36m-manylinux1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 7d82f72c74ba7c5c8b2a681fcd4c5b0c36565ab803199f089e4abe8acdc6380f |
|
MD5 | 6be4cb9c060390b7027511a0905ca4d3 |
|
BLAKE2b-256 | 51c0a922105a47a6422ab1f652dabc41393fc6d2a31068ac4728485052992cdf |
Hashes for pyodps-0.11.2.2-cp36-cp36m-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 1ed5ec6a5c396d57627f82bf76057173aa01cf787b2d139311b3f6c2e790d043 |
|
MD5 | 2323c573760054df648a1e4eb03d3db6 |
|
BLAKE2b-256 | 9941a02d389bc34df1e083848371311c60047870ae0abe6e94935d7a674b3d13 |
Hashes for pyodps-0.11.2.2-cp35-cp35m-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 9b3273cda4ddca246c7c6b89736348a87ec3a4c37d25d1f046be7ea3d33f29b1 |
|
MD5 | 5f4bef4d5248485b4db872ad80c08d94 |
|
BLAKE2b-256 | d1b06783a94b16bc5b96e5ac7b30c3fbb96315741950a6384e6484702345ebde |
Hashes for pyodps-0.11.2.2-cp35-cp35m-win32.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 0dae853b8183bd2cf97c417550ccf71d787c046465221cfe6edc53cb09647210 |
|
MD5 | c128c5780c02c4798cc9f28b875e84dc |
|
BLAKE2b-256 | 13e37ce24bc9b2cc96401eb26e49c7993f162118c2151f48096da647d2b46280 |
Hashes for pyodps-0.11.2.2-cp35-cp35m-manylinux1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 258841164b3a3c698578da97bb12e6bba4c4633b3b2b36a61194e9a943206e55 |
|
MD5 | 18c4fb19415421598e9e78ab81d57d18 |
|
BLAKE2b-256 | fcd3047965a65e8b90468fe17675983ad6fccd793d0242e3947511dab9a699cd |
Hashes for pyodps-0.11.2.2-cp27-cp27mu-manylinux1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | e08ab11c8840995eaf4f50f3a380fa21a6856f8f0586cf24bcccc4a0480f2db5 |
|
MD5 | 79527d498be58631bb9d741fecf16d01 |
|
BLAKE2b-256 | 1aea271329619c0705208a2dd2522127b6276b302b361228047ae545db04d961 |
Hashes for pyodps-0.11.2.2-cp27-cp27m-manylinux1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | d77e9e79f6bebe856399deba938972c5a5fba1ca9cd9edc387253d7aaebf6c5f |
|
MD5 | 45181bcb276fa854bfdca8e8e69662cf |
|
BLAKE2b-256 | 01fb7584dc3ece7f0968428f0c3297b58da111b0d9715967dfb9448382363493 |
Hashes for pyodps-0.11.2.2-cp27-cp27m-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 4a5fe379568a340d41331fb92ed4f2bc32295a5a3bc166a531f0dc714940f126 |
|
MD5 | f247fab141eca3828179afb7ca1c8190 |
|
BLAKE2b-256 | d4c7f7c45669bb3f7226a4ce45981d03d0967b8fe8a5532731ea0f0cf37f5987 |