ODPS Python SDK and data analysis framework
Project description
Elegent way to access ODPS API. Documentation
Installation
The quick way:
pip install pyodps[full]
To integrate with Mars:
pip install pyodps#egg=pyodps[mars]
If you don’t need to use Jupyter, just type
pip install pyodps
The dependencies will be installed automatically.
Or from source code:
$ virtualenv pyodps_env
$ source pyodps_env/bin/activate
$ git clone <git clone URL> pyodps
$ cd pyodps
$ python setup.py install
Dependencies
Python (>=2.6), including Python 3+, pypy, Python 2.7 recommended
setuptools (>=3.0)
requests (>=2.4.0)
Run Unittest
copy conf/test.conf.template to odps/tests/test.conf, and fill it with your account
run python -m unittest discover
Usage
>>> from odps import ODPS
>>> o = ODPS('**your-access-id**', '**your-secret-access-key**',
... project='**your-project**', endpoint='**your-end-point**')
>>> dual = o.get_table('dual')
>>> dual.name
'dual'
>>> dual.schema
odps.Schema {
c_int_a bigint
c_int_b bigint
c_double_a double
c_double_b double
c_string_a string
c_string_b string
c_bool_a boolean
c_bool_b boolean
c_datetime_a datetime
c_datetime_b datetime
}
>>> dual.creation_time
datetime.datetime(2014, 6, 6, 13, 28, 24)
>>> dual.is_virtual_view
False
>>> dual.size
448
>>> dual.schema.columns
[<column c_int_a, type bigint>,
<column c_int_b, type bigint>,
<column c_double_a, type double>,
<column c_double_b, type double>,
<column c_string_a, type string>,
<column c_string_b, type string>,
<column c_bool_a, type boolean>,
<column c_bool_b, type boolean>,
<column c_datetime_a, type datetime>,
<column c_datetime_b, type datetime>]
DataFrame API
>>> from odps.df import DataFrame
>>> df = DataFrame(o.get_table('pyodps_iris'))
>>> df.dtypes
odps.Schema {
sepallength float64
sepalwidth float64
petallength float64
petalwidth float64
name string
}
>>> df.head(5)
|==========================================| 1 / 1 (100.00%) 0s
sepallength sepalwidth petallength petalwidth name
0 5.1 3.5 1.4 0.2 Iris-setosa
1 4.9 3.0 1.4 0.2 Iris-setosa
2 4.7 3.2 1.3 0.2 Iris-setosa
3 4.6 3.1 1.5 0.2 Iris-setosa
4 5.0 3.6 1.4 0.2 Iris-setosa
>>> df[df.sepalwidth > 3]['name', 'sepalwidth'].head(5)
|==========================================| 1 / 1 (100.00%) 12s
name sepalwidth
0 Iris-setosa 3.5
1 Iris-setosa 3.2
2 Iris-setosa 3.1
3 Iris-setosa 3.6
4 Iris-setosa 3.9
Command-line and IPython enhancement
In [1]: %load_ext odps In [2]: %enter Out[2]: <odps.inter.Room at 0x10fe0e450> In [3]: %sql select * from pyodps_iris limit 5 |==========================================| 1 / 1 (100.00%) 2s Out[3]: sepallength sepalwidth petallength petalwidth name 0 5.1 3.5 1.4 0.2 Iris-setosa 1 4.9 3.0 1.4 0.2 Iris-setosa 2 4.7 3.2 1.3 0.2 Iris-setosa 3 4.6 3.1 1.5 0.2 Iris-setosa 4 5.0 3.6 1.4 0.2 Iris-setosa
Python UDF Debugging Tool
#file: plus.py
from odps.udf import annotate
@annotate('bigint,bigint->bigint')
class Plus(object):
def evaluate(self, a, b):
return a + b
$ cat plus.input 1,1 3,2 $ pyou plus.Plus < plus.input 2 5
Contributing
For a development install, clone the repository and then install from source:
git clone https://github.com/aliyun/aliyun-odps-python-sdk cd pyodps pip install -r requirements.txt -e .
If you need to modify the frontend code, you need to install nodejs/npm. To build and install your frontend code, use
python setup.py build_js python setup.py install_js
License
Licensed under the Apache License 2.0
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distributions
Hashes for pyodps-0.11.1-cp310-cp310-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 4d15dec5bb5ccdc5b55765e51b15aaed2eecbc4a8e609623d277e67d836e008f |
|
MD5 | 0855ed74a1829e4c965e0302b0e3194a |
|
BLAKE2b-256 | 63b53ef836679f6484b0c59f82b5db98ea456fc439d9cd4327b8243998d76424 |
Hashes for pyodps-0.11.1-cp310-cp310-win32.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | a1901a0e104713dae2474a481fbb503bb887b31cd9bb83616548f2b8459332fd |
|
MD5 | 032081789feb194ba263ce2979fa319e |
|
BLAKE2b-256 | 4bb8fe3163e44739442868b4da7a6c6d46782cd262d49d11db55068fbac5f741 |
Hashes for pyodps-0.11.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 837f1e7b2c7d611a1f1cc1cbc3f56ef7254a021e3e8d5caac18dc6c35bfba13f |
|
MD5 | afab1b7bcaada4419155d63152ade452 |
|
BLAKE2b-256 | 8945bb5d0d578fe185e0abda1f356810495d57b6407e330f80beb21c682fa121 |
Hashes for pyodps-0.11.1-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 70f8bdd3387217d7b92435e4d0578affd10f055629359e6c10597d7130b85566 |
|
MD5 | 0cad80dd9cc39c31c4893f8c273a6a3d |
|
BLAKE2b-256 | 5051ddd2bc1fb83967e3b88f687f23db84fd374bbbefcd4f3198976404ecdfde |
Hashes for pyodps-0.11.1-cp310-cp310-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 739c66dd6a820c41ffcf7453f2a98cb0dc1c317b8856da8fed9f1179a1bc690d |
|
MD5 | 935ab8b5c8f76bf0e9d9824099a45077 |
|
BLAKE2b-256 | 968271dafbb6e326d7cc5ef6b660fb256b79e2a25b9c5303e1f70556118e3109 |
Hashes for pyodps-0.11.1-cp310-cp310-macosx_10_9_universal2.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 3de988b2ec940bdc849ea06380f879e217a86808726f1a1d130fbdd9f31c6008 |
|
MD5 | d018deb135f401d3109396d7fba57ffb |
|
BLAKE2b-256 | 89430347d27d088a2a0f246189bc3a20ba3817fe38529386868bdb1c9619be0d |
Hashes for pyodps-0.11.1-cp39-cp39-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 6aaf06efed10f5c9d7699b10159dfe815bb7c344d47598e8952413f7d590a737 |
|
MD5 | be9affec7afd882938265a877ba73e2f |
|
BLAKE2b-256 | f223cf9f4be166efa731f37aac4d3fb5c8203912bda0e7f5afea60cd0fab4c0d |
Hashes for pyodps-0.11.1-cp39-cp39-win32.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 31326cbccf7c659fc3749d473f44c1aa62be78e1d9176c5da142394cf778034c |
|
MD5 | 6e0af5a499659ccaeb1a471c110390b6 |
|
BLAKE2b-256 | 2c148a0beacbe45a73a858ae8f1fb0342a6e5e84d9d16948e4d531fc598475e8 |
Hashes for pyodps-0.11.1-cp39-cp39-manylinux2014_aarch64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | e564e57f439fec89eeaa4051ee23fbef6bf5f6e6cff2c575a661584edb916f1b |
|
MD5 | 08fd3b180969100a0cbbe165ca9863a2 |
|
BLAKE2b-256 | 39685f0a0e51d8c57bf60931cec3b657bb752b093a115ebb7bb74d6e8e6b90a7 |
Hashes for pyodps-0.11.1-cp39-cp39-manylinux1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | fad8b172d4a351e9ae04deade898586e66a079e452123bd20f815e48ffc33ff2 |
|
MD5 | 02dbcc26567d3b096e08b6cdc3fafd23 |
|
BLAKE2b-256 | 811dcff927a43c71812969805299ff547eace1eae3c61f2a757cdf4f1e378c85 |
Hashes for pyodps-0.11.1-cp39-cp39-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 3c25772d15e33b27a0e9da37061f03cd7051cf836456008a4dd9328690dc639a |
|
MD5 | 0807d8cd87bcac0e25fb15edf3e19b34 |
|
BLAKE2b-256 | 89ccb3f5919d892f0cdd8bb781c0763e25a888290b504085b9e14edd15a94640 |
Hashes for pyodps-0.11.1-cp39-cp39-macosx_10_9_universal2.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | f34c1ed6cfb6473bcd8e223c97d969adfc89eb084b14de45587526d762ff3d4f |
|
MD5 | 3369775f4e218c783d398609a7d462da |
|
BLAKE2b-256 | 981dbea564fc9f242ec92cf476c988816c4038f7e824280b595b0fb132737eed |
Hashes for pyodps-0.11.1-cp38-cp38-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 098c3d1c98243d1b3fbbe96198b98018c41b69aabc93e49997e116f7d1168acb |
|
MD5 | 377553210c682af5907a73027b43402a |
|
BLAKE2b-256 | 1a88b0bcf55b3c077fe010554b3870626325ee5d8bb31b972769a454d310c613 |
Hashes for pyodps-0.11.1-cp38-cp38-win32.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | eec6a7ce17596e685e4062dd6da524cd3f134c13bf05690ad7e5388f30de1ce5 |
|
MD5 | 3b36656d988718472b2084d627c4c3ec |
|
BLAKE2b-256 | ac2220fc203d463c7ff3a56d1dec52d14438a443e0a25de8526217741b810dc9 |
Hashes for pyodps-0.11.1-cp38-cp38-manylinux2014_aarch64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | e3b28b699ec149c2b724279797a5bb022006d45fb385bd284884e9db511480c1 |
|
MD5 | 46b33e26d553e88ac7fe572f9442c2f4 |
|
BLAKE2b-256 | b34064f924d43be03790db3987b136506e288e8606db3b0020ffd013bcdb2d10 |
Hashes for pyodps-0.11.1-cp38-cp38-manylinux1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 954df4e1f34b73fbd0f7828d80af039c7bdb10457dae947a17bd77a74d71b6b4 |
|
MD5 | fab204fd8a3ed695dccae8dade142140 |
|
BLAKE2b-256 | 646c9693347de85243f86ea2c012704394a7a96af9f3c18b87b1dc48c6b86b3e |
Hashes for pyodps-0.11.1-cp38-cp38-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 6200cbeb02725c67c75cda9647f428bac3f0b1a7c6e9ceb541f1998562059e6c |
|
MD5 | a61ff35ab3ce9be012817d1fa7438cd8 |
|
BLAKE2b-256 | c235b1bd740f3c8e853f5448fa517558bcfd9dd5da3c69cd726f71add473bf88 |
Hashes for pyodps-0.11.1-cp37-cp37m-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 0ecbbcf4d5c17f1f44ef8f8887156ff2f0013702e9de1536435abb2f362f3ef2 |
|
MD5 | 2272d9ef32382663550b5eaf577bc35c |
|
BLAKE2b-256 | 5a267bc9378d917557a4947268ec31563468e5b6f48909aa4846006be61adadc |
Hashes for pyodps-0.11.1-cp37-cp37m-win32.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 17090ad02d3b3e525f935b3e6dd3fcb77bed5254343580d8b8f40117e75161a9 |
|
MD5 | bbcaf4736a5fc66da3c18d273f56caf5 |
|
BLAKE2b-256 | f6004c41962f37021a0d2f5e6ab6f0cc3bffd62c5e9d46d045274be6a52229dc |
Hashes for pyodps-0.11.1-cp37-cp37m-manylinux2014_aarch64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | ce15dc2731d7cb38eb93d3b6e8bcdc3d3142766f982b9684f244cba4e36338df |
|
MD5 | 260f5427e42e29fc955954641edd3697 |
|
BLAKE2b-256 | 13c1e35e7883a8ec25be194f9c196c80ee7ab92abb9428fe23c75e87e2d7f74f |
Hashes for pyodps-0.11.1-cp37-cp37m-manylinux1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | b10801edbf0a4a93db4a8e20ebff6e8560b791e82d93c6ba0085fa00d38c6043 |
|
MD5 | 4a3227f467b3836d840e50717d121a9c |
|
BLAKE2b-256 | f6a19429946facf67f5240b0476e66cc6bf4c63882e12f4a738b4150253821cc |
Hashes for pyodps-0.11.1-cp37-cp37m-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 19c040c3ffb1e519b57f7f2aa49c6e6b332e027bf71295dc9bd0c1b95d8c02a8 |
|
MD5 | c9d1176078464e09cf045440c560c912 |
|
BLAKE2b-256 | ec96f333e4a0514e7522f2f776a0ef50c5bcd0b5c4a4cc90ad7b6f5b0632e968 |
Hashes for pyodps-0.11.1-cp36-cp36m-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | d2209101408b9983eaf3b5a3c4bc36bf26b74a5cf5806ca8c56d40a04d55b2c8 |
|
MD5 | 598c64f9ce2db10b8a0d5859bcd14582 |
|
BLAKE2b-256 | 68d3d8f8d59d50324849005dd2944132cb72ee0a27bdc00fa4b228f99bbf1112 |
Hashes for pyodps-0.11.1-cp36-cp36m-win32.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 30580666ff1799683d0c1ec71ac0bf610cb2f4062efc8fb842ff213257447b11 |
|
MD5 | 9c28da3be2578baed88da0e59e434b62 |
|
BLAKE2b-256 | 82391b426c9c9845353071630b3b832768e2fae4dcbd22ab0386f1f411dbeac9 |
Hashes for pyodps-0.11.1-cp36-cp36m-manylinux1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | d23b5bded9a69a6652052ca503394e2a5f2b92e3e9fefd58dea18d1453d8109b |
|
MD5 | fa61e6bdf7ee3d643b4428255ec4b44c |
|
BLAKE2b-256 | 6c355a89986a9b38886f717cc91f07e6c37c5eb3d30b011dd4d5f33517dde176 |
Hashes for pyodps-0.11.1-cp36-cp36m-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 35d26a4b81bbd39abc87fc7d8a9816f0e7b81728120a927ba25163c1a512800c |
|
MD5 | e64fea245e465d577abb141c439e3abe |
|
BLAKE2b-256 | d067c9758dd9e483d60eb80d37ca8275d2c1a2d44db452a216efe7d79e2c8d58 |
Hashes for pyodps-0.11.1-cp35-cp35m-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | e64abbdb191513d7fb5d2dfdb0f91d5139fba3f7924540be1013fa07ea7368ec |
|
MD5 | fd47057fe8eb6b7dd161243791beb266 |
|
BLAKE2b-256 | 9ba458ee693206df0a5f501c963892d1b87a78aa4bc1144aa45887f9c5a7f602 |
Hashes for pyodps-0.11.1-cp35-cp35m-win32.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 563ada8b94ee6db3bdffa8b323bbc39c1bb27ed4cf83e9f4a8530eb005bbe8ed |
|
MD5 | f04d530b91bd7471e899834e179c8bf9 |
|
BLAKE2b-256 | 352196dfefb571ed3c5255e7e035a87d12be2b8237cb793f596487b817859575 |
Hashes for pyodps-0.11.1-cp35-cp35m-manylinux1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 83b58cb70dc0e3f5be2ce28413e509b4f0214251fb00a21799a6eeb71486aa97 |
|
MD5 | e5214c3bfffbf0b94a9f71276c7d030f |
|
BLAKE2b-256 | a91a7315b147a4ccaa0549a3ec5b92c686143f76976d0e6d3242f7fc1f5cbe53 |
Hashes for pyodps-0.11.1-cp27-cp27mu-manylinux1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 67223181a5331c625730e28f02eb52328a230e0fd15268cb43d949a64a9f391c |
|
MD5 | d9ab925a3aceea9723092d9c5bffcf2f |
|
BLAKE2b-256 | 52ce91d7a959f8dbe11eff83e0222a5194dbea09c336b04e5328f27322d3d3e3 |
Hashes for pyodps-0.11.1-cp27-cp27m-manylinux1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 7d8a303cfab4df035be68d3e4f0cf18851f6d6dd2ec16432b4091a65f06edb7b |
|
MD5 | 073cecc28573e785c9698c6d30721a7a |
|
BLAKE2b-256 | ec424b073f9b58d2414d9750db976d4ebd0352240d93cd213045485506c2fd20 |
Hashes for pyodps-0.11.1-cp27-cp27m-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | f8dd357a61c24df9f4c5f7dd2f76732205b205488a2db543d3d443addc07239a |
|
MD5 | 67bad204704856113cb50733f72cf619 |
|
BLAKE2b-256 | d87565db54667578fb788e16d5776cd84bcabc13bf6e307f916109010552abfb |