ODPS Python SDK and data analysis framework
Project description
Elegent way to access ODPS API. Documentation
Installation
The quick way:
pip install pyodps[full]
If you don’t need to use Jupyter, just type
pip install pyodps
The dependencies will be installed automatically.
Or from source code:
$ virtualenv pyodps_env
$ source pyodps_env/bin/activate
$ pip install git+https://github.com/aliyun/aliyun-odps-python-sdk.git
Dependencies
Python (>=2.7), including Python 3+, pypy, Python 3.7 recommended
setuptools (>=3.0)
Run Tests
install pytest
copy conf/test.conf.template to odps/tests/test.conf, and fill it with your account
run pytest odps
Usage
>>> from odps import ODPS
>>> o = ODPS('**your-access-id**', '**your-secret-access-key**',
... project='**your-project**', endpoint='**your-end-point**')
>>> dual = o.get_table('dual')
>>> dual.name
'dual'
>>> dual.table_schema
odps.Schema {
c_int_a bigint
c_int_b bigint
c_double_a double
c_double_b double
c_string_a string
c_string_b string
c_bool_a boolean
c_bool_b boolean
c_datetime_a datetime
c_datetime_b datetime
}
>>> dual.creation_time
datetime.datetime(2014, 6, 6, 13, 28, 24)
>>> dual.is_virtual_view
False
>>> dual.size
448
>>> dual.table_schema.columns
[<column c_int_a, type bigint>,
<column c_int_b, type bigint>,
<column c_double_a, type double>,
<column c_double_b, type double>,
<column c_string_a, type string>,
<column c_string_b, type string>,
<column c_bool_a, type boolean>,
<column c_bool_b, type boolean>,
<column c_datetime_a, type datetime>,
<column c_datetime_b, type datetime>]
DataFrame API
>>> from odps.df import DataFrame
>>> df = DataFrame(o.get_table('pyodps_iris'))
>>> df.dtypes
odps.Schema {
sepallength float64
sepalwidth float64
petallength float64
petalwidth float64
name string
}
>>> df.head(5)
|==========================================| 1 / 1 (100.00%) 0s
sepallength sepalwidth petallength petalwidth name
0 5.1 3.5 1.4 0.2 Iris-setosa
1 4.9 3.0 1.4 0.2 Iris-setosa
2 4.7 3.2 1.3 0.2 Iris-setosa
3 4.6 3.1 1.5 0.2 Iris-setosa
4 5.0 3.6 1.4 0.2 Iris-setosa
>>> df[df.sepalwidth > 3]['name', 'sepalwidth'].head(5)
|==========================================| 1 / 1 (100.00%) 12s
name sepalwidth
0 Iris-setosa 3.5
1 Iris-setosa 3.2
2 Iris-setosa 3.1
3 Iris-setosa 3.6
4 Iris-setosa 3.9
Command-line and IPython enhancement
In [1]: %load_ext odps In [2]: %enter Out[2]: <odps.inter.Room at 0x10fe0e450> In [3]: %sql select * from pyodps_iris limit 5 |==========================================| 1 / 1 (100.00%) 2s Out[3]: sepallength sepalwidth petallength petalwidth name 0 5.1 3.5 1.4 0.2 Iris-setosa 1 4.9 3.0 1.4 0.2 Iris-setosa 2 4.7 3.2 1.3 0.2 Iris-setosa 3 4.6 3.1 1.5 0.2 Iris-setosa 4 5.0 3.6 1.4 0.2 Iris-setosa
Python UDF Debugging Tool
#file: plus.py
from odps.udf import annotate
@annotate('bigint,bigint->bigint')
class Plus(object):
def evaluate(self, a, b):
return a + b
$ cat plus.input 1,1 3,2 $ pyou plus.Plus < plus.input 2 5
Contributing
For a development install, clone the repository and then install from source:
git clone https://github.com/aliyun/aliyun-odps-python-sdk.git cd pyodps pip install -r requirements.txt -e .
If you need to modify the frontend code, you need to install nodejs/npm. To build and install your frontend code, use
python setup.py build_js python setup.py install_js
License
Licensed under the Apache License 2.0
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distributions
Hashes for pyodps-0.11.4.post0-cp310-cp310-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 002fff346c0733ba45fe5b6865ac981586b3b3e87c66b24fb0fa873212e20cfb |
|
MD5 | 4a8b16fda1ceb0f8f01a884d8a80bb77 |
|
BLAKE2b-256 | 84c55fccde646a1471bd0ed21d84df10143ac8ef75f2498530604635ae40b2a4 |
Hashes for pyodps-0.11.4.post0-cp310-cp310-win32.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 9ef4e7f62321c3ba017a0f871bcbb723e0eeec48dc377bb19a302eef7c5f0505 |
|
MD5 | 1e96c8209172265654d2c45766e3c7f0 |
|
BLAKE2b-256 | 7b0c2b7e048736e1aeba1d5406e20e37958a7944ded42abc59cedee527aed2cb |
Hashes for pyodps-0.11.4.post0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 4ad87d73ec06b6097bb19ddca416f92c6bb245a95a87a30b379ef877f94791d5 |
|
MD5 | f3b65c75595922278c042a707e597ff3 |
|
BLAKE2b-256 | 3b8c481f81d56eb4356e4bc3a56980dfaa580393c0f6696f768e1ae5c1d563b5 |
Hashes for pyodps-0.11.4.post0-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | b47d5ced9884c6256aacd857a4e97889c19be5bfbb61e2e630c281c04b415799 |
|
MD5 | 6863af837afc07858b4f6ab9feb1ff6b |
|
BLAKE2b-256 | af7693a3a4a7833e277c477813018b52a63140f1e50939a7cbeef6f95bd0225a |
Hashes for pyodps-0.11.4.post0-cp310-cp310-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | e3cf62d45741e07bbdad7be49d4a03bd1c07f407a78b75f2c4db86222e543ecc |
|
MD5 | 64aa0254b051b0418d9dbd2554bc9446 |
|
BLAKE2b-256 | 9e2e648adc5cdfcbf34f06616b28c84065fbf62dfd52d839ee77a2925a1d2677 |
Hashes for pyodps-0.11.4.post0-cp310-cp310-macosx_10_9_universal2.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 2298bf3b93df62034b73a69bdccc3857245fd2e08b5e1bb5f824769cc1344d38 |
|
MD5 | 9221237c5cd3245cea61b4577672a05d |
|
BLAKE2b-256 | 209e4a88c5d38898c32388d9c3f4bb119ec37f2aee56a8389d88d3c32960b81f |
Hashes for pyodps-0.11.4.post0-cp39-cp39-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 49423e768512268779a01eb8a67c896058024c81a611f0d64c3b9544ec8c5702 |
|
MD5 | b1a56859ff39c744af6908a76efe3c09 |
|
BLAKE2b-256 | a506751adfae0e61161677db52c5ad331004fab299fe8ab39b9dfbeca9624d73 |
Hashes for pyodps-0.11.4.post0-cp39-cp39-win32.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | ca806e8bec815e50eeb5b67b0f7cb05f7e343854e2e23d035502fd4271d2d12e |
|
MD5 | 6d4b435ee464eac9d359732016211078 |
|
BLAKE2b-256 | 0fbca8d953c840a6d5c741e4842d9e61b750076129208c3f43d7b1cb3ba9460a |
Hashes for pyodps-0.11.4.post0-cp39-cp39-manylinux2014_aarch64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 91f7dddafe0c94cba14d5c4249c95d4410a1bcd97e13783f2b9ab1d021360df0 |
|
MD5 | 0a970f335b7eef86c3db7e1153d54e48 |
|
BLAKE2b-256 | 5ebc88fef63a997086a035e897100e23199454a29b4a934fa6f5a3a9194a7dd3 |
Hashes for pyodps-0.11.4.post0-cp39-cp39-manylinux1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | f30898c482c36ec7257a8b1e92728a558ff270fb028802bece622a0aaa9ffdd2 |
|
MD5 | 4f199fee72b229fe44654198dd83e433 |
|
BLAKE2b-256 | 8d1045db02488239d99141644048b5092ecff0bea4f03946f087a4345032cbd1 |
Hashes for pyodps-0.11.4.post0-cp39-cp39-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 995ee251685a17293972fe1d29be270d5bd5a8a6f1c7e79de6d48da729c961d7 |
|
MD5 | b1050ba9f31acd07070ce24719a1ea0b |
|
BLAKE2b-256 | 2b1eea201acedfece1168cf2cc14bd993ac2e2c77baf836a82650c0e1aa60a74 |
Hashes for pyodps-0.11.4.post0-cp39-cp39-macosx_10_9_universal2.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 7c9c87d16713dda6ba938e3df23dd79eee4c840ae3d4df3c68549ad89c9a3215 |
|
MD5 | be44aea71d26f2d5a5068ea019aab3cd |
|
BLAKE2b-256 | dc06e163b3d20aae9ef0194a81fca0d228c6289f5616e6f961a39fab00794ef5 |
Hashes for pyodps-0.11.4.post0-cp38-cp38-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | bd76b86ec6466019a32b51a70d98462a08e3d95bd2ed366fa112ace731db8c40 |
|
MD5 | cd3e9576e964dfbd7366dd97d50670aa |
|
BLAKE2b-256 | 32032cc9066cb30805892baa7b10abe92c2653dd0ce51fa4bdbf9e8f68b68189 |
Hashes for pyodps-0.11.4.post0-cp38-cp38-win32.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | d47a7c9180d1680961c0e362f964481d19f2c63e3d9e25fab107e07ad5ffb92d |
|
MD5 | 3bf94056166056568401daf822544cae |
|
BLAKE2b-256 | a7c5d5456dd8a2d0a8732bc8da9bb3d7961d8fd6ab23083054c4ce1ba35f7243 |
Hashes for pyodps-0.11.4.post0-cp38-cp38-manylinux2014_aarch64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | a9365269c945aa9be195a39e544fd727ef1e60a5ecce4b59f63df19fd710b2b3 |
|
MD5 | b055994ba87060013d3ad0a09d631a5a |
|
BLAKE2b-256 | e6970c82a1a671575cb4f9b02cfea4a1191715ee5bef76806c8713a79acf4b3b |
Hashes for pyodps-0.11.4.post0-cp38-cp38-manylinux1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 393ba6dd166c448606423ea2047e7a2a5a4f3f16044808a15b32a469d083138b |
|
MD5 | 887888ec2220781428c91ea93598a4f4 |
|
BLAKE2b-256 | 887212ee04a64b3ebfcbb9cb62b67b5954ad4d796ef201c43e5bf585ae730e25 |
Hashes for pyodps-0.11.4.post0-cp38-cp38-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | a8fa6e1d82dabaa1873e592324913c2cf0ecfdf62b010c1d2ca0c55ec89d2eec |
|
MD5 | 31a15751a272efdf2e32f13f73cbb78e |
|
BLAKE2b-256 | 1a18bf9fb6868ff00bc099d462dcdc60c6cd6d4c9c72e8c68f2651f21963af8b |
Hashes for pyodps-0.11.4.post0-cp37-cp37m-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 9b7663fde4381a295ee9d5f067982a1ee4c7a4201b0876ba1057a59ad0fe2b79 |
|
MD5 | 2d39ab74789cde79a3fa0ce958f0cd04 |
|
BLAKE2b-256 | 3a3aecb6c7113c7901ebce178c888f402773497d8289e0e19cf661ba6638e89e |
Hashes for pyodps-0.11.4.post0-cp37-cp37m-win32.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | cc2da22d551ed2885d440125055436d2b8d567c35508181254935491271742a8 |
|
MD5 | aa93c0487336c5deab6dd51c0cfcfa17 |
|
BLAKE2b-256 | 6dbbd94b869ff85de3b9258883f3464a3c2db64e112caa46dc49458754e56bb7 |
Hashes for pyodps-0.11.4.post0-cp37-cp37m-manylinux2014_aarch64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 5a2c8d6b50994203b8dcf672a19ed5806bb60e11e86c4c6cd0beb9fb781c6ddb |
|
MD5 | 945a244be0789867c86ee5c860a9ac09 |
|
BLAKE2b-256 | a572bebdc20f587d389fb06795b65d969faf45ad1da7d2aeb89b044494162279 |
Hashes for pyodps-0.11.4.post0-cp37-cp37m-manylinux1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | e2375d55abd7639f94da79e0f972d2ed8551b2585351151130669959dc37fb14 |
|
MD5 | a073f979825d14413cef7b12afb501da |
|
BLAKE2b-256 | 64769264c8de3b106fa564096c73caef9528ee1beab9897d522946751ea727e8 |
Hashes for pyodps-0.11.4.post0-cp37-cp37m-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | cbab40a7ef4e0dd5cc23c5cd607a79ed52fd0f8a7b240e5b204cda37b6cbe5a6 |
|
MD5 | fd8109389ba8ea49ee8c851439641f2d |
|
BLAKE2b-256 | 13efb840013827ead005a3bed430c61d43a6006f591748d48363e95abb3643e3 |
Hashes for pyodps-0.11.4.post0-cp36-cp36m-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | a80d86318716af690a96751904b89ef26edb01b1d735ea14cb4e8770c25ea40d |
|
MD5 | e3cc6e0103a8b30a923b68727edf0e1d |
|
BLAKE2b-256 | 2d6219049df5df3383e362b393151b4ce7de697eb0a94bb3e295c46927937382 |
Hashes for pyodps-0.11.4.post0-cp36-cp36m-win32.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 643f2391a434e93b56e8a8124946d96a5e9b671fa9829f6174e1eb863e237905 |
|
MD5 | 8eb92e49c90a14e659e09c8548e9b24a |
|
BLAKE2b-256 | 0eb0713014f5eeb4b0ec345aa5b00052c30d924151b13e382c2f56da89390194 |
Hashes for pyodps-0.11.4.post0-cp36-cp36m-manylinux1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 13fcaf1fb892469787717aefa1c59b31d14e323382b7e8d14da46650fa509526 |
|
MD5 | f009a9725613b403bc2d021766b09c09 |
|
BLAKE2b-256 | 6072852c12ed6e6e71b4907eb50a699775844b35dc6328aaad31dafd7c6a238d |
Hashes for pyodps-0.11.4.post0-cp36-cp36m-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | e64bd9d2e0da9d905c5dd8f84a237cdd0bf0293e7e233c5df9a59d4971240bd5 |
|
MD5 | 33609f719fe899b841d47617fb97eda9 |
|
BLAKE2b-256 | 69a849c860650adb5c3d7b4b379263f2612cdba6b27171d0cf46ffa88d7e1d33 |
Hashes for pyodps-0.11.4.post0-cp35-cp35m-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 5ce94345ec7cfd5b5bde22b2116daad0004d4f8afcc21646cae6e77805a78400 |
|
MD5 | 685c784807619ef16363bbd4755824fd |
|
BLAKE2b-256 | 7574ac666474b09c97886955edd1f714fc39d1eaf712c417b57aff54e91c1aaa |
Hashes for pyodps-0.11.4.post0-cp35-cp35m-win32.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | bf724f9e1e573b810ac370e3ec26ada6d7317f3269611bd4eaabce6b9176c938 |
|
MD5 | 70b8c4751f520c80d4ef5dac9044d56b |
|
BLAKE2b-256 | ead438e9b441aaae978c22c1ef915f6678225fd11c9ff46029dad698206db7f8 |
Hashes for pyodps-0.11.4.post0-cp35-cp35m-manylinux1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 515719e689e19f81be493d2d9b5d76953fb8acaa08df80761649d50a3b9348f1 |
|
MD5 | c177300cddb2fd73f956e4881382280f |
|
BLAKE2b-256 | f2d888911ad129f28c6da3724691721321894ce4bd512bef6b4a96b2f7d5d2bd |
Hashes for pyodps-0.11.4.post0-cp27-cp27mu-manylinux1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 346c69d92197a84526097069f58ceed6144483e80acdb5778e5b5f6beef0eeba |
|
MD5 | 0ec82cf7a8b6b593b05c39dc42ec86d2 |
|
BLAKE2b-256 | 34d1bb5ba918e8261dffcb3716fbf27d849cba0f9056668681c2ffb2643cd509 |
Hashes for pyodps-0.11.4.post0-cp27-cp27m-manylinux1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 3b1e6a23634ddfedee3914766b899fa0ac2c74e6d45beda85cf76a96ab0453ae |
|
MD5 | 8584e83e87798d17ab1e7717e3360567 |
|
BLAKE2b-256 | a8ab8fcecbf790e3c4671e6e32e2ab6ab38a5c9c3502273107835ea8ff6896ae |
Hashes for pyodps-0.11.4.post0-cp27-cp27m-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 657e1827d56ad99f47f736af03c03a514c7bb095b8f8d6f3ada0232ea11f43e0 |
|
MD5 | 28c10a4d5b5fa5631efd6796d3c96379 |
|
BLAKE2b-256 | d148aa68a1425402bf82083c6d37286ff7787e28b7956560e8a7789f0f1118f5 |