A wrapper library that provides one API to read, manipulate and writedata in different excel formats
Project description
Support the project
If your company has embedded pyexcel and its components into a revenue generating product, please support me on patreon or bounty source to maintain the project and develop it further.
If you are an individual, you are welcome to support me too and for however long you feel like. As my backer, you will receive early access to pyexcel related contents.
And your issues will get prioritized if you would like to become my patreon as pyexcel pro user.
With your financial support, I will be able to invest a little bit more time in coding, documentation and writing interesting posts.
Known constraints
Fonts, colors and charts are not supported.
Introduction
Feature Highlights
One application programming interface(API) to handle multiple data sources:
physical file
memory file
SQLAlchemy table
Django Model
Python data structures: dictionary, records and array
One API to read and write data in various excel file formats.
For large data sets, data streaming are supported. A genenerator can be returned to you. Checkout iget_records, iget_array, isave_as and isave_book_as.
Installation
You can install pyexcel via pip:
$ pip install pyexcel
or clone it and install it:
$ git clone https://github.com/pyexcel/pyexcel.git
$ cd pyexcel
$ python setup.py install
Usage
Please note that you will have to use ‘.sortable.html’ in order to replicate the example.
>>> # pip install pyexcel-text==0.2.7.1
>>> import pyexcel as p
>>> ccs_insight2 = p.Sheet()
>>> ccs_insight2.name = "Worldwide Mobile Phone Shipments (Billions), 2017-2021"
>>> ccs_insight2.ndjson = """
... {"year": ["2017", "2018", "2019", "2020", "2021"]}
... {"smart phones": [1.53, 1.64, 1.74, 1.82, 1.90]}
... {"feature phones": [0.46, 0.38, 0.30, 0.23, 0.17]}
... """.strip()
>>> ccs_insight2
pyexcel sheet:
+----------------+------+------+------+------+------+
| year | 2017 | 2018 | 2019 | 2020 | 2021 |
+----------------+------+------+------+------+------+
| smart phones | 1.53 | 1.64 | 1.74 | 1.82 | 1.9 |
+----------------+------+------+------+------+------+
| feature phones | 0.46 | 0.38 | 0.3 | 0.23 | 0.17 |
+----------------+------+------+------+------+------+
Suppose you have the following data in a dictionary:
Name |
Age |
---|---|
Adam |
28 |
Beatrice |
29 |
Ceri |
30 |
Dean |
26 |
you can easily save it into an excel file using the following code:
>>> import pyexcel
>>> # make sure you had pyexcel-xls installed
>>> a_list_of_dictionaries = [
... {
... "Name": 'Adam',
... "Age": 28
... },
... {
... "Name": 'Beatrice',
... "Age": 29
... },
... {
... "Name": 'Ceri',
... "Age": 30
... },
... {
... "Name": 'Dean',
... "Age": 26
... }
... ]
>>> pyexcel.save_as(records=a_list_of_dictionaries, dest_file_name="your_file.xls")
And here’s how to obtain the records:
>>> import pyexcel as p
>>> records = p.iget_records(file_name="your_file.xls")
>>> for record in records:
... print("%s is aged at %d" % (record['Name'], record['Age']))
Adam is aged at 28
Beatrice is aged at 29
Ceri is aged at 30
Dean is aged at 26
>>> p.free_resources()
Advanced usage :fire:
If you are dealing with big data, please consider these usages:
>>> def increase_everyones_age(generator): ... for row in generator: ... row['Age'] += 1 ... yield row >>> def duplicate_each_record(generator): ... for row in generator: ... yield row ... yield row >>> records = p.iget_records(file_name="your_file.xls") >>> io=p.isave_as(records=duplicate_each_record(increase_everyones_age(records)), ... dest_file_type='csv', dest_lineterminator='\n') >>> print(io.getvalue()) Age,Name 29,Adam 29,Adam 30,Beatrice 30,Beatrice 31,Ceri 31,Ceri 27,Dean 27,Dean <BLANKLINE>
Two advantages of above method:
Add as many wrapping functions as you want.
Constant memory consumption
Available Plugins
Package name |
Supported file formats |
Dependencies |
Python versions |
---|---|---|---|
2.6, 2.7, 3.3, 3.4, 3.5, 3.6 pypy |
|||
xls, xlsx(read only), xlsm(read only) |
same as above |
||
xlsx |
same as above |
||
ods |
pyexcel-ezodf, lxml |
2.6, 2.7, 3.3, 3.4 3.5, 3.6 |
|
ods |
same as above |
Package name |
Supported file formats |
Dependencies |
Python versions |
---|---|---|---|
xlsx(write only) |
Python 2 and 3 |
||
xlsx(read only) |
lxml |
same as above |
|
read only for ods, fods |
lxml |
same as above |
|
html(read only) |
lxml,html5lib |
same as above |
Package name |
Supported file formats |
Dependencies |
Python versions |
---|---|---|---|
write only:rst, mediawiki, html, latex, grid, pipe, orgtbl, plain simple read only: ndjson r/w: json |
2.6, 2.7, 3.3, 3.4 3.5, 3.6, pypy |
||
handsontable in html |
same as above |
||
svg chart |
2.7, 3.3, 3.4, 3.5 3.6, pypy |
||
sortable table in html |
same as above |
||
gantt chart in html |
except pypy, same as above |
In order to manage the list of plugins installed, you need to use pip to add or remove a plugin. When you use virtualenv, you can have different plugins per virtual environment. In the situation where you have multiple plugins that does the same thing in your environment, you need to tell pyexcel which plugin to use per function call. For example, pyexcel-ods and pyexcel-odsr, and you want to get_array to use pyexcel-odsr. You need to append get_array(…, library=’pyexcel-odsr’).
Footnotes
Acknowledgement
All great work have been done by odf, ezodf, xlrd, xlwt, tabulate and other individual developers. This library unites only the data access code.
License
New BSD License
Change log
0.5.13 - 12.03.2019
updated
#176: get_sheet {IndexError}list index out of range // XLSX can’t be opened
0.5.12 - 25.02.2019
updated
#174: include examples in tarbar
0.5.11 - 22.02.2019
updated
#169: remove pyexcel-handsontalbe in test
add tests, and docs folder in distribution
0.5.10 - 3.12.2018
updated
#157: Please use scan_plugins_regex, which lml 0.7 complains about
updated dependency on pyexcel-io to 0.5.11
0.5.9.1 - 30.08.2018
updated
to require pyexcel-io 0.5.9.1 and use lml at least version 0.0.2
0.5.9 - 30.08.2018
added
support __len__. len(book) returns the number of sheets and len(sheet) returns the number of rows
#144: memory-efficient way to read sheet names.
#148: force_file_type is introduced. When reading a file on a disk, this parameter allows you to choose a reader. i.e. csv reader for a text file. xlsx reader for a xlsx file but with .blob file suffix.
finally, pyexcel got import pyexcel.__version__
updated
0.5.8 - unreleased
added
#125, sort book sheets
updated
0.5.7 - 11.01.2018
added
pyexcel-io#46, expose bulk_save to developer.
0.5.6 - 23.10.2017
removed
0.5.5 - 20.10.2017
removed
0.5.4 - 27.09.2017
fixed
#100, Sheet.to_dict() gets out of range error because there is only one row.
updated
Updated the baseline of pyexcel-io to 0.5.1.
0.5.3 - 01-08-2017
added
0.5.2 - 26-07-2017
Updated
embeded the enabler for pyexcel-htmlr. http source does not support text/html as mime type.
0.5.1 - 12.06.2017
Updated
support saving SheetStream and BookStream to database targets. This is needed for pyexcel-webio and its downstream projects.
0.5.0 - 19.06.2017
Added
Sheet.top() and Sheet.top_left() for data browsing
add html as default rich display in Jupyter notebook when pyexcel-text and pyexcel-chart is installed
add svg as default rich display in Jupyter notebook when pyexcel-chart and one of its implementation plugin(pyexcel-pygal, etc.) are is installed
new dictionary source supported: a dictionary of key value pair could be read into a sheet.
added dynamic external plugin loading. meaning if a pyexcel plugin is installed, it will be loaded implicitly. And this change would remove unnecessary info log for those who do not use pyexcel-text and pyexcel-gal
save_book_as before 0.5.0 becomes isave_book_as and save_book_as in 0.5.0 convert BookStream to Book before saving.
#83, file closing mechanism is enfored. free_resource is added and it should be called when iget_array, iget_records, isave_as and/or isave_book_as are used.
Updated
array is passed to pyexcel.Sheet as reference. it means your array data will be modified.
Removed
pyexcel.Writer and pyexcel.BookWriter were removed
pyexcel.load_book_from_sql and pyexcel.load_from_sql were removed
pyexcel.deprecated.load_from_query_sets, pyexcel.deprecated.load_book_from_django_models and pyexcel.deprecated.load_from_django_model were removed
Removed plugin loading code and lml is used instead
0.4.5 - 17.03.2017
Updated
#80: remove pyexcel-chart import from v0.4.x
0.4.4 - 06.02.2017
Updated
Removed
remove get_{{file_type}}_stream functions from pyexcel.Sheet and pyexcel.Book introduced since 0.4.3.
0.4.3 - 26.01.2017
Added
‘.stream’ attribute are attached to ~pyexcel.Sheet and ~pyexcel.Book to get direct access the underneath stream in responding to file type attributes, such as sheet.xls. it helps provide a custom stream to external world, for example, Sheet.stream.csv gives a text stream that contains csv formatted data. Book.stream.xls returns a xls format data in a byte stream.
Updated
Better error reporting when an unknown parameters or unsupported file types were given to the signature functions.
0.4.2 - 17.01.2017
Updated
Raise exception if the incoming sheet does not have column names. In other words, only sheet with column names could be saved to database. sheet with row names cannot be saved. The alternative is to transpose the sheet, then name_columns_by_row and then save.
fix iget_records where a non-uniform content should be given, e.g. [[“x”, “y”], [1, 2], [3]], some record would become non-uniform, e.g. key ‘y’ would be missing from the second record.
skip_empty_rows is applicable when saving a python data structure to another data source. For example, if your array contains a row which is consisted of empty string, such as [‘’, ‘’, ‘’ … ‘’], please specify skip_empty_rows=False in order to preserve it. This becomes subtle when you try save a python dictionary where empty rows is not easy to be spotted.
#69: better documentation for save_book_as.
0.4.1 - 23.12.2016
Updated
#68: regression save_to_memory() should have returned a stream instance.
0.4.0 - 22.12.2016
Added
Flask-Excel#19 allow sheet_name parameter
pyexcel-xls#11 case-insensitive for file_type. xls and XLS are treated in the same way
Updated
#66: export_columns is ignored
Update dependency on pyexcel-io v0.3.0
0.3.3 - 07.11.2016
Updated
#63: cannot display empty sheet(hence book with empty sheet) as texttable
0.3.2 - 02.11.2016
Updated
#62: optional module import error become visible.
0.3.0 - 28.10.2016
Added:
file type setters for Sheet and Book, and its documentation
iget_records returns a generator for a list of records and should have better memory performance, especially dealing with large csv files.
iget_array returns a generator for a list of two dimensional array and should have better memory performance, especially dealing with large csv files.
Enable pagination support, and custom row renderer via pyexcel-io v0.2.3
Updated
Take isave_as out from save_as. Hence two functions are there for save a sheet as
#60: encode ‘utf-8’ if the console is of ascii encoding.
#59: custom row renderer
#56: set cell value does not work
pyexcel.transpose becomes pyexcel.sheets.transpose
iterator functions of pyexcel.Sheet were converted to generator functions
pyexcel.Sheet.enumerate()
pyexcel.Sheet.reverse()
pyexcel.Sheet.vertical()
pyexcel.Sheet.rvertical()
pyexcel.Sheet.rows()
pyexcel.Sheet.rrows()
pyexcel.Sheet.columns()
pyexcel.Sheet.rcolumns()
pyexcel.Sheet.named_rows()
pyexcel.Sheet.named_columns()
~pyexcel.Sheet.save_to_memory and ~pyexcel.Book.save_to_memory return the actual content. No longer they will return a io object hence you cannot call getvalue() on them.
Removed:
content and out_file as function parameters to the signature functions are no longer supported.
SourceFactory and RendererFactory are removed
The following methods are removed
pyexcel.to_array
pyexcel.to_dict
pyexcel.utils.to_one_dimensional_array
pyexcel.dict_to_array
pyexcel.from_records
pyexcel.to_records
pyexcel.Sheet.filter has been re-implemented and all filters were removed:
pyexcel.filters.ColumnIndexFilter
pyexcel.filters.ColumnFilter
pyexcel.filters.RowFilter
pyexcel.filters.EvenColumnFilter
pyexcel.filters.OddColumnFilter
pyexcel.filters.EvenRowFilter
pyexcel.filters.OddRowFilter
pyexcel.filters.RowIndexFilter
pyexcel.filters.SingleColumnFilter
pyexcel.filters.RowValueFilter
pyexcel.filters.NamedRowValueFilter
pyexcel.filters.ColumnValueFilter
pyexcel.filters.NamedColumnValueFilter
pyexcel.filters.SingleRowFilter
the following functions have been removed
add_formatter
remove_formatter
clear_formatters
freeze_formatters
add_filter
remove_filter
clear_filters
freeze_formatters
pyexcel.Sheet.filter has been re-implemented and all filters were removed:
pyexcel.formatters.SheetFormatter
0.2.5 - 31.08.2016
Updated:
#58: texttable should have been made as compulsory requirement
0.2.4 - 14.07.2016
Updated:
For python 2, writing to sys.stdout by pyexcel-cli raise IOError.
0.2.3 - 11.07.2016
Updated:
For python 3, do not seek 0 when saving to memory if sys.stdout is passed on. Hence, adding support for sys.stdin and sys.stdout.
0.2.2 - 01.06.2016
Updated:
Explicit imports, no longer needed
Depends on latest setuptools 18.0.1
NotImplementedError will be raised if parameters to core functions are not supported, e.g. get_sheet(cannot_find_me_option=”will be thrown out as NotImplementedError”)
0.2.1 - 23.04.2016
Added:
add pyexcel-text file types as attributes of pyexcel.Sheet and pyexcel.Book, related to #31
auto import pyexcel-text if it is pip installed
Updated:
code refactoring done for easy addition of sources.
bug fix #29, Even if the format is a string it is displayed as a float
pyexcel-text is no longer a plugin to pyexcel-io but to pyexcel.sources, see pyexcel-text#22
Removed:
pyexcel.presentation is removed. No longer the internal decorate @outsource is used. related to #31
0.2.0 - 17.01.2016
Updated
adopt pyexcel-io yield key word to return generator as content
pyexcel.save_as and pyexcel.save_book_as get performance improvements
0.1.7 - 03.07.2015
Added
Support pyramid-excel which does the database commit on its own.
0.1.6 - 13.06.2015
Added
get excel data from a http url
0.0.13 - 07.02.2015
Added
Support django
texttable as default renderer
0.0.12 - 25.01.2015
Added
Added sqlalchemy support
0.0.10 - 15.12.2015
Added
added csvz and tsvz format
0.0.4 - 12.10.2014
Updated
Support python 3
0.0.1 - 14.09.2014
Features:
read and write csv, ods, xls, xlsx and xlsm files(which are referred later as excel files)
various iterators for the reader
row and column filters for the reader
utilities to get array and dictionary out from excel files.
cookbok receipes for some common and simple usage of this library.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for pyexcel-0.5.13-py2.py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | f34f4bcc0322ab1f7dccd38b77ecd8f1f7964cf820bdd682494fc47d9f2fb6e3 |
|
MD5 | e2f5c8725e01cde291d31f8d387134bf |
|
BLAKE2b-256 | db9c306d5860bb14b2d902104ee5235be89ef9407c4c131f8d00f4a45612d0d5 |