A small Python library to deal with big tables
Sometimes you cannot or don't want to use Pandas or similar advanced tool for data analysis, but still have a need to manipulate large tables with code. In such cases you can use Lemuras – it is a pure Python library without external dependencies. And if you have some experience of Pandas or SQL, then you can easily work with Lemuras.
Again, this library may be considered as a simplified analogue of Pandas, but not as a replacement. However, Lemuras is capable of processing an operation on a few tables with several thousands of rows in less than a second on a simple web server. So, if you need a tiny library to generate analytical reports or convert table formats, Lemuras is a good choice!
- Integration with Jupyter IPython Notebook: Lemuras objects are printed as nice tables.
- Save / load CSV files, JSON, HTML tables, SQL (both query result and table creation code).
- Automatic columns types detection, simple type conversion.
- Access, add, edit, delete cells, rows, columns. Apply custom of built-in functions, lambdas, sort the data, iterate over rows.
- Advanced processing of columns: you can take any table column, apply any function or lambda, do math with several columns and discrete values, compare them, check existing in other columns or lists, filter a table by it, or add it to a table, etc... In other words, you can do anything!
- Grouping by none, one, or multiple columns, aggregation with built-in or user-defined functions and lambdas for specified or just all the columns.
- Merge (Join): inner / left / right / outer.
- Tables concatenation and appending.
- Pivot tables creation.
It is tasted on Python 2.7 and Python 3.4-3.7
All the features are described in notebook examples:
- Basic things – access to columns, cells, rows; add, delete, change their values; also filtering and sorting. 1.5) Functions applying – apply functions or lambda expressions to columns or tables, change types, aggregate values, use your own or one of lots predefined useful functions (oncluding statistical ones).
- Group by – grouping and combining (aggregating).
- Merge / Join – such types: inner, outer, left, right.
- Pivot table – create new tables with columns, rows and cells from another table.
- Tables Concatenate / Append – simple tables concatenation and appending.
- Types, Read/Write, CSV, SQL, JSON, HTML – description of Lemuras supported data types, saving to and loading from CSV, SQL, JSON, HTML formats.
In addition, there are several complex examples of solving a real world problems:
OLAP reports parser with CSV, TSV, and even XLS support
The code of Lemuras is well-commented, also there are many unit-tests, so, you can easily find useful information there. Contributions are welcome.
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.