Data Processing is used for data processing through MinIO, databases, Web APIs, etc.
Project description
Data Processing
Current Version Main Features
Data Processing is used for data processing through MinIO, databases, Web APIs, etc. The data types handled include:
- txt
- json
- doc
- html
- excel
- csv
- markdown
- ppt
Current Text Type Processing
The data processing process includes: cleaning abnormal data, filtering, de-duplication, and anonymization.
Design
Local Development
Software Requirements
Before setting up the local data-process environment, please make sure the following software is installed:
- Python 3.10.x
Environment Setup
Install the Python dependencies in the requirements.txt file
Running
Run the server.py file in the src directory
isort
isort is a tool for sorting imports alphabetically within your Python code. It helps maintain a consistent and clean import order.
install
pip install isort
isort a file
isort src/server.py
isort a directory
isort .
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for one-data-processing-0.0.1.tar.gz
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 | 28c4eaccc42432378407ffa0769d69a5d79fdbd5acd65aa8aab9506f3dfcd8d1 |
|
| MD5 | dcc795744fe02ba05b07d49b0f51f721 |
|
| BLAKE2b-256 | 03820e75af207784703ef4d9c63046dd57dd67f32269d89496850b5c6e54ca70 |
Hashes for one_data_processing-0.0.1-py3-none-any.whl
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 | dd53eb0824b898176146d457587149b0eaded4ca040bec1c5a847ec8e324721e |
|
| MD5 | a31162a5ded05444cff73e661b70df2b |
|
| BLAKE2b-256 | ee6d2a2582686ea3e0bab01f22dc06f4036b89388f746c5081dd326fcce88208 |
