DataRecipe

Table of Contents

  1. Overview
  2. Functions
  3. Contact Information

Overview

This toolkit provides a variety of Python functions to facilitate common data manipulation, data import/export, and database operations.

Functions

General Features

send_email

Sends an email using SMTP with SSL/TLS options, supporting attachments if provided.

  • Parameters:
    • subject: Email subject as a string.
    • body: Main content of the email.
    • send_email_address: Sender's email address.
    • send_email_password: Sender's email password for SMTP authentication.
    • receive_email_address: Recipient's email address.
    • attachment_path: Directory path where attachments are stored (optional).
    • attachment_list: List of filenames to be attached (optional).
    • smtp_address: SMTP server address (default: 'smtp.feishu.cn').
    • smtp_port: SMTP server port (default: 465).

Example with Attachments:

send_email(
    "Meeting Documents", 
    "Please see attached documents for the upcoming meeting.", 
    "sender@example.com", 
    "password123", 
    "receiver@example.com", 
    attachment_path="/path/to/documents", 
    attachment_list=["agenda.pdf", "minutes.docx"]
)

Data Validation and Cleaning

check_empty

Checks for empty entries in specified DataFrame columns.

  • Parameters:
    • df: DataFrame to check.
    • columns: Columns to check for missing values.
    • output_cols: Columns to include in the output.

Example:

empty_data = check_empty(df, columns=["name", "email"])

clean_dataframe

Cleans DataFrame by replacing infinite values with NaN.

  • Parameters:
    • df: DataFrame to clean.

Example:

clean_dataframe(df)

Data Import/Export

local_to_df

Converts files from a local directory to a pandas DataFrame.

  • Parameters:
    • path: Directory path to search for files.
    • partial_file_name: File name pattern to match.
    • skip_rows: Number of rows to skip at the start of each file.
    • keep_file_name: If True, adds a column with the file name.
    • sheet_num: For Excel files, specifies the sheet number to read.
    • encoding: Character encoding of the files.

Example with CSV files:

df = local_to_df("./data", "sample", keep_file_name=True)

Example with Excel files:

df = local_to_df("./data", "report", sheet_num=2, encoding='utf-8')

df_to_xlsx

Saves a DataFrame to an Excel file.

  • Parameters:
    • df: DataFrame to save.
    • directory_path: Path to directory where the file will be saved.
    • file_name: Name of the output file.

Example:

df_to_xlsx(df, "./output", "output_data")

df_to_csv

Saves a DataFrame to a CSV file.

  • Parameters:
    • df: DataFrame to save.
    • directory_path: Path to directory where the file will be saved.
    • file_name: Name of the output file.

Example:

df_to_csv(df, "./output", "output_data")

Database Operations

update

Updates records in a database table based on conditions.

  • Parameters:
    • raw_df: DataFrame containing new data to update.
    • database: Database name.
    • table: Table name.
    • yaml_file_name: YAML file name with DB configuration.
    • clause: SQL clause for record deletion.
    • date_col: Column name containing date data.
    • custom_path: Path to directory containing the YAML file.

Example:

update(df, "test_db", "user_data", clause="user_id > 10")

sql_query

Executes a SELECT SQL query and returns a DataFrame.

  • Parameters:
    • database: Database name.
    • sql: SQL SELECT statement.
    • yaml_file_name: YAML file name with DB configuration.
    • custom_path: Optional path to directory containing the YAML file.

Example:

result_df = sql_query("test_db", "SELECT * FROM users")

Contact Information

For any questions or suggestions regarding the toolkit, please contact us at:

Project links

Key dates

PyPI data
Data sourced directly from PyPI's database.
  • Released:
Latest release

2 maintainers

PyPI data
Data sourced directly from PyPI's database.
Avatar for HanfanC from gravatar.com HanfanC Avatar for lucy_zhou_12138 from gravatar.com lucy_zhou_12138

Credits

Author: Hanfan Chen

License

MIT License

Classifiers

Report project as malware

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

datarecipe-2.1.1.tar.gz (13.4 kB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

datarecipe-2.1.1-py3-none-any.whl (12.9 kB view details)

Uploaded Python 3

File details

Details for the file datarecipe-2.1.1.tar.gz.

File metadata

  • Download URL: datarecipe-2.1.1.tar.gz
  • Upload date:
  • Size: 13.4 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.2.0 CPython/3.12.7

File hashes

Hashes for datarecipe-2.1.1.tar.gz
Algorithm Hash digest
SHA256 a854d21de54be64fd1562b7ad422ae9fb50471344d7745a3de6b20d071c5dd88
MD5 0b3c61234315f1585f87a079aafd61b3
BLAKE2b-256 17857217b6e159304611c41c496fb9d4142836ba108a3e12866f3c6443e5f014

See more details on using hashes here.

File details

Details for the file datarecipe-2.1.1-py3-none-any.whl.

File metadata

  • Download URL: datarecipe-2.1.1-py3-none-any.whl
  • Upload date:
  • Size: 12.9 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.2.0 CPython/3.12.7

File hashes

Hashes for datarecipe-2.1.1-py3-none-any.whl
Algorithm Hash digest
SHA256 cdb159379638b7116cfb2a91dc2aba7503552f3479b1fb44ddbfec31556be50b
MD5 b87c2f291ddd0ef04e5380af6a684322
BLAKE2b-256 097c6c6a6a3b3b81e056c6b729bc8be0ec70e496f91f0ebc6db8c696130c50f8

See more details on using hashes here.

Release history Release notifications | RSS feed

This release

2.1.1 This release

2 files

2.1.0

2 files

2.0.8

2 files

2.0.7

2 files

2.0.6

2 files

2.0.5

2 files

2.0.4

2 files

2.0.3

2 files

2.0.2

2 files

2.0.1

2 files

1.2.13

2 files

1.2.12

2 files

1.2.11

2 files

1.2.10

1 file

1.2.9

2 files

1.2.8

2 files

1.2.7

2 files

1.2.6

2 files

1.2.5

2 files

1.2.4

2 files

1.2.3

2 files

1.1.2

2 files

1.1.1

2 files

1.0.11

2 files

1.0.10

2 files

1.0.9

2 files

1.0.8

2 files

1.0.7

2 files

1.0.6

2 files

1.0.5

2 files

1.0.4

2 files

1.0.3

2 files

1.0.2

2 files

1.0.1

2 files

0.0.13

2 files

0.0.12

2 files

0.0.9

2 files

0.0.8

2 files

0.0.7

2 files

0.0.6

2 files

0.0.5

2 files

0.0.4

2 files

0.0.3

2 files

0.0.2

2 files

0.0.1

2 files

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page