Skip to main content

Analysis Tool Box

Description

Analysis Tool Box (i.e. "analysistoolbox") is a collection of tools in Python for data collection and processing, statisitics, analytics, and intelligence analysis.

Getting Started

To install the package, run the following command in the root directory of the project:

pip install analysistoolbox

Visualizations are created using the matplotlib and seaborn libraries. While you can select whichever seaborn style you'd like, the following Seaborn style tends to get the best looking plots:

sns.set(
    style="white",
    font="Arial",
    context="paper"
)

Usage

There are many modules in the analysistoolbox package, each with their own functions. The following is a list of the modules:

  • Calculus
  • Data collection
  • Data processing
  • Descriptive analytics
  • File management
  • Hypothesis testing
  • Linear algebra
  • Predictive analytics
  • Statistics
  • Visualizations

Calculus

FindDerivative

The FindDerivative function calculates the derivative of a given function. It uses the sympy library, a Python library for symbolic mathematics, to perform the differentiation. The function also has the capability to print the original function and its derivative, return the derivative function, and plot both the original function and its derivative.

# Load the FindDerivative function from the Calculus submodule
from analysistoolbox.calculus import FindDerivative
import sympy

# Set Seaborn style
sns.set(
    style="white",
    font="Arial",
    context="paper"
)

# Define a symbolic variable
x = sympy.symbols('x')

# Define a function
f_of_x = x**3 + 2*x**2 + 3*x + 4

# Use the FindDerivative function
FindDerivative(
    f_of_x, 
    print_functions=True, 
    return_derivative_function=True, 
    plot_functions=True
)

FindLimitOfFunction

The FindLimitOfFunction function finds the limit of a function at a specific point and optionally plot the function and its tangent line at that point. The script uses the matplotlib and numpy libraries for plotting and numerical operations respectively.

# Import the necessary libraries
import numpy as np
from analysistoolbox.calculus import FindLimitOfFunction

# Set Seaborn style
sns.set(
    style="white",
    font="Arial",
    context="paper"
)

# Define a symbolic variable
x = sympy.symbols('x')

# Define a function
f_of_x = np.sin(x) / x

# Use the FindLimitOfFunction function
FindLimitOfFunction(
    f_of_x, 
    point=0, 
    step=0.01, 
    plot_function=True, 
    x_minimum=-10, 
    x_maximum=10, 
    n=1000, 
    tangent_line_window=1
)

FindMinimumSquareLoss

The FindMinimumSquareLoss function calculates the minimum square loss between observed and predicted values. This function is often used in machine learning and statistics to measure the average squared difference between the actual and predicted outcomes.

# Import the necessary libraries
from analysistoolbox.calculus import FindMinimumSquareLoss

# Set Seaborn style
sns.set(
    style="white",
    font="Arial",
    context="paper"
)

# Define observed and predicted values
observed_values = [1, 2, 3, 4, 5]
predicted_values = [1.1, 1.9, 3.2, 3.7, 5.1]

# Use the FindMinimumSquareLoss function
minimum_square_loss = FindMinimumSquareLoss(
    observed_values, 
    predicted_values, 
    show_plot=True
)

print(f"The minimum square loss is: {minimum_square_loss}")

PlotFunction

The PlotFunction function plots a mathematical function of x. It takes a lambda function as input and allows for customization of the plot.

# Import the necessary libraries
from analysistoolbox.calculus import PlotFunction

# Set Seaborn style
sns.set(
    style="white",
    font="Arial",
    context="paper"
)

# Define the function to plot
f_of_x = lambda x: x**2

# Plot the function with default settings
PlotFunction(f_of_x)

Data Collection

ExtractTextFromPDF

The ExtractTextFromPDF function extracts text from a PDF file, cleans it, then saves it to a text file.

# Import the function
from analysistoolbox.data_collection import ExtractTextFromPDF

# Define the path to the PDF file
filepath_to_pdf = "/path/to/your/input.pdf"

# Define the path to the text file
filepath_for_exported_text = "/path/to/your/output.txt"

# Call the function
ExtractTextFromPDF(
    filepath_to_pdf=filepath_to_pdf, 
    filepath_for_exported_text=filepath_for_exported_text, 
    start_page=1, 
    end_page=None
)

FetchPDFFromURL

The FetchPDFFromURL function downloads a PDF file from a URL and saves it to a specified location.

from analysistoolbox.data_collection import FetchPDFFromURL

# URL of the PDF file to download
url = "https://example.com/sample.pdf"

# Name of the file to save the PDF as
filename = "sample.pdf"

# Call the function to download the PDF
FetchPDFFromURL(url, filename)

FetchUSShapefile

The FetchUSShapefile function fetches a geographical shapefile from the TIGER database of the U.S. Census Bureau.

from analysistoolbox.data_collection import FetchUSShapefile

# Fetch the shapefile for the census tracts in King County, Washington, for the 2021 census year
shapefile = FetchUSShapefile(state='WA', county='King', geography='tract', census_year=2021)

# Print the first few rows of the shapefile
print(shapefile.head())

FetchWebsiteText

The FetchWebsiteText function fetches the text from a website and saves it to a text file.

# Import the function from the module
from analysistoolbox.data_collection import FetchWebsiteText

# Define the URL to fetch
url = "https://www.example.com"

# Define the Browserless API key
browserless_api_key = "your_browserless_api_key"

# Call the function
text = FetchWebsiteText(url, browserless_api_key)

# Print the fetched text
print(text)

GetGoogleSearchResults

The GetGoogleSearchResults function fetches Google search results for a given query using the Serper API.

# Import the function
from analysistoolbox.data_collection import GetGoogleSearchResults

# Define a search query
query = "Python programming"

# Call the function with the query
# Make sure to replace 'your_serper_api_key' with your actual Serper API key
results = GetGoogleSearchResults(query, serper_api_key='your_serper_api_key', number_of_results=5, apply_autocorrect=True, display_results=True)

# Print the results
print(results)

GetZipFile

The GetZipFile function downloads a zip file from a url and saves it to a specified folder. It can also unzip the file and print the contents of the zip file.

# Import the function
from analysistoolbox.data_collection import GetZipFile

# URL of the zip file to download
url = "http://example.com/file.zip"

# Path to the folder where the zip file will be saved
path_to_save_folder = "/path/to/save/folder"

# Call the function
GetZipFile(url, path_to_save_folder)

Data Processing

AddDateNumberColumns

The AddDateNumberColumns function adds columns for the year, month, quarter, week, day of the month, and day of the week to a dataframe.

# Import necessary packages
from analysistoolbox.data_processing import AddDateNumberColumns
from datetime import datetime
import pandas as pd

# Create a sample dataframe
data = {'Date': [datetime(2020, 1, 1), datetime(2020, 2, 1), datetime(2020, 3, 1), datetime(2020, 4, 1)]}
df = pd.DataFrame(data)

# Use the function on the sample dataframe
df = AddDateNumberColumns(df, 'Date')

# Print the updated dataframe
print(df)

AddLeadingZeros

The AddLeadingZeros function adds leading zeros to a column. If fixed_length is not specified, the longest string in the column is used as the fixed length. If add_as_new_column is set to True, the new column is added to the dataframe. Otherwise, the original column is updated.

# Import necessary packages
from analysistoolbox.data_processing import AddLeadingZeros
import pandas as pd

# Create a sample dataframe
data = {'ID': [1, 23, 456, 7890]}
df = pd.DataFrame(data)

# Print original dataframe
print("Original dataframe:")
print(df)

# Use the AddLeadingZeros function
df = AddLeadingZeros(df, 'ID', add_as_new_column=True)

# Print updated dataframe
print("Updated dataframe:")
print(df)

AddRowCountColumn

The AddRowCountColumn function adds a column to a dataframe that contains the row number for each row, based on a group (or groups) of columns. The function can also sort the dataframe by a column or columns before adding the row count column.

# Import necessary packages
from analysistoolbox.data_processing import AddRowCountColumn
import pandas as pd

# Create a sample dataframe
data = {
    'Payment Method': ['Check', 'Credit Card', 'Check', 'Credit Card', 'Check', 'Credit Card', 'Check', 'Credit Card'],
    'Transaction Value': [100, 200, 300, 400, 500, 600, 700, 800],
    'Transaction Order': [1, 2, 3, 4, 5, 6, 7, 8]
}
df = pd.DataFrame(data)

# Define the parameters for the function
grouping_variables = ['Payment Method']
order_columns = ['Transaction Order']
ascending_order_args = [True]
row_count_column_name = 'Row Count'

# Call the function
new_df = AddRowCountColumn(df, grouping_variables, order_columns, ascending_order_args, row_count_column_name)

AddTPeriodColumn

The AddTPeriodColumn function adds a T-period column to a dataframe. The T-period column is the number of intervals (e.g., days or weeks) since the earliest date in the dataframe.

# Import necessary libraries
from analysistoolbox.data_processing import AddTPeriodColumn
from datetime import datetime
import pandas as pd

# Create a sample dataframe
data = {
    'date': pd.date_range(start='1/1/2020', end='1/10/2020'),
    'value': range(1, 11)
}
df = pd.DataFrame(data)

# Use the function
df_updated = AddTPeriodColumn(df, 'date', 'days')

# Print the updated dataframe
print(df_updated)

Contributions

To report an issue, request a feature, or contribute to the project, please see the CONTRIBUTING.md file (in progress).

License

This project is licensed under the MIT License - see the LICENSE.md file for details.

Metadata

Release files for analysistoolbox 1.0.4

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for analysistoolbox 1.0.4
File Size Uploaded Provenance Actions
analysistoolbox-1.0.4.tar.gz 97.1 kB Without provenance Details for analysistoolbox-1.0.4.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for analysistoolbox 1.0.4
File Interpreter ABI Platform Actions
analysistoolbox-1.0.4-py3-none-any.whl Python 3 none any Details for analysistoolbox-1.0.4-py3-none-any.whl

Total release size: 271.3 kB

Release files / analysistoolbox-1.0.4.tar.gz

Download URL analysistoolbox-1.0.4.tar.gz
Size 97.1 kB
Tags Source
SHA-256 checksum
How to use checksums
2192b4fb7d358b8bd25cd3a8c23208bac4e28e810f1f218032b5bb8e98f721e3
BLAKE2b-256 checksum
How to use checksums
de87b43a1c160dc7738521905f431e38617293f146cc63d3a183124cf73115b2
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/4.0.2 CPython/3.9.6

Release files / analysistoolbox-1.0.4-py3-none-any.whl

Download URL analysistoolbox-1.0.4-py3-none-any.whl
Size 174.3 kB
Tags Python 3
SHA-256 checksum
How to use checksums
e0862ec1ad05715d2165cf906aadbdddbf7d325b8d6015bf7fcb330b7398e392
BLAKE2b-256 checksum
How to use checksums
419bfc14a404e4b860e7ea510b57c2625245261a48b8c5535a3e0dd562be2b64
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/4.0.2 CPython/3.9.6

Security for analysistoolbox 1.0.4

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Release history Release notifications | RSS feed

This release

1.0.4 This release

2 release files No provenance

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page