Programming language to create machine learning pipelines.
Project description
plAI
Plai is a programming language to create machine learning pipelines with focus on data treatment, validation, and generation of integration tests to ensure more confiability to machine learning systems on production.
Examples
# This is a commnt
# Import statements
import math
from math import cos
# To get a specific column of the main dataset on
# a pipeline call:
# .colname or ."colname"
# Function definition
def fn(x: int):
return 2 * x + 1
# Inline function definition
fn(x: int) = 2 * x + 1
# Specifing columns type of a dataset
#
# Expression can be used to specify a certain type
# to all columns that match the pattern
type T = {
timestamp: datetime,
name: str,
num*: float64,
}
# Pipeline definition
#
# Output of each expression became the input of the next
pipeline alt(df: T):
foo(.name)
bar(.timestamp)
# Operation that must be applied term a term on a dataset
# must use the operator `$` when calling the column
# if there is no specification of target column
# the result will be loaded to the column being used on the operation
# if there is more than one column being used a target column must be specified
$.name + 'foo'
# To specify a column as target of an operation use the operator `as`
$.name + 'bar' as barname
# Exemple of pipeline
#
pipeline main(df: T, df2):
# Drop timestamp column
drop(.timestamp)
bar($.name + '-foo') as name
# Create new column
$.name + '-' + $.country + df2.x as id
# Copy column
$.id as id2
fn(.name)
dropna(.foo)
merge df2 on x
# Calling another pipeline
alt
Development
- Install the dependencies by running the command on the root folder of the project:
pip install -r requirements.txt
- To run all the tests execute:
pytest tests
To run a specific test execute:
# For a specific test file
pytest tests/test_grammar.py
# For a specific test class
pytest tests/test_grammar.py::TestBasicTokens
# For a specific tests method
pytest tests/test_grammar.py::TestBasicTokens::test_token_number
- To run the interactive terminal execute on the root folder:
python -m plai
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
plAI-0.0.0.tar.gz
(8.3 kB
view hashes)
Built Distribution
plAI-0.0.0-py3-none-any.whl
(11.9 kB
view hashes)