Skip to main content

Package for generating and evaluating patterns in quantitative reports

Project description

data-patterns

Pypi Version Build Status Documentation Status License

Package for generating and evaluating data-patterns in quantitative reports

Features

Here is what the package does:

  • Generating and evaluating patterns in structured datasets and exporting to Excel and JSON

  • Transforming generated patterns into XBRL validation rules and Pandas code

  • Evaluating reporting data with data quality rules published by De Nederlandsche Bank (to be provided)

Quick overview

To install the package

pip install data_patterns

To introduce the features of the this package define the following Pandas DataFrame:

df = pd.DataFrame(columns = ['Name',       'Type',             'Assets', 'TV-life', 'TV-nonlife' , 'Own funds', 'Excess'],
                  data   = [['Insurer  1', 'life insurer',     1000,     800,       0,             200,         200],
                            ['Insurer  2', 'non-life insurer', 4000,     0,         3200,          800,         800],
                            ['Insurer  3', 'non-life insurer', 800,      0,         700,           100,         100],
                            ['Insurer  4', 'life insurer',     2500,     1800,      0,             700,         700],
                            ['Insurer  5', 'non-life insurer', 2100,     0,         2200,          200,         200],
                            ['Insurer  6', 'life insurer',     9000,     8800,      0,             200,         200],
                            ['Insurer  7', 'life insurer',     9000,     0,         8800,          200,         200],
                            ['Insurer  8', 'life insurer',     9000,     8800,      0,             200,         200],
                            ['Insurer  9', 'non-life insurer', 9000,     0,         8800,          200,         200],
                            ['Insurer 10', 'non-life insurer', 9000,     0,         8800,          200,         199.99]])
df.set_index('Name', inplace = True)

Start by defining a PatternMiner:

miner = data_patterns.PatternMiner(df)

To generate patterns use the find-function of this object:

df_patterns = miner.find({'name'      : 'equal values',
                          'pattern'   : '=',
                          'parameters': {"min_confidence": 0.5,
                                         "min_support"   : 2}})

The result is a DataFrame with the patterns that were found. The first part of the DataFrame now contains

id

pattern_id

P columns

relation type

Q columns

support

exceptions

confidence

0

equal values

[Own funds]

=

[Excess]

9

1

0.9

1

equal values

[Excess]

=

[Own funds]

9

1

0.9

The miner finds two patterns; the first states that the ‘Own funds’-column is identical to the ‘Excess’-column in 9 of the 10 cases (with a confidence of 90 %, there is one case where the equal-pattern does not hold), and the second pattern is identical to the first but with the columns reversed.

To analyze data with the generated set of data-patterns use the analyze function with the dataframe with the data as input:

df_results = miner.analyze(df)

The result is a DataFrame with the results. If we select result_type = False then the first part of the output contains

index

result_type

pattern_id

P columns

relation type

Q columns

P values

Q values

Insurer 10

False

equal values

[Own funds]

=

[Excess]

[200]

[199.99]

Insurer 10

False

equal values

[Excess]

=

[Own funds]

[199.99]

[200]

Other patterns you can use are ‘>’, ‘<’, ‘<=’, ‘>=’, ‘!=’, ‘sum’, and ‘–>’.

Read the documentation for more features.

History

0.1.0 (2019-10-27)

  • Development release.

0.1.11 (2019-11-6)

  • First release on PyPI.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

data_patterns-0.1.12.tar.gz (24.0 kB view hashes)

Uploaded Source

Built Distribution

data_patterns-0.1.12-py2.py3-none-any.whl (14.7 kB view hashes)

Uploaded Python 2 Python 3

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page