A Python package to access ML datasets (UCI, Kaggle, synthetic, etc.) in a normalized format.

Example real-life datasets

Loading the data

>>> from kxy_datasets.uci_regressions import AirQuality
>>> air_quality = AirQuality()
>>> print(air_quality.name)
UCIAirQuality

Retrieving target and explanatory variables as numpy arrays

>>> y, x = air_quality.x, air_quality.y
>>> print(air_quality.x.shape)
(8991, 14)
>>> print(air_quality.y.shape)
(8991, 1)
>>> print(len(air_quality))
8991

Reading the problem type (classification/regression)

>>> print(air_quality.problem_type)
regression

Retrieving the data as a dataframe

>>> air_quality.df
       Date  Time  CO(GT)  PT08.S1(CO)  NMHC(GT)  C6H6(GT)  PT08.S2(NMHC)  NOx(GT)  PT08.S3(NOx)  NO2(GT)  PT08.S4(NO2)  PT08.S5(O3)     T    RH      AH
0     273.0    18     2.6       1360.0     150.0      11.9         1046.0    166.0        1056.0    113.0        1692.0       1268.0  13.6  48.9  0.7578
1     273.0    19     2.0       1292.0     112.0       9.4          955.0    103.0        1174.0     92.0        1559.0        972.0  13.3  47.7  0.7255
2     273.0    20     2.2       1402.0      88.0       9.0          939.0    131.0        1140.0    114.0        1555.0       1074.0  11.9  54.0  0.7502
3     273.0    21     2.2       1376.0      80.0       9.2          948.0    172.0        1092.0    122.0        1584.0       1203.0  11.0  60.0  0.7867
4     273.0    22     1.6       1272.0      51.0       6.5          836.0    131.0        1205.0    116.0        1490.0       1110.0  11.2  59.6  0.7888
...     ...   ...     ...          ...       ...       ...            ...      ...           ...      ...           ...          ...   ...   ...     ...
9352  456.0    10     3.1       1314.0    -200.0      13.5         1101.0    472.0         539.0    190.0        1374.0       1729.0  21.9  29.3  0.7568
9353  456.0    11     2.4       1163.0    -200.0      11.4         1027.0    353.0         604.0    179.0        1264.0       1269.0  24.3  23.7  0.7119
9354  456.0    12     2.4       1142.0    -200.0      12.4         1063.0    293.0         603.0    175.0        1241.0       1092.0  26.9  18.3  0.6406
9355  456.0    13     2.1       1003.0    -200.0       9.5          961.0    235.0         702.0    156.0        1041.0        770.0  28.3  13.5  0.5139
9356  456.0    14     2.2       1071.0    -200.0      11.9         1047.0    265.0         654.0    168.0        1129.0        816.0  28.5  13.1  0.5028

[8991 rows x 15 columns]
>>> air_quality.y_column
'C6H6(GT)'
>>> air_quality.x_columns
['Date', 'Time', 'CO(GT)', 'PT08.S1(CO)', 'NMHC(GT)', 'PT08.S2(NMHC)', 'NOx(GT)', 'PT08.S3(NOx)', 'NO2(GT)', 'PT08.S4(NO2)', 'PT08.S5(O3)', 'T', 'RH', 'AH']

UCI classification datasets

>>> from kxy_datasets.uci_classifications import BankNote

Kaggle regression datasets

>>> from kxy_datasets.kaggle_regressions import HousePricesAdvanced

Kaggle classification datasets

>>> from kxy_datasets.kaggle_classifications import Titanic

Example synthetic datasets

Synthetic regression datasets (with known theoretical-best performance achievable)

>>> from kxy_datasets.synthetic_regressions import SQRTABSReg

Synthetic classification datasets (with known theoretical-best performance achievable)

>>> from kxy_datasets.synthetic_classifications import EllipticalBoundaryBin

Data valuation and model-free variable selection with the kxy package

Data valuation

>>> from kxy_datasets.kaggle_classifications import Titanic
>>> titanic = Titanic()
>>> titanic.data_valuation()
[====================================================================================================] 100% ETA: 0s   
  Achievable R-Squared Achievable Log-Likelihood Per Sample Achievable Accuracy
0                 0.53                            -2.89e-01                0.92

Model-free variable selection

>>> titanic.variable_selection()
[====================================================================================================] 100% ETA: 0s   
                    Variable Running Achievable R-Squared Running Achievable Accuracy
Selection Order                                                                      
0                No Variable                         0.00                        0.62
1                        Sex                         0.26                        0.79
2                PassengerId                         0.27                        0.79
3                     Pclass                         0.37                        0.84
4                      Parch                         0.37                        0.84
5                        Age                         0.48                        0.90
6                   Embarked                         0.48                        0.90
7                      SibSp                         0.53                        0.92
8                       Fare                         0.53                        0.92