lor-gdp-tools 1.0.1

This is a utility package designed to enable data scientitists and analysts to easily access GDP data within a python environment

Gdp Tools Package

This is a utility package designed to enable data scientists and analysts to easily access GDP data within a python environment

Requirements:

• The data professional should be able to clone the package at the start of new project and when in production
• The package will contain a number of support functions serving the following objectives: -- Accessing data on GDP Base/CIM/Warehouse -- Accessing data in temp storage (LAB) -- Querying that data in SQL -- utlising that data as a Python/PySpark DataFrame -- Storing processed data to the temp storage space (LAB)
• any data access configs should available to be used

Credentials

Contact a member of the Laing O'Rourke Data Science team to set up up your credentials.

USERNAME and PASSWORD need to be entered in as arguments when initailising the tool, for example '''gdp_tools = GlobalDataPlatformTools(USERNAME,PWD)'''

Functionality

Should be used in Azure ML Studio. NO access to GDP locally.

validate_odbc_drivers - check if the correct ODBC connectors are installed, install them as necessary automatically [check_odbc_driver, install_odbc_driver]

setup_odbc_connection - set up the ODBC with your credentials. Happens when class is initalised

query_gdp_to_pd - enter a SQL query as a string, return the data as a pandas dataframe

search_tables - will return a list of all table in the GDP. give it the argument 'source_system' = '[COINS]' to narrow your search for any table with the term 'COINS' in it (for example)