cloupy 1.0.0

The package allows to download, process and visualize climatological data from reliable sources

cloupy is a Python package for climatological data downloading, processing and visualizing. The main goal of the package is to help its author in writing a BA thesis. The package is well adapted to academic work - used data sources are reliable and graphs are easy to modify. What's more, cloupy is intuitive and really easy to use, so even users who are not related to the climatological environment should have no problems with the library usage.

Installation

Run the following command on the command line.

Windows:

pip install cloupy

Linux:

pip3 install cloupy

Usage/Examples

1. Import cloupy library

import cloupy as cl

2. Download climatological data for a single station and for the whole country from the WMO database

single_station = cl.d_wmo_data(station_name='TOKYO', elements_to_scrape=['temp', 'preci'])
whole_country = cl.d_wmo_data(station_name='couJAPAN', elements_to_scrape=['temp', 'preci'])

3. Download climatological data for the station in Poznań (WMO ID: 12330) from the IMGW database and draw a Walter-Lieth diagram

wl = cl.g_WalterLieth(station_name='POZNAŃ')
wl.d_imgw_data(years_range=range(1966, 2020))
wl.draw()

4. How does the graph look like?

5. Download data, set as global dataframe and draw a Walter-Lieth diagram based on the global dataframe

global_df = cl.d_imgw_data(
    interval='monthly', 
    stations_kind='synop', 
    years_range=range(1966, 2020)
    )
cl.set_global_df(global_df)

wl = cl.g_WalterLieth('WARSZAWA')
wl.import_global_df(columns_order='imgw_monthly')
wl.draw()

6. The graph style that better fits a scientific article

cl.choose_diagStyle('retro')
wl.draw()

7. Select which graph elements are to be drawn

As you can see, the graph for POZNAŃ displays information about the coordinates, while the graphs for WARSZAWA do not. The coordinates for POZNAŃ are automatically imported when the wl.d_imgw_data() method is used. However, when we import data from the global dataframe for WARSZAWA, the wl.import_global_df() method does not add coordinates automatically and the coordinates have to be added manually when the cl.g_WalterLieth() object is being created. In our case, for the graph for WARSZAWA it would be:

cl.g_WalterLieth(station_name='WARSZAWA', lat=52.2, lon=21.0, elevation=100)

Now the cl.draw() method will display the coordinates box. So, if the cl.g_WalterLieth() object does not find the unnecessary data, it will just simply not display missing values. What is more, you can manually decide which elements on the graph are to be drawn.

The wl.draw() method takes several arguments that let you to decide which element on the graph will be displayed. For example, if you do not want to display the title (which actually is the station name), the yearly means box and the bottom freeze rectangles, you have to pass the following arguments to the wl.draw() method:

wl.draw(title_text=False, yearly_means_box=False, freeze_rectangles=False)

8. Provide drawing data manually

cloupy graphs can be drawn from the data provided manually. Every graph has its required data structure which must be preserved in the pandas.DataFrame() object. For a Walter-Lieth graph, the pandas.DataFrame() object must contain 5 or 6 columns, depending on the data interval (5 for a monthly interval, 6 for a daily interval). Data can be passed to the dataframe argument in the cl.g_WalterLieth() object. For example, the process might look like this:

import pandas as pd
data = pd.DataFrame(
    {
        'months': [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12],
        'temp': [-2, -1, 0, 7, 15, 18, 19, 20, 18, 14, 8, 3],
        'preci': [50, 25, 55, 60, 70, 80, 90, 80, 68, 50, 45, 49],
        'max_temp': [10, 15, 17, 18, 19, 20, 35, 34, 25, 20, 15, 10],
        'min_temp': [-36, -29, -20, -15, -5, -1, 1, 2, -1, -4, -18, -22]
                            }
)
wl = cl.g_WalterLieth(station_name='TEST', dataframe=data)
wl.draw()

More detailed information on the required data structure is available in the graphs classes docstrings.

9. Recap of the drawing process

cloupy drawing system is easy and can be summarized as follows:

• create a graph object, provide data for further processing and drawing (follow required data structure)
• optionally, use the graph object methods to download and process data
• use the draw() method to specify which elements of the graph will be displayed. Enjoy the graph!

Brief Documentation (the most important functions and classes)

DATA PROCESSING FUNCTIONS/CLASSES

• set_global_df(...) -> set global data frame from which data can be imported in any time.
• read_global_df(...) -> return the global data frame as pandas.DataFrame.
• DataFrame(...) -> create pandas.DataFrame object.

DATA SCRAPING FUNCTIONS

• d_imgw_data(...) -> download IMGW data files from the IMGW database and return it as one merged pd.DataFrame.
• i_imgw_get_file_formats(...) -> return the available file formats for the given 'interval' and 'stations_kind' in the IMGW database (different file formats contain different data)
• i_imgw_serach_keywords_in_columns(...) -> search for the given keywords in the column names and return a dictionary with the file formats in which the keywords were found
• d_wmo_data(...) -> download climatological data for specified station/stations from the WMO website.
• i_wmo_get_stations(...) -> return pandas.DataFrame with WMO stations information (WMO ids, coordinates, etc.).
• i_wmo_search_near_station(...) -> return the nearest stations from WMO database for the given coordinates.

DATA VISUALIZATION FUNCTIONS

• choose_diagStyle(...) -> choose global style for diagrams.
• change_diagStyle_params(...) -> change global parameters for drawing diagrams.
• save_graph(...) -> save created graph (this is the function 'savefig' from the matplotlib library).

DATA VISUALIZATION CLASSES

Note that every class for drawing diagrams contains some of the above functions as its methods (for data scraping and processing)

• g_WalterLieth(...) -> create a WalterLieth object where data for drawing a Walter-Lieth diagram can be downloaded, modified, manually provided.

More detailed documentation for every function, class and method is available in the cloupy's Python files

Future Features

• more climatological data processing functions (e.g. completing missing data)
• more climatological graphs
• drawing maps (e.g. interpolation maps)

Dependencies

Packages/libraries:

• Pandas version: 1.1.4; 1.3.4 or higher
• Matplotlib version: 3.3.2; 3.4.3 or higher
• Requests version: 2.24.0; 2.26.0 or higher
• bs4 version: 4.9.3 or higher
• Numpy version: 1.19.4; 1.21.4 or higher
• Pytest version: 6.2.5 or higher (only for running tests)
• Mock version: 4.0.3 or higher (only for running tests)

Python version: 3.8.2; 3.9.6

OS: Windows; Linux

All of the above versions of packages/Python have been tested. Note that cloupy should also be compatible with the versions between mentioned (e.g. cloupy should work fine on any Pandas version between 1.1.4 and 1.3.4; any Python version between 3.8.2 and 3.9.6). However, it has not been tested and it is recommended to use the most recent version of the packages/libraries. At this moment, Python versions after 3.9.6 do not compile with some of the required packages, so Python 3.9.6 is recommended for using cloupy.

Running Tests

To run tests, run the following command from the root directory:

# change directory to the cloupy root directory, for example:
cd c:\Python\Lib\site-packages\cloupy

python -m pytest 

If you want to run only unit/integration testing:

python -m pytest test\test_unit

python -m pytest test\test_integration

Note that the integration testing may take some time (during the process, data downloading functions are being tested, so duration depends on several factors). The process usually takes about 9-10 minutes.

If you want to skip data downloading tests, run the following command:

python -m pytest -k "not downloading"

Support

For support, please contact me via email: [e-mail]

If you want to report a bug, see CONTRIBUTING.md

License

cloupy is licensed under MIT.

Author

@Kamil Grala