Python module able to download a file from FTP and subset it using time-range,bounding-box,variables and depths
FTPsubsetMO - FTP subsetting service
I developed this software because motivated to improve my efficiency and productivity. It is just an attemp/idea/prototype and it is not fully optimased or considered stable.
Be aware that:
The tool is in development so it can be possible find bugs, errors and imprecisions. Please to report them if you find one.
In the Database just the MY datasets are avaiable (still missing NRT datasets).
For more detailed information about the MULTI YEAR datasets please to look the MY_datasets file.
Python software which is able to download files over FTP protocol and subset the files retrieved by parameters as time-range, bounding box, variables and single/range Depth levels (below the GUI interface displayed on a Linux system).
Together with this tool is distribuited a database which store all the information needed to download the files from each datasets (type of data-set (NRT/MY), time steps (DAILY/MONTLY) and other two parameters needed to correctly identify and select the files prior the download. The key value to retrive such information is the FTP URL of the targeted dataset. It was ideated and implemented by me to adress the download automatization. However I really push the Producers to uniform all the data_structure/file_names and Metadata info (which will make easier the database creation/updates).
After the download which uses the python module "ftplib" (The files are downloaded in the same directory where the tool run) all the subsequent analyses are mainly performed with xarray (another python module). Below the full list of dependencies required (They are installed automatically during the installation if not avaiable in the python environment used):
- <input type="checkbox" checked="" disabled="" /> xarray
- <input type="checkbox" checked="" disabled="" /> ftputil>=3.4
- <input type="checkbox" checked="" disabled="" /> netCDF4
- <input type="checkbox" checked="" disabled="" /> pandas
- <input type="checkbox" checked="" disabled="" /> datetime
- <input type="checkbox" checked="" disabled="" /> os
- <input type="checkbox" checked="" disabled="" /> json
- <input type="checkbox" checked="" disabled="" /> hdf5
- <input type="checkbox" checked="" disabled="" /> h5py
- <input type="checkbox" checked="" disabled="" /> h5netcdf
What the user needs to input?
- CMEMS personal login credential
- FTP Link of the dataset (Our key value to extract from the data-base all the parameters needed to make the Tool works) as example below:
- Time range
- Date start
- Date end
Date format as YYYY-MM-DD also in the case of the MONTHLY dataset where the term "DD" can be set to any real value.
Geographic bounding box (if interested to subset by geographic area)
Variables name (if interested in extract a selection rather than all)
Depths information parameter values (if interested in a SINGLE/RANGE or all the depths)
Python module/script installation:
Just type in the terminal/command-prompt:
pip install FTPsubsetMO
After that you are free to decide if you want to use the GUI interface or exexute it as script. The latter will allow the maximum customization but as cons it is not very user friendly.
To use the GUI interface just type on terminal/command-prompt:
After that just type on the pop upped screen the parameters asked/required. Finally just click on the red download button to start the download/subsetting process.
To use the program as a script and then be able to be free in look/modify/customise the code please to:
Open the Terminal/command_prompt in the location where you desire download the files or anyway have the script
Activate your python environment and import the module:
from FTPsubsetMO import script
- Run the function "script" as follow:
The above function will allow you to add, in the path folder where you run the command, the files needed to run the subsetting process in a pure scripting way. "FTPsubsetMO.py" is the only file to modify based on your data request needs. The script's inputs are highlighted with "". More information can be found as form of comments in FTPsubsetMO.py script.
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
|Filename, size||File type||Python version||Upload date||Hashes|
|Filename, size FTPsubsetMO-0.5.6-py3-none-any.whl (193.4 kB)||File type Wheel||Python version py3||Upload date||Hashes View|
|Filename, size FTPsubsetMO-0.5.6.tar.gz (8.8 kB)||File type Source||Python version None||Upload date||Hashes View|
Hashes for FTPsubsetMO-0.5.6-py3-none-any.whl