StationsET

class weatherdb.stations.StationsET

Bases: StationsBaseTET

A class to work with and download potential Evapotranspiration (VPGB) data for several stations.

count_holes(stids='all', **kwargs)

Count holes in timeseries depending on there length.

Parameters:

• stids (string or list of int, optional) – The Stations to return. Can either be “all”, for all possible stations or a list with the Station IDs. The default is “all”.

• **kwargs (dict, optional)

• **kwargs –

  This is a list of parameters, that is supported by the StationBase.count_holes method.

  Furthermore the kwargs are passed to the get_stations method.

  possible values are:

  • weekslist, optional

    A list of hole length to count. Every hole longer than the duration of weeks specified is counted. The default is [2, 4, 8, 12, 16, 20, 24]

  • kindstr

    The kind of the timeserie to analyze. Should be one of [‘raw’, ‘qc’, ‘filled’]. For N also “corr” is possible. Normally only “raw” and “qc” make sense, because the other timeseries should not have holes.

  • periodTimestampPeriod or (tuple or list of datetime.datetime or None), optional

    The minimum and maximum Timestamp for which to analyze the timeseries. If None is given, the maximum and minimal possible Timestamp is taken. The default is (None, None).

  • between_meta_periodbool, optional

    Only check between the respective period that is defined in the meta table. If “qc” is chosen as kind, then the “raw” meta period is taken. The default is True.

  • crop_periodbool, optional

    should the period get cropped to the maximum filled period. This will result in holes being ignored when they are at the end or at the beginning of the timeserie. If period = (None, None) is given, then this parameter is set to True. The default is False.

Returns:

A Pandas Dataframe, with station_id as index and one column per week. The numbers in the table are the amount of NA-periods longer than the respective amount of weeks.

Return type:

pandas.DataFrame

Raises:

ValueError – If the input parameters were not correct.

download_meta()

Download the meta file(s) from the CDC server.

Returns:

The meta file from the CDC server. If there are several meta files on the server, they are joined together.

Return type:

geopandas.GeoDataFrame

fillup(only_real=False, stids='all', **kwargs)

Fill up the quality checked data with data from nearby stations to get complete timeseries.

Parameters:

• only_real (bool, optional) – Whether only real stations are computed or also virtual ones. True: only stations with own data are returned. The default is True.

• stids (string or list of int, optional) – The Stations for which to compute. Can either be “all”, for all possible stations or a list with the Station IDs. The default is “all”.

• do_mp (bool, optional) – Should the method be done in multiprocessing mode? If False the methods will be called in threading mode. Multiprocessing needs more memory and a bit more initiating time. Therefor it is only usefull for methods with a lot of computation effort in the python code. If the most computation of a method is done in the postgresql database, then threading is enough to speed the process up. The default is False.

• **kwargs (dict, optional) – The additional keyword arguments for the _run_method and get_stations method

Raises:

ValueError – If the given stids (Station_IDs) are not all valid.

get_df(stids, **kwargs)

Get a DataFrame with the corresponding data.

Parameters:

• stids (string or list of int, optional) – The Stations for which to compute. Can either be “all”, for all possible stations or a list with the Station IDs. The default is “all”.

• **kwargs (optional keyword arguments) – Those keyword arguments are passed to the get_df function of the station class. Possible parameters are period, agg_to, kinds. Furthermore the kwargs are passed to the get_stations method.

Returns:

A DataFrame with the timeseries for the selected stations, kind(s) and the given period. If multiple columns are selected, the columns in this DataFrame is a MultiIndex with the station IDs as first level and the kind as second level.

Return type:

pd.Dataframe

get_meta(infos=['station_id', 'filled_from', 'filled_until', 'geometry'], stids='all', only_real=True)

Get the meta Dataframe from the Database.

Parameters:

• infos (list or str, optional) – A list of information from the meta file to return If “all” than all possible columns are returned, but only one geometry column. The default is: [“Station_id”, “filled_from”, “filled_until”, “geometry”]

• only_real (bool, optional) – Whether only real stations are returned or also virtual ones. True: only stations with own data are returned. The default is True.

Returns:

The meta DataFrame.

Return type:

pandas.DataFrame or geopandas.GeoDataFrae

classmethod get_meta_explanation(infos='all')

Get the explanations of the available meta fields.

Parameters:

infos (list or string, optional) – The infos you wish to get an explanation for. If “all” then all the available information get returned. The default is “all”

Returns:

a pandas Series with the information names as index and the explanation as values.

Return type:

pd.Series

get_quotient(kinds_num, kinds_denom, stids='all', return_as='df', **kwargs)

Get the quotient of multi-annual means of two different kinds or the timeserie and the multi annual raster value.

$quotient = overline{ts}_{kind_num} / overline{ts}_{denom}$

Parameters:

• kinds_num (list of str or str) – The timeseries kinds of the numerators. Should be one of [‘raw’, ‘qc’, ‘filled’]. For precipitation also “corr” is possible.

• kinds_denom (list of str or str) – The timeseries kinds of the denominator or the multi annual raster key. If the denominator is a multi annual raster key, then the result is the quotient of the timeserie and the raster value. Possible values are: - for timeserie kinds: ‘raw’, ‘qc’, ‘filled’ or for precipitation also “corr”. - for raster keys: ‘hyras’, ‘dwd’ or ‘regnie’, depending on your defined raster files.

• stids (list of Integer) – The stations IDs for which to compute the quotient.

• return_as (str, optional) – The format of the return value. If “df” then a pandas DataFrame is returned. If “json” then a list with dictionaries is returned.

• **kwargs (dict, optional) – The additional keyword arguments are passed to the get_stations method.

Returns:

The quotient of the two timeseries as DataFrame or list of dictionaries (JSON) depending on the return_as parameter. The default is pd.DataFrame.

Return type:

pandas.DataFrame or list of dict

Raises:

ValueError – If the input parameters were not correct.

get_stations(only_real=True, stids='all', skip_missing_stids=False, **kwargs)

Get a list with all the stations as Station-objects.

Parameters:

• only_real (bool, optional) – Whether only real stations are returned or also virtual ones. True: only stations with own data are returned. The default is True.

• stids (string or list of int, optional) – The Stations to return. Can either be “all”, for all possible stations or a list with the Station IDs. The default is “all”.

• skip_missing_stids (bool, optional) – Should the method skip the missing stations from input stids? If False, then a ValueError is raised if a station is not found. The default is False.

• **kwargs (dict, optional) – The additional keyword arguments aren’t used in this method.

Returns:

returns a list with the corresponding station objects.

Return type:

Station-object

Raises:

ValueError – If the given stids (Station_IDs) are not all valid.

last_imp_fillup(stids='all', do_mp=False, **kwargs)

Do the gap filling of the last import.

Parameters:

• do_mp (bool, optional) – Should the method be done in multiprocessing mode? If False the methods will be called in threading mode. Multiprocessing needs more memory and a bit more initiating time. Therefor it is only usefull for methods with a lot of computation effort in the python code. If the most computation of a method is done in the postgresql database, then threading is enough to speed the process up. The default is False.

• stids (string or list of int, optional) – The Stations for which to compute. Can either be “all”, for all possible stations or a list with the Station IDs. The default is “all”.

• **kwargs (dict, optional) – The additional keyword arguments for the _run_method and get_stations method

last_imp_quality_check(stids='all', do_mp=False, **kwargs)

Do the quality check of the last import.

Parameters:

• do_mp (bool, optional) – Should the method be done in multiprocessing mode? If False the methods will be called in threading mode. Multiprocessing needs more memory and a bit more initiating time. Therefor it is only usefull for methods with a lot of computation effort in the python code. If the most computation of a method is done in the postgresql database, then threading is enough to speed the process up. The default is False.

• stids (string or list of int, optional) – The Stations for which to compute. Can either be “all”, for all possible stations or a list with the Station IDs. The default is “all”.

• **kwargs (dict, optional) – The additional keyword arguments for the _run_method and get_stations method

quality_check(period=(None, None), only_real=True, stids='all', do_mp=False, **kwargs)

Quality check the raw data for a given period.

Parameters:

• period (tuple or list of datetime.datetime or None, optional) – The minimum and maximum Timestamp for which to get the timeseries. If None is given, the maximum or minimal possible Timestamp is taken. The default is (None, None).

• stids (string or list of int, optional) – The Stations for which to compute. Can either be “all”, for all possible stations or a list with the Station IDs. The default is “all”.

• do_mp (bool, optional) – Should the method be done in multiprocessing mode? If False the methods will be called in threading mode. Multiprocessing needs more memory and a bit more initiating time. Therefor it is only usefull for methods with a lot of computation effort in the python code. If the most computation of a method is done in the postgresql database, then threading is enough to speed the process up. The default is False.

• **kwargs (dict, optional) – The additional keyword arguments for the _run_method and get_stations method

update(only_new=True, **kwargs)

Make a complete update of the stations.

Does the update_raw, quality check and fillup of the stations.

Parameters:

only_new (bool, optional) – Should a only new values be computed? If False: The stations are updated for the whole possible period. If True, the stations are only updated for new values. The default is True.

update_ma_raster(stids='all', do_mp=False, **kwargs)

Update the multi annual raster values for the stations.

Get a multi annual value from the corresponding raster and save to the multi annual table in the database.

Parameters:

• stids (string or list of int, optional) – The Stations for which to compute. Can either be “all”, for all possible stations or a list with the Station IDs. The default is “all”.

• do_mp (bool, optional) – Should the method be done in multiprocessing mode? If False the methods will be called in threading mode. Multiprocessing needs more memory and a bit more initiating time. Therefor it is only usefull for methods with a lot of computation effort in the python code. If the most computation of a method is done in the postgresql database, then threading is enough to speed the process up. The default is False.

• **kwargs (dict, optional) – The additional keyword arguments for the _run_method and get_stations method

Raises:

ValueError – If the given stids (Station_IDs) are not all valid.

update_ma_timeseries(kind, stids='all', do_mp=False, **kwargs)

Update the multi annual timeseries values for the stations.

Get a multi annual value from the corresponding timeseries and save to the database.

Parameters:

• kind (str or list of str) – The timeseries data kind to update theire multi annual value. Must be a column in the timeseries DB. Must be one of “raw”, “qc”, “filled”. For the precipitation also “corr” is valid.

• stids (string or list of int, optional) – The Stations for which to compute. Can either be “all”, for all possible stations or a list with the Station IDs. The default is “all”.

• do_mp (bool, optional) – Should the method be done in multiprocessing mode? If False the methods will be called in threading mode. Multiprocessing needs more memory and a bit more initiating time. Therefor it is only usefull for methods with a lot of computation effort in the python code. If the most computation of a method is done in the postgresql database, then threading is enough to speed the process up. The default is False.

• **kwargs (dict, optional) – The additional keyword arguments for the _run_method and get_stations method

Raises:

ValueError – If the given stids (Station_IDs) are not all valid.

update_meta(stids='all', **kwargs)

Update the meta table by comparing to the CDC server.

The “von_datum” and “bis_datum” is ignored because it is better to set this by the filled period of the stations in the database. Often the CDC period is not correct.

Parameters:

stids (string or list of int, optional) – The Stations for which to compute. Can either be “all”, for all possible stations or a list with the Station IDs. The default is “all”.

update_period_meta(stids='all', **kwargs)

Update the period in the meta table of the raw data.

Parameters:

• stids (string or list of int, optional) – The Stations for which to compute. Can either be “all”, for all possible stations or a list with the Station IDs. The default is “all”.

• **kwargs (dict, optional)

• **kwargs – The additional keyword arguments are passed to the get_stations method.

Raises:

ValueError – If the given stids (Station_IDs) are not all valid.

update_raw(only_new=True, only_real=True, stids='all', remove_nas=True, do_mp=True, **kwargs)

Download all stations data from CDC and upload to database.

Parameters:

• only_new (bool, optional) – Get only the files that are not yet in the database? If False all the available files are loaded again. The default is True

• only_real (bool, optional) – Whether only real stations are tried to download. True: only stations with a date in raw_from in meta are downloaded. The default is True.

• stids (string or list of int, optional) – The Stations to return. Can either be “all”, for all possible stations or a list with the Station IDs. The default is “all”.

• do_mp (bool, optional) – Should the method be done in multiprocessing mode? If False the methods will be called in threading mode. Multiprocessing needs more memory and a bit more initiating time. Therefor it is only usefull for methods with a lot of computation effort in the python code. If the most computation of a method is done in the postgresql database, then threading is enough to speed the process up. The default is True.

• remove_nas (bool, optional) – Remove the NAs from the downloaded data before updating it to the database. This has computational advantages. The default is True.

• **kwargs (dict, optional) – The additional keyword arguments for the _run_method and get_stations method

Raises:

ValueError – If the given stids (Station_IDs) are not all valid.