StationPD

class weatherdb.station.StationPD(id, **kwargs)

Bases: StationPBase, StationCanVirtualBase

A class to work with and download daily precipitation data for one station.

Those station data are only downloaded to do some quality checks on the 10 minute data. Therefor there is no special quality check and richter correction done on this data. If you want daily precipitation data, better use the 10 minutes station(StationP) and aggregate to daily values.

Create a Station object.

Parameters:

• id (int) – The stations ID.

• _skip_meta_check (bool, optional) – Should the check if the station is in the database meta file get skiped. Pay attention, when skipping this, because it can lead to problems. This is for computational reasons, because it makes the initialization faster. Is used by the stations classes, because the only initialize objects that are in the meta table. The default is False

Raises:

NotImplementedError – If the class is initiated with a station ID that is not in the database. To prevent this error, set _skip_meta_check=True.

count_holes(weeks=[2, 4, 8, 12, 16, 20, 24], kind='qc', period=(None, None), between_meta_period=True, crop_period=False, **kwargs)

Count holes in timeseries depending on there length.

Parameters:

• weeks (list, optional) – A list of hole length to count. Every hole longer than the duration of weeks specified is counted. The default is [2, 4, 8, 12, 16, 20, 24]

• kind (str) – The kind of the timeserie to analyze. Should be one of [‘raw’, ‘qc’, ‘filled’]. For N also “corr” is possible. Normally only “raw” and “qc” make sense, because the other timeseries should not have holes.

• period (TimestampPeriod or (tuple or list of datetime.datetime or None), optional) – The minimum and maximum Timestamp for which to analyze the timeseries. If None is given, the maximum and minimal possible Timestamp is taken. The default is (None, None).

• between_meta_period (bool, optional) – Only check between the respective period that is defined in the meta table. If “qc” is chosen as kind, then the “raw” meta period is taken. The default is True.

• crop_period (bool, optional) – should the period get cropped to the maximum filled period. This will result in holes being ignored when they are at the end or at the beginning of the timeserie. If period = (None, None) is given, then this parameter is set to True. The default is False.

Returns:

A Pandas Dataframe, with station_id as index and one column per week. The numbers in the table are the amount of NA-periods longer than the respective amount of weeks.

Return type:

pandas.DataFrame

Raises:

ValueError – If the input parameters were not correct.

download_raw(only_new=False)

Download the timeserie from the CDC Server.

This function only returns the timeserie, but is not updating the database.

Parameters:

only_new (bool, optional) – Get only the files that are not yet in the database? If False all the available files are loaded again. The default is False.

Returns:

The Timeseries as a DataFrame with a Timestamp Index.

Return type:

pandas.DataFrame

fillup(period=(None, None), **kwargs)

Fill up missing data with measurements from nearby stations.

Parameters:

• period (TimestampPeriod or (tuple or list of datetime.datetime or None), optional) – The minimum and maximum Timestamp for which to gap fill the timeseries. If None is given, the maximum or minimal possible Timestamp is taken. The default is (None, None).

• **kwargs (dict, optional) – Additional arguments for the fillup function. e.g. p_elev to consider the elevation to select nearest stations. (only for T and ET)

get_coef(other_stid, in_db_unit=False)

Get the regionalisation coefficients due to the height.

Those are the values from the dwd grid, HYRAS or REGNIE grids.

Parameters:

• other_stid (int) – The Station Id of the other station from wich to regionalise for own station.

• in_db_unit (bool, optional) – Should the coefficients be returned in the unit as stored in the database? This is only relevant for the temperature. The default is False.

Returns:

A list of coefficients. For T, ET and N-daily only the the yearly coefficient is returned. For N the winter and summer half yearly coefficient is returned in tuple. None is returned if either the own or other stations multi-annual value is not available.

Return type:

list of floats or None

get_df(kinds, period=(None, None), agg_to=None, nas_allowed=True, add_na_share=False, db_unit=False, sql_add_where=None, **kwargs)

Get a timeseries DataFrame from the database.

Parameters:

• kinds (str or list of str) – The data kinds to update. Must be a column in the timeseries DB. Must be one of “raw”, “qc”, “filled”, “adj”, “filled_by”, “filled_share”. For the precipitation also “qn” and “corr” are valid. If “filled_by” is given together with an aggregation step, the “filled_by” is replaced by the “filled_share”. The “filled_share” gives the share of filled values in the aggregation group in percent.

• period (TimestampPeriod or (tuple or list of datetime.datetime or None), optional) – The minimum and maximum Timestamp for which to get the timeseries. If None is given, the maximum or minimal possible Timestamp is taken. The default is (None, None).

• agg_to (str or None, optional) – Aggregate to a given timespan. If more than 20% of missing values in the aggregation group, the aggregated value will be None. Can be anything smaller than the maximum timespan of the saved data. If a Timeperiod smaller than the saved data is given, than the maximum possible timeperiod is returned. For T and ET it can be “month”, “year”. For N it can also be “hour”. If None than the maximum timeperiod is taken. The default is None.

• nas_allowed (bool, optional) – Should NAs be allowed? If True, then the maximum possible period is returned, even if there are NAs in the timeserie. If False, then the minimal filled period is returned. The default is True.

• add_na_share (bool, optional) – Should one or several columns be added to the Dataframe with the share of NAs in the data. This is especially important, when the stations data get aggregated, because the aggregation doesn’t make sense if there are a lot of NAs in the original data. If True, one column per asked kind is added with the respective share of NAs, if the aggregation step is not the smallest. The “kind”_na_share column is in percentage. The default is False.

• db_unit (bool, optional) – Should the result be in the Database unit. If False the unit is getting converted to normal unit, like mm or °C. The numbers are saved as integer in the database and got therefor multiplied by 10 or 100 to get to an integer. The default is False.

• sql_add_where (str or None, optional) – additional sql where statement to filter the output. E.g. “EXTRACT(MONTH FROM timestamp) == 2” The default is None

Returns:

The timeserie Dataframe with a DatetimeIndex.

Return type:

pandas.DataFrame

get_dist(period=(None, None))

Get the timeserie with the infomation from which station the data got filled and the corresponding distance to this station.

Parameters:

period (TimestampPeriod or (tuple or list of datetime.datetime or None), optional) – The minimum and maximum Timestamp for which to get the timeserie. If None is given, the maximum or minimal possible Timestamp is taken. The default is (None, None).

Returns:

The timeserie for this station and the given period with the station_id and the distance in meters from which the data got filled from.

Return type:

pd.DataFrame

get_filled(period=(None, None), with_dist=False, **kwargs)

Get the filled timeserie.

Either only the timeserie is returned or also the id of the station from which the station data got filled, together with the distance to this station in m.

Parameters:

• period (TimestampPeriod or (tuple or list of datetime.datetime or None), optional) – The minimum and maximum Timestamp for which to get the timeserie. If None is given, the maximum or minimal possible Timestamp is taken. The default is (None, None).

• with_dist (bool, optional) – Should the distance to the stations from which the timeseries got filled be added. The default is False.

Returns:

The filled timeserie for this station and the given period.

Return type:

pd.DataFrame

get_filled_period(kind, from_meta=False)

Get the min and max Timestamp for which there is data in the corresponding timeserie.

Computes the period from the timeserie or meta table.

Parameters:

• kind (str) – The data kind to look for filled period. Must be a column in the timeseries DB. Must be one of “raw”, “qc”, “filled”, “adj”. If “best” is given, then depending on the parameter of the station the best kind is selected. For Precipitation this is “corr” and for the other this is “filled”. For the precipitation also “qn” and “corr” are valid.

• from_meta (bool, optional) – Should the period be from the meta table? If False: the period is returned from the timeserie. In this case this function is only a wrapper for .get_period_meta. The default is False.

Raises:

• NotImplementedError – If the given kind is not valid.

• ValueError – If the given kind is not a string.

Returns:

A TimestampPeriod of the filled timeserie. (NaT, NaT) if the timeserie is all empty or not defined.

Return type:

TimestampPeriod

get_geom(crs=None)

Get the point geometry of the station.

Parameters:

crs (str, int or None, optional) – The coordinate reference system of the geometry. If None, then the geometry is returned in WGS84 (EPSG:4326). If string, then it should be in a pyproj readable format. If int, then it should be the EPSG code. The default is None.

Returns:

The location of the station as shapely Point in the given coordinate reference system.

Return type:

shapely.geometries.Point

get_geom_shp(crs=None)

Get the geometry of the station as a shapely Point object.

Deprecated since version 1.0.0: get_geom_shp is deprecated and will be removed in future releases. It is replaced by get_geom.

Parameters:

crs (str, int or None, optional) – If None, then the geometry is returned in WGS84 (EPSG:4326). If string, then it should be one of “WGS84” or “UTM”. If int, then it should be the EPSG code.

Returns:

The location of the station as shapely Point.

Return type:

shapely.geometries.Point

get_last_imp_period(all=False)

Get the last imported Period for this Station.

Parameters:

all (bool, optional) – Should the maximum Timespan for all the last imports be returned. If False only the period for this station is returned. The default is False.

Returns:

(minimal datetime, maximal datetime)

Return type:

TimespanPeriod or tuple of datetime.datetime

get_ma_raster()

Wrapper for get_multi_annual.

get_max_period(kinds, nas_allowed=False, **kwargs)

Get the maximum available period for this stations timeseries.

If nas_allowed is True, then the maximum range of the timeserie is returned. Else the minimal filled period is returned

Parameters:

• kinds (str or list of str) – The data kinds to update. Must be a column in the timeseries DB. Must be one of “raw”, “qc”, “filled”, “adj”. For the precipitation also “qn” and “corr” are valid.

• nas_allowed (bool, optional) – Should NAs be allowed? If True, then the maximum possible period is returned, even if there are NAs in the timeserie. If False, then the minimal filled period is returned. The default is False.

Returns:

The maximum Timestamp Period

Return type:

TimestampPeriod

get_meta(infos='all')

Get Information from the meta table.

Parameters:

infos (list of str or str, optional) – A list of the information to get from the database. If “all” then all the information are returned. The default is “all”.

Returns:

dict with the meta information. The first level has one entry per parameter. The second level has one entry per information, asked for. If only one information is asked for, then it is returned as single value and not as subdict.

Return type:

dict or int/string

classmethod get_meta_explanation(infos='all')

Get the explanations of the available meta fields.

Parameters:

infos (list or string, optional) – The infos you wish to get an explanation for. If “all” then all the available information get returned. The default is “all”

Returns:

a pandas Series with the information names as index and the explanation as values.

Return type:

pd.Series

get_multi_annual_raster()

Get the multi annual raster value(s) for this station.

Returns:

The corresponding multi annual value. For T en ET the yearly value is returned. For N the winter and summer half yearly sum is returned in tuple. The returned unit is mm or °C.

Return type:

list or None

get_name()

get_neighboor_stids(n=5, only_real=True, p_elev=None, period=None, **kwargs)

Get a list with Station Ids of the nearest neighboor stations.

Parameters:

• n (int, optional) – The number of stations to return. If None, then all the possible stations are returned. The default is 5.

• only_real (bool, optional) – Should only real station get considered? If false also virtual stations are part of the result. The default is True.

• p_elev (tuple of float or None, optional) –

  The parameters (P_1, P_2) to weight the height differences between stations. The elevation difference is considered with the formula from LARSIM (equation 3-18 & 3-19 from the LARSIM manual [1] ):

  \[L_{weighted} = L_{horizontal} * (1 + (\frac{|\delta H|}{P_1})^{P_2})\]

  If None, then the height difference is not considered and only the nearest stations are returned. The default is None.

• period (TimestampPeriod or None, optional) – The period for which the nearest neighboors are returned. The neighboor station needs to have raw data for at least one half of the period. If None, then the availability of the data is not checked. The default is None.

Returns:

A list of station Ids in order of distance. The closest station is the first in the list.

Return type:

list of int

References

[1]

LARSIM Dokumentation, last check on 06.04.2023, online available under https://www.larsim.info/dokumentation/LARSIM-Dokumentation.pdf

get_period_meta(kind, all=False)

Get a specific period from the meta information table.

This functions returns the information from the meta table. In this table there are several periods saved, like the period of the last import.

Parameters:

• kind (str) – The kind of period to return. Should be one of [‘filled’, ‘raw’, ‘last_imp’]. filled: the maximum filled period of the filled timeserie. raw: the maximum filled timeperiod of the raw data. last_imp: the maximum filled timeperiod of the last import.

• all (bool, optional) – Should the maximum Timespan for all the filled periods be returned. If False only the period for this station is returned. The default is False.

Returns:

The TimestampPeriod of the station or of all the stations if all=True.

Return type:

TimestampPeriod

Raises:

ValueError – If a wrong kind is handed in.

get_quotient(kinds_num, kinds_denom, return_as='df')

Get the quotient of multi-annual means of two different kinds or the timeserie and the multi annual raster value.

$quotient = overline{ts}_{kind_num} / overline{ts}_{denom}$

Parameters:

• kinds_num (list of str or str) – The timeseries kinds of the numerators. Should be one of [‘raw’, ‘qc’, ‘filled’]. For precipitation also “corr” is possible.

• kinds_denom (list of str or str) – The timeseries kinds of the denominator or the multi annual raster key. If the denominator is a multi annual raster key, then the result is the quotient of the timeserie and the raster value. Possible values are: - for timeserie kinds: ‘raw’, ‘qc’, ‘filled’ or for precipitation also “corr”. - for raster keys: ‘hyras’, ‘dwd’ or ‘regnie’, depending on your defined raster files.

• return_as (str, optional) – The format of the return value. If “df” then a pandas DataFrame is returned. If “json” then a list with dictionaries is returned.

Returns:

The quotient of the two timeseries as DataFrame or list of dictionaries (JSON) depending on the return_as parameter. The default is pd.DataFrame.

Return type:

pandas.DataFrame or list of dict

Raises:

ValueError – If the input parameters were not correct.

get_raw(**kwargs)

Get the raw timeserie.

Parameters:

**kwargs (dict, optional) – The keyword arguments get passed to the get_df function. Possible parameters are “period”, “agg_to” or “nas_allowed”

Returns:

The raw timeserie for this station and the given period.

Return type:

pd.DataFrame

get_zipfiles(only_new=True, ftp_file_list=None)

Get the zipfiles on the CDC server with the raw data.

Parameters:

• only_new (bool, optional) – Get only the files that are not yet in the database? If False all the available files are loaded again. The default is True

• ftp_file_list (list of (strings, datetime), optional) – A list of files on the FTP server together with their modification time. If None, then the list is fetched from the server. The default is None

Returns:

A DataFrame of zipfiles and the corresponding modification time on the CDC server to import.

Return type:

pandas.DataFrame or None

is_last_imp_done(kind)

Is the last import for the given kind already worked in?

Parameters:

kind (str) – The data kind to look for filled period. Must be a column in the timeseries DB. Must be one of “raw”, “qc”, “filled”, “adj”, “best”. If “best” is given, then depending on the parameter of the station the best kind is selected. For Precipitation this is “corr” and for the other this is “filled”. For the precipitation also “qn” and “corr” are valid.

Returns:

True if the last import of the given kind is already treated.

Return type:

bool

is_real()

Check if the station is a real station or only a virtual one.

Real means that the DWD is measuring here. Virtual means, that there are no measurements here, but the station got created to have timeseries for every parameter for every precipitation station.

Returns:

true if the station is real, false if it is virtual.

Return type:

bool

is_virtual()

Check if the station is a real station or only a virtual one.

Real means that the DWD is measuring here. Virtual means, that there are no measurements here, but the station got created to have timeseries for every parameter for every precipitation station.

Returns:

true if the station is virtual, false if it is real.

Return type:

bool

isin_db()

Check if Station is already in a timeseries table.

Returns:

True if Station has a table in DB, no matter if it is filled or not.

Return type:

bool

isin_ma()

Check if Station is already in the multi annual table.

Returns:

True if Station is in multi annual table.

Return type:

bool

isin_meta()

Check if Station is already in the meta table.

Returns:

True if Station is in meta table.

Return type:

bool

isin_meta_p()

Check if Station is in the precipitation meta table.

Returns:

True if Station is in the precipitation meta table.

Return type:

bool

last_imp_fillup(_last_imp_period=None, **kwargs)

Do the gap filling of the last import.

Parameters:

• _last_imp_period (TimestampPeriod or (tuple or list of datetime.datetime or None), optional) – The minimum and maximum Timestamp for which to do the gap filling. If None is given, the last import period is taken. This is only for internal use, to speed up the process if run in a batch. The default is None.

• **kwargs (dict, optional) – Additional keyword arguments passed to the fillup function.

last_imp_qc(**kwargs)

plot(period=(None, None), kind='filled', agg_to=None, **kwargs)

Plot the data of this station.

Parameters:

• period (TimestampPeriod or (tuple or list of datetime.datetime or None), optional) – The minimum and maximum Timestamp for which to get the timeseries. If None is given, the maximum or minimal possible Timestamp is taken. The default is (None, None).

• kind (str, optional) – The data kind to plot. Must be a column in the timeseries DB. Must be one of “raw”, “qc”, “filled”, “adj”. For the precipitation also “qn” and “corr” are valid. The default is “filled.

• agg_to (str or None, optional) – Aggregate to a given timespan. Can be anything smaller than the maximum timespan of the saved data. If a Timeperiod smaller than the saved data is given, than the maximum possible timeperiod is returned. For T and ET it can be “month”, “year”. For N it can also be “hour”. If None than the maximum timeperiod is taken. The default is None.

update_ma_raster(skip_if_exist=True, drop_when_error=True, **kwargs)

Update the multi annual values in the station_ma_raster table.

Get new values from the raster and put in the table.

Parameters:

• skip_if_exist (bool, optional) – Skip the update if the stations multi annual data is already in the table. The default is True.

• drop_when_error (bool, optional) – Drop the station from the database if there is an error. The default is True.

• **kwargs (dict, optional) – Additional keyword arguments catch all, but unused here.

update_ma_timeseries(kind, **kwargs)

Update the mean annual value from the station timeserie.

Parameters:

• kind (str or list of str) – The timeseries data kind to update theire multi annual value. Must be a column in the timeseries DB. Must be one of “raw”, “qc”, “filled”. For the precipitation also “corr” is valid.

• **kwargs (dict, optional) – Additional keyword arguments catch all, but unused here.

update_period_meta(kind, **kwargs)

Update the time period in the meta file.

Compute teh filled period of a timeserie and save in the meta table.

Parameters:

• kind (str) – The data kind to look for filled period. Must be a column in the timeseries DB. Must be one of “raw”, “qc”, “filled”. If “best” is given, then depending on the parameter of the station the best kind is selected. For Precipitation this is “corr” and for the other this is “filled”. For the precipitation also “corr” are valid.

• **kwargs (dict, optional) – Additional keyword arguments catch all, but unused here.

update_raw(only_new=True, ftp_file_list=None, remove_nas=True, **kwargs)

Download data from CDC and upload to database.

Parameters:

• only_new (bool, optional) – Get only the files that are not yet in the database? If False all the available files are loaded again. The default is True

• ftp_file_list (list of (strings, datetime), optional) – A list of files on the FTP server together with their modification time. If None, then the list is fetched from the server. The default is None

• remove_nas (bool, optional) – Remove the NAs from the downloaded data before updating it to the database. This has computational advantages. The default is True.

• **kwargs (dict) – Additional keyword arguments catch all, but unused here.

Returns:

The raw Dataframe of the Stations data.

Return type:

pandas.DataFrame