broker

Broker

class weatherDB.broker.Broker

Bases: object

A class to manage and update the database.

Can get used to update all the stations and parameters at once.

This class is only working with SELECT and UPDATE user privileges. Even Better is to also have DELETE and INSERT privileges.

Public Methods:

__init__()
create_db_schema([if_exists, silent, owner])	Create the database schema.
upgrade_db_schema([revision])	Upgrade the database schema to a specific revision.
initiate_db(**kwargs)	Initiate the Database.
update_raw([only_new, paras])	Update the raw data from the DWD-CDC server to the database.
update_meta([paras])	Update the meta file from the CDC Server to the Database.
update_ma_raster([paras])	Update the multi-annual data from raster to table.
update_ma_timeseries([paras])	Update the multi-annual values from timeseries in the database.
update_period_meta([paras])	Update the periods in the meta table.
quality_check([paras, with_fillup_nd])	Do the quality check on the stations raw data.
last_imp_quality_check([paras, with_fillup_nd])	Quality check the last imported data.
fillup([paras])	Fillup the timeseries.
last_imp_fillup([paras])	Fillup the last imported data.
richter_correct(**kwargs)	Richter correct all of the precipitation data.
last_imp_corr(**kwargs)	Richter correct the last imported precipitation data.
update_db([paras])	The regular Update of the database.
vacuum([do_analyze])
get_setting(key)	Get a specific settings value from the database.
set_setting(key, value)	Set a specific setting.
get_db_version()	Get the package version that the databases state is at.
set_db_version([version])	Set the package version that the databases state is at.
activate()	Activate the broker in a context manager.

---

create_db_schema(if_exists=None, silent=False, owner=None)

Create the database schema.

Parameters:

• if_exists (str, optional) – What to do if the tables already exist. If None the user gets asked. If “D” or “drop” the tables get dropped and recreated. If “I” or “ignore” the existing tables get ignored and the creation of the schema continues for the other. If “E” er “exit” the creation of the schema gets exited. The default is None.

• silent (bool, optional) – If True the user gets not asked if the tables already exist. If True, if_exists must not be None. The default is False.

• owner (str, optional) – The user that should get the ownership of the tables and schemas. If None the current database user will be the owner. The default is None.

upgrade_db_schema(revision='head')

Upgrade the database schema to a specific revision.

Parameters:

revision (str, optional) – The revision to upgrade to. If “head” the database gets upgraded to the latest revision. The default is “head”.

initiate_db(**kwargs)

Initiate the Database.

Downloads all the data from the CDC server for the first time. Updates the multi-annual data and the richter-class for all the stations. Quality checks and fills up the timeseries.

Parameters:

**kwargs (dict) – The keyword arguments to pass to the called methods of the stations

update_raw(only_new=True, paras=['p_d', 'p', 't', 'et'], **kwargs)

Update the raw data from the DWD-CDC server to the database.

Parameters:

• only_new (bool, optional) – Get only the files that are not yet in the database? If False all the available files are loaded again. The default is True.

• paras (list of str, optional) – The parameters for which to do the actions. Can be one, some or all of [“p_d”, “p”, “t”, “et”]. The default is [“p_d”, “p”, “t”, “et”].

• **kwargs (dict) – The keyword arguments to pass to the update_raw method of the stations

update_meta(paras=['p_d', 'p', 't', 'et'], **kwargs)

Update the meta file from the CDC Server to the Database.

Parameters:

• paras (list of str, optional) – The parameters for which to do the actions. Can be one, some or all of [“p_d”, “p”, “t”, “et”]. The default is [“p_d”, “p”, “t”, “et”].

• **kwargs (dict) – The keyword arguments to pass to update_meta method of the stations

update_ma_raster(paras=['p_d', 'p', 't', 'et'], **kwargs)

Update the multi-annual data from raster to table.

Parameters:

• paras (list of str, optional) – The parameters for which to do the actions. Can be one, some or all of [“p_d”, “p”, “t”, “et”]. The default is [“p_d”, “p”, “t”, “et”].

• **kwargs (dict) – The keyword arguments to pass to update_ma_raster method of the stations

update_ma_timeseries(paras=['p_d', 'p', 't', 'et'], **kwargs)

Update the multi-annual values from timeseries in the database.

Parameters:

• paras (list of str, optional) – The parameters for which to do the actions. Can be one, some or all of [“p_d”, “p”, “t”, “et”]. The default is [“p_d”, “p”, “t”, “et”].

• **kwargs (dict) – The keyword arguments to pass to update_ma_timeseries method of the stations

update_period_meta(paras=['p_d', 'p', 't', 'et'], **kwargs)

Update the periods in the meta table.

Parameters:

• paras (list of str, optional) – The parameters for which to do the actions. Can be one, some or all of [“p_d”, “p”, “t”, “et”]. The default is [“p_d”, “p”, “t”, “et”].

• **kwargs (dict) – The keyword arguments to pass to update_period_meta method of the stations

quality_check(paras=['p', 't', 'et'], with_fillup_nd=True, **kwargs)

Do the quality check on the stations raw data.

Parameters:

• paras (list of str, optional) – The parameters for which to do the actions. Can be one, some or all of [“p”, “t”, “et”]. The default is [“p”, “t”, “et”].

• with_fillup_nd (bool, optional) – Should the daily precipitation data get filled up if the 10 minute precipitation data gets quality checked. The default is True.

• **kwargs (dict) – The keyword arguments to pass to quality_check method of the stations

last_imp_quality_check(paras=['p', 't', 'et'], with_fillup_nd=True, **kwargs)

Quality check the last imported data.

Also fills up the daily precipitation data if the 10 minute precipitation data should get quality checked.

Parameters:

• paras (list of str, optional) – The parameters for which to do the actions. Can be one, some or all of [“p”, “t”, “et”]. The default is [“p”, “t”, “et”].

• with_fillup_nd (bool, optional) – Should the daily precipitation data get filled up if the 10 minute precipitation data gets quality checked. The default is True.

• **kwargs (dict) – The keyword arguments to pass to last_imp_quality_check method of the stations. If with_fillup_nd is True, the keyword arguments are also passed to the last_imp_fillup method of the stations_pd.

fillup(paras=['p', 't', 'et'], **kwargs)

Fillup the timeseries.

Parameters:

• paras (list of str, optional) – The parameters for which to do the actions. Can be one, some or all of [“p_d”, “p”, “t”, “et”]. The default is [“p_d”, “p”, “t”, “et”].

• **kwargs (dict) – The keyword arguments to pass to fillup method of the stations

last_imp_fillup(paras=['p', 't', 'et'], **kwargs)

Fillup the last imported data.

Parameters:

• paras (list of str, optional) – The parameters for which to do the actions. Can be one, some or all of [“p_d”, “p”, “t”, “et”]. The default is [“p_d”, “p”, “t”, “et”].

• **kwargs (dict) – The keyword arguments to pass to last_imp_fillup method of the stations

richter_correct(**kwargs)

Richter correct all of the precipitation data.

Parameters:

**kwargs (dict) – The keyword arguments to pass to richter_correct method of the stations_p

last_imp_corr(**kwargs)

Richter correct the last imported precipitation data.

Parameters:

**kwargs (dict) – The keyword arguments to pass to last_imp_corr method of the stations

update_db(paras=['p_d', 'p', 't', 'et'], **kwargs)

The regular Update of the database.

Downloads new data. Quality checks the newly imported data. Fills up the newly imported data.

Parameters:

• paras (list of str, optional) – The parameters for which to do the actions. Can be one, some or all of [“p_d”, “p”, “t”, “et”]. The default is [“p_d”, “p”, “t”, “et”].

• **kwargs (dict) – The keyword arguments to pass to the called methods of the stations

vacuum(do_analyze=True, **kwargs)

get_setting(key)

Get a specific settings value from the database.

Parameters:

key (str) – The key of the setting.

Returns:

value – The database settings value.

Return type:

str or None

set_setting(key: str, value: str)

Set a specific setting.

Parameters:

• key (str) – The key of the setting.

• value (str) – The value of the setting.

get_db_version()

Get the package version that the databases state is at.

Returns:

The version of the database.

Return type:

version

set_db_version(version=<Version('1.0.6')>)

Set the package version that the databases state is at.

Parameters:

version (pv.Version, optional) – The Version of the python package The default is the version of this package.

property is_any_active

Get the state of the broker.

Returns:

Whether the broker is active.

Return type:

bool

property is_active

Get the state of the broker.

Returns:

Whether the broker is active.

Return type:

bool

activate()

Activate the broker in a context manager.