forecast_smoke

Smoke checks for CNBS forecast output dataframes.

Lightweight sanity checks that the forecast pipeline produced an output that is structurally correct and within plausible ranges. Distinct from the team’s skill-validation work (RMSE/CRPS/R^2 against the forecast archive) — this only confirms the machinery ran and produced a sensible shape; it does not assess forecast quality.

The forecast pipeline (notebooks/production/2_LEF_forecast_model.ipynb) produces a tidy DataFrame with one row per (cfs_run, forecast_month, model, lake) and columns for each component (precipitation, evaporation, runoff, nbs). These checks assert that output matches the expected contract before it is written to disk or database.

Failures raise ValueError with a message describing the offending rows / columns, so a regression in the upstream pipeline (sign flip, unit bug, schema drift, NaN leak) fails loudly at the boundary rather than silently corrupting downstream artifacts.

Typical use, from notebook 2 right before the to_csv / to_sql calls:

from src.forecast_smoke import smoke_check_forecast smoke_check_forecast(df_pivoted)

Tests can call individual checks (smoke_check_schema, smoke_check_no_nans, smoke_check_ranges) for finer-grained assertions.

src.forecast_smoke.smoke_check_schema(df)

Check that the forecast dataframe has the expected columns, lake names, and forecast_month / cfs_run formats.

Raises:

ValueError – If required columns are missing, lake names are wrong, or time index strings don’t match the documented format.

src.forecast_smoke.smoke_check_no_nans(df)

Check that no NaN or sentinel fill values appear in id columns or component columns.

Raises:

ValueError – If any required column contains NaN, or if any component column contains a known sentinel fill value (-99990.0 / -9999.0).

src.forecast_smoke.smoke_check_ranges(df, ranges=None)

Check that each component value falls within a plausible mm/month range.

Parameters:

• df (pandas.DataFrame) – Forecast output; expected to have columns from EXPECTED_COMPONENTS.

• ranges (dict, optional) – Override the default per-component (low, high) bounds. Defaults to COMPONENT_RANGES_MM.

Raises:

ValueError – If any component value falls outside its bounds. The message identifies the column, bound, and an example offending row.

src.forecast_smoke.smoke_check_forecast(df, ranges=None)

Run all forecast output smoke checks: schema, no-NaNs, value ranges.

Call this from notebook 2 just before writing CSVs / to_sql.

Raises:

ValueError – On the first failed check, with a message describing the failure.