Welcome to nbs-predictor! This software package is designed to forecast key components of Net Basin Supply (NBS) for the Laurentian Great Lakes using atmospheric forecast data. Currently, the package uses forecast data from National Oceanic and Atmospheric Administration’s Climate Forecast System (CFS) to predict precipitation, evaporation, runoff, and net basin supply (NBS) at monthly intervals up to 12 months into the future.
- Advanced Predictive Algorithms: Leverages methods like Gaussian Processes and Random Forest for data-driven forecasting.
- Real-Time Data Processing: Processes real-time operational forecast data for up-to-date predictions.
- User-Friendly Interface: Streamlined setup and interactive notebooks for ease of use.
- Continuous Improvements: Actively maintained with regular updates to improve performance, expand functionality, and enhance modeling capabilities.
- Data Source: NOAA forecast data from the Climate Forecast System (CFS), which must be downloaded and preprocessed before generating forecasts. Additionally, sea surface temperatures from GLSEA are required as initial conditions. This repository includes notebooks to handle both downloading and preprocessing of both datasets.
- Required Datasets: Please refer to the 'Data Sources' section below for specific files and data organization.
Forecast monthly precipitation (P), evaporation (E), runoff (R), and net basin supply (NBS) for all of the Laurentian Great Lakes twelve (12) months into the future. By forecasting NBS directly, rather than deriving it solely from the individual component forecasts (NBS = P − E + R), the model minimizes accumulated uncertainty and improves reliability.
| Abbreviation | Name | Source | Data Use |
|---|---|---|---|
| CFSR | Climate Forecast System Reanalysis | NOAA | Training |
| CFS | Climate Forecast System v2 | NOAA | Forecasting |
| GLSEA | Great Lakes Surface Environmental Analysis | NOAA GLERL | Forecasting |
| L2SWBM | Large Lake Statistical Water Balance Model | GLCC | Training |
| RNBS | Residual Net Basin Supply | GLCC | Training |
Before you begin, make sure that the following are installed on your system:
- Conda (Anaconda or Miniconda)
- Python 3.11+
On a Mac, begin by opening a Terminal window.
On a Windows computer, you will need to open the Anaconda Prompt.
- If you don’t already have Anaconda or Miniconda installed, download and install it from: https://www.anaconda.com/download
-
Clone the Repository: Decide where you want the tool to live on your local machine and clone the repository. In this example, we will add it directly to the Desktop.
cd Desktopgit clone https://github.com/great-lakes-ai-lab/cnbs-predictor.git
cd cnbs-predictor -
Set Up the Conda Environment: Create and activate the Conda environment.
conda env create -f requirements/environment.yml
conda activate nbs_env
-
Set Up Jupyter Kernel: Register the Conda environment as a Jupyter kernel. This is a one time step. After initial set up, you will not need to run this command again.
python -m ipykernel install --user --name nbs_env --display-name "Python (nbs_env)"
After setting up your Conda environment, you can launch Jupyter Lab to work with the notebooks.
jupyter lab- After starting Jupyter Lab, a new browser window should open.
- On the left, navigate to the notebooks/production/ directory.
- First time users who do not have the trained model files (e.g. data/input/models/GP_trained_model_anom.joblib) will need to first run
0_LEF_model_training_anomalies.ipynb. This script will create and train the scalers, models, and attributes and save them to the appropriate folders. If you already have the required files, skip to step 8. - Set your local directory paths in the User Input section.
- Double check the kernel in the top right shows something similar to
Python 3 (ipykernel)orPython (nbs_env). If it does not, click on it to switch kernels. - Run the notebook by clicking on the ▶▶ symbol to run all cells or the ▶ to run one cell at a time.
- You should now have joblib files under models, scalers, and csv/json files under climatology.
- Open
1_LEF_download_preprocess.ipynb. Set your local directory paths in the User Input section. - Run the notebook. This will download CFS data to the specified directory, process the data, and save it to a database
cfs_forecast_data.db. If you have a database started, you can set auto to 'yes' in the notebook and it will automatically pull the last date in the database and pick up where it left off, bringing the database up-to-date. If you are starting a database from scratch, the script will automatically begin downloading CFS data from 9 months prior until today. You can also set auto to 'no' and set the specified date range to download and process CFS data. This script takes a while to run, depending on how much data it needs to download and process. - Open
2_LEF_forecast_model.ipynb. Set your local directory paths in the User Input section. - Run the notebook. This will read in the CFS database, run the data through the saved trained models, and produce a 12 month forecast for precipitation, evaporation, runoff, and NBS.
- Script
3_LEF_visualization.ipynbis an optional notebook you can run to create timeseries plots of the forecast data created in script 2.
cnbs-predictor/
├── assets # Project logos and branding assets
├── CITATION.cff # Citation metadata for the project
├── CODE_OF_CONDUCT.md # Code of conduct for contributors
├── CONTRIBUTING.md # Contribution guidelines
├── data # Directory for storing input and forecast data
│ ├── cfs # Archived pre-processed CFS data
│ ├── cfsr # Archived pre-processed CFSR data used for training
│ ├── forecast # Forecast output data
│ ├── glcc # NBS observations from GLCC used for training and validation
│ ├── glsea # GLSEA surface water temperature data
│ ├── input # Model inputs, trained models, scalers, masks, etc.
│ ├── l2swbm # L2SWBM target data for precipitation, evaporation, and runoff
│ ├── probabilities # USACE probability files used for CEP calculations
│ └── snodas # SNODAS snow data used for model inputs
├── docs # Project documentation and supporting materials
│ ├── development_history.md # Notes on project development and changes
│ ├── experiment_skill_metrics.md # Skill metric summaries from model experiments
│ ├── experiments.md # Experiment descriptions and results
│ ├── Great Lakes NBS Predictor Flyer.pdf # Project flyer
│ ├── sphinx # Sphinx documentation source files
│ └── wiki-drafts # Draft content for GitHub wiki pages
├── images # Team member or project-related images
├── LICENSE # Project license
├── notebooks # Jupyter notebooks
│ ├── exploratory # Exploratory and development notebooks
│ ├── production # Production-ready notebooks for generating forecasts
│ └── validation # Notebooks for model validation
├── README.md # Project overview and setup instructions
├── requirements # Conda environment files
│ ├── environment-test.yml # Testing environment
│ └── environment.yml # Main project environment
├── ROADMAP.md # Project roadmap and planned improvements
├── SECURITY.md # Security policy and reporting guidance
├── src # Source code for data processing, modeling, and utilities
│ ├── data_downloader.py # Utilities for downloading input datasets
│ ├── data_loader.py # Functions for loading data
│ ├── data_processor.py # Data cleaning, transformation, and preprocessing tools
│ ├── database_utils.py # Database creation, querying, and management utilities
│ ├── forecast_smoke.py # Forecast smoke tests or quick forecast checks
│ ├── hydro_utils.py # Hydrologic calculation utilities
│ ├── plotting.py # Plotting and visualization functions
│ └── utilities.py # General helper functions
└── tests # Test suite
├── conftest.py # Shared pytest fixtures and configuration
├── fixtures # Test fixture data
├── integration # Integration tests across multiple components
└── unit # Unit tests for individual functionsWe welcome contributions to cnbs-predictor! Please start a discussion with the maintainers first (@lefitzpatrick, @danijonesocean), preferably in the GitHub Discussions or Issues sections of this repository. Once you have discussed possible changes with the developers:
- Fork the repository.
- Clone your fork to your target machine.
- Create a feature branch for your changes.
- Make and commit your changes.
- Push your branch to your fork.
- Submit a pull request to the main repository.
For detailed contributing instructions, please refer to the Contributing guidelines. For a summary of current development plans to help map out possible contributions, see our Roadmap document.
This project adheres to a Code of Conduct. By participating, you agree to follow this code and foster a welcoming and respectful community.
This project is licensed under the GNU Affero General Public License Version 3.0.
 Dani Jones Principal Investigator |
 Lindsay Fitzpatrick Model Developer |
 Deanna Fielder Model Testing |
 Jamie Ward Model Testing |
 Trece Bye Model Testing |
 Matt McAnear Contributor |
 Bryan Mroczka Contributor |
nbs-predictor is powered by institutional collaboration from:
Funding for this project provided by NOAA