Skip to content

Add missing metric config to sweep example in credit scorecards notebook#632

Open
johndmulhausen wants to merge 1 commit into
mainfrom
fix/issue-601-sweep-metric
Open

Add missing metric config to sweep example in credit scorecards notebook#632
johndmulhausen wants to merge 1 commit into
mainfrom
fix/issue-601-sweep-metric

Conversation

@johndmulhausen

Copy link
Copy Markdown
Contributor

Bug

The sweep_config in colabs/boosting/Credit_Scorecards_with_XGBoost_and_W&B.ipynb defines method and parameters but no metric key. Without it, W&B sweep visualizations (parallel coordinates, parameter importance, best-run selection) pick an arbitrary parameter as the optimization goal instead of the validation loss the notebook actually cares about.

Fix

Insert a metric block into sweep_config, right after method:

  "metric" : {
    "name" : "val_log_loss",
    "goal" : "minimize"
  },

val_log_loss matches exactly what the sweep's train() function records (run.summary["val_log_loss"]), and it is a mean negative log-likelihood, so minimize is the correct goal. The block matches the cell's existing indentation and spacing style, and the change is a pure 4-line insertion in the raw notebook JSON — no other cells, outputs, or metadata touched.

Verification

  • nbformat.validate() passes on the edited notebook (still valid nbformat 4 JSON).
  • Extracted the edited cell's source (minus the wandb.sweep(...) call) and executed it: it is valid Python and sweep_config["metric"] == {"name": "val_log_loss", "goal": "minimize"}.
  • Programmatically confirmed the sweep train() cell (the function passed to wandb.agent) still logs run.summary["val_log_loss"], so the metric name matches what training logs.

Fixes #601

🤖 Generated with Claude Code

The sweep_config in the XGBoost credit scorecards notebook had no
"metric" key, so W&B sweep visualizations picked an arbitrary
parameter as the optimization goal. Add the metric block pointing at
val_log_loss (minimize), which matches what the sweep train() function
logs via run.summary["val_log_loss"].

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Copilot AI review requested due to automatic review settings July 17, 2026 17:16
@review-notebook-app

Copy link
Copy Markdown

Check out this pull request on  ReviewNB

See visual diffs & provide feedback on Jupyter Notebooks.


Powered by ReviewNB

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds the missing W&B sweep metric configuration to the Credit Scorecards hyperparameter sweep example so the sweep correctly optimizes validation log loss (instead of defaulting to an arbitrary parameter).

Changes:

  • Inserted a "metric" block into sweep_config with name: "val_log_loss" and goal: "minimize".

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@johndmulhausen
johndmulhausen marked this pull request as ready for review July 21, 2026 21:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: Missing Metric Configuration in Sweep Example

2 participants