Skip to content

Add standalone OWASP cheat sheet refresh script#952

Open
Bornunique911 wants to merge 2 commits into
OWASP:mainfrom
Bornunique911:review/issue-471-cheatsheet-refresh-script
Open

Add standalone OWASP cheat sheet refresh script#952
Bornunique911 wants to merge 2 commits into
OWASP:mainfrom
Bornunique911:review/issue-471-cheatsheet-refresh-script

Conversation

@Bornunique911

@Bornunique911 Bornunique911 commented Jul 1, 2026

Copy link
Copy Markdown
Contributor

Summary

This PR is split out from the larger issue-471 review flow to make review smaller and more focused.

It adds a standalone script for refreshing OWASP Cheat Sheet data and normalizing cheat sheet links in the local cache.

Issue reference:

Problem Fixed

The earlier refresh-scripts review became too large because it was mixed with broader OWASP importer and follow-up work.

For this part of the work, the useful standalone contribution is:

  • rebuilding cheat sheet data into the cache
  • normalizing GitHub-style cheat sheet links to official OWASP Cheat Sheet Series URLs
  • providing that workflow as a reusable script with backup and environment setup

Solution

This PR adds a single standalone script:

  • scripts/update-cheatsheets.sh

The script:

  • prepares the local virtual environment if needed
  • installs Python dependencies if required
  • verifies the target database exists
  • creates a timestamped backup
  • reimports cheat sheet data using --cheatsheets_in
  • normalizes stored cheat sheet links from GitHub paths to official OWASP Cheat Sheet Series URLs

Tests

bash -n scripts/update-cheatsheets.sh

Reviewer Notes

This PR is intentionally narrow because it was split to reduce review size:

  • no frontend changes
  • no parser-stack expansion
  • no unrelated OWASP importer changes
  • no bundle artifacts

This PR is meant to be reviewed as a standalone operational helper.

@coderabbitai

coderabbitai Bot commented Jul 1, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

Warning

Review limit reached

@Bornunique911, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 29 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: Path: .coderabbit.yml

Review profile: CHILL

Plan: Pro

Run ID: 071f9dfc-f434-4d95-9dbd-bb9667a3bd69

📥 Commits

Reviewing files that changed from the base of the PR and between 716f8b2 and 76283a2.

📒 Files selected for processing (1)
  • scripts/update-cheatsheets.sh

Walkthrough

This PR adds a new Bash script, scripts/update-cheatsheets.sh, which prepares a Python virtual environment, backs up the SQLite cache database, runs cre.py to update cheatsheet data, and rewrites OWASP Cheat Sheets node links from GitHub URLs to official cheatsheets URLs.

Changes

Cheatsheets Update Script

Layer / File(s) Summary
Script setup, backup, and update flow
scripts/update-cheatsheets.sh
Sets strict Bash mode, prepares a Python venv, installs dependencies if needed, backs up the cache DB with a timestamp, runs cre.py with gap analysis/embedding disabled, then updates matching node links from GitHub URLs to official OWASP cheatsheets URLs via inline Python.

Estimated code review effort: 2 (Simple) | ~10 minutes

Sequence Diagram(s)

sequenceDiagram
  participant Script as update-cheatsheets.sh
  participant Venv as Python venv
  participant DB as standards_cache.sqlite
  participant Cre as cre.py

  Script->>Venv: create/activate venv, install requirements if flask missing
  Script->>DB: verify DB exists
  Script->>DB: create timestamped .bak backup
  Script->>Cre: run cre.py --cheatsheets_in --cache_file DB
  Script->>DB: query node table for OWASP Cheat Sheets links
  Script->>DB: update matching links to official cheatsheets URL
Loading

Related issues: None specified.

Related PRs: None specified.

Suggested labels: scripts, maintenance

Suggested reviewers: None specified.

🐰 A rabbit hops through cache and code,
Backing up before the load,
Links once pointed to GitHub's tree,
Now cheatsheets point where they should be!

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly summarizes the main change: adding a standalone OWASP cheat sheet refresh script.
Description check ✅ Passed The description is directly related to the script addition and its cache refresh workflow.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@Bornunique911 Bornunique911 force-pushed the review/issue-471-cheatsheet-refresh-script branch from d08d662 to 716f8b2 Compare July 7, 2026 11:55

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (2)
scripts/update-cheatsheets.sh (2)

23-24: 🗄️ Data Integrity & Integration | 🔵 Trivial | ⚡ Quick win

Backup filename collision within the same second.

date +%Y%m%d%H%M%S has 1-second resolution; two runs within the same second silently overwrite each other's backup.

🛡️ Add uniqueness
-BACKUP_FILE="${DB_PATH}.$(date +%Y%m%d%H%M%S).bak"
+BACKUP_FILE="${DB_PATH}.$(date +%Y%m%d%H%M%S)_$$.bak"
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@scripts/update-cheatsheets.sh` around lines 23 - 24, The backup creation in
the update-cheatsheets script can collide when run multiple times within the
same second because BACKUP_FILE is based only on date +%Y%m%d%H%M%S. Update the
BACKUP_FILE naming logic in the script so each run produces a unique filename,
for example by adding higher-resolution time, the process ID, or another unique
suffix, and keep the cp backup step using that new unique name.

14-16: 🩺 Stability & Availability | 🔵 Trivial | ⚡ Quick win

Fragile dependency-check proxy.

Using import flask success as a stand-in for "all deps installed" means new/updated packages in requirements.txt won't get installed if flask is already present in the venv from a prior run.

♻️ Simpler and more robust alternative
-if ! python -c "import flask" >/dev/null 2>&1; then
-  pip install -r "$ROOT_DIR/requirements.txt"
-fi
+pip install -q -r "$ROOT_DIR/requirements.txt"

pip install is idempotent and fast when nothing changed, so always running it avoids silently skipping updated dependencies.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@scripts/update-cheatsheets.sh` around lines 14 - 16, The dependency check in
the update-cheatsheets.sh script is too narrow because it only tests flask via
the python import gate, so updated or newly added packages in requirements.txt
can be skipped. Remove the import-based conditional around the pip install step
and always run the requirements installation in the script flow so dependency
updates are applied reliably; the relevant logic is the shell block that wraps
pip install -r "$ROOT_DIR/requirements.txt".
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@scripts/update-cheatsheets.sh`:
- Around line 23-24: The backup creation in the update-cheatsheets script can
collide when run multiple times within the same second because BACKUP_FILE is
based only on date +%Y%m%d%H%M%S. Update the BACKUP_FILE naming logic in the
script so each run produces a unique filename, for example by adding
higher-resolution time, the process ID, or another unique suffix, and keep the
cp backup step using that new unique name.
- Around line 14-16: The dependency check in the update-cheatsheets.sh script is
too narrow because it only tests flask via the python import gate, so updated or
newly added packages in requirements.txt can be skipped. Remove the import-based
conditional around the pip install step and always run the requirements
installation in the script flow so dependency updates are applied reliably; the
relevant logic is the shell block that wraps pip install -r
"$ROOT_DIR/requirements.txt".

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yml

Review profile: CHILL

Plan: Pro

Run ID: 4a99f980-fd4a-4622-a8a0-b5532012fd97

📥 Commits

Reviewing files that changed from the base of the PR and between 0e16c2e and 716f8b2.

📒 Files selected for processing (1)
  • scripts/update-cheatsheets.sh

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant