Skip to content

FEAT: Adding Garak Web Injection Scenario#2067

Open
rlundeen2 wants to merge 1 commit into
microsoft:mainfrom
rlundeen2:rlundeen2-web-injection-scenario
Open

FEAT: Adding Garak Web Injection Scenario#2067
rlundeen2 wants to merge 1 commit into
microsoft:mainfrom
rlundeen2:rlundeen2-web-injection-scenario

Conversation

@rlundeen2

Copy link
Copy Markdown
Contributor

This PR ports garak's web_injection probe family into PyRIT as a new WebInjection scenario (registered as garak.web_injection), covering all 8 garak probe variants. These split into two groups: markdown-based data exfiltration (markdown-image exfil, Colab AI data leakage, string-assembly exfil, playground markdown exfil, and two combinatorial "extended" URI probes) and cross-site scripting (task-carrier XSS and markdown XSS). The implementation uses PyRIT concepts throughout — each strategy packages a SeedObjective (the attack goal) alongside the literal injection SeedPrompt in a SeedAttackGroup, and aggregate strategies (ALL, DEFAULT, EXFIL, XSS) let callers select probe subsets.

Rather than ship a parallel garak-faithful scorer set, the PR enhances the existing XSSOutputScorer and MarkdownInjectionScorer with the detection cases garak relies on, and wires them together as a composite OR objective scorer. Three new local seed datasets back the probes (garak_example_domains_xss, garak_markdown_js, garak_xss_normal_instructions), while the existing garak_web_html_js dataset is reused; all four are declared as scenario defaults so they auto-load via LoadDefaultDatasets. Unit tests cover the scenario and the enhanced scorers, and the garak scanner docs are updated.

Example usage

pyrit_scan garak.web_injection --target openai_chat --strategies xss --max-dataset-size 1

Port garak's web_injection probe family as a PyRIT WebInjection scenario covering all 8 probe variants (markdown data-exfiltration and XSS). Each strategy pairs a SeedObjective with a SeedPrompt in a SeedAttackGroup. Enhances the existing XSSOutputScorer and MarkdownInjectionScorer with garak detection cases and adds local seed datasets.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant