Skip to content

ci: run only fast tests and skip slow notebooks on draft PRs #562

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
FBumann merged 8 commits into from
Jul 22, 2026
Merged

ci: run only fast tests and skip slow notebooks on draft PRs #562

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
b7c4936
Speed up ci by only ranning fast tests on drafts
FBumann Jan 10, 2026
ecf9265
Remove deprecated from drafts
FBumann Jan 10, 2026
cfaeda0
Improve test selection
FBumann Jan 10, 2026
a594b56
Make docs match this pattern
FBumann Jan 10, 2026
80e52e5
Add include paths to workflows:
FBumann Jan 10, 2026
9609435
Added .github/workflows/** to both tests and docs workflows. Now CI c…
FBumann Jan 10, 2026
2cc84c2
Retrigger CI
FBumann Jan 10, 2026
ee49258
Ensure workflows are triggered on ready_for_review
FBumann Jan 10, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 11 additions & 0 deletions .github/workflows/docs.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,11 +7,14 @@ on:
- 'docs/**'
- 'mkdocs.yml'
- 'flixopt/**'
- '.github/workflows/**'
pull_request:
types: [opened, synchronize, reopened, ready_for_review]
paths:
- 'docs/**'
- 'mkdocs.yml'
- 'flixopt/**' # notebooks import flixopt; catch library changes that break docs
- '.github/workflows/**'
workflow_dispatch:
inputs:
deploy:
Expand Down Expand Up @@ -91,6 +94,14 @@ jobs:
grep -vFf slow_notebooks.txt | \
xargs -P 4 -I {} sh -c 'jupyter execute --inplace "$1" || exit 255' _ {}

- name: Execute slow notebooks
if: steps.notebook-cache.outputs.cache-hit != 'true' && github.event.pull_request.draft != 'true'
run: |
set -eo pipefail
# Execute slow notebooks in parallel (skip on draft PRs)
cd docs/notebooks && cat slow_notebooks.txt | \
xargs -P 4 -I {} sh -c 'jupyter execute --inplace "$1" || exit 255' _ {}
Comment on lines +97 to +103

@coderabbitai coderabbitai Bot Jul 22, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🗄️ Data Integrity & Integration | 🟠 Major | ⚡ Quick win

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
set -euo pipefail

git ls-files .github/workflows/docs.yaml
wc -l .github/workflows/docs.yaml
sed -n '1,220p' .github/workflows/docs.yaml

Repository: flixOpt/flixopt

Length of output: 6367

🏁 Script executed:

#!/bin/bash
set -euo pipefail

# Inspect cache-related references in the workflow and nearby docs notebook paths.
rg -n "notebook-cache|slow_notebooks|ready_for_review|draft != 'true'|actions/cache|cache-key|cache-hit" .github/workflows/docs.yaml docs -S

Repository: flixOpt/flixopt

Length of output: 1563

🏁 Script executed:

#!/bin/bash
set -euo pipefail

# If the workflow references separate cache keys or restore-keys, inspect the exact expressions.
python3 - <<'PY'
from pathlib import Path
p = Path('.github/workflows/docs.yaml')
text = p.read_text()
for needle in ['notebook-cache', 'slow_notebooks', 'ready_for_review', 'cache-hit', 'actions/cache', 'restore-keys', 'key:']:
    print(f'## {needle}')
    for i, line in enumerate(text.splitlines(), 1):
        if needle in line:
            print(f'{i}: {line}')
PY

Repository: flixOpt/flixopt

Length of output: 1670

🏁 Script executed:

#!/bin/bash
set -euo pipefail

sed -n '1,220p' .github/workflows/docs.yaml

Repository: flixOpt/flixopt

Length of output: 6307

Separate draft and full notebook caches. The cache key only hashes notebooks and flixopt source, so a draft PR run can save fast-only outputs under the same key that a later ready_for_review run will hit and then skip the slow notebooks. Include execution mode/slow_notebooks.txt in the key, or prevent draft runs from saving the cache.

🤖 Prompt for AI Agents 
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In @.github/workflows/docs.yaml around lines 97 - 103, Update the notebook cache
configuration associated with the “Execute slow notebooks” step so draft and
ready-for-review/full runs cannot share a cache entry containing different
notebook outputs. Include the execution mode and slow_notebooks.txt in the cache
key, or disable cache saving for draft runs, while preserving the existing
slow-notebook skip behavior on cache hits.


- name: Build docs
env:
MKDOCS_JUPYTER_EXECUTE: "false"
Expand Down
20 changes: 19 additions & 1 deletion .github/workflows/tests.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,8 +3,19 @@ name: Tests
on:
push:
branches: [main]
paths:
- 'flixopt/**'
- 'tests/**'
- 'pyproject.toml'
- '.github/workflows/**'
pull_request:
types: [opened, synchronize, reopened, ready_for_review]
branches: ["**"]
paths:
- 'flixopt/**'
- 'tests/**'
- 'pyproject.toml'
- '.github/workflows/**'
workflow_dispatch:

concurrency:
Expand Down Expand Up @@ -61,7 +72,14 @@ jobs:
run: uv pip install --system .[dev]

- name: Run tests
run: pytest -v --numprocesses=auto
run: |
if [[ "${{ github.event.pull_request.draft }}" == "true" ]]; then
# Draft PR: skip examples, slow, and deprecated_api
pytest -v --numprocesses=auto -m "not examples and not slow and not deprecated_api"
else
# Ready PR & main push: examples excluded via addopts
pytest -v --numprocesses=auto
fi
Comment on lines +75 to +82

@coderabbitai coderabbitai Bot Jul 22, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔒 Security & Privacy | 🟠 Major | ⚡ Quick win

🧩 Analysis chain

🏁 Script executed:

sed -n '1,220p' .github/workflows/tests.yaml

Repository: flixOpt/flixopt

Length of output: 3440

🌐 Web query:

GitHub Actions GITHUB_TOKEN permissions default contents read upload-artifact permissions required actions/upload-artifact checkout least privilege workflow permissions

💡 Result:

The GitHub Actions GITHUB_TOKEN is a temporary, automatically generated secret used to authenticate API requests within workflows [1][2]. Applying the principle of least privilege is a critical security best practice for GitHub Actions [3][4]. Default Permissions For most new repositories and organizations, the default permission for the GITHUB_TOKEN is set to read-only (specifically for contents and packages) [5][6][2]. You can verify or change this default in your repository or organization settings under Actions > General > Workflow permissions [5][2]. Permissions for Standard Actions 1. actions/checkout: This action typically requires read access to repository contents to fetch the code [1][6]. 2. actions/upload-artifact: This action does not rely on the GITHUB_TOKEN for its core functionality; it uses a separate, internal token managed by the GitHub Actions runner [7]. Therefore, you do not need to grant explicit GITHUB_TOKEN permissions for artifact uploads [7]. Note that "permission denied" errors encountered with this action are almost always related to file system permissions on the runner (e.g., trying to access a file created by a different user, such as root in Docker) rather than GITHUB_TOKEN scopes [8][9][10]. Implementing Least Privilege To enforce the principle of least privilege, explicitly define permissions in your workflow YAML file [3][4]. Any permission not listed under the permissions key is set to none by default [1]. Example of a minimal permissions configuration: permissions: contents: read If a specific job requires more access (e.g., to create an issue), you can scope the permissions for that job specifically: jobs: job_name: permissions: contents: read issues: write steps: - uses: actions/checkout@v4 - run: gh issue create... For complex workflows, you can use monitoring tools to identify the minimum set of permissions required before locking them down [2].

Citations:

Restrict the workflow token here. Add a permissions block with contents: read at the workflow level; none of these jobs need additional scopes.

🧰 Tools
🪛 zizmor (1.26.1)

[warning] 51-82: overly broad permissions (excessive-permissions): default permissions used due to no permissions: block

(excessive-permissions)

🤖 Prompt for AI Agents 
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In @.github/workflows/tests.yaml around lines 75 - 82, Add a workflow-level
permissions block granting only contents: read, alongside the existing workflow
configuration in tests.yaml. Keep the pytest execution branches unchanged and
ensure no broader token scopes are introduced.

Source: Linters/SAST tools


test-examples:
runs-on: ubuntu-24.04
Expand Down
Loading