Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
55 changes: 55 additions & 0 deletions .github/skills/overall-status/references/common.md
Original file line number Diff line number Diff line change
Expand Up @@ -298,6 +298,61 @@ line. Rules:
to a plain `**bold paragraph**` — the admonition class is what makes
it visually prominent.

### C4.2.1 Per-link status markers — Inspection & Report cells (mandatory)

Inspection cells (`Req. Inspection`, `Arch. Inspection`, `Impl. Inspection`)
and the **Module Verification Report** cells routinely list many source
links (Baselibs alone has 10 checklists per inspection column). A reader
cannot tell which checklist is finished from the aggregate `3/10` count
alone. Therefore **every source link in these cells MUST be prefixed with a
status marker** derived from the *linked* file's document-level `:status:`
(the same `:status:` that drives the cell count — see
[C3.3](#c33-inspections)):

| Linked file `:status:` | Marker |
|---|---|
| `valid` | ✅ |
| `draft` | 🔄 |
| `invalid` | ❌ |
| missing / no `:status:`| ⚠️ (investigate — usually a wrong path) |

Format — the marker sits **between** the line-block bar and the backtick:

```rst
- 🔄 30% (3/10)

| ✅ `baselibs <…/docs/requirements/chklst_req_inspection.rst>`__
| ✅ `bitmanipulation <…/docs/requirements/chklst_req_inspection.rst>`__
| 🔄 `concurrency <…/docs/requirements/chklst_req_inspection.rst>`__
```

Hard rules:
- Applies to **every** cell whose links point at `chklst_*_inspection.rst`
files (`Req./Arch./Impl. Inspection`) **and** at
`module_verification_report.rst` / `platform_ver_report.rst` files.
- The marker reflects the **document-level** `:status:` of the *linked*
file (parse with `count_checklist_status`, see
[Step 2](#step-2--count-directives)), **not** the aggregate cell status.
- Self-check: the number of ✅ markers in a cell **MUST equal** the `valid`
numerator of that cell (e.g. `3/10` ⇒ exactly three ✅). A mismatch means
a stale marker or a wrong link.
- Requirements / Architecture / Detailed-Design / Code cells (which link
`index.rst` requirement/architecture files, **not** checklists) do **not**
get per-link markers — only inspection and report links do.
- Markers are additive to the existing link rules ([C4.2](#c42-source-links--mandatory)):
keep the component-name label, the pinned-ref URL and the no-cap listing.

Helper (renders one marked inspection/report link):

```python
INSP_MARK = {"valid": "✅", "draft": "🔄", "invalid": "❌"}

def insp_link(label, url, status):
"""status is the document :status: of the linked chklst_*/report file,
from count_checklist_status()'s underlying :status: match."""
return f"| {INSP_MARK.get(status, '⚠️')} `{label} <{url}>`__"
```

### C4.3 Per-PA progress figures

Each Process Area embeds one SVG figure under `docs/_assets/`:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,10 @@ embedding) see [common.md C4](./common.md#c4-rst-formatting-rules).
(`.trlc`, not parsed) — render the Component Req cell as
`0/0 comp_req [TRLC]` plus a link to the `.trlc` file (see
[C4.2](./common.md#c42-source-links--mandatory)).
- **Per-link status markers (Req. Inspection):** prefix every
`chklst_req_inspection.rst` link with ✅/🔄/❌ by its document `:status:`
(count of ✅ must equal the `valid` numerator) — see
[common.md C4.2.1](./common.md#c421-per-link-status-markers--inspection--report-cells-mandatory).

---

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,11 @@ Never counted as Arch directives: `stkh_req`, `tool_req`, `document`,
`needtable`, `needpie`, `needextend`, `figure`, `uml`, `note`, `attention`,
`toctree`, `grid`.

- **Per-link status markers (Arch. Inspection):** prefix every
`chklst_arc_inspection.rst` link with ✅/🔄/❌ by its document `:status:`
(count of ✅ must equal the `valid` numerator) — see
[common.md C4.2.1](./common.md#c421-per-link-status-markers--inspection--report-cells-mandatory).

---

## Path filters
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,12 @@ estimate (Code), directive counting (Detailed Design) and checklist status
| Detailed Design | `dd` | `dd`, `dd_sta`, `dd_dyn` |
| Impl. Inspection | `chklst` | document `:status:` of each `chklst_impl_inspection.rst` / `chklst_dd_inspection.rst` (see [C3.3](./common.md#c33-inspections)) |

- **Per-link status markers (Impl. Inspection):** prefix every
`chklst_impl_inspection.rst` / `chklst_dd_inspection.rst` link with
✅/🔄/❌ by its document `:status:` (count of ✅ must equal the `valid`
numerator) — see
[common.md C4.2.1](./common.md#c421-per-link-status-markers--inspection--report-cells-mandatory).

### Lines of Code (Code column)

LOC counts every line in source files (`.cpp .h .c .rs .py`) outside `docs/`,
Expand Down
58 changes: 58 additions & 0 deletions .github/skills/overall-status/references/pa5-verification.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,11 @@ status and the module verification report. For shared conventions see
| **Module Ver. Report** | `verification/module_verification_report.rst` `:status: valid` and contains data | `:status: draft` | absent or template only |
| **Platform Ver. Report** | _no column_ — the platform verification report exists **once** for the entire platform; do not render it as a per-module column. Add it as a **bold one-liner immediately after the PA5 table** (see [Platform Verification Report](#platform-verification-report)). | | |

- **Per-link status marker (Module Ver. Report):** prefix the
`module_verification_report.rst` link with ✅/🔄/❌ by its document
`:status:` — see
[common.md C4.2.1](./common.md#c421-per-link-status-markers--inspection--report-cells-mandatory).

> **Static / Dynamic Analysis cells render the status only, no source-code
> link.** The PA5 link discipline of
> [common.md C4.2](./common.md#c42-source-links--mandatory) does not apply to
Expand All @@ -43,6 +48,59 @@ singular/plural (`1 test` vs `N tests`).

---

## Test-counting pitfalls (read before touching Unit / Comp. IT numbers)

Test discovery is the **most error-prone** part of PA5. Verified failure
modes (each has produced a wrong number in the past — Logging was `340`
unit / `❌` comp when the truth is `650` / `8`):

1. **Count all components of a module, not just one subtree.** Two-component
modules keep tests in several roots — e.g. Logging has **both**
`score/datarouter/**` (~324 cases) **and** `score/mw/log/**` (~326).
Missing one silently halves the count. Always walk the *whole* repo tree.
2. **Rust inline unit tests live in `src/`, not in `tests/`.** Counting only
files under a `test/`-style path (the old `is_test_path`) **misses every
`#[cfg(test)] mod tests { #[test] … }`** in `src/*.rs`. Lifecycle has
**235** such inline tests, all in `src/`. Count `#[test]` /
`#[tokio::test]` in **all** `.rs` files (outside `docs/`, `third_party/`,
`bazel-*`).
3. **Unit vs. Component/Integration is per-repo convention, not one rule.**
There is no single reliable heuristic; classify with the module's own
convention:

| Module | Component/Integration tests live in | Count |
|---|---|---|
| Logging | `score/test/component/**` (Bazel `tags=["integration"]`, ITF `py_*_itf_test`) | 8 |
| Persistency | `tests/test_cases/tests/test_cit_*.py` (**CIT** = Component Integration Test) | 44 |
| Baselibs | `score/os/utils/test/mqueueintegration_test.cpp` | 13 |
| Communication | `score/mw/com/test/**/integration_test/*.py` | 63 |
| Time | `score/**/tests/integration_tests/**` | 4 |
| Security/Crypto | `tests/integration_tests/integration_test.py` | 11 |
| Some/IP | `tests/integration_test/**` + `quality/integration_testing/**` | 21 |

A **Rust `tests/` directory is a *crate integration* test**, but a
C++ component's `tests/` subfolder (e.g. `futurecpp/tests/`) is that
component's **unit** tests — do **not** blanket-classify `tests/` as
integration.
4. **Two different "test count" metrics exist — do not mix them.**
- *Test-case declarations* — count of `TEST*(` / `#[test]` / `def test_`
macros (Logging 650, Baselibs ~5.7k).
- *Executed test cases* — the same expanded over `TYPED_TEST` types and
parametrised instances (Baselibs `25833`). Bazel CI reports *test
**targets*** (`Executed N tests` ≈ 297), a **third** number.

The current table predominantly uses the **executed-case** metric. When a
single cell is corrected, match the metric already used in that column;
do **not** silently switch the whole column to a different metric.

> **When in doubt, verify a suspicious cell manually** against the repo at
> the pinned ref (fetch the test files, count) before overwriting — a
> `✅ Available` test cell dropping to `❌ Open` is almost always a discovery
> bug, not a real regression (see the plausibility gate in
> [common.md Step 5b](./common.md#5b-plausibility--diff-each-cell)).

---

## Coverage CI keys

Coverage data is read from the latest successful run of workflow id
Expand Down
62 changes: 34 additions & 28 deletions docs/_assets/pa2_impl_progress.svg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
12 changes: 6 additions & 6 deletions docs/_assets/pa3_arch_progress.svg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
8 changes: 4 additions & 4 deletions docs/_assets/pa4_impl_progress.svg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Loading