From 64611200bb8288e141ccbada5babbc50e6b11e42 Mon Sep 17 00:00:00 2001 From: Xiangzhe Date: Mon, 22 Jun 2026 11:10:33 +0800 Subject: [PATCH 01/85] docs: add rewrite architecture records Document the Flutter app, reusable Rust getter submodule, Lua package repository model, SQLite split, migration bridge, and testing strategy for the UpgradeAll rewrite. --- docs/README.md | 29 + docs/adr/0001-flutter-shell-rust-core.md | 28 + docs/adr/0002-rust-sqlite-storage.md | 24 + .../0003-source-level-page-customization.md | 24 + docs/adr/0004-legacy-room-migration.md | 26 + docs/adr/0005-tdd-bdd-cucumber-policy.md | 35 + docs/adr/0006-getter-library-and-cli.md | 32 + docs/adr/0007-getter-cli-command-contract.md | 90 + docs/adr/README.md | 15 + docs/ai-development.md | 54 + .../flutter-ui-feature-parity-and-testing.md | 89 + docs/architecture/README.md | 25 + ...pp-centric-lua-package-repository-model.md | 99 ++ .../0002-getter-flutter-platform-boundary.md | 53 + .../adr/0003-legacy-room-migration.md | 83 + .../adr/0004-sqlite-main-db-and-cache-db.md | 47 + docs/architecture/adr/0005-lua-package-api.md | 58 + ...06-package-centric-cli-command-contract.md | 120 ++ docs/architecture/target-architecture.md | 143 ++ .../upgradeall-getter-rewrite-wiki.md | 1207 ++++++++++++++ docs/implementation/coding-agent-handoff.md | 124 ++ docs/lua-api/package-lifecycle.md | 66 + docs/lua-api/permissions.md | 33 + docs/lua-api/repository-layout.md | 56 + docs/lua-api/templates.md | 67 + docs/migration/legacy-room-mapping.md | 98 ++ docs/refactor/2026-06-20-refactor-plan.md | 104 ++ ...ll-flutter-getter-rewrite-complete-plan.md | 1468 +++++++++++++++++ ...2026-06-21-reconciled-full-rewrite-plan.md | 602 +++++++ docs/refactor/phase-1-getter-cli-bdd-plan.md | 242 +++ docs/refactor/phase-1a-work-plan.md | 78 + ...phase-1b-getter-workspace-skeleton-plan.md | 48 + docs/testing/bdd-plan.md | 115 ++ 33 files changed, 5382 insertions(+) create mode 100644 docs/README.md create mode 100644 docs/adr/0001-flutter-shell-rust-core.md create mode 100644 docs/adr/0002-rust-sqlite-storage.md create mode 100644 docs/adr/0003-source-level-page-customization.md create mode 100644 docs/adr/0004-legacy-room-migration.md create mode 100644 docs/adr/0005-tdd-bdd-cucumber-policy.md create mode 100644 docs/adr/0006-getter-library-and-cli.md create mode 100644 docs/adr/0007-getter-cli-command-contract.md create mode 100644 docs/adr/README.md create mode 100644 docs/ai-development.md create mode 100644 docs/app/flutter-ui-feature-parity-and-testing.md create mode 100644 docs/architecture/README.md create mode 100644 docs/architecture/adr/0001-app-centric-lua-package-repository-model.md create mode 100644 docs/architecture/adr/0002-getter-flutter-platform-boundary.md create mode 100644 docs/architecture/adr/0003-legacy-room-migration.md create mode 100644 docs/architecture/adr/0004-sqlite-main-db-and-cache-db.md create mode 100644 docs/architecture/adr/0005-lua-package-api.md create mode 100644 docs/architecture/adr/0006-package-centric-cli-command-contract.md create mode 100644 docs/architecture/target-architecture.md create mode 100644 docs/architecture/upgradeall-getter-rewrite-wiki.md create mode 100644 docs/implementation/coding-agent-handoff.md create mode 100644 docs/lua-api/package-lifecycle.md create mode 100644 docs/lua-api/permissions.md create mode 100644 docs/lua-api/repository-layout.md create mode 100644 docs/lua-api/templates.md create mode 100644 docs/migration/legacy-room-mapping.md create mode 100644 docs/refactor/2026-06-20-refactor-plan.md create mode 100644 docs/refactor/2026-06-20-upgradeall-flutter-getter-rewrite-complete-plan.md create mode 100644 docs/refactor/2026-06-21-reconciled-full-rewrite-plan.md create mode 100644 docs/refactor/phase-1-getter-cli-bdd-plan.md create mode 100644 docs/refactor/phase-1a-work-plan.md create mode 100644 docs/refactor/phase-1b-getter-workspace-skeleton-plan.md create mode 100644 docs/testing/bdd-plan.md diff --git a/docs/README.md b/docs/README.md new file mode 100644 index 00000000..e04bc2b8 --- /dev/null +++ b/docs/README.md @@ -0,0 +1,29 @@ +# UpgradeAll Rewrite Documentation + +> Status: Draft / living design record +> Date: 2026-06-21 +> Project: UpgradeAll rewrite — Flutter APP + Rust getter core + Lua package repository model + +This documentation set records the design decisions for the UpgradeAll rewrite. It exists so coding agents and human maintainers can trace every major implementation choice back to a written decision. + +Start here: + +1. `architecture/upgradeall-getter-rewrite-wiki.md` — main living architecture wiki. +2. `architecture/adr/0001-app-centric-lua-package-repository-model.md` — package/repository/Lua model. +3. `architecture/adr/0002-getter-flutter-platform-boundary.md` — getter vs Flutter/platform adapter boundary. +4. `architecture/adr/0003-legacy-room-migration.md` — old Room DB migration strategy. +5. `architecture/adr/0004-sqlite-main-db-and-cache-db.md` — storage and cache split. +6. `architecture/adr/0005-lua-package-api.md` — Lua package API and Rust validation boundary. +7. `lua-api/` — practical Lua package authoring docs. +8. `migration/legacy-room-mapping.md` — old data mapping rules. +9. `app/flutter-ui-feature-parity-and-testing.md` — Flutter feature parity and BDD/TDD test boundary. +10. `implementation/coding-agent-handoff.md` — coding-agent / pi-agent handoff instructions. + +Canonical architecture ADRs live in `docs/architecture/adr/*`. The `docs/adr/*` directory is kept for historical/refactor-phase ADRs and transition notes. + +Documentation policy: + +- Every major decision must be captured in the wiki or an ADR. +- Every cross-boundary API must have schema documentation before implementation stabilizes. +- Every migration must have source/target mapping and failure behavior documented. +- Every coding agent must read `../AGENTS.md` and this docs index before implementation. diff --git a/docs/adr/0001-flutter-shell-rust-core.md b/docs/adr/0001-flutter-shell-rust-core.md new file mode 100644 index 00000000..9fa73f01 --- /dev/null +++ b/docs/adr/0001-flutter-shell-rust-core.md @@ -0,0 +1,28 @@ +# 0001: Flutter shell with getter-owned product logic + +- Date: 2026-06-20 +- Status: Accepted for the refactor plan + +## Context + +UpgradeAll is currently an Android/Kotlin multi-module application with a Rust `getter` submodule already integrated through native Android build tooling. The 2026-06-20 rewrite plan chooses Flutter for the new app shell and moves durable product logic into `getter`. + +The key trade-off is whether the application remains Android/Kotlin-centered or becomes a thin cross-platform shell around a reusable headless engine. + +## Decision + +The rewritten UpgradeAll App will be a Flutter UI/platform shell. `getter` is the headless product engine and owns durable product behavior: source interpretation, update checks, release discovery, download orchestration, provider/downloader registration, storage, migrations, and event streams. + +Flutter must not grow a second copy of getter product logic. UI code may adapt presentation, navigation, platform permissions, and source-level pages, but product decisions must flow through getter contracts. + +## Consequences + +- The app can become cross-platform without duplicating update logic per UI host. +- Getter contracts must be intentionally designed, versioned, documented, and tested. +- UI work cannot start by drawing screens around mock logic; it must be driven by getter-facing behavior scenarios and DTO contracts. +- Android compatibility work remains important because existing installed users must migrate safely. + +## Alternatives considered + +- Keep Android/Kotlin as the product center and call Rust only for selected helpers. This preserves current shape but keeps logic split across platform code and makes Flutter a risky rewrite. +- Make Flutter own product logic and use getter only as a library of utilities. This weakens the reusable engine goal and makes CLI/library support secondary. diff --git a/docs/adr/0002-rust-sqlite-storage.md b/docs/adr/0002-rust-sqlite-storage.md new file mode 100644 index 00000000..da7e1210 --- /dev/null +++ b/docs/adr/0002-rust-sqlite-storage.md @@ -0,0 +1,24 @@ +# 0002: Rust-managed SQLite storage + +- Date: 2026-06-20 +- Status: Accepted for the refactor plan + +## Context + +The rewrite needs a durable storage model that is owned by the headless engine rather than by a specific UI host. The 2026-06-20 plan rejects ad-hoc JSONL as the long-term store and requires a tested migration path from legacy Android Room data. + +## Decision + +Getter will own the new canonical SQLite storage. Legacy Android Room is a migration source, not the long-term source of truth. JSON/JSONL may exist only as import/export, diagnostics, fixtures, or alpha compatibility data, not as the official durable store for the rewritten product. + +## Consequences + +- Storage migrations can be tested at the getter layer without a UI. +- Flutter, CLI, and other hosts share the same durable model. +- A legacy import path must preserve supported existing Android data before the official Flutter Android release. +- Storage schema and canonical ID rules require tests before implementation changes. + +## Alternatives considered + +- Keep Room as the primary store. This preserves existing Android implementation but conflicts with a reusable getter engine. +- Keep JSONL initially and migrate later. This reduces early work but creates a second migration and risks shipping unstable persistence semantics. diff --git a/docs/adr/0003-source-level-page-customization.md b/docs/adr/0003-source-level-page-customization.md new file mode 100644 index 00000000..d0f14968 --- /dev/null +++ b/docs/adr/0003-source-level-page-customization.md @@ -0,0 +1,24 @@ +# 0003: Source-level page customization + +- Date: 2026-06-20 +- Status: Accepted for the refactor plan + +## Context + +UpgradeAll users may want customized pages and flows. Runtime UI plugins would increase app complexity, safety risk, test surface, and compatibility burden. The rewrite plan instead emphasizes source-level downstream customization. + +## Decision + +UpgradeAll will support page customization through source-level modules and typed contracts, not through a v1 runtime UI plugin system. Upstream should provide stable page contracts, default pages, examples, and compile/test failures when custom pages drift from contracts. + +## Consequences + +- Downstream builders can fork, modify pages, run tests, and rebuild. +- Runtime app complexity stays lower than a plugin UI framework. +- Stable route IDs, semantic/test IDs, and page contracts become product requirements. +- Upstream should avoid needless churn in customization surfaces. + +## Alternatives considered + +- Runtime UI plugins. More flexible for installed apps, but much harder to secure, test, and keep compatible during the rewrite. +- No customization boundary. Simpler initially, but conflicts with the selected distribution philosophy. diff --git a/docs/adr/0004-legacy-room-migration.md b/docs/adr/0004-legacy-room-migration.md new file mode 100644 index 00000000..80e1259d --- /dev/null +++ b/docs/adr/0004-legacy-room-migration.md @@ -0,0 +1,26 @@ +# 0004: First-class legacy Room migration + +- Date: 2026-06-20 +- Status: Accepted for the refactor plan + +## Context + +Existing Android users have data in the legacy UpgradeAll Room database. The official Android upgrade path must preserve package identity and user data. Migration failure must be visible and recoverable rather than silently destructive. + +## Decision + +Legacy Android Room migration is a first-class compatibility subsystem. The official Flutter Android upgrade must keep the existing application identity and use a tested import flow from supported legacy Room schemas into getter-owned storage. + +Migration must be transactional from the user's perspective: a failure must not leave a partially usable new app state. The app must provide recovery actions such as retry, report export, and explicit start-fresh confirmation. + +## Consequences + +- The project needs migration fixtures and end-to-end migration tests before release. +- Legacy schema support boundaries must be explicit. +- The legacy migrator can be removed only after a separately documented support decision. +- Android signing/package identity is part of the migration contract. + +## Alternatives considered + +- Best-effort startup migration. Easier to implement but risky for user data. +- Manual export/import only. Avoids direct migration complexity but breaks the official upgrade expectation. diff --git a/docs/adr/0005-tdd-bdd-cucumber-policy.md b/docs/adr/0005-tdd-bdd-cucumber-policy.md new file mode 100644 index 00000000..5f088a25 --- /dev/null +++ b/docs/adr/0005-tdd-bdd-cucumber-policy.md @@ -0,0 +1,35 @@ +# 0005: TDD and Cucumber behavior coverage policy + +- Date: 2026-06-20 +- Status: Accepted for the refactor plan + +## Context + +The refactor must be test-driven. The user clarified that Cucumber/Gherkin BDD is required for user-facing behavior, especially the UpgradeAll App and Getter CLI. BDD should cover integration-level behavior, while internal algorithms and module boundaries should keep faster traditional tests. + +Cucumber documentation defines behavior specs as Gherkin `Feature`, `Scenario`, `Given`, `When`, and `Then` files with tags, data tables, and scenario outlines. Cucumber step definitions bind those phrases to executable code. The Rust Cucumber implementation uses `.feature` files, a per-scenario `World`, and async step functions. + +## Decision + +Every behavior-changing implementation must start from a failing automated test. + +Cucumber/Gherkin is mandatory for supported user-facing interfaces: + +- UpgradeAll App workflows. +- Getter CLI commands, output contracts, errors, and exit codes. +- User-visible migration success and recovery behavior. +- Cross-boundary acceptance behavior where a user action depends on getter outcomes. + +Internal interfaces do not require Gherkin unless promoted to supported user-facing contracts. They should use the fastest appropriate traditional tests: Rust unit/integration/property tests, storage migration tests, Kotlin/Dart unit tests, widget tests, and focused integration tests. + +## Consequences + +- BDD scenarios become acceptance contracts, not a replacement for all unit tests. +- Getter CLI must be designed before implementation because its behavior scenarios need stable commands, JSON/human output rules, and exit-code semantics. +- UI screens must expose stable test IDs so scenarios do not depend on localized text. +- CI/verification must separate fast internal tests from slower BDD acceptance tests while keeping both required before release. + +## Alternatives considered + +- Require Gherkin for every test. This maximizes uniformity but slows feedback and makes low-level Rust/Kotlin/Dart tests verbose. +- Use only native unit/integration tests. Faster initially, but fails the requirement that user-facing behavior be expressed as executable behavior specs. diff --git a/docs/adr/0006-getter-library-and-cli.md b/docs/adr/0006-getter-library-and-cli.md new file mode 100644 index 00000000..c6edf380 --- /dev/null +++ b/docs/adr/0006-getter-library-and-cli.md @@ -0,0 +1,32 @@ +# 0006: Getter as both library and CLI + +- Date: 2026-06-20 +- Status: Accepted for the refactor plan + +## Context + +Getter must serve multiple hosts. The UpgradeAll App needs an embeddable engine, while AI/operator workflows need a scriptable command-line surface. The current Rust crate already has a library entrypoint and a placeholder binary, but that does not define a supported library or CLI contract. + +## Decision + +Getter will be both: + +1. A library: the stable embeddable engine surface for UI hosts and integration adapters. +2. A CLI: the supported command-line user interface for verification, automation, diagnostics, and developer workflows. + +The CLI is a user-facing interface and therefore requires complete Cucumber/Gherkin coverage for supported commands. The library requires traditional unit/integration tests for internal behavior and contract tests where exposed to supported hosts. + +The CLI must not become an unrelated second implementation. It should call the same getter core behavior as the library. + +## Consequences + +- CLI command shape, output mode, error model, and exit codes need explicit design before implementation. +- Behavior scenarios for CLI can drive core workflow design without needing Flutter first. +- The library/CLI split helps prevent UI code from becoming the only way to exercise product behavior. +- Public module visibility must be distinguished from supported API contract. + +## Alternatives considered + +- Library only. Simpler, but weaker for AI/operator workflows and headless verification. +- CLI only. Useful for automation, but not sufficient for embedding in the app. +- Separate CLI logic. Faster to prototype but risks drift from app behavior. diff --git a/docs/adr/0007-getter-cli-command-contract.md b/docs/adr/0007-getter-cli-command-contract.md new file mode 100644 index 00000000..f254d81d --- /dev/null +++ b/docs/adr/0007-getter-cli-command-contract.md @@ -0,0 +1,90 @@ +# 0007: Getter CLI command contract + +- Date: 2026-06-20 +- Status: Accepted for the Phase 1a CLI contract + +## Context + +Getter CLI is a user-facing interface. Once Cucumber/Gherkin scenarios and step assertions are written, command names, output schemas, error schemas, side effects, and exit codes become supported behavior. ADR 0006 says the CLI needs explicit design before implementation. + +The canonical 06-20 plan gives examples such as `getter app list`, `getter hub list`, and `getter legacy import-room-bundle `. The refactor plan is AI/operator/CLI-first, so machine-readable output should be stable from the first slice. Phase 1a implemented this contract in the committed BDD-backed CLI spine. + +## Decision + +Getter CLI uses domain-noun subcommands and machine-readable JSON by default during the rewrite. + +Initial supported command grammar: + +```text +getter --data-dir init +getter --data-dir app list +getter --data-dir hub list +getter --data-dir legacy import-room-bundle +``` + +Global conventions: + +- `--data-dir ` is mandatory in early development and all BDD scenarios. +- JSON is the default output for supported commands. +- Human-readable output can be added later behind an explicit flag, but is not the first automation contract. +- Success payloads go to stdout. +- Error envelopes go to stdout when the command can run far enough to emit structured JSON; invalid CLI usage may use stderr/help text. +- Unstructured diagnostics must not be mixed into JSON stdout. + +Success envelope shape: + +```json +{ + "ok": true, + "command": "app list", + "data": {}, + "warnings": [] +} +``` + +Error envelope shape: + +```json +{ + "ok": false, + "command": "legacy import-room-bundle", + "error": { + "code": "migration.invalid_bundle", + "message": "Legacy Room export bundle is invalid", + "report_path": "/path/to/report.json" + } +} +``` + +Initial exit-code classes: + +- `0`: success. +- `1`: generic failure not covered by a more specific class. +- `2`: invalid CLI usage. +- `10`: data/storage error. +- `20`: migration/import error. +- `30`: network/provider error. +- `40`: download error. + +Storage convention: + +- `getter init` creates or opens the canonical getter-owned SQLite storage. It must not initialize JSONL as durable product storage. +- `legacy import-room-bundle` returns a stable unsupported/not-implemented failure for syntactically valid bundles until the real Room import phase is implemented. +- Minimal Phase 1 storage may contain only metadata and empty app/hub tables, but it must be compatible with the accepted Rust-managed SQLite direction. + +## Consequences + +- BDD scenarios can assert stable JSON fields instead of vague text. +- AI/operator workflows get deterministic output from the beginning. +- Early development avoids accidentally treating platform defaults as part of the contract. +- Human-friendly CLI output remains possible later, but it must not destabilize automation. + +## Alternatives considered + +- Human-readable output by default with `--json` opt-in. Friendlier for terminals, but risks making prose the accidental contract. +- Plural commands such as `apps list`. This is common in some CLIs, but the canonical plan already uses singular `app list` and `hub list`. +- Platform-default data directory from the start. This is convenient for users but makes early BDD tests less isolated and can hide state leakage. + +## Implementation note + +Phase 1a executable CLI feature files are now implemented. Future changes should extend this contract explicitly rather than treating it as provisional. diff --git a/docs/adr/README.md b/docs/adr/README.md new file mode 100644 index 00000000..0ee95a8e --- /dev/null +++ b/docs/adr/README.md @@ -0,0 +1,15 @@ +# Architecture Decision Records + +ADRs in this directory explain decisions that are costly to reverse, surprising without context, and the result of real trade-offs. + +This directory is historical/refactor-phase material. The canonical architecture ADR set lives under `docs/architecture/adr/*`. + +Current refactor ADRs: + +- [0001: Flutter shell with getter-owned product logic](0001-flutter-shell-rust-core.md) +- [0002: Rust-managed SQLite storage](0002-rust-sqlite-storage.md) +- [0003: Source-level page customization](0003-source-level-page-customization.md) +- [0004: First-class legacy Room migration](0004-legacy-room-migration.md) +- [0005: TDD and Cucumber behavior coverage policy](0005-tdd-bdd-cucumber-policy.md) +- [0006: Getter as both library and CLI](0006-getter-library-and-cli.md) +- [0007: Getter CLI command contract](0007-getter-cli-command-contract.md) diff --git a/docs/ai-development.md b/docs/ai-development.md new file mode 100644 index 00000000..c98e4fed --- /dev/null +++ b/docs/ai-development.md @@ -0,0 +1,54 @@ +# AI Development Workflow + +This repository is being prepared for a test-driven Flutter + getter rewrite. + +## Baseline protection + +- Preserve user work before syncing or rewriting. +- Current planning baseline: superproject `4a1aae1d44a418989b0d3d28528cacff0cc066c0`, getter submodule `f011d9b4b9a15f83cd39c86e781ad8830a8ecae6`. +- The canonical 06-20 plan is copied at `docs/refactor/2026-06-20-upgradeall-flutter-getter-rewrite-complete-plan.md`. +- The pre-sync implementation stash is historical context, not accepted architecture. +- Do not apply stash contents wholesale without a fresh review against the ADRs and the canonical plan. + +## Required loop + +For every behavior change: + +1. Identify whether the behavior is user-facing or internal. +2. User-facing App/CLI behavior: add or update a Cucumber/Gherkin scenario first. +3. Internal behavior: add or update the smallest native unit/integration test first. +4. Confirm the test fails for the expected reason. +5. Implement the smallest change. +6. Run focused validation. +7. Run `just verify` before reporting completion. + +## User-facing BDD scope + +Complete BDD coverage is required for: + +- UpgradeAll App workflows. +- Getter CLI commands, outputs, errors, and exit codes. +- User-visible migration success/failure/recovery behavior. + +BDD is not required for every private function or algorithm. Internal behavior still requires automated tests through the appropriate native framework. + +## Planning rules + +- Update `CONTEXT.md` immediately when domain terms become clear. +- Add ADRs only for costly, surprising, trade-off decisions. +- Keep getter product behavior out of UI-only code. +- Keep stable test IDs in UI contracts. +- Do not start Flutter screen work before getter contracts and acceptance scenarios exist. + +## Commands + +Use `just --list` to see available commands. + +Phase 0 command expectations: + +- `just status` checks branch/submodule state. +- `just cargo-metadata` checks Rust manifests stay loadable. +- `just gradle-projects` checks Gradle can configure the current project graph. +- `just verify` runs the current lightweight verification skeleton. + +Later phases must extend `just verify` to include the real Cucumber, Rust, Flutter, migration, and Android release checks. diff --git a/docs/app/flutter-ui-feature-parity-and-testing.md b/docs/app/flutter-ui-feature-parity-and-testing.md new file mode 100644 index 00000000..9f3122fb --- /dev/null +++ b/docs/app/flutter-ui-feature-parity-and-testing.md @@ -0,0 +1,89 @@ +# Flutter UI Feature Parity and Testing Strategy + +> Status: Draft / living design record +> Date: 2026-06-21 +> Project: UpgradeAll rewrite — Flutter APP + Rust getter core + Lua package repository model + +## UI feature parity + +The Flutter UI should preserve these user-visible product capabilities unless explicitly deferred: + +- Home module entry and update summary. +- Apps list and Magisk list. +- App detail with version/source/artifact selection. +- App settings/editing. +- Repository/source visibility. +- Installed-app autogen preview and confirmation. +- Download task view and controls. +- Settings. +- Logs. +- Migration/recovery status. +- Yellow warning tag for free-network Lua scripts. + +## BDD vs TDD boundary + +Use mixed BDD and TDD. + +### TDD + +Use TDD for function/domain behavior: + +- Rust functions. +- repository resolution. +- Lua validation. +- migration mapping. +- cache invalidation. +- version comparison. +- download action generation. +- error classification. + +TDD tests should be small, deterministic and focused. + +### BDD + +Use BDD for UI and integration behavior: + +- Flutter flows. +- migration UX. +- installed autogen confirmation. +- yellow network warning tag. +- update/download task flow. + +BDD scenarios act as self-explaining documentation tests. Do not over-test BDD: each scenario should document a meaningful user behavior or integration boundary. + +## Suggested BDD style + +```gherkin +Feature: Installed app autogen + + Scenario: Generate package scripts for installed apps + Given the device has installed apps not covered by official repository + When the user opens Installed Autogen + And confirms the generated list + Then getter writes package scripts to local_autogen + And the apps appear in the app list as generated fallback packages +``` + +## Current Flutter shell slice + +The first Flutter implementation slice is intentionally a shell, not product logic: + +- App project lives under `app_flutter/`. +- Android identity remains `net.xzos.upgradeall` for future direct upgrade work. +- `UpgradeAllApp` exposes stable route/action/state keys such as `route.home`, `action.open_apps`, `state.apps_list`, and `state.migration_ready`. +- The temporary `GetterAdapter` is fake in-memory data only. It exists to keep UI routes testable until the Rust getter FFI/RPC binding is wired. +- Product decisions such as repository resolution, updates, migrations, storage, and downloads still belong in Rust getter. + +## Test pyramid + +- Many Rust unit tests. +- Moderate Rust integration tests for Lua/package/repository behavior. +- Focused Flutter widget tests for component states. +- Few BDD end-to-end scenarios for critical user flows. + +## Anti-goals + +- Do not use BDD for every function branch. +- Do not test Flutter UI by asserting brittle localized visible strings only. +- Do not duplicate Rust unit coverage in UI tests. +- Do not make migration tests depend on network. diff --git a/docs/architecture/README.md b/docs/architecture/README.md new file mode 100644 index 00000000..4fd130f6 --- /dev/null +++ b/docs/architecture/README.md @@ -0,0 +1,25 @@ +# Architecture Documentation + +This directory records the architecture decisions and design notes for the UpgradeAll rewrite. + +Start here: + +- `upgradeall-getter-rewrite-wiki.md` — main living wiki for the Flutter + Rust getter + Lua package repository redesign. + +Planned / active ADRs: + +- `adr/0001-app-centric-lua-package-repository-model.md` +- `adr/0002-getter-flutter-platform-boundary.md` +- `adr/0003-legacy-room-migration.md` +- `adr/0004-sqlite-main-db-and-cache-db.md` +- `adr/0005-lua-package-api.md` +- `adr/0006-package-centric-cli-command-contract.md` + +Documentation policy: + +- Every important architecture decision should be recorded in this wiki or an ADR. +- Every new module should have a documented responsibility boundary. +- Every cross-boundary API should have a schema document. +- Every migration step should have source/target mapping documentation. +- `docs/architecture/adr/*` is the canonical architecture ADR set. +- `docs/adr/*` is historical/refactor-phase material kept for transition context unless a doc explicitly says otherwise. diff --git a/docs/architecture/adr/0001-app-centric-lua-package-repository-model.md b/docs/architecture/adr/0001-app-centric-lua-package-repository-model.md new file mode 100644 index 00000000..0413572d --- /dev/null +++ b/docs/architecture/adr/0001-app-centric-lua-package-repository-model.md @@ -0,0 +1,99 @@ +# ADR-0001: App-centric Lua package repository model + +> Status: Draft / living design record +> Date: 2026-06-21 +> Project: UpgradeAll rewrite — Flutter APP + Rust getter core + Lua package repository model + +## Decision + +UpgradeAll will replace the old hub-app model with an app/package-centric repository model. + +- The primary user-facing object is an App/package, not a Hub. +- Package IDs are readable UpgradeAll namespaces, not UUIDs. +- Examples: `android/org.fdroid.fdroid`, `android/com.termux`, `magisk/zygisk-next`. +- GitHub, F-Droid, Google Play, CoolApk and similar systems are providers/sources/backends, not package identity. +- A single package may have multiple sources. +- Package definitions are Lua files stored in repositories/overlays. +- Repositories have priorities; higher priority wins. +- getter only sees the top-level resolved package for a given package id. + +## Context + +The previous model represented update logic as App + enabled Hub list. This became insufficient because providers describe where metadata comes from, not what the package is; projects publish artifacts in many different layouts; and different sources for the same installed app should normally be sources of one package. + +The new model takes inspiration from Portage/emerge overlays and Funtoo Metatools/autogen, but does not copy ebuild syntax. It uses Lua as an embedded package definition language via Rust getter. + +## Repository layout + +```text +repo/ + repo.toml + packages/ + android/ + org.fdroid.fdroid.lua + magisk/ + zygisk-next.lua + lib/ + github.lua + android.lua + github_android_apk.lua + templates/ + android_installed_app.lua + github_android_apk.lua +``` + +`packages/` contains final package definitions consumed by getter. + +`lib/` contains reusable Lua modules. These are conceptually like eclasses, but the project does not introduce an `eclass` keyword or syntax. + +`templates/` contains Lua generators that output new package Lua file content. Templates are for autogen workflows, not runtime package evaluation. + +## Repository priority + +Default priority convention: + +```text +local 100 user-written overrides, default highest priority +community/official 0 normal remote repositories +local_autogen -1 generated fallback packages from installed inventory +``` + +The user may edit priorities manually. The only hard rule is: higher priority wins. + +## Import and override + +Reusable Lua modules should use native Lua `require` where practical: + +```lua +local github_android = require("lib.github_android_apk") +``` + +Parent package import uses a host helper because package ids contain slashes/dots and repo id must be explicit: + +```lua +local base = package_from("official", "android/org.fdroid.fdroid") +``` + +Override is a Lua helper/metatable concern, not a Rust API concern. Rust validates only the final returned data object. + +## Consequences + +Positive: + +- App identity is readable and user-supportable. +- Multiple sources become package internals rather than top-level user confusion. +- Users can maintain patch stacks by overriding individual package files. +- Autogen can create fallback local package definitions without contaminating user-authored `local` overrides. + +Costs: + +- getter must implement repository resolution, priority, package loading, Lua execution, validation and cache invalidation. +- Package authors need documentation and examples. +- Lua outputs must be strictly validated by Rust. + +## Non-goals + +- No UUID primary identity for packages. +- No runtime UI customization framework. +- No static-template-only system. +- No guarantee that arbitrary user forks never require rebasing. diff --git a/docs/architecture/adr/0002-getter-flutter-platform-boundary.md b/docs/architecture/adr/0002-getter-flutter-platform-boundary.md new file mode 100644 index 00000000..20d6faf2 --- /dev/null +++ b/docs/architecture/adr/0002-getter-flutter-platform-boundary.md @@ -0,0 +1,53 @@ +# ADR-0002: getter / Flutter / platform boundary + +> Status: Draft / living design record +> Date: 2026-06-21 +> Project: UpgradeAll rewrite — Flutter APP + Rust getter core + Lua package repository model + +## Decision + +All product and domain logic belongs in the Rust getter core. Flutter is a UI shell and platform adapter. + +Getter remains a separate reusable git submodule at `core-getter/src/main/rust/getter`, tracking `https://github.com/DUpdateSystem/getter`. UpgradeAll records a gitlink to a getter commit; getter CLI/core implementation belongs in that submodule, not as vendored superproject files. + +The Android app embeds getter as a Rust library / FFI-style core. The app does not use a standalone getter daemon as the primary path. + +Platform-specific APIs are exposed to getter through RPC/callback-style boundaries so that thread management and platform complexity remain isolated. + +## getter owns + +- Package/repository model. +- Lua package evaluation. +- Provider/source orchestration. +- Version normalization and comparison. +- Release/artifact selection. +- Update status calculation. +- Download request/action generation. +- Download task state machine. +- SQLite main DB and cache DB. +- Legacy migration/import. +- Diagnostics and event streams. +- CLI behavior. + +## Flutter APP owns + +- UI rendering and navigation. +- Android permission prompts. +- Android PackageManager inventory scanning. +- Installed version lookup through platform APIs. +- APK install / Shizuku/root/system installer adapters. +- Notifications / foreground service integration. +- SAF/file picker and URI permissions. +- User confirmation flows. + +## Boundary rule + +If a workflow should be possible from getter CLI without Flutter UI, it belongs in getter. + +If a workflow requires Android APIs or user-interface rendering, it belongs in the Flutter/platform adapter and is exposed to getter as a platform capability. + +## Testing consequence + +- Rust getter behavior is TDD-tested with unit/integration tests. +- Flutter UI and platform flows are BDD-tested through user-visible scenarios. +- Platform adapters get focused integration tests or fake adapter tests. diff --git a/docs/architecture/adr/0003-legacy-room-migration.md b/docs/architecture/adr/0003-legacy-room-migration.md new file mode 100644 index 00000000..cca7f722 --- /dev/null +++ b/docs/architecture/adr/0003-legacy-room-migration.md @@ -0,0 +1,83 @@ +# ADR-0003: Legacy Room migration + +> Status: Draft / living design record +> Date: 2026-06-21 +> Project: UpgradeAll rewrite — Flutter APP + Rust getter core + Lua package repository model + +## Decision + +Old UpgradeAll user data must migrate automatically and without normal-user manual export/import. + +Migration is intentionally limited and simple. It preserves core user-visible app tracking state, but does not attempt to migrate every complex legacy behavior. + +Complex legacy data such as API keys, auth tokens and unusual Hub configuration may be dropped. + +## Source data + +Legacy Room database: + +```text +app_metadata_database.db +version = 17 +entities = app, hub, extra_app, extra_hub +``` + +## Target data + +Migration writes to: + +- getter main SQLite user state. +- `local` repository package Lua files when necessary. +- migration records table. + +Normal installed-app autogen writes to `local_autogen`, but legacy migration is special: it may generate `local` package files once to preserve explicit old user data. + +## Package ID mapping + +- Android apps: `android/`. +- Magisk modules: `magisk/`. + +## Mapping strategy + +1. Detect legacy Room DB. +2. Use bundled official repository snapshot for matching; do not require network at first launch. +3. For common cases, convert legacy app/cloud config to the new package/user state model. +4. If a package is covered by official repository, point user state at that package. +5. If not covered but common conversion exists, generate a `local` package Lua file. +6. Rare/complex cases migrate installed id/tracked state and surface a missing-package diagnostic. +7. Record migration completion. + +## What can be dropped + +- API keys. +- Provider auth tokens. +- Complex or ambiguous Hub auth. +- Legacy settings whose meaning no longer exists. +- Exotic URL replacement rules that cannot be safely mapped. + +## Implemented CLI bridge-bundle slice + +The current Rust CLI implementation does not read Android Room files directly yet. It accepts a JSON bridge bundle for deterministic host-side tests: + +```json +{ + "format": "upgradeall-legacy-room-bundle", + "version": 17, + "apps": [ + { + "kind": "android", + "installed_id": "org.fdroid.fdroid", + "official_package_available": true, + "common_conversion_available": false, + "ignored_version": "1.20.0", + "favorite": true + } + ] +} +``` + +This slice maps `apps[]` into getter tracked package state in `main.db`, writes a sanitized report under `migration-reports/`, and records `legacy-room-v17` completion. Unsupported bundle formats/versions still fail with a sanitized recovery report. + +## Failure behavior + +A single unmapped app must not block the whole app. Global migration failure should lead to a migration/recovery page. A per-app mapping failure should be visible on that app or diagnostics page. diff --git a/docs/architecture/adr/0004-sqlite-main-db-and-cache-db.md b/docs/architecture/adr/0004-sqlite-main-db-and-cache-db.md new file mode 100644 index 00000000..426d0bc3 --- /dev/null +++ b/docs/architecture/adr/0004-sqlite-main-db-and-cache-db.md @@ -0,0 +1,47 @@ +# ADR-0004: SQLite main DB and cache DB + +> Status: Draft / living design record +> Date: 2026-06-21 +> Project: UpgradeAll rewrite — Flutter APP + Rust getter core + Lua package repository model + +## Decision + +getter uses SQLite for backend storage. + +The storage is split into: + +1. Main DB: authoritative user and getter state. +2. Cache DB: derived/evaluated/provider/cache state. + +Users manually corrupting backend storage is considered non-standard usage. getter may fail fast with a clear error. + +## Main DB stores + +- Repository registry and priority. +- Enabled/tracked apps. +- User source priority overrides. +- Ignored versions, pins, favorites. +- Migration records. +- Settings and credential references. +- Download task persistent state. + +## Cache DB stores + +- Evaluated package metadata. +- Lua validation results. +- Release candidates and selected latest versions. +- Artifact metadata. +- Provider response cache. +- Search index. + +## Cache invalidation keys + +Cache keys should include repo id, repo revision/hash, package file hash, Lua API version, getter/package schema version, platform target and permissions/network mode. + +## Repo source files + +Package Lua files live in filesystem repositories, not inside the main DB. SQLite records repository path/revision/priority and evaluated/cache results. + +## Rationale + +SQLite is chosen over transparent text files for backend state because mobile app data needs atomic updates, reliable migrations, consistent concurrent operations and robust cache/query behavior. Text-like transparency is preserved at the package repository layer through Lua files. diff --git a/docs/architecture/adr/0005-lua-package-api.md b/docs/architecture/adr/0005-lua-package-api.md new file mode 100644 index 00000000..9a5bd087 --- /dev/null +++ b/docs/architecture/adr/0005-lua-package-api.md @@ -0,0 +1,58 @@ +# ADR-0005: Lua package API and Rust validation boundary + +> Status: Draft / living design record +> Date: 2026-06-21 +> Project: UpgradeAll rewrite — Flutter APP + Rust getter core + Lua package repository model + +## Decision + +getter embeds Lua for package definitions, reusable package helpers and autogen templates. + +The Lua/Rust boundary is treated as an RPC/serialization boundary: Lua returns JSON-like tables; Rust validates and deserializes them into typed structs. + +Lua scripts do not receive mutable Rust domain objects. + +## Language + +Use Lua via `mlua` unless implementation evidence later proves a blocker. + +## Boundaries + +Lua can use normal Lua tables/functions/metatables, reusable modules via `require`, package import helper for parent packages, and host-provided provider/network APIs based on permissions. + +Rust owns schema validation, typed domain model, persistence, event dispatch, download task state and platform callback dispatch. + +## Lifecycle phases + +App-centric phase names: + +```text +preflight +setup +match +discover +prepare +select +resolve # name still open; alternative: make_actions +post_update +``` + +`plan` is rejected because it is too vague. + +## Network permission model + +Lua has no direct network API by default. + +If a package declares free network permission, getter exposes a direct network host API to that Lua environment and Flutter displays a yellow warning tag in App detail source/version UI. + +This tag is informative and does not block use. + +## Templates + +Templates under `templates/` are Lua generators that output Lua package file content. They are distinct from runtime package modules. + +## Validation + +Rust validates package id/path consistency, known package kind, required fields, installed target schema, phase function presence/type where required, permission schema, action schema and URL/action validity. + +Errors must distinguish Lua runtime errors, schema validation errors and domain validation errors. diff --git a/docs/architecture/adr/0006-package-centric-cli-command-contract.md b/docs/architecture/adr/0006-package-centric-cli-command-contract.md new file mode 100644 index 00000000..8e995f53 --- /dev/null +++ b/docs/architecture/adr/0006-package-centric-cli-command-contract.md @@ -0,0 +1,120 @@ +# ADR-0006: Package-centric getter CLI command contract + +> Status: Draft / implementation slice accepted +> Date: 2026-06-21 +> Project: UpgradeAll rewrite — Flutter APP + Rust getter core + Lua package repository model + +## Decision + +The getter CLI is a first-class user-facing interface for exercising Rust getter behavior without Flutter. + +The supported rewrite CLI vocabulary is package/repository-centric. New commands should use `repo`, `package`, `app`, `storage`, and `legacy` nouns. The old `hub` noun is not a new domain model; it is kept only as a temporary Phase 1a compatibility command for legacy/background plans and must not grow into a hub-app architecture. + +Initial implemented grammar: + +```text +getter --data-dir init +getter --data-dir app list +getter --data-dir repo list +getter --data-dir repo add [--priority ] +getter --data-dir repo eval +getter --data-dir package eval [--repo ] +getter --data-dir storage validate +getter --data-dir legacy import-room-bundle +getter --data-dir hub list # temporary compatibility only +``` + +Global conventions: + +- `--data-dir ` is mandatory in early development and BDD tests. +- JSON is the default output contract. +- Successful command payloads go to stdout. +- Structured command failures go to stdout as JSON error envelopes when possible. +- Invalid CLI usage may additionally use stderr/help text and exit code `2`. +- The CLI must call Rust getter/storage behavior; it must not duplicate product logic outside getter. +- CLI scenarios must invoke the built binary as an external process. +- `package eval ` without `--repo` evaluates the package from the highest-priority registered repository that contains that package id. `--repo ` evaluates that exact repository and bypasses overlay resolution. + +Success envelope shape: + +```json +{ + "ok": true, + "command": "repo list", + "data": {}, + "warnings": [] +} +``` + +Error envelope shape: + +```json +{ + "ok": false, + "command": "legacy import-room-bundle", + "error": { + "code": "migration.invalid_bundle", + "message": "Legacy Room export bundle is invalid", + "report_path": "/path/to/report.json" + } +} +``` + +The first supported `legacy import-room-bundle` slice accepts a JSON bridge bundle with this shape: + +```json +{ + "format": "upgradeall-legacy-room-bundle", + "version": 17, + "apps": [ + { + "kind": "android", + "installed_id": "org.fdroid.fdroid", + "official_package_available": true, + "common_conversion_available": false, + "ignored_version": "1.20.0", + "favorite": true + } + ] +} +``` + +It maps `apps[]` into getter tracked package state in `main.db`, writes a sanitized report under `migration-reports/`, and records `legacy-room-v17` migration completion. Malformed JSON uses `migration.invalid_bundle`; wrong format/version uses `migration.unsupported_bundle`. + +Exit-code classes: + +- `0`: success. +- `1`: generic structured command failure. +- `2`: invalid CLI usage. +- `10`: data/storage error. +- `20`: migration/import error. +- `30`: future network/provider error. +- `40`: future download error. + +## Context + +The rewrite architecture requires getter core to be independently exercisable before Flutter UI work. A CLI-first spine proves that storage, repository loading, Lua package evaluation, migration error reporting, and later update workflows can run without platform/UI code. + +Older Phase 1a docs accepted `getter hub list` as a temporary smoke command. Newer architecture docs reject hub-app as the future model. This ADR reconciles those facts: `hub list` may remain as a no-op compatibility smoke while package/repository commands become the forward path. + +## Consequences + +Positive: + +- The CLI can be used for BDD/runtime evidence and AI/operator workflows. +- Flutter cannot hide missing getter behavior behind UI code. +- Package/repository terminology stays aligned with ADR-0001. +- Legacy import failures can be tested non-destructively before full Room import exists. + +Costs: + +- Command grammar changes must be documented and covered by Gherkin tests. +- The temporary `hub list` compatibility command must be removed or clearly deprecated later. +- CLI output schemas become a supported automation contract. + +## Non-goals + +- No old hub-app model revival. +- No live network provider behavior in the initial CLI smoke slice. +- No direct Android Room database reader in the JSON bridge-bundle slice. +- No Flutter UI behavior in CLI tests. diff --git a/docs/architecture/target-architecture.md b/docs/architecture/target-architecture.md new file mode 100644 index 00000000..ea426847 --- /dev/null +++ b/docs/architecture/target-architecture.md @@ -0,0 +1,143 @@ +# Target Architecture + +Date: 2026-06-20 + +## Source basis + +This document is based on the copied root 2026-06-20 rewrite plan at `docs/refactor/2026-06-20-upgradeall-flutter-getter-rewrite-complete-plan.md`, the synced repository state, current code inspection, Cucumber documentation lookup, and the user's clarified testing rule. + +Canonical plan hash: + +- SHA-256: `a9d02ce7fb88112506580a6e5e723494016ff75cc950083f66ab93701bbc3a0a` +- Copied from `xz@100.65.231.22:/home/xz/.hermes/plans/2026-06-20_181038-upgradeall-flutter-getter-rewrite-complete-plan.md` +- Matches the plan captured in the pre-sync stash untracked parent. + +> All user-facing functions/interfaces need BDD Cucumber coverage. The main user-facing surfaces are the UpgradeAll App and Getter CLI. Internal interfaces use unit/integration/traditional tests because BDD fits integration behavior better than algorithm-level unit tests. + +## Exact repository baseline + +Superproject: + +- Branch used for planning: `refactor/phase0-planning-20260620` +- Synced upstream branch: `master` / `origin/master` +- Baseline commit: `4a1aae1d44a418989b0d3d28528cacff0cc066c0` +- Baseline commit subject: `feat: hub authentication UI with auth_keywords support` +- Pre-sync local backup branch: `backup/pre-sync-master-20260620-183445` at `8a820a76bfee22228272912e4e10127b63284583` + +Getter submodule: + +- Path: `core-getter/src/main/rust/getter` +- Baseline commit: `f011d9b4b9a15f83cd39c86e781ad8830a8ecae6` +- Baseline subject: `feat: add auth_keywords to HubItem and manager_update_hub_auth RPC` +- Pre-sync submodule backup branch: `backup/pre-sync-20260620-183445` at `73a5fc921ef4644346f8b984ac4f10394b7ba291` + +Stash backups: + +- Superproject WIP backup: `stash@{1}` / `b9462fb0c8f15b1ffddd2cd36125e21e2a4b9a09`, message `backup before 2026-06-20 refactor planning 20260620-183445 (superproject)` +- Submodule WIP backup: `core-getter/src/main/rust/getter` `stash@{0}` / `ac6c76288d069b047a784df6aceb82536e870e49`, message `backup before 2026-06-20 refactor planning 20260620-183445 (submodule getter)` +- Agent artifact backup: `stash@{0}` / `7d668a1e0514972c23911f29ec11b08763db222a` in the superproject, message `agent artifacts after refactor planning context 20260620-185313` + +Current Android app identity: + +- `applicationId`: `net.xzos.upgradeall` +- `namespace`: `net.xzos.upgradeall` +- `versionCode`: `105` +- `versionName`: `0.20-alpha.4` +- `compileSdk`: `36` +- `targetSdk`: `36` +- `minSdk`: `23` + +Current module graph: + +- `:app` +- `:core` +- `:core-websdk` +- `:core-utils` +- `:core-shell` +- `:core-downloader` +- `:core-installer` +- `:core-android-utils` +- `:app-backup` +- `:core-getter` +- `:core-websdk:data` +- `:core-getter:provider` +- `:core-getter:rpc` + +Current build facts: + +- Gradle wrapper: `9.3.1` +- AGP: `9.0.1` +- Kotlin: `2.3.10` +- Android Rust Gradle plugin: `0.6.0` +- Java/Kotlin toolchain: `21` +- `core-getter` builds Rust `api_proxy` for Android ABIs through the Android Rust Gradle plugin. +- Top-level Gradle configuration runs Cargo metadata for `core-getter/src/main/rust/api_proxy/Cargo.toml`; breaking Cargo metadata can break Gradle configuration before tests run. + +Current getter facts: + +- The Rust getter crate already has `src/lib.rs` and `src/main.rs`. +- `src/main.rs` currently only prints `Hello, world!`, so the CLI exists structurally but not as a supported interface. +- Existing Rust tests use traditional Rust test tooling and fixtures; no Cucumber/Gherkin dependency is present yet. + +## Target runtime layers + +1. **Getter Core** owns product behavior and durable state. +2. **Getter Library** exposes the embeddable engine contract used by app/platform adapters. +3. **Getter CLI** exposes the command-line user interface for automation, diagnostics, and AI/operator workflows. +4. **UpgradeAll App** is the graphical shell and platform integration layer. +5. **Legacy Migrator** preserves supported Android user data during official upgrade. +6. **Source-level page modules** provide downstream UI customization through typed contracts and stable test IDs. + +## Testing architecture + +Testing is layered by audience and feedback speed: + +- **BDD Cucumber/Gherkin acceptance tests**: required for user-facing UpgradeAll App behavior and Getter CLI behavior. +- **UI/widget tests**: required for page states, stable IDs, and rendering contracts. +- **Getter traditional tests**: required for algorithms, parsers, provider behavior, storage, migration, download orchestration, and library contracts. +- **Migration tests**: required before Android release, including success and failure recovery paths. +- **Black-box UI flows**: required for primary app flows using stable semantic/test IDs; these may be generated from or mapped to Gherkin scenarios. + +## Phase gates + +### Phase 0: Planning and verification skeleton + +- Record glossary and ADRs. +- Create a single verification entrypoint. +- Do not revive stashed implementation work as accepted architecture. +- Do not implement product behavior before the first failing test is defined. + +### Phase 1: Getter workspace and API seams + +- Split getter by domain boundaries only after ADRs are accepted. +- Preserve Cargo metadata compatibility for Gradle during transitions. +- Define library and CLI contracts before filling behavior. + +### Phase 2: Storage and migration foundation + +- Implement Rust-managed SQLite behind getter tests. +- Create legacy import fixtures and failure semantics. + +### Phase 3: CLI-first behavior slices + +- Use Getter CLI Cucumber scenarios to drive headless product behavior. +- Reuse the same core behavior from library and CLI. + +### Phase 4: Flutter app shell and UI BDD + +- Build UI around getter contracts. +- Every public route/action/state receives stable IDs. +- App behavior scenarios drive integration tests. + +### Phase 5: Android migration release readiness + +- End-to-end migration tests on supported legacy states. +- Official Android identity preserved for direct upgrade. +- Recovery/reporting behavior verified. + +## Non-goals for Phase 0 + +- No production code rewrite. +- No choice to delete `:core-getter:rpc` unless an ADR explicitly replaces that boundary. +- No assumption that the stashed direct-JNI work is the approved direction. +- No Flutter screen implementation before getter contracts and behavior tests exist. diff --git a/docs/architecture/upgradeall-getter-rewrite-wiki.md b/docs/architecture/upgradeall-getter-rewrite-wiki.md new file mode 100644 index 00000000..1d1b31d0 --- /dev/null +++ b/docs/architecture/upgradeall-getter-rewrite-wiki.md @@ -0,0 +1,1207 @@ +# UpgradeAll getter 重构架构设计 Wiki + +> 状态：设计草案 / living document +> 日期：2026-06-21 17:27 CST +> 适用范围：UpgradeAll 从旧 Android/Kotlin + Room + hub-app 模型，重构为 Flutter UI + Rust getter core + Lua package repository 模型。 +> 设计原则：所有重要代码边界、数据模型、迁移策略和架构决策都必须记录在案；后续实现必须同步更新本文或对应 ADR。 + +--- + +## 0. 文档目的 + +本文是 UpgradeAll 新架构的主设计文档，用来约束后续代码实现、重构计划、迁移策略和 wiki/开发文档。 + +本文不是单纯的想法记录，而是用于回答这些问题： + +1. 为什么放弃旧的 `hub-app` 模型。 +2. 新的 `getter` 和 `APP` 边界是什么。 +3. 为什么所有 product/domain logic 都进入 Rust getter。 +4. 为什么新 UI 使用 Flutter。 +5. 为什么 getter backend storage 使用 SQLite。 +6. 为什么 package/update 模型采用 Lua package repository，而不是固定模板或旧 Hub。 +7. Lua package 脚本如何组织、导入、覆写、生成和校验。 +8. 旧数据如何无感迁移。 +9. 用户二次开发、AI fork、patch stack 如何不被架构拖累。 +10. 哪些决策已经锁定，哪些仍是 open question。 + +后续规则： + +- 每个重要代码模块都应能在本文或后续 ADR 中找到设计依据。 +- 每个破坏性决策都应有「为什么不选其他方案」。 +- 每个迁移逻辑都应记录数据来源、目标、保留字段和丢弃字段。 +- 每个 Lua API / Rust API / Flutter adapter API 都应有边界说明。 + +--- + +## 1. 背景：现有 UpgradeAll 的事实基础 + +### 1.1 当前产品定位 + +当前 UpgradeAll 是 Android 上的更新检查/下载工具，核心能力包括： + +- 检查 Android apps、Magisk modules 等对象的更新。 +- 从多个来源获取 release/update 信息，例如 GitHub、GitLab、F-Droid、Google Play、CoolApk、Source List / cloud config。 +- 支持用户自定义规则、Hub/App 配置、外部下载器、本地/云备份、日志、安装器等能力。 + +代码审计来源： + +- `/home/xz/workspace/upgradeall-audit/upgradeall-current-context-map.md` +- `settings.gradle` +- `app/build.gradle` +- `app/src/main/AndroidManifest.xml` +- `core/src/main/java/net/xzos/upgradeall/core/database/MetaDatabase.kt` +- `core-getter/rpc/src/main/java/net/xzos/upgradeall/getter/rpc/GetterService.kt` + +关键事实： + +- 官方 Android applicationId 是 `net.xzos.upgradeall`。 +- 当前版本信息：`versionCode = 105`, `versionName = "0.20-alpha.4"`。 +- Debug build 使用 `applicationIdSuffix ".debug"`，不能代表正式升级路径。 +- 当前 app 仍以 Activity / Fragment / XML / DataBinding / ViewBinding 为主。 +- Compose 依赖存在，但不是主 UI 架构。 +- `core-getter` 已经有 Rust getter 的 JNI/RPC 集成，但目前仍是过渡形态。 + +### 1.2 当前 Gradle 模块 + +现有模块： + +```text +:app +:core +:core-websdk +:core-utils +:core-shell +:core-downloader +:core-installer +:core-android-utils +:app-backup +:core-getter +:core-websdk:data +:core-getter:provider +:core-getter:rpc +``` + +当前职责概括： + +- `:app`：Android UI、Activity/Fragment、WorkManager、偏好设置、日志、文件管理等。 +- `:core`：Room DB、App/Hub/domain 状态、版本比较、更新状态推导、manager 薄壳。 +- `:core-websdk`：旧 Web SDK API 与 Rust getter 代理桥接；Kotlin hub RPC server；GooglePlay/CoolApk 回调。 +- `:core-downloader`：下载相关 Android/Kotlin 层能力。 +- `:core-installer`：安装器相关能力。 +- `:core-android-utils`：PackageManager / Android 文件与系统工具。 +- `:app-backup`：本地 zip 备份/恢复与 WebDAV 云备份。 +- `:core-getter`：JNI/native Rust api_proxy + GetterPort。 +- `:core-getter:rpc`：Kotlin WebSocket JSON-RPC client 和 DTO。 + +### 1.3 当前用户可见功能 + +新架构必须理解并有意识地处理这些现有功能： + +- Home：模块入口、检查更新、自动检查更新、更新数量展示、普通/简化模式。 +- Apps/Magisk：按 app type 展示，包含 Updates/Star/All/Applications 条件 tab，支持添加、编辑、删除、批量更新/忽略。 +- Discover：发现 cloud config/source list 中的 app 配置，搜索、刷新、导入。 +- Hub Manager：启用/禁用 Hub、applications mode、认证、URL replace、全局设置。 +- App Detail：版本选择、查看 changelog/more URL、下载 asset、编辑 App、改 source/Hub 优先级、忽略当前版本。 +- File Management：下载任务状态、暂停/继续/重试/删除/安装/打开文件。 +- Settings：Backup、Downloader、UI、Updates、Language、Installation。 +- Log：分类查看、清空、导出。 +- Restore/Migration：恢复/迁移进度页。 + +这些功能不一定一比一保留旧 UI，但产品语义必须被新架构覆盖或明确标记为 v1 非目标。 + +--- + +## 2. 旧架构的问题 + +### 2.1 `hub-app` 模型已经不够 + +旧模型大致是： + +```text +App + app_id + enable_hub_list + cloud_config + +Hub + GitHub / F-Droid / GooglePlay / CoolApk / Source List +``` + +这个模型的问题： + +1. GitHub/F-Droid/Google Play/CoolApk 本质上不是「包」，而是 provider/source/backend。 +2. 同一个 App 可以来自多个来源，但它仍应是同一个更新对象。 +3. 不同项目的发布方式差异极大，固定 Hub 模板会无限膨胀。 +4. App 的打包、版本选择、asset 选择、校验、安装对象匹配都应是 package 级别逻辑，而不是 Hub 级别逻辑。 +5. 旧模型难以表达类似 package manager 的 repository/overlay/override 关系。 + +结论：新架构放弃 `hub-app` 模型，改为 app/package-centric 模型。 + +### 2.2 渐进式剥离失败 + +当前代码已经尝试将部分逻辑迁移到 Rust getter，但仍存在： + +- Room 与 Rust JSONL 并存。 +- Kotlin AppManager/HubManager 仍承担大量状态/业务逻辑。 +- `migrateRoomToRust()` 是一次性倒账，不是正式迁移系统。 +- 旧 UI、旧 DB、旧 Hub、Rust getter 的边界复杂交错。 +- 兴趣开发无法长期维持这种双架构过渡成本。 + +结论：新版本从零重构，不继续渐进式剥离。 + +### 2.3 当前 Room -> Rust 迁移技术债 + +当前 `migrateRoomToRust()` 的问题： + +- 只在 `apps.jsonl` 不存在或为空时执行。 +- 从 Room 读取 apps/hubs/extra_hub。 +- 没有覆盖 `extra_app`。 +- AppEntity 迁移时 Rust 会重新分配 app UUID。 +- 没有持续同步或双向同步。 +- 它是启动时一次性倒账，不是版本化、事务性、可验证的正式迁移。 + +结论：正式重构不能沿用该方案。 + +--- + +## 3. 新架构总览 + +### 3.1 核心决策 + +已锁定决策： + +1. 新 UI 使用 Flutter。 +2. getter 使用 Rust。 +3. 所有 product/domain logic 都放在 getter。 +4. Android App 只是 Flutter UI + platform adapter。 +5. App 内 getter 形态采用嵌入式 Rust library / FFI 风格，不以 daemon 作为主路径。 +6. 平台专用 API 通过 RPC/callback 暴露给 getter，用于隐藏平台线程/API 复杂度。 +7. 后端存储使用 SQLite。 +8. 用户通过非标准方式改坏 backend storage 时，getter fail fast 报错，不提供复杂恢复引导。 +9. 用户二次开发采用 patch stack/source fork 模式，不设计复杂 runtime customization/plugin 系统。 +10. 旧数据迁移必须对普通用户无感自动完成，同时可提供手动导入。 + +### 3.2 顶层结构 + +目标结构： + +```text +Flutter APP + - UI rendering + - navigation + - Android permissions + - Android PackageManager inventory + - installer adapter + - notification adapter + - SAF/file picker adapter + - platform RPC/callback server + | + | FFI / RPC-like boundary + v +Rust getter core + - app/package model + - repository/overlay resolution + - Lua package evaluation + - update discovery/select/resolve + - provider/source backends + - download task state machine + - SQLite main DB + - cache DB + - legacy migration + - CLI API +``` + +### 3.3 Getter 必须拥有的能力 + +必须进入 getter core： + +- App/package identity。 +- Repository/overlay 管理。 +- Lua package loading/evaluation。 +- Package update lifecycle。 +- Provider/source backend。 +- Version parsing/comparison/filtering。 +- Release/artifact normalization。 +- Update status calculation。 +- Download request/action generation。 +- Download task state machine。 +- Main SQLite storage。 +- Cache DB。 +- Legacy migration/import。 +- Event stream。 +- CLI API。 +- Diagnostics/error reporting。 + +### 3.4 APP/platform adapter 保留的能力 + +保留在 Flutter/Android adapter： + +- Android PackageManager installed app scanning。 +- Android installed version lookup。 +- APK install / package installer / Shizuku/root installer。 +- Android permission request。 +- Notification / foreground service integration。 +- SAF/file picker/URI permission。 +- Activity/UI navigation。 +- Android-specific file opening intents。 +- Theme/localization/user-facing UI preferences。 + +--- + +## 4. Package-centric 模型 + +### 4.1 Package ID + +Package 主 ID 使用 UpgradeAll 自己的可读 namespace，不使用 UUID 作为主身份。 + +示例： + +```text +android/org.fdroid.fdroid +android/com.termux +magisk/zygisk-next +generic/example-tool +``` + +设计理由： + +- UUID 对用户无意义。 +- package ID 应可读、可 diff、可手写、可在 issue/文档中引用。 +- Android 和 Magisk 迁移可以自然映射。 + +旧数据映射： + +- 旧 Android app：`android/`。 +- 旧 Magisk module：`magisk/`。 + +### 4.2 APP/package-centric，而不是 hub-centric + +用户界面和 getter 的用户可见概念应围绕 App/package，而不是 Hub。 + +旧 Hub 的概念拆分为： + +- repository：一组 package Lua 文件和 reusable modules。 +- provider/source：GitHub、F-Droid、Google Play、CoolApk 等访问后端。 +- package：一个可维护更新单元。 +- installed target：本机安装对象，如 Android package 或 Magisk module。 +- user state：enabled、ignore、source priority、favorite、overrides 等用户状态。 + +CLI/UI 命名建议： + +- UI：Apps / Modules / Repositories / Sources。 +- CLI：可以使用 `getter app ...` 面向用户。 +- Rust 内部：使用 `Package` / `ResolvedPackage`。 + +### 4.3 多来源同一 package + +同一个 Android App 如果可来自 F-Droid、GitHub、Google Play，它应是同一个 package 的多个 source/provider，而不是多个 package。 + +例如： + +```lua +return android_app { + id = "android/org.fdroid.fdroid", + name = "F-Droid", + installed = android.package("org.fdroid.fdroid"), + sources = { + fdroid.package { package = "org.fdroid.fdroid" }, + github.release { repo = "f-droid/fdroidclient" }, + }, +} +``` + +source priority 可以来自 package 默认值，也可以被 user state 覆盖。 + +--- + +## 5. Repository / overlay 模型 + +### 5.1 Repository 类型 + +新架构使用 repository/overlay 模型，参考 Portage/emerge 的 overlay 思路。 + +Repository 可以是： + +- official：官方包定义仓库。 +- community：社区包定义仓库。 +- local_autogen：自动生成的本地包仓库。 +- local：用户手写/覆盖仓库。 + +### 5.2 Priority 规则 + +优先级规则： + +- 数字越大优先级越高。 +- getter resolved view 只看最高优先级 package。 +- 用户可以手动修改 repo priority。 + +默认建议： + +```text +local 100 用户手写覆盖，默认最高 +official 0 官方仓库 +community 0 或用户配置 +local_autogen -1 根据已安装应用自动生成的 fallback +``` + +注意：`local` 只是默认最高，用户可以自己改优先级。 + +### 5.3 local 与 local_autogen 的区别 + +`local`： + +- 用户手写/编辑。 +- 用于明确覆盖上游 package。 +- 默认 priority 最高。 +- 普通清理按钮不应删除 `local`。 + +`local_autogen`： + +- 用户点击“从已安装应用生成”后产生。 +- 是低优先级 fallback。 +- 上游 official package 出现后，official 会覆盖它。 +- 清理按钮只作用于该 autogen 仓库。 + +仓库名固定为 `local_autogen`。它表达“本地自动生成的 fallback 仓库”。 + +### 5.4 首次旧数据迁移与 autogen 的区别 + +旧数据迁移是特殊情况： + +- 首启迁移必须无感。 +- 迁移可以一次性生成 `local` package 文件，以保留用户旧配置。 +- 该行为只发生一次。 + +普通 installed autogen： + +- 是用户主动点击按钮触发。 +- 生成到 `local_autogen`。 +- 不是首启迁移的一部分。 + +--- + +## 6. Repository 文件布局 + +建议 layout： + +```text +repo/ + repo.toml + + packages/ + android/ + org.fdroid.fdroid.lua + com.termux.lua + magisk/ + zygisk-next.lua + + lib/ + std.lua + github.lua + fdroid.lua + google_play.lua + coolapk.lua + android.lua + magisk.lua + github_android_apk.lua + fdroid_android_apk.lua + + templates/ + android_installed_app.lua + magisk_installed_module.lua + github_android_apk.lua + fdroid_android_apk.lua +``` + +`repo.toml` 示例： + +```toml +id = "official" +name = "UpgradeAll Official" +priority = 0 +api_version = "getter.repo.v1" +``` + +### 6.1 packages/ + +`packages/` 里是最终被 getter 解析的 package Lua 文件。 + +路径建议： + +```text +packages/android/org.fdroid.fdroid.lua +packages/magisk/zygisk-next.lua +``` + +路径可推导 package id： + +```text +packages/android/org.fdroid.fdroid.lua -> android/org.fdroid.fdroid +``` + +文件内也应声明同样 id，getter 校验路径 id 和声明 id 一致。 + +### 6.2 lib/ + +`lib/` 里是 reusable Lua module。 + +注意：这里的角色类似 Gentoo eclass，但项目语法里不需要真的叫 eclass。 + +原则： + +- 不限定 lib 里写什么。 +- 只抽象重复代码。 +- 可以提供高层 helper，例如 `github_android_apk { ... }`。 +- package 文件通过 Lua 原生 `require()` 导入。 + +示例： + +```lua +local github_android = require("lib.github_android_apk") +``` + +### 6.3 templates/ + +`templates/` 里是 Lua 生成器，用于生成新的 package Lua 文件内容。 + +这参考 Funtoo Metatools/autogen： + +- Funtoo metatools 用 autogen.py/autogen.yaml 查询 upstream 并生成 ebuild。 +- UpgradeAll 的 templates 用 Lua 根据 installed inventory 或用户输入生成 package Lua。 + +template 直接返回文件路径和文本内容，而不是返回 AST。 + +示例： + +```lua +return template { + id = "android_installed_app", + + generate = function(ctx, input) + return { + path = "packages/android/" .. input.package_name .. ".lua", + content = [[ +local android = require("lib.android") + +return android.local_app { + id = "android/]] .. input.package_name .. [[", + name = "]] .. input.label .. [[", + package_name = "]] .. input.package_name .. [[", +} +]] + } + end +} +``` + +--- + +## 7. Lua package API + +### 7.1 语言选择 + +内嵌语言：Lua。 + +优先实现：`mlua`。 + +理由： + +- Rust 集成成熟。 +- 语言小，适合作为嵌入式脚本。 +- 支持 metatable，可实现继承/override/object helper。 +- 适合 ebuild/eclass-like 的可编程 package definition。 +- AI 和用户都比较容易读写。 + +### 7.2 不发明自定义语法 + +原则： + +- 尽可能使用 Lua 原生语法。 +- 不维护复杂自定义语法。 +- 不引入新的 DSL parser。 +- package override/object 行为用 Lua table/metatable/helper 实现。 + +### 7.3 Parent package import + +父包导入使用 host helper： + +```lua +local base = package_from("official", "android/org.fdroid.fdroid") +``` + +理由： + +- package id 里有 `/`、`.`、`-` 等字符。 +- Lua 原生 `require()` 会把 `.` 当模块路径分隔。 +- parent package import 需要显式 repo id，避免 priority/递归歧义。 +- 这是 host function，不是新语法。 + +Reusable module 仍使用 Lua `require()`： + +```lua +local github = require("lib.github") +``` + +### 7.4 Lua/Rust boundary / Lua/Rust 边界 + +Lua package scripts 在边界返回 JSON-like object/table。 + +原则： + +- Lua↔Rust crossing 视为 RPC/serialization boundary。 +- Lua 返回 plain data。 +- Rust validate/deserialize 成 typed structs。 +- 如果 mlua 能直接把 Lua table 映射到 Rust struct，可以作为实现细节。 +- 概念上不暴露可变 Rust domain object 给 Lua。 + +好处： + +- Lua API 简单。 +- cache/debug 输出可检查。 +- 不绑定 Rust 内部对象生命周期。 +- 错误模型清晰。 + +错误分层： + +1. Lua runtime error：脚本执行失败。 +2. Schema validation error：Lua 返回 table，但字段不符合 schema。 +3. Domain error：schema 合法，但语义不成立。 + +### 7.5 Package 文件示例 + +官方 package： + +```lua +local github_android = require("lib.github_android_apk") + +return github_android { + id = "android/org.fdroid.fdroid", + name = "F-Droid", + android_package = "org.fdroid.fdroid", + repo = "f-droid/fdroidclient", + asset_pattern = "%.apk$", +} +``` + +本地 override： + +```lua +local base = package_from("official", "android/org.fdroid.fdroid") + +return base:override(function(pkg) + pkg.name = "F-Droid Custom" + pkg.source_priority = { "github", "fdroid" } + + local parent_select = pkg.select + function pkg:select(ctx, candidates, installed, user_state) + local selected = parent_select(self, ctx, candidates, installed, user_state) + selected.channel = "custom" + return selected + end +end) +``` + +--- + +## 8. Override API + +### 8.1 为什么需要 override helper + +用户如果想修改上游 package，不应复制整个上游文件。 + +目标： + +- 用户可以引用父包。 +- 用户只改需要改的字段或 hook。 +- 上游更新时，用户 patch 尽量不冲突。 + +### 8.2 Table override + +适合简单字段替换： + +```lua +local base = package_from("official", "android/org.fdroid.fdroid") + +return base:override { + name = "F-Droid Custom", + source_priority = { "github", "fdroid" }, +} +``` + +语义： + +- getter/lib 克隆 base package。 +- 表中出现的字段替换父字段。 +- 简单、直观。 +- 不适合复杂函数覆写。 + +### 8.3 Function override + +适合复杂逻辑： + +```lua +local base = package_from("official", "android/org.fdroid.fdroid") + +return base:override(function(pkg) + pkg.name = "F-Droid Custom" + + local parent_select = pkg.select + function pkg:select(ctx, candidates, installed, user_state) + local selected = parent_select(self, ctx, candidates, installed, user_state) + selected.channel = "custom" + return selected + end +end) +``` + +语义： + +- getter/lib 克隆 base package。 +- 用户函数修改 clone。 +- 可以替换字段，也可以替换 hook。 +- 可以调用父函数。 + +### 8.4 推荐策略 + +建议同时支持 table override 和 function override。 + +文档推荐： + +- 简单 metadata 修改用 table override。 +- 非平凡修改用 function override。 + +注意：override helper 是 Lua lib/helper 问题，不是 Rust API 问题。Rust 只关心最终返回的 JSON-like package object 是否符合 schema。 + +--- + +## 9. Package lifecycle phases + +### 9.1 参考 emerge，但不照搬 + +Gentoo ebuild phase 包括： + +```text +pkg_pretend +pkg_setup +src_unpack +src_prepare +src_configure +src_compile +src_test +src_install +pkg_preinst +pkg_postinst +``` + +UpgradeAll 不是源码编译系统，因此不复制 `src_compile/src_install` 这些名字。 + +参考点是： + +- package 文件提供一组生命周期 hook。 +- 默认 hook 由 reusable module 提供。 +- package 可以 override hook。 +- getter 按固定顺序执行。 + +### 9.2 新 phase 名称 + +采用 app-centric 命名： + +```text +preflight +setup +match +discover +prepare +select + +post_update +``` + +`plan` 这个名字过于模糊，已拒绝。 + +推荐替代： + +- `resolve`：把 selected candidate 解析成可执行 actions。 +- `make_actions`：更直白，返回 action list。 + +目前建议：`resolve`。 + +### 9.3 Phase 语义 + +#### preflight(ctx) + +用途： + +- 预检查。 +- 检查平台是否支持。 +- 检查权限声明。 +- 检查 provider/backend 可用性。 +- 检查明显不兼容的 user state。 + +参考 Gentoo：`pkg_pretend`。 + +#### setup(ctx) + +用途： + +- 初始化 package evaluation。 +- 解析 provider config。 +- 检查 auth 是否存在。 +- 确定默认 source priority。 + +参考 Gentoo：`pkg_setup`。 + +#### match(ctx, installed_item) + +用途： + +- 判断一个 installed inventory item 是否匹配本 package。 +- 替代旧 `checkAppAvailable` 的一部分语义。 + +#### discover(ctx) + +用途： + +- 查询 provider/source。 +- 返回 release candidates。 + +替代旧： + +- `getAppReleaseList` +- `getAppUpdate` + +#### prepare(ctx, candidates) + +用途： + +- 将 provider-specific release 规范化为 canonical candidates。 +- 过滤 prerelease。 +- 过滤 arch/variant。 +- 提取/规范化 version。 +- 处理 changelog/asset metadata。 + +参考 Gentoo：`src_prepare`。 + +#### select(ctx, candidates, installed, user_state) + +用途： + +- 从 candidates 中选择应更新的版本和 artifact。 +- 应用 version compare。 +- 应用 ignore/pin/source priority。 + +#### resolve(ctx, selected) + +用途： + +- 将 selected candidate 转成可执行动作。 +- 返回 DownloadRequest / InstallAction / warnings。 + +示例输出： + +```lua +return { + actions = { + { + type = "download", + url = selected.artifact.url, + file_name = selected.artifact.name, + headers = {}, + }, + { + type = "install", + installer = "android_package", + file = selected.artifact.name, + }, + }, + warnings = {}, +} +``` + +#### post_update(ctx, result) + +用途： + +- 可选的更新后 message / metadata。 +- 应尽量少用。 +- 大部分状态变更应由 Rust core 处理。 + +--- + +## 10. Permissions / network model + +### 10.1 默认无直接网络 + +默认情况下，Lua package script 不获得直接网络 API。 + +它可以通过 getter 暴露的 provider/source API 间接获取 release 信息。 + +### 10.2 自由网络权限 + +如果 package 声明自由网络权限，getter 才向 Lua 环境暴露直接网络接口。 + +该权限用于类似 live/9999 包或特殊 upstream 逻辑。 + +UI 行为： + +- 在 App detail 的 source/version 层显示黄色 warning tag。 +- 该 tag 只提示，不阻止使用。 + +### 10.3 不做脚本超时 + +不对 Lua 脚本本身设置 runtime timeout/fuel limit。 + +理由： + +- 停机问题无法一般解决。 +- 脚本速度受本地机器、网络、provider 等影响。 +- 网络操作使用正常 network timeout。 + +### 10.4 v1 暂不强制校验 + +v1 暂不做 repo/script/artifact 强校验。 + +理由： + +- 先信任 Git 仓库。 +- 校验系统会显著增加复杂度。 +- 可以先保留 schema 字段，后续再 enforce。 + +--- + +## 11. Storage model + +### 11.1 Main SQLite DB + +主 DB 存储权威用户状态和 getter 状态。 + +建议内容： + +- repositories registry。 +- repo priority。 +- enabled apps/packages。 +- user source priority override。 +- ignore versions。 +- pins。 +- favorites/star。 +- migration records。 +- settings。 +- credentials references。 +- download task persistent state。 + +### 11.2 Cache DB + +缓存 DB 单独文件，不与主 DB 混用。 + +缓存内容： + +- evaluated package metadata。 +- version/release candidates。 +- selected latest version。 +- asset metadata。 +- provider response cache。 +- search index。 +- validation result。 + +Cache key 应包含： + +```text +repo id +repo revision/hash +package file hash +Lua API version +getter version or package API version +platform target +permissions/network mode +``` + +### 11.3 Repo files + +package Lua source files 存在本地文件夹中。 + +SQLite 只记录 repo registry/path/revision/priority 等元信息。 + +Android 上 repo sync 可以先采用 archive zip/tar 或 bundled repo snapshot，避免直接依赖完整 git CLI。 + +--- + +## 12. URL rewrite / bashrc-like hooks + +旧 `extra_hub` 的 URL replace 语义保留，但改为全局策略。 + +要求： + +- 是全局的，不散落到每个 source。 +- 可按 package/repository scope 区分。 +- 参考 emerge bashrc 的精神：全局 hook 根据上下文做调整。 + +建议文件： + +```text +config/hooks/download_rewrite.lua +``` + +示例： + +```lua +return function(ctx, req) + if ctx.repo_id == "official" and ctx.package_id == "android/com.foo" then + req.url = req.url:gsub("https://github.com/", "https://mirror.example/github/") + end + + return req +end +``` + +执行阶段： + +- `resolve` 生成 DownloadRequest 后。 +- downloader submit 前。 + +--- + +## 13. Legacy migration + +### 13.1 迁移原则 + +旧数据迁移必须无感自动完成。 + +但迁移是有限/简单迁移，不追求完整复刻旧语义。 + +可以丢弃： + +- API key。 +- auth token。 +- 复杂 Hub 配置。 +- 无法可靠映射的特殊规则。 + +必须保留： + +- saved apps 的基本 identity。 +- Android package / Magisk module installed id。 +- ignore version / mark version 能力，如果可映射。 +- user-visible tracked app 列表。 +- 常见 source/cloud config 能力，如果可内置转换。 + +### 13.2 迁移输入 + +旧 Room DB： + +- `app` +- `hub` +- `extra_app` +- `extra_hub` + +Room DB 信息： + +- name：`app_metadata_database.db` +- version：17 +- migrations：6->17 + +### 13.3 迁移输出 + +输出到： + +- getter main SQLite user state。 +- 必要时生成 `local` repo package Lua 文件。 + +迁移生成 `local` 是特殊情况，只做一次。 + +普通 installed autogen 不写 local，而写 `local_autogen`。 + +### 13.4 迁移匹配策略 + +建议流程： + +1. 使用 bundled official repo snapshot 做本地匹配，不依赖首启联网。 +2. 能匹配 official package 的旧 App：写入 user state，指向 official package。 +3. 不能匹配但常见类型可转换：生成 `local` package Lua。 +4. 稀有情况：迁移 installed id list，状态为 missing package，提示用户自己写或提交 issue。 +5. 迁移完成后记录 migration_runs。 + +### 13.5 迁移 UX + +- 普通用户无感进入新 App。 +- 迁移失败时进入 migration/recovery 页面。 +- 单个 package 无法匹配不应阻塞整个 App。 +- 该 package 显示 missing/needs package script 状态。 + +--- + +## 14. Installed autogen UX + +### 14.1 生成流程 + +用户点击“从已安装应用生成”： + +1. Android adapter 扫描 installed inventory。 +2. getter 找出可生成的候选列表。 +3. UI 展示列表。 +4. 用户 yes/no 确认。 +5. getter 写入 `local_autogen` repo。 +6. 生成后不会自动消失。 + +### 14.2 清理流程 + +用户点击“清除不存在的应用”： + +1. getter 计算将删除列表。 +2. UI 展示列表。 +3. 用户 yes/no 确认。 +4. getter 删除 `local_autogen` 中不再安装的记录/文件。 + +普通清理按钮只作用于 `local_autogen`，不删除 `local`。 + +--- + +## 15. Patch stack / user fork 模型 + +### 15.1 不设计复杂 runtime customization + +决策：用户二次开发采用 patch stack/source fork，不做复杂 runtime plugin/customization 框架。 + +原因： + +- 无法预测用户如何修改软件。 +- 为任意 customization 设计稳定 runtime API 会显著拖累兴趣项目维护。 +- Flutter 本身不是为了用户 runtime custom UI 设计的。 + +### 15.2 仍需降低 rebase 成本 + +参考 Linux kernel 的模块分离思想： + +- subsystem 目录清晰。 +- API 边界明确。 +- generated files 不手改。 +- 上游经常变的代码和用户常改代码尽量分离。 +- repository/package Lua 文件天然适合 patch stack。 + +### 15.3 稳定性承诺层级 + +建议承诺： + +- Rust internal API：不稳定。 +- Lua package boundary schema：相对稳定。 +- ResolvedPackage / UpdateCandidate / UpdateAction schema：稳定。 +- Platform RPC API：相对稳定。 +- CLI user-facing commands：稳定。 +- Individual package Lua scripts：可变。 + +--- + +## 16. Flutter APP 边界 + +Flutter APP 负责： + +- Home / App list / App detail / Settings / Log / Migration UI。 +- Android platform adapter。 +- 展示 getter 状态和事件。 +- 用户确认流程，如 autogen list yes/no、cleanup list yes/no。 +- 显示 free-network yellow tag。 + +Flutter APP 不负责： + +- provider/source logic。 +- package update selection。 +- version comparison。 +- storage migration。 +- download task state machine。 +- repository resolution。 +- Lua evaluation。 + +--- + +## 17. CLI 方向 + +getter CLI 应围绕 app/package，而不是 hub。 + +建议命令： + +```bash +getter app list +getter app show android/org.fdroid.fdroid +getter app check android/org.fdroid.fdroid +getter app update android/org.fdroid.fdroid +getter app sources android/org.fdroid.fdroid + +getter repo list +getter repo sync +getter repo eval official + +getter template list +getter template run android_installed_app --input ... + +getter storage validate +getter legacy migrate +``` + +CLI 是验证 getter core 独立性的关键： + +如果 CLI 无法完成核心更新流程，说明逻辑仍然泄漏在 Flutter/Android APP 里。 + +--- + +## 18. 非目标 + +v1 非目标： + +- 不做复杂 runtime UI customization framework。 +- 不做 Wasm plugin runtime。 +- 不做完整旧 auth/API key 迁移。 +- 不强制 repo/script/artifact 校验。 +- 不做 Lua script timeout/fuel limit。 +- 不保证任意用户 fork 不冲突。 +- 不继续维护旧 hub-app 逻辑模型。 + +--- + +## 19. Open questions + +仍需决策： + +1. `plan` 替代 phase 最终名字：`resolve` 还是 `make_actions`。 +2. template conflict policy：目标文件存在时 skip、overwrite、还是询问。 +4. repo priority 默认值精确设定。 +5. URL rewrite hook 的最终 Lua schema。 +6. Android repo sync v1 使用 bundled snapshot、zip/tar archive，还是 git/libgit2。 +7. main DB/cache DB 具体 schema。 +8. legacy migration 的字段级 mapping。 +9. Flutter UI route/page 具体信息架构。 +10. provider/source host API 细节。 + +--- + +## 20. Documentation policy + +从本文开始，UpgradeAll 重构文档采用以下规则： + +1. 每个重要架构决策写入 wiki 或 ADR。 +2. 每个新模块必须有 README 或 docs section，说明职责和非职责。 +3. 每个跨边界 API 必须有 schema 文档。 +4. 每个迁移步骤必须有 source/target mapping 文档。 +5. 每个 Lua host API 必须有示例。 +6. 每个用户可见破坏性行为必须有 UX 说明。 +7. 每次设计变更必须更新本文或后续 ADR。 + +推荐后续文档拆分： + +```text +docs/ + architecture/ + upgradeall-getter-rewrite-wiki.md + adr/ + 0001-app-centric-lua-package-repository-model.md + 0002-getter-flutter-platform-boundary.md + 0003-legacy-room-migration.md + 0004-sqlite-main-db-and-cache-db.md + 0005-lua-package-api.md + lua-api/ + package-lifecycle.md + repository-layout.md + templates.md + permissions.md + migration/ + legacy-room-mapping.md + app/ + flutter-ui-feature-parity.md +``` diff --git a/docs/implementation/coding-agent-handoff.md b/docs/implementation/coding-agent-handoff.md new file mode 100644 index 00000000..d0bb396e --- /dev/null +++ b/docs/implementation/coding-agent-handoff.md @@ -0,0 +1,124 @@ +# Coding Agent Handoff: UpgradeAll Rewrite + +> Status: Ready for coding-agent bootstrap +> Date: 2026-06-21 +> Target agent: pi agent / coding agents running in the UpgradeAll repository + +## Read first + +Before coding, read these files in order: + +1. `AGENTS.md` +2. `docs/README.md` +3. `docs/architecture/upgradeall-getter-rewrite-wiki.md` +4. `docs/architecture/adr/0001-app-centric-lua-package-repository-model.md` +5. `docs/architecture/adr/0002-getter-flutter-platform-boundary.md` +6. `docs/architecture/adr/0003-legacy-room-migration.md` +7. `docs/architecture/adr/0004-sqlite-main-db-and-cache-db.md` +8. `docs/architecture/adr/0005-lua-package-api.md` +9. `docs/architecture/adr/0006-package-centric-cli-command-contract.md` +10. `docs/app/flutter-ui-feature-parity-and-testing.md` + +## Mission + +Rewrite UpgradeAll from scratch around: + +```text +Flutter APP + Rust getter core + Lua package repositories +``` + +The old hub-app model must not be reintroduced. + +## Non-negotiable architecture rules + +- Rust getter owns all product/domain logic. +- Flutter owns UI and platform adapter only. +- getter lives in the reusable `core-getter/src/main/rust/getter` git submodule (`https://github.com/DUpdateSystem/getter`); make getter changes in that submodule and update the superproject gitlink. +- getter storage uses SQLite main DB plus separate cache DB. +- Package definitions are Lua files in repositories. +- Lua returns JSON-like tables across the Lua/Rust boundary; Rust validates typed structs. +- Package IDs are readable, e.g. `android/org.fdroid.fdroid`, not UUID primary identities. +- Legacy Room migration must be automatic for normal users, but it is intentionally limited/simple. +- Patch stack/source fork is the supported customization model; do not design a runtime UI customization framework. + +## First implementation tranche + +Do not start with Flutter screens. + +Recommended order: + +1. Create Rust getter workspace skeleton. +2. Define core Rust types: + - PackageId + - RepositoryId + - RepositoryPriority + - ResolvedPackage + - InstalledTarget + - UpdateCandidate + - SelectedUpdate + - UpdateAction +3. Implement repository layout loader: + - `repo.toml` + - `packages/` + - `lib/` + - `templates/` +4. Integrate `mlua` minimally: + - load a Lua package file; + - expose `require` search path for repo `lib/`; + - expose `package_from(repo, id)` later; + - return JSON-like Lua table; + - validate into Rust structs. +5. Implement repository priority resolution. +6. Implement main DB and cache DB skeleton. +7. Write migration mapping tests before writing migration implementation. +8. Only after getter CLI can evaluate/list packages should Flutter shell begin. + +## Testing strategy + +Use mixed TDD and BDD. + +### TDD + +Use TDD for function/domain behavior: + +- PackageId parsing/formatting. +- Repository priority resolution. +- Lua table -> Rust validation. +- lifecycle phase output validation. +- cache invalidation key calculation. +- legacy Room mapping functions. +- version comparison and update selection. + +### BDD + +Use BDD for UI and integration behavior: + +- Flutter app list and app detail flows. +- installed autogen preview and confirmation. +- cleanup preview and confirmation. +- yellow network warning tag display. +- legacy migration success/warning UX. +- update/download task flow. + +BDD scenarios should be self-explaining documentation tests. Do not over-test BDD. + +## Documentation update rule + +If coding changes a boundary, model, phase, migration rule, repository layout, or testing rule, update docs in the same patch. + +Prefer adding/updating ADRs for decisions rather than burying major changes in code comments. + +## Repository naming + +- `local` is the default highest-priority user-authored override repository. +- `local_autogen` is the generated fallback repository used by ordinary installed-app autogen. +- Legacy migration is special and may generate `local` package files once for compatibility. +- Cleanup of missing generated apps only touches `local_autogen`. + +## Open questions to resolve before implementation hardens + +- Final name for the `resolve`/`make_actions` lifecycle phase. +- Template conflict behavior when generated target already exists. +- Concrete main DB/cache DB schema. +- Android repo sync mechanism: bundled snapshot vs archive download vs git/libgit2. +- URL rewrite hook schema. diff --git a/docs/lua-api/package-lifecycle.md b/docs/lua-api/package-lifecycle.md new file mode 100644 index 00000000..9b5a6f46 --- /dev/null +++ b/docs/lua-api/package-lifecycle.md @@ -0,0 +1,66 @@ +# Lua Package Lifecycle + +> Status: Draft / living design record +> Date: 2026-06-21 +> Project: UpgradeAll rewrite — Flutter APP + Rust getter core + Lua package repository model + +UpgradeAll uses an app/update lifecycle inspired by Gentoo ebuild phases, but does not copy source-build phase names. + +## Phases + +```text +preflight(ctx) +setup(ctx) +match(ctx, installed_item) +discover(ctx) +prepare(ctx, candidates) +select(ctx, candidates, installed, user_state) +resolve(ctx, selected) +post_update(ctx, result) +``` + +`resolve` is the current recommended replacement for the rejected name `plan`. It means: convert selected candidate/artifact into executable update actions. + +## preflight + +Validate whether the package can be evaluated on this platform and with current permissions/settings. + +## setup + +Resolve package/provider setup such as default source priority, credential availability and provider config. + +## match + +Match installed inventory items to this package. + +## discover + +Query sources/providers and return release candidates. + +## prepare + +Normalize, filter and enrich release candidates. + +## select + +Choose the candidate/artifact to update to, using installed version and user state. + +The first getter-core selection helper uses deterministic tokenized version comparison: digit runs compare numerically, text suffixes compare case-insensitively, separators are ignored, and a prerelease-like text suffix (for example `beta`/`rc`) sorts before the final release with the same numeric prefix. The selector skips the user's ignored version and returns the highest candidate newer than the installed version. + +## resolve + +Return executable update actions: + +```lua +return { + actions = { + { type = "download", url = "https://...", file_name = "app.apk" }, + { type = "install", installer = "android_package", file = "app.apk" }, + }, + warnings = {}, +} +``` + +## post_update + +Optional post-update hook. Most persistent state changes should remain in Rust core, not Lua. diff --git a/docs/lua-api/permissions.md b/docs/lua-api/permissions.md new file mode 100644 index 00000000..e30b2e78 --- /dev/null +++ b/docs/lua-api/permissions.md @@ -0,0 +1,33 @@ +# Lua Permissions + +> Status: Draft / living design record +> Date: 2026-06-21 +> Project: UpgradeAll rewrite — Flutter APP + Rust getter core + Lua package repository model + +## Default + +Lua package scripts do not receive direct network access by default. + +They can use getter-provided provider/source APIs. + +## Free network permission + +A package may declare free network access for live/9999-like logic or unusual upstreams. + +When declared: + +- getter exposes a direct Lua network host API; +- Flutter displays a yellow warning tag at App detail source/version level; +- use is not blocked. + +## Timeouts + +Network operations use normal network timeouts. + +Lua script runtime itself does not use a timeout/fuel limit. + +## v1 verification policy + +v1 does not enforce repo/script/artifact verification. + +Schema fields may exist for future verification, but enforcement is not a v1 requirement. diff --git a/docs/lua-api/repository-layout.md b/docs/lua-api/repository-layout.md new file mode 100644 index 00000000..15b8d1e6 --- /dev/null +++ b/docs/lua-api/repository-layout.md @@ -0,0 +1,56 @@ +# Lua Repository Layout + +> Status: Draft / living design record +> Date: 2026-06-21 +> Project: UpgradeAll rewrite — Flutter APP + Rust getter core + Lua package repository model + +Recommended layout: + +```text +repo/ + repo.toml + packages/ + android/ + org.fdroid.fdroid.lua + magisk/ + zygisk-next.lua + lib/ + github.lua + android.lua + github_android_apk.lua + templates/ + android_installed_app.lua +``` + +## repo.toml + +```toml +id = "official" +name = "UpgradeAll Official" +priority = 0 +api_version = "getter.repo.v1" +``` + +## packages/ + +Package files are final package definitions consumed by getter. + +Path-derived package id: + +```text +packages/android/org.fdroid.fdroid.lua -> android/org.fdroid.fdroid +``` + +The file should declare the same id. getter validates consistency. + +## lib/ + +Reusable Lua modules. These are conceptually similar to eclasses but are plain Lua modules. + +```lua +local github_android = require("lib.github_android_apk") +``` + +## templates/ + +Lua generators that output package Lua file content. diff --git a/docs/lua-api/templates.md b/docs/lua-api/templates.md new file mode 100644 index 00000000..3ae87b42 --- /dev/null +++ b/docs/lua-api/templates.md @@ -0,0 +1,67 @@ +# Lua Templates / Autogen + +> Status: Draft / living design record +> Date: 2026-06-21 +> Project: UpgradeAll rewrite — Flutter APP + Rust getter core + Lua package repository model + +Templates are Lua generators that output package Lua file content. + +They are inspired by Funtoo Metatools/autogen, where autogen code produces ebuilds from upstream or structured inputs. + +## Template role + +Templates are used for: + +- generating package files from installed Android apps; +- generating package files from Magisk modules; +- repository maintainer batch generation; +- assisted package creation from GitHub/F-Droid metadata. + +Templates are not runtime package definitions. + +## Example + +```lua +return template { + id = "android_installed_app", + + generate = function(ctx, input) + return { + path = "packages/android/" .. input.package_name .. ".lua", + content = [[ +local android = require("lib.android") + +return android.local_app { + id = "android/]] .. input.package_name .. [[", + name = "]] .. input.label .. [[", + package_name = "]] .. input.package_name .. [[", +} +]] + } + end +} +``` + +## UX contract + +Generation flow: + +1. User clicks generate. +2. getter computes candidate list. +3. Flutter shows preview list. +4. User confirms yes/no. +5. getter writes files. + +Cleanup flow: + +1. User clicks clear missing generated apps. +2. getter computes deletion list. +3. Flutter shows preview list. +4. User confirms yes/no. +5. getter deletes only autogen-managed files/state. + +## Repositories + +Ordinary installed-app autogen writes to `local_autogen`. + +Legacy migration may generate `local` files once as a special compatibility path. diff --git a/docs/migration/legacy-room-mapping.md b/docs/migration/legacy-room-mapping.md new file mode 100644 index 00000000..db6aa42a --- /dev/null +++ b/docs/migration/legacy-room-mapping.md @@ -0,0 +1,98 @@ +# Legacy Room Migration Mapping + +> Status: Draft / living design record +> Date: 2026-06-21 +> Project: UpgradeAll rewrite — Flutter APP + Rust getter core + Lua package repository model + +## Source + +Legacy Room DB: + +```text +app_metadata_database.db +version 17 +``` + +Tables: + +- `app` +- `hub` +- `extra_app` +- `extra_hub` + +## Target + +- getter main SQLite DB user state. +- `local` repository package Lua files when migration needs compatibility stubs. +- migration record table. + +## Principles + +- Migration must be automatic for normal users. +- Migration is limited and simple. +- Complex API keys/auth may be discarded. +- Per-app mapping failures should not block the entire app. + +## App mapping + +Legacy app -> new package id: + +```text +Android package -> android/ +Magisk module -> magisk/ +``` + +If bundled official repo contains a matching package, link user state to it. + +If no official match but common conversion is possible, generate local package Lua. + +If no conversion is possible, preserve installed/tracked id and mark missing package definition. + +## Hub mapping + +Legacy Hub does not map to a top-level new object. + +Its semantics are split into: + +- provider/source config; +- package source priority; +- credentials/auth settings; +- URL rewrite policy; +- migration diagnostics. + +Complex auth may be dropped. + +## ExtraApp mapping + +Map mark/ignore version state when possible. + +## ExtraHub mapping + +Map URL replace semantics into global download rewrite policy if safe. Otherwise drop and record warning. + +## Current CLI bridge bundle + +The host-side CLI implementation currently accepts a deterministic JSON bridge bundle instead of reading Room directly: + +```json +{ + "format": "upgradeall-legacy-room-bundle", + "version": 17, + "apps": [ + { + "kind": "android", + "installed_id": "org.fdroid.fdroid", + "official_package_available": true, + "common_conversion_available": false, + "ignored_version": "1.20.0", + "favorite": true + } + ] +} +``` + +Each app maps to `tracked_packages` in getter main DB. Success reports are sanitized and include counts only; raw bundles are not copied into reports. + +## Completion + +After successful migration, write a migration record so the same migration does not rerun. diff --git a/docs/refactor/2026-06-20-refactor-plan.md b/docs/refactor/2026-06-20-refactor-plan.md new file mode 100644 index 00000000..c686678f --- /dev/null +++ b/docs/refactor/2026-06-20-refactor-plan.md @@ -0,0 +1,104 @@ +# 2026-06-20 Refactor Plan + +## Objective + +Prepare the UpgradeAll Flutter + getter rewrite from a clean, synced master while preserving all temporary work in stashes/backup branches. + +## Canonical source plan + +The detailed 06-20 plan has been copied into this repository at: + +- `docs/refactor/2026-06-20-upgradeall-flutter-getter-rewrite-complete-plan.md` + +Provenance: + +- Remote source: `xz@100.65.231.22:/home/xz/.hermes/plans/2026-06-20_181038-upgradeall-flutter-getter-rewrite-complete-plan.md` +- SHA-256: `a9d02ce7fb88112506580a6e5e723494016ff75cc950083f66ab93701bbc3a0a` +- The hash matches the plan that was preserved inside the pre-sync stash's untracked parent. + +## Completed preparation + +- Superproject WIP was stashed before sync. +- Getter submodule WIP was stashed before sync. +- Local pre-sync commits were preserved on backup branches. +- `master` was synced to upstream `origin/master` commit `4a1aae1d44a418989b0d3d28528cacff0cc066c0`. +- Getter submodule was synced to recorded commit `f011d9b4b9a15f83cd39c86e781ad8830a8ecae6`. +- Planning branch created: `refactor/phase0-planning-20260620`. + +## User clarification captured + +BDD Cucumber coverage is required for all user-facing functions/interfaces. The complete BDD coverage targets are the UpgradeAll App and Getter CLI. Internal interfaces should use unit tests, integration tests, and other traditional test frameworks because BDD fits integration/acceptance behavior better than algorithm-level tests. + +## Phase 0 deliverables + +- Glossary: `CONTEXT.md`. +- ADRs: `docs/adr/0001` through `0006`. +- Target architecture: `docs/architecture/target-architecture.md`. +- BDD/TDD plan: `docs/testing/bdd-plan.md`. +- Agent workflow: `docs/ai-development.md` and root `AGENTS.md`. +- Verification skeleton: `justfile`. + +## Phase 1 recommendation + +Detailed Phase 1a plan: [`phase-1-getter-cli-bdd-plan.md`](phase-1-getter-cli-bdd-plan.md). Phase 1a is the Getter CLI BDD spine inside the broader canonical Phase 1 getter workspace refactor. + +Detailed Phase 1b plan: [`phase-1b-getter-workspace-skeleton-plan.md`](phase-1b-getter-workspace-skeleton-plan.md). Phase 1b is the transitional workspace skeleton that keeps behavior in the root getter package while introducing the split-crate scaffold. The single current verification entrypoint is `just verify`, which includes Phase 1a focused behavior tests plus Phase 1b structural workspace checks. + +Do not start by implementing Flutter screens. + +Start with a testable headless slice: + +1. ADR 0007 is accepted for the Phase 1a Getter CLI command contract; future CLI changes must explicitly extend or revise that ADR. +2. Define the first Getter CLI Gherkin scenarios for initialization, app listing, hub listing, and malformed legacy bundle failure reporting. +3. Wire a minimal Cucumber runner for Getter CLI. +4. Implement the smallest CLI contract needed to make the first scenario pass. +5. Add internal Rust tests for the core behavior behind that CLI scenario. +6. Only then expose the same behavior through the app shell. + +## Decision gates before implementation + +- Choose the concrete Cucumber runner strategy for Flutter App scenarios. +- Choose the concrete command/output/error contract for the first Getter CLI slice. +- Decide whether to mine, split, or discard each part of the stashed direct-JNI/RPC rewrite. +- Confirm the first supported legacy DB schema range for migration fixtures. + +## First proposed BDD scenarios + +### Getter CLI smoke + +```gherkin +@getter-cli @smoke +Feature: Getter CLI initialization + Scenario: User initializes a new getter data directory + Given an empty getter data directory + When I run getter init for that directory + Then the command succeeds + And the getter data directory is usable +``` + +### Getter CLI migration recovery + +```gherkin +@getter-cli @migration +Feature: Legacy import failure recovery + Scenario: User receives a non-destructive report when legacy import fails + Given a corrupted legacy export bundle + When I run getter legacy import for that bundle + Then the command fails with a documented migration error + And no partially usable getter state is created + And a sanitized migration report is available +``` + +### UpgradeAll App migration recovery + +```gherkin +@app @migration +Feature: App migration recovery + Scenario: User can retry or report a failed migration + Given the app starts with a legacy database that cannot be imported + When migration fails + Then the app shows the migration recovery screen + And the user can retry migration + And the user can export a sanitized report + And starting fresh requires explicit confirmation +``` diff --git a/docs/refactor/2026-06-20-upgradeall-flutter-getter-rewrite-complete-plan.md b/docs/refactor/2026-06-20-upgradeall-flutter-getter-rewrite-complete-plan.md new file mode 100644 index 00000000..fb65d99e --- /dev/null +++ b/docs/refactor/2026-06-20-upgradeall-flutter-getter-rewrite-complete-plan.md @@ -0,0 +1,1468 @@ +# UpgradeAll Flutter + getter Rust Core Rewrite Implementation Plan + +> **For Hermes:** Use `subagent-driven-development` skill to implement this plan task-by-task after the user explicitly asks for execution. + +**Goal:** Rewrite `DUpdateSystem/UpgradeAll` as a Flutter app shell whose durable logic lives in `DUpdateSystem/getter` as a Rust-first core, while preserving existing Android users' Room database data through a tested upgrade path. + +**Architecture:** `getter` becomes the headless product engine: storage, migrations, providers, downloads, version comparison, update orchestration, plugin registry, event streams, CLI/TUI API. `UpgradeAll` becomes a Flutter UI/platform shell with source-level customizable page modules, typed generated contracts, stable test IDs, and Android platform adapters. Android legacy migration is treated as a first-class compatibility subsystem, not a best-effort startup hack. + +**Tech Stack:** Rust workspace (`getter-core`, `getter-storage`, `getter-provider`, `getter-downloader`, `getter-plugin-api`, `getter-ffi`, `getter-rpc`, `getter-cli`), Rust-managed SQLite, Flutter/Dart, `flutter_rust_bridge` v2 or equivalent Dart FFI generator, Flutter `integration_test`, Maestro for black-box semantic UI flows, Patrol only for native OS automation, Android legacy Room migrator module for old installed users. + +--- + +## 0. Source and docs basis + +User-selected decision: + +- UI framework: **Flutter**. +- Distribution philosophy: source-level downstream customization. Users can fork, ask AI to modify pages, merge upstream, compile their own build, and rely on strong module boundaries, type checks, tests, and compile-time failures. +- Development posture: CLI/opencode/Emacs first; do not assume Android Studio. + +Read-only source inspection used: + +- `DUpdateSystem/UpgradeAll` +- `DUpdateSystem/getter` + +Relevant current-code facts: + +- `UpgradeAll/settings.gradle:13-25` defines modules: `:app`, `:core`, `:core-websdk`, `:core-utils`, `:core-shell`, `:core-downloader`, `:core-installer`, `:core-android-utils`, `:app-backup`, `:core-getter`, `:core-websdk:data`, `:core-getter:provider`, `:core-getter:rpc`. +- `UpgradeAll/app/build.gradle:71-74` still enables `dataBinding` and `viewBinding`; `app/build.gradle:131-143` already has Compose deps, but we are now choosing Flutter for the rewrite. +- `UpgradeAll/core-getter/build.gradle:37-52` already builds a Rust `api_proxy` for Android ABIs via an Android Rust Gradle plugin. +- `UpgradeAll/core-getter/src/main/java/net/xzos/upgradeall/getter/NativeLib.kt:17-26` loads `api_proxy` via `System.loadLibrary("api_proxy")` and exposes JNI `runServer`. +- `UpgradeAll/core-getter/src/main/java/net/xzos/upgradeall/getter/GetterPort.kt:25-35` starts the Rust service and creates a Kotlin `GetterService` client. +- `GetterPort.kt:147-168` already exposes `registerProvider` and `registerDownloader`. +- `UpgradeAll/core-getter/rpc/.../GetterService.kt:13-187` already defines a broad async service API for init, release lookup, cloud config, provider/downloader registration, download tasks, app manager, hub manager, extra records, Android API, notification, and cloud config manager. +- `getter/src/websdk/repo/provider.rs:22-40` has built-in Rust provider registry for GitHub, F-Droid, GitLab, and LSPosed. +- `getter/src/websdk/repo/provider.rs:48-55` supports dynamic `add_provider`. +- `getter/src/rpc/server.rs:71-85` starts JSON-RPC server; `server.rs:173-180` registers an external provider; `server.rs:187-220` handles download info and URL replacement. +- `UpgradeAll/core/src/main/java/net/xzos/upgradeall/core/database/MetaDatabase.kt:21-24` declares Room `MetaDatabase` with entities `AppEntity`, `HubEntity`, `ExtraAppEntity`, `ExtraHubEntity`, version `17`. +- `MetaDatabase.kt:55-77` registers migrations `6->7`, `7->8`, `8->9`, `9->10`, `8->10`, `10->11`, ..., `16->17`, and uses database name `app_metadata_database.db`. +- Current legacy Room v17 tables contain: + - `app`: `name`, `app_id`, `invalid_version_number_field_regex`, `include_version_number_field_regex`, `ignore_version_number`, `cloud_config`, `enable_hub_list`, `star`, `id`. + - `hub`: `uuid`, `hub_config`, `auth`, `ignore_app_id_list`, `applications_mode`, `user_ignore_app_id_list`, `sort_point`. + - `extra_app`: `id`, `app_id`, `mark_version_number`. + - `extra_hub`: `id`, `enable_global`, `url_replace_search`, `url_replace_string`. +- `UpgradeAll/core/src/main/java/net/xzos/upgradeall/core/database/migration/RustMigration.kt` already attempts Room -> Rust JSONL migration, but it currently migrates apps, hubs, and extra hubs only; it does not migrate `extra_app`, skips if `apps.jsonl` exists, and lets Rust assign random new app UUIDs. This is not enough for a safe official Flutter rewrite migration. +- `getter/src/database/mod.rs` currently uses JSONL stores: `apps.jsonl`, `hubs.jsonl`, `extra_apps.jsonl`, `extra_hubs.jsonl`. +- `getter/src/database/store.rs` rewrites whole JSONL files under file locks. This is simple, but it lacks a formal schema migration system and is not ideal as the long-term compatibility storage for old installed Android users. + +Docs checked / used as design constraints: + +- Flutter official integration testing docs: Flutter supports unit/widget/integration tests; integration tests can be run with `flutter test integration_test` on supported targets. +- Flutter official native-code binding docs: Flutter can bind to native code through Dart FFI; for Rust, a binding generator such as `flutter_rust_bridge` is the practical high-level path. +- Android Room migration docs: Room migration errors can crash users; migrations should preserve user data, rely on exported schemas, and be tested. Manual migrations are needed for complex schema changes. Exported schema JSON files should be version-controlled and used in migration tests. +- Maestro Flutter docs/search result: Maestro tests Flutter apps through the Flutter Semantics Tree; use semantic labels / `Semantics` / semantic identifiers instead of brittle localized text. +- Patrol docs: Flutter `integration_test` cannot interact with the OS itself; Patrol native automation is useful for permissions, notifications, and other native OS interactions. + +Note: the requested `grab-docs` skill is not installed in this Hermes profile. I used the closest available workflow: source-code audit + official documentation lookup. + +--- + +## 1. Non-negotiable architecture decisions + +### Decision 1: `getter` owns product logic + +`getter` owns: + +- providers and provider registry; +- download-info extraction; +- downloader task management; +- version comparison and filtering; +- update status calculation; +- app/hub/extra record storage; +- cloud config parsing/application; +- plugin manifests and plugin runtime; +- event stream; +- legacy import and new storage migrations; +- CLI/TUI command API. + +Flutter owns: + +- navigation; +- page rendering; +- platform widgets; +- source-level customizable page modules; +- Android/iOS/desktop platform adapters; +- user interaction and accessibility/semantics identifiers. + +Flutter must not own provider logic, downloader logic, version comparison, URL replacement, durable update state, or DB migration semantics. + +### Decision 2: official Android upgrade keeps package identity + +For users updating from old UpgradeAll to the Flutter rewrite: + +- Keep Android `applicationId = "net.xzos.upgradeall"` for official releases. +- Use the same signing key lineage for official upgrade builds. +- If application ID or signing key changes, the new app cannot access the old app-private Room DB path. In that case, a separate migration bridge/export release is required. + +### Decision 3: Rust storage should move from ad-hoc JSONL to Rust-managed SQLite + +Current `getter` JSONL storage is useful for early extraction but is not ideal for long-lived mobile app compatibility. + +Recommended v1 storage for Flutter rewrite: + +- Rust-managed SQLite database, e.g. `getter.db`. +- Embedded Rust migrations, versioned by `PRAGMA user_version` plus a `schema_migrations` / `migration_runs` table. +- Access through `getter-storage`, not through Dart Drift/sqflite. +- Android legacy Room DB is imported into `getter.db` exactly once. +- Existing `apps.jsonl` / `hubs.jsonl` / `extra_*.jsonl` alpha data gets its own importer. + +Rationale: + +- Old app data is already SQLite. +- SQLite has transactionality and schema migration semantics. +- Flutter/Dart storage would split ownership away from Rust core. +- JSONL whole-file rewrite becomes fragile as the data model grows. + +### Decision 4: source-level page customization, not runtime UI plugins + +The user-customization model is: + +```text +upstream source release + -> downstream user fork + -> AI modifies page modules + -> user merges upstream later + -> compiler/tests reveal breakages + -> user builds their own APK/desktop app +``` + +So the app must provide: + +- stable typed `ui_contract`; +- stable `ui_kit` components; +- upstream-owned default pages; +- downstream-owned custom page package/registry; +- strict analyzer settings; +- generated API bindings that users do not edit; +- one-command verification. + +### Decision 5: UI testability is a product requirement + +Every public page/action must have: + +- stable route ID; +- stable semantic identifier/test ID; +- loading/empty/error/content state IDs; +- widget tests where possible; +- integration tests for primary flows; +- Maestro flows for black-box AI/manual clicking; +- Patrol only where native OS automation is needed. + +--- + +## 2. Target repository layout + +Keep the two public repos conceptually separate, but make local development easy. + +### `DUpdateSystem/getter` + +```text +getter/ + Cargo.toml # workspace + crates/ + getter-core/ # pure domain: apps/hubs/releases/status/version logic + getter-storage/ # Rust SQLite, migrations, legacy imports + getter-providers/ # built-in providers + provider traits + getter-downloader/ # downloader tasks and backend routing + getter-plugin-api/ # plugin manifest, permissions, schema, ABI + getter-rpc/ # JSON-RPC/WebSocket for external plugins/automation + getter-ffi/ # Flutter-facing facade for flutter_rust_bridge + getter-cli/ # headless CLI; proves core is UI-independent + getter-tui/ # optional later; ratatui/crossterm + migrations/ + getter/ # new Rust SQLite schema migrations + legacy-room/ # docs/schema snapshots for import reference + fixtures/ + legacy-room/ # old DB fixtures for v6-v17 migration tests + providers/ # GitHub/GitLab/F-Droid/LSPosed fixtures + docs/ + adr/ + api/ + migration/ +``` + +### `DUpdateSystem/UpgradeAll` + +```text +UpgradeAll/ + AGENTS.md + justfile + pubspec.yaml # Flutter app workspace root if desired + native/ + getter/ # git submodule or pinned workspace checkout of DUpdateSystem/getter + apps/ + upgradeall_flutter/ + pubspec.yaml + lib/ + main.dart + app_shell.dart + bootstrap.dart + platform/ + routing/ + android/ # same applicationId for official upgrade + ios/ + linux/ + windows/ + macos/ + integration_test/ + test/ + packages/ + upgradeall_contract/ # generated typed Dart DTO/client facade; do not edit manually + upgradeall_ui_contract/ # PageContext, RouteSpec, UiId, PageDescriptor + upgradeall_ui_kit/ # reusable widgets/components + upgradeall_pages_default/ # upstream maintained default pages + upgradeall_pages_custom/ # downstream/user maintained page overlay; upstream touches minimally + upgradeall_pages_examples/ # examples/templates; safe for upstream edits + tools/ + gen_contract/ + verify_custom_pages/ + migrate_contract/ + ai_review/ + docs/ + adr/ + architecture/ + migration/ + ai-development.md + custom-pages.md + testing.md +``` + +Important downstream merge rule: + +- Upstream should avoid editing `packages/upgradeall_pages_custom/` after initial skeleton creation. +- Upstream examples/templates go under `packages/upgradeall_pages_examples/`. +- Users should modify `pages_custom`, not `app_shell`, not `getter`, not generated bindings. + +--- + +## 3. Flutter app architecture + +### 3.1 Runtime layers + +```text +Flutter main() + -> bootstrap platform paths + -> Android legacy migration check/import if needed + -> getter_ffi.init(data_dir, cache_dir, platform_capabilities) + -> AppShell + -> PageRegistry(default pages + custom pages) + -> PageContext(getter client, event stream, navigation, theme, platform services) +``` + +### 3.2 Dart package responsibilities + +`upgradeall_contract`: + +- Generated from `getter-ffi` / Rust DTO declarations. +- Contains `GetterClient`, DTOs, event models, error models. +- Do not manually edit. + +`upgradeall_ui_contract`: + +- Source-stable API for custom pages. +- Contains: + +```dart +abstract interface class UpgradeAllPage { + RouteSpec get route; + UiText get title; + Widget build(PageContext ctx); +} + +final class PageContext { + final GetterClient getter; + final AppNavigator nav; + final Stream events; + final PlatformServices platform; + final UpgradeAllTheme theme; +} + +final class UiId { + final String value; + const UiId(this.value); +} +``` + +`upgradeall_ui_kit`: + +- App list widget. +- Release list widget. +- Hub selector widget. +- Plugin config schema renderer. +- Download task card. +- Error panel. +- Loading/empty state components. +- Test ID / semantics helpers. + +`upgradeall_pages_default`: + +- Home page. +- App list page. +- App detail page. +- Release/download page. +- Hub manager page. +- Discover/cloud config page. +- Download task manager page. +- Settings page. +- Logs/diagnostics page. +- Migration status page. + +`upgradeall_pages_custom`: + +- User-owned replacement/additional pages. +- Custom page registry. +- Optional theme overrides. +- Must depend only on `upgradeall_ui_contract`, `upgradeall_ui_kit`, and `upgradeall_contract`. + +### 3.3 State management + +Keep state management simple and AI-readable. + +Recommended v1: + +- Use plain typed service classes + `ValueNotifier`/`StreamBuilder` where sufficient. +- If app complexity requires provider injection, use `flutter_riverpod` without codegen initially. +- Do not add heavy code generation in UI packages except generated Rust bindings. + +Rule: + +- Domain state comes from `getter` snapshots/events. +- Flutter state is view state only: selected tab, visible filter, form draft, scroll state, local animation state. + +--- + +## 4. Rust API and FFI plan + +### 4.1 Use a narrow Flutter-facing facade + +Do not expose internal Rust modules directly to Dart. + +Create `getter-ffi` facade: + +```rust +pub struct GetterHandle { /* opaque */ } + +pub async fn init(config: InitConfig) -> Result; +pub async fn list_apps(handle: &GetterHandle, query: AppQuery) -> Result; +pub async fn get_app_detail(handle: &GetterHandle, app_id: AppRecordId) -> Result; +pub async fn renew_all(handle: &GetterHandle) -> Result; +pub async fn renew_app(handle: &GetterHandle, app_id: AppRecordId) -> Result; +pub async fn list_hubs(handle: &GetterHandle) -> Result>; +pub async fn save_hub(handle: &GetterHandle, draft: HubDraft) -> Result; +pub async fn submit_download(handle: &GetterHandle, req: DownloadRequest) -> Result; +pub fn event_stream(handle: &GetterHandle) -> impl Stream; +``` + +Expose only DTOs that are stable and serializable. + +### 4.2 Keep JSON-RPC for external extensibility + +`getter-rpc` remains useful for: + +- external provider plugins; +- external downloader plugins; +- CLI/debug automation; +- eventual local daemon mode; +- integration tests independent of Flutter. + +But Flutter should normally use direct FFI bindings, not local WebSocket JSON-RPC for every UI operation. + +### 4.3 Error model + +Define typed errors in Rust and generated Dart: + +```rust +pub enum GetterError { + Storage(StorageError), + Network(NetworkError), + Provider(ProviderError), + Migration(MigrationError), + Platform(PlatformError), + Permission(PermissionError), + InvalidInput(ValidationError), +} +``` + +Each error must include: + +- stable code; +- human-readable message; +- optional recoverability flag; +- optional diagnostic ID; +- optional source record ID. + +Do not pass raw panics/strings across FFI. + +--- + +## 5. Storage design + +### 5.1 New Rust SQLite schema v1 + +Recommended core tables: + +```text +meta( + key TEXT PRIMARY KEY, + value TEXT NOT NULL +) + +schema_migrations( + version INTEGER PRIMARY KEY, + name TEXT NOT NULL, + applied_at INTEGER NOT NULL, + checksum TEXT NOT NULL +) + +migration_runs( + id TEXT PRIMARY KEY, + source_kind TEXT NOT NULL, -- legacy_room, legacy_jsonl, fresh + source_version TEXT, + source_hash TEXT, + status TEXT NOT NULL, -- started, completed, failed + started_at INTEGER NOT NULL, + completed_at INTEGER, + report_json TEXT +) + +apps( + id TEXT PRIMARY KEY, + name TEXT NOT NULL, + app_id_json TEXT NOT NULL, + app_id_hash TEXT NOT NULL, + invalid_version_number_field_regex TEXT, + include_version_number_field_regex TEXT, + ignore_version_number TEXT, + cloud_config_json TEXT, + enable_hub_list_json TEXT, + star INTEGER, + legacy_room_id INTEGER, + created_at INTEGER NOT NULL, + updated_at INTEGER NOT NULL +) + +hubs( + uuid TEXT PRIMARY KEY, + hub_config_json TEXT NOT NULL, + auth_json TEXT NOT NULL, + ignore_app_id_list_json TEXT NOT NULL, + applications_mode INTEGER NOT NULL, + user_ignore_app_id_list_json TEXT NOT NULL, + sort_point INTEGER NOT NULL, + created_at INTEGER NOT NULL, + updated_at INTEGER NOT NULL +) + +extra_apps( + id TEXT PRIMARY KEY, + app_id_json TEXT NOT NULL, + app_id_hash TEXT NOT NULL, + mark_version_number TEXT, + legacy_room_id INTEGER, + created_at INTEGER NOT NULL, + updated_at INTEGER NOT NULL +) + +extra_hubs( + id TEXT PRIMARY KEY, + enable_global INTEGER NOT NULL, + url_replace_search TEXT, + url_replace_string TEXT, + created_at INTEGER NOT NULL, + updated_at INTEGER NOT NULL +) + +download_tasks(...) +provider_plugins(...) +downloader_plugins(...) +event_log(...) -- optional, bounded/rotated +``` + +For v1, JSON columns are acceptable for compatibility with current UpgradeAll model. Normalize only when there is a real query/index need. + +### 5.2 Deterministic IDs for migrated records + +Do not assign random app IDs during legacy migration. + +Use deterministic IDs: + +```text +new_app_id = UUIDv5(UPGRADEALL_LEGACY_NAMESPACE, "room-app:{legacy_room_id}:{canonical_app_id_json}") +new_extra_app_id = UUIDv5(UPGRADEALL_LEGACY_NAMESPACE, "room-extra-app:{legacy_room_id}:{canonical_app_id_json}") +``` + +Rationale: + +- migration is repeatable; +- tests are deterministic; +- logs and support reports are stable; +- migration can be retried safely. + +For hubs, preserve existing `uuid`. +For extra hubs, preserve existing text `id` (`GLOBAL` or hub UUID). + +### 5.3 Canonical JSON + +All maps/lists used as identity must be canonicalized before hashing: + +- sort object keys; +- preserve null vs missing where semantically meaningful; +- remove blank values only if legacy behavior did so; +- no whitespace; +- UTF-8. + +Write tests for canonicalization. + +--- + +## 6. Legacy Android migration strategy + +### 6.1 Supported source states + +Support these startup cases: + +1. Fresh Flutter install: no old Room DB, no getter DB. +2. Old Android UpgradeAll install with Room DB schema v6-v17. +3. Old Android UpgradeAll install with Room DB plus WAL/SHM files. +4. Intermediate alpha install with current getter JSONL store. +5. Partially completed previous migration attempt. +6. Failed migration with preserved backup. + +### 6.2 Official Android upgrade invariant + +Official upgrade can only read app-private old DB if: + +- package name/applicationId remains `net.xzos.upgradeall`; +- signing key lineage permits app update; +- Android system treats it as the same app data directory. + +If either changes, the plan must include a migration bridge release before the Flutter rewrite: + +```text +old Kotlin UpgradeAll bridge release + -> exports encrypted/signed migration bundle through SAF or app-private backup + -> Flutter rewrite imports bundle on first launch +``` + +### 6.3 Use an Android-only legacy migrator module + +Create a tiny Android library in the Flutter app, not a product logic dependency: + +```text +apps/upgradeall_flutter/android/legacy_migrator/ + src/main/kotlin/net/xzos/upgradeall/legacy_migration/ + LegacyMetaDatabase.kt + LegacyEntities.kt + LegacyConverters.kt + LegacyMigrations.kt + LegacyExportBundle.kt + LegacyMigrationRunner.kt +``` + +This module exists only to: + +- open/copy old Room DB; +- apply existing Room migrations to v17; +- export a typed migration bundle; +- never serve runtime product logic. + +Why not direct Rust import from every old schema only? + +- The existing Room migration chain already encodes legacy quirks from v6-v17. +- Room exported schema docs and `room-testing` make migration verification possible. +- Implementing every old schema conversion directly in Rust would be more error-prone. + +Long-term: after several major releases, this module can be removed only if the project formally drops direct migration from old Kotlin releases. + +### 6.4 Migration flow + +First Flutter Android launch: + +```text +1. Flutter bootstrap calls Android LegacyMigrationRunner.checkNeeded(). +2. If getter.db exists and migration_runs has completed legacy_room import, skip. +3. If old Room DB does not exist, create fresh getter.db. +4. If old Room DB exists: + a. create migration session ID; + b. copy app_metadata_database.db, -wal, -shm into private backup directory; + c. copy same files into a working DB name, e.g. legacy_migration_work.db; + d. open working DB with LegacyMetaDatabase + migrations 6->17; + e. force checkpoint on working DB; + f. export LegacyExportBundle v1; + g. close Room DB; + h. pass bundle path/hash to Rust getter-storage; + i. Rust imports bundle into getter.db inside a transaction; + j. Rust validates counts, canonical hashes, required fields; + k. mark migration_runs completed; + l. keep backup for at least N releases or until user explicitly deletes it. +``` + +Never delete old DB during the first successful migration. It can be ignored after success, but keep it for recovery. + +### 6.5 Legacy export bundle + +Use JSON for auditability initially. If size becomes an issue, add CBOR later. + +```json +{ + "format": "upgradeall.legacy.room.export.v1", + "source": { + "database_name": "app_metadata_database.db", + "room_schema_version": 17, + "identity_hash": "...", + "source_sha256": "...", + "exported_at": 1234567890, + "app_version_name": "...", + "app_version_code": 105 + }, + "apps": [ ... ], + "hubs": [ ... ], + "extra_apps": [ ... ], + "extra_hubs": [ ... ], + "warnings": [ ... ] +} +``` + +Include all four legacy tables. Current `RustMigration.kt` omits `extra_app`; the new migration must not repeat that omission. + +### 6.6 Mapping rules + +Legacy `app` -> Rust `apps`: + +- `name` -> `name` +- `app_id` JSON string -> canonical map -> `app_id_json`, `app_id_hash` +- `invalid_version_number_field_regex` -> same +- `include_version_number_field_regex` -> same +- `ignore_version_number` -> same +- `cloud_config` -> same JSON, validated against AppConfig DTO if possible +- `enable_hub_list` space-separated string -> ordered list JSON, while preserving original string if needed for compatibility +- `star` integer/null -> bool/null +- `id` long -> `legacy_room_id` +- new `id` -> deterministic UUIDv5 + +Legacy `hub` -> Rust `hubs`: + +- preserve `uuid` +- `hub_config` -> same JSON, validate against HubConfig DTO +- `auth` -> auth JSON; do not log tokens +- `ignore_app_id_list` -> canonical list JSON +- `applications_mode` -> integer/bool semantic +- `user_ignore_app_id_list` -> canonical list JSON +- `sort_point` -> integer + +Legacy `extra_app` -> Rust `extra_apps`: + +- old `id` long -> `legacy_room_id` +- `app_id` -> canonical map/hash +- `mark_version_number` -> same +- new `id` -> deterministic UUIDv5 + +Legacy `extra_hub` -> Rust `extra_hubs`: + +- preserve `id` (`GLOBAL` or hub UUID) +- `enable_global` -> bool/integer +- `url_replace_search` -> same +- `url_replace_string` -> same + +### 6.7 Migration failure behavior + +If migration fails: + +- Do not create a partially usable app state. +- Show Migration Recovery page. +- Save: + - migration session ID; + - error code; + - sanitized log; + - backup path; + - source DB hash; + - failed phase. +- Offer actions: + - retry migration; + - export migration report; + - start fresh only after explicit user confirmation; + - open issue template with sanitized details. + +No destructive fallback by default. + +### 6.8 Migration tests + +Create fixtures for at least: + +- v6 database with sample app/hub. +- v8 database after major table rewrite. +- v10 database without unique app index. +- v13 database with `extra_app` table. +- v16 database with `extra_hub` but without `include_version_number_field_regex`. +- v17 database with all fields. +- DB with WAL/SHM uncheckpointed writes. +- DB with malformed optional JSON field. +- DB with auth token; verify logs redact it. +- Existing JSONL store; import to SQLite. +- Partial migration run; retry idempotently. + +Commands: + +```bash +just test-migration +cargo test -p getter-storage legacy_room +./gradlew :legacy_migrator:testDebugUnitTest # Android side, if kept as Gradle module +flutter test test/migration_bootstrap_test.dart +``` + +--- + +## 7. Flutter UI pages and source customization + +### 7.1 Page registry + +Define page registry composition: + +```dart +final pages = [ + ...defaultPages, + ...customPages, +]; +``` + +Conflict rule: + +- Custom page with same route ID overrides default page only if explicitly declared. +- Otherwise duplicate route IDs are compile/test failures. + +### 7.2 Stable UI IDs + +Create a single source of truth: + +```dart +abstract final class UiIds { + static const homePage = UiId('home.page'); + static const homeCheckUpdates = UiId('home.check_updates'); + static const homeOpenApps = UiId('home.open_apps'); + static const appListPage = UiId('app_list.page'); + static const appListItemPrefix = 'app_list.item.'; + static const appDetailPage = UiId('app_detail.page'); + static const migrationPage = UiId('migration.page'); + static const migrationRetry = UiId('migration.retry'); +} +``` + +Every interactive widget must use semantic identifiers/labels through helper widgets: + +```dart +Widget testableButton({ + required UiId id, + required VoidCallback? onPressed, + required Widget child, +}) { + return Semantics( + identifier: id.value, // Flutter 3.19+ where available + label: id.value, // fallback for tools using labels + button: true, + child: ElevatedButton( + key: ValueKey(id.value), + onPressed: onPressed, + child: child, + ), + ); +} +``` + +Avoid localized visible text as the only selector. + +### 7.3 Custom page guardrails + +`AGENTS.md` and custom-page docs must instruct AI agents: + +```text +Allowed to edit: +- packages/upgradeall_pages_custom/** +- custom theme files +- tests under packages/upgradeall_pages_custom/test/** + +Do not edit unless explicitly requested: +- native/getter/** +- generated bindings +- platform adapters +- migration code +- app_shell bootstrap +- storage schema migrations +``` + +Never silence type errors with `dynamic`, unchecked casts, or broad `catch (_) {}`. + +--- + +## 8. AI-friendly CLI workflow + +Create one-command verification through `justfile`. + +Example: + +```make +setup: + flutter doctor + cargo --version + rustup target list --installed + +gen: + cargo run -p getter-codegen + flutter_rust_bridge_codegen generate + +format: + cargo fmt --all + dart format apps packages tools + +check: + cargo clippy --workspace --all-targets -- -D warnings + flutter analyze --fatal-infos + +test: + cargo test --workspace + flutter test + +test-migration: + cargo test -p getter-storage legacy + flutter test test/migration_bootstrap_test.dart + +test-ui: + flutter test integration_test + +build-android-debug: + flutter build apk --debug + +e2e-android: + maestro test e2e/maestro/android + +verify: gen format check test test-migration build-android-debug +``` + +AI agents should usually run: + +```bash +just verify +``` + +For page-only changes: + +```bash +just format +just check +flutter test packages/upgradeall_pages_custom +just test-ui +``` + +--- + +## 9. UI testing plan + +### 9.1 Test layers + +Layer 1: Rust core tests + +- provider fixtures; +- version comparison; +- update status; +- storage migrations; +- legacy import; +- downloader task state transitions; +- plugin permission validation. + +Layer 2: Flutter widget tests + +- page renders loading/empty/error/content states; +- page actions call typed fake `GetterClient`; +- custom page registry override works; +- semantics IDs exist. + +Layer 3: Flutter integration tests + +- app boots fresh; +- app boots after migration success; +- home -> app list -> app detail -> release list; +- renew all progress event updates UI; +- download task flow with fake backend. + +Layer 4: Maestro black-box flows + +- uses semantic IDs, not localized text; +- verifies app can be clicked by external automation; +- good for AI/manual click testing. + +Layer 5: Patrol native automation, only where needed + +- Android notification permission; +- file picker/SAF; +- install permission/system dialogs; +- notification tray interactions. + +### 9.2 Required Maestro flows + +```text +e2e/maestro/android/ + 001_fresh_launch.yaml + 002_migration_success.yaml + 003_open_app_list.yaml + 004_open_app_detail.yaml + 005_renew_all.yaml + 006_download_task.yaml + 007_migration_failure_recovery.yaml +``` + +Every flow should prefer: + +```yaml +- tapOn: + id: home.check_updates +``` + +not: + +```yaml +- tapOn: "Check updates" +``` + +### 9.3 Screenshot/visual tests + +Use screenshots for regression, not as primary selectors. + +- Golden tests for stable widgets. +- Mask dynamic data: time, progress, network text. +- Store baselines per theme/locale if needed. + +--- + +## 10. Plugin and extension plan + +### 10.1 Plugin layers + +Separate: + +1. Provider plugins: release source logic. +2. Downloader plugins: download backend logic. +3. UI configuration: declarative schemas rendered by Flutter/TUI. +4. Source-level page customizations: user-owned Flutter page modules. + +Do not conflate runtime provider plugins with source-level UI customizations. + +### 10.2 V1 plugins + +V1 should support: + +- built-in Rust providers; +- external JSON-RPC provider registration, continuing current concept; +- external JSON-RPC downloader registration; +- plugin manifest; +- config schema; +- permission declaration. + +Example manifest: + +```toml +id = "github" +kind = "provider" +version = "1.0.0" +api_version = "getter.plugin.v1" + +[permissions] +network = ["api.github.com", "github.com"] +filesystem = false + +[ui] +config_schema = "schemas/github-config.schema.json" +``` + +V2 can add Wasm/WASI sandbox plugins after the core rewrite stabilizes. + +--- + +## 11. Implementation phases + +### Phase 0: Freeze legacy baseline and document decisions + +Objective: establish known-good source points before rewriting. + +Tasks: + +1. Tag current Android/Kotlin state in `UpgradeAll`, e.g. `legacy-android-room-v17-baseline`. +2. Tag current `getter` state before storage rewrite. +3. Create ADRs: + - `docs/adr/0001-flutter-shell-rust-core.md` + - `docs/adr/0002-rust-sqlite-storage.md` + - `docs/adr/0003-source-level-page-customization.md` + - `docs/adr/0004-legacy-room-migration.md` +4. Create `docs/architecture/target-architecture.md`. +5. Create `docs/ai-development.md` and root `AGENTS.md`. + +Verification: + +```bash +git status --short +``` + +Expected: only docs/plan changes in planning stage; no code changes until execution begins. + +### Phase 1: Refactor `getter` into a Rust workspace + +Objective: isolate core logic before Flutter integration. + +Tasks: + +1. Create Cargo workspace. +2. Move storage code into `getter-storage`. +3. Move provider code into `getter-providers`. +4. Move manager/version/update logic into `getter-core`. +5. Move downloader code into `getter-downloader`. +6. Move JSON-RPC into `getter-rpc`. +7. Add `getter-cli` with minimal commands. +8. Add `getter-ffi` facade crate. + +Verification: + +```bash +cargo fmt --all --check +cargo test --workspace +cargo clippy --workspace --all-targets -- -D warnings +``` + +Acceptance: + +- No Android/JNI dependency in `getter-core`. +- CLI can initialize storage and list empty apps/hubs. +- Existing provider fixture tests still pass. + +### Phase 2: Replace JSONL storage with Rust SQLite + +Objective: create migration-capable storage foundation. + +Tasks: + +1. Add `getter-storage` SQLite backend. +2. Add embedded migrations. +3. Add schema metadata table. +4. Add models for apps, hubs, extra apps, extra hubs. +5. Add JSONL importer for existing alpha data. +6. Keep JSONL reader as compatibility-only module. +7. Update managers to use storage trait rather than direct JSONL store. + +Verification: + +```bash +cargo test -p getter-storage +cargo test -p getter-core +``` + +Acceptance: + +- Fresh `getter.db` creates schema v1. +- JSONL import test passes. +- Re-running import is idempotent. +- Storage transaction tests pass. + +### Phase 3: Build Android legacy Room export module + +Objective: support old installed UpgradeAll users. + +Tasks: + +1. Create Android legacy migrator module under Flutter Android host. +2. Copy/minimize legacy Room entities, converters, and migrations v6-v17. +3. Add legacy DB work-copy logic. +4. Add checkpoint logic for WAL/SHM. +5. Export `LegacyExportBundle` containing apps, hubs, extra apps, extra hubs. +6. Redact sensitive auth tokens in logs. +7. Add migration status/error DTOs for Flutter. + +Verification: + +```bash +./gradlew :legacy_migrator:testDebugUnitTest +``` + +Acceptance: + +- Can open sample v17 DB and export all four tables. +- Can open older fixture DB and migrate/export to v17 bundle. +- ExtraApp is included. +- Auth fields are present in bundle but redacted in logs. + +### Phase 4: Implement Rust legacy import + +Objective: import legacy Room export bundle into Rust SQLite. + +Tasks: + +1. Define `LegacyRoomExportBundle` Rust DTO. +2. Validate bundle format/version/hash. +3. Canonicalize app IDs and app ID lists. +4. Generate deterministic IDs. +5. Import apps/hubs/extra apps/extra hubs in one transaction. +6. Record migration run. +7. Add rollback/failed migration reporting. + +Verification: + +```bash +cargo test -p getter-storage legacy_room_import +``` + +Acceptance: + +- v17 export imports into `getter.db`. +- v6-v17 fixture exports import correctly. +- Count and field parity tests pass. +- Re-import same bundle does not duplicate records. +- Failed import leaves no partial DB state. + +### Phase 5: Create Flutter app shell + +Objective: minimal Flutter app booting against `getter`. + +Tasks: + +1. Create `apps/upgradeall_flutter`. +2. Preserve Android `applicationId = net.xzos.upgradeall`. +3. Add `native/getter` checkout/submodule. +4. Add `flutter_rust_bridge` or selected FFI generator. +5. Generate minimal Dart bindings. +6. Implement `bootstrap.dart`: + - platform paths; + - legacy migration check; + - getter init; + - error handling. +7. Implement basic AppShell and route host. + +Verification: + +```bash +flutter analyze --fatal-infos +flutter test +flutter build apk --debug +``` + +Acceptance: + +- Fresh app launches to Home page. +- `getter` initializes. +- No domain logic in Flutter shell. + +### Phase 6: Implement page contracts and default pages + +Objective: make page customization safe and typed. + +Tasks: + +1. Create `upgradeall_ui_contract`. +2. Create `upgradeall_ui_kit`. +3. Create `upgradeall_pages_default`. +4. Create `upgradeall_pages_custom` skeleton. +5. Add `UiIds` constants. +6. Add semantic/testable widget wrappers. +7. Implement default pages: + - Home. + - App list. + - App detail. + - Hub manager. + - Discover/cloud config. + - Download tasks. + - Settings. + - Migration status. + +Verification: + +```bash +flutter analyze --fatal-infos +flutter test packages/upgradeall_ui_kit +flutter test packages/upgradeall_pages_default +``` + +Acceptance: + +- Default pages compile only through `ui_contract` and `getter` client. +- Custom package can override a route. +- Widget tests verify semantic IDs. + +### Phase 7: Implement feature parity through getter API + +Objective: migrate current UpgradeAll flows to Rust-backed Flutter UI. + +Feature slices: + +1. App list and status. +2. App detail and release list. +3. Renew all / renew one. +4. Hub manager and auth editing. +5. Cloud config discover/apply. +6. Download info and download tasks. +7. URL replacement and extra hub settings. +8. Extra app mark version. +9. Settings and logs. +10. Android platform installed app scanning. +11. Android installer adapter. +12. Backup/export/import if still required. + +For each slice: + +- Write Rust core tests first. +- Add/extend FFI DTO. +- Add fake `GetterClient` for Flutter tests. +- Implement UI page. +- Add widget test. +- Add integration/Maestro flow if user-visible. + +Verification: + +```bash +just verify +just e2e-android +``` + +Acceptance: + +- Core flow works without Flutter through `getter-cli`. +- Flutter UI only renders/calls commands. + +### Phase 8: Migration end-to-end testing on Android + +Objective: prove real upgrade path. + +Tasks: + +1. Build old legacy APK with test fixture data. +2. Install old APK on emulator. +3. Seed app/hub/extra data. +4. Upgrade in-place to Flutter APK with same applicationId/signing. +5. Verify migration screen. +6. Verify data appears in Flutter UI. +7. Verify `getter.db` has imported records. +8. Verify old DB backup exists. +9. Repeat for v6/v8/v13/v16/v17 fixtures. + +Commands: + +```bash +just build-legacy-fixture-apk +just install-legacy-fixture +just seed-legacy-db-v17 +just build-android-debug +just upgrade-to-flutter-debug +just e2e-migration-android +``` + +Acceptance: + +- No data loss for apps/hubs/extra apps/extra hubs. +- WAL/SHM fixture migrates. +- Failed migration shows recovery page, not crash. +- Migration report is exportable and sanitized. + +### Phase 9: CLI/TUI proof + +Objective: prove `getter` is truly headless. + +CLI commands: + +```text +getter init +getter app list +getter app detail +getter app renew +getter renew-all +getter hub list +getter hub save +getter download submit +getter task list +getter plugin list +getter plugin register +getter legacy import-room-bundle +``` + +Verification: + +```bash +cargo run -p getter-cli -- app list +cargo run -p getter-cli -- legacy import-room-bundle fixtures/legacy-room/v17/export.json +``` + +Acceptance: + +- Main update check and migration import can run without Flutter. + +### Phase 10: Release strategy + +Objective: minimize risk for existing users. + +Stages: + +1. Internal migration test builds. +2. Public alpha with manual export/import only. +3. Beta with automatic Room migration but opt-in. +4. Release candidate with automatic migration by default. +5. Stable Flutter release. + +Release rules: + +- Same applicationId/signing for official Android upgrade. +- No destructive migration fallback. +- Keep old DB backup for at least two stable releases. +- Keep legacy migrator for enough versions to cover direct upgrades from last Kotlin release. +- Publish migration known-issues doc. + +--- + +## 12. Validation matrix + +Rust: + +```bash +cargo fmt --all --check +cargo clippy --workspace --all-targets -- -D warnings +cargo test --workspace +``` + +Flutter: + +```bash +flutter analyze --fatal-infos +flutter test +flutter test integration_test +flutter build apk --debug +``` + +Android legacy migration: + +```bash +./gradlew :legacy_migrator:testDebugUnitTest +just e2e-migration-android +``` + +Maestro: + +```bash +maestro test e2e/maestro/android +``` + +Patrol, only for native OS flows: + +```bash +patrol test -t integration_test/native_permissions_test.dart +``` + +Migration invariants: + +- Every legacy app row maps to exactly one Rust app row. +- Every legacy hub row maps to exactly one Rust hub row. +- Every legacy extra_app row maps to exactly one Rust extra_app row. +- Every legacy extra_hub row maps to exactly one Rust extra_hub row. +- Auth values are preserved in storage, redacted in logs. +- Migration is idempotent. +- Failed migration is recoverable. +- Old DB backup is kept. + +UI/testability invariants: + +- Every route has a stable route ID. +- Every primary action has a stable UI ID. +- No Maestro flow relies only on localized text. +- Custom pages compile against `ui_contract` only. +- Generated bindings are not manually edited. + +--- + +## 13. Risks and mitigations + +Risk: Flutter rewrite loses access to old app-private DB. + +- Mitigation: keep same applicationId and signing key. If not possible, ship bridge export release. + +Risk: current Room -> Rust migration misses data. + +- Mitigation: replace `RustMigration.kt` approach with explicit export bundle including all four tables; add fixture tests for `extra_app`. + +Risk: JSONL storage cannot support long-term schema evolution. + +- Mitigation: move to Rust SQLite before official Flutter release; keep JSONL importer only for alpha compatibility. + +Risk: AI/user custom pages create merge conflicts. + +- Mitigation: stable `ui_contract`, `ui_kit`, and downstream-owned `pages_custom`; upstream avoids touching custom package. + +Risk: AI UI tests become brittle. + +- Mitigation: semantic identifiers/test IDs, Maestro flows by ID, widget tests by `ValueKey`, screenshot tests only for visual regression. + +Risk: generated FFI code becomes confusing to AI. + +- Mitigation: `AGENTS.md` says never edit generated bindings; run `just gen`. + +Risk: platform-specific Android features leak into core. + +- Mitigation: define `PlatformServices` / Rust platform callback traits; keep PackageManager, installer, notifications, SAF in Flutter Android platform adapter. + +Risk: migration failure bricks startup. + +- Mitigation: migration recovery page, retry, backup, sanitized report, explicit fresh-start option only. + +--- + +## 14. Open questions to settle before execution + +1. Are official Flutter Android builds guaranteed to keep `applicationId = net.xzos.upgradeall` and signing key lineage? + - Recommended answer: yes, required for direct migration. + +2. Should `getter` use Rust SQLite immediately, or first keep current JSONL and migrate later? + - Recommended answer: Rust SQLite before official Flutter release. JSONL only as alpha compatibility import. + +3. How long should the legacy Room migrator remain in the Flutter app? + - Recommended answer: at least two stable release cycles, or until analytics/support indicates old Kotlin direct upgrades are negligible. + +4. What is the minimum old DB schema version to support? + - Recommended answer: support v6-v17 because current code has migrations from v6; below v6 requires manual bridge export or unsupported warning. + +5. Should the first Flutter release include desktop targets? + - Recommended answer: use Linux desktop as a development/test target, but Android is the official migration target first. + +6. Should user custom pages be tracked in upstream? + - Recommended answer: upstream provides skeleton and examples; after initial skeleton, upstream avoids changes in `pages_custom` except major contract migration. + +7. Should plugin runtime use Wasm in v1? + - Recommended answer: no. Use built-in Rust + external JSON-RPC first; add Wasm after core storage/migration/UI stabilizes. + +--- + +## 15. First execution batch recommendation + +Do not start by writing Flutter screens. + +Start with this order: + +1. ADRs + AGENTS.md + justfile skeleton. +2. `getter` workspace split. +3. Rust SQLite storage and migration framework. +4. Legacy Room export/import tests. +5. Minimal Flutter app shell + getter init. +6. Migration status page. +7. Home/AppList feature slice. + +Reason: if migration and headless core are wrong, Flutter page work will hide architectural mistakes. + +First concrete task after approval: + +```text +Create ADRs and an executable repo verification skeleton: +- docs/adr/0001-flutter-shell-rust-core.md +- docs/adr/0002-rust-sqlite-storage.md +- docs/adr/0003-source-level-page-customization.md +- docs/adr/0004-legacy-room-migration.md +- AGENTS.md +- justfile +``` + +Then run: + +```bash +just verify +``` + +Expected initially: verify may only check available existing pieces, but it becomes the single AI/operator entrypoint for the rest of the rewrite. diff --git a/docs/refactor/2026-06-21-reconciled-full-rewrite-plan.md b/docs/refactor/2026-06-21-reconciled-full-rewrite-plan.md new file mode 100644 index 00000000..dd60420d --- /dev/null +++ b/docs/refactor/2026-06-21-reconciled-full-rewrite-plan.md @@ -0,0 +1,602 @@ +# 2026-06-21 Reconciled Full Rewrite Plan + +> Status: implementation-grade plan, not implementation completion +> Scope: UpgradeAll rewrite toward **Flutter APP + Rust getter core + Lua package repository** +> Basis: `AGENTS.md`, `docs/README.md`, `docs/architecture/**`, `docs/app/flutter-ui-feature-parity-and-testing.md`, current source inspection, and context-builder/oracle findings from 2026-06-21. + +## 0. Purpose + +The user asked that the work must not stop at passing tests: the CLI and APP must actually run, and the result must be cross-platform. After clarification, the selected deliverable for this pass is **Full rewrite plan**. + +Therefore this document does **not** claim the Flutter UI, CLI, migration, or cross-platform runtime are already complete. It defines the implementation sequence and acceptance gates required before anyone may claim completion. + +## 1. Source-of-truth reconciliation + +### 1.1 Authoritative docs for future implementation + +Implementation must follow these files first: + +1. `AGENTS.md` +2. `docs/README.md` +3. `docs/architecture/upgradeall-getter-rewrite-wiki.md` +4. `docs/architecture/adr/0001-app-centric-lua-package-repository-model.md` +5. `docs/architecture/adr/0002-getter-flutter-platform-boundary.md` +6. `docs/architecture/adr/0003-legacy-room-migration.md` +7. `docs/architecture/adr/0004-sqlite-main-db-and-cache-db.md` +8. `docs/architecture/adr/0005-lua-package-api.md` +9. `docs/app/flutter-ui-feature-parity-and-testing.md` + +Older files under `docs/adr/**` and `docs/refactor/2026-06-20-*` are useful background, but where they conflict with the current architecture docs they must be revised or superseded. + +### 1.2 Conflicts to resolve before coding + +| Conflict | Current rule | Required action | +|---|---|---| +| Older docs use `hub` as a new CLI/domain concept (`getter hub list`, Hub Manager, hub tables). | Do not reintroduce the old hub-app model. Providers/sources/backends are not package identity. | Supersede old CLI ADR with package/repository/source vocabulary. Keep `hub` only as legacy migration input terminology. | +| Older plan uses a single `getter.db`. | Current ADR-0004 requires SQLite **main DB + cache DB** split. | Implement two DBs from the beginning of the new getter storage path. | +| Older plan describes source-level page customization and plugin ideas. | Runtime UI customization/plugin framework is explicitly not allowed for v1. | Only source fork/patch-stack customization is allowed. Runtime provider extensibility must be gated separately and must not become UI plugins. | +| Older plan leans toward a specific FFI generator. | Current docs require embedded Rust library / FFI-style boundary; generator is not fixed. | Choose FFI approach through an explicit gate before Flutter integration. | +| Older BDD plan sounds exhaustive. | Current testing rules say BDD is for meaningful user-visible flows; do not over-test with BDD. | Use BDD for CLI/App/migration flows; use TDD/unit/integration tests for domain algorithms. | + +## 2. Current implementation baseline + +As of this planning pass: + +- The repository is still mainly the legacy Android/Kotlin app. +- There is no Flutter project (`pubspec.yaml`/Dart entry point absent). +- A partial Rust getter workspace exists under `core-getter/src/main/rust/getter`. +- `getter-core` currently has package id, repository layout, and minimal Lua table validation tests. +- `getter-storage` currently has main/cache SQLite skeleton and pure legacy mapping helper tests. +- `getter-cli` is only a library skeleton; no runnable binary exists. +- The Android JNI/RPC path currently binds a placeholder local TCP endpoint and parks forever; it is not a full getter RPC surface. +- Existing legacy Kotlin Room → Rust migration writes toward old JSONL/RPC concepts and must not be treated as the new migration implementation. +- The git worktree is already dirty/staged from prior work, including a staged deletion of the old getter gitlink and untracked replacement workspace/docs. Before implementation work, reconcile the baseline deliberately. + +## 3. Completion definition: “actually runs” + +Passing unit tests is insufficient. A milestone may be called complete only when it provides runtime evidence. + +### 3.1 Required runtime evidence types + +1. **CLI runtime evidence** + - The `getter` CLI is invoked as an external process. + - It creates/opens real SQLite main/cache DB files under a temp data directory. + - It loads/evaluates real Lua package files from a fixture repository. + - It emits stable JSON stdout for success/failure envelopes. + - Its output is saved as test artifacts for at least smoke scenarios. + +2. **APP runtime evidence** + - Flutter app boots through the real app entry point, not just widget tests. + - At least one desktop/dev target and Android debug target are launched in smoke gates. + - UI flows use stable route/action/state IDs, not localized text-only selectors. + - App interacts with a fake/offline getter backend first, then the real FFI getter when ready. + +3. **Cross-platform evidence** + - Rust getter/core/CLI tests and smoke commands run on a host CI matrix. + - Flutter builds and smoke-runs on explicitly approved app targets. + - Path handling, data-dir handling, and fixture loading use platform-neutral temp dirs. + +4. **Migration evidence** + - Legacy Room fixture bundles import into new getter main DB inside a transaction. + - Dropped fields are documented. + - Per-app failures do not block whole-app migration. + - Global migration failure reaches a recovery UI, not a crash. + +### 3.2 Recommended first target matrix + +This matrix should be confirmed before implementation: + +| Layer | Required first target | Later expansion | +|---|---|---| +| Rust getter core/CLI | Linux host now; CI matrix Linux/macOS/Windows before release | Additional Android target builds through Gradle/NDK | +| Flutter APP | Android debug + Linux desktop dev smoke | Windows/macOS desktop smoke if they are official supported targets | +| Legacy migration | Android official upgrade path | Manual import/export recovery path for non-official builds | + +## 4. Decision gates before implementation + +Do not launch broad implementation until these decisions are recorded: + +1. **Cross-platform target scope**: Android + Linux dev smoke, or Android/Linux/Windows/macOS as release targets? +2. **Android upgrade identity**: Will official Flutter builds keep `applicationId = net.xzos.upgradeall` and signing key lineage? +3. **CLI vocabulary**: Supersede old `hub` CLI commands with `repo/source/provider/package` commands. +4. **FFI approach**: `flutter_rust_bridge`, manual C ABI, or a staged temporary JSON/RPC dev bridge. +5. **Main DB/cache DB schema**: exact v1 tables and migration mechanism. +6. **Legacy migration range**: which old Room schema versions are supported directly; which fields are dropped. +7. **Provider extensibility**: v1 built-in providers only, external JSON-RPC providers, or deferred plugin runtime. +8. **Repository layout in this repo**: keep transitional `core-getter/src/main/rust/getter` or move toward a cleaner workspace path. +9. **Baseline cleanup**: resolve staged deletion/untracked replacement workspace before code-writing subagents start. + +## 5. New CLI contract direction + +The older `getter hub list` contract must be revised. The new CLI should be package/repository-centric. + +Recommended initial grammar: + +```text +getter --data-dir init +getter --data-dir repo list +getter --data-dir repo add [--priority ] +getter --data-dir repo eval +getter --data-dir package eval [--repo ] +getter --data-dir app list +getter --data-dir app show +getter --data-dir app check [--offline-fixtures] +getter --data-dir template list [--repo ] +getter --data-dir template run --input +getter --data-dir legacy import-room-bundle +getter --data-dir storage validate +getter --data-dir diagnostics +``` + +Conventions: + +- JSON stdout is the default automation contract. +- Invalid CLI usage may use stderr/help text and exit code `2`. +- Structured command failures should emit JSON error envelopes on stdout. +- No command should require Flutter/Android APIs unless it explicitly declares a platform adapter/mock. + +Success envelope: + +```json +{ + "ok": true, + "command": "repo list", + "data": {}, + "warnings": [] +} +``` + +Error envelope: + +```json +{ + "ok": false, + "command": "legacy import-room-bundle", + "error": { + "code": "migration.invalid_bundle", + "message": "Legacy Room export bundle is invalid", + "report_path": "/path/to/report.json" + } +} +``` + +## 6. Implementation phases + +### Phase 0 — Baseline, docs reconciliation, and verification skeleton + +Goal: start from a known, reviewable baseline. + +Tasks: + +1. Resolve the current git/submodule/workspace state deliberately. +2. Add/supersede ADR for the package-centric CLI contract. +3. Mark older hub-oriented docs as legacy background or update terminology. +4. Add a root verification entrypoint (`justfile` or equivalent) that can run available checks. +5. Document the target platform matrix. + +Validation: + +```bash +git status --short +cargo test --manifest-path core-getter/src/main/rust/getter/Cargo.toml --workspace +./gradlew projects +``` + +Acceptance: + +- No hidden dirty baseline. +- New docs state that `hub` is legacy migration terminology only. +- Verification command is present even if later targets are initially skipped. + +### Phase 1 — Getter CLI executable spine + +Goal: make getter independently runnable before Flutter UI work. + +Tasks: + +1. Add a real `getter-cli` binary target. +2. Implement minimal CLI parser and JSON envelopes. +3. Implement `init`, `repo list`, `app list`, `storage validate`, and structured errors. +4. Add BDD/Gherkin CLI smoke scenarios that invoke the binary as an external process. +5. Add internal unit tests for output serialization and data-dir handling. + +Validation: + +```bash +cargo test --manifest-path core-getter/src/main/rust/getter/Cargo.toml --workspace +cargo run --manifest-path core-getter/src/main/rust/getter/Cargo.toml -p getter-cli -- --data-dir /tmp/ua-getter-smoke init +cargo run --manifest-path core-getter/src/main/rust/getter/Cargo.toml -p getter-cli -- --data-dir /tmp/ua-getter-smoke repo list +cargo run --manifest-path core-getter/src/main/rust/getter/Cargo.toml -p getter-cli -- --data-dir /tmp/ua-getter-smoke app list +cargo test --manifest-path core-getter/src/main/rust/getter/Cargo.toml -p getter-cli --test bdd_cli +``` + +Acceptance: + +- CLI creates real main/cache DB files. +- CLI returns stable JSON. +- No Android/JNI dependency appears in `getter-core` or `getter-cli`. + +### Phase 2 — Repository overlay and Lua package evaluation + +Goal: prove the app/package-centric repository model with real Lua files. + +Tasks: + +1. Implement multi-repository registry and priority resolution. +2. Implement resolved view: highest-priority package wins by package id. +3. Complete Lua evaluation boundary for JSON-like tables. +4. Add `package_from(repo, id)` with explicit repo id. +5. Add Lua override helper support through repo `lib` modules. +6. Add template listing/running skeleton. +7. Add fixture repositories: `official`, `local`, `local_autogen`. + +Validation: + +```bash +cargo run -p getter-cli -- --data-dir /tmp/ua-getter-smoke repo add official fixtures/repos/official --priority 0 +cargo run -p getter-cli -- --data-dir /tmp/ua-getter-smoke repo add local_autogen fixtures/repos/local_autogen --priority -1 +cargo run -p getter-cli -- --data-dir /tmp/ua-getter-smoke repo eval official +cargo run -p getter-cli -- --data-dir /tmp/ua-getter-smoke package eval android/org.fdroid.fdroid --repo official +cargo test -p getter-core repository lua +``` + +Acceptance: + +- `local` > `official` > `local_autogen` priority behavior is tested. +- Path-derived package id must match declared id. +- Lua runtime/schema/domain errors are distinct. +- Free-network permission is surfaced as metadata, not executed by default. + +### Phase 3 — SQLite main/cache DB foundation + +Goal: replace skeleton storage with a real package/repository/user-state schema. + +Main DB v1 should store: + +- repositories registry and priorities; +- tracked packages and enabled/favorite state; +- user source priority overrides; +- ignored versions, pins, and per-package user state; +- migration records; +- settings and credential references; +- download task persistent state. + +Cache DB v1 should store: + +- evaluated package metadata; +- Lua validation result; +- provider responses; +- release candidates; +- selected latest version; +- search/cache indexes where needed. + +Tasks: + +1. Define schema migrations for main DB and cache DB. +2. Add storage traits used by CLI/core. +3. Add cache key calculation tests including repo id/revision/package hash/API version/getter version/platform/permission mode. +4. Add fail-fast corruption/error behavior with clear diagnostics. + +Validation: + +```bash +cargo test -p getter-storage +cargo run -p getter-cli -- --data-dir /tmp/ua-getter-smoke storage validate +sqlite3 /tmp/ua-getter-smoke/main.db '.schema' +sqlite3 /tmp/ua-getter-smoke/cache.db '.schema' +``` + +Acceptance: + +- Main DB and cache DB are separate files. +- Storage operations are transactional. +- Cache can be cleared without losing user state. + +### Phase 4 — Update lifecycle and offline provider/download proof + +Goal: prove update behavior without relying on flaky live network. + +Tasks: + +1. Implement lifecycle validation for `preflight`, `setup`, `match`, `discover`, `prepare`, `select`, `resolve`, `post_update` where applicable. +2. Add fake/offline provider fixture responses. +3. Implement version comparison and candidate selection in Rust getter. +4. Implement update action generation (`Download`, `Install`, `OpenUrl`) with schema validation. +5. Implement download task state machine skeleton. +6. Keep direct network disabled unless package permission allows and user warning is visible. + +Validation: + +```bash +cargo test -p getter-core version repository lua lifecycle +cargo test -p getter-providers --features fixtures +cargo test -p getter-downloader +cargo run -p getter-cli -- --data-dir /tmp/ua-getter-smoke app check android/org.fdroid.fdroid --offline-fixtures +cargo run -p getter-cli -- --data-dir /tmp/ua-getter-smoke task list +``` + +Acceptance: + +- Main update flow works from CLI without Flutter. +- Offline provider fixture can produce a selected update and actions. +- Live network is not required for smoke gates. + +### Phase 5 — Getter platform boundary and FFI facade + +Goal: expose getter to hosts without leaking domain logic into Flutter. + +Tasks: + +1. Choose and document FFI approach. +2. Define narrow stable DTOs for Flutter-facing facade. +3. Define platform capability traits/callbacks for PackageManager inventory, installer, notifications, SAF/file picker, and installed version lookup. +4. Provide fake platform adapter for desktop/integration tests. +5. Keep JSON-RPC/local daemon path as optional/dev/external plugin path, not main Flutter path. + +Validation: + +```bash +cargo test -p getter-ffi +cargo test -p getter-rpc +cargo metadata --format-version 1 --manifest-path core-getter/src/main/rust/api_proxy/Cargo.toml +./gradlew projects +``` + +Acceptance: + +- Flutter/UI hosts call facade DTOs, not internal storage/provider modules. +- Android-only APIs are behind platform capabilities. +- Existing Gradle Cargo metadata path remains intact or is intentionally replaced with docs and working build files. + +### Phase 6 — Minimal Flutter app shell + +Goal: create a real cross-platform UI shell that boots. + +Tasks: + +1. Create Flutter project/workspace in the chosen repo layout. +2. Add packages for: + - app shell; + - getter contract/generated bindings; + - UI contract; + - UI kit; + - default pages; + - user/source-fork custom pages skeleton if desired, but no runtime UI plugin framework. +3. Add stable route/action/state IDs. +4. Implement bootstrap with fake getter first, then real getter init. +5. Implement minimal Home, App list, Repositories, Downloads, Logs, Settings, Migration status shell pages. + +Validation: + +```bash +flutter pub get +flutter analyze --fatal-infos +flutter test +flutter test integration_test +flutter build linux --debug +flutter build apk --debug +``` + +Runtime smoke: + +```bash +flutter run -d linux --debug +adb install -r build/app/outputs/flutter-apk/app-debug.apk +adb shell am start -W -n net.xzos.upgradeall.debug/ +adb shell pidof net.xzos.upgradeall.debug +``` + +Acceptance: + +- App boots on Linux desktop dev target and Android debug target. +- UI tests use stable IDs. +- Flutter contains no provider/update/version/storage logic. + +### Phase 7 — Flutter UI feature parity slices + +Goal: implement user-visible flows through getter APIs. + +Implement one vertical slice at a time: + +1. Home summary and update count. +2. App/package list. +3. App detail with source/version/artifact information. +4. Repository/source visibility. +5. Free-network yellow warning tag. +6. Installed autogen preview and confirmation. +7. Download task view and controls. +8. Settings. +9. Logs/diagnostics. +10. Migration/recovery page. + +For each slice: + +- Write BDD scenario for user-visible behavior. +- Add/extend getter CLI/core test if logic is new. +- Add Flutter widget/integration tests. +- Run a real UI smoke flow when the slice affects navigation or launch. + +Acceptance: + +- Every route and primary action has stable IDs. +- App pages render loading/empty/error/content states. +- BDD scenarios are meaningful and not duplicated low-level unit tests. + +### Phase 8 — Android legacy Room migration + +Goal: automatic migration for normal official Android users. + +Tasks: + +1. Confirm official app id/signing lineage. +2. Implement Android-only legacy migrator that exports Room DB v6-v17 to a sanitized bundle. +3. Include `app`, `hub`, `extra_app`, and `extra_hub` legacy tables in the export. +4. Import bundle into getter main DB transactionally. +5. Generate `local` Lua packages only for legacy migration cases where needed. +6. Preserve mapped user state; document dropped fields. +7. Implement migration success/warning/failure UI. + +Validation: + +```bash +cargo test -p getter-storage legacy_room +./gradlew :legacy_migrator:testDebugUnitTest +flutter test test/migration_bootstrap_test.dart +flutter test integration_test/migration_recovery_test.dart +``` + +End-to-end Android evidence: + +```bash +# outline; exact names depend on fixture tooling +./gradlew :app:installLegacyFixtureDebug +adb shell am start -W -n net.xzos.upgradeall/ +./gradlew :upgradeall_flutter:installDebug +adb shell am start -W -n net.xzos.upgradeall.debug/ +adb shell run-as net.xzos.upgradeall.debug ls files +``` + +Acceptance: + +- Single unmapped package does not block migration. +- Global migration failure reaches recovery UI and exportable report. +- Old DB backup is retained. +- No auth/token secret leaks in logs/reports. + +### Phase 9 — Installed autogen and local/local_autogen behavior + +Goal: implement user-visible generated fallback packages without corrupting user overrides. + +Tasks: + +1. Android adapter scans installed inventory. +2. getter computes autogen candidates. +3. UI shows confirmation list. +4. Confirm writes package files to `local_autogen`. +5. Cleanup only removes missing generated packages from `local_autogen`, never `local`. + +Validation: + +```bash +cargo test -p getter-core autogen +cargo run -p getter-cli -- --data-dir /tmp/ua-getter-smoke template run android_installed_app --input fixtures/installed/fdroid.json +flutter test integration_test/installed_autogen_test.dart +``` + +Acceptance: + +- Generated files are visible and can be evaluated by CLI. +- `local` remains untouched by ordinary cleanup. + +### Phase 10 — Cross-platform release readiness + +Goal: prove the project can be built, tested, and run on selected platforms. + +Required before release candidate: + +```bash +cargo fmt --all --check +cargo clippy --workspace --all-targets -- -D warnings +cargo test --workspace --all-targets +cargo run -p getter-cli -- --data-dir "$TMPDIR/ua-getter" init +cargo run -p getter-cli -- --data-dir "$TMPDIR/ua-getter" repo eval official +flutter analyze --fatal-infos +flutter test +flutter test integration_test +flutter build apk --debug +flutter build linux --debug +``` + +CI matrix: + +- Linux: full Rust + Flutter Linux + Android APK build. +- macOS: Rust core/CLI + Flutter tests/build where available. +- Windows: Rust core/CLI + Flutter tests/build where available. +- Android emulator lane: install/launch smoke and critical migration/autogen/download flows. + +Acceptance: + +- Cross-platform means explicit matrix rows are green, not an informal claim. +- Any unsupported platform is named as unsupported or not-yet-release-gated. + +## 7. BDD scenario inventory + +Use Gherkin for these user-visible behaviors: + +### Getter CLI + +- Initialize a new getter data directory. +- List repositories in JSON. +- Add/evaluate a local repository. +- Evaluate a package from a fixture Lua repo. +- List tracked apps before/after adding state. +- Check one app through offline provider fixtures. +- Submit/list a download task through fake downloader. +- Reject malformed legacy import bundle without partial state. +- Report unsupported valid legacy bundle until full import is implemented. + +### Flutter APP + +- Fresh launch reaches Home. +- Home opens App list. +- App list opens App detail. +- App detail displays source/version/artifact data from fake getter. +- Free-network package displays yellow warning tag. +- Installed autogen preview asks confirmation before writing. +- Cleanup preview only targets `local_autogen`. +- Download task flow shows queued/running/succeeded/failed states. +- Migration success reaches migrated App list. +- Migration failure reaches recovery page. + +### Migration + +- Legacy v17 export imports apps and user state. +- Legacy export with extra_app preserves ignored/marked version where mapped. +- Auth/token values are preserved where supported but redacted from reports. +- Unmapped package creates warning/missing-package state, not global failure. + +## 8. Documentation updates required with implementation + +Update docs in the same patch when implementation changes any of these: + +- package/repository/Lua schema; +- CLI command grammar or JSON envelope; +- main/cache DB schema; +- migration mapping/dropped fields; +- FFI/platform capability boundary; +- UI route/action/state IDs; +- validation matrix/CI gates. + +Prefer new ADRs for costly decisions: + +- `0006-package-centric-cli-command-contract.md` under `docs/architecture/adr/`. +- `0007-ffi-binding-approach.md` if/when FFI generator is chosen. +- `0008-platform-target-matrix.md` once cross-platform targets are fixed. + +## 9. Stop rules + +Stop and ask for a decision if any implementation requires: + +- changing official Android application id or signing assumptions; +- introducing runtime UI customization/plugin framework; +- reusing old hub-app model as the new product model; +- dropping legacy migration fields not documented in migration docs; +- adding Android-specific APIs to getter core; +- putting provider/update/version/storage logic in Flutter; +- claiming cross-platform support without a runnable gate for that platform. + +## 10. First recommended implementation batch + +Do not start with Flutter screens. + +Recommended first batch: + +1. Reconcile docs and supersede `getter hub list` with package/repo CLI contract. +2. Resolve git/submodule dirty baseline. +3. Add getter CLI binary. +4. Add CLI BDD smoke for `init`, `repo list`, `app list`, and malformed legacy import failure. +5. Make CLI create real main/cache DB files and return stable JSON. +6. Add fixture repository and package evaluation CLI smoke. +7. Only then start minimal Flutter shell. + +This sequence keeps the core honest: if the CLI cannot perform the domain workflow, Flutter must not paper over the missing getter behavior. diff --git a/docs/refactor/phase-1-getter-cli-bdd-plan.md b/docs/refactor/phase-1-getter-cli-bdd-plan.md new file mode 100644 index 00000000..2d282a18 --- /dev/null +++ b/docs/refactor/phase-1-getter-cli-bdd-plan.md @@ -0,0 +1,242 @@ +# Phase 1a Plan: Getter CLI BDD Spine + +Date: 2026-06-20 + +## Purpose + +Phase 1a creates the first executable TDD spine for the rewrite without starting Flutter screen work. It is the entry spine for canonical Phase 1, not a replacement for the full getter workspace refactor. The goal is to make `getter` usable as a CLI and library-backed engine through behavior-first development. + +This phase follows the clarified testing rule: + +- User-facing interfaces require Cucumber/Gherkin BDD coverage. +- Getter CLI is a user-facing interface and needs complete BDD coverage for supported commands. +- Getter internals use traditional Rust unit/integration/property tests. + +## Strict review of the plan + +### Assumption: Start with the CLI before Flutter UI + +Verdict: keep it. + +Reason: the canonical 06-20 plan says `getter` owns product logic. A CLI-first slice exercises getter behavior without hiding engine mistakes behind UI scaffolding. + +### Assumption: Use Cucumber/Gherkin for every Rust test + +Verdict: reject it. + +Reason: the user clarified that BDD is for user-facing integration/acceptance behavior. Internal Rust behavior should keep fast traditional tests. + +### Assumption: Current `src/main.rs` means the CLI already exists + +Verdict: reject it. + +Reason: `src/main.rs` currently prints `Hello, world!`. The binary exists structurally, but the supported command contract does not exist yet. + +### Assumption: The stashed direct-JNI rewrite can be resumed as implementation + +Verdict: reject for Phase 1. + +Reason: Phase 1 is CLI/library test spine work. Stash mining is allowed only after comparing each piece against ADRs and the canonical plan. + +## Proposed test/tooling shape + +### Getter CLI BDD + +Initial runner direction: Rust Cucumber (`cucumber-rs`) for `.feature` files that invoke the `getter` binary. + +Target-aligned layout for the future getter workspace: + +```text +getter/ + crates/ + getter-cli/ + features/ + cli/ + init.feature + app_list.feature + hub_list.feature + legacy_import_room_bundle_failure.feature + tests/ + bdd_cli.rs + support/ + cli_world.rs + fixtures.rs +``` + +If implementation starts before the repository is moved to this target workspace, the temporary path under `core-getter/src/main/rust/getter/` must be treated as transitional. The test language and command contracts should still match the target layout. + +Step definitions should: + +- create an isolated temporary data directory per scenario; +- invoke the compiled `getter` binary as an external process; +- assert exit code, stdout/stderr, output schema, and filesystem/database side effects; +- avoid depending on network unless the scenario explicitly needs a mocked provider/server; +- preserve sanitized failure artifacts for debugging. + +### Internal Rust tests + +Use traditional Rust tests for: + +- command parser units; +- output schema serialization; +- storage initialization; +- canonical IDs; +- legacy import mapping; +- migration report creation; +- provider parsing; +- version comparison; +- download orchestration edge cases. + +## CLI command contract + +The executable CLI contract must be accepted before feature files are implemented. The proposed contract is recorded in [`../adr/0007-getter-cli-command-contract.md`](../adr/0007-getter-cli-command-contract.md). + +This Phase 1a plan uses that proposed grammar consistently: + +```text +getter --data-dir init +getter --data-dir app list +getter --data-dir hub list +getter --data-dir legacy import-room-bundle +``` + +Until ADR 0007 is accepted or revised, these commands are planning placeholders rather than executable supported contracts. + +## First behavior slices + +### Slice 1: CLI initializes an empty data directory + +Feature: + +```gherkin +@getter-cli @smoke +Feature: Getter CLI initialization + Scenario: User initializes a new getter data directory + Given an empty getter data directory + When I run getter init for that directory + Then the command succeeds + And the output is valid JSON + And the getter data directory is usable +``` + +Implementation work allowed by this slice: + +- Replace `Hello, world!` with minimal CLI parsing. +- Create or open canonical getter-owned SQLite storage with minimal metadata and empty app/hub tables. +- Add JSON success/error output envelope. +- Add internal tests for SQLite storage init and output serialization. + +Implementation work not allowed by this slice: + +- Full provider registry. +- Flutter UI. +- Android migration. +- Downloader implementation. + +### Slice 2: CLI lists empty app and hub catalogs + +Feature: + +```gherkin +@getter-cli @smoke +Feature: Getter CLI app listing + Scenario: User lists apps before adding any app records + Given an initialized getter data directory + When I run getter app list for that directory + Then the command succeeds + And the output contains an empty app list + +Feature: Getter CLI hub listing + Scenario: User lists hubs before adding any hub records + Given an initialized getter data directory + When I run getter hub list for that directory + Then the command succeeds + And the output contains an empty hub list +``` + +Implementation work allowed: + +- Minimal read path through getter core/library. +- Stable app-list and hub-list output DTOs. +- Internal tests for empty app and hub listing. + +### Slice 3: CLI reports non-destructive legacy import failure + +Feature: + +```gherkin +@getter-cli @migration +Feature: Legacy import failure recovery + Scenario: User receives a non-destructive report when legacy import fails + Given a corrupted legacy export bundle + And an initialized getter data directory + When I run getter legacy import-room-bundle for that bundle + Then the command fails with a documented migration error + And no partially usable imported state is created + And a sanitized migration report is available + + Scenario: User receives a not-implemented failure when a valid bundle is supplied + Given a syntactically valid but unsupported legacy export bundle + And an initialized getter data directory + When I run getter legacy import-room-bundle for that bundle + Then the command fails because import is not implemented yet +``` + +Implementation work allowed: + +- Malformed-bundle detection. +- Unsupported/Not-Implemented classification for syntactically valid bundles. +- Import error classification for `migration.invalid_bundle` and `migration.unsupported_bundle`. +- Non-destructive transaction boundary for failed import. +- Minimal sanitized JSON migration report for malformed and unsupported bundles. +- Internal tests for report redaction and no-state-change semantics. + +Implementation work not allowed: + +- Full Room export implementation. +- Full Flutter migration page. +- Real legacy schema mapping beyond malformed/corrupted bundle rejection and unsupported valid bundle handling. + +## Commit-sized sequence + +1. Add Cucumber runner dependencies and a failing `init.feature` with step skeleton. +2. Add minimal CLI parser and JSON output envelope to make `init.feature` pass. +3. Add internal Rust tests for storage init and output serialization. +4. Add failing `app_list.feature` and `hub_list.feature` for empty catalog listing. +5. Implement minimal library/core read paths to make empty app/hub listing pass. +6. Add failing `legacy_import_room_bundle_failure.feature` for malformed bundle behavior. +7. Implement migration report/error skeleton and no-state-change semantics for malformed bundles only. +8. Extend `just verify` to run getter CLI BDD and internal Rust tests. + +## Verification targets to add in Phase 1 + +Proposed future just targets: + +```make +test-getter-unit: + cargo test --manifest-path core-getter/src/main/rust/getter/Cargo.toml --lib --tests + +test-getter-bdd: + cargo test --manifest-path core-getter/src/main/rust/getter/Cargo.toml --test bdd_cli + +verify: status cargo-metadata gradle-projects test-getter-unit test-getter-bdd bdd-plan-check +``` + +## Mapping to canonical Phase 1 acceptance + +Canonical Phase 1 requires more than this CLI spine. Phase 1a contributes the first executable behavior spine, then the broader Phase 1 must still complete: + +- target getter workspace split (`getter-core`, `getter-storage`, `getter-providers`, `getter-downloader`, `getter-plugin-api`, `getter-ffi`, `getter-rpc`, `getter-cli` or accepted equivalents); +- no Android/JNI dependency inside Getter Core; +- CLI can initialize canonical storage; +- CLI can list empty apps and hubs; +- provider fixture tests for core behavior; +- `cargo test --workspace` or transitional equivalent passes. + +## Phase 1a decisions now captured + +1. ADR 0007 is accepted for the Phase 1a CLI contract; future CLI changes must explicitly extend or revise it. +2. `getter init` creates/opens SQLite immediately, not JSONL durable storage. +3. The malformed-bundle scenario is only a migration failure skeleton, not full legacy import implementation. +4. The supported legacy schema range and bundle version remain deferred until real import mapping starts. +5. Migration reports are JSON-first. Markdown support summaries can be generated later for issue templates/support. diff --git a/docs/refactor/phase-1a-work-plan.md b/docs/refactor/phase-1a-work-plan.md new file mode 100644 index 00000000..17682cb5 --- /dev/null +++ b/docs/refactor/phase-1a-work-plan.md @@ -0,0 +1,78 @@ +# Phase 1a Work Plan: Getter CLI BDD Spine + +Date: 2026-06-20 +Status: Approved to start implementation + +## Goal + +Create the first executable TDD/BDD spine for the rewrite through the Getter CLI, without starting Flutter screen work and without reviving the stashed direct-JNI/RPC rewrite as accepted architecture. + +This work implements the first user-facing CLI behavior slices from `docs/refactor/phase-1-getter-cli-bdd-plan.md` and follows the CLI contract in `docs/adr/0007-getter-cli-command-contract.md`. + +## Approved contract for this slice + +Initial supported commands: + +```text +getter --data-dir init +getter --data-dir app list +getter --data-dir hub list +getter --data-dir legacy import-room-bundle +``` + +Phase 1a constraints: + +- JSON output is the default machine-readable CLI contract. +- `--data-dir ` is mandatory in tests and early development. +- `getter init` creates/opens canonical getter-owned SQLite storage, not JSONL durable storage. +- `app list` and `hub list` return empty collections for newly initialized storage. +- `legacy import-room-bundle` in Phase 1a only covers malformed/corrupted bundle rejection plus explicit unsupported/not-implemented handling for syntactically valid bundles; no full Room import mapping yet. +- All user-facing CLI behavior added here needs Cucumber/Gherkin BDD coverage. +- Internal storage/output/parser behavior should use traditional Rust tests where appropriate. + +## Validation contract + +A successful Phase 1a implementation must provide evidence for: + +1. A Cucumber/Gherkin CLI BDD runner exists for getter CLI scenarios. +2. A failing `init` scenario was added first and is made green. +3. `getter --data-dir init` succeeds and emits valid JSON with `ok: true`. +4. `getter --data-dir app list` succeeds after init and emits an empty app list. +5. `getter --data-dir hub list` succeeds after init and emits an empty hub list. +6. `getter --data-dir legacy import-room-bundle ` fails non-destructively with a structured migration error and a sanitized JSON report path. +7. `getter --data-dir legacy import-room-bundle ` fails with a stable unsupported/not-implemented migration error and does not mutate the initialized store. +8. SQLite is used for the durable getter store initialized in this slice. +9. Traditional Rust tests cover core/internal pieces that are not best expressed as Gherkin. +10. `just verify` is extended to include the new getter CLI BDD/internal tests or a transitional target that proves them. + +## Non-goals + +- Do not implement Flutter UI. +- Do not implement full legacy Room export/import mapping. +- Do not implement provider registry, update checks, or downloads beyond what empty list scenarios require. +- Do not delete or replace Android RPC/JNI integration as part of this slice. +- Do not use JSONL as the durable product store. +- Do not apply the pre-sync stash wholesale. + +## Expected implementation order + +1. Make sure the getter submodule is on a working branch rather than detached HEAD. +2. Add Rust CLI test dependencies and the Cucumber runner skeleton. +3. Add the first Gherkin feature for `getter init` and observe it fail. +4. Implement minimal CLI parsing/output/storage init to pass `init`. +5. Add internal tests for SQLite init and output envelope serialization. +6. Add `app list` and `hub list` scenarios and implementation. +7. Add malformed legacy bundle failure scenario and minimal non-destructive report implementation. +8. Extend `just verify` with the new getter test command(s). +9. Run focused validation and report changed files, commands, failures, and residual risks. + +## Handoff requirements + +The worker must report: + +- changed files in the superproject and getter submodule; +- tests/features added; +- commands run with exit codes; +- whether SQLite storage is actually initialized; +- whether each BDD scenario passes; +- any blocked items or decisions needed before continuing. diff --git a/docs/refactor/phase-1b-getter-workspace-skeleton-plan.md b/docs/refactor/phase-1b-getter-workspace-skeleton-plan.md new file mode 100644 index 00000000..81cc2db2 --- /dev/null +++ b/docs/refactor/phase-1b-getter-workspace-skeleton-plan.md @@ -0,0 +1,48 @@ +# Phase 1b Plan: Getter Workspace Skeleton + +## Goal + +Create the Cargo workspace shape for the Getter rewrite without moving or rewriting existing behavior. Phase 1b is a transitional skeleton milestone, not completion of canonical Phase 1. + +## Scope + +- Add a Cargo workspace inside `core-getter/src/main/rust/getter`. +- Keep the existing root package named `getter` and keep its current CLI behavior in place. +- Add skeleton crates under `core-getter/src/main/rust/getter/crates/`: + - `getter-core` + - `getter-storage` + - `getter-providers` + - `getter-downloader` + - `getter-plugin-api` + - `getter-rpc` + - `getter-cli` + - `getter-ffi` +- Keep `api_proxy` compatible with `getter = { path = "../getter", features = ["rustls-platform-verifier-android"] }`. +- Resolve ADR 0007 status drift so the committed Phase 1a CLI contract is no longer treated as provisional. + +## Non-goals + +- No behavior/module moves from `core-getter/src/main/rust/getter/src/`. +- No change to supported CLI behavior. +- No claim that canonical Phase 1 is complete. +- No clippy `-D warnings` gate. +- No `cargo test --workspace` gate for this milestone. + +## Validation + +Phase 1b should validate the new workspace shape while preserving Phase 1a behavior: + +- `cargo metadata --manifest-path core-getter/src/main/rust/getter/Cargo.toml --no-deps --format-version 1` +- `cargo metadata --manifest-path core-getter/src/main/rust/api_proxy/Cargo.toml --no-deps --format-version 1` +- `cargo fmt --manifest-path core-getter/src/main/rust/getter/Cargo.toml --all --check` +- `cargo check --manifest-path core-getter/src/main/rust/getter/Cargo.toml --workspace --all-targets` +- `cargo check --manifest-path core-getter/src/main/rust/api_proxy/Cargo.toml` +- `just verify-workspace-skeleton` +- `just verify` +- `./gradlew --no-daemon projects` if not already covered by `just verify-workspace-skeleton` + +`just verify` is the single current verification entrypoint. It runs the scoped Phase 1a behavior/storage gates and the Phase 1b workspace skeleton checks without adding known-red broad getter tests, `cargo test --workspace`, or clippy `-D warnings`. + +## Notes + +This milestone creates the split-crate scaffold only. The root `getter` package remains the transitional monolith until a later approved behavior move. The `getter-core` Android/JNI guard is a structural metadata/text check for the new crate boundary; it prevents obvious dependency/reference drift but does not prove that product logic has already been isolated. diff --git a/docs/testing/bdd-plan.md b/docs/testing/bdd-plan.md new file mode 100644 index 00000000..8a010a6f --- /dev/null +++ b/docs/testing/bdd-plan.md @@ -0,0 +1,115 @@ +# BDD and TDD Plan + +Date: 2026-06-20 + +## Rule + +Every behavior-changing implementation starts with a failing automated test. + +Cucumber/Gherkin is required for user-facing behavior. The mandatory user-facing coverage surfaces are: + +1. UpgradeAll App workflows. +2. Getter CLI commands and contracts. +3. Migration success/failure/recovery behavior visible to users. + +Internal interfaces use traditional unit/integration/property tests unless they become supported user-facing contracts. + +## Why this split + +BDD is strongest at integration and acceptance behavior. It is not the best tool for every low-level algorithm test. Therefore: + +- Use Gherkin for observable workflows and supported command behavior. +- Use Rust/Kotlin/Dart native tests for internal logic, parsing, storage invariants, migration units, DTO serialization, and edge-case algorithms. +- Use widget/UI tests for rendering states and stable IDs. + +## Cucumber conventions + +Feature files should use product language from `CONTEXT.md`. + +Required tags: + +- `@app` for UpgradeAll App scenarios. +- `@getter-cli` for Getter CLI scenarios. +- `@migration` for legacy migration scenarios. +- `@smoke` for scenarios that must run in the fastest acceptance pass. +- `@regression` for scenarios created from bug fixes. + +Scenario naming should describe behavior, not implementation. Prefer: + +```gherkin +Scenario: User sees recoverable migration failure +``` + +not: + +```gherkin +Scenario: Rust importer returns error code 17 +``` + +## Planned suites + +### Getter CLI BDD + +Purpose: drive headless user-facing behavior before Flutter UI depends on it. + +Coverage examples: + +- Initialize a new data directory. +- Import a legacy bundle successfully. +- Report migration failure without destructive fallback. +- List apps in stable JSON output. +- Renew one app and report progress/events. +- Submit a download and report task state. +- Return documented non-zero exit codes for invalid input, network failure, and migration failure. + +Implementation direction: + +- Use Rust Cucumber for CLI behavior where practical. +- Step definitions invoke the built CLI binary and assert stdout/stderr/exit status and resulting state. +- Lower-level getter behavior stays covered by native Rust tests. + +### UpgradeAll App BDD + +Purpose: cover user-visible app behavior with stable route/action/state IDs. + +Coverage examples: + +- Fresh launch reaches the home route. +- Legacy migration success reaches the migrated app list. +- Legacy migration failure reaches recovery actions. +- User opens app list, app detail, and renew-all flow. +- User submits a download and sees task progress/failure/success state. +- Empty/loading/error/content states are addressable by stable IDs. + +Implementation direction: + +- Feature files are the acceptance source of truth. +- UI automation must use stable IDs, not localized text, wherever possible. +- The concrete runner can be implemented through Cucumber-compatible step definitions over Flutter integration tests and/or black-box automation, but the scenarios remain Gherkin. + +### Internal traditional tests + +Required for: + +- Version comparison. +- Provider parsing. +- Storage migrations and canonical IDs. +- Download orchestration edge cases. +- DTO serialization compatibility. +- Library API contract behavior that is not directly a CLI/App workflow. + +## Red-green-refactor loop + +1. Select a user-facing behavior or internal behavior. +2. Write the smallest failing test: + - Gherkin scenario for App/CLI behavior. + - Native unit/integration test for internal behavior. +3. Run the focused target and confirm failure for the expected reason. +4. Implement the smallest change. +5. Run focused tests until green. +6. Refactor with tests green. +7. Run `just verify` before handing off. + +## Phase 0 acceptance + +Phase 0 is complete when docs and verification skeleton exist. It does not need the full Cucumber runner wired yet, but it must prevent feature implementation from proceeding without a test plan and a failing-test entrypoint. From 952728730c1cca252f9549ee0aaa9b1b97e3d865 Mon Sep 17 00:00:00 2001 From: Xiangzhe Date: Mon, 22 Jun 2026 11:10:33 +0800 Subject: [PATCH 02/85] chore: add rewrite agent guardrails Record required docs, submodule ownership, architecture rules, and local pi ignore policy for coding agents. --- .gitignore | 1 + AGENTS.md | 73 ++++++++++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 74 insertions(+) create mode 100644 AGENTS.md diff --git a/.gitignore b/.gitignore index 0adc1232..0fb45b92 100644 --- a/.gitignore +++ b/.gitignore @@ -34,6 +34,7 @@ /android-studio/sdk out/ /tmp +/.pi/ /intellij workspace.xml *.versionsBackup diff --git a/AGENTS.md b/AGENTS.md new file mode 100644 index 00000000..5030ffdf --- /dev/null +++ b/AGENTS.md @@ -0,0 +1,73 @@ +# AGENTS.md — UpgradeAll rewrite coding agent bootstrap + +This repository is being rewritten toward a Flutter APP + Rust getter core + Lua package repository architecture. + +Before coding, every agent MUST read: + +1. `docs/README.md` +2. `docs/architecture/upgradeall-getter-rewrite-wiki.md` +3. `docs/architecture/adr/0001-app-centric-lua-package-repository-model.md` +4. `docs/architecture/adr/0002-getter-flutter-platform-boundary.md` +5. `docs/architecture/adr/0003-legacy-room-migration.md` +6. `docs/architecture/adr/0004-sqlite-main-db-and-cache-db.md` +7. `docs/architecture/adr/0005-lua-package-api.md` +8. `docs/architecture/adr/0006-package-centric-cli-command-contract.md` +9. `docs/app/flutter-ui-feature-parity-and-testing.md` + +## Core architecture rules + +- Rust getter owns all product/domain logic. +- Rust getter lives in the `core-getter/src/main/rust/getter` git submodule (`https://github.com/DUpdateSystem/getter`) so it remains independently reusable; implement getter CLI/core changes inside that submodule and update the superproject gitlink, do not vendor getter source into the UpgradeAll superproject. +- Flutter owns UI and platform adapter only. +- Do not reintroduce the old hub-app model. +- Use readable package ids such as `android/org.fdroid.fdroid`, not UUID primary ids. +- Lua package files return JSON-like tables; Rust validates/deserializes them. +- Backend state uses SQLite main DB plus separate cache DB. +- Package Lua source files live in repository folders. +- `local` is user-authored override repo. +- `local_autogen` is generated fallback repo. +- Do not add runtime UI customization/plugin framework unless a later ADR changes this. + +## Testing rules + +Use mixed BDD and TDD. + +TDD is for function/domain behavior: + +- Rust functions. +- repository resolution. +- Lua validation. +- migration mapping. +- cache invalidation. +- version comparison. + +BDD is for UI/integration behavior: + +- Flutter flows. +- migration UX. +- installed autogen confirmation. +- yellow network warning tag. +- update/download task flow. + +BDD scenarios are self-explaining documentation tests. Do not over-test with BDD; keep scenarios meaningful and user-visible. + +## Implementation discipline + +- Make small, reviewable changes. +- Update docs/ADR when behavior or architecture changes. +- Do not edit generated files manually. +- Do not silently drop migration fields; document dropped fields. +- Do not put Android-specific APIs into getter core. +- Do not put provider/update/version/storage logic into Flutter UI. +- If uncertain, add a small ADR or update the architecture wiki before coding. + +## Suggested first implementation order + +1. Create Rust workspace skeleton for getter. +2. Define package id, repository, and Lua validation structs. +3. Implement repository layout loader. +4. Add mlua evaluation returning JSON-like tables. +5. Implement Rust schema validation. +6. Implement main DB/cache DB skeleton. +7. Implement legacy migration mapping tests. +8. Build minimal Flutter shell only after getter core can be exercised by CLI. From ae0d72c220a6804f4bd48a40be1ad379cb392f76 Mon Sep 17 00:00:00 2001 From: Xiangzhe Date: Mon, 22 Jun 2026 11:10:33 +0800 Subject: [PATCH 03/85] feat(app): add Flutter shell scaffold Add the first Flutter UI/platform-adapter shell with stable route/action/state keys, fake getter adapter data, Android/Linux scaffolds, and widget tests. --- app_flutter/.gitignore | 43 ++ app_flutter/.metadata | 33 ++ app_flutter/README.md | 19 + app_flutter/analysis_options.yaml | 28 ++ app_flutter/android/.gitignore | 10 + app_flutter/android/app/build.gradle | 64 +++ .../android/app/src/main/AndroidManifest.xml | 33 ++ .../net/xzos/upgradeall/MainActivity.kt | 6 + .../res/drawable-v21/launch_background.xml | 12 + .../main/res/drawable/launch_background.xml | 12 + .../src/main/res/mipmap-hdpi/ic_launcher.png | Bin 0 -> 544 bytes .../src/main/res/mipmap-mdpi/ic_launcher.png | Bin 0 -> 442 bytes .../src/main/res/mipmap-xhdpi/ic_launcher.png | Bin 0 -> 721 bytes .../main/res/mipmap-xxhdpi/ic_launcher.png | Bin 0 -> 1031 bytes .../main/res/mipmap-xxxhdpi/ic_launcher.png | Bin 0 -> 1443 bytes .../app/src/main/res/values-night/styles.xml | 18 + .../app/src/main/res/values/styles.xml | 18 + .../app/src/profile/AndroidManifest.xml | 7 + app_flutter/android/build.gradle | 30 ++ app_flutter/android/gradle.properties | 4 + .../android/gradle/wrapper/gradle-wrapper.jar | Bin 0 -> 53636 bytes .../gradle/wrapper/gradle-wrapper.properties | 5 + app_flutter/android/gradlew | 160 +++++++ app_flutter/android/gradlew.bat | 90 ++++ app_flutter/android/settings.gradle | 29 ++ app_flutter/lib/main.dart | 411 ++++++++++++++++++ app_flutter/linux/.gitignore | 1 + app_flutter/linux/CMakeLists.txt | 145 ++++++ app_flutter/linux/flutter/CMakeLists.txt | 88 ++++ .../flutter/generated_plugin_registrant.cc | 11 + .../flutter/generated_plugin_registrant.h | 15 + .../linux/flutter/generated_plugins.cmake | 23 + app_flutter/linux/main.cc | 6 + app_flutter/linux/my_application.cc | 104 +++++ app_flutter/linux/my_application.h | 18 + app_flutter/pubspec.lock | 188 ++++++++ app_flutter/pubspec.yaml | 90 ++++ app_flutter/test/widget_test.dart | 83 ++++ 38 files changed, 1804 insertions(+) create mode 100644 app_flutter/.gitignore create mode 100644 app_flutter/.metadata create mode 100644 app_flutter/README.md create mode 100644 app_flutter/analysis_options.yaml create mode 100644 app_flutter/android/.gitignore create mode 100644 app_flutter/android/app/build.gradle create mode 100644 app_flutter/android/app/src/main/AndroidManifest.xml create mode 100644 app_flutter/android/app/src/main/kotlin/net/xzos/upgradeall/MainActivity.kt create mode 100644 app_flutter/android/app/src/main/res/drawable-v21/launch_background.xml create mode 100644 app_flutter/android/app/src/main/res/drawable/launch_background.xml create mode 100644 app_flutter/android/app/src/main/res/mipmap-hdpi/ic_launcher.png create mode 100644 app_flutter/android/app/src/main/res/mipmap-mdpi/ic_launcher.png create mode 100644 app_flutter/android/app/src/main/res/mipmap-xhdpi/ic_launcher.png create mode 100644 app_flutter/android/app/src/main/res/mipmap-xxhdpi/ic_launcher.png create mode 100644 app_flutter/android/app/src/main/res/mipmap-xxxhdpi/ic_launcher.png create mode 100644 app_flutter/android/app/src/main/res/values-night/styles.xml create mode 100644 app_flutter/android/app/src/main/res/values/styles.xml create mode 100644 app_flutter/android/app/src/profile/AndroidManifest.xml create mode 100644 app_flutter/android/build.gradle create mode 100644 app_flutter/android/gradle.properties create mode 100644 app_flutter/android/gradle/wrapper/gradle-wrapper.jar create mode 100644 app_flutter/android/gradle/wrapper/gradle-wrapper.properties create mode 100755 app_flutter/android/gradlew create mode 100644 app_flutter/android/gradlew.bat create mode 100644 app_flutter/android/settings.gradle create mode 100644 app_flutter/lib/main.dart create mode 100644 app_flutter/linux/.gitignore create mode 100644 app_flutter/linux/CMakeLists.txt create mode 100644 app_flutter/linux/flutter/CMakeLists.txt create mode 100644 app_flutter/linux/flutter/generated_plugin_registrant.cc create mode 100644 app_flutter/linux/flutter/generated_plugin_registrant.h create mode 100644 app_flutter/linux/flutter/generated_plugins.cmake create mode 100644 app_flutter/linux/main.cc create mode 100644 app_flutter/linux/my_application.cc create mode 100644 app_flutter/linux/my_application.h create mode 100644 app_flutter/pubspec.lock create mode 100644 app_flutter/pubspec.yaml create mode 100644 app_flutter/test/widget_test.dart diff --git a/app_flutter/.gitignore b/app_flutter/.gitignore new file mode 100644 index 00000000..29a3a501 --- /dev/null +++ b/app_flutter/.gitignore @@ -0,0 +1,43 @@ +# Miscellaneous +*.class +*.log +*.pyc +*.swp +.DS_Store +.atom/ +.buildlog/ +.history +.svn/ +migrate_working_dir/ + +# IntelliJ related +*.iml +*.ipr +*.iws +.idea/ + +# The .vscode folder contains launch configuration and tasks you configure in +# VS Code which you may wish to be included in version control, so this line +# is commented out by default. +#.vscode/ + +# Flutter/Dart/Pub related +**/doc/api/ +**/ios/Flutter/.last_build_id +.dart_tool/ +.flutter-plugins +.flutter-plugins-dependencies +.pub-cache/ +.pub/ +/build/ + +# Symbolication related +app.*.symbols + +# Obfuscation related +app.*.map.json + +# Android Studio will place build artifacts here +/android/app/debug +/android/app/profile +/android/app/release diff --git a/app_flutter/.metadata b/app_flutter/.metadata new file mode 100644 index 00000000..c1f9a6bf --- /dev/null +++ b/app_flutter/.metadata @@ -0,0 +1,33 @@ +# This file tracks properties of this Flutter project. +# Used by Flutter tool to assess capabilities and perform upgrades etc. +# +# This file should be version controlled and should not be manually edited. + +version: + revision: "78666c8dc57e9f7548ca9f8dd0740fbf0c658dc9" + channel: "stable" + +project_type: app + +# Tracks metadata for the flutter migrate command +migration: + platforms: + - platform: root + create_revision: 78666c8dc57e9f7548ca9f8dd0740fbf0c658dc9 + base_revision: 78666c8dc57e9f7548ca9f8dd0740fbf0c658dc9 + - platform: android + create_revision: 78666c8dc57e9f7548ca9f8dd0740fbf0c658dc9 + base_revision: 78666c8dc57e9f7548ca9f8dd0740fbf0c658dc9 + - platform: linux + create_revision: 78666c8dc57e9f7548ca9f8dd0740fbf0c658dc9 + base_revision: 78666c8dc57e9f7548ca9f8dd0740fbf0c658dc9 + + # User provided section + + # List of Local paths (relative to this file) that should be + # ignored by the migrate tool. + # + # Files that are not part of the templates will be ignored by default. + unmanaged_files: + - 'lib/main.dart' + - 'ios/Runner.xcodeproj/project.pbxproj' diff --git a/app_flutter/README.md b/app_flutter/README.md new file mode 100644 index 00000000..545a870f --- /dev/null +++ b/app_flutter/README.md @@ -0,0 +1,19 @@ +# UpgradeAll Flutter app + +This is the new Flutter shell for the UpgradeAll rewrite. It must remain a UI and platform adapter around the Rust getter core; product logic, repository resolution, storage, and migration behavior belong in getter. + +## Current slice + +- Android application identity: `net.xzos.upgradeall` +- Stable route/action/state keys for widget and future integration tests +- Placeholder routes for apps, repositories, downloads, logs, settings, and legacy migration +- Fake in-memory getter adapter until the Rust getter FFI/RPC binding is wired + +## Verification + +```bash +flutter analyze +flutter test +``` + +From the repository root, `just verify` also runs the Flutter analyzer and widget tests. diff --git a/app_flutter/analysis_options.yaml b/app_flutter/analysis_options.yaml new file mode 100644 index 00000000..0d290213 --- /dev/null +++ b/app_flutter/analysis_options.yaml @@ -0,0 +1,28 @@ +# This file configures the analyzer, which statically analyzes Dart code to +# check for errors, warnings, and lints. +# +# The issues identified by the analyzer are surfaced in the UI of Dart-enabled +# IDEs (https://dart.dev/tools#ides-and-editors). The analyzer can also be +# invoked from the command line by running `flutter analyze`. + +# The following line activates a set of recommended lints for Flutter apps, +# packages, and plugins designed to encourage good coding practices. +include: package:flutter_lints/flutter.yaml + +linter: + # The lint rules applied to this project can be customized in the + # section below to disable rules from the `package:flutter_lints/flutter.yaml` + # included above or to enable additional rules. A list of all available lints + # and their documentation is published at https://dart.dev/lints. + # + # Instead of disabling a lint rule for the entire project in the + # section below, it can also be suppressed for a single line of code + # or a specific dart file by using the `// ignore: name_of_lint` and + # `// ignore_for_file: name_of_lint` syntax on the line or in the file + # producing the lint. + rules: + # avoid_print: false # Uncomment to disable the `avoid_print` rule + # prefer_single_quotes: true # Uncomment to enable the `prefer_single_quotes` rule + +# Additional information about this file can be found at +# https://dart.dev/guides/language/analysis-options diff --git a/app_flutter/android/.gitignore b/app_flutter/android/.gitignore new file mode 100644 index 00000000..7760dbbd --- /dev/null +++ b/app_flutter/android/.gitignore @@ -0,0 +1,10 @@ +/.gradle +/captures/ +/local.properties +GeneratedPluginRegistrant.java + +# Remember to never publicly share your keystore. +# See https://flutter.dev/docs/deployment/android#reference-the-keystore-from-the-app +key.properties +**/*.keystore +**/*.jks diff --git a/app_flutter/android/app/build.gradle b/app_flutter/android/app/build.gradle new file mode 100644 index 00000000..76d0742d --- /dev/null +++ b/app_flutter/android/app/build.gradle @@ -0,0 +1,64 @@ +plugins { + id "com.android.application" + id "kotlin-android" + id "dev.flutter.flutter-gradle-plugin" +} + +def localProperties = new Properties() +def localPropertiesFile = rootProject.file('local.properties') +if (localPropertiesFile.exists()) { + localPropertiesFile.withReader('UTF-8') { reader -> + localProperties.load(reader) + } +} + +def flutterVersionCode = localProperties.getProperty('flutter.versionCode') +if (flutterVersionCode == null) { + flutterVersionCode = '105' +} + +def flutterVersionName = localProperties.getProperty('flutter.versionName') +if (flutterVersionName == null) { + flutterVersionName = '0.20.0-alpha.4' +} + +android { + namespace "net.xzos.upgradeall" + compileSdkVersion 36 + ndkVersion flutter.ndkVersion + + compileOptions { + sourceCompatibility JavaVersion.VERSION_17 + targetCompatibility JavaVersion.VERSION_17 + } + + kotlinOptions { + jvmTarget = '17' + } + + sourceSets { + main.java.srcDirs += 'src/main/kotlin' + } + + defaultConfig { + applicationId "net.xzos.upgradeall" + minSdkVersion 23 + targetSdkVersion 36 + versionCode flutterVersionCode.toInteger() + versionName flutterVersionName + } + + buildTypes { + release { + // TODO: Add your own signing config for the release build. + // Signing with the debug keys for now, so `flutter run --release` works. + signingConfig signingConfigs.debug + } + } +} + +flutter { + source '../..' +} + +dependencies {} diff --git a/app_flutter/android/app/src/main/AndroidManifest.xml b/app_flutter/android/app/src/main/AndroidManifest.xml new file mode 100644 index 00000000..4a2e8949 --- /dev/null +++ b/app_flutter/android/app/src/main/AndroidManifest.xml @@ -0,0 +1,33 @@ + + + + + + + + + + + + + + diff --git a/app_flutter/android/app/src/main/kotlin/net/xzos/upgradeall/MainActivity.kt b/app_flutter/android/app/src/main/kotlin/net/xzos/upgradeall/MainActivity.kt new file mode 100644 index 00000000..f3d65028 --- /dev/null +++ b/app_flutter/android/app/src/main/kotlin/net/xzos/upgradeall/MainActivity.kt @@ -0,0 +1,6 @@ +package net.xzos.upgradeall + +import io.flutter.embedding.android.FlutterActivity + +class MainActivity: FlutterActivity() { +} diff --git a/app_flutter/android/app/src/main/res/drawable-v21/launch_background.xml b/app_flutter/android/app/src/main/res/drawable-v21/launch_background.xml new file mode 100644 index 00000000..f74085f3 --- /dev/null +++ b/app_flutter/android/app/src/main/res/drawable-v21/launch_background.xml @@ -0,0 +1,12 @@ + + + + + + + + diff --git a/app_flutter/android/app/src/main/res/drawable/launch_background.xml b/app_flutter/android/app/src/main/res/drawable/launch_background.xml new file mode 100644 index 00000000..304732f8 --- /dev/null +++ b/app_flutter/android/app/src/main/res/drawable/launch_background.xml @@ -0,0 +1,12 @@ + + + + + + + + diff --git a/app_flutter/android/app/src/main/res/mipmap-hdpi/ic_launcher.png b/app_flutter/android/app/src/main/res/mipmap-hdpi/ic_launcher.png new file mode 100644 index 0000000000000000000000000000000000000000..db77bb4b7b0906d62b1847e87f15cdcacf6a4f29 GIT binary patch literal 544 zcmeAS@N?(olHy`uVBq!ia0vp^9w5xY3?!3`olAj~WQl7;NpOBzNqJ&XDuZK6ep0G} zXKrG8YEWuoN@d~6R2!h8bpbvhu0Wd6uZuB!w&u2PAxD2eNXD>P5D~Wn-+_Wa#27Xc zC?Zj|6r#X(-D3u$NCt}(Ms06KgJ4FxJVv{GM)!I~&n8Bnc94O7-Hd)cjDZswgC;Qs zO=b+9!WcT8F?0rF7!Uys2bs@gozCP?z~o%U|N3vA*22NaGQG zlg@K`O_XuxvZ&Ks^m&R!`&1=spLvfx7oGDKDwpwW`#iqdw@AL`7MR}m`rwr|mZgU`8P7SBkL78fFf!WnuYWm$5Z0 zNXhDbCv&49sM544K|?c)WrFfiZvCi9h0O)B3Pgg&ebxsLQ05GG~ AQ2+n{ literal 0 HcmV?d00001 diff --git a/app_flutter/android/app/src/main/res/mipmap-mdpi/ic_launcher.png b/app_flutter/android/app/src/main/res/mipmap-mdpi/ic_launcher.png new file mode 100644 index 0000000000000000000000000000000000000000..17987b79bb8a35cc66c3c1fd44f5a5526c1b78be GIT binary patch literal 442 zcmeAS@N?(olHy`uVBq!ia0vp^1|ZDA3?vioaBc-sk|nMYCBgY=CFO}lsSJ)O`AMk? zp1FzXsX?iUDV2pMQ*D5Xx&nMcT!A!W`0S9QKQy;}1Cl^CgaH=;G9cpY;r$Q>i*pfB zP2drbID<_#qf;rPZx^FqH)F_D#*k@@q03KywUtLX8Ua?`H+NMzkczFPK3lFz@i_kW%1NOn0|D2I9n9wzH8m|-tHjsw|9>@K=iMBhxvkv6m8Y-l zytQ?X=U+MF$@3 zt`~i=@j|6y)RWMK--}M|=T`o&^Ni>IoWKHEbBXz7?A@mgWoL>!*SXo`SZH-*HSdS+ yn*9;$7;m`l>wYBC5bq;=U}IMqLzqbYCidGC!)_gkIk_C@Uy!y&wkt5C($~2D>~)O*cj@FGjOCM)M>_ixfudOh)?xMu#Fs z#}Y=@YDTwOM)x{K_j*Q;dPdJ?Mz0n|pLRx{4n|)f>SXlmV)XB04CrSJn#dS5nK2lM zrZ9#~WelCp7&e13Y$jvaEXHskn$2V!!DN-nWS__6T*l;H&Fopn?A6HZ-6WRLFP=R` zqG+CE#d4|IbyAI+rJJ`&x9*T`+a=p|0O(+s{UBcyZdkhj=yS1>AirP+0R;mf2uMgM zC}@~JfByORAh4SyRgi&!(cja>F(l*O+nd+@4m$|6K6KDn_&uvCpV23&>G9HJp{xgg zoq1^2_p9@|WEo z*X_Uko@K)qYYv~>43eQGMdbiGbo>E~Q& zrYBH{QP^@Sti!`2)uG{irBBq@y*$B zi#&(U-*=fp74j)RyIw49+0MRPMRU)+a2r*PJ$L5roHt2$UjExCTZSbq%V!HeS7J$N zdG@vOZB4v_lF7Plrx+hxo7(fCV&}fHq)$ literal 0 HcmV?d00001 diff --git a/app_flutter/android/app/src/main/res/mipmap-xxhdpi/ic_launcher.png b/app_flutter/android/app/src/main/res/mipmap-xxhdpi/ic_launcher.png new file mode 100644 index 0000000000000000000000000000000000000000..d5f1c8d34e7a88e3f88bea192c3a370d44689c3c GIT binary patch literal 1031 zcmeAS@N?(olHy`uVBq!ia0vp^6F``Q8Ax83A=Cw=BuiW)N`mv#O3D+9QW+dm@{>{( zJaZG%Q-e|yQz{EjrrIztFa`(sgt!6~Yi|1%a`XoT0ojZ}lNrNjb9xjc(B0U1_% zz5^97Xt*%oq$rQy4?0GKNfJ44uvxI)gC`h-NZ|&0-7(qS@?b!5r36oQ}zyZrNO3 zMO=Or+<~>+A&uN&E!^Sl+>xE!QC-|oJv`ApDhqC^EWD|@=#J`=d#Xzxs4ah}w&Jnc z$|q_opQ^2TrnVZ0o~wh<3t%W&flvYGe#$xqda2bR_R zvPYgMcHgjZ5nSA^lJr%;<&0do;O^tDDh~=pIxA#coaCY>&N%M2^tq^U%3DB@ynvKo}b?yu-bFc-u0JHzced$sg7S3zqI(2 z#Km{dPr7I=pQ5>FuK#)QwK?Y`E`B?nP+}U)I#c1+FM*1kNvWG|a(TpksZQ3B@sD~b zpQ2)*V*TdwjFOtHvV|;OsiDqHi=6%)o4b!)x$)%9pGTsE z-JL={-Ffv+T87W(Xpooq<`r*VzWQcgBN$$`u}f>-ZQI1BB8ykN*=e4rIsJx9>z}*o zo~|9I;xof literal 0 HcmV?d00001 diff --git a/app_flutter/android/app/src/main/res/mipmap-xxxhdpi/ic_launcher.png b/app_flutter/android/app/src/main/res/mipmap-xxxhdpi/ic_launcher.png new file mode 100644 index 0000000000000000000000000000000000000000..4d6372eebdb28e45604e46eeda8dd24651419bc0 GIT binary patch literal 1443 zcmb`G{WsKk6vsdJTdFg%tJav9_E4vzrOaqkWF|A724Nly!y+?N9`YV6wZ}5(X(D_N(?!*n3`|_r0Hc?=PQw&*vnU?QTFY zB_MsH|!j$PP;I}?dppoE_gA(4uc!jV&0!l7_;&p2^pxNo>PEcNJv za5_RT$o2Mf!<+r?&EbHH6nMoTsDOa;mN(wv8RNsHpG)`^ymG-S5By8=l9iVXzN_eG%Xg2@Xeq76tTZ*dGh~Lo9vl;Zfs+W#BydUw zCkZ$o1LqWQO$FC9aKlLl*7x9^0q%0}$OMlp@Kk_jHXOjofdePND+j!A{q!8~Jn+s3 z?~~w@4?egS02}8NuulUA=L~QQfm;MzCGd)XhiftT;+zFO&JVyp2mBww?;QByS_1w! zrQlx%{^cMj0|Bo1FjwY@Q8?Hx0cIPF*@-ZRFpPc#bBw{5@tD(5%sClzIfl8WU~V#u zm5Q;_F!wa$BSpqhN>W@2De?TKWR*!ujY;Yylk_X5#~V!L*Gw~;$%4Q8~Mad z@`-kG?yb$a9cHIApZDVZ^U6Xkp<*4rU82O7%}0jjHlK{id@?-wpN*fCHXyXh(bLt* zPc}H-x0e4E&nQ>y%B-(EL=9}RyC%MyX=upHuFhAk&MLbsF0LP-q`XnH78@fT+pKPW zu72MW`|?8ht^tz$iC}ZwLp4tB;Q49K!QCF3@!iB1qOI=?w z7In!}F~ij(18UYUjnbmC!qKhPo%24?8U1x{7o(+?^Zu0Hx81|FuS?bJ0jgBhEMzf< zCgUq7r2OCB(`XkKcN-TL>u5y#dD6D!)5W?`O5)V^>jb)P)GBdy%t$uUMpf$SNV31$ zb||OojAbvMP?T@$h_ZiFLFVHDmbyMhJF|-_)HX3%m=CDI+ID$0^C>kzxprBW)hw(v zr!Gmda);ICoQyhV_oP5+C%?jcG8v+D@9f?Dk*!BxY}dazmrT@64UrP3hlslANK)bq z$67n83eh}OeW&SV@HG95P|bjfqJ7gw$e+`Hxo!4cx`jdK1bJ>YDSpGKLPZ^1cv$ek zIB?0S<#tX?SJCLWdMd{-ME?$hc7A$zBOdIJ)4!KcAwb=VMov)nK;9z>x~rfT1>dS+ zZ6#`2v@`jgbqq)P22H)Tx2CpmM^o1$B+xT6`(v%5xJ(?j#>Q$+rx_R|7TzDZe{J6q zG1*EcU%tE?!kO%^M;3aM6JN*LAKUVb^xz8-Pxo#jR5(-KBeLJvA@-gxNHx0M-ZJLl z;#JwQoh~9V?`UVo#}{6ka@II>++D@%KqGpMdlQ}?9E*wFcf5(#XQnP$Dk5~%iX^>f z%$y;?M0BLp{O3a(-4A?ewryHrrD%cx#Q^%KY1H zNre$ve+vceSLZcNY4U(RBX&)oZn*Py()h)XkE?PL$!bNb{N5FVI2Y%LKEm%yvpyTP z(1P?z~7YxD~Rf<(a@_y` literal 0 HcmV?d00001 diff --git a/app_flutter/android/app/src/main/res/values-night/styles.xml b/app_flutter/android/app/src/main/res/values-night/styles.xml new file mode 100644 index 00000000..06952be7 --- /dev/null +++ b/app_flutter/android/app/src/main/res/values-night/styles.xml @@ -0,0 +1,18 @@ + + + + + + + diff --git a/app_flutter/android/app/src/main/res/values/styles.xml b/app_flutter/android/app/src/main/res/values/styles.xml new file mode 100644 index 00000000..cb1ef880 --- /dev/null +++ b/app_flutter/android/app/src/main/res/values/styles.xml @@ -0,0 +1,18 @@ + + + + + + + diff --git a/app_flutter/android/app/src/profile/AndroidManifest.xml b/app_flutter/android/app/src/profile/AndroidManifest.xml new file mode 100644 index 00000000..399f6981 --- /dev/null +++ b/app_flutter/android/app/src/profile/AndroidManifest.xml @@ -0,0 +1,7 @@ + + + + diff --git a/app_flutter/android/build.gradle b/app_flutter/android/build.gradle new file mode 100644 index 00000000..802640d0 --- /dev/null +++ b/app_flutter/android/build.gradle @@ -0,0 +1,30 @@ +buildscript { + ext.kotlin_version = '1.9.22' + repositories { + google() + mavenCentral() + } + + dependencies { + classpath "org.jetbrains.kotlin:kotlin-gradle-plugin:$kotlin_version" + } +} + +allprojects { + repositories { + google() + mavenCentral() + } +} + +rootProject.buildDir = '../build' +subprojects { + project.buildDir = "${rootProject.buildDir}/${project.name}" +} +subprojects { + project.evaluationDependsOn(':app') +} + +tasks.register("clean", Delete) { + delete rootProject.buildDir +} diff --git a/app_flutter/android/gradle.properties b/app_flutter/android/gradle.properties new file mode 100644 index 00000000..5d46147a --- /dev/null +++ b/app_flutter/android/gradle.properties @@ -0,0 +1,4 @@ +org.gradle.jvmargs=-Xmx4G +android.useAndroidX=true +android.enableJetifier=true +android.suppressUnsupportedCompileSdk=36 diff --git a/app_flutter/android/gradle/wrapper/gradle-wrapper.jar b/app_flutter/android/gradle/wrapper/gradle-wrapper.jar new file mode 100644 index 0000000000000000000000000000000000000000..13372aef5e24af05341d49695ee84e5f9b594659 GIT binary patch literal 53636 zcmafaW0a=B^559DjdyHo$F^PVt zzd|cWgMz^T0YO0lQ8%TE1O06v|NZl~LH{LLQ58WtNjWhFP#}eWVO&eiP!jmdp!%24 z{&z-MK{-h=QDqf+S+Pgi=_wg$I{F28X*%lJ>A7Yl#$}fMhymMu?R9TEB?#6@|Q^e^AHhxcRL$z1gsc`-Q`3j+eYAd<4@z^{+?JM8bmu zSVlrVZ5-)SzLn&LU9GhXYG{{I+u(+6ES+tAtQUanYC0^6kWkks8cG;C&r1KGs)Cq}WZSd3k1c?lkzwLySimkP5z)T2Ox3pNs;PdQ=8JPDkT7#0L!cV? zzn${PZs;o7UjcCVd&DCDpFJvjI=h(KDmdByJuDYXQ|G@u4^Kf?7YkE67fWM97kj6F z973tGtv!k$k{<>jd~D&c(x5hVbJa`bILdy(00%lY5}HZ2N>)a|))3UZ&fUa5@uB`H z+LrYm@~t?g`9~@dFzW5l>=p0hG%rv0>(S}jEzqQg6-jImG%Pr%HPtqIV_Ym6yRydW z4L+)NhcyYp*g#vLH{1lK-hQQSScfvNiNx|?nSn-?cc8}-9~Z_0oxlr~(b^EiD`Mx< zlOLK)MH?nl4dD|hx!jBCIku-lI(&v~bCU#!L7d0{)h z;k4y^X+=#XarKzK*)lv0d6?kE1< zmCG^yDYrSwrKIn04tG)>>10%+ zEKzs$S*Zrl+GeE55f)QjY$ zD5hi~J17k;4VSF_`{lPFwf^Qroqg%kqM+Pdn%h#oOPIsOIwu?JR717atg~!)*CgXk zERAW?c}(66rnI+LqM^l7BW|9dH~5g1(_w$;+AAzSYlqop*=u5}=g^e0xjlWy0cUIT7{Fs2Xqx*8% zW71JB%hk%aV-wjNE0*$;E-S9hRx5|`L2JXxz4TX3nf8fMAn|523ssV;2&145zh{$V z#4lt)vL2%DCZUgDSq>)ei2I`*aeNXHXL1TB zC8I4!uq=YYVjAdcCjcf4XgK2_$y5mgsCdcn2U!VPljXHco>+%`)6W=gzJk0$e%m$xWUCs&Ju-nUJjyQ04QF_moED2(y6q4l+~fo845xm zE5Esx?~o#$;rzpCUk2^2$c3EBRNY?wO(F3Pb+<;qfq;JhMFuSYSxiMejBQ+l8(C-- zz?Xufw@7{qvh$;QM0*9tiO$nW(L>83egxc=1@=9Z3)G^+*JX-z92F((wYiK>f;6 zkc&L6k4Ua~FFp`x7EF;ef{hb*n8kx#LU|6{5n=A55R4Ik#sX{-nuQ}m7e<{pXq~8#$`~6| zi{+MIgsBRR-o{>)CE8t0Bq$|SF`M0$$7-{JqwFI1)M^!GMwq5RAWMP!o6G~%EG>$S zYDS?ux;VHhRSm*b^^JukYPVb?t0O%^&s(E7Rb#TnsWGS2#FdTRj_SR~YGjkaRFDI=d)+bw$rD;_!7&P2WEmn zIqdERAbL&7`iA^d?8thJ{(=)v>DgTF7rK-rck({PpYY$7uNY$9-Z< ze4=??I#p;$*+-Tm!q8z}k^%-gTm59^3$*ByyroqUe02Dne4?Fc%JlO>*f9Zj{++!^ zBz0FxuS&7X52o6-^CYq>jkXa?EEIfh?xdBPAkgpWpb9Tam^SXoFb3IRfLwanWfskJ zIbfU-rJ1zPmOV)|%;&NSWIEbbwj}5DIuN}!m7v4($I{Rh@<~-sK{fT|Wh?<|;)-Z; zwP{t@{uTsmnO@5ZY82lzwl4jeZ*zsZ7w%a+VtQXkigW$zN$QZnKw4F`RG`=@eWowO zFJ6RC4e>Y7Nu*J?E1*4*U0x^>GK$>O1S~gkA)`wU2isq^0nDb`);Q(FY<8V6^2R%= zDY}j+?mSj{bz2>F;^6S=OLqiHBy~7h4VVscgR#GILP!zkn68S^c04ZL3e$lnSU_(F zZm3e`1~?eu1>ys#R6>Gu$`rWZJG&#dsZ?^)4)v(?{NPt+_^Ak>Ap6828Cv^B84fa4 z_`l$0SSqkBU}`f*H#<14a)khT1Z5Z8;=ga^45{l8y*m|3Z60vgb^3TnuUKaa+zP;m zS`za@C#Y;-LOm&pW||G!wzr+}T~Q9v4U4ufu*fLJC=PajN?zN=?v^8TY}wrEeUygdgwr z7szml+(Bar;w*c^!5txLGKWZftqbZP`o;Kr1)zI}0Kb8yr?p6ZivtYL_KA<+9)XFE z=pLS5U&476PKY2aKEZh}%|Vb%!us(^qf)bKdF7x_v|Qz8lO7Ro>;#mxG0gqMaTudL zi2W!_#3@INslT}1DFJ`TsPvRBBGsODklX0`p-M6Mrgn~6&fF`kdj4K0I$<2Hp(YIA z)fFdgR&=qTl#sEFj6IHzEr1sYM6 zNfi!V!biByA&vAnZd;e_UfGg_={}Tj0MRt3SG%BQYnX$jndLG6>ssgIV{T3#=;RI% zE}b!9z#fek19#&nFgC->@!IJ*Fe8K$ZOLmg|6(g}ccsSBpc`)3;Ar8;3_k`FQ#N9&1tm>c|2mzG!!uWvelm zJj|oDZ6-m(^|dn3em(BF&3n12=hdtlb@%!vGuL*h`CXF?^=IHU%Q8;g8vABm=U!vX zT%Ma6gpKQC2c;@wH+A{)q+?dAuhetSxBDui+Z;S~6%oQq*IwSMu-UhMDy{pP z-#GB-a0`0+cJ%dZ7v0)3zfW$eV>w*mgU4Cma{P$DY3|w364n$B%cf()fZ;`VIiK_O zQ|q|(55+F$H(?opzr%r)BJLy6M&7Oq8KCsh`pA5^ohB@CDlMKoDVo5gO&{0k)R0b(UOfd>-(GZGeF}y?QI_T+GzdY$G{l!l% zHyToqa-x&X4;^(-56Lg$?(KYkgJn9W=w##)&CECqIxLe@+)2RhO*-Inpb7zd8txFG6mY8E?N8JP!kRt_7-&X{5P?$LAbafb$+hkA*_MfarZxf zXLpXmndnV3ubbXe*SYsx=eeuBKcDZI0bg&LL-a8f9>T(?VyrpC6;T{)Z{&|D5a`Aa zjP&lP)D)^YYWHbjYB6ArVs+4xvrUd1@f;;>*l zZH``*BxW+>Dd$be{`<&GN(w+m3B?~3Jjz}gB8^|!>pyZo;#0SOqWem%xeltYZ}KxOp&dS=bg|4 zY-^F~fv8v}u<7kvaZH`M$fBeltAglH@-SQres30fHC%9spF8Ld%4mjZJDeGNJR8+* zl&3Yo$|JYr2zi9deF2jzEC) zl+?io*GUGRp;^z+4?8gOFA>n;h%TJC#-st7#r&-JVeFM57P7rn{&k*z@+Y5 zc2sui8(gFATezp|Te|1-Q*e|Xi+__8bh$>%3|xNc2kAwTM!;;|KF6cS)X3SaO8^z8 zs5jV(s(4_NhWBSSJ}qUzjuYMKlkjbJS!7_)wwVsK^qDzHx1u*sC@C1ERqC#l%a zk>z>m@sZK{#GmsB_NkEM$$q@kBrgq%=NRBhL#hjDQHrI7(XPgFvP&~ZBJ@r58nLme zK4tD}Nz6xrbvbD6DaDC9E_82T{(WRQBpFc+Zb&W~jHf1MiBEqd57}Tpo8tOXj@LcF zwN8L-s}UO8%6piEtTrj@4bLH!mGpl5mH(UJR1r9bBOrSt0tSJDQ9oIjcW#elyMAxl7W^V(>8M~ss0^>OKvf{&oUG@uW{f^PtV#JDOx^APQKm& z{*Ysrz&ugt4PBUX@KERQbycxP%D+ApR%6jCx7%1RG2YpIa0~tqS6Xw6k#UN$b`^l6d$!I z*>%#Eg=n#VqWnW~MurJLK|hOQPTSy7G@29g@|g;mXC%MF1O7IAS8J^Q6D&Ra!h^+L&(IBYg2WWzZjT-rUsJMFh@E)g)YPW_)W9GF3 zMZz4RK;qcjpnat&J;|MShuPc4qAc)A| zVB?h~3TX+k#Cmry90=kdDoPYbhzs#z96}#M=Q0nC{`s{3ZLU)c(mqQQX;l~1$nf^c zFRQ~}0_!cM2;Pr6q_(>VqoW0;9=ZW)KSgV-c_-XdzEapeLySavTs5-PBsl-n3l;1jD z9^$^xR_QKDUYoeqva|O-+8@+e??(pRg@V|=WtkY!_IwTN~ z9Rd&##eWt_1w$7LL1$-ETciKFyHnNPjd9hHzgJh$J(D@3oYz}}jVNPjH!viX0g|Y9 zDD`Zjd6+o+dbAbUA( zEqA9mSoX5p|9sDVaRBFx_8)Ra4HD#xDB(fa4O8_J2`h#j17tSZOd3%}q8*176Y#ak zC?V8Ol<*X{Q?9j{Ys4Bc#sq!H;^HU$&F_`q2%`^=9DP9YV-A!ZeQ@#p=#ArloIgUH%Y-s>G!%V3aoXaY=f<UBrJTN+*8_lMX$yC=Vq+ zrjLn-pO%+VIvb~>k%`$^aJ1SevcPUo;V{CUqF>>+$c(MXxU12mxqyFAP>ki{5#;Q0 zx7Hh2zZdZzoxPY^YqI*Vgr)ip0xnpQJ+~R*UyFi9RbFd?<_l8GH@}gGmdB)~V7vHg z>Cjy78TQTDwh~+$u$|K3if-^4uY^|JQ+rLVX=u7~bLY29{lr>jWV7QCO5D0I>_1?; zx>*PxE4|wC?#;!#cK|6ivMzJ({k3bT_L3dHY#h7M!ChyTT`P#%3b=k}P(;QYTdrbe z+e{f@we?3$66%02q8p3;^th;9@y2vqt@LRz!DO(WMIk?#Pba85D!n=Ao$5NW0QVgS zoW)fa45>RkjU?H2SZ^#``zs6dG@QWj;MO4k6tIp8ZPminF`rY31dzv^e-3W`ZgN#7 z)N^%Rx?jX&?!5v`hb0-$22Fl&UBV?~cV*{hPG6%ml{k;m+a-D^XOF6DxPd$3;2VVY zT)E%m#ZrF=D=84$l}71DK3Vq^?N4``cdWn3 zqV=mX1(s`eCCj~#Nw4XMGW9tK>$?=cd$ule0Ir8UYzhi?%_u0S?c&j7)-~4LdolkgP^CUeE<2`3m)I^b ztV`K0k$OS^-GK0M0cNTLR22Y_eeT{<;G(+51Xx}b6f!kD&E4; z&Op8;?O<4D$t8PB4#=cWV9Q*i4U+8Bjlj!y4`j)^RNU#<5La6|fa4wLD!b6?RrBsF z@R8Nc^aO8ty7qzlOLRL|RUC-Bt-9>-g`2;@jfNhWAYciF{df9$n#a~28+x~@x0IWM zld=J%YjoKm%6Ea>iF){z#|~fo_w#=&&HRogJmXJDjCp&##oVvMn9iB~gyBlNO3B5f zXgp_1I~^`A0z_~oAa_YBbNZbDsnxLTy0@kkH!=(xt8|{$y<+|(wSZW7@)#|fs_?gU5-o%vpsQPRjIxq;AED^oG%4S%`WR}2(*!84Pe8Jw(snJ zq~#T7+m|w#acH1o%e<+f;!C|*&_!lL*^zRS`;E}AHh%cj1yR&3Grv&0I9k9v0*w8^ zXHEyRyCB`pDBRAxl;ockOh6$|7i$kzCBW$}wGUc|2bo3`x*7>B@eI=-7lKvI)P=gQ zf_GuA+36kQb$&{ZH)6o^x}wS}S^d&Xmftj%nIU=>&j@0?z8V3PLb1JXgHLq)^cTvB zFO6(yj1fl1Bap^}?hh<>j?Jv>RJdK{YpGjHxnY%d8x>A{k+(18J|R}%mAqq9Uzm8^Us#Ir_q^w9-S?W07YRD`w%D(n;|8N%_^RO`zp4 z@`zMAs>*x0keyE)$dJ8hR37_&MsSUMlGC*=7|wUehhKO)C85qoU}j>VVklO^TxK?! zO!RG~y4lv#W=Jr%B#sqc;HjhN={wx761vA3_$S>{j+r?{5=n3le|WLJ(2y_r>{)F_ z=v8Eo&xFR~wkw5v-{+9^JQukxf8*CXDWX*ZzjPVDc>S72uxAcY+(jtg3ns_5R zRYl2pz`B)h+e=|7SfiAAP;A zk0tR)3u1qy0{+?bQOa17SpBRZ5LRHz(TQ@L0%n5xJ21ri>^X420II1?5^FN3&bV?( zCeA)d9!3FAhep;p3?wLPs`>b5Cd}N!;}y`Hq3ppDs0+><{2ey0yq8o7m-4|oaMsWf zsLrG*aMh91drd-_QdX6t&I}t2!`-7$DCR`W2yoV%bcugue)@!SXM}fJOfG(bQQh++ zjAtF~zO#pFz})d8h)1=uhigDuFy`n*sbxZ$BA^Bt=Jdm}_KB6sCvY(T!MQnqO;TJs zVD{*F(FW=+v`6t^6{z<3-fx#|Ze~#h+ymBL^^GKS%Ve<)sP^<4*y_Y${06eD zH_n?Ani5Gs4&1z)UCL-uBvq(8)i!E@T_*0Sp5{Ddlpgke^_$gukJc_f9e=0Rfpta@ ze5~~aJBNK&OJSw!(rDRAHV0d+eW#1?PFbr==uG-$_fu8`!DWqQD~ef-Gx*ZmZx33_ zb0+I(0!hIK>r9_S5A*UwgRBKSd6!ieiYJHRigU@cogJ~FvJHY^DSysg)ac=7#wDBf zNLl!E$AiUMZC%%i5@g$WsN+sMSoUADKZ}-Pb`{7{S>3U%ry~?GVX!BDar2dJHLY|g zTJRo#Bs|u#8ke<3ohL2EFI*n6adobnYG?F3-#7eZZQO{#rmM8*PFycBR^UZKJWr(a z8cex$DPOx_PL^TO<%+f^L6#tdB8S^y#+fb|acQfD(9WgA+cb15L+LUdHKv)wE6={i zX^iY3N#U7QahohDP{g`IHS?D00eJC9DIx0V&nq!1T* z4$Bb?trvEG9JixrrNRKcjX)?KWR#Y(dh#re_<y*=5!J+-Wwb*D>jKXgr5L8_b6pvSAn3RIvI5oj!XF^m?otNA=t^dg z#V=L0@W)n?4Y@}49}YxQS=v5GsIF3%Cp#fFYm0Bm<}ey& zOfWB^vS8ye?n;%yD%NF8DvOpZqlB++#4KnUj>3%*S(c#yACIU>TyBG!GQl7{b8j#V z;lS})mrRtT!IRh2B-*T58%9;!X}W^mg;K&fb7?2#JH>JpCZV5jbDfOgOlc@wNLfHN z8O92GeBRjCP6Q9^Euw-*i&Wu=$>$;8Cktx52b{&Y^Ise-R1gTKRB9m0*Gze>$k?$N zua_0Hmbcj8qQy{ZyJ%`6v6F+yBGm>chZxCGpeL@os+v&5LON7;$tb~MQAbSZKG$k z8w`Mzn=cX4Hf~09q8_|3C7KnoM1^ZGU}#=vn1?1^Kc-eWv4x^T<|i9bCu;+lTQKr- zRwbRK!&XrWRoO7Kw!$zNQb#cJ1`iugR(f_vgmu!O)6tFH-0fOSBk6$^y+R07&&B!(V#ZV)CX42( zTC(jF&b@xu40fyb1=_2;Q|uPso&Gv9OSM1HR{iGPi@JUvmYM;rkv#JiJZ5-EFA%Lu zf;wAmbyclUM*D7>^nPatbGr%2aR5j55qSR$hR`c?d+z z`qko8Yn%vg)p=H`1o?=b9K0%Blx62gSy)q*8jWPyFmtA2a+E??&P~mT@cBdCsvFw4 zg{xaEyVZ|laq!sqN}mWq^*89$e6%sb6Thof;ml_G#Q6_0-zwf80?O}D0;La25A0C+ z3)w-xesp6?LlzF4V%yA9Ryl_Kq*wMk4eu&)Tqe#tmQJtwq`gI^7FXpToum5HP3@;N zpe4Y!wv5uMHUu`zbdtLys5)(l^C(hFKJ(T)z*PC>7f6ZRR1C#ao;R&_8&&a3)JLh* zOFKz5#F)hJqVAvcR#1)*AWPGmlEKw$sQd)YWdAs_W-ojA?Lm#wCd}uF0^X=?AA#ki zWG6oDQZJ5Tvifdz4xKWfK&_s`V*bM7SVc^=w7-m}jW6U1lQEv_JsW6W(| zkKf>qn^G!EWn~|7{G-&t0C6C%4)N{WRK_PM>4sW8^dDkFM|p&*aBuN%fg(I z^M-49vnMd%=04N95VO+?d#el>LEo^tvnQsMop70lNqq@%cTlht?e+B5L1L9R4R(_6 z!3dCLeGXb+_LiACNiqa^nOELJj%q&F^S+XbmdP}`KAep%TDop{Pz;UDc#P&LtMPgH zy+)P1jdgZQUuwLhV<89V{3*=Iu?u#v;v)LtxoOwV(}0UD@$NCzd=id{UuDdedeEp| z`%Q|Y<6T?kI)P|8c!K0Za&jxPhMSS!T`wlQNlkE(2B*>m{D#`hYYD>cgvsKrlcOcs7;SnVCeBiK6Wfho@*Ym9 zr0zNfrr}0%aOkHd)d%V^OFMI~MJp+Vg-^1HPru3Wvac@-QjLX9Dx}FL(l>Z;CkSvC zOR1MK%T1Edv2(b9$ttz!E7{x4{+uSVGz`uH&)gG`$)Vv0^E#b&JSZp#V)b6~$RWwe zzC3FzI`&`EDK@aKfeqQ4M(IEzDd~DS>GB$~ip2n!S%6sR&7QQ*=Mr(v*v-&07CO%# zMBTaD8-EgW#C6qFPPG1Ph^|0AFs;I+s|+A@WU}%@WbPI$S0+qFR^$gim+Fejs2f!$ z@Xdlb_K1BI;iiOUj`j+gOD%mjq^S~J0cZZwuqfzNH9}|(vvI6VO+9ZDA_(=EAo;( zKKzm`k!s!_sYCGOm)93Skaz+GF7eY@Ra8J$C)`X)`aPKym?7D^SI}Mnef4C@SgIEB z>nONSFl$qd;0gSZhNcRlq9VVHPkbakHlZ1gJ1y9W+@!V$TLpdsbKR-VwZrsSM^wLr zL9ob&JG)QDTaf&R^cnm5T5#*J3(pSpjM5~S1 z@V#E2syvK6wb?&h?{E)CoI~9uA(hST7hx4_6M(7!|BW3TR_9Q zLS{+uPoNgw(aK^?=1rFcDO?xPEk5Sm=|pW%-G2O>YWS^(RT)5EQ2GSl75`b}vRcD2 z|HX(x0#Qv+07*O|vMIV(0?KGjOny#Wa~C8Q(kF^IR8u|hyyfwD&>4lW=)Pa311caC zUk3aLCkAFkcidp@C%vNVLNUa#1ZnA~ZCLrLNp1b8(ndgB(0zy{Mw2M@QXXC{hTxr7 zbipeHI-U$#Kr>H4}+cu$#2fG6DgyWgq{O#8aa)4PoJ^;1z7b6t&zt zPei^>F1%8pcB#1`z`?f0EAe8A2C|}TRhzs*-vN^jf(XNoPN!tONWG=abD^=Lm9D?4 zbq4b(in{eZehKC0lF}`*7CTzAvu(K!eAwDNC#MlL2~&gyFKkhMIF=32gMFLvKsbLY z1d$)VSzc^K&!k#2Q?(f>pXn){C+g?vhQ0ijV^Z}p5#BGrGb%6n>IH-)SA$O)*z3lJ z1rtFlovL`cC*RaVG!p!4qMB+-f5j^1)ALf4Z;2X&ul&L!?`9Vdp@d(%(>O=7ZBV;l z?bbmyPen>!P{TJhSYPmLs759b1Ni1`d$0?&>OhxxqaU|}-?Z2c+}jgZ&vCSaCivx| z-&1gw2Lr<;U-_xzlg}Fa_3NE?o}R-ZRX->__}L$%2ySyiPegbnM{UuADqwDR{C2oS zPuo88%DNfl4xBogn((9j{;*YGE0>2YoL?LrH=o^SaAcgO39Ew|vZ0tyOXb509#6{7 z0<}CptRX5(Z4*}8CqCgpT@HY3Q)CvRz_YE;nf6ZFwEje^;Hkj0b1ESI*8Z@(RQrW4 z35D5;S73>-W$S@|+M~A(vYvX(yvLN(35THo!yT=vw@d(=q8m+sJyZMB7T&>QJ=jkwQVQ07*Am^T980rldC)j}}zf!gq7_z4dZ zHwHB94%D-EB<-^W@9;u|(=X33c(G>q;Tfq1F~-Lltp|+uwVzg?e$M96ndY{Lcou%w zWRkjeE`G*i)Bm*|_7bi+=MPm8by_};`=pG!DSGBP6y}zvV^+#BYx{<>p0DO{j@)(S zxcE`o+gZf8EPv1g3E1c3LIbw+`rO3N+Auz}vn~)cCm^DlEi#|Az$b z2}Pqf#=rxd!W*6HijC|u-4b~jtuQS>7uu{>wm)PY6^S5eo=?M>;tK`=DKXuArZvaU zHk(G??qjKYS9G6Du)#fn+ob=}C1Hj9d?V$_=J41ljM$CaA^xh^XrV-jzi7TR-{{9V zZZI0;aQ9YNEc`q=Xvz;@q$eqL<}+L(>HR$JA4mB6~g*YRSnpo zTofY;u7F~{1Pl=pdsDQx8Gg#|@BdoWo~J~j%DfVlT~JaC)he>he6`C`&@@#?;e(9( zgKcmoidHU$;pi{;VXyE~4>0{kJ>K3Uy6`s*1S--*mM&NY)*eOyy!7?9&osK*AQ~vi z{4qIQs)s#eN6j&0S()cD&aCtV;r>ykvAzd4O-fG^4Bmx2A2U7-kZR5{Qp-R^i4H2yfwC7?9(r3=?oH(~JR4=QMls>auMv*>^^!$}{}R z;#(gP+O;kn4G|totqZGdB~`9yzShMze{+$$?9%LJi>4YIsaPMwiJ{`gocu0U}$Q$vI5oeyKrgzz>!gI+XFt!#n z7vs9Pn`{{5w-@}FJZn?!%EQV!PdA3hw%Xa2#-;X4*B4?`WM;4@bj`R-yoAs_t4!!` zEaY5OrYi`3u3rXdY$2jZdZvufgFwVna?!>#t#DKAD2;U zqpqktqJ)8EPY*w~yj7r~#bNk|PDM>ZS?5F7T5aPFVZrqeX~5_1*zTQ%;xUHe#li?s zJ*5XZVERVfRjwX^s=0<%nXhULK+MdibMjzt%J7#fuh?NXyJ^pqpfG$PFmG!h*opyi zmMONjJY#%dkdRHm$l!DLeBm#_0YCq|x17c1fYJ#5YMpsjrFKyU=y>g5QcTgbDm28X zYL1RK)sn1@XtkGR;tNb}(kg#9L=jNSbJizqAgV-TtK2#?LZXrCIz({ zO^R|`ZDu(d@E7vE}df5`a zNIQRp&mDFbgyDKtyl@J|GcR9!h+_a$za$fnO5Ai9{)d7m@?@qk(RjHwXD}JbKRn|u z=Hy^z2vZ<1Mf{5ihhi9Y9GEG74Wvka;%G61WB*y7;&L>k99;IEH;d8-IR6KV{~(LZ zN7@V~f)+yg7&K~uLvG9MAY+{o+|JX?yf7h9FT%7ZrW7!RekjwgAA4jU$U#>_!ZC|c zA9%tc9nq|>2N1rg9uw-Qc89V}I5Y`vuJ(y`Ibc_?D>lPF0>d_mB@~pU`~)uWP48cT@fTxkWSw{aR!`K{v)v zpN?vQZZNPgs3ki9h{An4&Cap-c5sJ!LVLtRd=GOZ^bUpyDZHm6T|t#218}ZA zx*=~9PO>5IGaBD^XX-_2t7?7@WN7VfI^^#Csdz9&{1r z9y<9R?BT~-V8+W3kzWWQ^)ZSI+R zt^Lg`iN$Z~a27)sC_03jrD-%@{ArCPY#Pc*u|j7rE%}jF$LvO4vyvAw3bdL_mg&ei zXys_i=Q!UoF^Xp6^2h5o&%cQ@@)$J4l`AG09G6Uj<~A~!xG>KjKSyTX)zH*EdHMK0 zo;AV-D+bqWhtD-!^+`$*P0B`HokilLd1EuuwhJ?%3wJ~VXIjIE3tj653PExvIVhE& zFMYsI(OX-Q&W$}9gad^PUGuKElCvXxU_s*kx%dH)Bi&$*Q(+9j>(Q>7K1A#|8 zY!G!p0kW29rP*BNHe_wH49bF{K7tymi}Q!Vc_Ox2XjwtpM2SYo7n>?_sB=$c8O5^? z6as!fE9B48FcE`(ruNXP%rAZlDXrFTC7^aoXEX41k)tIq)6kJ*(sr$xVqsh_m3^?? zOR#{GJIr6E0Sz{-( z-R?4asj|!GVl0SEagNH-t|{s06Q3eG{kZOoPHL&Hs0gUkPc&SMY=&{C0&HDI)EHx9 zm#ySWluxwp+b~+K#VG%21%F65tyrt9RTPR$eG0afer6D`M zTW=y!@y6yi#I5V#!I|8IqU=@IfZo!@9*P+f{yLxGu$1MZ%xRY(gRQ2qH@9eMK0`Z> zgO`4DHfFEN8@m@dxYuljsmVv}c4SID+8{kr>d_dLzF$g>urGy9g+=`xAfTkVtz56G zrKNsP$yrDyP=kIqPN9~rVmC-wH672NF7xU>~j5M06Xr&>UJBmOV z%7Ie2d=K=u^D`~i3(U7x?n=h!SCSD1`aFe-sY<*oh+=;B>UVFBOHsF=(Xr(Cai{dL z4S7Y>PHdfG9Iav5FtKzx&UCgg)|DRLvq7!0*9VD`e6``Pgc z1O!qSaNeBBZnDXClh(Dq@XAk?Bd6+_rsFt`5(E+V2c)!Mx4X z47X+QCB4B7$B=Fw1Z1vnHg;x9oDV1YQJAR6Q3}_}BXTFg$A$E!oGG%`Rc()-Ysc%w za(yEn0fw~AaEFr}Rxi;if?Gv)&g~21UzXU9osI9{rNfH$gPTTk#^B|irEc<8W+|9$ zc~R${X2)N!npz1DFVa%nEW)cgPq`MSs)_I*Xwo<+ZK-2^hD(Mc8rF1+2v7&qV;5SET-ygMLNFsb~#u+LpD$uLR1o!ha67gPV5Q{v#PZK5X zUT4aZ{o}&*q7rs)v%*fDTl%}VFX?Oi{i+oKVUBqbi8w#FI%_5;6`?(yc&(Fed4Quy8xsswG+o&R zO1#lUiA%!}61s3jR7;+iO$;1YN;_*yUnJK=$PT_}Q%&0T@2i$ zwGC@ZE^A62YeOS9DU9me5#`(wv24fK=C)N$>!!6V#6rX3xiHehfdvwWJ>_fwz9l)o`Vw9yi z0p5BgvIM5o_ zgo-xaAkS_mya8FXo1Ke4;U*7TGSfm0!fb4{E5Ar8T3p!Z@4;FYT8m=d`C@4-LM121 z?6W@9d@52vxUT-6K_;1!SE%FZHcm0U$SsC%QB zxkTrfH;#Y7OYPy!nt|k^Lgz}uYudos9wI^8x>Y{fTzv9gfTVXN2xH`;Er=rTeAO1x znaaJOR-I)qwD4z%&dDjY)@s`LLSd#FoD!?NY~9#wQRTHpD7Vyyq?tKUHKv6^VE93U zt_&ePH+LM-+9w-_9rvc|>B!oT>_L59nipM-@ITy|x=P%Ezu@Y?N!?jpwP%lm;0V5p z?-$)m84(|7vxV<6f%rK3!(R7>^!EuvA&j@jdTI+5S1E{(a*wvsV}_)HDR&8iuc#>+ zMr^2z*@GTnfDW-QS38OJPR3h6U&mA;vA6Pr)MoT7%NvA`%a&JPi|K8NP$b1QY#WdMt8-CDA zyL0UXNpZ?x=tj~LeM0wk<0Dlvn$rtjd$36`+mlf6;Q}K2{%?%EQ+#FJy6v5cS+Q-~ ztk||Iwr$(CZQHi38QZF;lFFBNt+mg2*V_AhzkM<8#>E_S^xj8%T5tXTytD6f)vePG z^B0Ne-*6Pqg+rVW?%FGHLhl^ycQM-dhNCr)tGC|XyES*NK%*4AnZ!V+Zu?x zV2a82fs8?o?X} zjC1`&uo1Ti*gaP@E43NageV^$Xue3%es2pOrLdgznZ!_a{*`tfA+vnUv;^Ebi3cc$?-kh76PqA zMpL!y(V=4BGPQSU)78q~N}_@xY5S>BavY3Sez-+%b*m0v*tOz6zub9%*~%-B)lb}t zy1UgzupFgf?XyMa+j}Yu>102tP$^S9f7;b7N&8?_lYG$okIC`h2QCT_)HxG1V4Uv{xdA4k3-FVY)d}`cmkePsLScG&~@wE?ix2<(G7h zQ7&jBQ}Kx9mm<0frw#BDYR7_HvY7En#z?&*FurzdDNdfF znCL1U3#iO`BnfPyM@>;#m2Lw9cGn;(5*QN9$zd4P68ji$X?^=qHraP~Nk@JX6}S>2 zhJz4MVTib`OlEAqt!UYobU0-0r*`=03)&q7ubQXrt|t?^U^Z#MEZV?VEin3Nv1~?U zuwwSeR10BrNZ@*h7M)aTxG`D(By$(ZP#UmBGf}duX zhx;7y1x@j2t5sS#QjbEPIj95hV8*7uF6c}~NBl5|hgbB(}M3vnt zu_^>@s*Bd>w;{6v53iF5q7Em>8n&m&MXL#ilSzuC6HTzzi-V#lWoX zBOSBYm|ti@bXb9HZ~}=dlV+F?nYo3?YaV2=N@AI5T5LWWZzwvnFa%w%C<$wBkc@&3 zyUE^8xu<=k!KX<}XJYo8L5NLySP)cF392GK97(ylPS+&b}$M$Y+1VDrJa`GG7+%ToAsh z5NEB9oVv>as?i7f^o>0XCd%2wIaNRyejlFws`bXG$Mhmb6S&shdZKo;p&~b4wv$ z?2ZoM$la+_?cynm&~jEi6bnD;zSx<0BuCSDHGSssT7Qctf`0U!GDwG=+^|-a5%8Ty z&Q!%m%geLjBT*#}t zv1wDzuC)_WK1E|H?NZ&-xr5OX(ukXMYM~_2c;K}219agkgBte_#f+b9Al8XjL-p}1 z8deBZFjplH85+Fa5Q$MbL>AfKPxj?6Bib2pevGxIGAG=vr;IuuC%sq9x{g4L$?Bw+ zvoo`E)3#bpJ{Ij>Yn0I>R&&5B$&M|r&zxh+q>*QPaxi2{lp?omkCo~7ibow#@{0P> z&XBocU8KAP3hNPKEMksQ^90zB1&&b1Me>?maT}4xv7QHA@Nbvt-iWy7+yPFa9G0DP zP82ooqy_ku{UPv$YF0kFrrx3L=FI|AjG7*(paRLM0k1J>3oPxU0Zd+4&vIMW>h4O5G zej2N$(e|2Re z@8xQ|uUvbA8QVXGjZ{Uiolxb7c7C^nW`P(m*Jkqn)qdI0xTa#fcK7SLp)<86(c`A3 zFNB4y#NHe$wYc7V)|=uiW8gS{1WMaJhDj4xYhld;zJip&uJ{Jg3R`n+jywDc*=>bW zEqw(_+j%8LMRrH~+M*$V$xn9x9P&zt^evq$P`aSf-51`ZOKm(35OEUMlO^$>%@b?a z>qXny!8eV7cI)cb0lu+dwzGH(Drx1-g+uDX;Oy$cs+gz~?LWif;#!+IvPR6fa&@Gj zwz!Vw9@-Jm1QtYT?I@JQf%`=$^I%0NK9CJ75gA}ff@?I*xUD7!x*qcyTX5X+pS zAVy4{51-dHKs*OroaTy;U?zpFS;bKV7wb}8v+Q#z<^$%NXN(_hG}*9E_DhrRd7Jqp zr}2jKH{avzrpXj?cW{17{kgKql+R(Ew55YiKK7=8nkzp7Sx<956tRa(|yvHlW zNO7|;GvR(1q}GrTY@uC&ow0me|8wE(PzOd}Y=T+Ih8@c2&~6(nzQrK??I7DbOguA9GUoz3ASU%BFCc8LBsslu|nl>q8Ag(jA9vkQ`q2amJ5FfA7GoCdsLW znuok(diRhuN+)A&`rH{$(HXWyG2TLXhVDo4xu?}k2cH7QsoS>sPV)ylb45Zt&_+1& zT)Yzh#FHRZ-z_Q^8~IZ+G~+qSw-D<{0NZ5!J1%rAc`B23T98TMh9ylkzdk^O?W`@C??Z5U9#vi0d<(`?9fQvNN^ji;&r}geU zSbKR5Mv$&u8d|iB^qiLaZQ#@)%kx1N;Og8Js>HQD3W4~pI(l>KiHpAv&-Ev45z(vYK<>p6 z6#pU(@rUu{i9UngMhU&FI5yeRub4#u=9H+N>L@t}djC(Schr;gc90n%)qH{$l0L4T z;=R%r>CuxH!O@+eBR`rBLrT0vnP^sJ^+qE^C8ZY0-@te3SjnJ)d(~HcnQw@`|qAp|Trrs^E*n zY1!(LgVJfL?@N+u{*!Q97N{Uu)ZvaN>hsM~J?*Qvqv;sLnXHjKrtG&x)7tk?8%AHI zo5eI#`qV1{HmUf-Fucg1xn?Kw;(!%pdQ)ai43J3NP4{%x1D zI0#GZh8tjRy+2{m$HyI(iEwK30a4I36cSht3MM85UqccyUq6$j5K>|w$O3>`Ds;`0736+M@q(9$(`C6QZQ-vAKjIXKR(NAH88 zwfM6_nGWlhpy!_o56^BU``%TQ%tD4hs2^<2pLypjAZ;W9xAQRfF_;T9W-uidv{`B z{)0udL1~tMg}a!hzVM0a_$RbuQk|EG&(z*{nZXD3hf;BJe4YxX8pKX7VaIjjDP%sk zU5iOkhzZ&%?A@YfaJ8l&H;it@;u>AIB`TkglVuy>h;vjtq~o`5NfvR!ZfL8qS#LL` zD!nYHGzZ|}BcCf8s>b=5nZRYV{)KK#7$I06s<;RyYC3<~`mob_t2IfR*dkFJyL?FU zvuo-EE4U(-le)zdgtW#AVA~zjx*^80kd3A#?vI63pLnW2{j*=#UG}ISD>=ZGA$H&` z?Nd8&11*4`%MQlM64wfK`{O*ad5}vk4{Gy}F98xIAsmjp*9P=a^yBHBjF2*Iibo2H zGJAMFDjZcVd%6bZ`dz;I@F55VCn{~RKUqD#V_d{gc|Z|`RstPw$>Wu+;SY%yf1rI=>51Oolm>cnjOWHm?ydcgGs_kPUu=?ZKtQS> zKtLS-v$OMWXO>B%Z4LFUgw4MqA?60o{}-^6tf(c0{Y3|yF##+)RoXYVY-lyPhgn{1 z>}yF0Ab}D#1*746QAj5c%66>7CCWs8O7_d&=Ktu!SK(m}StvvBT1$8QP3O2a*^BNA z)HPhmIi*((2`?w}IE6Fo-SwzI_F~OC7OR}guyY!bOQfpNRg3iMvsFPYb9-;dT6T%R zhLwIjgiE^-9_4F3eMHZ3LI%bbOmWVe{SONpujQ;3C+58=Be4@yJK>3&@O>YaSdrevAdCLMe_tL zl8@F}{Oc!aXO5!t!|`I zdC`k$5z9Yf%RYJp2|k*DK1W@AN23W%SD0EdUV^6~6bPp_HZi0@dku_^N--oZv}wZA zH?Bf`knx%oKB36^L;P%|pf#}Tp(icw=0(2N4aL_Ea=9DMtF})2ay68V{*KfE{O=xL zf}tcfCL|D$6g&_R;r~1m{+)sutQPKzVv6Zw(%8w&4aeiy(qct1x38kiqgk!0^^X3IzI2ia zxI|Q)qJNEf{=I$RnS0`SGMVg~>kHQB@~&iT7+eR!Ilo1ZrDc3TVW)CvFFjHK4K}Kh z)dxbw7X%-9Ol&Y4NQE~bX6z+BGOEIIfJ~KfD}f4spk(m62#u%k<+iD^`AqIhWxtKGIm)l$7=L`=VU0Bz3-cLvy&xdHDe-_d3%*C|Q&&_-n;B`87X zDBt3O?Wo-Hg6*i?f`G}5zvM?OzQjkB8uJhzj3N;TM5dSM$C@~gGU7nt-XX_W(p0IA6$~^cP*IAnA<=@HVqNz=Dp#Rcj9_6*8o|*^YseK_4d&mBY*Y&q z8gtl;(5%~3Ehpz)bLX%)7|h4tAwx}1+8CBtu9f5%^SE<&4%~9EVn4*_!r}+{^2;} zwz}#@Iw?&|8F2LdXUIjh@kg3QH69tqxR_FzA;zVpY=E zcHnWh(3j3UXeD=4m_@)Ea4m#r?axC&X%#wC8FpJPDYR~@65T?pXuWdPzEqXP>|L`S zKYFF0I~%I>SFWF|&sDsRdXf$-TVGSoWTx7>7mtCVUrQNVjZ#;Krobgh76tiP*0(5A zs#<7EJ#J`Xhp*IXB+p5{b&X3GXi#b*u~peAD9vr0*Vd&mvMY^zxTD=e(`}ybDt=BC(4q)CIdp>aK z0c?i@vFWjcbK>oH&V_1m_EuZ;KjZSiW^i30U` zGLK{%1o9TGm8@gy+Rl=-5&z`~Un@l*2ne3e9B+>wKyxuoUa1qhf?-Pi= zZLCD-b7*(ybv6uh4b`s&Ol3hX2ZE<}N@iC+h&{J5U|U{u$XK0AJz)!TSX6lrkG?ris;y{s zv`B5Rq(~G58?KlDZ!o9q5t%^E4`+=ku_h@~w**@jHV-+cBW-`H9HS@o?YUUkKJ;AeCMz^f@FgrRi@?NvO3|J zBM^>4Z}}!vzNum!R~o0)rszHG(eeq!#C^wggTgne^2xc9nIanR$pH1*O;V>3&#PNa z7yoo?%T(?m-x_ow+M0Bk!@ow>A=skt&~xK=a(GEGIWo4AW09{U%(;CYLiQIY$bl3M zxC_FGKY%J`&oTS{R8MHVe{vghGEshWi!(EK*DWmoOv|(Ff#(bZ-<~{rc|a%}Q4-;w z{2gca97m~Nj@Nl{d)P`J__#Zgvc@)q_(yfrF2yHs6RU8UXxcU(T257}E#E_A}%2_IW?%O+7v((|iQ{H<|$S7w?;7J;iwD>xbZc$=l*(bzRXc~edIirlU0T&0E_EXfS5%yA zs0y|Sp&i`0zf;VLN=%hmo9!aoLGP<*Z7E8GT}%)cLFs(KHScNBco(uTubbxCOD_%P zD7XlHivrSWLth7jf4QR9`jFNk-7i%v4*4fC*A=;$Dm@Z^OK|rAw>*CI%E z3%14h-)|Q%_$wi9=p!;+cQ*N1(47<49TyB&B*bm_m$rs+*ztWStR~>b zE@V06;x19Y_A85N;R+?e?zMTIqdB1R8>(!4_S!Fh={DGqYvA0e-P~2DaRpCYf4$-Q z*&}6D!N_@s`$W(|!DOv%>R0n;?#(HgaI$KpHYpnbj~I5eeI(u4CS7OJajF%iKz)*V zt@8=9)tD1ML_CrdXQ81bETBeW!IEy7mu4*bnU--kK;KfgZ>oO>f)Sz~UK1AW#ZQ_ic&!ce~@(m2HT@xEh5u%{t}EOn8ET#*U~PfiIh2QgpT z%gJU6!sR2rA94u@xj3%Q`n@d}^iMH#X>&Bax+f4cG7E{g{vlJQ!f9T5wA6T`CgB%6 z-9aRjn$BmH=)}?xWm9bf`Yj-f;%XKRp@&7?L^k?OT_oZXASIqbQ#eztkW=tmRF$~% z6(&9wJuC-BlGrR*(LQKx8}jaE5t`aaz#Xb;(TBK98RJBjiqbZFyRNTOPA;fG$;~e` zsd6SBii3^(1Y`6^#>kJ77xF{PAfDkyevgox`qW`nz1F`&w*DH5Oh1idOTLES>DToi z8Qs4|?%#%>yuQO1#{R!-+2AOFznWo)e3~_D!nhoDgjovB%A8< zt%c^KlBL$cDPu!Cc`NLc_8>f?)!FGV7yudL$bKj!h;eOGkd;P~sr6>r6TlO{Wp1%xep8r1W{`<4am^(U} z+nCDP{Z*I?IGBE&*KjiaR}dpvM{ZFMW%P5Ft)u$FD373r2|cNsz%b0uk1T+mQI@4& zFF*~xDxDRew1Bol-*q>F{Xw8BUO;>|0KXf`lv7IUh%GgeLUzR|_r(TXZTbfXFE0oc zmGMwzNFgkdg><=+3MnncRD^O`m=SxJ6?}NZ8BR)=ag^b4Eiu<_bN&i0wUaCGi60W6 z%iMl&`h8G)y`gfrVw$={cZ)H4KSQO`UV#!@@cDx*hChXJB7zY18EsIo1)tw0k+8u; zg(6qLysbxVbLFbkYqKbEuc3KxTE+%j5&k>zHB8_FuDcOO3}FS|eTxoUh2~|Bh?pD| zsmg(EtMh`@s;`(r!%^xxDt(5wawK+*jLl>_Z3shaB~vdkJ!V3RnShluzmwn7>PHai z3avc`)jZSAvTVC6{2~^CaX49GXMtd|sbi*swkgoyLr=&yp!ASd^mIC^D;a|<=3pSt zM&0u%#%DGzlF4JpMDs~#kU;UCtyW+d3JwNiu`Uc7Yi6%2gfvP_pz8I{Q<#25DjM_D z(>8yI^s@_tG@c=cPoZImW1CO~`>l>rs=i4BFMZT`vq5bMOe!H@8q@sEZX<-kiY&@u3g1YFc zc@)@OF;K-JjI(eLs~hy8qOa9H1zb!3GslI!nH2DhP=p*NLHeh^9WF?4Iakt+b( z-4!;Q-8c|AX>t+5I64EKpDj4l2x*!_REy9L_9F~i{)1?o#Ws{YG#*}lg_zktt#ZlN zmoNsGm7$AXLink`GWtY*TZEH!J9Qv+A1y|@>?&(pb(6XW#ZF*}x*{60%wnt{n8Icp zq-Kb($kh6v_voqvA`8rq!cgyu;GaWZ>C2t6G5wk! zcKTlw=>KX3ldU}a1%XESW71))Z=HW%sMj2znJ;fdN${00DGGO}d+QsTQ=f;BeZ`eC~0-*|gn$9G#`#0YbT(>O(k&!?2jI z&oi9&3n6Vz<4RGR}h*1ggr#&0f%Op(6{h>EEVFNJ0C>I~~SmvqG+{RXDrexBz zw;bR@$Wi`HQ3e*eU@Cr-4Z7g`1R}>3-Qej(#Dmy|CuFc{Pg83Jv(pOMs$t(9vVJQJ zXqn2Ol^MW;DXq!qM$55vZ{JRqg!Q1^Qdn&FIug%O3=PUr~Q`UJuZ zc`_bE6i^Cp_(fka&A)MsPukiMyjG$((zE$!u>wyAe`gf-1Qf}WFfi1Y{^ zdCTTrxqpQE#2BYWEBnTr)u-qGSVRMV7HTC(x zb(0FjYH~nW07F|{@oy)rlK6CCCgyX?cB;19Z(bCP5>lwN0UBF}Ia|L0$oGHl-oSTZ zr;(u7nDjSA03v~XoF@ULya8|dzH<2G=n9A)AIkQKF0mn?!BU(ipengAE}6r`CE!jd z=EcX8exgDZZQ~~fgxR-2yF;l|kAfnjhz|i_o~cYRdhnE~1yZ{s zG!kZJ<-OVnO{s3bOJK<)`O;rk>=^Sj3M76Nqkj<_@Jjw~iOkWUCL+*Z?+_Jvdb!0cUBy=(5W9H-r4I zxAFts>~r)B>KXdQANyaeKvFheZMgoq4EVV0|^NR@>ea* zh%<78{}wsdL|9N1!jCN-)wH4SDhl$MN^f_3&qo?>Bz#?c{ne*P1+1 z!a`(2Bxy`S^(cw^dv{$cT^wEQ5;+MBctgPfM9kIQGFUKI#>ZfW9(8~Ey-8`OR_XoT zflW^mFO?AwFWx9mW2-@LrY~I1{dlX~jBMt!3?5goHeg#o0lKgQ+eZcIheq@A&dD}GY&1c%hsgo?z zH>-hNgF?Jk*F0UOZ*bs+MXO(dLZ|jzKu5xV1v#!RD+jRrHdQ z>>b){U(I@i6~4kZXn$rk?8j(eVKYJ2&k7Uc`u01>B&G@c`P#t#x@>Q$N$1aT514fK zA_H8j)UKen{k^ehe%nbTw}<JV6xN_|| z(bd-%aL}b z3VITE`N~@WlS+cV>C9TU;YfsU3;`+@hJSbG6aGvis{Gs%2K|($)(_VfpHB|DG8Nje+0tCNW%_cu3hk0F)~{-% zW{2xSu@)Xnc`Dc%AOH)+LT97ImFR*WekSnJ3OYIs#ijP4TD`K&7NZKsfZ;76k@VD3py?pSw~~r^VV$Z zuUl9lF4H2(Qga0EP_==vQ@f!FLC+Y74*s`Ogq|^!?RRt&9e9A&?Tdu=8SOva$dqgYU$zkKD3m>I=`nhx-+M;-leZgt z8TeyQFy`jtUg4Ih^JCUcq+g_qs?LXSxF#t+?1Jsr8c1PB#V+f6aOx@;ThTIR4AyF5 z3m$Rq(6R}U2S}~Bn^M0P&Aaux%D@ijl0kCCF48t)+Y`u>g?|ibOAJoQGML@;tn{%3IEMaD(@`{7ByXQ`PmDeK*;W?| zI8%%P8%9)9{9DL-zKbDQ*%@Cl>Q)_M6vCs~5rb(oTD%vH@o?Gk?UoRD=C-M|w~&vb z{n-B9>t0EORXd-VfYC>sNv5vOF_Wo5V)(Oa%<~f|EU7=npanpVX^SxPW;C!hMf#kq z*vGNI-!9&y!|>Zj0V<~)zDu=JqlQu+ii387D-_U>WI_`3pDuHg{%N5yzU zEulPN)%3&{PX|hv*rc&NKe(bJLhH=GPuLk5pSo9J(M9J3v)FxCo65T%9x<)x+&4Rr2#nu2?~Glz|{28OV6 z)H^`XkUL|MG-$XE=M4*fIPmeR2wFWd>5o*)(gG^Y>!P4(f z68RkX0cRBOFc@`W-IA(q@p@m>*2q-`LfujOJ8-h$OgHte;KY4vZKTxO95;wh#2ZDL zKi8aHkz2l54lZd81t`yY$Tq_Q2_JZ1d(65apMg}vqwx=ceNOWjFB)6m3Q!edw2<{O z4J6+Un(E8jxs-L-K_XM_VWahy zE+9fm_ZaxjNi{fI_AqLKqhc4IkqQ4`Ut$=0L)nzlQw^%i?bP~znsbMY3f}*nPWqQZ zz_CQDpZ?Npn_pEr`~SX1`OoSkS;bmzQ69y|W_4bH3&U3F7EBlx+t%2R02VRJ01cfX zo$$^ObDHK%bHQaOcMpCq@@Jp8!OLYVQO+itW1ZxlkmoG#3FmD4b61mZjn4H|pSmYi2YE;I#@jtq8Mhjdgl!6({gUsQA>IRXb#AyWVt7b=(HWGUj;wd!S+q z4S+H|y<$yPrrrTqQHsa}H`#eJFV2H5Dd2FqFMA%mwd`4hMK4722|78d(XV}rz^-GV(k zqsQ>JWy~cg_hbp0=~V3&TnniMQ}t#INg!o2lN#H4_gx8Tn~Gu&*ZF8#kkM*5gvPu^ zw?!M^05{7q&uthxOn?%#%RA_%y~1IWly7&_-sV!D=Kw3DP+W)>YYRiAqw^d7vG_Q%v;tRbE1pOBHc)c&_5=@wo4CJTJ1DeZErEvP5J(kc^GnGYX z|LqQjTkM{^gO2cO#-(g!7^di@$J0ibC(vsnVkHt3osnWL8?-;R1BW40q5Tmu_9L-s z7fNF5fiuS-%B%F$;D97N-I@!~c+J>nv%mzQ5vs?1MgR@XD*Gv`A{s8 z5Cr>z5j?|sb>n=c*xSKHpdy667QZT?$j^Doa%#m4ggM@4t5Oe%iW z@w~j_B>GJJkO+6dVHD#CkbC(=VMN8nDkz%44SK62N(ZM#AsNz1KW~3(i=)O;q5JrK z?vAVuL}Rme)OGQuLn8{3+V352UvEBV^>|-TAAa1l-T)oiYYD&}Kyxw73shz?Bn})7 z_a_CIPYK(zMp(i+tRLjy4dV#CBf3s@bdmwXo`Y)dRq9r9-c@^2S*YoNOmAX%@OYJOXs zT*->in!8Ca_$W8zMBb04@|Y)|>WZ)-QGO&S7Zga1(1#VR&)X+MD{LEPc%EJCXIMtr z1X@}oNU;_(dfQ_|kI-iUSTKiVzcy+zr72kq)TIp(GkgVyd%{8@^)$%G)pA@^Mfj71FG%d?sf(2Vm>k%X^RS`}v0LmwIQ7!_7cy$Q8pT?X1VWecA_W68u==HbrU& z@&L6pM0@8ZHL?k{6+&ewAj%grb6y@0$3oamTvXsjGmPL_$~OpIyIq%b$(uI1VKo zk_@{r>1p84UK3}B>@d?xUZ}dJk>uEd+-QhwFQ`U?rA=jj+$w8sD#{492P}~R#%z%0 z5dlltiAaiPKv9fhjmuy{*m!C22$;>#85EduvdSrFES{QO$bHpa7E@&{bWb@<7VhTF zXCFS_wB>7*MjJ3$_i4^A2XfF2t7`LOr3B@??OOUk=4fKkaHne4RhI~Lm$JrHfUU*h zgD9G66;_F?3>0W{pW2A^DR7Bq`ZUiSc${S8EM>%gFIqAw0du4~kU#vuCb=$I_PQv? zZfEY7X6c{jJZ@nF&T>4oyy(Zr_XqnMq)ZtGPASbr?IhZOnL|JKY()`eo=P5UK9(P-@ zOJKFogtk|pscVD+#$7KZs^K5l4gC}*CTd0neZ8L(^&1*bPrCp23%{VNp`4Ld*)Fly z)b|zb*bCzp?&X3_=qLT&0J+=p01&}9*xbk~^hd^@mV!Ha`1H+M&60QH2c|!Ty`RepK|H|Moc5MquD z=&$Ne3%WX+|7?iiR8=7*LW9O3{O%Z6U6`VekeF8lGr5vd)rsZu@X#5!^G1;nV60cz zW?9%HgD}1G{E(YvcLcIMQR65BP50)a;WI*tjRzL7diqRqh$3>OK{06VyC=pj6OiardshTnYfve5U>Tln@y{DC99f!B4> zCrZa$B;IjDrg}*D5l=CrW|wdzENw{q?oIj!Px^7DnqAsU7_=AzXxoA;4(YvN5^9ag zwEd4-HOlO~R0~zk>!4|_Z&&q}agLD`Nx!%9RLC#7fK=w06e zOK<>|#@|e2zjwZ5aB>DJ%#P>k4s0+xHJs@jROvoDQfSoE84l8{9y%5^POiP+?yq0> z7+Ymbld(s-4p5vykK@g<{X*!DZt1QWXKGmj${`@_R~=a!qPzB357nWW^KmhV!^G3i zsYN{2_@gtzsZH*FY!}}vNDnqq>kc(+7wK}M4V*O!M&GQ|uj>+8!Q8Ja+j3f*MzwcI z^s4FXGC=LZ?il4D+Y^f89wh!d7EU-5dZ}}>_PO}jXRQ@q^CjK-{KVnmFd_f&IDKmx zZ5;PDLF%_O);<4t`WSMN;Ec^;I#wU?Z?_R|Jg`#wbq;UM#50f@7F?b7ySi-$C-N;% zqXowTcT@=|@~*a)dkZ836R=H+m6|fynm#0Y{KVyYU=_*NHO1{=Eo{^L@wWr7 zjz9GOu8Fd&v}a4d+}@J^9=!dJRsCO@=>K6UCM)Xv6};tb)M#{(k!i}_0Rjq z2kb7wPcNgov%%q#(1cLykjrxAg)By+3QueBR>Wsep&rWQHq1wE!JP+L;q+mXts{j@ zOY@t9BFmofApO0k@iBFPeKsV3X=|=_t65QyohXMSfMRr7Jyf8~ogPVmJwbr@`nmml zov*NCf;*mT(5s4K=~xtYy8SzE66W#tW4X#RnN%<8FGCT{z#jRKy@Cy|!yR`7dsJ}R z!eZzPCF+^b0qwg(mE=M#V;Ud9)2QL~ z-r-2%0dbya)%ui_>e6>O3-}4+Q!D+MU-9HL2tH)O`cMC1^=rA=q$Pcc;Zel@@ss|K zH*WMdS^O`5Uv1qNTMhM(=;qjhaJ|ZC41i2!kt4;JGlXQ$tvvF8Oa^C@(q6(&6B^l) zNG{GaX?`qROHwL-F1WZDEF;C6Inuv~1&ZuP3j53547P38tr|iPH#3&hN*g0R^H;#) znft`cw0+^Lwe{!^kQat+xjf_$SZ05OD6~U`6njelvd+4pLZU(0ykS5&S$)u?gm!;} z+gJ8g12b1D4^2HH!?AHFAjDAP^q)Juw|hZfIv{3Ryn%4B^-rqIF2 zeWk^za4fq#@;re{z4_O|Zj&Zn{2WsyI^1%NW=2qA^iMH>u>@;GAYI>Bk~u0wWQrz* zdEf)7_pSYMg;_9^qrCzvv{FZYwgXK}6e6ceOH+i&+O=x&{7aRI(oz3NHc;UAxMJE2 zDb0QeNpm$TDcshGWs!Zy!shR$lC_Yh-PkQ`{V~z!AvUoRr&BAGS#_*ZygwI2-)6+a zq|?A;+-7f0Dk4uuht z6sWPGl&Q$bev1b6%aheld88yMmBp2j=z*egn1aAWd?zN=yEtRDGRW&nmv#%OQwuJ; zqKZ`L4DsqJwU{&2V9f>2`1QP7U}`6)$qxTNEi`4xn!HzIY?hDnnJZw+mFnVSry=bLH7ar+M(e9h?GiwnOM?9ZJcTJ08)T1-+J#cr&uHhXkiJ~}&(}wvzCo33 zLd_<%rRFQ3d5fzKYQy41<`HKk#$yn$Q+Fx-?{3h72XZrr*uN!5QjRon-qZh9-uZ$rWEKZ z!dJMP`hprNS{pzqO`Qhx`oXGd{4Uy0&RDwJ`hqLw4v5k#MOjvyt}IkLW{nNau8~XM z&XKeoVYreO=$E%z^WMd>J%tCdJx5-h+8tiawu2;s& zD7l`HV!v@vcX*qM(}KvZ#%0VBIbd)NClLBu-m2Scx1H`jyLYce;2z;;eo;ckYlU53 z9JcQS+CvCwj*yxM+e*1Vk6}+qIik2VzvUuJyWyO}piM1rEk%IvS;dsXOIR!#9S;G@ zPcz^%QTf9D<2~VA5L@Z@FGQqwyx~Mc-QFzT4Em?7u`OU!PB=MD8jx%J{<`tH$Kcxz zjIvb$x|`s!-^^Zw{hGV>rg&zb;=m?XYAU0LFw+uyp8v@Y)zmjj&Ib7Y1@r4`cfrS%cVxJiw`;*BwIU*6QVsBBL;~nw4`ZFqs z1YSgLVy=rvA&GQB4MDG+j^)X1N=T;Ty2lE-`zrg(dNq?=Q`nCM*o8~A2V~UPArX<| zF;e$5B0hPSo56=ePVy{nah#?e-Yi3g*z6iYJ#BFJ-5f0KlQ-PRiuGwe29fyk1T6>& zeo2lvb%h9Vzi&^QcVNp}J!x&ubtw5fKa|n2XSMlg#=G*6F|;p)%SpN~l8BaMREDQN z-c9O}?%U1p-ej%hzIDB!W_{`9lS}_U==fdYpAil1E3MQOFW^u#B)Cs zTE3|YB0bKpXuDKR9z&{4gNO3VHDLB!xxPES+)yaJxo<|}&bl`F21};xsQnc!*FPZA zSct2IU3gEu@WQKmY-vA5>MV?7W|{$rAEj4<8`*i)<%fj*gDz2=ApqZ&MP&0UmO1?q!GN=di+n(#bB_mHa z(H-rIOJqamMfwB%?di!TrN=x~0jOJtvb0e9uu$ZCVj(gJyK}Fa5F2S?VE30P{#n3eMy!-v7e8viCooW9cfQx%xyPNL*eDKL zB=X@jxulpkLfnar7D2EeP*0L7c9urDz{XdV;@tO;u`7DlN7#~ zAKA~uM2u8_<5FLkd}OzD9K zO5&hbK8yakUXn8r*H9RE zO9Gsipa2()=&x=1mnQtNP#4m%GXThu8Ccqx*qb;S{5}>bU*V5{SY~(Hb={cyTeaTM zMEaKedtJf^NnJrwQ^Bd57vSlJ3l@$^0QpX@_1>h^+js8QVpwOiIMOiSC_>3@dt*&| zV?0jRdlgn|FIYam0s)a@5?0kf7A|GD|dRnP1=B!{ldr;N5s)}MJ=i4XEqlC}w)LEJ}7f9~c!?It(s zu>b=YBlFRi(H-%8A!@Vr{mndRJ z_jx*?BQpK>qh`2+3cBJhx;>yXPjv>dQ0m+nd4nl(L;GmF-?XzlMK zP(Xeyh7mFlP#=J%i~L{o)*sG7H5g~bnL2Hn3y!!r5YiYRzgNTvgL<(*g5IB*gcajK z86X3LoW*5heFmkIQ-I_@I_7b!Xq#O;IzOv(TK#(4gd)rmCbv5YfA4koRfLydaIXUU z8(q?)EWy!sjsn-oyUC&uwJqEXdlM}#tmD~*Ztav=mTQyrw0^F=1I5lj*}GSQTQOW{ z=O12;?fJfXxy`)ItiDB@0sk43AZo_sRn*jc#S|(2*%tH84d|UTYN!O4R(G6-CM}84 zpiyYJ^wl|w@!*t)dwn0XJv2kuHgbfNL$U6)O-k*~7pQ?y=sQJdKk5x`1>PEAxjIWn z{H$)fZH4S}%?xzAy1om0^`Q$^?QEL}*ZVQK)NLgmnJ`(we z21c23X1&=^>k;UF-}7}@nzUf5HSLUcOYW&gsqUrj7%d$)+d8ZWwTZq)tOgc%fz95+ zl%sdl)|l|jXfqIcjKTFrX74Rbq1}osA~fXPSPE?XO=__@`7k4Taa!sHE8v-zfx(AM zXT_(7u;&_?4ZIh%45x>p!(I&xV|IE**qbqCRGD5aqLpCRvrNy@uT?iYo-FPpu`t}J zSTZ}MDrud+`#^14r`A%UoMvN;raizytxMBV$~~y3i0#m}0F}Dj_fBIz+)1RWdnctP z>^O^vd0E+jS+$V~*`mZWER~L^q?i-6RPxxufWdrW=%prbCYT{5>Vgu%vPB)~NN*2L zB?xQg2K@+Xy=sPh$%10LH!39p&SJG+3^i*lFLn=uY8Io6AXRZf;p~v@1(hWsFzeKzx99_{w>r;cypkPVJCKtLGK>?-K0GE zGH>$g?u`)U_%0|f#!;+E>?v>qghuBwYZxZ*Q*EE|P|__G+OzC-Z+}CS(XK^t!TMoT zc+QU|1C_PGiVp&_^wMxfmMAuJDQ%1p4O|x5DljN6+MJiO%8s{^ts8$uh5`N~qK46c`3WY#hRH$QI@*i1OB7qBIN*S2gK#uVd{ zik+wwQ{D)g{XTGjKV1m#kYhmK#?uy)g@idi&^8mX)Ms`^=hQGY)j|LuFr8SJGZjr| zzZf{hxYg)-I^G|*#dT9Jj)+wMfz-l7ixjmwHK9L4aPdXyD-QCW!2|Jn(<3$pq-BM; zs(6}egHAL?8l?f}2FJSkP`N%hdAeBiD{3qVlghzJe5s9ZUMd`;KURm_eFaK?d&+TyC88v zCv2R(Qg~0VS?+p+l1e(aVq`($>|0b{{tPNbi} zaZDffTZ7N|t2D5DBv~aX#X+yGagWs1JRsqbr4L8a`B`m) z1p9?T`|*8ZXHS7YD8{P1Dk`EGM`2Yjsy0=7M&U6^VO30`Gx!ZkUoqmc3oUbd&)V*iD08>dk=#G!*cs~^tOw^s8YQqYJ z!5=-4ZB7rW4mQF&YZw>T_in-c9`0NqQ_5Q}fq|)%HECgBd5KIo`miEcJ>~a1e2B@) zL_rqoQ;1MowD34e6#_U+>D`WcnG5<2Q6cnt4Iv@NC$*M+i3!c?6hqPJLsB|SJ~xo! zm>!N;b0E{RX{d*in3&0w!cmB&TBNEjhxdg!fo+}iGE*BWV%x*46rT@+cXU;leofWy zxst{S8m!_#hIhbV7wfWN#th8OI5EUr3IR_GOIzBgGW1u4J*TQxtT7PXp#U#EagTV* zehVkBFF06`@5bh!t%L)-)`p|d7D|^kED7fsht#SN7*3`MKZX};Jh0~nCREL_BGqNR zxpJ4`V{%>CAqEE#Dt95u=;Un8wLhrac$fao`XlNsOH%&Ey2tK&vAcriS1kXnntDuttcN{%YJz@!$T zD&v6ZQ>zS1`o!qT=JK-Y+^i~bZkVJpN8%<4>HbuG($h9LP;{3DJF_Jcl8CA5M~<3s^!$Sg62zLEnJtZ z0`)jwK75Il6)9XLf(64~`778D6-#Ie1IR2Ffu+_Oty%$8u+bP$?803V5W6%(+iZzp zp5<&sBV&%CJcXUIATUakP1czt$&0x$lyoLH!ueNaIpvtO z*eCijxOv^-D?JaLzH<3yhOfDENi@q#4w(#tl-19(&Yc2K%S8Y&r{3~-)P17sC1{rQ zOy>IZ6%814_UoEi+w9a4XyGXF66{rgE~UT)oT4x zg9oIx@|{KL#VpTyE=6WK@Sbd9RKEEY)5W{-%0F^6(QMuT$RQRZ&yqfyF*Z$f8>{iT zq(;UzB-Ltv;VHvh4y%YvG^UEkvpe9ugiT97ErbY0ErCEOWs4J=kflA!*Q}gMbEP`N zY#L`x9a?E)*~B~t+7c8eR}VY`t}J;EWuJ-6&}SHnNZ8i0PZT^ahA@@HXk?c0{)6rC zP}I}_KK7MjXqn1E19gOwWvJ3i9>FNxN67o?lZy4H?n}%j|Dq$p%TFLUPJBD;R|*0O z3pLw^?*$9Ax!xy<&fO@;E2w$9nMez{5JdFO^q)B0OmGwkxxaDsEU+5C#g+?Ln-Vg@ z-=z4O*#*VJa*nujGnGfK#?`a|xfZsuiO+R}7y(d60@!WUIEUt>K+KTI&I z9YQ6#hVCo}0^*>yr-#Lisq6R?uI=Ms!J7}qm@B}Zu zp%f-~1Cf!-5S0xXl`oqq&fS=tt0`%dDWI&6pW(s zJXtYiY&~t>k5I0RK3sN;#8?#xO+*FeK#=C^%{Y>{k{~bXz%(H;)V5)DZRk~(_d0b6 zV!x54fwkl`1y;%U;n|E#^Vx(RGnuN|T$oJ^R%ZmI{8(9>U-K^QpDcT?Bb@|J0NAfvHtL#wP ziYupr2E5=_KS{U@;kyW7oy*+UTOiF*e+EhYqVcV^wx~5}49tBNSUHLH1=x}6L2Fl^4X4633$k!ZHZTL50Vq+a5+ z<}uglXQ<{x&6ey)-lq6;4KLHbR)_;Oo^FodsYSw3M-)FbLaBcPI=-ao+|))T2ksKb z{c%Fu`HR1dqNw8%>e0>HI2E_zNH1$+4RWfk}p-h(W@)7LC zwVnUO17y+~kw35CxVtokT44iF$l8XxYuetp)1Br${@lb(Q^e|q*5%7JNxp5B{r<09 z-~8o#rI1(Qb9FhW-igcsC6npf5j`-v!nCrAcVx5+S&_V2D>MOWp6cV$~Olhp2`F^Td{WV`2k4J`djb#M>5D#k&5XkMu*FiO(uP{SNX@(=)|Wm`@b> z_D<~{ip6@uyd7e3Rn+qM80@}Cl35~^)7XN?D{=B-4@gO4mY%`z!kMIZizhGtCH-*7 z{a%uB4usaUoJwbkVVj%8o!K^>W=(ZzRDA&kISY?`^0YHKe!()(*w@{w7o5lHd3(Us zUm-K=z&rEbOe$ackQ3XH=An;Qyug2g&vqf;zsRBldxA+=vNGoM$Zo9yT?Bn?`Hkiq z&h@Ss--~+=YOe@~JlC`CdSHy zcO`;bgMASYi6`WSw#Z|A;wQgH@>+I3OT6(*JgZZ_XQ!LrBJfVW2RK%#02|@V|H4&8DqslU6Zj(x!tM{h zRawG+Vy63_8gP#G!Eq>qKf(C&!^G$01~baLLk#)ov-Pqx~Du>%LHMv?=WBx2p2eV zbj5fjTBhwo&zeD=l1*o}Zs%SMxEi9yokhbHhY4N!XV?t8}?!?42E-B^Rh&ABFxovs*HeQ5{{*)SrnJ%e{){Z_#JH+jvwF7>Jo zE+qzWrugBwVOZou~oFa(wc7?`wNde>~HcC@>fA^o>ll?~aj-e|Ju z+iJzZg0y1@eQ4}rm`+@hH(|=gW^;>n>ydn!8%B4t7WL)R-D>mMw<7Wz6>ulFnM7QA ze2HEqaE4O6jpVq&ol3O$46r+DW@%glD8Kp*tFY#8oiSyMi#yEpVIw3#t?pXG?+H>v z$pUwT@0ri)_Bt+H(^uzp6qx!P(AdAI_Q?b`>0J?aAKTPt>73uL2(WXws9+T|%U)Jq zP?Oy;y6?{%J>}?ZmfcnyIQHh_jL;oD$`U#!v@Bf{5%^F`UiOX%)<0DqQ^nqA5Ac!< z1DPO5C>W0%m?MN*x(k>lDT4W3;tPi=&yM#Wjwc5IFNiLkQf`7GN+J*MbB4q~HVePM zeDj8YyA*btY&n!M9$tuOxG0)2um))hsVsY+(p~JnDaT7x(s2If0H_iRSju7!z7p|8 zzI`NV!1hHWX3m)?t68k6yNKvop{Z>kl)f5GV(~1InT4%9IxqhDX-rgj)Y|NYq_NTlZgz-)=Y$=x9L7|k0=m@6WQ<4&r=BX@pW25NtCI+N{e&`RGSpR zeb^`@FHm5?pWseZ6V08{R(ki}--13S2op~9Kzz;#cPgL}Tmrqd+gs(fJLTCM8#&|S z^L+7PbAhltJDyyxAVxqf(2h!RGC3$;hX@YNz@&JRw!m5?Q)|-tZ8u0D$4we+QytG^ zj0U_@+N|OJlBHdWPN!K={a$R1Zi{2%5QD}s&s-Xn1tY1cwh)8VW z$pjq>8sj4)?76EJs6bA0E&pfr^Vq`&Xc;Tl2T!fm+MV%!H|i0o;7A=zE?dl)-Iz#P zSY7QRV`qRc6b&rON`BValC01zSLQpVemH5y%FxK8m^PeNN(Hf1(%C}KPfC*L?Nm!nMW0@J3(J=mYq3DPk;TMs%h`-amWbc%7{1Lg3$ z^e=btuqch-lydbtLvazh+fx?87Q7!YRT(=-Vx;hO)?o@f1($e5B?JB9jcRd;zM;iE zu?3EqyK`@_5Smr#^a`C#M>sRwq2^|ym)X*r;0v6AM`Zz1aK94@9Ti)Lixun2N!e-A z>w#}xPxVd9AfaF$XTTff?+#D(xwOpjZj9-&SU%7Z-E2-VF-n#xnPeQH*67J=j>TL# z<v}>AiTXrQ(fYa%82%qlH=L z6Fg8@r4p+BeTZ!5cZlu$iR?EJpYuTx>cJ~{{B7KODY#o*2seq=p2U0Rh;3mX^9sza zk^R_l7jzL5BXWlrVkhh!+LQ-Nc0I`6l1mWkp~inn)HQWqMTWl4G-TBLglR~n&6J?4 z7J)IO{wkrtT!Csntw3H$Mnj>@;QbrxC&Shqn^VVu$Ls*_c~TTY~fri6fO-=eJsC*8(3(H zSyO>=B;G`qA398OvCHRvf3mabrPZaaLhn*+jeA`qI!gP&i8Zs!*bBqMXDJpSZG$N) zx0rDLvcO>EoqCTR)|n7eOp-jmd>`#w`6`;+9+hihW2WnKVPQ20LR94h+(p)R$Y!Q zj_3ZEY+e@NH0f6VjLND)sh+Cvfo3CpcXw?`$@a^@CyLrAKIpjL8G z`;cDLqvK=ER)$q)+6vMKlxn!!SzWl>Ib9Ys9L)L0IWr*Ox;Rk#(Dpqf;wapY_EYL8 zKFrV)Q8BBKO4$r2hON%g=r@lPE;kBUVYVG`uxx~QI>9>MCXw_5vnmDsm|^KRny929 zeKx>F(LDs#K4FGU*k3~GX`A!)l8&|tyan-rBHBm6XaB5hc5sGKWwibAD7&3M-gh1n z2?eI7E2u{(^z#W~wU~dHSfy|m)%PY454NBxED)y-T3AO`CLQxklcC1I@Y`v4~SEI#Cm> z-cjqK6I?mypZapi$ZK;y&G+|#D=woItrajg69VRD+Fu8*UxG6KdfFmFLE}HvBJ~Y) zC&c-hr~;H2Idnsz7_F~MKpBZldh)>itc1AL0>4knbVy#%pUB&9vqL1Kg*^aU`k#(p z=A%lur(|$GWSqILaWZ#2xj(&lheSiA|N6DOG?A|$!aYM)?oME6ngnfLw0CA79WA+y zhUeLbMw*VB?drVE_D~3DWVaD>8x?_q>f!6;)i3@W<=kBZBSE=uIU60SW)qct?AdM zXgti8&O=}QNd|u%Fpxr172Kc`sX^@fm>Fxl8fbFalJYci_GGoIzU*~U*I!QLz? z4NYk^=JXBS*Uph@51da-v;%?))cB^(ps}y8yChu7CzyC9SX{jAq13zdnqRHRvc{ha zcPmgCUqAJ^1RChMCCz;ZN*ap{JPoE<1#8nNObDbAt6Jr}Crq#xGkK@w2mLhIUecvy z#?s~?J()H*?w9K`_;S+8TNVkHSk}#yvn+|~jcB|he}OY(zH|7%EK%-Tq=)18730)v zM3f|=oFugXq3Lqn={L!wx|u(ycZf(Te11c3?^8~aF; zNMC)gi?nQ#S$s{46yImv_7@4_qu|XXEza~);h&cr*~dO@#$LtKZa@@r$8PD^jz{D6 zk~5;IJBuQjsKk+8i0wzLJ2=toMw4@rw7(|6`7*e|V(5-#ZzRirtkXBO1oshQ&0>z&HAtSF8+871e|ni4gLs#`3v7gnG#^F zDv!w100_HwtU}B2T!+v_YDR@-9VmoGW+a76oo4yy)o`MY(a^GcIvXW+4)t{lK}I-& zl-C=(w_1Z}tsSFjFd z3iZjkO6xnjLV3!EE?ex9rb1Zxm)O-CnWPat4vw08!GtcQ3lHD+ySRB*3zQu-at$rj zzBn`S?5h=JlLXX8)~Jp%1~YS6>M8c-Mv~E%s7_RcvIYjc-ia`3r>dvjxZ6=?6=#OM zfsv}?hGnMMdi9C`J9+g)5`M9+S79ug=!xE_XcHdWnIRr&hq$!X7aX5kJV8Q(6Lq?|AE8N2H z37j{DPDY^Jw!J>~>Mwaja$g%q1sYfH4bUJFOR`x=pZQ@O(-4b#5=_Vm(0xe!LW>YF zO4w`2C|Cu%^C9q9B>NjFD{+qt)cY3~(09ma%mp3%cjFsj0_93oVHC3)AsbBPuQNBO z`+zffU~AgGrE0K{NVR}@oxB4&XWt&pJ-mq!JLhFWbnXf~H%uU?6N zWJ7oa@``Vi$pMWM#7N9=sX1%Y+1qTGnr_G&h3YfnkHPKG}p>i{fAG+(klE z(g~u_rJXF48l1D?;;>e}Ra{P$>{o`jR_!s{hV1Wk`vURz`W2c$-#r9GM7jgs2>um~ zouGlCm92rOiLITzf`jgl`v2qYw^!Lh0YwFHO1|3Krp8ztE}?#2+>c)yQlNw%5e6w5 zIm9BKZN5Q9b!tX`Zo$0RD~B)VscWp(FR|!a!{|Q$={;ZWl%10vBzfgWn}WBe!%cug z^G%;J-L4<6&aCKx@@(Grsf}dh8fuGT+TmhhA)_16uB!t{HIAK!B-7fJLe9fsF)4G- zf>(~ⅅ8zCNKueM5c!$)^mKpZNR!eIlFST57ePGQcqCqedAQ3UaUEzpjM--5V4YO zY22VxQm%$2NDnwfK+jkz=i2>NjAM6&P1DdcO<*Xs1-lzdXWn#LGSxwhPH7N%D8-zCgpFWt@`LgNYI+Fh^~nSiQmwH0^>E>*O$47MqfQza@Ce z1wBw;igLc#V2@y-*~Hp?jA1)+MYYyAt|DV_8RQCrRY@sAviO}wv;3gFdO>TE(=9o? z=S(r=0oT`w24=ihA=~iFV5z$ZG74?rmYn#eanx(!Hkxcr$*^KRFJKYYB&l6$WVsJ^ z-Iz#HYmE)Da@&seqG1fXsTER#adA&OrD2-T(z}Cwby|mQf{0v*v3hq~pzF`U`jenT z=XHXeB|fa?Ws$+9ADO0rco{#~+`VM?IXg7N>M0w1fyW1iiKTA@p$y zSiAJ%-Mg{m>&S4r#Tw@?@7ck}#oFo-iZJCWc`hw_J$=rw?omE{^tc59ftd`xq?jzf zo0bFUI=$>O!45{!c4?0KsJmZ#$vuYpZLo_O^oHTmmLMm0J_a{Nn`q5tG1m=0ecv$T z5H7r0DZGl6be@aJ+;26EGw9JENj0oJ5K0=^f-yBW2I0jqVIU};NBp*gF7_KlQnhB6 z##d$H({^HXj@il`*4^kC42&3)(A|tuhs;LygA-EWFSqpe+%#?6HG6}mE215Z4mjO2 zY2^?5$<8&k`O~#~sSc5Fy`5hg5#e{kG>SAbTxCh{y32fHkNryU_c0_6h&$zbWc63T z7|r?X7_H!9XK!HfZ+r?FvBQ$x{HTGS=1VN<>Ss-7M3z|vQG|N}Frv{h-q623@Jz*@ ziXlZIpAuY^RPlu&=nO)pFhML5=ut~&zWDSsn%>mv)!P1|^M!d5AwmSPIckoY|0u9I zTDAzG*U&5SPf+@c_tE_I!~Npfi$?gX(kn=zZd|tUZ_ez(xP+)xS!8=k(<{9@<+EUx zYQgZhjn(0qA#?~Q+EA9oh_Jx5PMfE3#KIh#*cFIFQGi)-40NHbJO&%ZvL|LAqU=Rw zf?Vr4qkUcKtLr^g-6*N-tfk+v8@#Lpl~SgKyH!+m9?T8B>WDWK22;!i5&_N=%f{__ z-LHb`v-LvKqTJZCx~z|Yg;U_f)VZu~q7trb%C6fOKs#eJosw&b$nmwGwP;Bz`=zK4 z>U3;}T_ptP)w=vJaL8EhW;J#SHA;fr13f=r#{o)`dRMOs-T;lp&Toi@u^oB_^pw=P zp#8Geo2?@!h2EYHY?L;ayT}-Df0?TeUCe8Cto{W0_a>!7Gxmi5G-nIIS;X{flm2De z{SjFG%knZoVa;mtHR_`*6)KEf=dvOT3OgT7C7&-4P#4X^B%VI&_57cBbli()(%zZC?Y0b;?5!f22UleQ=9h4_LkcA!Xsqx@q{ko&tvP_V@7epFs}AIpM{g??PA>U(sk$Gum>2Eu zD{Oy{$OF%~?B6>ixQeK9I}!$O0!T3#Ir8MW)j2V*qyJ z8Bg17L`rg^B_#rkny-=<3fr}Y42+x0@q6POk$H^*p3~Dc@5uYTQ$pfaRnIT}Wxb;- zl!@kkZkS=l)&=y|21veY8yz$t-&7ecA)TR|=51BKh(@n|d$EN>18)9kSQ|GqP?aeM ztXd9C&Md$PPF*FVs*GhoHM2L@D$(Qf%%x zwQBUt!jM~GgwluBcwkgwQ!249uPkNz3u@LSYZgmpHgX|P#8!iKk^vSKZ;?)KE$92d z2U>y}VWJ0&zjrIqddM3dz-nU%>bL&KU%SA|LiiUU7Ka|c=jF|vQ1V)Jz`JZe*j<5U6~RVuBEVJoY~ z&GE+F$f>4lN=X4-|9v*5O*Os>>r87u z!_1NSV?_X&HeFR1fOFb8_P)4lybJ6?1BWK`Tv2;4t|x1<#@17UO|hLGnrB%nu)fDk zfstJ4{X4^Y<8Lj<}g2^kksSefQTMuTo?tJLCh zC~>CR#a0hADw!_Vg*5fJwV{~S(j8)~sn>Oyt(ud2$1YfGck77}xN@3U_#T`q)f9!2 zf>Ia;Gwp2_C>WokU%(z2ec8z94pZyhaK+e>3a9sj^-&*V494;p9-xk+u1Jn#N_&xs z59OI2w=PuTErv|aNcK*>3l^W*p3}fjXJjJAXtBA#%B(-0--s;1U#f8gFYW!JL+iVG zV0SSx5w8eVgE?3Sg@eQv)=x<+-JgpVixZQNaZr}3b8sVyVs$@ndkF5FYKka@b+YAh z#nq_gzlIDKEs_i}H4f)(VQ!FSB}j>5znkVD&W0bOA{UZ7h!(FXrBbtdGA|PE1db>s z$!X)WY)u#7P8>^7Pjjj-kXNBuJX3(pJVetTZRNOnR5|RT5D>xmwxhAn)9KF3J05J; z-Mfb~dc?LUGqozC2p!1VjRqUwwDBnJhOua3vCCB-%ykW_ohSe?$R#dz%@Gym-8-RA zjMa_SJSzIl8{9dV+&63e9$4;{=1}w2=l+_j_Dtt@<(SYMbV-18&%F@Zl7F_5! z@xwJ0wiDdO%{}j9PW1(t+8P7Ud79yjY>x>aZYWJL_NI?bI6Y02`;@?qPz_PRqz(7v``20`- z033Dy|4;y6di|>cz|P-z|6c&3f&g^OAt8aN0Zd&0yZ>dq2aFCsE<~Ucf$v{sL=*++ zBxFSa2lfA+Y%U@B&3D=&CBO&u`#*nNc|PCY7XO<}MnG0VR764XrHtrb5zwC*2F!Lp zE<~Vj0;z!S-|3M4DFxuQ=`ShTf28<9p!81(0hFbGNqF%0gg*orez9!qt8e%o@Yfl@ zhvY}{@3&f??}7<`p>FyU;7?VkKbh8_=csozU=|fH&szgZ{=NDCylQ>EH^x5!K3~-V z)_2Y>0uJ`Z0Pb58y`RL+&n@m9tJ)O<%q#&u#DAIt+-rRt0eSe1MTtMl@W)H$b3D)@ z*A-1bUgZI)>HdcI4&W>P4W5{-j=s5p5`cbQ+{(g0+RDnz!TR^mxSLu_y#SDVKrj8i zA^hi6>jMGM;`$9Vfb-Yf!47b)Ow`2OKtNB=z|Kxa$5O}WPo;(Dc^`q(7X8kkeFyO8 z{XOq^07=u|7*P2`m;>PIFf=i80MKUxsN{d2cX0M+REsE*20+WQ79T9&cqT>=I_U% z{=8~^Isg(Nzo~`4iQfIb_#CVCD>#5h>=-Z#5dH}WxYzn%0)GAm6L2WdUdP=0_h>7f z(jh&7%1i(ZOn+}D8$iGK4Vs{pmHl_w4Qm-46H9>4^{3dz^DZDh+dw)6Xd@CpQNK$j z{CU;-cmpK=egplZ3y3%y=sEnCJ^eYVKXzV8H2_r*fJ*%*B;a1_lOpt6)IT1IAK2eB z{rie|uDJUrbgfUE>~C>@RO|m5ex55F{=~Bb4Cucp{ok7Yf9V}QuZ`#Gc|WaqsQlK- zKaV)iMRR__&Ak2Z=IM9R9g5$WM4u{a^C-7uX*!myEym z#_#p^T!P~#Dx$%^K>Y_nj_3J*E_LwJ60-5Xu=LkJAwcP@|0;a&+|+ZX`Jbj9P5;T% z|KOc}4*#4o{U?09`9Hz`Xo-I!P=9XfIrr*MQ}y=$!qgv?_J38^bNb4kM&_OVg^_=Eu-qG5U(fw0KMgH){C8pazq~51rN97hf#20-7=aK0)N|UM H-+%o-(+5aQ literal 0 HcmV?d00001 diff --git a/app_flutter/android/gradle/wrapper/gradle-wrapper.properties b/app_flutter/android/gradle/wrapper/gradle-wrapper.properties new file mode 100644 index 00000000..559efb4c --- /dev/null +++ b/app_flutter/android/gradle/wrapper/gradle-wrapper.properties @@ -0,0 +1,5 @@ +distributionBase=GRADLE_USER_HOME +distributionPath=wrapper/dists +zipStoreBase=GRADLE_USER_HOME +zipStorePath=wrapper/dists +distributionUrl=https\://services.gradle.org/distributions/gradle-8.5-bin.zip diff --git a/app_flutter/android/gradlew b/app_flutter/android/gradlew new file mode 100755 index 00000000..9d82f789 --- /dev/null +++ b/app_flutter/android/gradlew @@ -0,0 +1,160 @@ +#!/usr/bin/env bash + +############################################################################## +## +## Gradle start up script for UN*X +## +############################################################################## + +# Add default JVM options here. You can also use JAVA_OPTS and GRADLE_OPTS to pass JVM options to this script. +DEFAULT_JVM_OPTS="" + +APP_NAME="Gradle" +APP_BASE_NAME=`basename "$0"` + +# Use the maximum available, or set MAX_FD != -1 to use that value. +MAX_FD="maximum" + +warn ( ) { + echo "$*" +} + +die ( ) { + echo + echo "$*" + echo + exit 1 +} + +# OS specific support (must be 'true' or 'false'). +cygwin=false +msys=false +darwin=false +case "`uname`" in + CYGWIN* ) + cygwin=true + ;; + Darwin* ) + darwin=true + ;; + MINGW* ) + msys=true + ;; +esac + +# Attempt to set APP_HOME +# Resolve links: $0 may be a link +PRG="$0" +# Need this for relative symlinks. +while [ -h "$PRG" ] ; do + ls=`ls -ld "$PRG"` + link=`expr "$ls" : '.*-> \(.*\)$'` + if expr "$link" : '/.*' > /dev/null; then + PRG="$link" + else + PRG=`dirname "$PRG"`"/$link" + fi +done +SAVED="`pwd`" +cd "`dirname \"$PRG\"`/" >/dev/null +APP_HOME="`pwd -P`" +cd "$SAVED" >/dev/null + +CLASSPATH=$APP_HOME/gradle/wrapper/gradle-wrapper.jar + +# Determine the Java command to use to start the JVM. +if [ -n "$JAVA_HOME" ] ; then + if [ -x "$JAVA_HOME/jre/sh/java" ] ; then + # IBM's JDK on AIX uses strange locations for the executables + JAVACMD="$JAVA_HOME/jre/sh/java" + else + JAVACMD="$JAVA_HOME/bin/java" + fi + if [ ! -x "$JAVACMD" ] ; then + die "ERROR: JAVA_HOME is set to an invalid directory: $JAVA_HOME + +Please set the JAVA_HOME variable in your environment to match the +location of your Java installation." + fi +else + JAVACMD="java" + which java >/dev/null 2>&1 || die "ERROR: JAVA_HOME is not set and no 'java' command could be found in your PATH. + +Please set the JAVA_HOME variable in your environment to match the +location of your Java installation." +fi + +# Increase the maximum file descriptors if we can. +if [ "$cygwin" = "false" -a "$darwin" = "false" ] ; then + MAX_FD_LIMIT=`ulimit -H -n` + if [ $? -eq 0 ] ; then + if [ "$MAX_FD" = "maximum" -o "$MAX_FD" = "max" ] ; then + MAX_FD="$MAX_FD_LIMIT" + fi + ulimit -n $MAX_FD + if [ $? -ne 0 ] ; then + warn "Could not set maximum file descriptor limit: $MAX_FD" + fi + else + warn "Could not query maximum file descriptor limit: $MAX_FD_LIMIT" + fi +fi + +# For Darwin, add options to specify how the application appears in the dock +if $darwin; then + GRADLE_OPTS="$GRADLE_OPTS \"-Xdock:name=$APP_NAME\" \"-Xdock:icon=$APP_HOME/media/gradle.icns\"" +fi + +# For Cygwin, switch paths to Windows format before running java +if $cygwin ; then + APP_HOME=`cygpath --path --mixed "$APP_HOME"` + CLASSPATH=`cygpath --path --mixed "$CLASSPATH"` + JAVACMD=`cygpath --unix "$JAVACMD"` + + # We build the pattern for arguments to be converted via cygpath + ROOTDIRSRAW=`find -L / -maxdepth 1 -mindepth 1 -type d 2>/dev/null` + SEP="" + for dir in $ROOTDIRSRAW ; do + ROOTDIRS="$ROOTDIRS$SEP$dir" + SEP="|" + done + OURCYGPATTERN="(^($ROOTDIRS))" + # Add a user-defined pattern to the cygpath arguments + if [ "$GRADLE_CYGPATTERN" != "" ] ; then + OURCYGPATTERN="$OURCYGPATTERN|($GRADLE_CYGPATTERN)" + fi + # Now convert the arguments - kludge to limit ourselves to /bin/sh + i=0 + for arg in "$@" ; do + CHECK=`echo "$arg"|egrep -c "$OURCYGPATTERN" -` + CHECK2=`echo "$arg"|egrep -c "^-"` ### Determine if an option + + if [ $CHECK -ne 0 ] && [ $CHECK2 -eq 0 ] ; then ### Added a condition + eval `echo args$i`=`cygpath --path --ignore --mixed "$arg"` + else + eval `echo args$i`="\"$arg\"" + fi + i=$((i+1)) + done + case $i in + (0) set -- ;; + (1) set -- "$args0" ;; + (2) set -- "$args0" "$args1" ;; + (3) set -- "$args0" "$args1" "$args2" ;; + (4) set -- "$args0" "$args1" "$args2" "$args3" ;; + (5) set -- "$args0" "$args1" "$args2" "$args3" "$args4" ;; + (6) set -- "$args0" "$args1" "$args2" "$args3" "$args4" "$args5" ;; + (7) set -- "$args0" "$args1" "$args2" "$args3" "$args4" "$args5" "$args6" ;; + (8) set -- "$args0" "$args1" "$args2" "$args3" "$args4" "$args5" "$args6" "$args7" ;; + (9) set -- "$args0" "$args1" "$args2" "$args3" "$args4" "$args5" "$args6" "$args7" "$args8" ;; + esac +fi + +# Split up the JVM_OPTS And GRADLE_OPTS values into an array, following the shell quoting and substitution rules +function splitJvmOpts() { + JVM_OPTS=("$@") +} +eval splitJvmOpts $DEFAULT_JVM_OPTS $JAVA_OPTS $GRADLE_OPTS +JVM_OPTS[${#JVM_OPTS[*]}]="-Dorg.gradle.appname=$APP_BASE_NAME" + +exec "$JAVACMD" "${JVM_OPTS[@]}" -classpath "$CLASSPATH" org.gradle.wrapper.GradleWrapperMain "$@" diff --git a/app_flutter/android/gradlew.bat b/app_flutter/android/gradlew.bat new file mode 100644 index 00000000..8a0b282a --- /dev/null +++ b/app_flutter/android/gradlew.bat @@ -0,0 +1,90 @@ +@if "%DEBUG%" == "" @echo off +@rem ########################################################################## +@rem +@rem Gradle startup script for Windows +@rem +@rem ########################################################################## + +@rem Set local scope for the variables with windows NT shell +if "%OS%"=="Windows_NT" setlocal + +@rem Add default JVM options here. You can also use JAVA_OPTS and GRADLE_OPTS to pass JVM options to this script. +set DEFAULT_JVM_OPTS= + +set DIRNAME=%~dp0 +if "%DIRNAME%" == "" set DIRNAME=. +set APP_BASE_NAME=%~n0 +set APP_HOME=%DIRNAME% + +@rem Find java.exe +if defined JAVA_HOME goto findJavaFromJavaHome + +set JAVA_EXE=java.exe +%JAVA_EXE% -version >NUL 2>&1 +if "%ERRORLEVEL%" == "0" goto init + +echo. +echo ERROR: JAVA_HOME is not set and no 'java' command could be found in your PATH. +echo. +echo Please set the JAVA_HOME variable in your environment to match the +echo location of your Java installation. + +goto fail + +:findJavaFromJavaHome +set JAVA_HOME=%JAVA_HOME:"=% +set JAVA_EXE=%JAVA_HOME%/bin/java.exe + +if exist "%JAVA_EXE%" goto init + +echo. +echo ERROR: JAVA_HOME is set to an invalid directory: %JAVA_HOME% +echo. +echo Please set the JAVA_HOME variable in your environment to match the +echo location of your Java installation. + +goto fail + +:init +@rem Get command-line arguments, handling Windowz variants + +if not "%OS%" == "Windows_NT" goto win9xME_args +if "%@eval[2+2]" == "4" goto 4NT_args + +:win9xME_args +@rem Slurp the command line arguments. +set CMD_LINE_ARGS= +set _SKIP=2 + +:win9xME_args_slurp +if "x%~1" == "x" goto execute + +set CMD_LINE_ARGS=%* +goto execute + +:4NT_args +@rem Get arguments from the 4NT Shell from JP Software +set CMD_LINE_ARGS=%$ + +:execute +@rem Setup the command line + +set CLASSPATH=%APP_HOME%\gradle\wrapper\gradle-wrapper.jar + +@rem Execute Gradle +"%JAVA_EXE%" %DEFAULT_JVM_OPTS% %JAVA_OPTS% %GRADLE_OPTS% "-Dorg.gradle.appname=%APP_BASE_NAME%" -classpath "%CLASSPATH%" org.gradle.wrapper.GradleWrapperMain %CMD_LINE_ARGS% + +:end +@rem End local scope for the variables with windows NT shell +if "%ERRORLEVEL%"=="0" goto mainEnd + +:fail +rem Set variable GRADLE_EXIT_CONSOLE if you need the _script_ return code instead of +rem the _cmd.exe /c_ return code! +if not "" == "%GRADLE_EXIT_CONSOLE%" exit 1 +exit /b 1 + +:mainEnd +if "%OS%"=="Windows_NT" endlocal + +:omega diff --git a/app_flutter/android/settings.gradle b/app_flutter/android/settings.gradle new file mode 100644 index 00000000..08a15043 --- /dev/null +++ b/app_flutter/android/settings.gradle @@ -0,0 +1,29 @@ +pluginManagement { + def flutterSdkPath = { + def properties = new Properties() + file("local.properties").withInputStream { properties.load(it) } + def flutterSdkPath = properties.getProperty("flutter.sdk") + assert flutterSdkPath != null, "flutter.sdk not set in local.properties" + return flutterSdkPath + } + settings.ext.flutterSdkPath = flutterSdkPath() + + includeBuild("${settings.ext.flutterSdkPath}/packages/flutter_tools/gradle") + + repositories { + google() + mavenCentral() + gradlePluginPortal() + } + + plugins { + id "dev.flutter.flutter-gradle-plugin" version "1.0.0" apply false + } +} + +plugins { + id "dev.flutter.flutter-plugin-loader" version "1.0.0" + id "com.android.application" version "8.2.2" apply false +} + +include ":app" diff --git a/app_flutter/lib/main.dart b/app_flutter/lib/main.dart new file mode 100644 index 00000000..5a082922 --- /dev/null +++ b/app_flutter/lib/main.dart @@ -0,0 +1,411 @@ +import 'package:flutter/material.dart'; + +void main() { + runApp(const UpgradeAllApp()); +} + +@visibleForTesting +class AppKeys { + static const homeRoute = ValueKey('route.home'); + static const appsRoute = ValueKey('route.apps'); + static const appDetailRoute = ValueKey('route.app_detail'); + static const repositoriesRoute = ValueKey('route.repositories'); + static const downloadsRoute = ValueKey('route.downloads'); + static const logsRoute = ValueKey('route.logs'); + static const settingsRoute = ValueKey('route.settings'); + static const migrationRoute = ValueKey('route.migration'); + + static const openApps = ValueKey('action.open_apps'); + static const openRepositories = ValueKey('action.open_repositories'); + static const openDownloads = ValueKey('action.open_downloads'); + static const openLogs = ValueKey('action.open_logs'); + static const openSettings = ValueKey('action.open_settings'); + static const openMigration = ValueKey('action.open_migration'); + static const openFirstApp = ValueKey('action.open_first_app'); + + static const updateSummary = ValueKey('state.update_summary'); + static const getterStatus = ValueKey('state.getter_status'); + static const appsList = ValueKey('state.apps_list'); + static const repositoriesList = ValueKey('state.repositories_list'); + static const downloadsEmpty = ValueKey('state.downloads_empty'); + static const logsEmpty = ValueKey('state.logs_empty'); + static const settingsShell = ValueKey('state.settings_shell'); + static const migrationReady = ValueKey('state.migration_ready'); + + static ValueKey appRow(String packageId) => + ValueKey('state.app.$packageId'); + static ValueKey repoRow(String repositoryId) => + ValueKey('state.repository.$repositoryId'); +} + +class UpgradeAllApp extends StatelessWidget { + const UpgradeAllApp({super.key, this.getter = const FakeGetterAdapter()}); + + final GetterAdapter getter; + + @override + Widget build(BuildContext context) { + return MaterialApp( + title: 'UpgradeAll', + theme: ThemeData( + colorScheme: ColorScheme.fromSeed(seedColor: Colors.blue), + useMaterial3: true, + ), + routes: { + '/': (context) => HomePage(getter: getter), + '/apps': (context) => AppsPage(getter: getter), + '/repositories': (context) => RepositoriesPage(getter: getter), + '/downloads': (context) => const DownloadsPage(), + '/logs': (context) => const LogsPage(), + '/settings': (context) => const SettingsPage(), + '/migration': (context) => const MigrationPage(), + }, + onGenerateRoute: (settings) { + if (settings.name == '/apps/detail') { + final app = settings.arguments! as AppSummary; + return MaterialPageRoute( + builder: (context) => AppDetailPage(app: app), + settings: settings, + ); + } + return null; + }, + ); + } +} + +abstract interface class GetterAdapter { + GetterSnapshot loadSnapshot(); +} + +class FakeGetterAdapter implements GetterAdapter { + const FakeGetterAdapter(); + + @override + GetterSnapshot loadSnapshot() { + return const GetterSnapshot( + status: 'Fake getter ready', + updateCount: 0, + apps: [ + AppSummary( + id: 'android/org.fdroid.fdroid', + name: 'F-Droid', + installedVersion: '1.20.0', + latestVersion: '1.20.0', + hasFreeNetworkWarning: true, + ), + ], + repositories: [ + RepositorySummary(id: 'local', priority: 100), + RepositorySummary(id: 'official', priority: 0), + RepositorySummary(id: 'local_autogen', priority: -1), + ], + ); + } +} + +class GetterSnapshot { + const GetterSnapshot({ + required this.status, + required this.updateCount, + required this.apps, + required this.repositories, + }); + + final String status; + final int updateCount; + final List apps; + final List repositories; +} + +class AppSummary { + const AppSummary({ + required this.id, + required this.name, + required this.installedVersion, + required this.latestVersion, + required this.hasFreeNetworkWarning, + }); + + final String id; + final String name; + final String installedVersion; + final String latestVersion; + final bool hasFreeNetworkWarning; +} + +class RepositorySummary { + const RepositorySummary({required this.id, required this.priority}); + + final String id; + final int priority; +} + +class HomePage extends StatelessWidget { + const HomePage({super.key, required this.getter}); + + final GetterAdapter getter; + + @override + Widget build(BuildContext context) { + final snapshot = getter.loadSnapshot(); + return Scaffold( + key: AppKeys.homeRoute, + appBar: AppBar(title: const Text('UpgradeAll')), + body: ListView( + padding: const EdgeInsets.all(16), + children: [ + Card( + key: AppKeys.updateSummary, + child: ListTile( + title: const Text('Updates'), + subtitle: Text('${snapshot.updateCount} updates available'), + ), + ), + Card( + key: AppKeys.getterStatus, + child: ListTile( + title: const Text('Getter core'), + subtitle: Text(snapshot.status), + ), + ), + const SizedBox(height: 16), + const _RouteButton( + key: AppKeys.openApps, + icon: Icons.apps, + label: 'Apps', + routeName: '/apps', + ), + const _RouteButton( + key: AppKeys.openRepositories, + icon: Icons.source, + label: 'Repositories', + routeName: '/repositories', + ), + const _RouteButton( + key: AppKeys.openDownloads, + icon: Icons.download, + label: 'Downloads', + routeName: '/downloads', + ), + const _RouteButton( + key: AppKeys.openLogs, + icon: Icons.receipt_long, + label: 'Logs', + routeName: '/logs', + ), + const _RouteButton( + key: AppKeys.openSettings, + icon: Icons.settings, + label: 'Settings', + routeName: '/settings', + ), + const _RouteButton( + key: AppKeys.openMigration, + icon: Icons.move_down, + label: 'Legacy migration', + routeName: '/migration', + ), + ], + ), + ); + } +} + +class AppsPage extends StatelessWidget { + const AppsPage({super.key, required this.getter}); + + final GetterAdapter getter; + + @override + Widget build(BuildContext context) { + final apps = getter.loadSnapshot().apps; + return Scaffold( + key: AppKeys.appsRoute, + appBar: AppBar(title: const Text('Apps')), + body: ListView.builder( + key: AppKeys.appsList, + itemCount: apps.length, + itemBuilder: (context, index) { + final app = apps[index]; + return ListTile( + key: AppKeys.appRow(app.id), + title: Text(app.name), + subtitle: Text('${app.id} • ${app.installedVersion}'), + trailing: app.hasFreeNetworkWarning + ? const Chip( + label: Text('Network'), + backgroundColor: Colors.amber, + ) + : null, + onTap: () { + Navigator.of(context).pushNamed('/apps/detail', arguments: app); + }, + ); + }, + ), + ); + } +} + +class AppDetailPage extends StatelessWidget { + const AppDetailPage({super.key, required this.app}); + + final AppSummary app; + + @override + Widget build(BuildContext context) { + return Scaffold( + key: AppKeys.appDetailRoute, + appBar: AppBar(title: Text(app.name)), + body: ListView( + padding: const EdgeInsets.all(16), + children: [ + Text(app.id, style: Theme.of(context).textTheme.titleMedium), + const SizedBox(height: 12), + Text('Installed: ${app.installedVersion}'), + Text('Latest: ${app.latestVersion}'), + if (app.hasFreeNetworkWarning) + const Padding( + padding: EdgeInsets.only(top: 12), + child: Chip( + label: Text('Network access required'), + backgroundColor: Colors.amber, + ), + ), + ], + ), + ); + } +} + +class RepositoriesPage extends StatelessWidget { + const RepositoriesPage({super.key, required this.getter}); + + final GetterAdapter getter; + + @override + Widget build(BuildContext context) { + final repositories = getter.loadSnapshot().repositories; + return Scaffold( + key: AppKeys.repositoriesRoute, + appBar: AppBar(title: const Text('Repositories')), + body: ListView.builder( + key: AppKeys.repositoriesList, + itemCount: repositories.length, + itemBuilder: (context, index) { + final repository = repositories[index]; + return ListTile( + key: AppKeys.repoRow(repository.id), + title: Text(repository.id), + subtitle: Text('Priority ${repository.priority}'), + ); + }, + ), + ); + } +} + +class DownloadsPage extends StatelessWidget { + const DownloadsPage({super.key}); + + @override + Widget build(BuildContext context) { + return const _PlaceholderPage( + key: AppKeys.downloadsRoute, + title: 'Downloads', + stateKey: AppKeys.downloadsEmpty, + message: 'No download tasks yet', + ); + } +} + +class LogsPage extends StatelessWidget { + const LogsPage({super.key}); + + @override + Widget build(BuildContext context) { + return const _PlaceholderPage( + key: AppKeys.logsRoute, + title: 'Logs', + stateKey: AppKeys.logsEmpty, + message: 'No getter events yet', + ); + } +} + +class SettingsPage extends StatelessWidget { + const SettingsPage({super.key}); + + @override + Widget build(BuildContext context) { + return const _PlaceholderPage( + key: AppKeys.settingsRoute, + title: 'Settings', + stateKey: AppKeys.settingsShell, + message: 'Settings shell ready', + ); + } +} + +class MigrationPage extends StatelessWidget { + const MigrationPage({super.key}); + + @override + Widget build(BuildContext context) { + return const _PlaceholderPage( + key: AppKeys.migrationRoute, + title: 'Legacy migration', + stateKey: AppKeys.migrationReady, + message: 'Ready to show migration reports', + ); + } +} + +class _RouteButton extends StatelessWidget { + const _RouteButton({ + super.key, + required this.icon, + required this.label, + required this.routeName, + }); + + final IconData icon; + final String label; + final String routeName; + + @override + Widget build(BuildContext context) { + return Padding( + padding: const EdgeInsets.symmetric(vertical: 4), + child: FilledButton.icon( + onPressed: () => Navigator.of(context).pushNamed(routeName), + icon: Icon(icon), + label: Text(label), + ), + ); + } +} + +class _PlaceholderPage extends StatelessWidget { + const _PlaceholderPage({ + super.key, + required this.title, + required this.stateKey, + required this.message, + }); + + final String title; + final Key stateKey; + final String message; + + @override + Widget build(BuildContext context) { + return Scaffold( + appBar: AppBar(title: Text(title)), + body: Center( + child: Text(key: stateKey, message), + ), + ); + } +} diff --git a/app_flutter/linux/.gitignore b/app_flutter/linux/.gitignore new file mode 100644 index 00000000..d3896c98 --- /dev/null +++ b/app_flutter/linux/.gitignore @@ -0,0 +1 @@ +flutter/ephemeral diff --git a/app_flutter/linux/CMakeLists.txt b/app_flutter/linux/CMakeLists.txt new file mode 100644 index 00000000..7492bd8a --- /dev/null +++ b/app_flutter/linux/CMakeLists.txt @@ -0,0 +1,145 @@ +# Project-level configuration. +cmake_minimum_required(VERSION 3.10) +project(runner LANGUAGES CXX) + +# The name of the executable created for the application. Change this to change +# the on-disk name of your application. +set(BINARY_NAME "upgradeall") +# The unique GTK application identifier for this application. See: +# https://wiki.gnome.org/HowDoI/ChooseApplicationID +set(APPLICATION_ID "net.xzos.upgradeall") + +# Explicitly opt in to modern CMake behaviors to avoid warnings with recent +# versions of CMake. +cmake_policy(SET CMP0063 NEW) + +# Load bundled libraries from the lib/ directory relative to the binary. +set(CMAKE_INSTALL_RPATH "$ORIGIN/lib") + +# Root filesystem for cross-building. +if(FLUTTER_TARGET_PLATFORM_SYSROOT) + set(CMAKE_SYSROOT ${FLUTTER_TARGET_PLATFORM_SYSROOT}) + set(CMAKE_FIND_ROOT_PATH ${CMAKE_SYSROOT}) + set(CMAKE_FIND_ROOT_PATH_MODE_PROGRAM NEVER) + set(CMAKE_FIND_ROOT_PATH_MODE_PACKAGE ONLY) + set(CMAKE_FIND_ROOT_PATH_MODE_LIBRARY ONLY) + set(CMAKE_FIND_ROOT_PATH_MODE_INCLUDE ONLY) +endif() + +# Define build configuration options. +if(NOT CMAKE_BUILD_TYPE AND NOT CMAKE_CONFIGURATION_TYPES) + set(CMAKE_BUILD_TYPE "Debug" CACHE + STRING "Flutter build mode" FORCE) + set_property(CACHE CMAKE_BUILD_TYPE PROPERTY STRINGS + "Debug" "Profile" "Release") +endif() + +# Compilation settings that should be applied to most targets. +# +# Be cautious about adding new options here, as plugins use this function by +# default. In most cases, you should add new options to specific targets instead +# of modifying this function. +function(APPLY_STANDARD_SETTINGS TARGET) + target_compile_features(${TARGET} PUBLIC cxx_std_14) + target_compile_options(${TARGET} PRIVATE -Wall -Werror) + target_compile_options(${TARGET} PRIVATE "$<$>:-O3>") + target_compile_definitions(${TARGET} PRIVATE "$<$>:NDEBUG>") +endfunction() + +# Flutter library and tool build rules. +set(FLUTTER_MANAGED_DIR "${CMAKE_CURRENT_SOURCE_DIR}/flutter") +add_subdirectory(${FLUTTER_MANAGED_DIR}) + +# System-level dependencies. +find_package(PkgConfig REQUIRED) +pkg_check_modules(GTK REQUIRED IMPORTED_TARGET gtk+-3.0) + +add_definitions(-DAPPLICATION_ID="${APPLICATION_ID}") + +# Define the application target. To change its name, change BINARY_NAME above, +# not the value here, or `flutter run` will no longer work. +# +# Any new source files that you add to the application should be added here. +add_executable(${BINARY_NAME} + "main.cc" + "my_application.cc" + "${FLUTTER_MANAGED_DIR}/generated_plugin_registrant.cc" +) + +# Apply the standard set of build settings. This can be removed for applications +# that need different build settings. +apply_standard_settings(${BINARY_NAME}) + +# Add dependency libraries. Add any application-specific dependencies here. +target_link_libraries(${BINARY_NAME} PRIVATE flutter) +target_link_libraries(${BINARY_NAME} PRIVATE PkgConfig::GTK) + +# Run the Flutter tool portions of the build. This must not be removed. +add_dependencies(${BINARY_NAME} flutter_assemble) + +# Only the install-generated bundle's copy of the executable will launch +# correctly, since the resources must in the right relative locations. To avoid +# people trying to run the unbundled copy, put it in a subdirectory instead of +# the default top-level location. +set_target_properties(${BINARY_NAME} + PROPERTIES + RUNTIME_OUTPUT_DIRECTORY "${CMAKE_BINARY_DIR}/intermediates_do_not_run" +) + + +# Generated plugin build rules, which manage building the plugins and adding +# them to the application. +include(flutter/generated_plugins.cmake) + + +# === Installation === +# By default, "installing" just makes a relocatable bundle in the build +# directory. +set(BUILD_BUNDLE_DIR "${PROJECT_BINARY_DIR}/bundle") +if(CMAKE_INSTALL_PREFIX_INITIALIZED_TO_DEFAULT) + set(CMAKE_INSTALL_PREFIX "${BUILD_BUNDLE_DIR}" CACHE PATH "..." FORCE) +endif() + +# Start with a clean build bundle directory every time. +install(CODE " + file(REMOVE_RECURSE \"${BUILD_BUNDLE_DIR}/\") + " COMPONENT Runtime) + +set(INSTALL_BUNDLE_DATA_DIR "${CMAKE_INSTALL_PREFIX}/data") +set(INSTALL_BUNDLE_LIB_DIR "${CMAKE_INSTALL_PREFIX}/lib") + +install(TARGETS ${BINARY_NAME} RUNTIME DESTINATION "${CMAKE_INSTALL_PREFIX}" + COMPONENT Runtime) + +install(FILES "${FLUTTER_ICU_DATA_FILE}" DESTINATION "${INSTALL_BUNDLE_DATA_DIR}" + COMPONENT Runtime) + +install(FILES "${FLUTTER_LIBRARY}" DESTINATION "${INSTALL_BUNDLE_LIB_DIR}" + COMPONENT Runtime) + +foreach(bundled_library ${PLUGIN_BUNDLED_LIBRARIES}) + install(FILES "${bundled_library}" + DESTINATION "${INSTALL_BUNDLE_LIB_DIR}" + COMPONENT Runtime) +endforeach(bundled_library) + +# Copy the native assets provided by the build.dart from all packages. +set(NATIVE_ASSETS_DIR "${PROJECT_BUILD_DIR}native_assets/linux/") +install(DIRECTORY "${NATIVE_ASSETS_DIR}" + DESTINATION "${INSTALL_BUNDLE_LIB_DIR}" + COMPONENT Runtime) + +# Fully re-copy the assets directory on each build to avoid having stale files +# from a previous install. +set(FLUTTER_ASSET_DIR_NAME "flutter_assets") +install(CODE " + file(REMOVE_RECURSE \"${INSTALL_BUNDLE_DATA_DIR}/${FLUTTER_ASSET_DIR_NAME}\") + " COMPONENT Runtime) +install(DIRECTORY "${PROJECT_BUILD_DIR}/${FLUTTER_ASSET_DIR_NAME}" + DESTINATION "${INSTALL_BUNDLE_DATA_DIR}" COMPONENT Runtime) + +# Install the AOT library on non-Debug builds only. +if(NOT CMAKE_BUILD_TYPE MATCHES "Debug") + install(FILES "${AOT_LIBRARY}" DESTINATION "${INSTALL_BUNDLE_LIB_DIR}" + COMPONENT Runtime) +endif() diff --git a/app_flutter/linux/flutter/CMakeLists.txt b/app_flutter/linux/flutter/CMakeLists.txt new file mode 100644 index 00000000..d5bd0164 --- /dev/null +++ b/app_flutter/linux/flutter/CMakeLists.txt @@ -0,0 +1,88 @@ +# This file controls Flutter-level build steps. It should not be edited. +cmake_minimum_required(VERSION 3.10) + +set(EPHEMERAL_DIR "${CMAKE_CURRENT_SOURCE_DIR}/ephemeral") + +# Configuration provided via flutter tool. +include(${EPHEMERAL_DIR}/generated_config.cmake) + +# TODO: Move the rest of this into files in ephemeral. See +# https://github.com/flutter/flutter/issues/57146. + +# Serves the same purpose as list(TRANSFORM ... PREPEND ...), +# which isn't available in 3.10. +function(list_prepend LIST_NAME PREFIX) + set(NEW_LIST "") + foreach(element ${${LIST_NAME}}) + list(APPEND NEW_LIST "${PREFIX}${element}") + endforeach(element) + set(${LIST_NAME} "${NEW_LIST}" PARENT_SCOPE) +endfunction() + +# === Flutter Library === +# System-level dependencies. +find_package(PkgConfig REQUIRED) +pkg_check_modules(GTK REQUIRED IMPORTED_TARGET gtk+-3.0) +pkg_check_modules(GLIB REQUIRED IMPORTED_TARGET glib-2.0) +pkg_check_modules(GIO REQUIRED IMPORTED_TARGET gio-2.0) + +set(FLUTTER_LIBRARY "${EPHEMERAL_DIR}/libflutter_linux_gtk.so") + +# Published to parent scope for install step. +set(FLUTTER_LIBRARY ${FLUTTER_LIBRARY} PARENT_SCOPE) +set(FLUTTER_ICU_DATA_FILE "${EPHEMERAL_DIR}/icudtl.dat" PARENT_SCOPE) +set(PROJECT_BUILD_DIR "${PROJECT_DIR}/build/" PARENT_SCOPE) +set(AOT_LIBRARY "${PROJECT_DIR}/build/lib/libapp.so" PARENT_SCOPE) + +list(APPEND FLUTTER_LIBRARY_HEADERS + "fl_basic_message_channel.h" + "fl_binary_codec.h" + "fl_binary_messenger.h" + "fl_dart_project.h" + "fl_engine.h" + "fl_json_message_codec.h" + "fl_json_method_codec.h" + "fl_message_codec.h" + "fl_method_call.h" + "fl_method_channel.h" + "fl_method_codec.h" + "fl_method_response.h" + "fl_plugin_registrar.h" + "fl_plugin_registry.h" + "fl_standard_message_codec.h" + "fl_standard_method_codec.h" + "fl_string_codec.h" + "fl_value.h" + "fl_view.h" + "flutter_linux.h" +) +list_prepend(FLUTTER_LIBRARY_HEADERS "${EPHEMERAL_DIR}/flutter_linux/") +add_library(flutter INTERFACE) +target_include_directories(flutter INTERFACE + "${EPHEMERAL_DIR}" +) +target_link_libraries(flutter INTERFACE "${FLUTTER_LIBRARY}") +target_link_libraries(flutter INTERFACE + PkgConfig::GTK + PkgConfig::GLIB + PkgConfig::GIO +) +add_dependencies(flutter flutter_assemble) + +# === Flutter tool backend === +# _phony_ is a non-existent file to force this command to run every time, +# since currently there's no way to get a full input/output list from the +# flutter tool. +add_custom_command( + OUTPUT ${FLUTTER_LIBRARY} ${FLUTTER_LIBRARY_HEADERS} + ${CMAKE_CURRENT_BINARY_DIR}/_phony_ + COMMAND ${CMAKE_COMMAND} -E env + ${FLUTTER_TOOL_ENVIRONMENT} + "${FLUTTER_ROOT}/packages/flutter_tools/bin/tool_backend.sh" + ${FLUTTER_TARGET_PLATFORM} ${CMAKE_BUILD_TYPE} + VERBATIM +) +add_custom_target(flutter_assemble DEPENDS + "${FLUTTER_LIBRARY}" + ${FLUTTER_LIBRARY_HEADERS} +) diff --git a/app_flutter/linux/flutter/generated_plugin_registrant.cc b/app_flutter/linux/flutter/generated_plugin_registrant.cc new file mode 100644 index 00000000..e71a16d2 --- /dev/null +++ b/app_flutter/linux/flutter/generated_plugin_registrant.cc @@ -0,0 +1,11 @@ +// +// Generated file. Do not edit. +// + +// clang-format off + +#include "generated_plugin_registrant.h" + + +void fl_register_plugins(FlPluginRegistry* registry) { +} diff --git a/app_flutter/linux/flutter/generated_plugin_registrant.h b/app_flutter/linux/flutter/generated_plugin_registrant.h new file mode 100644 index 00000000..e0f0a47b --- /dev/null +++ b/app_flutter/linux/flutter/generated_plugin_registrant.h @@ -0,0 +1,15 @@ +// +// Generated file. Do not edit. +// + +// clang-format off + +#ifndef GENERATED_PLUGIN_REGISTRANT_ +#define GENERATED_PLUGIN_REGISTRANT_ + +#include + +// Registers Flutter plugins. +void fl_register_plugins(FlPluginRegistry* registry); + +#endif // GENERATED_PLUGIN_REGISTRANT_ diff --git a/app_flutter/linux/flutter/generated_plugins.cmake b/app_flutter/linux/flutter/generated_plugins.cmake new file mode 100644 index 00000000..2e1de87a --- /dev/null +++ b/app_flutter/linux/flutter/generated_plugins.cmake @@ -0,0 +1,23 @@ +# +# Generated file, do not edit. +# + +list(APPEND FLUTTER_PLUGIN_LIST +) + +list(APPEND FLUTTER_FFI_PLUGIN_LIST +) + +set(PLUGIN_BUNDLED_LIBRARIES) + +foreach(plugin ${FLUTTER_PLUGIN_LIST}) + add_subdirectory(flutter/ephemeral/.plugin_symlinks/${plugin}/linux plugins/${plugin}) + target_link_libraries(${BINARY_NAME} PRIVATE ${plugin}_plugin) + list(APPEND PLUGIN_BUNDLED_LIBRARIES $) + list(APPEND PLUGIN_BUNDLED_LIBRARIES ${${plugin}_bundled_libraries}) +endforeach(plugin) + +foreach(ffi_plugin ${FLUTTER_FFI_PLUGIN_LIST}) + add_subdirectory(flutter/ephemeral/.plugin_symlinks/${ffi_plugin}/linux plugins/${ffi_plugin}) + list(APPEND PLUGIN_BUNDLED_LIBRARIES ${${ffi_plugin}_bundled_libraries}) +endforeach(ffi_plugin) diff --git a/app_flutter/linux/main.cc b/app_flutter/linux/main.cc new file mode 100644 index 00000000..e7c5c543 --- /dev/null +++ b/app_flutter/linux/main.cc @@ -0,0 +1,6 @@ +#include "my_application.h" + +int main(int argc, char** argv) { + g_autoptr(MyApplication) app = my_application_new(); + return g_application_run(G_APPLICATION(app), argc, argv); +} diff --git a/app_flutter/linux/my_application.cc b/app_flutter/linux/my_application.cc new file mode 100644 index 00000000..313ff25e --- /dev/null +++ b/app_flutter/linux/my_application.cc @@ -0,0 +1,104 @@ +#include "my_application.h" + +#include +#ifdef GDK_WINDOWING_X11 +#include +#endif + +#include "flutter/generated_plugin_registrant.h" + +struct _MyApplication { + GtkApplication parent_instance; + char** dart_entrypoint_arguments; +}; + +G_DEFINE_TYPE(MyApplication, my_application, GTK_TYPE_APPLICATION) + +// Implements GApplication::activate. +static void my_application_activate(GApplication* application) { + MyApplication* self = MY_APPLICATION(application); + GtkWindow* window = + GTK_WINDOW(gtk_application_window_new(GTK_APPLICATION(application))); + + // Use a header bar when running in GNOME as this is the common style used + // by applications and is the setup most users will be using (e.g. Ubuntu + // desktop). + // If running on X and not using GNOME then just use a traditional title bar + // in case the window manager does more exotic layout, e.g. tiling. + // If running on Wayland assume the header bar will work (may need changing + // if future cases occur). + gboolean use_header_bar = TRUE; +#ifdef GDK_WINDOWING_X11 + GdkScreen* screen = gtk_window_get_screen(window); + if (GDK_IS_X11_SCREEN(screen)) { + const gchar* wm_name = gdk_x11_screen_get_window_manager_name(screen); + if (g_strcmp0(wm_name, "GNOME Shell") != 0) { + use_header_bar = FALSE; + } + } +#endif + if (use_header_bar) { + GtkHeaderBar* header_bar = GTK_HEADER_BAR(gtk_header_bar_new()); + gtk_widget_show(GTK_WIDGET(header_bar)); + gtk_header_bar_set_title(header_bar, "upgradeall"); + gtk_header_bar_set_show_close_button(header_bar, TRUE); + gtk_window_set_titlebar(window, GTK_WIDGET(header_bar)); + } else { + gtk_window_set_title(window, "upgradeall"); + } + + gtk_window_set_default_size(window, 1280, 720); + gtk_widget_show(GTK_WIDGET(window)); + + g_autoptr(FlDartProject) project = fl_dart_project_new(); + fl_dart_project_set_dart_entrypoint_arguments(project, self->dart_entrypoint_arguments); + + FlView* view = fl_view_new(project); + gtk_widget_show(GTK_WIDGET(view)); + gtk_container_add(GTK_CONTAINER(window), GTK_WIDGET(view)); + + fl_register_plugins(FL_PLUGIN_REGISTRY(view)); + + gtk_widget_grab_focus(GTK_WIDGET(view)); +} + +// Implements GApplication::local_command_line. +static gboolean my_application_local_command_line(GApplication* application, gchar*** arguments, int* exit_status) { + MyApplication* self = MY_APPLICATION(application); + // Strip out the first argument as it is the binary name. + self->dart_entrypoint_arguments = g_strdupv(*arguments + 1); + + g_autoptr(GError) error = nullptr; + if (!g_application_register(application, nullptr, &error)) { + g_warning("Failed to register: %s", error->message); + *exit_status = 1; + return TRUE; + } + + g_application_activate(application); + *exit_status = 0; + + return TRUE; +} + +// Implements GObject::dispose. +static void my_application_dispose(GObject* object) { + MyApplication* self = MY_APPLICATION(object); + g_clear_pointer(&self->dart_entrypoint_arguments, g_strfreev); + G_OBJECT_CLASS(my_application_parent_class)->dispose(object); +} + +static void my_application_class_init(MyApplicationClass* klass) { + G_APPLICATION_CLASS(klass)->activate = my_application_activate; + G_APPLICATION_CLASS(klass)->local_command_line = my_application_local_command_line; + G_OBJECT_CLASS(klass)->dispose = my_application_dispose; +} + +static void my_application_init(MyApplication* self) {} + +MyApplication* my_application_new() { + return MY_APPLICATION(g_object_new(my_application_get_type(), + "application-id", APPLICATION_ID, + "flags", G_APPLICATION_NON_UNIQUE, + nullptr)); +} diff --git a/app_flutter/linux/my_application.h b/app_flutter/linux/my_application.h new file mode 100644 index 00000000..72271d5e --- /dev/null +++ b/app_flutter/linux/my_application.h @@ -0,0 +1,18 @@ +#ifndef FLUTTER_MY_APPLICATION_H_ +#define FLUTTER_MY_APPLICATION_H_ + +#include + +G_DECLARE_FINAL_TYPE(MyApplication, my_application, MY, APPLICATION, + GtkApplication) + +/** + * my_application_new: + * + * Creates a new Flutter-based application. + * + * Returns: a new #MyApplication. + */ +MyApplication* my_application_new(); + +#endif // FLUTTER_MY_APPLICATION_H_ diff --git a/app_flutter/pubspec.lock b/app_flutter/pubspec.lock new file mode 100644 index 00000000..81c00a98 --- /dev/null +++ b/app_flutter/pubspec.lock @@ -0,0 +1,188 @@ +# Generated by pub +# See https://dart.dev/tools/pub/glossary#lockfile +packages: + async: + dependency: transitive + description: + name: async + sha256: "947bfcf187f74dbc5e146c9eb9c0f10c9f8b30743e341481c1e2ed3ecc18c20c" + url: "https://pub.dev" + source: hosted + version: "2.11.0" + boolean_selector: + dependency: transitive + description: + name: boolean_selector + sha256: "6cfb5af12253eaf2b368f07bacc5a80d1301a071c73360d746b7f2e32d762c66" + url: "https://pub.dev" + source: hosted + version: "2.1.1" + characters: + dependency: transitive + description: + name: characters + sha256: "04a925763edad70e8443c99234dc3328f442e811f1d8fd1a72f1c8ad0f69a605" + url: "https://pub.dev" + source: hosted + version: "1.3.0" + clock: + dependency: transitive + description: + name: clock + sha256: cb6d7f03e1de671e34607e909a7213e31d7752be4fb66a86d29fe1eb14bfb5cf + url: "https://pub.dev" + source: hosted + version: "1.1.1" + collection: + dependency: transitive + description: + name: collection + sha256: ee67cb0715911d28db6bf4af1026078bd6f0128b07a5f66fb2ed94ec6783c09a + url: "https://pub.dev" + source: hosted + version: "1.18.0" + cupertino_icons: + dependency: "direct main" + description: + name: cupertino_icons + sha256: ba631d1c7f7bef6b729a622b7b752645a2d076dba9976925b8f25725a30e1ee6 + url: "https://pub.dev" + source: hosted + version: "1.0.8" + fake_async: + dependency: transitive + description: + name: fake_async + sha256: "511392330127add0b769b75a987850d136345d9227c6b94c96a04cf4a391bf78" + url: "https://pub.dev" + source: hosted + version: "1.3.1" + flutter: + dependency: "direct main" + description: flutter + source: sdk + version: "0.0.0" + flutter_lints: + dependency: "direct dev" + description: + name: flutter_lints + sha256: a25a15ebbdfc33ab1cd26c63a6ee519df92338a9c10f122adda92938253bef04 + url: "https://pub.dev" + source: hosted + version: "2.0.3" + flutter_test: + dependency: "direct dev" + description: flutter + source: sdk + version: "0.0.0" + lints: + dependency: transitive + description: + name: lints + sha256: "0a217c6c989d21039f1498c3ed9f3ed71b354e69873f13a8dfc3c9fe76f1b452" + url: "https://pub.dev" + source: hosted + version: "2.1.1" + matcher: + dependency: transitive + description: + name: matcher + sha256: "1803e76e6653768d64ed8ff2e1e67bea3ad4b923eb5c56a295c3e634bad5960e" + url: "https://pub.dev" + source: hosted + version: "0.12.16" + material_color_utilities: + dependency: transitive + description: + name: material_color_utilities + sha256: "9528f2f296073ff54cb9fee677df673ace1218163c3bc7628093e7eed5203d41" + url: "https://pub.dev" + source: hosted + version: "0.5.0" + meta: + dependency: transitive + description: + name: meta + sha256: a6e590c838b18133bb482a2745ad77c5bb7715fb0451209e1a7567d416678b8e + url: "https://pub.dev" + source: hosted + version: "1.10.0" + path: + dependency: transitive + description: + name: path + sha256: "8829d8a55c13fc0e37127c29fedf290c102f4e40ae94ada574091fe0ff96c917" + url: "https://pub.dev" + source: hosted + version: "1.8.3" + sky_engine: + dependency: transitive + description: flutter + source: sdk + version: "0.0.99" + source_span: + dependency: transitive + description: + name: source_span + sha256: "53e943d4206a5e30df338fd4c6e7a077e02254531b138a15aec3bd143c1a8b3c" + url: "https://pub.dev" + source: hosted + version: "1.10.0" + stack_trace: + dependency: transitive + description: + name: stack_trace + sha256: "73713990125a6d93122541237550ee3352a2d84baad52d375a4cad2eb9b7ce0b" + url: "https://pub.dev" + source: hosted + version: "1.11.1" + stream_channel: + dependency: transitive + description: + name: stream_channel + sha256: ba2aa5d8cc609d96bbb2899c28934f9e1af5cddbd60a827822ea467161eb54e7 + url: "https://pub.dev" + source: hosted + version: "2.1.2" + string_scanner: + dependency: transitive + description: + name: string_scanner + sha256: "556692adab6cfa87322a115640c11f13cb77b3f076ddcc5d6ae3c20242bedcde" + url: "https://pub.dev" + source: hosted + version: "1.2.0" + term_glyph: + dependency: transitive + description: + name: term_glyph + sha256: a29248a84fbb7c79282b40b8c72a1209db169a2e0542bce341da992fe1bc7e84 + url: "https://pub.dev" + source: hosted + version: "1.2.1" + test_api: + dependency: transitive + description: + name: test_api + sha256: "5c2f730018264d276c20e4f1503fd1308dfbbae39ec8ee63c5236311ac06954b" + url: "https://pub.dev" + source: hosted + version: "0.6.1" + vector_math: + dependency: transitive + description: + name: vector_math + sha256: "80b3257d1492ce4d091729e3a67a60407d227c27241d6927be0130c98e741803" + url: "https://pub.dev" + source: hosted + version: "2.1.4" + web: + dependency: transitive + description: + name: web + sha256: afe077240a270dcfd2aafe77602b4113645af95d0ad31128cc02bce5ac5d5152 + url: "https://pub.dev" + source: hosted + version: "0.3.0" +sdks: + dart: ">=3.2.3 <4.0.0" diff --git a/app_flutter/pubspec.yaml b/app_flutter/pubspec.yaml new file mode 100644 index 00000000..a222b7a8 --- /dev/null +++ b/app_flutter/pubspec.yaml @@ -0,0 +1,90 @@ +name: upgradeall +description: "UpgradeAll Flutter shell backed by the Rust getter core." +# The following line prevents the package from being accidentally published to +# pub.dev using `flutter pub publish`. This is preferred for private packages. +publish_to: 'none' # Remove this line if you wish to publish to pub.dev + +# The following defines the version and build number for your application. +# A version number is three numbers separated by dots, like 1.2.43 +# followed by an optional build number separated by a +. +# Both the version and the builder number may be overridden in flutter +# build by specifying --build-name and --build-number, respectively. +# In Android, build-name is used as versionName while build-number used as versionCode. +# Read more about Android versioning at https://developer.android.com/studio/publish/versioning +# In iOS, build-name is used as CFBundleShortVersionString while build-number is used as CFBundleVersion. +# Read more about iOS versioning at +# https://developer.apple.com/library/archive/documentation/General/Reference/InfoPlistKeyReference/Articles/CoreFoundationKeys.html +# In Windows, build-name is used as the major, minor, and patch parts +# of the product and file versions while build-number is used as the build suffix. +version: 0.20.0-alpha.4+105 + +environment: + sdk: '>=3.2.3 <4.0.0' + +# Dependencies specify other packages that your package needs in order to work. +# To automatically upgrade your package dependencies to the latest versions +# consider running `flutter pub upgrade --major-versions`. Alternatively, +# dependencies can be manually updated by changing the version numbers below to +# the latest version available on pub.dev. To see which dependencies have newer +# versions available, run `flutter pub outdated`. +dependencies: + flutter: + sdk: flutter + + + # The following adds the Cupertino Icons font to your application. + # Use with the CupertinoIcons class for iOS style icons. + cupertino_icons: ^1.0.2 + +dev_dependencies: + flutter_test: + sdk: flutter + + # The "flutter_lints" package below contains a set of recommended lints to + # encourage good coding practices. The lint set provided by the package is + # activated in the `analysis_options.yaml` file located at the root of your + # package. See that file for information about deactivating specific lint + # rules and activating additional ones. + flutter_lints: ^2.0.0 + +# For information on the generic Dart part of this file, see the +# following page: https://dart.dev/tools/pub/pubspec + +# The following section is specific to Flutter packages. +flutter: + + # The following line ensures that the Material Icons font is + # included with your application, so that you can use the icons in + # the material Icons class. + uses-material-design: true + + # To add assets to your application, add an assets section, like this: + # assets: + # - images/a_dot_burr.jpeg + # - images/a_dot_ham.jpeg + + # An image asset can refer to one or more resolution-specific "variants", see + # https://flutter.dev/assets-and-images/#resolution-aware + + # For details regarding adding assets from package dependencies, see + # https://flutter.dev/assets-and-images/#from-packages + + # To add custom fonts to your application, add a fonts section here, + # in this "flutter" section. Each entry in this list should have a + # "family" key with the font family name, and a "fonts" key with a + # list giving the asset and other descriptors for the font. For + # example: + # fonts: + # - family: Schyler + # fonts: + # - asset: fonts/Schyler-Regular.ttf + # - asset: fonts/Schyler-Italic.ttf + # style: italic + # - family: Trajan Pro + # fonts: + # - asset: fonts/TrajanPro.ttf + # - asset: fonts/TrajanPro_Bold.ttf + # weight: 700 + # + # For details regarding fonts from package dependencies, + # see https://flutter.dev/custom-fonts/#from-packages diff --git a/app_flutter/test/widget_test.dart b/app_flutter/test/widget_test.dart new file mode 100644 index 00000000..dcaee08f --- /dev/null +++ b/app_flutter/test/widget_test.dart @@ -0,0 +1,83 @@ +import 'package:flutter_test/flutter_test.dart'; + +import 'package:upgradeall/main.dart'; + +void main() { + testWidgets('fresh launch exposes home route and getter state', + (tester) async { + await tester.pumpWidget(const UpgradeAllApp()); + + expect(find.byKey(AppKeys.homeRoute), findsOneWidget); + expect(find.byKey(AppKeys.updateSummary), findsOneWidget); + expect(find.byKey(AppKeys.getterStatus), findsOneWidget); + expect(find.text('0 updates available'), findsOneWidget); + expect(find.text('Fake getter ready'), findsOneWidget); + }); + + testWidgets('app list and detail routes use stable keys', (tester) async { + await tester.pumpWidget(const UpgradeAllApp()); + + await tester.tap(find.byKey(AppKeys.openApps)); + await tester.pumpAndSettle(); + + expect(find.byKey(AppKeys.appsRoute), findsOneWidget); + expect(find.byKey(AppKeys.appsList), findsOneWidget); + expect(find.byKey(AppKeys.appRow('android/org.fdroid.fdroid')), + findsOneWidget); + expect(find.text('Network'), findsOneWidget); + + await tester.tap(find.byKey(AppKeys.appRow('android/org.fdroid.fdroid'))); + await tester.pumpAndSettle(); + + expect(find.byKey(AppKeys.appDetailRoute), findsOneWidget); + expect(find.text('android/org.fdroid.fdroid'), findsOneWidget); + expect(find.text('Installed: 1.20.0'), findsOneWidget); + expect(find.text('Latest: 1.20.0'), findsOneWidget); + expect(find.text('Network access required'), findsOneWidget); + }); + + testWidgets('repository route lists priority ordered repository IDs', + (tester) async { + await tester.pumpWidget(const UpgradeAllApp()); + + await tester.tap(find.byKey(AppKeys.openRepositories)); + await tester.pumpAndSettle(); + + expect(find.byKey(AppKeys.repositoriesRoute), findsOneWidget); + expect(find.byKey(AppKeys.repositoriesList), findsOneWidget); + expect(find.byKey(AppKeys.repoRow('local')), findsOneWidget); + expect(find.byKey(AppKeys.repoRow('official')), findsOneWidget); + expect(find.byKey(AppKeys.repoRow('local_autogen')), findsOneWidget); + }); + + testWidgets('placeholder routes expose stable empty-state keys', + (tester) async { + await tester.pumpWidget(const UpgradeAllApp()); + + await tester.tap(find.byKey(AppKeys.openDownloads)); + await tester.pumpAndSettle(); + expect(find.byKey(AppKeys.downloadsRoute), findsOneWidget); + expect(find.byKey(AppKeys.downloadsEmpty), findsOneWidget); + + await tester.pageBack(); + await tester.pumpAndSettle(); + await tester.tap(find.byKey(AppKeys.openLogs)); + await tester.pumpAndSettle(); + expect(find.byKey(AppKeys.logsRoute), findsOneWidget); + expect(find.byKey(AppKeys.logsEmpty), findsOneWidget); + + await tester.pageBack(); + await tester.pumpAndSettle(); + await tester.tap(find.byKey(AppKeys.openSettings)); + await tester.pumpAndSettle(); + expect(find.byKey(AppKeys.settingsRoute), findsOneWidget); + expect(find.byKey(AppKeys.settingsShell), findsOneWidget); + + await tester.pageBack(); + await tester.pumpAndSettle(); + await tester.tap(find.byKey(AppKeys.openMigration)); + await tester.pumpAndSettle(); + expect(find.byKey(AppKeys.migrationRoute), findsOneWidget); + expect(find.byKey(AppKeys.migrationReady), findsOneWidget); + }); +} From 4756f7c2ad1329b836d0fe2e74faeb40cb90a11b Mon Sep 17 00:00:00 2001 From: Xiangzhe Date: Mon, 22 Jun 2026 11:10:33 +0800 Subject: [PATCH 04/85] feat(getter): wire package-centric getter submodule Point the UpgradeAll superproject at the reusable getter submodule rewrite and keep api_proxy on the Android-compatible getter feature set. --- .../src/main/rust/api_proxy/Cargo.toml | 5 +-- .../src/main/rust/api_proxy/src/lib.rs | 45 ++++++++----------- core-getter/src/main/rust/getter | 2 +- 3 files changed, 21 insertions(+), 31 deletions(-) diff --git a/core-getter/src/main/rust/api_proxy/Cargo.toml b/core-getter/src/main/rust/api_proxy/Cargo.toml index 5a1469ce..3d12d3c6 100644 --- a/core-getter/src/main/rust/api_proxy/Cargo.toml +++ b/core-getter/src/main/rust/api_proxy/Cargo.toml @@ -7,16 +7,15 @@ edition = "2021" [dependencies] jni = "0.21" # from rustls-platform-verifier-android, sync version -getter = { path = "../getter", features = ["rustls-platform-verifier-android"] } +getter = { path = "../getter", default-features = false, features = ["rustls-platform-verifier-android"] } serde = { version = "1.0", features = ["derive"] } serde_json = "1.0.145" -tokio = "1.48.0" +tokio = { version = "1.48.0", features = ["rt-multi-thread"] } [lib] crate-type = ["cdylib"] [profile.release] -crate-type = ["rlib", "cdylib"] strip = true opt-level = 3 lto = true diff --git a/core-getter/src/main/rust/api_proxy/src/lib.rs b/core-getter/src/main/rust/api_proxy/src/lib.rs index 57b924c3..8006bd74 100644 --- a/core-getter/src/main/rust/api_proxy/src/lib.rs +++ b/core-getter/src/main/rust/api_proxy/src/lib.rs @@ -26,35 +26,39 @@ pub extern "C" fn Java_net_xzos_upgradeall_getter_NativeLib_runServer<'local>( .expect("Failed to create Java string"); } } - let (url_tx, url_rx) = channel(); - let (completion_tx, completion_rx) = channel::>(); + let (startup_tx, startup_rx) = channel::>(); thread::spawn(move || { let runtime = match tokio::runtime::Runtime::new() { Ok(rt) => rt, Err(e) => { - let err_msg = format!("Error creating Tokio runtime: {}", e); - completion_tx.send(Some(err_msg)).unwrap(); + let _ = startup_tx.send(Err(format!("Error creating Tokio runtime: {}", e))); return; } }; runtime.block_on(async move { let address = "127.0.0.1:0"; - match run_server_hanging(address, |url| { - url_tx.send(url.to_string()).unwrap(); + let startup_error_tx = startup_tx.clone(); + if let Err(e) = run_server_hanging(address, move |url| { + startup_tx + .send(Ok(url.to_string())) + .map_err(|_| getter::rpc::server::RpcServerError::StartupCallback)?; Ok(()) }) .await { - Ok(_) => completion_tx.send(None).unwrap(), // No error, send completion signal - Err(e) => { - let err_msg = format!("Error running server: {}", e); - completion_tx.send(Some(err_msg)).unwrap(); - } + // If startup failed before the URL callback, report it to JNI. + // If startup succeeded, NativeLib.runServer has already returned + // to Kotlin and the placeholder server intentionally lives for + // the lifetime of this background thread. + let _ = startup_error_tx.send(Err(format!("Error running server: {}", e))); } }); }); - let url = match url_rx.recv() { - Ok(url) => url, + let url = match startup_rx.recv() { + Ok(Ok(url)) => url, + Ok(Err(error)) => { + return env.new_string(error).expect("Failed to create Java string"); + } Err(e) => { return env .new_string(format!("Error receiving URL from server thread: {}", e)) @@ -82,18 +86,5 @@ pub extern "C" fn Java_net_xzos_upgradeall_getter_NativeLib_runServer<'local>( .expect("Failed to create Java string"); } - let error = match completion_rx.recv() { - Ok(error) => error, - Err(e) => { - return env - .new_string(format!("Error receiving error from server thread: {}", e)) - .expect("Failed to create Java string"); - } - }; - match error { - None => env.new_string("").expect("Failed to create Java string"), - Some(error) => env - .new_string(format!("Error running server: {}", error)) - .expect("Failed to create Java string"), - } + env.new_string("").expect("Failed to create Java string") } diff --git a/core-getter/src/main/rust/getter b/core-getter/src/main/rust/getter index f011d9b4..f746c961 160000 --- a/core-getter/src/main/rust/getter +++ b/core-getter/src/main/rust/getter @@ -1 +1 @@ -Subproject commit f011d9b4b9a15f83cd39c86e781ad8830a8ecae6 +Subproject commit f746c96161de03548e2ebaccf57e7800a98becc7 From a5730a98822e8f806fd8282f7d1d114178947dd0 Mon Sep 17 00:00:00 2001 From: Xiangzhe Date: Mon, 22 Jun 2026 11:10:33 +0800 Subject: [PATCH 05/85] ci: add rewrite validation workflow Add just-based local and GitHub Actions validation for getter tests, CLI BDD, Flutter shell tests, workspace checks, and Flutter Android debug builds. --- .../upgradeall-rewrite-validation.yml | 51 +++++++++++++++++++ justfile | 38 ++++++++++++++ 2 files changed, 89 insertions(+) create mode 100644 .github/workflows/upgradeall-rewrite-validation.yml create mode 100644 justfile diff --git a/.github/workflows/upgradeall-rewrite-validation.yml b/.github/workflows/upgradeall-rewrite-validation.yml new file mode 100644 index 00000000..2e76e5ff --- /dev/null +++ b/.github/workflows/upgradeall-rewrite-validation.yml @@ -0,0 +1,51 @@ +name: UpgradeAll Rewrite Validation + +on: + pull_request: + push: + branches: + - master + workflow_dispatch: + +jobs: + rewrite-validation: + name: Rewrite validation + runs-on: ubuntu-latest + steps: + - name: Checkout repository + uses: actions/checkout@v6 + with: + submodules: true + fetch-depth: 0 + + - name: Set up Java + uses: actions/setup-java@v5 + with: + distribution: temurin + java-version: 21 + + - name: Set up Android SDK + uses: android-actions/setup-android@v3 + + - name: Install Android SDK packages + run: sdkmanager --install "platforms;android-36" "build-tools;36.0.0" "platform-tools" + + - name: Install Flutter + uses: subosito/flutter-action@v2 + with: + channel: stable + cache: true + + - name: Set up Rust + uses: dtolnay/rust-toolchain@stable + with: + components: clippy, rustfmt + + - name: Install just + run: | + if ! command -v just >/dev/null 2>&1; then + cargo install just --locked + fi + + - name: Run rewrite validation + run: just verify diff --git a/justfile b/justfile new file mode 100644 index 00000000..04a91c83 --- /dev/null +++ b/justfile @@ -0,0 +1,38 @@ +set shell := ["bash", "-eu", "-o", "pipefail", "-c"] + +GETTER_MANIFEST := "core-getter/src/main/rust/getter/Cargo.toml" +API_PROXY_MANIFEST := "core-getter/src/main/rust/api_proxy/Cargo.toml" + +verify: + just test-getter-unit + just test-getter-bdd + just test-flutter-widget + just verify-workspace-skeleton + just build-flutter-android-debug + +verify-fast: + just test-getter-unit + just test-getter-bdd + just test-flutter-widget + +test-getter-unit: + cargo test --manifest-path {{ GETTER_MANIFEST }} --workspace --lib --bins + +test-getter-bdd: + cargo test --manifest-path {{ GETTER_MANIFEST }} -p getter-cli --test bdd_cli + +test-flutter-widget: + cd app_flutter && flutter test + +build-flutter-android-debug: + cd app_flutter && flutter build apk --debug + +verify-workspace-skeleton: + test "$(git ls-files -s core-getter/src/main/rust/getter | awk '{print $1}')" = "160000" + cargo metadata --manifest-path {{ GETTER_MANIFEST }} --no-deps --format-version 1 >/tmp/upgradeall-getter-metadata.json + cargo metadata --manifest-path {{ API_PROXY_MANIFEST }} --no-deps --format-version 1 >/tmp/upgradeall-api-proxy-metadata.json + cargo fmt --manifest-path {{ GETTER_MANIFEST }} --all --check + cargo check --manifest-path {{ GETTER_MANIFEST }} --workspace --all-targets + cargo check --manifest-path {{ API_PROXY_MANIFEST }} + cd app_flutter && flutter analyze + ./gradlew --no-daemon projects From 59c1a0df9a67f99739f6c2a36980f2d23597eb94 Mon Sep 17 00:00:00 2001 From: Xiangzhe Date: Mon, 22 Jun 2026 11:27:06 +0800 Subject: [PATCH 06/85] fix(getter): keep Android proxy off Lua deps Update the getter submodule to the Android facade fix and enable native-tokio explicitly for api_proxy without pulling the getter domain feature set. --- core-getter/src/main/rust/api_proxy/Cargo.toml | 2 +- core-getter/src/main/rust/getter | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/core-getter/src/main/rust/api_proxy/Cargo.toml b/core-getter/src/main/rust/api_proxy/Cargo.toml index 3d12d3c6..0dfb56f6 100644 --- a/core-getter/src/main/rust/api_proxy/Cargo.toml +++ b/core-getter/src/main/rust/api_proxy/Cargo.toml @@ -7,7 +7,7 @@ edition = "2021" [dependencies] jni = "0.21" # from rustls-platform-verifier-android, sync version -getter = { path = "../getter", default-features = false, features = ["rustls-platform-verifier-android"] } +getter = { path = "../getter", default-features = false, features = ["native-tokio", "rustls-platform-verifier-android"] } serde = { version = "1.0", features = ["derive"] } serde_json = "1.0.145" tokio = { version = "1.48.0", features = ["rt-multi-thread"] } diff --git a/core-getter/src/main/rust/getter b/core-getter/src/main/rust/getter index f746c961..3b7613d7 160000 --- a/core-getter/src/main/rust/getter +++ b/core-getter/src/main/rust/getter @@ -1 +1 @@ -Subproject commit f746c96161de03548e2ebaccf57e7800a98becc7 +Subproject commit 3b7613d709b405cb7229f2fbbf546c2d29ee96e6 From 3201d92d7b40c0be42b79b674e6a787406956028 Mon Sep 17 00:00:00 2001 From: Xiangzhe Date: Mon, 22 Jun 2026 11:43:39 +0800 Subject: [PATCH 07/85] fix(app): use Flutter-compatible Gradle wrapper Update the Flutter Android wrapper to Gradle 8.7 so current Flutter stable accepts the debug APK build in rewrite validation CI. --- app_flutter/android/gradle/wrapper/gradle-wrapper.properties | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/app_flutter/android/gradle/wrapper/gradle-wrapper.properties b/app_flutter/android/gradle/wrapper/gradle-wrapper.properties index 559efb4c..45181329 100644 --- a/app_flutter/android/gradle/wrapper/gradle-wrapper.properties +++ b/app_flutter/android/gradle/wrapper/gradle-wrapper.properties @@ -2,4 +2,4 @@ distributionBase=GRADLE_USER_HOME distributionPath=wrapper/dists zipStoreBase=GRADLE_USER_HOME zipStorePath=wrapper/dists -distributionUrl=https\://services.gradle.org/distributions/gradle-8.5-bin.zip +distributionUrl=https\://services.gradle.org/distributions/gradle-8.7-bin.zip From 35e6c3d1f7b35803eb76461b459c35ca71196b12 Mon Sep 17 00:00:00 2001 From: Xiangzhe Date: Mon, 22 Jun 2026 12:32:53 +0800 Subject: [PATCH 08/85] ci: restrict Telegram notifications to master pushes Guard the Telegram APK notification preparation and upload steps so manual branch runs and non-master events cannot post to the channel. --- .github/workflows/android.yml | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/.github/workflows/android.yml b/.github/workflows/android.yml index 2dbf886b..09fdf99a 100644 --- a/.github/workflows/android.yml +++ b/.github/workflows/android.yml @@ -118,7 +118,7 @@ jobs: # token: ${{secrets.APP_CENTER_TOKEN}} - name: Find debug APK - if: ${{ !github.event.pull_request }} + if: ${{ github.event_name == 'push' && github.ref == 'refs/heads/master' }} run: | if [ ! -z "${{ secrets.BOT_TOKEN }}" ]; then OUTPUT="app/build/outputs/apk/debug/" @@ -127,7 +127,7 @@ jobs: fi - name: Generate Commit Message - if: ${{ !github.event.pull_request }} + if: ${{ github.event_name == 'push' && github.ref == 'refs/heads/master' }} run: | COMMIT_MESSAGE=$(git log -1 --pretty=format:%s) AUTHOR_NAME=$(git log -1 --pretty=format:%an) @@ -146,7 +146,7 @@ jobs: echo "EOF" >> $GITHUB_ENV - name: Send commit to Telegram - if: ${{ !github.event.pull_request }} + if: ${{ github.event_name == 'push' && github.ref == 'refs/heads/master' }} uses: xz-dev/TelegramFileUploader@v1 env: BOT_TOKEN: ${{ secrets.TELEGRAM_TOKEN }} From 80e1eb60bed05eaac7875c477d377ed046111f19 Mon Sep 17 00:00:00 2001 From: Xiangzhe Date: Mon, 22 Jun 2026 14:02:59 +0800 Subject: [PATCH 09/85] fix(app): use Flutter-compatible AGP Upgrade the Flutter Android shell to Android Gradle Plugin 8.6.0 so current Flutter stable accepts the debug APK build in rewrite validation CI. --- app_flutter/android/settings.gradle | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/app_flutter/android/settings.gradle b/app_flutter/android/settings.gradle index 08a15043..4dedb24d 100644 --- a/app_flutter/android/settings.gradle +++ b/app_flutter/android/settings.gradle @@ -23,7 +23,7 @@ pluginManagement { plugins { id "dev.flutter.flutter-plugin-loader" version "1.0.0" - id "com.android.application" version "8.2.2" apply false + id "com.android.application" version "8.6.0" apply false } include ":app" From 384aee6c3ff100691c07c83973f5ba1ea7ea369b Mon Sep 17 00:00:00 2001 From: Xiangzhe Date: Mon, 22 Jun 2026 14:42:35 +0800 Subject: [PATCH 10/85] docs: add rewrite next-step audit plan --- todo.md | 581 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 581 insertions(+) create mode 100644 todo.md diff --git a/todo.md b/todo.md new file mode 100644 index 00000000..1bd96af9 --- /dev/null +++ b/todo.md @@ -0,0 +1,581 @@ +# UpgradeAll rewrite next-step audit and plan + +Date: 2026-06-22 14:36 CST +Repo: `DUpdateSystem/UpgradeAll` +Branch checked: `rewrite/flutter-getter-spine` +Superproject HEAD checked before this document: `80e1eb60 fix(app): use Flutter-compatible AGP` +Getter submodule checked: `core-getter/src/main/rust/getter` -> `3b7613d709b405cb7229f2fbbf546c2d29ee96e6` + +This document is the canonical next-step plan after reviewing: + +- `todo-next-step.md` +- `AGENTS.md` +- `docs/README.md` +- `docs/implementation/coding-agent-handoff.md` +- `docs/architecture/upgradeall-getter-rewrite-wiki.md` +- `docs/architecture/adr/0001..0006` +- `docs/migration/legacy-room-mapping.md` +- `docs/app/flutter-ui-feature-parity-and-testing.md` +- current superproject diff/status/log +- current getter submodule diff/status/log +- current GitHub Actions state for UpgradeAll PR #514 and getter PR #54 + +## 1. Audit conclusion + +There is no major architecture drift from the original rewrite plan. + +The completed work is broadly aligned with the intended direction: + +```text +Flutter shell / platform adapter only + -> no product decisions in Flutter yet +Rust getter core + -> product/domain/storage/repository/Lua/update/migration logic +Lua package repositories + -> JSON-like Lua package tables validated by Rust +SQLite main.db + cache.db + -> durable state split from rebuildable cache +``` + +The important caveat is that the branch is not merge-ready yet because UpgradeAll rewrite validation CI is red. The immediate blocker is Flutter/Kotlin Gradle Plugin compatibility, not a design issue. + +The second caveat is process discipline: the Flutter shell has been created, but it must stay a shell until the real getter bridge is designed and wired. Do not add more product UI behavior that duplicates getter logic. + +## 2. Current state evidence + +### Superproject + +```text +branch: rewrite/flutter-getter-spine +HEAD: 80e1eb60bed05eaac7875c477d377ed046111f19 +PR: https://github.com/DUpdateSystem/UpgradeAll/pull/514 +status before this todo.md: only untracked todo-next-step.md +``` + +Recent superproject commits: + +```text +80e1eb60 fix(app): use Flutter-compatible AGP +35e6c3d1 ci: restrict Telegram notifications to master pushes +3201d92d fix(app): use Flutter-compatible Gradle wrapper +59c1a0df fix(getter): keep Android proxy off Lua deps +a5730a98 ci: add rewrite validation workflow +4756f7c2 feat(getter): wire package-centric getter submodule +ae0d72c2 feat(app): add Flutter shell scaffold +95272873 chore: add rewrite agent guardrails +64611200 docs: add rewrite architecture records +``` + +### Getter submodule + +```text +path: core-getter/src/main/rust/getter +mode: 160000 gitlink, not vendored source +branch: rewrite/package-cli-spine +HEAD: 3b7613d709b405cb7229f2fbbf546c2d29ee96e6 +PR: https://github.com/DUpdateSystem/getter/pull/54 +``` + +Submodule integrity evidence: + +```bash +git ls-files -s core-getter/src/main/rust/getter +# expected/current: mode 160000 at 3b7613d709b405cb7229f2fbbf546c2d29ee96e6 +``` + +Getter PR checks were green at review time: + +```text +static-code-check: pass +test: pass +clippy-sarif: skipped as expected +``` + +### UpgradeAll CI state + +At review time: + +```text +Android CI: success +UpgradeAll Rewrite Validation: failure +``` + +Latest failure in rewrite validation: + +```text +Error: Your project's Kotlin version (1.9.22) is lower than Flutter's minimum supported version of 2.0.0. Please upgrade your Kotlin version. +``` + +Relevant files: + +```text +app_flutter/android/build.gradle +app_flutter/android/settings.gradle +app_flutter/android/app/build.gradle +``` + +Current Kotlin source: + +```groovy +// app_flutter/android/build.gradle +ext.kotlin_version = '1.9.22' +``` + +Also observed from the failed CI log: + +```text +Flutter support for Gradle 8.7.0 will soon be dropped; future minimum likely 8.14.0. +Flutter support for Android Gradle Plugin 8.6.0 will soon be dropped; future minimum likely 8.11.1. +``` + +Do not jump to AGP 9 as part of the immediate fix unless the minimal Kotlin fix proves impossible. The current failure is KGP < 2.0.0. + +## 3. Completed work vs original plan + +| Area | Plan expectation | Current implementation | Judgment | +|---|---|---|---| +| Docs / ADR first | Architecture, ADRs, AGENTS, handoff before broad coding | Present under `docs/architecture/**`, `docs/implementation/**`, `AGENTS.md` | Aligned | +| Getter as reusable core | `core-getter/src/main/rust/getter` remains a real submodule | Restored `.gitmodules`; gitlink is `160000`; getter PR exists | Aligned | +| CLI before real UI | Getter must be exercisable headlessly before product UI | CLI commands exist; BDD CLI tests exist; Flutter is still fake shell | Mostly aligned | +| Package-centric model | Avoid reviving old hub-app model | `repo/package/app/storage/legacy` CLI nouns; `hub list` documented compatibility-only | Aligned | +| SQLite storage | Use main DB + cache DB, not JSONL product store | `MainDb` and `CacheDb` implemented; `init` creates `main.db` and `cache.db` | Aligned | +| Lua package repositories | Lua files return JSON-like tables; Rust validates | `getter-core/src/lua.rs` and repository loader implemented; hardened lib search path | Aligned | +| Legacy migration | Automatic migration eventually; initial slice may be JSON bridge | JSON bridge bundle import exists; direct Room reader deferred | Partial but acceptable | +| ExtraApp preservation | Do not repeat old bug of skipping `extra_app` state | Current mapping preserves `ignored_version` and `favorite` from extra app slice | Aligned for current slice | +| Flutter UI | Flutter owns UI/platform only | `FakeGetterAdapter`, route keys, placeholder pages; no real product logic | Acceptable shell; freeze scope until bridge | +| Mixed TDD/BDD | TDD for Rust/domain, BDD for user-facing/integration | Rust unit tests + CLI BDD + Flutter widget tests | Aligned | +| Verification | `just verify` should be the main gate | `just verify` exists and is used by workflow | Aligned, but currently red in CI | + +## 4. Deviations / risks to control + +### 4.1 CI is red due Kotlin Gradle Plugin + +This is the immediate blocker. Do not continue feature work before making rewrite validation green. + +Current issue: + +```text +app_flutter/android/build.gradle: ext.kotlin_version = '1.9.22' +Flutter stable in CI requires Kotlin >= 2.0.0 +``` + +This is not an architecture deviation. It is a build compatibility issue. + +### 4.2 Flutter shell exists before real bridge + +This is acceptable only because it is still a shell: + +- fake in-memory getter adapter +- stable route/action/state keys +- no repository/update/storage decisions in Dart +- placeholders for downloads/logs/settings/migration + +Risk: if future work keeps adding screens using fake data, the project will drift into UI-first implementation and violate the original plan. + +Rule: after CI is green, the next product step must be bridge contract + real getter-backed data, not more fake UI. + +### 4.3 Getter rewrite is large and destructive by diff size + +Getter branch replaces a lot of old code: + +```text +getter diff vs master: ~4.7k insertions, ~14k deletions +``` + +This is acceptable for a rewrite branch, but PR review must explicitly call out deferred old capabilities: + +- downloader runtime +- provider implementations +- old RPC surface +- old websdk/cloud config machinery +- full migration/import/export + +Do not describe this PR as product-complete. + +### 4.4 Legacy migration is still a bridge slice, not full migration + +Current implementation accepts a deterministic JSON bridge bundle and maps `apps[]` to getter tracked package state. + +Still missing: + +- direct Android Room DB reader/exporter +- complete `hub`, `extra_app`, `extra_hub` ingestion +- WAL/SHM-safe DB copy/checkpoint path +- idempotence and partial-failure recovery +- Flutter migration UX beyond placeholder + +This is acceptable now, but must be called out in PR notes. + +### 4.5 Plain non-interactive SSH shell did not expose Flutter + +From this review shell, `flutter` was not found when running a simple SSH command. CI installs Flutter and previous local validation may have used a different shell/toolchain environment. + +Before claiming local validation on genx, the next agent must either: + +1. run from the environment where Flutter is actually on PATH, or +2. locate/source the Flutter installation explicitly, or +3. rely on GitHub Actions and report that local Flutter was unavailable. + +Do not claim `just verify` passed locally unless the command actually ran in the current environment. + +## 5. Immediate next plan: make rewrite validation CI green + +This must be the next implementation task. + +### Objective + +Make `UpgradeAll Rewrite Validation` pass for PR #514 without changing architecture or adding feature scope. + +### Files likely touched + +```text +app_flutter/android/build.gradle +app_flutter/android/settings.gradle +app_flutter/android/app/build.gradle +``` + +### Step 1: try the minimal Kotlin fix first + +Edit: + +```diff +// app_flutter/android/build.gradle +- ext.kotlin_version = '1.9.22' ++ ext.kotlin_version = '2.0.0' +``` + +Do not change AGP or Gradle wrapper in the same commit unless the Kotlin-only fix fails. Keeping the diff small makes the failure mode obvious. + +### Step 2: run focused local checks + +Preferred local commands: + +```bash +cd ~/Code/DUpdateSystem/UpgradeAll +cd app_flutter +flutter build apk --debug +cd .. +just verify +``` + +If `flutter` is not on PATH in the current SSH shell, first locate or source the Flutter environment. If that is not practical, push the minimal change and use GitHub Actions as the verification source, but report that local Flutter was unavailable. + +Do not commit a workaround that uses: + +```text +--android-skip-build-dependency-validation +``` + +That flag is diagnostic only, not the real fix. + +### Step 3: if minimal fix fails, modernize Kotlin plugin declaration + +If Flutter/Gradle still complains after `ext.kotlin_version = '2.0.0'`, switch to the modern plugin DSL. + +Likely shape: + +```groovy +// app_flutter/android/settings.gradle +plugins { + id "dev.flutter.flutter-plugin-loader" version "1.0.0" + id "com.android.application" version "8.6.0" apply false + id "org.jetbrains.kotlin.android" version "2.0.0" apply false +} +``` + +Then update: + +```diff +// app_flutter/android/app/build.gradle + plugins { + id "com.android.application" +- id "kotlin-android" ++ id "org.jetbrains.kotlin.android" + id "dev.flutter.flutter-gradle-plugin" + } +``` + +If this works, remove obsolete top-level `buildscript` Kotlin classpath only after verifying the Flutter template still builds. + +### Step 4: commit and push the CI fix + +```bash +git status --short --branch --untracked-files=all +git add app_flutter/android/build.gradle app_flutter/android/settings.gradle app_flutter/android/app/build.gradle +git commit --no-gpg-sign -m "fix(app): use Flutter-compatible Kotlin plugin" +git push +``` + +### Step 5: watch CI + +```bash +gh run list --repo DUpdateSystem/UpgradeAll --branch rewrite/flutter-getter-spine --limit 10 +gh pr checks 514 --repo DUpdateSystem/UpgradeAll +``` + +If PR checks still do not attach automatically, manually dispatch both branch workflows: + +```bash +gh workflow run upgradeall-rewrite-validation.yml --repo DUpdateSystem/UpgradeAll --ref rewrite/flutter-getter-spine +gh workflow run android.yml --repo DUpdateSystem/UpgradeAll --ref rewrite/flutter-getter-spine +``` + +Acceptance: + +```text +Android CI: success +UpgradeAll Rewrite Validation: success +Getter PR #54 checks: still green +``` + +## 6. After CI is green: PR stabilization checklist + +Do this before any new feature work. + +### 6.1 Confirm submodule integrity + +```bash +git ls-files -s core-getter/src/main/rust/getter +git submodule status core-getter/src/main/rust/getter +``` + +Expected: + +```text +core-getter/src/main/rust/getter remains mode 160000 +submodule points to getter branch commit 3b7613d or later pushed getter commit +``` + +### 6.2 Clean or consciously leave local notes + +Current local note: + +```text +?? todo-next-step.md +``` + +Decide explicitly: + +- keep it untracked as scratch, or +- delete it, or +- replace it with this committed `todo.md`. + +Do not accidentally include machine-local scratch files in feature commits. + +### 6.3 Update PR descriptions + +UpgradeAll PR #514 should say clearly: + +- this is a rewrite spine, not a product-complete release +- docs/ADR/AGENTS were added +- getter is a submodule and points to getter PR #54 +- Flutter shell is intentionally fake-adapter only +- current validation commands +- known deferred work: real bridge, direct Room migration, local_autogen, provider/downloader/update lifecycle + +Getter PR #54 should say clearly: + +- package-centric CLI/core rewrite +- old hub-app model is not coming back +- old provider/downloader/RPC behavior is deferred, not silently retained +- CI is green except skipped optional SARIF + +## 7. Next architecture gate: real Flutter-to-getter bridge + +Do not add more fake Flutter product screens before this gate. + +### Objective + +Define and implement the first real data path from Flutter shell to getter without moving product logic into Dart. + +### New doc / ADR to add + +```text +docs/architecture/adr/0007-flutter-getter-bridge-contract.md +``` + +This ADR should decide: + +1. short-term bridge for development and tests +2. Android production bridge path +3. whether the JSON envelope used by CLI is also the app bridge contract +4. error model and event model +5. how Flutter gets paged snapshots and event deltas +6. which APIs are forbidden in Flutter UI code + +Recommended default direction: + +- Use getter-owned DTOs and JSON envelopes as the stable behavior contract. +- Keep CLI as the headless test oracle. +- For in-app Flutter, prefer a direct generated/native bridge only after the DTO contract is stable. +- Local RPC remains acceptable for debug/external plugins, but do not force every mobile UI call through a heavyweight JSON-RPC server unless an ADR accepts the lifecycle cost. + +### First bridge API surface + +Start with read-only snapshot APIs. Do not start with downloads/installers. + +Minimum getter-facing operations: + +```text +initialize(data_dir) +list_repositories() +list_tracked_packages() +evaluate_package(package_id, repo_id?) +read_migration_reports() +``` + +Minimum Flutter-facing DTOs: + +```text +GetterSnapshot +AppSummary +RepositorySummary +MigrationReportSummary +GetterError +``` + +### Files likely touched + +```text +app_flutter/lib/main.dart # split only if needed +app_flutter/lib/getter_adapter.dart # new adapter interface / DTOs +app_flutter/lib/fake_getter_adapter.dart # keep fake test adapter separate +app_flutter/test/widget_test.dart +core-getter/src/main/rust/getter/crates/getter-cli/src/lib.rs +core-getter/src/main/rust/getter/crates/getter-ffi/src/lib.rs or future bridge crate +``` + +### Acceptance + +- Flutter tests can still run with fake adapter. +- A separate integration/dev test exercises a real getter data directory and returns real repository/app state. +- No repository resolution, update selection, migration mapping, or storage decision is implemented in Dart. +- Docs name the bridge decision and its limitations. + +## 8. Next product phases after bridge + +### Phase A: direct legacy Room migration + +Goal: replace bridge-only JSON import with the Android upgrade path. + +Tasks: + +1. Android migrator copies old DB plus `-wal` and `-shm` safely. +2. Opens/canonicalizes old Room schema to latest supported legacy version. +3. Exports a typed bundle including all durable tables: + - `app` + - `hub` + - `extra_app` + - `extra_hub` +4. Rust imports the bundle into `main.db` in one transaction. +5. Migration record prevents rerun. +6. Report is sanitized and visible in Flutter migration page. + +Acceptance: + +- Fixtures for fresh install, supported old DB, WAL/SHM pending writes, malformed optional JSON, partial prior migration. +- Per-app failures become warnings; global unreadable DB becomes recovery state, not crash. +- Dropped fields are documented in `docs/migration/legacy-room-mapping.md`. + +### Phase B: `local_autogen` generation + +Goal: convert installed/legacy state into generated fallback Lua packages without mixing with user-authored overrides. + +Rules: + +```text +local = user-authored, highest priority, never overwritten silently +local_autogen = generated fallback, safe to regenerate/clean after preview +``` + +Tasks: + +1. Define autogen output path and deterministic package file naming. +2. Generate Lua package stubs for installed apps not covered by official/local repos. +3. Add preview report before writing. +4. Add cleanup preview for missing generated apps. +5. Add invalidation rules when installed apps or repo metadata changes. + +Acceptance: + +- BDD for preview/confirm/cancel cleanup UX. +- TDD for deterministic Lua generation and no overwrite of `local`. +- Yellow/free-network warning tagging remains getter-driven metadata, not hardcoded UI behavior. + +### Phase C: repository tooling and diagnostics + +Goal: make Lua package repositories maintainable. + +Tasks: + +1. Add `repo validate` command. +2. Add clearer package eval diagnostics with path/location. +3. Add schema docs for package lifecycle phases. +4. Add fixture repositories for success and common failure cases. +5. Add cache invalidation rules for repo changes. + +Acceptance: + +- `getter --data-dir repo validate ` returns structured JSON. +- Invalid Lua/schema/domain errors point to file and field. +- No network is required for repository validation unless explicitly requested. + +### Phase D: update/download/install lifecycle + +Goal: move from static app/repo display to real update workflows. + +Tasks: + +1. Expand getter core update task model. +2. Implement provider/downloader crate behavior beyond placeholders. +3. Add event stream/backpressure model. +4. Add download task state and cancellation. +5. Add platform install handoff contract. +6. Add Flutter BDD for update/download user flows only after getter behavior exists. + +Acceptance: + +- CLI can run an offline fixture update check. +- Flutter displays getter events rather than calculating status itself. +- Android platform adapter owns permissions/notifications/installer handoff. + +## 9. Do-not-do list for the next agent + +- Do not add more fake product screens before fixing CI and defining the bridge. +- Do not move provider/update/storage/migration logic into Flutter. +- Do not vendor getter source into the UpgradeAll superproject. +- Do not revive old hub-app architecture; `hub list` is compatibility-only. +- Do not use random UUIDs as primary package identity. +- Do not claim migration complete while direct Room DB ingestion is missing. +- Do not bypass Flutter dependency validation as a committed workaround. +- Do not commit generated build outputs: + - `target/` + - `build/` + - `.dart_tool/` + - `.gradle/` + - APK/AAB/SO/class/object outputs + - `local.properties` + - `.pi/` + - `context-build/` + +## 10. Quick commands for the next session + +```bash +cd ~/Code/DUpdateSystem/UpgradeAll + +git status --short --branch --untracked-files=all +git submodule status --recursive + +gh run list --repo DUpdateSystem/UpgradeAll --branch rewrite/flutter-getter-spine --limit 10 +gh pr checks 514 --repo DUpdateSystem/UpgradeAll +gh pr checks 54 --repo DUpdateSystem/getter + +# after Kotlin fix +cd app_flutter && flutter build apk --debug +cd .. +just verify + +# if CI needs manual dispatch +gh workflow run upgradeall-rewrite-validation.yml --repo DUpdateSystem/UpgradeAll --ref rewrite/flutter-getter-spine +gh workflow run android.yml --repo DUpdateSystem/UpgradeAll --ref rewrite/flutter-getter-spine +``` From a1c43f43505924ce55095d8f342d699d4d470a2a Mon Sep 17 00:00:00 2001 From: Xiangzhe Date: Mon, 22 Jun 2026 14:59:35 +0800 Subject: [PATCH 11/85] fix(app): use Flutter-compatible Kotlin plugin Upgrade the Flutter Android shell Kotlin Gradle Plugin to 2.0.0 so current Flutter stable accepts rewrite validation builds. --- app_flutter/android/build.gradle | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/app_flutter/android/build.gradle b/app_flutter/android/build.gradle index 802640d0..cc71bc70 100644 --- a/app_flutter/android/build.gradle +++ b/app_flutter/android/build.gradle @@ -1,5 +1,5 @@ buildscript { - ext.kotlin_version = '1.9.22' + ext.kotlin_version = '2.0.0' repositories { google() mavenCentral() From c6838b50c27026f9654d2f21a5e9bbb5520dc62b Mon Sep 17 00:00:00 2001 From: Xiangzhe Date: Mon, 22 Jun 2026 15:18:34 +0800 Subject: [PATCH 12/85] docs: update rewrite next-step completion status Record the completed Kotlin CI fix, green PR checks, submodule verification, and updated PR descriptions in the rewrite todo plan. --- todo.md | 218 +++++++++++++++++++------------------------------------- 1 file changed, 72 insertions(+), 146 deletions(-) diff --git a/todo.md b/todo.md index 1bd96af9..f03fac29 100644 --- a/todo.md +++ b/todo.md @@ -1,9 +1,11 @@ # UpgradeAll rewrite next-step audit and plan Date: 2026-06-22 14:36 CST +Completion update: 2026-06-22 15:15 CST Repo: `DUpdateSystem/UpgradeAll` Branch checked: `rewrite/flutter-getter-spine` Superproject HEAD checked before this document: `80e1eb60 fix(app): use Flutter-compatible AGP` +Superproject HEAD after completing the immediate CI fix: `a1c43f43 fix(app): use Flutter-compatible Kotlin plugin` Getter submodule checked: `core-getter/src/main/rust/getter` -> `3b7613d709b405cb7229f2fbbf546c2d29ee96e6` This document is the canonical next-step plan after reviewing: @@ -24,6 +26,13 @@ This document is the canonical next-step plan after reviewing: There is no major architecture drift from the original rewrite plan. +Completion update: the immediate CI blocker described in this document has been fixed. The Kotlin Gradle Plugin was upgraded to `2.0.0`, the fix was pushed in `a1c43f43`, and both UpgradeAll PR checks are now green: + +```text +Android CI / Build: success +UpgradeAll Rewrite Validation / Rewrite validation: success +``` + The completed work is broadly aligned with the intended direction: ```text @@ -37,9 +46,7 @@ SQLite main.db + cache.db -> durable state split from rebuildable cache ``` -The important caveat is that the branch is not merge-ready yet because UpgradeAll rewrite validation CI is red. The immediate blocker is Flutter/Kotlin Gradle Plugin compatibility, not a design issue. - -The second caveat is process discipline: the Flutter shell has been created, but it must stay a shell until the real getter bridge is designed and wired. Do not add more product UI behavior that duplicates getter logic. +The earlier caveat that the branch was not merge-ready because rewrite validation CI was red is now resolved. The remaining caveat is process discipline: the Flutter shell has been created, but it must stay a shell until the real getter bridge is designed and wired. Do not add more product UI behavior that duplicates getter logic. ## 2. Current state evidence @@ -47,14 +54,16 @@ The second caveat is process discipline: the Flutter shell has been created, but ```text branch: rewrite/flutter-getter-spine -HEAD: 80e1eb60bed05eaac7875c477d377ed046111f19 +HEAD after completing the immediate CI fix: a1c43f43505924ce55095d8f342d699d4d470a2a PR: https://github.com/DUpdateSystem/UpgradeAll/pull/514 -status before this todo.md: only untracked todo-next-step.md +status after cleanup should be clean except this document update until committed ``` Recent superproject commits: ```text +a1c43f43 fix(app): use Flutter-compatible Kotlin plugin +384aee6c docs: add rewrite next-step audit plan 80e1eb60 fix(app): use Flutter-compatible AGP 35e6c3d1 ci: restrict Telegram notifications to master pushes 3201d92d fix(app): use Flutter-compatible Gradle wrapper @@ -93,19 +102,26 @@ clippy-sarif: skipped as expected ### UpgradeAll CI state -At review time: +At review time the state was: ```text Android CI: success UpgradeAll Rewrite Validation: failure ``` -Latest failure in rewrite validation: +The failure in rewrite validation was: ```text Error: Your project's Kotlin version (1.9.22) is lower than Flutter's minimum supported version of 2.0.0. Please upgrade your Kotlin version. ``` +Completion update: the Kotlin compatibility fix was committed and pushed, and the current PR checks are now: + +```text +Android CI / Build: success +UpgradeAll Rewrite Validation / Rewrite validation: success +``` + Relevant files: ```text @@ -114,11 +130,11 @@ app_flutter/android/settings.gradle app_flutter/android/app/build.gradle ``` -Current Kotlin source: +Current Kotlin source after the fix: ```groovy // app_flutter/android/build.gradle -ext.kotlin_version = '1.9.22' +ext.kotlin_version = '2.0.0' ``` Also observed from the failed CI log: @@ -144,22 +160,19 @@ Do not jump to AGP 9 as part of the immediate fix unless the minimal Kotlin fix | ExtraApp preservation | Do not repeat old bug of skipping `extra_app` state | Current mapping preserves `ignored_version` and `favorite` from extra app slice | Aligned for current slice | | Flutter UI | Flutter owns UI/platform only | `FakeGetterAdapter`, route keys, placeholder pages; no real product logic | Acceptable shell; freeze scope until bridge | | Mixed TDD/BDD | TDD for Rust/domain, BDD for user-facing/integration | Rust unit tests + CLI BDD + Flutter widget tests | Aligned | -| Verification | `just verify` should be the main gate | `just verify` exists and is used by workflow | Aligned, but currently red in CI | +| Verification | `just verify` should be the main gate | `just verify` exists, passes locally, and passes in the rewrite validation workflow | Aligned | ## 4. Deviations / risks to control -### 4.1 CI is red due Kotlin Gradle Plugin - -This is the immediate blocker. Do not continue feature work before making rewrite validation green. +### 4.1 Resolved Kotlin Gradle Plugin CI blocker -Current issue: +The immediate CI blocker has been resolved. The fix was intentionally minimal: ```text -app_flutter/android/build.gradle: ext.kotlin_version = '1.9.22' -Flutter stable in CI requires Kotlin >= 2.0.0 +app_flutter/android/build.gradle: ext.kotlin_version = '2.0.0' ``` -This is not an architecture deviation. It is a build compatibility issue. +This clears Flutter stable's Kotlin >= 2.0.0 dependency validation without changing architecture or feature scope. ### 4.2 Flutter shell exists before real bridge @@ -206,37 +219,23 @@ Still missing: This is acceptable now, but must be called out in PR notes. -### 4.5 Plain non-interactive SSH shell did not expose Flutter - -From this review shell, `flutter` was not found when running a simple SSH command. CI installs Flutter and previous local validation may have used a different shell/toolchain environment. - -Before claiming local validation on genx, the next agent must either: - -1. run from the environment where Flutter is actually on PATH, or -2. locate/source the Flutter installation explicitly, or -3. rely on GitHub Actions and report that local Flutter was unavailable. - -Do not claim `just verify` passed locally unless the command actually ran in the current environment. - -## 5. Immediate next plan: make rewrite validation CI green - -This must be the next implementation task. - -### Objective - -Make `UpgradeAll Rewrite Validation` pass for PR #514 without changing architecture or adding feature scope. +### 4.5 Validation environment note -### Files likely touched +The Kotlin fix was validated in the active agent environment with: ```text -app_flutter/android/build.gradle -app_flutter/android/settings.gradle -app_flutter/android/app/build.gradle +cd app_flutter && flutter build apk --debug +just verify +./gradlew --no-daemon ':core-getter:buildDebugApi_proxyRust[armeabi-v7a]' ``` -### Step 1: try the minimal Kotlin fix first +CI also validated the branch with Java 21 and the current Flutter stable action. Future agents should still report the actual local toolchain used when claiming local validation, because Flutter stable's minimum Gradle/AGP/Kotlin checks can move over time. + +## 5. Completed immediate plan: make rewrite validation CI green + +Status: completed in `a1c43f43 fix(app): use Flutter-compatible Kotlin plugin`. -Edit: +What changed: ```diff // app_flutter/android/build.gradle @@ -244,141 +243,68 @@ Edit: + ext.kotlin_version = '2.0.0' ``` -Do not change AGP or Gradle wrapper in the same commit unless the Kotlin-only fix fails. Keeping the diff small makes the failure mode obvious. +Why this was enough: -### Step 2: run focused local checks +- The latest failing Rewrite Validation log reported only Flutter's Kotlin Gradle Plugin minimum-version gate. +- The existing Flutter Android template remained coherent with the minimal `buildscript` Kotlin classpath bump. +- No architecture, feature, AGP, or Gradle wrapper scope was expanded in this fix. -Preferred local commands: +Validation completed after the fix: -```bash -cd ~/Code/DUpdateSystem/UpgradeAll -cd app_flutter -flutter build apk --debug -cd .. +```text +cd app_flutter && flutter build apk --debug just verify +./gradlew --no-daemon ':core-getter:buildDebugApi_proxyRust[armeabi-v7a]' ``` -If `flutter` is not on PATH in the current SSH shell, first locate or source the Flutter environment. If that is not practical, push the minimal change and use GitHub Actions as the verification source, but report that local Flutter was unavailable. - -Do not commit a workaround that uses: +GitHub Actions on PR #514 after the fix: ```text ---android-skip-build-dependency-validation +Android CI / Build: success +UpgradeAll Rewrite Validation / Rewrite validation: success ``` -That flag is diagnostic only, not the real fix. - -### Step 3: if minimal fix fails, modernize Kotlin plugin declaration +No committed workaround uses `--android-skip-build-dependency-validation`. -If Flutter/Gradle still complains after `ext.kotlin_version = '2.0.0'`, switch to the modern plugin DSL. - -Likely shape: - -```groovy -// app_flutter/android/settings.gradle -plugins { - id "dev.flutter.flutter-plugin-loader" version "1.0.0" - id "com.android.application" version "8.6.0" apply false - id "org.jetbrains.kotlin.android" version "2.0.0" apply false -} -``` +## 6. Completed PR stabilization checklist -Then update: - -```diff -// app_flutter/android/app/build.gradle - plugins { - id "com.android.application" -- id "kotlin-android" -+ id "org.jetbrains.kotlin.android" - id "dev.flutter.flutter-gradle-plugin" - } -``` - -If this works, remove obsolete top-level `buildscript` Kotlin classpath only after verifying the Flutter template still builds. - -### Step 4: commit and push the CI fix - -```bash -git status --short --branch --untracked-files=all -git add app_flutter/android/build.gradle app_flutter/android/settings.gradle app_flutter/android/app/build.gradle -git commit --no-gpg-sign -m "fix(app): use Flutter-compatible Kotlin plugin" -git push -``` - -### Step 5: watch CI - -```bash -gh run list --repo DUpdateSystem/UpgradeAll --branch rewrite/flutter-getter-spine --limit 10 -gh pr checks 514 --repo DUpdateSystem/UpgradeAll -``` - -If PR checks still do not attach automatically, manually dispatch both branch workflows: - -```bash -gh workflow run upgradeall-rewrite-validation.yml --repo DUpdateSystem/UpgradeAll --ref rewrite/flutter-getter-spine -gh workflow run android.yml --repo DUpdateSystem/UpgradeAll --ref rewrite/flutter-getter-spine -``` - -Acceptance: +### 6.1 Submodule integrity confirmed ```text -Android CI: success -UpgradeAll Rewrite Validation: success -Getter PR #54 checks: still green -``` - -## 6. After CI is green: PR stabilization checklist - -Do this before any new feature work. - -### 6.1 Confirm submodule integrity - -```bash -git ls-files -s core-getter/src/main/rust/getter -git submodule status core-getter/src/main/rust/getter +160000 3b7613d709b405cb7229f2fbbf546c2d29ee96e6 0 core-getter/src/main/rust/getter +3b7613d709b405cb7229f2fbbf546c2d29ee96e6 core-getter/src/main/rust/getter (heads/rewrite/package-cli-spine) ``` -Expected: +The getter remains a real `160000` gitlink and is not vendored into the UpgradeAll superproject. -```text -core-getter/src/main/rust/getter remains mode 160000 -submodule points to getter branch commit 3b7613d or later pushed getter commit -``` +### 6.2 Local scratch notes cleaned -### 6.2 Clean or consciously leave local notes - -Current local note: +Temporary local scratch/review artifacts were removed after their useful content was folded into this tracked `todo.md`: ```text -?? todo-next-step.md +todo-next-step.md +subagent-artifacts/review-kotlin-todo.md ``` -Decide explicitly: - -- keep it untracked as scratch, or -- delete it, or -- replace it with this committed `todo.md`. - -Do not accidentally include machine-local scratch files in feature commits. - -### 6.3 Update PR descriptions +### 6.3 PR descriptions updated -UpgradeAll PR #514 should say clearly: +UpgradeAll PR #514 now states: - this is a rewrite spine, not a product-complete release - docs/ADR/AGENTS were added -- getter is a submodule and points to getter PR #54 +- getter is a submodule and points to getter PR #54 / `3b7613d709b405cb7229f2fbbf546c2d29ee96e6` - Flutter shell is intentionally fake-adapter only -- current validation commands -- known deferred work: real bridge, direct Room migration, local_autogen, provider/downloader/update lifecycle +- Gradle/AGP/Kotlin compatibility fixes are included +- current CI validation is green +- deferred work includes real bridge, direct Room migration, `local_autogen`, provider/downloader/update lifecycle -Getter PR #54 should say clearly: +Getter PR #54 now states: - package-centric CLI/core rewrite -- old hub-app model is not coming back +- old hub-app model is not restored - old provider/downloader/RPC behavior is deferred, not silently retained -- CI is green except skipped optional SARIF +- Android JNI/API proxy consumers can depend on getter without pulling Lua/domain dependencies +- checks are green except optional SARIF skip ## 7. Next architecture gate: real Flutter-to-getter bridge From 76f771a918c3fd16af84acf1cfb07674f42063aa Mon Sep 17 00:00:00 2001 From: Xiangzhe Date: Mon, 22 Jun 2026 17:01:17 +0800 Subject: [PATCH 13/85] feat(app): add getter bridge contract Add ADR-0007, getter-owned Flutter DTOs, fake and CLI adapters, and a Flutter dev test that exercises a real getter-cli data directory. Wire the integration test into just verify and update the getter submodule to the legacy report-list command. --- AGENTS.md | 3 +- app_flutter/README.md | 10 +- .../dev_test/cli_getter_adapter_test.dart | 128 +++++++++++ app_flutter/lib/cli_getter_adapter.dart | 200 ++++++++++++++++++ app_flutter/lib/fake_getter_adapter.dart | 1 + app_flutter/lib/getter_adapter.dart | 194 +++++++++++++++++ app_flutter/lib/main.dart | 69 +----- core-getter/src/main/rust/getter | 2 +- docs/README.md | 10 +- .../flutter-ui-feature-parity-and-testing.md | 4 +- docs/architecture/README.md | 1 + ...06-package-centric-cli-command-contract.md | 3 + .../0007-flutter-getter-bridge-contract.md | 146 +++++++++++++ docs/implementation/coding-agent-handoff.md | 3 +- justfile | 5 + todo.md | 91 +++----- 16 files changed, 736 insertions(+), 134 deletions(-) create mode 100644 app_flutter/dev_test/cli_getter_adapter_test.dart create mode 100644 app_flutter/lib/cli_getter_adapter.dart create mode 100644 app_flutter/lib/fake_getter_adapter.dart create mode 100644 app_flutter/lib/getter_adapter.dart create mode 100644 docs/architecture/adr/0007-flutter-getter-bridge-contract.md diff --git a/AGENTS.md b/AGENTS.md index 5030ffdf..85946e51 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -12,7 +12,8 @@ Before coding, every agent MUST read: 6. `docs/architecture/adr/0004-sqlite-main-db-and-cache-db.md` 7. `docs/architecture/adr/0005-lua-package-api.md` 8. `docs/architecture/adr/0006-package-centric-cli-command-contract.md` -9. `docs/app/flutter-ui-feature-parity-and-testing.md` +9. `docs/architecture/adr/0007-flutter-getter-bridge-contract.md` +10. `docs/app/flutter-ui-feature-parity-and-testing.md` ## Core architecture rules diff --git a/app_flutter/README.md b/app_flutter/README.md index 545a870f..bde21de9 100644 --- a/app_flutter/README.md +++ b/app_flutter/README.md @@ -5,15 +5,19 @@ This is the new Flutter shell for the UpgradeAll rewrite. It must remain a UI an ## Current slice - Android application identity: `net.xzos.upgradeall` -- Stable route/action/state keys for widget and future integration tests +- Stable route/action/state keys for widget and future integration/dev tests - Placeholder routes for apps, repositories, downloads, logs, settings, and legacy migration -- Fake in-memory getter adapter until the Rust getter FFI/RPC binding is wired +- `FakeGetterAdapter` for deterministic widget tests +- `CliGetterAdapter` as a development/integration bridge against the real `getter-cli` JSON envelope + +`CliGetterAdapter` is not the final Android production bridge. It exists to keep the getter-owned DTO and error contract executable while the native bridge is designed in ADR-0007. ## Verification ```bash flutter analyze flutter test +GETTER_CLI_BIN=/path/to/getter-cli flutter test dev_test/cli_getter_adapter_test.dart ``` -From the repository root, `just verify` also runs the Flutter analyzer and widget tests. +From the repository root, `just verify` also runs the Flutter analyzer, widget tests, getter CLI integration/dev test, and Android debug build. diff --git a/app_flutter/dev_test/cli_getter_adapter_test.dart b/app_flutter/dev_test/cli_getter_adapter_test.dart new file mode 100644 index 00000000..8cbeb435 --- /dev/null +++ b/app_flutter/dev_test/cli_getter_adapter_test.dart @@ -0,0 +1,128 @@ +import 'dart:io'; + +import 'package:flutter_test/flutter_test.dart'; +import 'package:upgradeall/cli_getter_adapter.dart'; + +void main() { + test('CliGetterAdapter reads real getter repository and tracked state', () { + final getterCli = Platform.environment['GETTER_CLI_BIN']; + if (getterCli == null || getterCli.isEmpty) { + fail('GETTER_CLI_BIN must point to the built getter-cli binary'); + } + + final temp = Directory.systemTemp.createTempSync('upgradeall-getter-cli-'); + addTearDown(() => temp.deleteSync(recursive: true)); + + final dataDir = Directory('${temp.path}/data')..createSync(); + final repoDir = _createFixtureRepository(temp, 'official'); + final bundle = _createLegacyBundle(temp); + final adapter = + CliGetterAdapter(executable: getterCli, dataDir: dataDir.path); + + adapter.initialize(); + _runGetter(getterCli, dataDir.path, [ + 'repo', + 'add', + 'official', + repoDir.path, + '--priority', + '0', + ]); + _runGetter(getterCli, dataDir.path, [ + 'legacy', + 'import-room-bundle', + bundle.path, + ]); + + final repositories = adapter.listRepositories(); + expect(repositories.map((repo) => repo.id), contains('official')); + expect( + repositories.singleWhere((repo) => repo.id == 'official').priority, 0); + + final trackedPackages = adapter.listTrackedPackages(); + final tracked = trackedPackages.singleWhere( + (package) => package.id == 'android/org.fdroid.fdroid', + ); + expect(tracked.favorite, isTrue); + expect(tracked.ignoredVersion, '1.20.0'); + expect(tracked.packageResolution, 'official_repository_package'); + + final evaluated = adapter.evaluatePackage( + 'android/org.fdroid.fdroid', + repositoryId: 'official', + ); + expect(evaluated.name, 'F-Droid'); + expect(evaluated.repositoryId, 'official'); + expect(evaluated.hasFreeNetworkWarning, isTrue); + + final reports = adapter.readMigrationReports(); + expect( + reports.singleWhere((report) => report.code == 'migration.imported').ok, + isTrue); + + final snapshot = adapter.loadSnapshot(); + expect(snapshot.status, 'Getter CLI ready'); + expect(snapshot.repositories.map((repo) => repo.id), contains('official')); + final app = snapshot.apps.singleWhere( + (app) => app.id == 'android/org.fdroid.fdroid', + ); + expect(app.name, 'F-Droid'); + expect(app.installedVersion, 'unknown'); + expect(app.hasFreeNetworkWarning, isTrue); + }); +} + +Directory _createFixtureRepository(Directory temp, String repoId) { + final repoDir = Directory('${temp.path}/repo-$repoId')..createSync(); + Directory('${repoDir.path}/packages/android').createSync(recursive: true); + Directory('${repoDir.path}/lib').createSync(); + Directory('${repoDir.path}/templates').createSync(); + File('${repoDir.path}/repo.toml').writeAsStringSync(''' +id = "$repoId" +name = "Fixture $repoId" +priority = 0 +api_version = "getter.repo.v1" +'''); + File('${repoDir.path}/packages/android/org.fdroid.fdroid.lua') + .writeAsStringSync(''' +return package_def { + id = "android/org.fdroid.fdroid", + name = "F-Droid", + installed = { + { kind = "android_package", package_name = "org.fdroid.fdroid" }, + }, + permissions = { free_network = true }, +} +'''); + return repoDir; +} + +File _createLegacyBundle(Directory temp) { + return File('${temp.path}/legacy-bundle.json')..writeAsStringSync(''' +{ + "format": "upgradeall-legacy-room-bundle", + "version": 17, + "apps": [ + { + "kind": "android", + "installed_id": "org.fdroid.fdroid", + "official_package_available": true, + "ignored_version": "1.20.0", + "favorite": true + } + ] +} +'''); +} + +void _runGetter(String getterCli, String dataDir, List args) { + final result = Process.runSync( + getterCli, + ['--data-dir', dataDir, ...args], + ); + if (result.exitCode != 0) { + fail('getter ${args.join(' ')} failed with ${result.exitCode}\n' + 'stdout:\n${result.stdout}\n' + 'stderr:\n${result.stderr}'); + } +} diff --git a/app_flutter/lib/cli_getter_adapter.dart b/app_flutter/lib/cli_getter_adapter.dart new file mode 100644 index 00000000..ac36571d --- /dev/null +++ b/app_flutter/lib/cli_getter_adapter.dart @@ -0,0 +1,200 @@ +import 'dart:convert'; +import 'dart:io'; + +import 'getter_adapter.dart'; + +class CliGetterAdapter implements GetterAdapter { + const CliGetterAdapter({ + required this.executable, + required this.dataDir, + this.environment = const {}, + }); + + final String executable; + final String dataDir; + final Map environment; + + @override + void initialize() { + _runGetter(const ['init']); + } + + @override + List listRepositories() { + final json = _runGetter(const ['repo', 'list']); + final repositories = _asList(_data(json)['repositories'], 'repositories'); + return repositories.map(_repositoryFromJson).toList(growable: false); + } + + @override + List listTrackedPackages() { + final json = _runGetter(const ['app', 'list']); + final apps = _asList(_data(json)['apps'], 'apps'); + return apps.map(_trackedPackageFromJson).toList(growable: false); + } + + @override + PackageEvaluation evaluatePackage(String packageId, {String? repositoryId}) { + final args = ['package', 'eval', packageId]; + if (repositoryId != null) { + args.addAll(['--repo', repositoryId]); + } + final json = _runGetter(args); + final package = _asMap(_data(json)['package'], 'package'); + return _packageEvaluationFromJson(package); + } + + @override + List readMigrationReports() { + final json = _runGetter(const ['legacy', 'report-list']); + final reports = _asList(_data(json)['reports'], 'reports'); + return reports + .map((report) => _migrationReportFromJson(_asMap(report, 'report'))) + .toList(growable: false); + } + + @override + GetterSnapshot loadSnapshot() { + initialize(); + final repositories = listRepositories(); + final trackedPackages = listTrackedPackages(); + final apps = trackedPackages.map((tracked) { + final evaluated = evaluatePackage( + tracked.id, + repositoryId: tracked.repositoryId, + ); + return AppSummary( + id: tracked.id, + name: evaluated.name, + installedVersion: 'unknown', + latestVersion: 'unknown', + hasFreeNetworkWarning: evaluated.hasFreeNetworkWarning, + ); + }).toList(growable: false); + + return GetterSnapshot( + status: 'Getter CLI ready', + updateCount: 0, + apps: apps, + repositories: repositories, + ); + } + + Map _runGetter(List commandArgs) { + final result = Process.runSync( + executable, + ['--data-dir', dataDir, ...commandArgs], + environment: environment.isEmpty ? null : environment, + ); + final stdoutText = result.stdout.toString(); + final decoded = stdoutText.trim().isEmpty + ? {} + : _asMap(jsonDecode(stdoutText), 'getter stdout'); + if (result.exitCode != 0 || decoded['ok'] != true) { + final error = _errorFromEnvelope(decoded); + throw GetterBridgeException(error, exitCode: result.exitCode); + } + return decoded; + } +} + +Map _data(Map envelope) { + return _asMap(envelope['data'], 'data'); +} + +GetterError _errorFromEnvelope(Map envelope) { + final error = _asMap(envelope['error'], 'error'); + return GetterError( + code: _asString(error['code'], 'error.code'), + message: _asString(error['message'], 'error.message'), + detail: error['detail'] as String?, + ); +} + +RepositorySummary _repositoryFromJson(Object? value) { + final json = _asMap(value, 'repository'); + return RepositorySummary( + id: _asString(json['id'], 'repository.id'), + priority: _asInt(json['priority'], 'repository.priority'), + ); +} + +TrackedPackageSummary _trackedPackageFromJson(Object? value) { + final json = _asMap(value, 'tracked package'); + return TrackedPackageSummary( + id: _asString(json['id'], 'tracked.id'), + enabled: _asBool(json['enabled'], 'tracked.enabled'), + favorite: _asBool(json['favorite'], 'tracked.favorite'), + ignoredVersion: json['ignored_version'] as String?, + repositoryId: json['repository_id'] as String?, + packageResolution: _asString( + json['package_resolution'], + 'tracked.package_resolution', + ), + ); +} + +PackageEvaluation _packageEvaluationFromJson(Object? value) { + final json = _asMap(value, 'package'); + final permissions = _asMap(json['permissions'], 'package.permissions'); + return PackageEvaluation( + id: _asString(json['id'], 'package.id'), + repositoryId: _asString(json['repository'], 'package.repository'), + name: _asString(json['name'], 'package.name'), + hasFreeNetworkWarning: _asBool( + permissions['free_network'], + 'package.permissions.free_network', + ), + ); +} + +MigrationReportSummary _migrationReportFromJson(Map json) { + return MigrationReportSummary( + ok: _asBool(json['ok'], 'migration.ok'), + code: _asString(json['code'], 'migration.code'), + message: _asString(json['message'], 'migration.message'), + importedRecords: _asInt(json['imported_records'], 'migration.imported'), + trackedRecords: _asInt(json['tracked_records'], 'migration.tracked'), + ); +} + +Map _asMap(Object? value, String name) { + if (value is Map) { + return value; + } + if (value is Map) { + return value.cast(); + } + throw FormatException('$name should be a JSON object'); +} + +List _asList(Object? value, String name) { + if (value is List) { + return value; + } + if (value is List) { + return value.cast(); + } + throw FormatException('$name should be a JSON array'); +} + +String _asString(Object? value, String name) { + if (value is String) { + return value; + } + throw FormatException('$name should be a string'); +} + +int _asInt(Object? value, String name) { + if (value is int) { + return value; + } + throw FormatException('$name should be an integer'); +} + +bool _asBool(Object? value, String name) { + if (value is bool) { + return value; + } + throw FormatException('$name should be a boolean'); +} diff --git a/app_flutter/lib/fake_getter_adapter.dart b/app_flutter/lib/fake_getter_adapter.dart new file mode 100644 index 00000000..0934f8a9 --- /dev/null +++ b/app_flutter/lib/fake_getter_adapter.dart @@ -0,0 +1 @@ +export 'getter_adapter.dart' show FakeGetterAdapter; diff --git a/app_flutter/lib/getter_adapter.dart b/app_flutter/lib/getter_adapter.dart new file mode 100644 index 00000000..655b2a3d --- /dev/null +++ b/app_flutter/lib/getter_adapter.dart @@ -0,0 +1,194 @@ +/// Getter-facing UI bridge contracts for the Flutter shell. +/// +/// These DTOs are transport/rendering shapes. Product decisions such as +/// repository overlay resolution, update selection, Lua validation, migration +/// mapping, and storage behavior belong in Rust getter. +abstract interface class GetterAdapter { + void initialize(); + + List listRepositories(); + + List listTrackedPackages(); + + PackageEvaluation evaluatePackage(String packageId, {String? repositoryId}); + + List readMigrationReports(); + + GetterSnapshot loadSnapshot(); +} + +class FakeGetterAdapter implements GetterAdapter { + const FakeGetterAdapter(); + + static const _snapshot = GetterSnapshot( + status: 'Fake getter ready', + updateCount: 0, + apps: [ + AppSummary( + id: 'android/org.fdroid.fdroid', + name: 'F-Droid', + installedVersion: '1.20.0', + latestVersion: '1.20.0', + hasFreeNetworkWarning: true, + ), + ], + repositories: [ + RepositorySummary(id: 'local', priority: 100), + RepositorySummary(id: 'official', priority: 0), + RepositorySummary(id: 'local_autogen', priority: -1), + ], + ); + + @override + void initialize() {} + + @override + List listRepositories() => _snapshot.repositories; + + @override + List listTrackedPackages() { + return const [ + TrackedPackageSummary( + id: 'android/org.fdroid.fdroid', + enabled: true, + favorite: false, + ignoredVersion: null, + repositoryId: 'official', + packageResolution: 'official_repository_package', + ), + ]; + } + + @override + PackageEvaluation evaluatePackage(String packageId, {String? repositoryId}) { + if (packageId != 'android/org.fdroid.fdroid') { + throw const GetterBridgeException( + GetterError( + code: 'package.not_found', + message: 'Fake package not found', + ), + ); + } + return const PackageEvaluation( + id: 'android/org.fdroid.fdroid', + repositoryId: 'official', + name: 'F-Droid', + hasFreeNetworkWarning: true, + ); + } + + @override + List readMigrationReports() { + return const []; + } + + @override + GetterSnapshot loadSnapshot() => _snapshot; +} + +class GetterSnapshot { + const GetterSnapshot({ + required this.status, + required this.updateCount, + required this.apps, + required this.repositories, + }); + + final String status; + final int updateCount; + final List apps; + final List repositories; +} + +class AppSummary { + const AppSummary({ + required this.id, + required this.name, + required this.installedVersion, + required this.latestVersion, + required this.hasFreeNetworkWarning, + }); + + final String id; + final String name; + final String installedVersion; + final String latestVersion; + final bool hasFreeNetworkWarning; +} + +class RepositorySummary { + const RepositorySummary({required this.id, required this.priority}); + + final String id; + final int priority; +} + +class TrackedPackageSummary { + const TrackedPackageSummary({ + required this.id, + required this.enabled, + required this.favorite, + required this.ignoredVersion, + required this.repositoryId, + required this.packageResolution, + }); + + final String id; + final bool enabled; + final bool favorite; + final String? ignoredVersion; + final String? repositoryId; + final String packageResolution; +} + +class PackageEvaluation { + const PackageEvaluation({ + required this.id, + required this.repositoryId, + required this.name, + required this.hasFreeNetworkWarning, + }); + + final String id; + final String repositoryId; + final String name; + final bool hasFreeNetworkWarning; +} + +class MigrationReportSummary { + const MigrationReportSummary({ + required this.ok, + required this.code, + required this.message, + required this.importedRecords, + required this.trackedRecords, + }); + + final bool ok; + final String code; + final String message; + final int importedRecords; + final int trackedRecords; +} + +class GetterError { + const GetterError({required this.code, required this.message, this.detail}); + + final String code; + final String message; + final String? detail; +} + +class GetterBridgeException implements Exception { + const GetterBridgeException(this.error, {this.exitCode}); + + final GetterError error; + final int? exitCode; + + @override + String toString() { + final detail = error.detail == null ? '' : ': ${error.detail}'; + final exit = exitCode == null ? '' : ' (exit $exitCode)'; + return 'GetterBridgeException$exit: ${error.code}: ${error.message}$detail'; + } +} diff --git a/app_flutter/lib/main.dart b/app_flutter/lib/main.dart index 5a082922..52bfcf4f 100644 --- a/app_flutter/lib/main.dart +++ b/app_flutter/lib/main.dart @@ -1,5 +1,7 @@ import 'package:flutter/material.dart'; +import 'getter_adapter.dart'; + void main() { runApp(const UpgradeAllApp()); } @@ -74,73 +76,6 @@ class UpgradeAllApp extends StatelessWidget { } } -abstract interface class GetterAdapter { - GetterSnapshot loadSnapshot(); -} - -class FakeGetterAdapter implements GetterAdapter { - const FakeGetterAdapter(); - - @override - GetterSnapshot loadSnapshot() { - return const GetterSnapshot( - status: 'Fake getter ready', - updateCount: 0, - apps: [ - AppSummary( - id: 'android/org.fdroid.fdroid', - name: 'F-Droid', - installedVersion: '1.20.0', - latestVersion: '1.20.0', - hasFreeNetworkWarning: true, - ), - ], - repositories: [ - RepositorySummary(id: 'local', priority: 100), - RepositorySummary(id: 'official', priority: 0), - RepositorySummary(id: 'local_autogen', priority: -1), - ], - ); - } -} - -class GetterSnapshot { - const GetterSnapshot({ - required this.status, - required this.updateCount, - required this.apps, - required this.repositories, - }); - - final String status; - final int updateCount; - final List apps; - final List repositories; -} - -class AppSummary { - const AppSummary({ - required this.id, - required this.name, - required this.installedVersion, - required this.latestVersion, - required this.hasFreeNetworkWarning, - }); - - final String id; - final String name; - final String installedVersion; - final String latestVersion; - final bool hasFreeNetworkWarning; -} - -class RepositorySummary { - const RepositorySummary({required this.id, required this.priority}); - - final String id; - final int priority; -} - class HomePage extends StatelessWidget { const HomePage({super.key, required this.getter}); diff --git a/core-getter/src/main/rust/getter b/core-getter/src/main/rust/getter index 3b7613d7..6be047ac 160000 --- a/core-getter/src/main/rust/getter +++ b/core-getter/src/main/rust/getter @@ -1 +1 @@ -Subproject commit 3b7613d709b405cb7229f2fbbf546c2d29ee96e6 +Subproject commit 6be047ac13d6dc4dd94dc29fbe6c4f622acd640f diff --git a/docs/README.md b/docs/README.md index e04bc2b8..eef3346d 100644 --- a/docs/README.md +++ b/docs/README.md @@ -14,10 +14,12 @@ Start here: 4. `architecture/adr/0003-legacy-room-migration.md` — old Room DB migration strategy. 5. `architecture/adr/0004-sqlite-main-db-and-cache-db.md` — storage and cache split. 6. `architecture/adr/0005-lua-package-api.md` — Lua package API and Rust validation boundary. -7. `lua-api/` — practical Lua package authoring docs. -8. `migration/legacy-room-mapping.md` — old data mapping rules. -9. `app/flutter-ui-feature-parity-and-testing.md` — Flutter feature parity and BDD/TDD test boundary. -10. `implementation/coding-agent-handoff.md` — coding-agent / pi-agent handoff instructions. +7. `architecture/adr/0006-package-centric-cli-command-contract.md` — getter CLI automation contract. +8. `architecture/adr/0007-flutter-getter-bridge-contract.md` — Flutter/getter DTO and bridge contract. +9. `lua-api/` — practical Lua package authoring docs. +10. `migration/legacy-room-mapping.md` — old data mapping rules. +11. `app/flutter-ui-feature-parity-and-testing.md` — Flutter feature parity and BDD/TDD test boundary. +12. `implementation/coding-agent-handoff.md` — coding-agent / pi-agent handoff instructions. Canonical architecture ADRs live in `docs/architecture/adr/*`. The `docs/adr/*` directory is kept for historical/refactor-phase ADRs and transition notes. diff --git a/docs/app/flutter-ui-feature-parity-and-testing.md b/docs/app/flutter-ui-feature-parity-and-testing.md index 9f3122fb..dfbccf81 100644 --- a/docs/app/flutter-ui-feature-parity-and-testing.md +++ b/docs/app/flutter-ui-feature-parity-and-testing.md @@ -71,7 +71,9 @@ The first Flutter implementation slice is intentionally a shell, not product log - App project lives under `app_flutter/`. - Android identity remains `net.xzos.upgradeall` for future direct upgrade work. - `UpgradeAllApp` exposes stable route/action/state keys such as `route.home`, `action.open_apps`, `state.apps_list`, and `state.migration_ready`. -- The temporary `GetterAdapter` is fake in-memory data only. It exists to keep UI routes testable until the Rust getter FFI/RPC binding is wired. +- `FakeGetterAdapter` keeps UI routes deterministic for widget tests. +- `CliGetterAdapter` exercises a real getter data directory through the `getter-cli` JSON envelope for development/integration tests. +- ADR-0007 documents the bridge contract and explicitly treats the CLI adapter as a test/development bridge, not the final Android production path. - Product decisions such as repository resolution, updates, migrations, storage, and downloads still belong in Rust getter. ## Test pyramid diff --git a/docs/architecture/README.md b/docs/architecture/README.md index 4fd130f6..16b72f39 100644 --- a/docs/architecture/README.md +++ b/docs/architecture/README.md @@ -14,6 +14,7 @@ Planned / active ADRs: - `adr/0004-sqlite-main-db-and-cache-db.md` - `adr/0005-lua-package-api.md` - `adr/0006-package-centric-cli-command-contract.md` +- `adr/0007-flutter-getter-bridge-contract.md` Documentation policy: diff --git a/docs/architecture/adr/0006-package-centric-cli-command-contract.md b/docs/architecture/adr/0006-package-centric-cli-command-contract.md index 8e995f53..2ff7b3b4 100644 --- a/docs/architecture/adr/0006-package-centric-cli-command-contract.md +++ b/docs/architecture/adr/0006-package-centric-cli-command-contract.md @@ -21,6 +21,7 @@ getter --data-dir repo eval getter --data-dir package eval [--repo ] getter --data-dir storage validate getter --data-dir legacy import-room-bundle +getter --data-dir legacy report-list getter --data-dir hub list # temporary compatibility only ``` @@ -81,6 +82,8 @@ The first supported `legacy import-room-bundle` slice accepts a JSON bridge bund It maps `apps[]` into getter tracked package state in `main.db`, writes a sanitized report under `migration-reports/`, and records `legacy-room-v17` migration completion. Malformed JSON uses `migration.invalid_bundle`; wrong format/version uses `migration.unsupported_bundle`. +`legacy report-list` returns sanitized migration report summaries through the same JSON envelope so app/test adapters do not need to inspect getter's data-directory layout directly. + Exit-code classes: - `0`: success. diff --git a/docs/architecture/adr/0007-flutter-getter-bridge-contract.md b/docs/architecture/adr/0007-flutter-getter-bridge-contract.md new file mode 100644 index 00000000..f25f57f2 --- /dev/null +++ b/docs/architecture/adr/0007-flutter-getter-bridge-contract.md @@ -0,0 +1,146 @@ +# ADR-0007: Flutter / getter bridge contract + +> Status: Draft / first implementation slice accepted +> Date: 2026-06-22 +> Project: UpgradeAll rewrite — Flutter APP + Rust getter core + Lua package repository model + +## Decision + +Flutter talks to getter through getter-owned DTOs and JSON envelopes. The initial bridge contract is read-only and snapshot-oriented so Flutter can display real getter state without copying product/domain logic into Dart. + +The CLI JSON envelope from ADR-0006 is the first executable bridge oracle. It is used for development, integration/dev tests, and contract validation. Android production embedding still follows ADR-0002: the app embeds getter as a Rust library / native bridge rather than depending on a standalone long-lived getter daemon as the primary mobile path. + +The first bridge implementation in Flutter therefore has two adapters: + +- `FakeGetterAdapter` for deterministic widget tests and UI shell work. +- `CliGetterAdapter` for development/integration tests against a real getter data directory and the built `getter-cli` binary. + +The CLI adapter is not the final Android production bridge. It exists to make the contract executable before the FFI/native bridge is stabilized. + +## First bridge API surface + +The first accepted API surface is intentionally read-only: + +```text +initialize() +listRepositories() +listTrackedPackages() +evaluatePackage(packageId, repositoryId?) +readMigrationReports() +loadSnapshot() +``` + +`loadSnapshot()` composes the smaller getter-owned operations into the UI shell's first snapshot DTO. It must not perform repository resolution, version comparison, migration mapping, or update selection in Dart. `readMigrationReports()` must go through a getter operation such as `legacy report-list`; Flutter must not inspect getter's data-directory layout directly. + +## Flutter DTOs + +The Flutter shell may use DTOs that mirror getter output for rendering: + +```text +GetterSnapshot +AppSummary +RepositorySummary +TrackedPackageSummary +PackageEvaluation +MigrationReportSummary +GetterError +``` + +DTOs are a UI transport shape, not a new product model. Any field whose value requires domain interpretation must be supplied by getter or by a platform capability explicitly documented in a later ADR. + +## JSON envelope contract + +The CLI bridge consumes the ADR-0006 envelope shape: + +```json +{ + "ok": true, + "command": "repo list", + "data": {}, + "warnings": [] +} +``` + +and structured error envelopes: + +```json +{ + "ok": false, + "command": "package eval", + "error": { + "code": "package.eval_error", + "message": "Getter package evaluation failed", + "detail": "..." + } +} +``` + +Flutter adapter code may parse and display these fields, but it must not infer missing domain state from them. If the UI needs a richer field, add it to getter output first and cover it with getter tests. + +## Error model + +The bridge maps getter errors into `GetterError`: + +- `code`: stable machine-readable getter/platform code. +- `message`: short user/log-facing message. +- `detail`: optional diagnostic detail. + +Flutter may choose presentation, but the source classification belongs to getter or a documented platform adapter. + +## Event model + +The first bridge slice is snapshot-only. Streaming events, progress, cancellation, backpressure, foreground services, notification lifecycle, and installer handoff are explicitly deferred to the update/download/install lifecycle ADR/work. + +Until then, Flutter should refresh snapshots rather than maintaining its own task state machine. + +## Android production bridge direction + +The Android production path should embed getter through a native bridge once the DTO contract is stable. The native bridge should expose getter-owned operations and platform callbacks/capabilities; it should not force all in-app UI calls through a heavyweight local JSON-RPC server unless a future ADR accepts that lifecycle cost. + +Local RPC remains acceptable for debug tooling, external integration, and development workflows. + +## APIs forbidden in Flutter UI code + +Flutter UI code must not implement: + +- repository priority/overlay resolution +- Lua package validation or evaluation semantics +- version comparison/update selection +- legacy Room mapping decisions +- cache invalidation rules +- provider/source selection +- download task state machines +- package ID normalization beyond display-safe handling + +If a feature requires one of these decisions, add or extend a getter operation instead. + +## Consequences + +Positive: + +- The first bridge is executable in CI without waiting for full mobile FFI. +- CLI output remains the headless test oracle. +- Flutter can start consuming real getter data while preserving the Rust-owned domain boundary. +- Future native bridge work has a concrete DTO/error contract to preserve. + +Costs: + +- The CLI adapter is development/test infrastructure, not the final mobile path. +- Snapshot-only UI cannot yet represent long-running update/download/install flows. +- Getter output schemas must evolve carefully because they are now a cross-boundary contract. + +## Validation + +The first implementation slice must provide: + +- Flutter widget tests that continue to use `FakeGetterAdapter`. +- A Flutter/Dart integration test that builds or receives a real `getter-cli` binary, initializes a real getter data directory, and reads repositories, tracked packages, package evaluation output, and migration reports through `CliGetterAdapter`. +- `just verify` coverage for the bridge integration test. + +## Non-goals + +- No full FFI/native bridge implementation in this ADR. +- No update/download/install event stream. +- No direct Android Room DB reader. +- No product-complete Flutter UI. +- No product/domain decisions in Dart. diff --git a/docs/implementation/coding-agent-handoff.md b/docs/implementation/coding-agent-handoff.md index d0bb396e..54c2cc01 100644 --- a/docs/implementation/coding-agent-handoff.md +++ b/docs/implementation/coding-agent-handoff.md @@ -17,7 +17,8 @@ Before coding, read these files in order: 7. `docs/architecture/adr/0004-sqlite-main-db-and-cache-db.md` 8. `docs/architecture/adr/0005-lua-package-api.md` 9. `docs/architecture/adr/0006-package-centric-cli-command-contract.md` -10. `docs/app/flutter-ui-feature-parity-and-testing.md` +10. `docs/architecture/adr/0007-flutter-getter-bridge-contract.md` +11. `docs/app/flutter-ui-feature-parity-and-testing.md` ## Mission diff --git a/justfile b/justfile index 04a91c83..da3dddd1 100644 --- a/justfile +++ b/justfile @@ -8,6 +8,7 @@ verify: just test-getter-bdd just test-flutter-widget just verify-workspace-skeleton + just test-flutter-getter-cli-integration just build-flutter-android-debug verify-fast: @@ -24,6 +25,10 @@ test-getter-bdd: test-flutter-widget: cd app_flutter && flutter test +test-flutter-getter-cli-integration: + cargo build --manifest-path {{ GETTER_MANIFEST }} -p getter-cli --bin getter-cli + cd app_flutter && GETTER_CLI_BIN="../core-getter/src/main/rust/getter/target/debug/getter-cli" flutter test dev_test/cli_getter_adapter_test.dart + build-flutter-android-debug: cd app_flutter && flutter build apk --debug diff --git a/todo.md b/todo.md index f03fac29..7e994569 100644 --- a/todo.md +++ b/todo.md @@ -306,77 +306,56 @@ Getter PR #54 now states: - Android JNI/API proxy consumers can depend on getter without pulling Lua/domain dependencies - checks are green except optional SARIF skip -## 7. Next architecture gate: real Flutter-to-getter bridge +## 7. Completed first architecture gate: Flutter-to-getter bridge contract -Do not add more fake Flutter product screens before this gate. +Status: first implementation slice completed after the CI fix. -### Objective +What landed: -Define and implement the first real data path from Flutter shell to getter without moving product logic into Dart. +- Added `docs/architecture/adr/0007-flutter-getter-bridge-contract.md`. +- Added Flutter bridge DTO/interface file: `app_flutter/lib/getter_adapter.dart`. +- Split fake test adapter export: `app_flutter/lib/fake_getter_adapter.dart`. +- Added `CliGetterAdapter` in `app_flutter/lib/cli_getter_adapter.dart`. +- Added a real getter-backed Flutter dev test: `app_flutter/dev_test/cli_getter_adapter_test.dart`. +- Added `just test-flutter-getter-cli-integration` and included it in `just verify`. +- Added getter CLI `legacy report-list` so Flutter/test adapters consume sanitized migration reports through the getter JSON envelope instead of reading getter's data-directory layout directly. -### New doc / ADR to add +Bridge direction accepted: -```text -docs/architecture/adr/0007-flutter-getter-bridge-contract.md -``` - -This ADR should decide: - -1. short-term bridge for development and tests -2. Android production bridge path -3. whether the JSON envelope used by CLI is also the app bridge contract -4. error model and event model -5. how Flutter gets paged snapshots and event deltas -6. which APIs are forbidden in Flutter UI code - -Recommended default direction: - -- Use getter-owned DTOs and JSON envelopes as the stable behavior contract. -- Keep CLI as the headless test oracle. -- For in-app Flutter, prefer a direct generated/native bridge only after the DTO contract is stable. -- Local RPC remains acceptable for debug/external plugins, but do not force every mobile UI call through a heavyweight JSON-RPC server unless an ADR accepts the lifecycle cost. - -### First bridge API surface +- `FakeGetterAdapter` remains for deterministic widget tests. +- `CliGetterAdapter` is a development/integration bridge and test oracle against `getter-cli`; it is not the final Android production path. +- Android production should still embed getter through a native/FFI-style bridge after DTOs stabilize. +- The shared `GetterAdapter` interface now exposes the first read-only bridge surface: + - `initialize()` + - `listRepositories()` + - `listTrackedPackages()` + - `evaluatePackage(packageId, repositoryId?)` + - `readMigrationReports()` + - `loadSnapshot()` -Start with read-only snapshot APIs. Do not start with downloads/installers. - -Minimum getter-facing operations: - -```text -initialize(data_dir) -list_repositories() -list_tracked_packages() -evaluate_package(package_id, repo_id?) -read_migration_reports() -``` - -Minimum Flutter-facing DTOs: +Validation completed: ```text -GetterSnapshot -AppSummary -RepositorySummary -MigrationReportSummary -GetterError +just verify ``` -### Files likely touched +Result: ```text -app_flutter/lib/main.dart # split only if needed -app_flutter/lib/getter_adapter.dart # new adapter interface / DTOs -app_flutter/lib/fake_getter_adapter.dart # keep fake test adapter separate -app_flutter/test/widget_test.dart -core-getter/src/main/rust/getter/crates/getter-cli/src/lib.rs -core-getter/src/main/rust/getter/crates/getter-ffi/src/lib.rs or future bridge crate +getter unit/bin tests: pass +getter CLI BDD: 8 features, 9 scenarios, 65 steps passed +Flutter widget tests: pass +Flutter analyze: pass +Flutter getter CLI integration test: pass +Gradle project check: pass +Flutter Android debug APK build: pass ``` -### Acceptance +Important boundary note: -- Flutter tests can still run with fake adapter. -- A separate integration/dev test exercises a real getter data directory and returns real repository/app state. -- No repository resolution, update selection, migration mapping, or storage decision is implemented in Dart. -- Docs name the bridge decision and its limitations. +- Flutter parses getter envelopes and renders DTOs. +- Flutter still must not implement repository resolution, Lua validation/evaluation semantics, version comparison, migration mapping, provider/source selection, cache invalidation, or download task state machines. +- If Flutter needs richer state, extend getter output first and cover it with getter tests. ## 8. Next product phases after bridge From 47f58116cc1bc8ebb14a85fd98df0660debf56dd Mon Sep 17 00:00:00 2001 From: Xiangzhe Date: Mon, 22 Jun 2026 20:46:34 +0800 Subject: [PATCH 14/85] feat(getter): add offline repo validation Update the getter submodule to the repo validate implementation and document the offline diagnostics contract for Lua package repositories. --- core-getter/src/main/rust/getter | 2 +- docs/README.md | 2 +- ...06-package-centric-cli-command-contract.md | 3 ++ docs/lua-api/package-lifecycle.md | 43 +++++++++++++++++++ docs/lua-api/repository-layout.md | 12 ++++++ 5 files changed, 60 insertions(+), 2 deletions(-) diff --git a/core-getter/src/main/rust/getter b/core-getter/src/main/rust/getter index 6be047ac..255a9c7c 160000 --- a/core-getter/src/main/rust/getter +++ b/core-getter/src/main/rust/getter @@ -1 +1 @@ -Subproject commit 6be047ac13d6dc4dd94dc29fbe6c4f622acd640f +Subproject commit 255a9c7c25174355be24053c0e3876de7cf087db diff --git a/docs/README.md b/docs/README.md index eef3346d..8b67034f 100644 --- a/docs/README.md +++ b/docs/README.md @@ -16,7 +16,7 @@ Start here: 6. `architecture/adr/0005-lua-package-api.md` — Lua package API and Rust validation boundary. 7. `architecture/adr/0006-package-centric-cli-command-contract.md` — getter CLI automation contract. 8. `architecture/adr/0007-flutter-getter-bridge-contract.md` — Flutter/getter DTO and bridge contract. -9. `lua-api/` — practical Lua package authoring docs. +9. `lua-api/` — practical Lua package authoring docs, including offline `repo validate` diagnostics. 10. `migration/legacy-room-mapping.md` — old data mapping rules. 11. `app/flutter-ui-feature-parity-and-testing.md` — Flutter feature parity and BDD/TDD test boundary. 12. `implementation/coding-agent-handoff.md` — coding-agent / pi-agent handoff instructions. diff --git a/docs/architecture/adr/0006-package-centric-cli-command-contract.md b/docs/architecture/adr/0006-package-centric-cli-command-contract.md index 2ff7b3b4..40ec2b82 100644 --- a/docs/architecture/adr/0006-package-centric-cli-command-contract.md +++ b/docs/architecture/adr/0006-package-centric-cli-command-contract.md @@ -18,6 +18,7 @@ getter --data-dir app list getter --data-dir repo list getter --data-dir repo add [--priority ] getter --data-dir repo eval +getter --data-dir repo validate getter --data-dir package eval [--repo ] getter --data-dir storage validate getter --data-dir legacy import-room-bundle @@ -84,6 +85,8 @@ It maps `apps[]` into getter tracked package state in `main.db`, writes a saniti `legacy report-list` returns sanitized migration report summaries through the same JSON envelope so app/test adapters do not need to inspect getter's data-directory layout directly. +`repo validate ` validates a repository path offline without requiring it to be registered first. It returns `valid`, `diagnostics`, `package_count`, and `network_required = false`; diagnostics are getter-owned structured records with stable codes, message, severity, source path, and optional package id/field. + Exit-code classes: - `0`: success. diff --git a/docs/lua-api/package-lifecycle.md b/docs/lua-api/package-lifecycle.md index 9b5a6f46..95a8354f 100644 --- a/docs/lua-api/package-lifecycle.md +++ b/docs/lua-api/package-lifecycle.md @@ -64,3 +64,46 @@ return { ## post_update Optional post-update hook. Most persistent state changes should remain in Rust core, not Lua. + +## Offline validation + +`getter --data-dir repo validate ` validates repository layout and package schema without network access. The command evaluates local package Lua files with the same constrained `lib/` module loading used by `repo eval`/`package eval`, then returns a getter-owned diagnostic report: + +```json +{ + "valid": false, + "network_required": false, + "package_count": 0, + "diagnostics": [ + { + "severity": "error", + "code": "package.schema", + "message": "required string field 'name' is missing", + "package_id": "android/org.fdroid.fdroid", + "location": { + "path": "repo/packages/android/org.fdroid.fdroid.lua" + } + } + ] +} +``` + +Initial stable diagnostic codes include: + +- `repository.read_repo_toml` +- `repository.parse_repo_toml` +- `repository.invalid_id` +- `repository.unsupported_api_version` +- `repository.missing_directory` +- `repository.read_packages_dir` +- `repository.invalid_package_path` +- `repository.invalid_package_id` +- `repository.hash_package_file` +- `package.read_file` +- `package.lua_runtime` +- `package.not_a_table` +- `package.unsupported_value` +- `package.schema` +- `package.domain` + +The validation command is intentionally offline. Provider/network validation belongs to later provider/update workflow commands, not repository schema validation. diff --git a/docs/lua-api/repository-layout.md b/docs/lua-api/repository-layout.md index 15b8d1e6..9ac55c51 100644 --- a/docs/lua-api/repository-layout.md +++ b/docs/lua-api/repository-layout.md @@ -54,3 +54,15 @@ local github_android = require("lib.github_android_apk") ## templates/ Lua generators that output package Lua file content. + +## Offline validation + +Use getter's structured validator before publishing or registering a repository: + +```bash +getter --data-dir /tmp/ua-getter repo validate /path/to/repo +``` + +The command does not require the repository to be registered and does not use the network. It checks the local layout, `repo.toml`, package path-derived ids, constrained Lua evaluation, and Rust schema/domain validation. Results are returned as JSON with `valid`, `package_count`, `network_required`, and getter-owned `diagnostics`. + +Common diagnostic codes include `repository.missing_directory`, `repository.unsupported_api_version`, `package.lua_runtime`, `package.schema`, and `package.domain`. From 0976844c4dced276c76106a72cd26015be9c9109 Mon Sep 17 00:00:00 2001 From: Xiangzhe Date: Mon, 22 Jun 2026 22:24:49 +0800 Subject: [PATCH 15/85] feat(getter): import legacy Room databases --- core-getter/src/main/rust/getter | 2 +- .../adr/0003-legacy-room-migration.md | 16 +++++-- ...06-package-centric-cli-command-contract.md | 5 +- docs/migration/legacy-room-mapping.md | 33 ++++++++++++- todo.md | 47 ++++++++++++------- 5 files changed, 77 insertions(+), 26 deletions(-) diff --git a/core-getter/src/main/rust/getter b/core-getter/src/main/rust/getter index 255a9c7c..17b0e281 160000 --- a/core-getter/src/main/rust/getter +++ b/core-getter/src/main/rust/getter @@ -1 +1 @@ -Subproject commit 255a9c7c25174355be24053c0e3876de7cf087db +Subproject commit 17b0e281914d3c7d646aa0605bee283d31ffd60b diff --git a/docs/architecture/adr/0003-legacy-room-migration.md b/docs/architecture/adr/0003-legacy-room-migration.md index cca7f722..3cd39c3a 100644 --- a/docs/architecture/adr/0003-legacy-room-migration.md +++ b/docs/architecture/adr/0003-legacy-room-migration.md @@ -55,9 +55,17 @@ Normal installed-app autogen writes to `local_autogen`, but legacy migration is - Legacy settings whose meaning no longer exists. - Exotic URL replacement rules that cannot be safely mapped. -## Implemented CLI bridge-bundle slice +## Implemented direct DB and bridge-bundle slices -The current Rust CLI implementation does not read Android Room files directly yet. It accepts a JSON bridge bundle for deterministic host-side tests: +The Rust CLI now has a direct SQLite import slice for copied/checkpointed Room v17 databases: + +```text +getter --data-dir legacy import-room-db +``` + +The direct importer opens the DB read-only, requires `PRAGMA user_version = 17`, reads legacy `app` and `extra_app` rows, maps known app-id keys to `android/` or `magisk/`, writes getter tracked package state plus the `legacy-room-v17` migration record in one transaction, and emits sanitized report counts/warnings. Current `hub` and `extra_hub` rows are not imported as top-level objects; they are counted/dropped with warnings until a later accepted mapping exists. Android/platform code is still responsible for producing a WAL/SHM-consistent copied DB file before invoking getter. + +The host-side CLI also keeps the deterministic JSON bridge bundle for tests and non-Android fixtures: ```json { @@ -76,8 +84,8 @@ The current Rust CLI implementation does not read Android Room files directly ye } ``` -This slice maps `apps[]` into getter tracked package state in `main.db`, writes a sanitized report under `migration-reports/`, and records `legacy-room-v17` completion. Unsupported bundle formats/versions still fail with a sanitized recovery report. +Both slices map app state into getter tracked package state in `main.db`, write sanitized reports under `migration-reports/`, and record `legacy-room-v17` completion. Unsupported bundle formats/versions and unsupported/malformed databases fail with sanitized recovery reports. ## Failure behavior -A single unmapped app must not block the whole app. Global migration failure should lead to a migration/recovery page. A per-app mapping failure should be visible on that app or diagnostics page. +A single unmapped app must not block the whole app. Global migration failure should lead to a migration/recovery page. A per-app mapping failure should be visible on that app or diagnostics page. The direct DB importer treats malformed optional rows and mixed valid/invalid app rows as warnings, but unreadable DBs, unsupported `user_version`, missing required `app` table, and databases with app rows but zero importable app rows are global failures. diff --git a/docs/architecture/adr/0006-package-centric-cli-command-contract.md b/docs/architecture/adr/0006-package-centric-cli-command-contract.md index 40ec2b82..c3263822 100644 --- a/docs/architecture/adr/0006-package-centric-cli-command-contract.md +++ b/docs/architecture/adr/0006-package-centric-cli-command-contract.md @@ -22,6 +22,7 @@ getter --data-dir repo validate getter --data-dir package eval [--repo ] getter --data-dir storage validate getter --data-dir legacy import-room-bundle +getter --data-dir legacy import-room-db getter --data-dir legacy report-list getter --data-dir hub list # temporary compatibility only ``` @@ -83,6 +84,8 @@ The first supported `legacy import-room-bundle` slice accepts a JSON bridge bund It maps `apps[]` into getter tracked package state in `main.db`, writes a sanitized report under `migration-reports/`, and records `legacy-room-v17` migration completion. Malformed JSON uses `migration.invalid_bundle`; wrong format/version uses `migration.unsupported_bundle`. +`legacy import-room-db ` is the first direct Room database import slice. It opens a copied/checkpointed legacy SQLite database read-only, requires `PRAGMA user_version = 17`, reads `app` and `extra_app` rows, maps known legacy app-id keys to readable package ids (`android/` and `magisk/`), writes tracked package state and the `legacy-room-v17` migration record in one transaction, and emits sanitized report counts/warnings. Unsupported DB versions use `migration.unsupported_db`; unreadable or malformed DBs use `migration.invalid_db`. A DB with a mix of valid and invalid app rows imports valid rows and reports skipped-row warnings; a DB with app rows but zero importable app rows is treated as `migration.invalid_db` so migration completion is not recorded silently. This command does not import legacy `hub` as a new domain model; current hub/extra_hub rows are counted/dropped with warnings until a later accepted mapping exists. + `legacy report-list` returns sanitized migration report summaries through the same JSON envelope so app/test adapters do not need to inspect getter's data-directory layout directly. `repo validate ` validates a repository path offline without requiring it to be registered first. It returns `valid`, `diagnostics`, `package_count`, and `network_required = false`; diagnostics are getter-owned structured records with stable codes, message, severity, source path, and optional package id/field. @@ -122,5 +125,5 @@ Costs: - No old hub-app model revival. - No live network provider behavior in the initial CLI smoke slice. -- No direct Android Room database reader in the JSON bridge-bundle slice. +- No Android/platform DB copy or WAL checkpoint implementation in the CLI contract itself; platform adapters prepare a consistent DB file and getter owns import semantics. - No Flutter UI behavior in CLI tests. diff --git a/docs/migration/legacy-room-mapping.md b/docs/migration/legacy-room-mapping.md index db6aa42a..97065401 100644 --- a/docs/migration/legacy-room-mapping.md +++ b/docs/migration/legacy-room-mapping.md @@ -64,15 +64,44 @@ Complex auth may be dropped. ## ExtraApp mapping -Map mark/ignore version state when possible. +Map mark/ignore version state when possible. In the direct Room DB importer, `extra_app.mark_version_number` wins over `app.ignore_version_number` when both exist for the same package id because it is the more specific extra-app state. ## ExtraHub mapping Map URL replace semantics into global download rewrite policy if safe. Otherwise drop and record warning. +## Current CLI direct DB import + +The host-side CLI can import a copied/checkpointed Room SQLite database directly: + +```text +getter --data-dir legacy import-room-db +``` + +Current direct import scope: + +- requires `PRAGMA user_version = 17`; +- reads `app.app_id`, `app.ignore_version_number`, `app.star`; +- reads `extra_app.app_id` and `extra_app.mark_version_number`; +- maps app-id key `android_app_package` to `android/`; +- maps app-id key `android_magisk_module` to `magisk/`; +- writes getter `tracked_packages` plus the `legacy-room-v17` migration record in one transaction; +- imports valid app rows while reporting skipped-row warnings when other app rows are malformed or unsupported; +- treats a DB with app rows but zero importable app rows as `migration.invalid_db` and does not record migration completion; +- emits sanitized counts/warnings and never embeds raw DB contents, auth, or tokens in reports. + +Currently dropped with warnings: + +- `hub` rows; +- `extra_hub` rows and URL replacement policy; +- hub auth/API keys/provider credentials; +- app regex/cloud config fields whose new package equivalent is not accepted yet. + +The direct CLI reader expects Android/platform code to provide a WAL/SHM-consistent DB copy; it does not perform Android Room checkpointing itself. + ## Current CLI bridge bundle -The host-side CLI implementation currently accepts a deterministic JSON bridge bundle instead of reading Room directly: +The host-side CLI implementation also accepts a deterministic JSON bridge bundle: ```json { diff --git a/todo.md b/todo.md index 7e994569..89a72f3d 100644 --- a/todo.md +++ b/todo.md @@ -363,24 +363,35 @@ Important boundary note: Goal: replace bridge-only JSON import with the Android upgrade path. -Tasks: - -1. Android migrator copies old DB plus `-wal` and `-shm` safely. -2. Opens/canonicalizes old Room schema to latest supported legacy version. -3. Exports a typed bundle including all durable tables: - - `app` - - `hub` - - `extra_app` - - `extra_hub` -4. Rust imports the bundle into `main.db` in one transaction. -5. Migration record prevents rerun. -6. Report is sanitized and visible in Flutter migration page. - -Acceptance: - -- Fixtures for fresh install, supported old DB, WAL/SHM pending writes, malformed optional JSON, partial prior migration. -- Per-app failures become warnings; global unreadable DB becomes recovery state, not crash. -- Dropped fields are documented in `docs/migration/legacy-room-mapping.md`. +Status: first getter-owned direct DB slice completed. The getter CLI now supports `legacy import-room-db ` for copied/checkpointed Room v17 SQLite files. It reads `app` and `extra_app`, maps known legacy app-id keys, writes `tracked_packages` plus `legacy-room-v17` in one transaction, prevents rerun, emits sanitized reports, and documents dropped hub/extra_hub fields. Android-side WAL/SHM copy/checkpoint and Flutter migration UX remain future work. + +Completed tasks: + +1. Rust direct importer opens copied legacy Room DB read-only and requires `PRAGMA user_version = 17`. +2. Rust reads durable `app` and `extra_app` fields needed for tracked package/user state. +3. Rust imports into `main.db` in one transaction. +4. Migration record prevents rerun. +5. Reports are sanitized and visible through `legacy report-list`. +6. Dropped `hub`/`extra_hub` fields are documented in `docs/migration/legacy-room-mapping.md`. + +Remaining tasks: + +1. Android migrator copies old DB plus `-wal` and `-shm` safely before invoking getter. +2. Android/platform adapter opens/checkpoints/canonicalizes old Room schema to latest supported legacy version. +3. Extend accepted mapping if future ADR accepts direct `hub`/`extra_hub` semantics. +4. Flutter migration page starts the adapter flow and renders getter reports. + +Acceptance progress: + +- Supported old DB fixture: done. +- Malformed/unsupported DB recovery reports: done. +- Partial prior migration/idempotence: done across direct DB and bridge bundle paths. +- Malformed optional JSON becomes warning: covered in Rust storage tests for `extra_app`. +- Mixed valid/invalid app rows import valid rows and warn: done. +- DBs with app rows but zero importable rows fail with recovery report: done. +- Report sanitization for dropped `hub`/`extra_hub` secrets and URL rewrite data: done. +- WAL/SHM pending writes: pending Android adapter slice. +- Per-app failures become warnings; global unreadable DB becomes recovery state, not crash: done for the getter-owned direct importer. ### Phase B: `local_autogen` generation From 81a5dec56b9b7b046a6bed74e81c7e4199196d12 Mon Sep 17 00:00:00 2001 From: Xiangzhe Date: Tue, 23 Jun 2026 00:18:12 +0800 Subject: [PATCH 16/85] feat(getter): add installed app autogen --- core-getter/src/main/rust/getter | 2 +- ...06-package-centric-cli-command-contract.md | 6 ++++ docs/lua-api/templates.md | 27 +++++++++------- todo.md | 31 ++++++++++++++----- 4 files changed, 47 insertions(+), 19 deletions(-) diff --git a/core-getter/src/main/rust/getter b/core-getter/src/main/rust/getter index 17b0e281..05397e14 160000 --- a/core-getter/src/main/rust/getter +++ b/core-getter/src/main/rust/getter @@ -1 +1 @@ -Subproject commit 17b0e281914d3c7d646aa0605bee283d31ffd60b +Subproject commit 05397e1446f19e728737b5f94b8f8d96036c83f4 diff --git a/docs/architecture/adr/0006-package-centric-cli-command-contract.md b/docs/architecture/adr/0006-package-centric-cli-command-contract.md index c3263822..312d512a 100644 --- a/docs/architecture/adr/0006-package-centric-cli-command-contract.md +++ b/docs/architecture/adr/0006-package-centric-cli-command-contract.md @@ -21,6 +21,10 @@ getter --data-dir repo eval getter --data-dir repo validate getter --data-dir package eval [--repo ] getter --data-dir storage validate +getter --data-dir autogen installed preview --inventory +getter --data-dir autogen installed apply --preview (--accept-all|--accept ...) +getter --data-dir autogen cleanup preview --inventory +getter --data-dir autogen cleanup apply --preview (--accept-all|--accept ...) getter --data-dir legacy import-room-bundle getter --data-dir legacy import-room-db getter --data-dir legacy report-list @@ -88,6 +92,8 @@ It maps `apps[]` into getter tracked package state in `main.db`, writes a saniti `legacy report-list` returns sanitized migration report summaries through the same JSON envelope so app/test adapters do not need to inspect getter's data-directory layout directly. +The first installed-app autogen slice accepts an Android/platform-provided inventory DTO, computes generated fallback packages in Rust, previews before writing, and applies only after explicit `--accept-all` or `--accept ` confirmation. Getter owns the canonical repository path `/repositories/local_autogen`, fixed repo id `local_autogen`, default priority `-1`, deterministic package file path `packages//.lua`, and `autogen-manifest.json`. Candidates are skipped when any registered repository with priority higher than `local_autogen` already provides that package id. Applying installed autogen writes/registers `local_autogen` package files and tracks accepted packages in `main.db` when they are not already resolved; existing user state (`enabled`, `favorite`, `ignored_version`) and existing non-missing resolution metadata are preserved. Cleanup preview/apply only targets manifest-managed `local_autogen` packages missing from the current installed inventory. Cleanup refuses stale/tampered previews that do not match the current manifest and deletes tracked state only for rows still owned by `local_autogen` generated packages. If an existing autogen file was modified, getter preserves that content into the user-authored `local` repo before regenerating or deleting the managed autogen file. + `repo validate ` validates a repository path offline without requiring it to be registered first. It returns `valid`, `diagnostics`, `package_count`, and `network_required = false`; diagnostics are getter-owned structured records with stable codes, message, severity, source path, and optional package id/field. Exit-code classes: diff --git a/docs/lua-api/templates.md b/docs/lua-api/templates.md index 3ae87b42..0ca5b925 100644 --- a/docs/lua-api/templates.md +++ b/docs/lua-api/templates.md @@ -46,22 +46,27 @@ return android.local_app { Generation flow: -1. User clicks generate. -2. getter computes candidate list. -3. Flutter shows preview list. -4. User confirms yes/no. -5. getter writes files. +1. Android/platform adapter writes an installed-inventory DTO. +2. User clicks generate. +3. getter computes candidate list through `autogen installed preview --inventory `. +4. Flutter shows preview list. +5. User confirms yes/no. +6. getter applies the accepted preview through `autogen installed apply --preview --accept-all` or repeated `--accept `. +7. getter writes files under `/repositories/local_autogen`, registers the repo, records `autogen-manifest.json`, and tracks accepted packages in `main.db`. Cleanup flow: -1. User clicks clear missing generated apps. -2. getter computes deletion list. -3. Flutter shows preview list. -4. User confirms yes/no. -5. getter deletes only autogen-managed files/state. +1. Android/platform adapter writes the current installed-inventory DTO. +2. User clicks clear missing generated apps. +3. getter computes deletion list through `autogen cleanup preview --inventory `. +4. Flutter shows preview list. +5. User confirms yes/no. +6. getter deletes only accepted manifest-managed `local_autogen` files/state. + +Cleanup apply refuses stale/tampered previews that do not match the current manifest, and guarded tracked-state deletion only removes rows still owned by `local_autogen` generated packages. Installed apply preserves existing user state (`enabled`, `favorite`, `ignored_version`) and existing non-missing resolution metadata when a package is already tracked. If a managed autogen file has been edited, getter preserves that content into the user-authored `local` repo before regenerating or deleting the generated file. Ordinary autogen cleanup never deletes `local`. ## Repositories -Ordinary installed-app autogen writes to `local_autogen`. +Ordinary installed-app autogen writes to `local_autogen`, using fixed repo id `local_autogen`, default priority `-1`, and deterministic paths such as `packages/android/com.example.app.lua`. Candidates are skipped when any registered repository with priority higher than `local_autogen` already provides the same package id. Legacy migration may generate `local` files once as a special compatibility path. diff --git a/todo.md b/todo.md index 89a72f3d..384a0577 100644 --- a/todo.md +++ b/todo.md @@ -404,19 +404,36 @@ local = user-authored, highest priority, never overwritten silently local_autogen = generated fallback, safe to regenerate/clean after preview ``` -Tasks: +User-confirmed decisions: + +- getter creates/uses canonical `/repositories/local_autogen`. +- any registered repository with priority higher than `local_autogen` suppresses generation for a package id. +- autogen apply/cleanup are getter-managed; if a generated file has been edited, getter preserves it into `local` before regenerating/deleting. +- applying installed autogen also tracks accepted packages because user confirmation means the user wants update tracking. + +Status: first getter-owned CLI/core slice in progress. Implemented pure autogen planning, installed preview/apply, cleanup preview/apply, deterministic package Lua generation, manifest-managed cleanup, higher-priority coverage skips, local preservation for edited autogen files, guarded cleanup against stale/tampered previews, and preservation of existing tracked user state during autogen apply. Flutter/Android inventory collection and UX remain future adapter work. + +Completed tasks: 1. Define autogen output path and deterministic package file naming. -2. Generate Lua package stubs for installed apps not covered by official/local repos. +2. Generate Lua package stubs for installed apps not covered by higher-priority repos. 3. Add preview report before writing. 4. Add cleanup preview for missing generated apps. -5. Add invalidation rules when installed apps or repo metadata changes. +5. Track accepted generated packages in getter storage without clobbering existing user state. +6. Preserve edited generated files into `local` before autogen rewrite/delete. +7. Guard cleanup deletion by current autogen manifest, repository id, and generated-package resolution. -Acceptance: +Remaining tasks: + +1. Android adapter supplies real installed inventory DTO. +2. Flutter confirmation UX consumes getter preview/apply DTOs. +3. Cache invalidation hooks beyond file-hash-based repository reload need to be expanded when evaluated/provider caches become active. + +Acceptance progress: -- BDD for preview/confirm/cancel cleanup UX. -- TDD for deterministic Lua generation and no overwrite of `local`. -- Yellow/free-network warning tagging remains getter-driven metadata, not hardcoded UI behavior. +- BDD for preview/confirm cleanup UX: done for CLI slice. +- TDD for deterministic Lua generation and no overwrite of `local`: done for core/CLI slice. +- Yellow/free-network warning tagging remains getter-driven metadata, not hardcoded UI behavior: not needed for installed-target-only stubs in this slice. ### Phase C: repository tooling and diagnostics From 8f3ca1333fce3e956087b090ddeb019c98a5d3b2 Mon Sep 17 00:00:00 2001 From: Xiangzhe Date: Tue, 23 Jun 2026 00:57:52 +0800 Subject: [PATCH 17/85] feat(getter): add offline update check --- core-getter/src/main/rust/getter | 2 +- ...06-package-centric-cli-command-contract.md | 3 ++ docs/lua-api/package-lifecycle.md | 2 ++ todo.md | 31 ++++++++++++------- 4 files changed, 26 insertions(+), 12 deletions(-) diff --git a/core-getter/src/main/rust/getter b/core-getter/src/main/rust/getter index 05397e14..196cd8c2 160000 --- a/core-getter/src/main/rust/getter +++ b/core-getter/src/main/rust/getter @@ -1 +1 @@ -Subproject commit 05397e1446f19e728737b5f94b8f8d96036c83f4 +Subproject commit 196cd8c2c449a17ef4c58f4fa401a2f6d93efe7b diff --git a/docs/architecture/adr/0006-package-centric-cli-command-contract.md b/docs/architecture/adr/0006-package-centric-cli-command-contract.md index 312d512a..14133b9b 100644 --- a/docs/architecture/adr/0006-package-centric-cli-command-contract.md +++ b/docs/architecture/adr/0006-package-centric-cli-command-contract.md @@ -21,6 +21,7 @@ getter --data-dir repo eval getter --data-dir repo validate getter --data-dir package eval [--repo ] getter --data-dir storage validate +getter --data-dir update check --fixture getter --data-dir autogen installed preview --inventory getter --data-dir autogen installed apply --preview (--accept-all|--accept ...) getter --data-dir autogen cleanup preview --inventory @@ -94,6 +95,8 @@ It maps `apps[]` into getter tracked package state in `main.db`, writes a saniti The first installed-app autogen slice accepts an Android/platform-provided inventory DTO, computes generated fallback packages in Rust, previews before writing, and applies only after explicit `--accept-all` or `--accept ` confirmation. Getter owns the canonical repository path `/repositories/local_autogen`, fixed repo id `local_autogen`, default priority `-1`, deterministic package file path `packages//.lua`, and `autogen-manifest.json`. Candidates are skipped when any registered repository with priority higher than `local_autogen` already provides that package id. Applying installed autogen writes/registers `local_autogen` package files and tracks accepted packages in `main.db` when they are not already resolved; existing user state (`enabled`, `favorite`, `ignored_version`) and existing non-missing resolution metadata are preserved. Cleanup preview/apply only targets manifest-managed `local_autogen` packages missing from the current installed inventory. Cleanup refuses stale/tampered previews that do not match the current manifest and deletes tracked state only for rows still owned by `local_autogen` generated packages. If an existing autogen file was modified, getter preserves that content into the user-authored `local` repo before regenerating or deleting the managed autogen file. +`update check --fixture ` is the first Phase D offline update-check slice. The fixture contract is explicitly offline and uses `format = "getter-offline-update-check"`, `version = 1`, `package_id`, optional `installed_version`, optional `ignored_version`, and normalized candidate/artifact DTOs. The command returns `network_required = false`, a getter-owned status (`update_available`, `up_to_date`, `no_candidates`, or `ignored`), the selected update when one exists, and generated download/install action DTOs. An update-selected result must have an actionable artifact; a selected candidate without artifacts is a structured update-check error rather than `update_available` with no actions. It reuses Rust getter update selection and version comparison; it does not run providers, perform downloads, persist download tasks, stream events, or call Android installers. + `repo validate ` validates a repository path offline without requiring it to be registered first. It returns `valid`, `diagnostics`, `package_count`, and `network_required = false`; diagnostics are getter-owned structured records with stable codes, message, severity, source path, and optional package id/field. Exit-code classes: diff --git a/docs/lua-api/package-lifecycle.md b/docs/lua-api/package-lifecycle.md index 95a8354f..e83cb235 100644 --- a/docs/lua-api/package-lifecycle.md +++ b/docs/lua-api/package-lifecycle.md @@ -61,6 +61,8 @@ return { } ``` +The first Phase D implementation slice exposes this boundary only through an offline CLI fixture command: `getter --data-dir update check --fixture `. The fixture is normalized JSON, not live provider output, and the command returns `network_required = false`, update-check status, selected candidate/artifact, and generated download/install action DTOs. It does not execute network providers, download files, persist download tasks, stream progress events, or invoke Android installers. + ## post_update Optional post-update hook. Most persistent state changes should remain in Rust core, not Lua. diff --git a/todo.md b/todo.md index 384a0577..a00381e4 100644 --- a/todo.md +++ b/todo.md @@ -457,20 +457,29 @@ Acceptance: Goal: move from static app/repo display to real update workflows. -Tasks: +Status: first getter-owned offline update-check slice in progress. The accepted minimal slice is intentionally not a provider/downloader/installer runtime. It defines normalized offline fixture input/output DTOs, reuses Rust getter update selection/version comparison, and adds `getter --data-dir update check --fixture ` returning selected update and generated download/install action DTOs with `network_required = false`. + +Completed tasks: -1. Expand getter core update task model. -2. Implement provider/downloader crate behavior beyond placeholders. -3. Add event stream/backpressure model. -4. Add download task state and cancellation. -5. Add platform install handoff contract. -6. Add Flutter BDD for update/download user flows only after getter behavior exists. +1. Add offline update-check fixture DTO and result/status DTO in getter core. +2. Reuse existing getter-core update selection for update availability. +3. Generate minimal download/install action DTOs for the selected artifact. +4. Add CLI command `update check --fixture `. +5. Add BDD coverage for update available, up to date, ignored latest fallback, ignored-only, unknown installed version, and malformed fixture. -Acceptance: +Remaining tasks: + +1. Implement provider/downloader crate behavior beyond placeholders. +2. Add event stream/backpressure model. +3. Add persistent download task state and cancellation. +4. Add platform install handoff contract. +5. Add Flutter BDD for update/download user flows only after getter behavior exists. + +Acceptance progress: -- CLI can run an offline fixture update check. -- Flutter displays getter events rather than calculating status itself. -- Android platform adapter owns permissions/notifications/installer handoff. +- CLI can run an offline fixture update check: done for first slice. +- Flutter displays getter events rather than calculating status itself: deferred until event DTO/bridge slice. +- Android platform adapter owns permissions/notifications/installer handoff: documented/deferred; no Android execution added in first slice. ## 9. Do-not-do list for the next agent From 9bb18f2d2709cf5762e571908ac48d8245d9dc2c Mon Sep 17 00:00:00 2001 From: Xiangzhe Date: Tue, 23 Jun 2026 10:54:54 +0800 Subject: [PATCH 18/85] feat(getter): add offline task lifecycle --- core-getter/src/main/rust/getter | 2 +- ...06-package-centric-cli-command-contract.md | 10 ++++++- .../0007-flutter-getter-bridge-contract.md | 4 ++- docs/lua-api/package-lifecycle.md | 2 ++ todo.md | 26 ++++++++++++------- 5 files changed, 32 insertions(+), 12 deletions(-) diff --git a/core-getter/src/main/rust/getter b/core-getter/src/main/rust/getter index 196cd8c2..cb4cb655 160000 --- a/core-getter/src/main/rust/getter +++ b/core-getter/src/main/rust/getter @@ -1 +1 @@ -Subproject commit 196cd8c2c449a17ef4c58f4fa401a2f6d93efe7b +Subproject commit cb4cb655f3aefa69200ac23cfcc1e770d678d664 diff --git a/docs/architecture/adr/0006-package-centric-cli-command-contract.md b/docs/architecture/adr/0006-package-centric-cli-command-contract.md index 14133b9b..5ee2d85e 100644 --- a/docs/architecture/adr/0006-package-centric-cli-command-contract.md +++ b/docs/architecture/adr/0006-package-centric-cli-command-contract.md @@ -22,6 +22,12 @@ getter --data-dir repo validate getter --data-dir package eval [--repo ] getter --data-dir storage validate getter --data-dir update check --fixture +getter --data-dir task submit --request +getter --data-dir task run +getter --data-dir task list +getter --data-dir task cancel +getter --data-dir task events --after --limit +getter --data-dir task install-result --status getter --data-dir autogen installed preview --inventory getter --data-dir autogen installed apply --preview (--accept-all|--accept ...) getter --data-dir autogen cleanup preview --inventory @@ -97,6 +103,8 @@ The first installed-app autogen slice accepts an Android/platform-provided inven `update check --fixture ` is the first Phase D offline update-check slice. The fixture contract is explicitly offline and uses `format = "getter-offline-update-check"`, `version = 1`, `package_id`, optional `installed_version`, optional `ignored_version`, and normalized candidate/artifact DTOs. The command returns `network_required = false`, a getter-owned status (`update_available`, `up_to_date`, `no_candidates`, or `ignored`), the selected update when one exists, and generated download/install action DTOs. An update-selected result must have an actionable artifact; a selected candidate without artifacts is a structured update-check error rather than `update_available` with no actions. It reuses Rust getter update selection and version comparison; it does not run providers, perform downloads, persist download tasks, stream events, or call Android installers. +The first getter-owned task lifecycle slice is also explicitly offline/fake and command-driven. `task submit --request ` accepts `format = "getter-download-request"`, `version = 1`, `package_id`, `executor = "fake"`, and update actions containing at least one `download` action; an optional `install` action creates an abstract install handoff after a successful fake run. `task run ` deterministically advances the fake task to `succeeded`; it performs no network I/O and writes no downloaded bytes. `task list` returns persisted task summaries from `main.db`. `task cancel ` persists cancellation for `queued`/`running` tasks, is idempotent for already-canceled tasks, and rejects terminal success/failure with a structured download error. `task events --after --limit ` is a pollable CLI/dev event contract with a positive `limit`; it is not a native streaming API, and native streaming/backpressure remains deferred. `task install-result --status ` records the platform-side result of an abstract handoff; the getter-created `requested` handoff state is not accepted as a platform result. Getter records handoff requests/results but does not call Android installers, request permissions, create notifications, or decide Android URI/SAF semantics. + `repo validate ` validates a repository path offline without requiring it to be registered first. It returns `valid`, `diagnostics`, `package_count`, and `network_required = false`; diagnostics are getter-owned structured records with stable codes, message, severity, source path, and optional package id/field. Exit-code classes: @@ -107,7 +115,7 @@ Exit-code classes: - `10`: data/storage error. - `20`: migration/import error. - `30`: future network/provider error. -- `40`: future download error. +- `40`: download/task lifecycle error. ## Context diff --git a/docs/architecture/adr/0007-flutter-getter-bridge-contract.md b/docs/architecture/adr/0007-flutter-getter-bridge-contract.md index f25f57f2..ad48763f 100644 --- a/docs/architecture/adr/0007-flutter-getter-bridge-contract.md +++ b/docs/architecture/adr/0007-flutter-getter-bridge-contract.md @@ -91,7 +91,9 @@ Flutter may choose presentation, but the source classification belongs to getter The first bridge slice is snapshot-only. Streaming events, progress, cancellation, backpressure, foreground services, notification lifecycle, and installer handoff are explicitly deferred to the update/download/install lifecycle ADR/work. -Until then, Flutter should refresh snapshots rather than maintaining its own task state machine. +The first Phase D lifecycle slice defines getter-owned task/event/handoff DTOs through the CLI only: task state is persisted in getter `main.db`, task events are pollable with `after` cursor plus `limit`, and fake executor progress is command-driven rather than background-streamed. This pollable CLI/dev contract is not the final native stream API. Flutter should not maintain its own task state machine; future Flutter/bridge work must render getter task/event DTOs or ask getter for richer fields. + +Android platform install remains a handoff boundary. Getter may request/record an abstract install handoff, but Android permissions, notifications, PackageInstaller/Shizuku/root execution, and path-versus-URI/SAF semantics belong to platform adapter work and remain outside this bridge slice. ## Android production bridge direction diff --git a/docs/lua-api/package-lifecycle.md b/docs/lua-api/package-lifecycle.md index e83cb235..b58f836b 100644 --- a/docs/lua-api/package-lifecycle.md +++ b/docs/lua-api/package-lifecycle.md @@ -63,6 +63,8 @@ return { The first Phase D implementation slice exposes this boundary only through an offline CLI fixture command: `getter --data-dir update check --fixture `. The fixture is normalized JSON, not live provider output, and the command returns `network_required = false`, update-check status, selected candidate/artifact, and generated download/install action DTOs. It does not execute network providers, download files, persist download tasks, stream progress events, or invoke Android installers. +The second Phase D slice consumes those generated actions through an explicitly offline/fake task lifecycle: `task submit --request `, `task run `, `task list`, `task cancel `, `task events --after --limit `, and `task install-result --status `. This proves getter-owned persistent task state, cancellation, pollable event DTOs, and abstract install handoff recording without live network I/O, background runners, native streaming, Flutter task-state logic, or Android installer calls. + ## post_update Optional post-update hook. Most persistent state changes should remain in Rust core, not Lua. diff --git a/todo.md b/todo.md index a00381e4..8055bbad 100644 --- a/todo.md +++ b/todo.md @@ -457,7 +457,7 @@ Acceptance: Goal: move from static app/repo display to real update workflows. -Status: first getter-owned offline update-check slice in progress. The accepted minimal slice is intentionally not a provider/downloader/installer runtime. It defines normalized offline fixture input/output DTOs, reuses Rust getter update selection/version comparison, and adds `getter --data-dir update check --fixture ` returning selected update and generated download/install action DTOs with `network_required = false`. +Status: second getter-owned offline lifecycle slice in progress. The accepted minimal Phase D work remains intentionally offline/fake: it defines normalized offline update-check DTOs, reuses Rust getter update selection/version comparison, adds `getter --data-dir update check --fixture `, and now adds a command-driven fake task lifecycle for persisted task state, cancellation, pollable task events, and abstract install handoff result recording. It still does not run live providers, perform network downloads, run background workers, invoke Android installers, or add Flutter product task state. Completed tasks: @@ -466,20 +466,28 @@ Completed tasks: 3. Generate minimal download/install action DTOs for the selected artifact. 4. Add CLI command `update check --fixture `. 5. Add BDD coverage for update available, up to date, ignored latest fallback, ignored-only, unknown installed version, and malformed fixture. +6. Add getter-core task/event/install-handoff DTOs for the first offline lifecycle proof. +7. Add main DB task/event/install-handoff tables and storage APIs with TDD coverage. +8. Implement deterministic fake/offline downloader behavior beyond the previous placeholder crate: submit, run, cancel, list, poll events, and record install result. +9. Add CLI commands and BDD coverage for `task submit`, `task run`, `task list`, `task cancel`, `task events`, and `task install-result`. Remaining tasks: -1. Implement provider/downloader crate behavior beyond placeholders. -2. Add event stream/backpressure model. -3. Add persistent download task state and cancellation. -4. Add platform install handoff contract. -5. Add Flutter BDD for update/download user flows only after getter behavior exists. +1. Implement live provider/downloader behavior beyond the fake/offline proof. +2. Add native stream/backpressure runtime beyond the current pollable CLI/dev event contract. +3. Decide and implement background worker/restart/retry/resume policy for real downloads. +4. Define Android production install handoff URI/SAF/permission/notification details and wire platform adapter execution. +5. Add Flutter BDD for update/download user flows only after getter behavior exists and the bridge consumes getter task/event DTOs. Acceptance progress: -- CLI can run an offline fixture update check: done for first slice. -- Flutter displays getter events rather than calculating status itself: deferred until event DTO/bridge slice. -- Android platform adapter owns permissions/notifications/installer handoff: documented/deferred; no Android execution added in first slice. +- CLI can run an offline fixture update check: done. +- Getter can persist and list fake/offline task state: done for CLI/dev slice. +- Getter can cancel queued/running fake tasks and reject invalid terminal cancellation: done. +- Getter can expose pollable task events with cursor/limit: done for CLI/dev slice; native streaming remains deferred. +- Getter can record abstract install handoff requests/results: done for CLI/dev slice; Android installer execution remains deferred. +- Flutter displays getter events rather than calculating status itself: deferred until bridge/UI task DTO slice. +- Android platform adapter owns permissions/notifications/installer handoff: documented/deferred; no Android execution added in this slice. ## 9. Do-not-do list for the next agent From 1ba7614491d671216d96b537a16fbc5784e72bf2 Mon Sep 17 00:00:00 2001 From: Xiangzhe Date: Tue, 23 Jun 2026 11:37:12 +0800 Subject: [PATCH 19/85] feat(app): render getter task DTOs --- .../dev_test/cli_getter_adapter_test.dart | 51 +++++++ app_flutter/lib/cli_getter_adapter.dart | 77 +++++++++++ app_flutter/lib/getter_adapter.dart | 126 ++++++++++++++++++ app_flutter/lib/main.dart | 71 +++++++++- app_flutter/test/widget_test.dart | 45 ++++++- .../flutter-ui-feature-parity-and-testing.md | 1 + .../0007-flutter-getter-bridge-contract.md | 14 +- todo.md | 10 +- 8 files changed, 382 insertions(+), 13 deletions(-) diff --git a/app_flutter/dev_test/cli_getter_adapter_test.dart b/app_flutter/dev_test/cli_getter_adapter_test.dart index 8cbeb435..8292a836 100644 --- a/app_flutter/dev_test/cli_getter_adapter_test.dart +++ b/app_flutter/dev_test/cli_getter_adapter_test.dart @@ -16,6 +16,7 @@ void main() { final dataDir = Directory('${temp.path}/data')..createSync(); final repoDir = _createFixtureRepository(temp, 'official'); final bundle = _createLegacyBundle(temp); + final taskRequest = _createDownloadTaskRequest(temp); final adapter = CliGetterAdapter(executable: getterCli, dataDir: dataDir.path); @@ -33,6 +34,13 @@ void main() { 'import-room-bundle', bundle.path, ]); + _runGetter(getterCli, dataDir.path, [ + 'task', + 'submit', + '--request', + taskRequest.path, + ]); + _runGetter(getterCli, dataDir.path, ['task', 'run', 'task-1']); final repositories = adapter.listRepositories(); expect(repositories.map((repo) => repo.id), contains('official')); @@ -60,6 +68,26 @@ void main() { reports.singleWhere((report) => report.code == 'migration.imported').ok, isTrue); + final tasks = adapter.listDownloadTasks(); + final task = tasks.singleWhere((task) => task.id == 'task-1'); + expect(task.packageId, 'android/org.fdroid.fdroid'); + expect(task.status, 'succeeded'); + expect(task.downloadFileName, 'app.apk'); + expect(task.installHandoffId, 'handoff-1'); + + final eventPage = adapter.listTaskEvents(after: 0, limit: 10); + expect(eventPage.hasMore, isFalse); + expect(eventPage.nextCursor, greaterThanOrEqualTo(4)); + expect( + eventPage.events.map((event) => event.kind), + containsAll([ + 'task_created', + 'task_started', + 'task_succeeded', + 'install_handoff_requested', + ]), + ); + final snapshot = adapter.loadSnapshot(); expect(snapshot.status, 'Getter CLI ready'); expect(snapshot.repositories.map((repo) => repo.id), contains('official')); @@ -115,6 +143,29 @@ File _createLegacyBundle(Directory temp) { '''); } +File _createDownloadTaskRequest(Directory temp) { + return File('${temp.path}/download-request.json')..writeAsStringSync(''' +{ + "format": "getter-download-request", + "version": 1, + "package_id": "android/org.fdroid.fdroid", + "executor": "fake", + "actions": [ + { + "type": "download", + "url": "https://example.invalid/app.apk", + "file_name": "app.apk" + }, + { + "type": "install", + "installer": "android_package", + "file": "app.apk" + } + ] +} +'''); +} + void _runGetter(String getterCli, String dataDir, List args) { final result = Process.runSync( getterCli, diff --git a/app_flutter/lib/cli_getter_adapter.dart b/app_flutter/lib/cli_getter_adapter.dart index ac36571d..754d1c69 100644 --- a/app_flutter/lib/cli_getter_adapter.dart +++ b/app_flutter/lib/cli_getter_adapter.dart @@ -53,6 +53,28 @@ class CliGetterAdapter implements GetterAdapter { .toList(growable: false); } + @override + List listDownloadTasks() { + final json = _runGetter(const ['task', 'list']); + final tasks = _asList(_data(json)['tasks'], 'tasks'); + return tasks + .map((task) => _downloadTaskFromJson(_asMap(task, 'task'))) + .toList(growable: false); + } + + @override + TaskEventPage listTaskEvents({required int after, required int limit}) { + final json = _runGetter([ + 'task', + 'events', + '--after', + after.toString(), + '--limit', + limit.toString(), + ]); + return _taskEventPageFromJson(_data(json)); + } + @override GetterSnapshot loadSnapshot() { initialize(); @@ -158,6 +180,54 @@ MigrationReportSummary _migrationReportFromJson(Map json) { ); } +DownloadTaskSummary _downloadTaskFromJson(Map json) { + return DownloadTaskSummary( + id: _asString(json['id'], 'task.id'), + packageId: _asString(json['package_id'], 'task.package_id'), + status: _asString(json['status'], 'task.status'), + executor: _asString(json['executor'], 'task.executor'), + actions: _asList(json['actions'], 'task.actions') + .map((action) => _asMap(action, 'task.action')) + .toList(growable: false), + downloadFileName: _asString( + json['download_file_name'], + 'task.download_file_name', + ), + downloadedFile: _asOptionalString( + json['downloaded_file'], + 'task.downloaded_file', + ), + failureMessage: _asOptionalString( + json['failure_message'], + 'task.failure_message', + ), + installHandoffId: _asOptionalString( + json['install_handoff_id'], + 'task.install_handoff_id', + ), + ); +} + +TaskEventPage _taskEventPageFromJson(Map json) { + return TaskEventPage( + events: _asList(json['events'], 'task.events') + .map((event) => _taskEventFromJson(_asMap(event, 'task.event'))) + .toList(growable: false), + nextCursor: _asInt(json['next_cursor'], 'task.next_cursor'), + hasMore: _asBool(json['has_more'], 'task.has_more'), + ); +} + +TaskEventSummary _taskEventFromJson(Map json) { + return TaskEventSummary( + cursor: _asInt(json['cursor'], 'task.event.cursor'), + taskId: _asString(json['task_id'], 'task.event.task_id'), + kind: _asString(json['kind'], 'task.event.kind'), + status: _asOptionalString(json['status'], 'task.event.status'), + message: _asOptionalString(json['message'], 'task.event.message'), + ); +} + Map _asMap(Object? value, String name) { if (value is Map) { return value; @@ -185,6 +255,13 @@ String _asString(Object? value, String name) { throw FormatException('$name should be a string'); } +String? _asOptionalString(Object? value, String name) { + if (value == null || value is String) { + return value as String?; + } + throw FormatException('$name should be a string or null'); +} + int _asInt(Object? value, String name) { if (value is int) { return value; diff --git a/app_flutter/lib/getter_adapter.dart b/app_flutter/lib/getter_adapter.dart index 655b2a3d..f407d384 100644 --- a/app_flutter/lib/getter_adapter.dart +++ b/app_flutter/lib/getter_adapter.dart @@ -14,6 +14,10 @@ abstract interface class GetterAdapter { List readMigrationReports(); + List listDownloadTasks(); + + TaskEventPage listTaskEvents({required int after, required int limit}); + GetterSnapshot loadSnapshot(); } @@ -77,11 +81,81 @@ class FakeGetterAdapter implements GetterAdapter { ); } + static const _downloadTasks = [ + DownloadTaskSummary( + id: 'task-1', + packageId: 'android/org.fdroid.fdroid', + status: 'succeeded', + executor: 'fake', + actions: >[ + { + 'type': 'download', + 'url': 'https://example.invalid/app.apk', + 'file_name': 'app.apk', + }, + { + 'type': 'install', + 'installer': 'android_package', + 'file': 'app.apk', + }, + ], + downloadFileName: 'app.apk', + downloadedFile: 'app.apk', + failureMessage: null, + installHandoffId: 'handoff-1', + ), + ]; + + static const _taskEvents = TaskEventPage( + events: [ + TaskEventSummary( + cursor: 1, + taskId: 'task-1', + kind: 'task_created', + status: 'queued', + message: 'Task created', + ), + TaskEventSummary( + cursor: 2, + taskId: 'task-1', + kind: 'task_succeeded', + status: 'succeeded', + message: 'Task succeeded', + ), + TaskEventSummary( + cursor: 3, + taskId: 'task-1', + kind: 'install_handoff_requested', + status: 'succeeded', + message: 'Install handoff requested', + ), + ], + nextCursor: 3, + hasMore: false, + ); + @override List readMigrationReports() { return const []; } + @override + List listDownloadTasks() => _downloadTasks; + + @override + TaskEventPage listTaskEvents({required int after, required int limit}) { + final events = _taskEvents.events + .where((event) => event.cursor > after) + .take(limit) + .toList(growable: false); + final nextCursor = events.isEmpty ? after : events.last.cursor; + return TaskEventPage( + events: events, + nextCursor: nextCursor, + hasMore: _taskEvents.events.any((event) => event.cursor > nextCursor), + ); + } + @override GetterSnapshot loadSnapshot() => _snapshot; } @@ -171,6 +245,58 @@ class MigrationReportSummary { final int trackedRecords; } +class DownloadTaskSummary { + const DownloadTaskSummary({ + required this.id, + required this.packageId, + required this.status, + required this.executor, + required this.actions, + required this.downloadFileName, + required this.downloadedFile, + required this.failureMessage, + required this.installHandoffId, + }); + + final String id; + final String packageId; + final String status; + final String executor; + final List> actions; + final String downloadFileName; + final String? downloadedFile; + final String? failureMessage; + final String? installHandoffId; +} + +class TaskEventPage { + const TaskEventPage({ + required this.events, + required this.nextCursor, + required this.hasMore, + }); + + final List events; + final int nextCursor; + final bool hasMore; +} + +class TaskEventSummary { + const TaskEventSummary({ + required this.cursor, + required this.taskId, + required this.kind, + required this.status, + required this.message, + }); + + final int cursor; + final String taskId; + final String kind; + final String? status; + final String? message; +} + class GetterError { const GetterError({required this.code, required this.message, this.detail}); diff --git a/app_flutter/lib/main.dart b/app_flutter/lib/main.dart index 52bfcf4f..8e3c5caa 100644 --- a/app_flutter/lib/main.dart +++ b/app_flutter/lib/main.dart @@ -29,7 +29,9 @@ class AppKeys { static const getterStatus = ValueKey('state.getter_status'); static const appsList = ValueKey('state.apps_list'); static const repositoriesList = ValueKey('state.repositories_list'); + static const downloadsList = ValueKey('state.downloads_list'); static const downloadsEmpty = ValueKey('state.downloads_empty'); + static const taskEventsList = ValueKey('state.task_events_list'); static const logsEmpty = ValueKey('state.logs_empty'); static const settingsShell = ValueKey('state.settings_shell'); static const migrationReady = ValueKey('state.migration_ready'); @@ -38,6 +40,10 @@ class AppKeys { ValueKey('state.app.$packageId'); static ValueKey repoRow(String repositoryId) => ValueKey('state.repository.$repositoryId'); + static ValueKey downloadTaskRow(String taskId) => + ValueKey('state.download_task.$taskId'); + static ValueKey taskEventRow(int cursor) => + ValueKey('state.task_event.$cursor'); } class UpgradeAllApp extends StatelessWidget { @@ -57,7 +63,7 @@ class UpgradeAllApp extends StatelessWidget { '/': (context) => HomePage(getter: getter), '/apps': (context) => AppsPage(getter: getter), '/repositories': (context) => RepositoriesPage(getter: getter), - '/downloads': (context) => const DownloadsPage(), + '/downloads': (context) => DownloadsPage(getter: getter), '/logs': (context) => const LogsPage(), '/settings': (context) => const SettingsPage(), '/migration': (context) => const MigrationPage(), @@ -242,15 +248,68 @@ class RepositoriesPage extends StatelessWidget { } class DownloadsPage extends StatelessWidget { - const DownloadsPage({super.key}); + const DownloadsPage({super.key, required this.getter}); + + final GetterAdapter getter; @override Widget build(BuildContext context) { - return const _PlaceholderPage( + final tasks = getter.listDownloadTasks(); + final events = getter.listTaskEvents(after: 0, limit: 20).events; + return Scaffold( key: AppKeys.downloadsRoute, - title: 'Downloads', - stateKey: AppKeys.downloadsEmpty, - message: 'No download tasks yet', + appBar: AppBar(title: const Text('Downloads')), + body: tasks.isEmpty + ? const Center( + child: Text(key: AppKeys.downloadsEmpty, 'No download tasks yet'), + ) + : ListView( + padding: const EdgeInsets.all(16), + children: [ + Text('Tasks', style: Theme.of(context).textTheme.titleMedium), + const SizedBox(height: 8), + ListView.builder( + key: AppKeys.downloadsList, + shrinkWrap: true, + physics: const NeverScrollableScrollPhysics(), + itemCount: tasks.length, + itemBuilder: (context, index) { + final task = tasks[index]; + return Card( + child: ListTile( + key: AppKeys.downloadTaskRow(task.id), + title: Text(task.packageId), + subtitle: Text( + '${task.status} • ${task.downloadFileName}', + ), + trailing: task.installHandoffId == null + ? null + : const Chip(label: Text('Install handoff')), + ), + ); + }, + ), + const SizedBox(height: 16), + Text('Events', style: Theme.of(context).textTheme.titleMedium), + const SizedBox(height: 8), + ListView.builder( + key: AppKeys.taskEventsList, + shrinkWrap: true, + physics: const NeverScrollableScrollPhysics(), + itemCount: events.length, + itemBuilder: (context, index) { + final event = events[index]; + return ListTile( + key: AppKeys.taskEventRow(event.cursor), + title: Text(event.kind), + subtitle: Text( + '${event.taskId} • ${event.status ?? 'no status'}', + ), + ); + }, + ), + ], + ), ); } } diff --git a/app_flutter/test/widget_test.dart b/app_flutter/test/widget_test.dart index dcaee08f..47775c18 100644 --- a/app_flutter/test/widget_test.dart +++ b/app_flutter/test/widget_test.dart @@ -1,5 +1,6 @@ import 'package:flutter_test/flutter_test.dart'; +import 'package:upgradeall/getter_adapter.dart'; import 'package:upgradeall/main.dart'; void main() { @@ -50,17 +51,38 @@ void main() { expect(find.byKey(AppKeys.repoRow('local_autogen')), findsOneWidget); }); - testWidgets('placeholder routes expose stable empty-state keys', + testWidgets('downloads route renders getter task DTOs read-only', (tester) async { await tester.pumpWidget(const UpgradeAllApp()); await tester.tap(find.byKey(AppKeys.openDownloads)); await tester.pumpAndSettle(); + expect(find.byKey(AppKeys.downloadsRoute), findsOneWidget); - expect(find.byKey(AppKeys.downloadsEmpty), findsOneWidget); + expect(find.byKey(AppKeys.downloadsList), findsOneWidget); + expect(find.byKey(AppKeys.downloadTaskRow('task-1')), findsOneWidget); + expect(find.byKey(AppKeys.taskEventsList), findsOneWidget); + expect(find.byKey(AppKeys.taskEventRow(3)), findsOneWidget); + expect(find.text('Install handoff'), findsOneWidget); + }); - await tester.pageBack(); + testWidgets('downloads route exposes getter empty task state', + (tester) async { + await tester.pumpWidget( + const UpgradeAllApp(getter: _NoTaskGetterAdapter()), + ); + + await tester.tap(find.byKey(AppKeys.openDownloads)); await tester.pumpAndSettle(); + + expect(find.byKey(AppKeys.downloadsRoute), findsOneWidget); + expect(find.byKey(AppKeys.downloadsEmpty), findsOneWidget); + }); + + testWidgets('placeholder routes expose stable empty-state keys', + (tester) async { + await tester.pumpWidget(const UpgradeAllApp()); + await tester.tap(find.byKey(AppKeys.openLogs)); await tester.pumpAndSettle(); expect(find.byKey(AppKeys.logsRoute), findsOneWidget); @@ -81,3 +103,20 @@ void main() { expect(find.byKey(AppKeys.migrationReady), findsOneWidget); }); } + +class _NoTaskGetterAdapter extends FakeGetterAdapter { + const _NoTaskGetterAdapter(); + + @override + List listDownloadTasks() => + const []; + + @override + TaskEventPage listTaskEvents({required int after, required int limit}) { + return const TaskEventPage( + events: [], + nextCursor: 0, + hasMore: false, + ); + } +} diff --git a/docs/app/flutter-ui-feature-parity-and-testing.md b/docs/app/flutter-ui-feature-parity-and-testing.md index dfbccf81..74828026 100644 --- a/docs/app/flutter-ui-feature-parity-and-testing.md +++ b/docs/app/flutter-ui-feature-parity-and-testing.md @@ -75,6 +75,7 @@ The first Flutter implementation slice is intentionally a shell, not product log - `CliGetterAdapter` exercises a real getter data directory through the `getter-cli` JSON envelope for development/integration tests. - ADR-0007 documents the bridge contract and explicitly treats the CLI adapter as a test/development bridge, not the final Android production path. - Product decisions such as repository resolution, updates, migrations, storage, and downloads still belong in Rust getter. +- The downloads route may render getter task/event DTOs read-only, but it must not implement a Dart download task state machine, retry policy, or installer semantics. ## Test pyramid diff --git a/docs/architecture/adr/0007-flutter-getter-bridge-contract.md b/docs/architecture/adr/0007-flutter-getter-bridge-contract.md index ad48763f..2d9bbf60 100644 --- a/docs/architecture/adr/0007-flutter-getter-bridge-contract.md +++ b/docs/architecture/adr/0007-flutter-getter-bridge-contract.md @@ -30,7 +30,14 @@ readMigrationReports() loadSnapshot() ``` -`loadSnapshot()` composes the smaller getter-owned operations into the UI shell's first snapshot DTO. It must not perform repository resolution, version comparison, migration mapping, or update selection in Dart. `readMigrationReports()` must go through a getter operation such as `legacy report-list`; Flutter must not inspect getter's data-directory layout directly. +The second accepted API surface adds read-only task lifecycle DTO consumption for the already accepted offline/fake getter lifecycle: + +```text +listDownloadTasks() +listTaskEvents(after, limit) +``` + +`loadSnapshot()` composes the smaller getter-owned operations into the UI shell's first snapshot DTO. It must not perform repository resolution, version comparison, migration mapping, or update selection in Dart. `readMigrationReports()` must go through a getter operation such as `legacy report-list`; Flutter must not inspect getter's data-directory layout directly. `listDownloadTasks()` and `listTaskEvents()` render getter-owned task/event DTOs; Flutter must not synthesize task states, retry policy, installer behavior, or update decisions. ## Flutter DTOs @@ -43,6 +50,9 @@ RepositorySummary TrackedPackageSummary PackageEvaluation MigrationReportSummary +DownloadTaskSummary +TaskEventPage +TaskEventSummary GetterError ``` @@ -136,7 +146,7 @@ Costs: The first implementation slice must provide: - Flutter widget tests that continue to use `FakeGetterAdapter`. -- A Flutter/Dart integration test that builds or receives a real `getter-cli` binary, initializes a real getter data directory, and reads repositories, tracked packages, package evaluation output, and migration reports through `CliGetterAdapter`. +- A Flutter/Dart integration test that builds or receives a real `getter-cli` binary, initializes a real getter data directory, and reads repositories, tracked packages, package evaluation output, migration reports, and task lifecycle DTOs through `CliGetterAdapter`. - `just verify` coverage for the bridge integration test. ## Non-goals diff --git a/todo.md b/todo.md index 8055bbad..dd32364e 100644 --- a/todo.md +++ b/todo.md @@ -471,13 +471,19 @@ Completed tasks: 8. Implement deterministic fake/offline downloader behavior beyond the previous placeholder crate: submit, run, cancel, list, poll events, and record install result. 9. Add CLI commands and BDD coverage for `task submit`, `task run`, `task list`, `task cancel`, `task events`, and `task install-result`. +Completed additional UI/bridge slice: + +10. Extend Flutter getter bridge DTOs/adapters with read-only task list and event page APIs backed by existing getter CLI `task list` and `task events`. +11. Render getter-owned task/event DTOs on the Flutter Downloads route without adding a Dart task state machine. +12. Add Flutter widget/dev integration coverage for reading and rendering getter task lifecycle DTOs. + Remaining tasks: 1. Implement live provider/downloader behavior beyond the fake/offline proof. 2. Add native stream/backpressure runtime beyond the current pollable CLI/dev event contract. 3. Decide and implement background worker/restart/retry/resume policy for real downloads. 4. Define Android production install handoff URI/SAF/permission/notification details and wire platform adapter execution. -5. Add Flutter BDD for update/download user flows only after getter behavior exists and the bridge consumes getter task/event DTOs. +5. Add product-level Flutter BDD for update/download user flows after live/provider/background/installer decisions are accepted; the current slice only covers read-only DTO rendering. Acceptance progress: @@ -486,7 +492,7 @@ Acceptance progress: - Getter can cancel queued/running fake tasks and reject invalid terminal cancellation: done. - Getter can expose pollable task events with cursor/limit: done for CLI/dev slice; native streaming remains deferred. - Getter can record abstract install handoff requests/results: done for CLI/dev slice; Android installer execution remains deferred. -- Flutter displays getter events rather than calculating status itself: deferred until bridge/UI task DTO slice. +- Flutter displays getter task/event DTOs rather than calculating status itself: done for read-only CLI/dev bridge slice. - Android platform adapter owns permissions/notifications/installer handoff: documented/deferred; no Android execution added in this slice. ## 9. Do-not-do list for the next agent From aed83cb70cf9f8b2580ff811821acaa71d2af6bc Mon Sep 17 00:00:00 2001 From: Xiangzhe Date: Tue, 23 Jun 2026 15:00:55 +0800 Subject: [PATCH 20/85] ci(app): publish Flutter APK artifacts --- .github/workflows/android.yml | 189 +++++++++++------- app_flutter/README.md | 7 +- app_flutter/android/app/build.gradle | 31 ++- .../android/app/src/main/AndroidManifest.xml | 7 +- docs/README.md | 9 +- .../flutter-ui-feature-parity-and-testing.md | 6 +- docs/architecture/README.md | 1 + .../0002-getter-flutter-platform-boundary.md | 4 +- .../adr/0008-flutter-product-apk-entry.md | 50 +++++ .../upgradeall-getter-rewrite-wiki.md | 2 + todo.md | 23 ++- 11 files changed, 243 insertions(+), 86 deletions(-) create mode 100644 docs/architecture/adr/0008-flutter-product-apk-entry.md diff --git a/.github/workflows/android.yml b/.github/workflows/android.yml index 09fdf99a..48048de8 100644 --- a/.github/workflows/android.yml +++ b/.github/workflows/android.yml @@ -2,19 +2,9 @@ name: Android CI on: push: - branches: + branches: - master - paths-ignore: - - 'source/**' - - '**.md' - - '.**' - - 'fastlane/**' pull_request: - paths-ignore: - - 'source/**' - - '**.md' - - '.**' - - 'fastlane/**' workflow_dispatch: jobs: @@ -40,92 +30,153 @@ jobs: - name: Setup Android SDK uses: android-actions/setup-android@v3 - - name: Install NDK - run: echo "y" | sdkmanager --install "ndk;${{ env.NDK_VERSION }}" + - name: Install Android SDK packages + run: yes | sdkmanager --install "platforms;android-36" "build-tools;36.0.0" "platform-tools" "ndk;${{ env.NDK_VERSION }}" - - name: Install Cargo with aarch64-linux-android + - name: Add Android tools to environment + shell: bash + run: | + BUILD_TOOL_VERSION=$(ls "$ANDROID_HOME/build-tools" | sort -V | tail -n 1) + echo "BUILD_TOOL_VERSION=$BUILD_TOOL_VERSION" >> "$GITHUB_ENV" + echo "$ANDROID_HOME/build-tools/$BUILD_TOOL_VERSION" >> "$GITHUB_PATH" + echo "ANDROID_NDK_HOME=$ANDROID_HOME/ndk/$NDK_VERSION" >> "$GITHUB_ENV" + echo "ANDROID_NDK_ROOT=$ANDROID_HOME/ndk/$NDK_VERSION" >> "$GITHUB_ENV" + echo "Android build tools: $BUILD_TOOL_VERSION" + echo "Android NDK: $ANDROID_HOME/ndk/$NDK_VERSION" + + - name: Install Flutter + uses: subosito/flutter-action@v2 + with: + channel: stable + cache: true + + - name: Install Rust toolchain uses: dtolnay/rust-toolchain@stable with: - targets: aarch64-linux-android + components: clippy, rustfmt + + - name: Install Android Rust targets + run: rustup target add aarch64-linux-android armv7-linux-androideabi x86_64-linux-android - - name: Add Rust targe tarchitectures + - name: Install just run: | - rustup target add x86_64-linux-android - rustup target add armv7-linux-androideabi + if ! command -v just >/dev/null 2>&1; then + cargo install just --locked + fi - name: Retrieve version + shell: bash run: | - echo VERSION=$(git rev-parse --short HEAD) >> $GITHUB_ENV - - # Split due https://github.com/mozilla/rust-android-gradle/issues/38 - - name: Build with Gradle (debug) - run: ./gradlew -PappVerName=${{ env.VERSION }} assembleDebug - env: - ANDROID_NDK_HOME: ${{ steps.setup-ndk.outputs.ndk-path }} - - - name: Build with Gradle (release) - if: ${{ !github.event.pull_request }} - run: ./gradlew -PappVerName=${{ env.VERSION }} assembleRelease + VERSION=$(git rev-parse --short HEAD) + APP_VERSION=$(awk '/^version:/ {print $2}' app_flutter/pubspec.yaml) + FLUTTER_BASE_VERSION="${APP_VERSION%%+*}" + FLUTTER_BUILD_NUMBER="${APP_VERSION##*+}" + echo "VERSION=$VERSION" >> "$GITHUB_ENV" + echo "FLUTTER_BUILD_NAME=${FLUTTER_BASE_VERSION}_${VERSION}" >> "$GITHUB_ENV" + echo "FLUTTER_BUILD_NUMBER=$FLUTTER_BUILD_NUMBER" >> "$GITHUB_ENV" + + - name: Run rewrite validation + run: just verify + + - name: Build Android Rust bridge libraries + run: ./gradlew --no-daemon ':core-getter:buildDebugApi_proxyRust[arm64-v8a]' ':core-getter:buildDebugApi_proxyRust[armeabi-v7a]' ':core-getter:buildDebugApi_proxyRust[x86_64]' + + - name: Configure Flutter release signing + if: ${{ github.event_name != 'pull_request' }} + shell: bash env: - ANDROID_NDK_HOME: ${{ steps.setup-ndk.outputs.ndk-path }} + SIGNING_KEY: ${{ secrets.SIGNING_KEY }} + KEY_STORE_PASSWORD: ${{ secrets.KEY_STORE_PASSWORD }} + KEY_PASSWORD: ${{ secrets.KEY_PASSWORD }} + KEY_ALIAS: ${{ secrets.ALIAS }} + run: | + set -euo pipefail + : "${SIGNING_KEY:?SIGNING_KEY secret is required}" + : "${KEY_STORE_PASSWORD:?KEY_STORE_PASSWORD secret is required}" + : "${KEY_PASSWORD:?KEY_PASSWORD secret is required}" + : "${KEY_ALIAS:?ALIAS secret is required}" + printf '%s' "$SIGNING_KEY" | base64 --decode > app_flutter/android/upload-keystore.jks + { + printf 'storePassword=%s\n' "$KEY_STORE_PASSWORD" + printf 'keyPassword=%s\n' "$KEY_PASSWORD" + printf 'keyAlias=%s\n' "$KEY_ALIAS" + printf 'storeFile=../upload-keystore.jks\n' + } > app_flutter/android/key.properties + + - name: Build Flutter APK artifacts + shell: bash + run: | + cd app_flutter + flutter build apk --debug --build-name "$FLUTTER_BUILD_NAME" --build-number "$FLUTTER_BUILD_NUMBER" + flutter build apk --release --build-name "$FLUTTER_BUILD_NAME" --build-number "$FLUTTER_BUILD_NUMBER" - - name: Setup build tool version variable + - name: Locate Flutter APK artifacts shell: bash run: | - BUILD_TOOL_VERSION=$(ls /usr/local/lib/android/sdk/build-tools/ | tail -n 1) - echo "BUILD_TOOL_VERSION=$BUILD_TOOL_VERSION" >> $GITHUB_ENV - echo Last build tool version is: $BUILD_TOOL_VERSION - - - name: Sign Android release - if: ${{ !github.event.pull_request }} - id: sign - uses: r0adkll/sign-android-release@v1.0.4 + DEBUG_APK="app_flutter/build/app/outputs/flutter-apk/app-debug.apk" + RELEASE_APK="app_flutter/build/app/outputs/flutter-apk/app-release.apk" + test -f "$DEBUG_APK" + test -f "$RELEASE_APK" + echo "DEBUG_APK=$DEBUG_APK" >> "$GITHUB_ENV" + echo "RELEASE_APK=$RELEASE_APK" >> "$GITHUB_ENV" + + - name: Verify Flutter APK identity and signature + shell: bash env: - BUILD_TOOLS_VERSION: ${{ env.BUILD_TOOL_VERSION }} - with: - releaseDirectory: app/build/outputs/apk/release - signingKeyBase64: ${{ secrets.SIGNING_KEY }} - alias: ${{ secrets.ALIAS }} - keyStorePassword: ${{ secrets.KEY_STORE_PASSWORD }} - keyPassword: ${{ secrets.KEY_PASSWORD }} + EXPECTED_SIGNING_CERT_SHA256: ${{ secrets.EXPECTED_SIGNING_CERT_SHA256 }} + run: | + set -euo pipefail + aapt dump badging "$DEBUG_APK" | tee /tmp/upgradeall-debug-badging.txt + aapt dump badging "$RELEASE_APK" | tee /tmp/upgradeall-release-badging.txt + grep -F "package: name='net.xzos.upgradeall.debug'" /tmp/upgradeall-debug-badging.txt + grep -F "versionCode='$FLUTTER_BUILD_NUMBER'" /tmp/upgradeall-debug-badging.txt + grep -F "versionName='$FLUTTER_BUILD_NAME'" /tmp/upgradeall-debug-badging.txt + grep -F "package: name='net.xzos.upgradeall'" /tmp/upgradeall-release-badging.txt + grep -F "versionCode='$FLUTTER_BUILD_NUMBER'" /tmp/upgradeall-release-badging.txt + grep -F "versionName='$FLUTTER_BUILD_NAME'" /tmp/upgradeall-release-badging.txt + apksigner verify --print-certs "$RELEASE_APK" | tee /tmp/upgradeall-release-certs.txt + if [[ "${{ github.event_name }}" != "pull_request" ]] && grep -q "CN=Android Debug" /tmp/upgradeall-release-certs.txt; then + echo "::error::Flutter release APK is debug-signed; CI release signing did not take effect" + exit 1 + fi + actual_sha256=$(awk -F': ' '/Signer #1 certificate SHA-256 digest/ { gsub(":", "", $2); print toupper($2); exit }' /tmp/upgradeall-release-certs.txt) + expected_sha256=$(printf '%s' "$EXPECTED_SIGNING_CERT_SHA256" | tr -d ':[:space:]' | tr '[:lower:]' '[:upper:]') + if [[ -n "$expected_sha256" && "$actual_sha256" != "$expected_sha256" ]]; then + echo "::error::Flutter release APK signer SHA-256 does not match EXPECTED_SIGNING_CERT_SHA256" + exit 1 + fi + if [[ "${{ github.event_name }}" != "pull_request" && -z "$expected_sha256" ]]; then + echo "::warning::EXPECTED_SIGNING_CERT_SHA256 is not set; release signer identity was not pinned" + fi - - name: Upload debug apk + - name: Upload Flutter debug apk uses: actions/upload-artifact@v6 - if: ${{ !github.event.pull_request }} + if: ${{ github.event_name != 'pull_request' }} with: - path: './app/build/outputs/apk/debug/*.apk' - name: build_debug_${{ env.VERSION }} + path: ${{ env.DEBUG_APK }} + name: build_flutter_debug_${{ env.VERSION }} - - name: Upload release apk + - name: Upload Flutter release apk uses: actions/upload-artifact@v6 - if: ${{ !github.event.pull_request }} + if: ${{ github.event_name != 'pull_request' }} with: - path: ${{ steps.sign.outputs.signedReleaseFile }} - name: build_release_${{ env.VERSION }} + path: ${{ env.RELEASE_APK }} + name: build_flutter_release_${{ env.VERSION }} - name: Get apk info - if: ${{ !github.event.pull_request }} + if: ${{ github.event_name != 'pull_request' }} id: apk-info uses: hkusu/apk-info-action@v1 with: - apk-path: ${{ steps.sign.outputs.signedReleaseFile }} + apk-path: ${{ env.RELEASE_APK }} # - name: Upload mappings with App Center CLI -# if: ${{ !github.event.pull_request }} +# if: ${{ github.event_name == 'push' && github.ref == 'refs/heads/master' }} # uses: zhaobozhen/AppCenter-Github-Action@1.0.1 # with: -# command: appcenter crashes upload-mappings --mapping app/build/outputs/mapping/release/mapping.txt --version-name ${{ steps.apk-info.outputs.version-name }} --version-code ${{ steps.apk-info.outputs.version-code }} --app DUpdateSystem/UpgradeAll +# command: appcenter crashes upload-mappings --mapping app_flutter/build/app/outputs/mapping/release/mapping.txt --version-name ${{ steps.apk-info.outputs.version-name }} --version-code ${{ steps.apk-info.outputs.version-code }} --app DUpdateSystem/UpgradeAll # token: ${{secrets.APP_CENTER_TOKEN}} - - name: Find debug APK - if: ${{ github.event_name == 'push' && github.ref == 'refs/heads/master' }} - run: | - if [ ! -z "${{ secrets.BOT_TOKEN }}" ]; then - OUTPUT="app/build/outputs/apk/debug/" - DEBUG_APK=$(find $OUTPUT -name "*.apk") - echo "DEBUG_APK=$DEBUG_APK" >> $GITHUB_ENV - fi - - name: Generate Commit Message if: ${{ github.event_name == 'push' && github.ref == 'refs/heads/master' }} run: | @@ -139,7 +190,7 @@ jobs: \`\`\`$COMMIT_MESSAGE\`\`\` by \`$AUTHOR_NAME\` See commit detail [Here]($COMMIT_URL) - Snapshot apk is attached" + Flutter snapshot apk is attached" echo "TELEGRAM_MESSAGE<> $GITHUB_ENV echo "$TELEGRAM_MESSAGE" >> $GITHUB_ENV @@ -156,7 +207,7 @@ jobs: to-who: ${{ secrets.TELEGRAM_TO }} message: ${{ env.TELEGRAM_MESSAGE }} files: | - /github/workspace/${{ steps.sign.outputs.signedReleaseFile }} + /github/workspace/${{ env.RELEASE_APK }} /github/workspace/${{ env.DEBUG_APK }} - name: Delete workflow runs diff --git a/app_flutter/README.md b/app_flutter/README.md index bde21de9..cb6fef91 100644 --- a/app_flutter/README.md +++ b/app_flutter/README.md @@ -1,10 +1,11 @@ # UpgradeAll Flutter app -This is the new Flutter shell for the UpgradeAll rewrite. It must remain a UI and platform adapter around the Rust getter core; product logic, repository resolution, storage, and migration behavior belong in getter. +This is the new Flutter shell and product APK entry for the UpgradeAll rewrite. It must remain a UI and platform adapter around the Rust getter core; product logic, repository resolution, storage, and migration behavior belong in getter. The legacy Android `:app` UI is kept only as reference code during migration. ## Current slice -- Android application identity: `net.xzos.upgradeall` +- Android release application identity: `net.xzos.upgradeall` +- Android debug application identity: `net.xzos.upgradeall.debug` - Stable route/action/state keys for widget and future integration/dev tests - Placeholder routes for apps, repositories, downloads, logs, settings, and legacy migration - `FakeGetterAdapter` for deterministic widget tests @@ -21,3 +22,5 @@ GETTER_CLI_BIN=/path/to/getter-cli flutter test dev_test/cli_getter_adapter_test ``` From the repository root, `just verify` also runs the Flutter analyzer, widget tests, getter CLI integration/dev test, and Android debug build. + +Android CI/release artifacts are built from this Flutter project with `flutter build apk`; the root Gradle `:app` module is no longer the rewrite product APK path. diff --git a/app_flutter/android/app/build.gradle b/app_flutter/android/app/build.gradle index 76d0742d..d1cc0deb 100644 --- a/app_flutter/android/app/build.gradle +++ b/app_flutter/android/app/build.gradle @@ -22,6 +22,14 @@ if (flutterVersionName == null) { flutterVersionName = '0.20.0-alpha.4' } +// Release signing is configured by CI/local key.properties. Without it, +// release builds remain debug-signed so local Flutter builds keep working. +def keystoreProperties = new Properties() +def keystorePropertiesFile = rootProject.file('key.properties') +if (keystorePropertiesFile.exists()) { + keystoreProperties.load(new FileInputStream(keystorePropertiesFile)) +} + android { namespace "net.xzos.upgradeall" compileSdkVersion 36 @@ -48,11 +56,28 @@ android { versionName flutterVersionName } + signingConfigs { + release { + if (keystorePropertiesFile.exists()) { + keyAlias keystoreProperties['keyAlias'] + keyPassword keystoreProperties['keyPassword'] + storeFile keystoreProperties['storeFile'] ? file(keystoreProperties['storeFile']) : null + storePassword keystoreProperties['storePassword'] + } + } + } + buildTypes { + debug { + applicationIdSuffix ".debug" + } release { - // TODO: Add your own signing config for the release build. - // Signing with the debug keys for now, so `flutter run --release` works. - signingConfig signingConfigs.debug + if (keystorePropertiesFile.exists()) { + signingConfig signingConfigs.release + } else { + // Keep local release builds runnable when no private keystore is present. + signingConfig signingConfigs.debug + } } } } diff --git a/app_flutter/android/app/src/main/AndroidManifest.xml b/app_flutter/android/app/src/main/AndroidManifest.xml index 4a2e8949..7beddf2f 100644 --- a/app_flutter/android/app/src/main/AndroidManifest.xml +++ b/app_flutter/android/app/src/main/AndroidManifest.xml @@ -1,6 +1,11 @@ + + + Status: Accepted +> Date: 2026-06-23 +> Project: UpgradeAll rewrite — Flutter APP + Rust getter core + Lua package repository model + +## Decision + +`app_flutter/` is the only product Android application entry for the rewrite. + +The legacy Android `:app` module and its native Activity/Fragment/XML UI are kept temporarily as reference code only. They must not be treated as the shipped product APK path for the rewrite, and new product UI flows must not be added there. + +All user-visible flows migrate into Flutter. Android-native code remains allowed only for non-UI platform adapter responsibilities such as: + +- legacy Room database copy/checkpoint handoff; +- installed package inventory collection; +- Android permission prompts and capability adapters; +- SAF/file picker and URI permission plumbing; +- installer handoff adapters; +- notifications/foreground-service integration after the background-runtime design is accepted; +- native/FFI bridge code that exposes getter/platform DTOs to Flutter. + +## Build and release consequences + +- Android CI and release APK artifacts build from `app_flutter`, not the legacy root Gradle `:app` module. +- `app_flutter` keeps the production package name `net.xzos.upgradeall` for release builds. +- `app_flutter` debug builds use `net.xzos.upgradeall.debug` so debug snapshots can be installed beside the release package. +- Release signing belongs to the Flutter Android project. CI writes `app_flutter/android/key.properties` from repository secrets and runs `flutter build apk --release`. +- The old `:app` Gradle module may still be checked for reference/skeleton integrity, but `./gradlew :app:assembleDebug` or `./gradlew :app:assembleRelease` is no longer the product APK build path. + +## Rationale + +The rewrite goal is Flutter APP + Rust getter core + Lua package repositories. Keeping the native Android UI as the launcher would preserve the old shell as a product dependency and blur ownership boundaries. Making `app_flutter` the product APK entry lets Flutter own all screens and navigation while Rust getter owns product/domain/storage logic. + +Keeping the old native UI source temporarily reduces migration risk: it remains available for parity comparison while individual flows are rebuilt in Flutter. + +## Non-goals + +This ADR does not delete the legacy `:app` module yet. + +This ADR does not approve live provider/downloader/background-worker/installer runtime semantics. Those remain separate Phase D decisions. + +This ADR does not claim the current Flutter shell is product-complete. Until the production native/FFI bridge exists, CI can validate getter next to the Flutter APK, but the APK remains a rewrite shell/snapshot rather than a fully wired getter product. + +## Follow-up + +- Move every user-facing entry and flow into Flutter. +- Add platform adapters only where Android APIs are required. +- Delete or archive legacy native UI code after Flutter feature parity is reached. +- Once the production getter bridge exists, add APK-level validation that the Flutter product APK contains and exercises the intended native getter bridge. diff --git a/docs/architecture/upgradeall-getter-rewrite-wiki.md b/docs/architecture/upgradeall-getter-rewrite-wiki.md index 1d1b31d0..609e05bb 100644 --- a/docs/architecture/upgradeall-getter-rewrite-wiki.md +++ b/docs/architecture/upgradeall-getter-rewrite-wiki.md @@ -61,6 +61,8 @@ - Compose 依赖存在，但不是主 UI 架构。 - `core-getter` 已经有 Rust getter 的 JNI/RPC 集成，但目前仍是过渡形态。 +Rewrite 决策更新：`app_flutter/` 是新架构唯一产品 APK 入口；旧 `:app` 原生 UI 暂时保留为参考代码，但不再作为 rewrite 的发布/启动路径。Android CI/release 产物必须来自 Flutter app，旧 native UI 不能继续接收新的产品入口。 + ### 1.2 当前 Gradle 模块 现有模块： diff --git a/todo.md b/todo.md index dd32364e..0aa10fb3 100644 --- a/todo.md +++ b/todo.md @@ -357,7 +357,24 @@ Important boundary note: - Flutter still must not implement repository resolution, Lua validation/evaluation semantics, version comparison, migration mapping, provider/source selection, cache invalidation, or download task state machines. - If Flutter needs richer state, extend getter output first and cover it with getter tests. -## 8. Next product phases after bridge +## 8. Product APK entry switch + +Decision: `app_flutter/` is the only product APK entry for the rewrite. The old native `:app` module remains in the repository as reference code only; all user-visible entry points and future flows must move to Flutter. + +Completed tasks: + +1. Added ADR-0008 to record the Flutter product APK entry decision. +2. Switched Android CI away from root `./gradlew assembleDebug/assembleRelease` product builds. +3. Android CI now runs `just verify`, builds Android Rust bridge libraries for the supported ABIs, and builds Flutter debug/release APK artifacts from `app_flutter`. +4. Release artifacts, APK info, and Telegram upload paths now use `app_flutter/build/app/outputs/flutter-apk/*.apk`. +5. Flutter release builds keep package id `net.xzos.upgradeall`; Flutter debug builds use `net.xzos.upgradeall.debug`. + +Remaining follow-up: + +1. Once the production native/FFI getter bridge is wired into `app_flutter`, add APK-level validation that the Flutter product APK contains/exercises that bridge. +2. Delete/archive legacy native UI code after Flutter feature parity is reached. + +## 9. Next product phases after bridge ### Phase A: direct legacy Room migration @@ -495,7 +512,7 @@ Acceptance progress: - Flutter displays getter task/event DTOs rather than calculating status itself: done for read-only CLI/dev bridge slice. - Android platform adapter owns permissions/notifications/installer handoff: documented/deferred; no Android execution added in this slice. -## 9. Do-not-do list for the next agent +## 10. Do-not-do list for the next agent - Do not add more fake product screens before fixing CI and defining the bridge. - Do not move provider/update/storage/migration logic into Flutter. @@ -514,7 +531,7 @@ Acceptance progress: - `.pi/` - `context-build/` -## 10. Quick commands for the next session +## 11. Quick commands for the next session ```bash cd ~/Code/DUpdateSystem/UpgradeAll From 2e2276701b18656d4b0d9ea422e70e1d12d4b49f Mon Sep 17 00:00:00 2001 From: Xiangzhe Date: Tue, 23 Jun 2026 16:25:34 +0800 Subject: [PATCH 21/85] feat(app): add legacy Room migration adapter --- .../net/xzos/upgradeall/MainActivity.kt | 112 ++++++++++- .../dev_test/cli_getter_adapter_test.dart | 78 ++++++++ app_flutter/lib/cli_getter_adapter.dart | 74 +++++++ app_flutter/lib/getter_adapter.dart | 54 ++++++ .../lib/legacy_migration_platform.dart | 93 +++++++++ app_flutter/lib/main.dart | 182 +++++++++++++++++- app_flutter/test/widget_test.dart | 146 ++++++++++++++ .../adr/0003-legacy-room-migration.md | 4 +- .../0007-flutter-getter-bridge-contract.md | 36 +++- docs/migration/legacy-room-mapping.md | 2 +- todo.md | 9 +- 11 files changed, 771 insertions(+), 19 deletions(-) create mode 100644 app_flutter/lib/legacy_migration_platform.dart diff --git a/app_flutter/android/app/src/main/kotlin/net/xzos/upgradeall/MainActivity.kt b/app_flutter/android/app/src/main/kotlin/net/xzos/upgradeall/MainActivity.kt index f3d65028..babaf554 100644 --- a/app_flutter/android/app/src/main/kotlin/net/xzos/upgradeall/MainActivity.kt +++ b/app_flutter/android/app/src/main/kotlin/net/xzos/upgradeall/MainActivity.kt @@ -1,6 +1,116 @@ package net.xzos.upgradeall +import android.database.sqlite.SQLiteDatabase +import android.os.Handler +import android.os.Looper import io.flutter.embedding.android.FlutterActivity +import io.flutter.embedding.engine.FlutterEngine +import io.flutter.plugin.common.MethodChannel +import java.io.File +import java.util.concurrent.Executors -class MainActivity: FlutterActivity() { +class MainActivity : FlutterActivity() { + private val legacyMigrationExecutor = Executors.newSingleThreadExecutor() + private val mainHandler = Handler(Looper.getMainLooper()) + + override fun configureFlutterEngine(flutterEngine: FlutterEngine) { + super.configureFlutterEngine(flutterEngine) + MethodChannel( + flutterEngine.dartExecutor.binaryMessenger, + LEGACY_MIGRATION_CHANNEL, + ).setMethodCallHandler { call, result -> + when (call.method) { + "prepareLegacyRoomImport" -> { + legacyMigrationExecutor.execute { + try { + val candidate = prepareLegacyRoomImport() + mainHandler.post { result.success(candidate) } + } catch (error: Exception) { + mainHandler.post { + result.error( + "legacy.prepare_failed", + error.message ?: "Failed to prepare legacy Room database", + null, + ) + } + } + } + } + + else -> result.notImplemented() + } + } + } + + override fun onDestroy() { + legacyMigrationExecutor.shutdown() + super.onDestroy() + } + + private fun prepareLegacyRoomImport(): Map { + val source = getDatabasePath(LEGACY_ROOM_DB_NAME) + if (!source.exists()) { + return mapOf( + "found" to false, + "database_path" to null, + "message" to "No legacy Room database found", + ) + } + + val destination = File( + File(filesDir, "getter-imports/legacy-room"), + LEGACY_ROOM_DB_NAME, + ) + copySqliteTriplet(source, destination) + checkpointCopiedDatabase(destination) + + return mapOf( + "found" to true, + "database_path" to destination.absolutePath, + "message" to "Legacy Room database prepared", + ) + } + + private fun copySqliteTriplet(source: File, destination: File) { + destination.parentFile?.mkdirs() + SQLITE_SUFFIXES.forEach { suffix -> + val sourceFile = File(source.path + suffix) + val destinationFile = File(destination.path + suffix) + if (sourceFile.exists()) { + sourceFile.copyTo(destinationFile, overwrite = true) + } else if (destinationFile.exists()) { + destinationFile.delete() + } + } + } + + private fun checkpointCopiedDatabase(database: File) { + val db = SQLiteDatabase.openDatabase( + database.path, + null, + SQLiteDatabase.OPEN_READWRITE, + ) + try { + db.rawQuery("PRAGMA wal_checkpoint(FULL)", null).use { cursor -> + while (cursor.moveToNext()) { + // Drain the pragma result so SQLite performs the checkpoint. + } + } + db.rawQuery("PRAGMA journal_mode=DELETE", null).use { cursor -> + while (cursor.moveToNext()) { + // Drain the pragma result and leave a standalone import DB. + } + } + } finally { + db.close() + } + File(database.path + "-wal").delete() + File(database.path + "-shm").delete() + } + + private companion object { + const val LEGACY_MIGRATION_CHANNEL = "net.xzos.upgradeall/legacy_migration" + const val LEGACY_ROOM_DB_NAME = "app_metadata_database.db" + val SQLITE_SUFFIXES = listOf("", "-wal", "-shm") + } } diff --git a/app_flutter/dev_test/cli_getter_adapter_test.dart b/app_flutter/dev_test/cli_getter_adapter_test.dart index 8292a836..8b57ada5 100644 --- a/app_flutter/dev_test/cli_getter_adapter_test.dart +++ b/app_flutter/dev_test/cli_getter_adapter_test.dart @@ -4,6 +4,41 @@ import 'package:flutter_test/flutter_test.dart'; import 'package:upgradeall/cli_getter_adapter.dart'; void main() { + test('CliGetterAdapter imports a direct legacy Room database', () { + final getterCli = Platform.environment['GETTER_CLI_BIN']; + if (getterCli == null || getterCli.isEmpty) { + fail('GETTER_CLI_BIN must point to the built getter-cli binary'); + } + + final temp = Directory.systemTemp.createTempSync('upgradeall-getter-cli-'); + addTearDown(() => temp.deleteSync(recursive: true)); + + final dataDir = Directory('${temp.path}/data')..createSync(); + final legacyDb = _createLegacyRoomDatabase(temp); + final adapter = + CliGetterAdapter(executable: getterCli, dataDir: dataDir.path); + + adapter.initialize(); + final result = adapter.importLegacyRoomDatabase(legacyDb.path); + + expect(result.alreadyImported, isFalse); + expect(result.importedRecords, 1); + expect(result.sourceCounts?.appRows, 1); + expect(result.sourceCounts?.extraAppRows, 1); + final tracked = result.trackedPackages.singleWhere( + (package) => package.id == 'android/org.fdroid.fdroid', + ); + expect(tracked.favorite, isTrue); + expect(tracked.ignoredVersion, '1.20.0'); + expect(tracked.packageResolution, 'missing_package_definition'); + + final reports = adapter.readMigrationReports(); + expect( + reports.singleWhere((report) => report.code == 'migration.imported').ok, + isTrue, + ); + }); + test('CliGetterAdapter reads real getter repository and tracked state', () { final getterCli = Platform.environment['GETTER_CLI_BIN']; if (getterCli == null || getterCli.isEmpty) { @@ -16,6 +51,7 @@ void main() { final dataDir = Directory('${temp.path}/data')..createSync(); final repoDir = _createFixtureRepository(temp, 'official'); final bundle = _createLegacyBundle(temp); + final legacyDb = _createLegacyRoomDatabase(temp); final taskRequest = _createDownloadTaskRequest(temp); final adapter = CliGetterAdapter(executable: getterCli, dataDir: dataDir.path); @@ -68,6 +104,14 @@ void main() { reports.singleWhere((report) => report.code == 'migration.imported').ok, isTrue); + final alreadyImported = adapter.importLegacyRoomDatabase(legacyDb.path); + expect(alreadyImported.alreadyImported, isTrue); + expect(alreadyImported.importedRecords, 0); + expect( + alreadyImported.trackedPackages.map((package) => package.id), + contains('android/org.fdroid.fdroid'), + ); + final tasks = adapter.listDownloadTasks(); final task = tasks.singleWhere((task) => task.id == 'task-1'); expect(task.packageId, 'android/org.fdroid.fdroid'); @@ -143,6 +187,40 @@ File _createLegacyBundle(Directory temp) { '''); } +File _createLegacyRoomDatabase(Directory temp) { + final db = File('${temp.path}/app_metadata_database.db'); + final result = Process.runSync('python3', [ + '-c', + r''' +import sqlite3 +import sys +path = sys.argv[1] +conn = sqlite3.connect(path) +conn.execute('PRAGMA user_version = 17') +conn.execute('CREATE TABLE app (id INTEGER PRIMARY KEY AUTOINCREMENT, name TEXT NOT NULL, app_id TEXT NOT NULL, ignore_version_number TEXT, star INTEGER)') +conn.execute('CREATE TABLE extra_app (id INTEGER PRIMARY KEY AUTOINCREMENT, app_id TEXT NOT NULL, mark_version_number TEXT)') +app_id = '{"android_app_package":"org.fdroid.fdroid"}' +conn.execute( + 'INSERT INTO app(id, name, app_id, ignore_version_number, star) VALUES (1, ?, ?, ?, ?)', + ('F-Droid', app_id, '1.10.0', 1), +) +conn.execute( + 'INSERT INTO extra_app(id, app_id, mark_version_number) VALUES (1, ?, ?)', + (app_id, '1.20.0'), +) +conn.commit() +conn.close() +''', + db.path, + ]); + if (result.exitCode != 0) { + fail('failed to create legacy Room DB fixture\n' + 'stdout:\n${result.stdout}\n' + 'stderr:\n${result.stderr}'); + } + return db; +} + File _createDownloadTaskRequest(Directory temp) { return File('${temp.path}/download-request.json')..writeAsStringSync(''' { diff --git a/app_flutter/lib/cli_getter_adapter.dart b/app_flutter/lib/cli_getter_adapter.dart index 754d1c69..d6c75c0e 100644 --- a/app_flutter/lib/cli_getter_adapter.dart +++ b/app_flutter/lib/cli_getter_adapter.dart @@ -14,6 +14,9 @@ class CliGetterAdapter implements GetterAdapter { final String dataDir; final Map environment; + @override + bool get supportsLegacyRoomImport => true; + @override void initialize() { _runGetter(const ['init']); @@ -53,6 +56,16 @@ class CliGetterAdapter implements GetterAdapter { .toList(growable: false); } + @override + LegacyMigrationImportResult importLegacyRoomDatabase(String databasePath) { + final json = _runGetter([ + 'legacy', + 'import-room-db', + databasePath, + ]); + return _legacyMigrationImportResultFromJson(_data(json)); + } + @override List listDownloadTasks() { final json = _runGetter(const ['task', 'list']); @@ -180,6 +193,60 @@ MigrationReportSummary _migrationReportFromJson(Map json) { ); } +LegacyMigrationImportResult _legacyMigrationImportResultFromJson( + Map json, +) { + final warningsValue = json['warnings']; + final sourceCountsValue = json['source_counts']; + return LegacyMigrationImportResult( + alreadyImported: _asOptionalBool( + json['already_imported'], + 'migration.already_imported', + ) ?? + false, + importedRecords: _asInt(json['imported_records'], 'migration.imported'), + trackedPackages: _asList(json['apps'], 'migration.apps') + .map(_trackedPackageFromJson) + .toList(growable: false), + warnings: warningsValue == null + ? const [] + : _asList(warningsValue, 'migration.warnings') + .map((warning) => _migrationWarningFromJson( + _asMap(warning, 'migration.warning'), + )) + .toList(growable: false), + sourceCounts: sourceCountsValue == null + ? null + : _migrationSourceCountsFromJson( + _asMap(sourceCountsValue, 'migration.source_counts'), + ), + ); +} + +MigrationWarningSummary _migrationWarningFromJson(Map json) { + return MigrationWarningSummary( + code: _asString(json['code'], 'migration.warning.code'), + message: _asString(json['message'], 'migration.warning.message'), + ); +} + +MigrationSourceCounts _migrationSourceCountsFromJson( + Map json, +) { + return MigrationSourceCounts( + appRows: _asInt(json['app_rows'], 'migration.source_counts.app_rows'), + extraAppRows: _asInt( + json['extra_app_rows'], + 'migration.source_counts.extra_app_rows', + ), + hubRows: _asInt(json['hub_rows'], 'migration.source_counts.hub_rows'), + extraHubRows: _asInt( + json['extra_hub_rows'], + 'migration.source_counts.extra_hub_rows', + ), + ); +} + DownloadTaskSummary _downloadTaskFromJson(Map json) { return DownloadTaskSummary( id: _asString(json['id'], 'task.id'), @@ -275,3 +342,10 @@ bool _asBool(Object? value, String name) { } throw FormatException('$name should be a boolean'); } + +bool? _asOptionalBool(Object? value, String name) { + if (value == null || value is bool) { + return value as bool?; + } + throw FormatException('$name should be a boolean or null'); +} diff --git a/app_flutter/lib/getter_adapter.dart b/app_flutter/lib/getter_adapter.dart index f407d384..8d1a47fc 100644 --- a/app_flutter/lib/getter_adapter.dart +++ b/app_flutter/lib/getter_adapter.dart @@ -4,6 +4,8 @@ /// repository overlay resolution, update selection, Lua validation, migration /// mapping, and storage behavior belong in Rust getter. abstract interface class GetterAdapter { + bool get supportsLegacyRoomImport; + void initialize(); List listRepositories(); @@ -14,6 +16,8 @@ abstract interface class GetterAdapter { List readMigrationReports(); + LegacyMigrationImportResult importLegacyRoomDatabase(String databasePath); + List listDownloadTasks(); TaskEventPage listTaskEvents({required int after, required int limit}); @@ -43,6 +47,9 @@ class FakeGetterAdapter implements GetterAdapter { ], ); + @override + bool get supportsLegacyRoomImport => false; + @override void initialize() {} @@ -139,6 +146,16 @@ class FakeGetterAdapter implements GetterAdapter { return const []; } + @override + LegacyMigrationImportResult importLegacyRoomDatabase(String databasePath) { + throw const GetterBridgeException( + GetterError( + code: 'bridge.not_connected', + message: 'Getter migration import bridge is not connected', + ), + ); + } + @override List listDownloadTasks() => _downloadTasks; @@ -245,6 +262,43 @@ class MigrationReportSummary { final int trackedRecords; } +class LegacyMigrationImportResult { + const LegacyMigrationImportResult({ + required this.alreadyImported, + required this.importedRecords, + required this.trackedPackages, + required this.warnings, + required this.sourceCounts, + }); + + final bool alreadyImported; + final int importedRecords; + final List trackedPackages; + final List warnings; + final MigrationSourceCounts? sourceCounts; +} + +class MigrationWarningSummary { + const MigrationWarningSummary({required this.code, required this.message}); + + final String code; + final String message; +} + +class MigrationSourceCounts { + const MigrationSourceCounts({ + required this.appRows, + required this.extraAppRows, + required this.hubRows, + required this.extraHubRows, + }); + + final int appRows; + final int extraAppRows; + final int hubRows; + final int extraHubRows; +} + class DownloadTaskSummary { const DownloadTaskSummary({ required this.id, diff --git a/app_flutter/lib/legacy_migration_platform.dart b/app_flutter/lib/legacy_migration_platform.dart new file mode 100644 index 00000000..9b1a70a4 --- /dev/null +++ b/app_flutter/lib/legacy_migration_platform.dart @@ -0,0 +1,93 @@ +import 'package:flutter/services.dart'; + +import 'getter_adapter.dart'; + +/// Android platform boundary for preparing legacy Room databases for getter. +/// +/// This adapter is intentionally non-UI. Android code may locate, checkpoint, +/// and copy the legacy Room SQLite files, but Rust getter still owns migration +/// mapping/import semantics and Flutter owns all user-visible screens. +abstract interface class LegacyMigrationPlatform { + Future prepareLegacyRoomImport(); +} + +class LegacyRoomImportCandidate { + const LegacyRoomImportCandidate({ + required this.found, + required this.databasePath, + required this.message, + }); + + final bool found; + final String? databasePath; + final String? message; +} + +class MethodChannelLegacyMigrationPlatform implements LegacyMigrationPlatform { + const MethodChannelLegacyMigrationPlatform({ + MethodChannel channel = const MethodChannel( + 'net.xzos.upgradeall/legacy_migration', + ), + }) : _channel = channel; + + final MethodChannel _channel; + + @override + Future prepareLegacyRoomImport() async { + final Map? result; + try { + result = await _channel.invokeMapMethod( + 'prepareLegacyRoomImport', + ); + } on PlatformException catch (error) { + throw GetterBridgeException( + GetterError( + code: error.code, + message: error.message ?? 'Legacy migration platform adapter failed', + detail: error.details?.toString(), + ), + ); + } + if (result == null) { + throw const FormatException('legacy migration platform returned null'); + } + return _candidateFromJson(result); + } +} + +class NoopLegacyMigrationPlatform implements LegacyMigrationPlatform { + const NoopLegacyMigrationPlatform(); + + @override + Future prepareLegacyRoomImport() async { + return const LegacyRoomImportCandidate( + found: false, + databasePath: null, + message: 'Legacy migration platform adapter is not connected', + ); + } +} + +LegacyRoomImportCandidate _candidateFromJson(Map json) { + final found = json['found']; + final databasePath = json['database_path']; + final message = json['message']; + if (found is! bool) { + throw const FormatException('legacy migration found should be a boolean'); + } + if (databasePath != null && databasePath is! String) { + throw const FormatException( + 'legacy migration database_path should be a string or null', + ); + } + if (message != null && message is! String) { + throw const FormatException( + 'legacy migration message should be a string or null', + ); + } + return LegacyRoomImportCandidate( + found: found, + databasePath: databasePath as String?, + message: message as String?, + ); +} diff --git a/app_flutter/lib/main.dart b/app_flutter/lib/main.dart index 8e3c5caa..9d3dced6 100644 --- a/app_flutter/lib/main.dart +++ b/app_flutter/lib/main.dart @@ -1,9 +1,14 @@ import 'package:flutter/material.dart'; import 'getter_adapter.dart'; +import 'legacy_migration_platform.dart'; void main() { - runApp(const UpgradeAllApp()); + runApp( + const UpgradeAllApp( + legacyMigrationPlatform: MethodChannelLegacyMigrationPlatform(), + ), + ); } @visibleForTesting @@ -24,6 +29,8 @@ class AppKeys { static const openSettings = ValueKey('action.open_settings'); static const openMigration = ValueKey('action.open_migration'); static const openFirstApp = ValueKey('action.open_first_app'); + static const startLegacyMigration = + ValueKey('action.start_legacy_migration'); static const updateSummary = ValueKey('state.update_summary'); static const getterStatus = ValueKey('state.getter_status'); @@ -35,6 +42,13 @@ class AppKeys { static const logsEmpty = ValueKey('state.logs_empty'); static const settingsShell = ValueKey('state.settings_shell'); static const migrationReady = ValueKey('state.migration_ready'); + static const migrationStatus = ValueKey('state.migration_status'); + static const migrationBridgeUnavailable = + ValueKey('state.migration_bridge_unavailable'); + static const migrationImported = ValueKey('state.migration_imported'); + static const migrationError = ValueKey('state.migration_error'); + static const migrationReportsList = + ValueKey('state.migration_reports_list'); static ValueKey appRow(String packageId) => ValueKey('state.app.$packageId'); @@ -47,9 +61,14 @@ class AppKeys { } class UpgradeAllApp extends StatelessWidget { - const UpgradeAllApp({super.key, this.getter = const FakeGetterAdapter()}); + const UpgradeAllApp({ + super.key, + this.getter = const FakeGetterAdapter(), + this.legacyMigrationPlatform = const NoopLegacyMigrationPlatform(), + }); final GetterAdapter getter; + final LegacyMigrationPlatform legacyMigrationPlatform; @override Widget build(BuildContext context) { @@ -66,7 +85,10 @@ class UpgradeAllApp extends StatelessWidget { '/downloads': (context) => DownloadsPage(getter: getter), '/logs': (context) => const LogsPage(), '/settings': (context) => const SettingsPage(), - '/migration': (context) => const MigrationPage(), + '/migration': (context) => MigrationPage( + getter: getter, + legacyMigrationPlatform: legacyMigrationPlatform, + ), }, onGenerateRoute: (settings) { if (settings.name == '/apps/detail') { @@ -342,16 +364,158 @@ class SettingsPage extends StatelessWidget { } } -class MigrationPage extends StatelessWidget { - const MigrationPage({super.key}); +class MigrationPage extends StatefulWidget { + const MigrationPage({ + super.key, + required this.getter, + required this.legacyMigrationPlatform, + }); + + final GetterAdapter getter; + final LegacyMigrationPlatform legacyMigrationPlatform; + + @override + State createState() => _MigrationPageState(); +} + +class _MigrationPageState extends State { + late List _reports; + LegacyMigrationImportResult? _importResult; + String? _status; + GetterError? _error; + bool _running = false; + + @override + void initState() { + super.initState(); + _reports = widget.getter.readMigrationReports(); + } + + Future _startMigration() async { + setState(() { + _running = true; + _status = 'Preparing legacy Room database'; + _error = null; + }); + + try { + final candidate = + await widget.legacyMigrationPlatform.prepareLegacyRoomImport(); + if (!mounted) return; + if (!candidate.found || candidate.databasePath == null) { + setState(() { + _status = candidate.message ?? 'No legacy Room database found'; + _running = false; + }); + return; + } + + final importResult = + widget.getter.importLegacyRoomDatabase(candidate.databasePath!); + final reports = widget.getter.readMigrationReports(); + if (!mounted) return; + setState(() { + _importResult = importResult; + _reports = reports; + _status = importResult.alreadyImported + ? 'Legacy migration was already completed' + : 'Legacy migration imported ${importResult.importedRecords} records'; + _running = false; + }); + } on GetterBridgeException catch (error) { + if (!mounted) return; + setState(() { + _error = error.error; + _status = error.error.message; + _reports = widget.getter.readMigrationReports(); + _running = false; + }); + } catch (error) { + if (!mounted) return; + setState(() { + _error = GetterError( + code: 'platform.legacy_migration_error', + message: 'Legacy migration platform adapter failed', + detail: error.toString(), + ); + _status = 'Legacy migration platform adapter failed'; + _running = false; + }); + } + } @override Widget build(BuildContext context) { - return const _PlaceholderPage( + final canImportLegacyRoom = widget.getter.supportsLegacyRoomImport; + return Scaffold( key: AppKeys.migrationRoute, - title: 'Legacy migration', - stateKey: AppKeys.migrationReady, - message: 'Ready to show migration reports', + appBar: AppBar(title: const Text('Legacy migration')), + body: ListView( + padding: const EdgeInsets.all(16), + children: [ + ElevatedButton.icon( + key: AppKeys.startLegacyMigration, + onPressed: + _running || !canImportLegacyRoom ? null : _startMigration, + icon: const Icon(Icons.move_down), + label: Text(_running ? 'Migrating…' : 'Start legacy migration'), + ), + if (!canImportLegacyRoom) + const Padding( + padding: EdgeInsets.only(top: 12), + child: Text( + key: AppKeys.migrationBridgeUnavailable, + 'Getter migration bridge is not connected', + ), + ), + const SizedBox(height: 16), + if (_status == null && _reports.isEmpty) + const Text( + key: AppKeys.migrationReady, + 'Ready to show migration reports', + ), + if (_status != null) Text(key: AppKeys.migrationStatus, _status!), + if (_importResult != null) + Padding( + padding: const EdgeInsets.only(top: 12), + child: Text( + key: AppKeys.migrationImported, + '${_importResult!.trackedPackages.length} tracked packages after import', + ), + ), + if (_error != null) + Padding( + padding: const EdgeInsets.only(top: 12), + child: Text( + key: AppKeys.migrationError, + '${_error!.code}: ${_error!.message}', + ), + ), + if (_reports.isNotEmpty) ...[ + const SizedBox(height: 16), + Text('Reports', style: Theme.of(context).textTheme.titleMedium), + const SizedBox(height: 8), + ListView.builder( + key: AppKeys.migrationReportsList, + shrinkWrap: true, + physics: const NeverScrollableScrollPhysics(), + itemCount: _reports.length, + itemBuilder: (context, index) { + final report = _reports[index]; + return ListTile( + title: Text(report.code), + subtitle: Text( + '${report.message} • imported ${report.importedRecords}', + ), + trailing: report.ok + ? const Icon(Icons.check_circle, color: Colors.green) + : const Icon(Icons.error, color: Colors.red), + ); + }, + ), + ], + ], + ), ); } } diff --git a/app_flutter/test/widget_test.dart b/app_flutter/test/widget_test.dart index 47775c18..cc1d6c87 100644 --- a/app_flutter/test/widget_test.dart +++ b/app_flutter/test/widget_test.dart @@ -1,6 +1,8 @@ +import 'package:flutter/material.dart'; import 'package:flutter_test/flutter_test.dart'; import 'package:upgradeall/getter_adapter.dart'; +import 'package:upgradeall/legacy_migration_platform.dart'; import 'package:upgradeall/main.dart'; void main() { @@ -79,6 +81,70 @@ void main() { expect(find.byKey(AppKeys.downloadsEmpty), findsOneWidget); }); + testWidgets('migration route imports prepared legacy DB through getter', + (tester) async { + final getter = _MigrationGetterAdapter(); + await tester.pumpWidget( + UpgradeAllApp( + getter: getter, + legacyMigrationPlatform: const _PreparedLegacyMigrationPlatform( + '/tmp/app_metadata_database.db', + ), + ), + ); + + await tester.tap(find.byKey(AppKeys.openMigration)); + await tester.pumpAndSettle(); + await tester.tap(find.byKey(AppKeys.startLegacyMigration)); + await tester.pumpAndSettle(); + + expect(getter.importedDatabasePath, '/tmp/app_metadata_database.db'); + expect(find.byKey(AppKeys.migrationStatus), findsOneWidget); + expect(find.text('Legacy migration imported 1 records'), findsOneWidget); + expect(find.byKey(AppKeys.migrationImported), findsOneWidget); + expect(find.byKey(AppKeys.migrationReportsList), findsOneWidget); + expect(find.text('migration.imported'), findsOneWidget); + }); + + testWidgets('migration route reports missing legacy DB from platform adapter', + (tester) async { + await tester.pumpWidget( + const UpgradeAllApp( + getter: _LegacyMigrationCapableGetterAdapter(), + legacyMigrationPlatform: _MissingLegacyMigrationPlatform(), + ), + ); + + await tester.tap(find.byKey(AppKeys.openMigration)); + await tester.pumpAndSettle(); + await tester.tap(find.byKey(AppKeys.startLegacyMigration)); + await tester.pumpAndSettle(); + + expect(find.byKey(AppKeys.migrationStatus), findsOneWidget); + expect(find.text('No legacy Room database found'), findsOneWidget); + expect(find.byKey(AppKeys.migrationImported), findsNothing); + }); + + testWidgets('migration route disables import when getter bridge is absent', + (tester) async { + await tester.pumpWidget( + const UpgradeAllApp( + legacyMigrationPlatform: _PreparedLegacyMigrationPlatform( + '/tmp/app_metadata_database.db', + ), + ), + ); + + await tester.tap(find.byKey(AppKeys.openMigration)); + await tester.pumpAndSettle(); + + final button = tester.widget( + find.byKey(AppKeys.startLegacyMigration), + ); + expect(button.onPressed, isNull); + expect(find.byKey(AppKeys.migrationBridgeUnavailable), findsOneWidget); + }); + testWidgets('placeholder routes expose stable empty-state keys', (tester) async { await tester.pumpWidget(const UpgradeAllApp()); @@ -120,3 +186,83 @@ class _NoTaskGetterAdapter extends FakeGetterAdapter { ); } } + +class _LegacyMigrationCapableGetterAdapter extends FakeGetterAdapter { + const _LegacyMigrationCapableGetterAdapter(); + + @override + bool get supportsLegacyRoomImport => true; +} + +class _MigrationGetterAdapter extends FakeGetterAdapter { + String? importedDatabasePath; + @override + bool get supportsLegacyRoomImport => true; + var _reports = const []; + + @override + LegacyMigrationImportResult importLegacyRoomDatabase(String databasePath) { + importedDatabasePath = databasePath; + _reports = const [ + MigrationReportSummary( + ok: true, + code: 'migration.imported', + message: 'Legacy Room database imported', + importedRecords: 1, + trackedRecords: 1, + ), + ]; + return const LegacyMigrationImportResult( + alreadyImported: false, + importedRecords: 1, + trackedPackages: [ + TrackedPackageSummary( + id: 'android/org.fdroid.fdroid', + enabled: true, + favorite: true, + ignoredVersion: '1.20.0', + repositoryId: null, + packageResolution: 'missing_package_definition', + ), + ], + warnings: [], + sourceCounts: MigrationSourceCounts( + appRows: 1, + extraAppRows: 1, + hubRows: 0, + extraHubRows: 0, + ), + ); + } + + @override + List readMigrationReports() => _reports; +} + +class _PreparedLegacyMigrationPlatform implements LegacyMigrationPlatform { + const _PreparedLegacyMigrationPlatform(this.databasePath); + + final String databasePath; + + @override + Future prepareLegacyRoomImport() async { + return LegacyRoomImportCandidate( + found: true, + databasePath: databasePath, + message: 'Legacy Room database prepared', + ); + } +} + +class _MissingLegacyMigrationPlatform implements LegacyMigrationPlatform { + const _MissingLegacyMigrationPlatform(); + + @override + Future prepareLegacyRoomImport() async { + return const LegacyRoomImportCandidate( + found: false, + databasePath: null, + message: 'No legacy Room database found', + ); + } +} diff --git a/docs/architecture/adr/0003-legacy-room-migration.md b/docs/architecture/adr/0003-legacy-room-migration.md index 3cd39c3a..48c7c208 100644 --- a/docs/architecture/adr/0003-legacy-room-migration.md +++ b/docs/architecture/adr/0003-legacy-room-migration.md @@ -63,7 +63,9 @@ The Rust CLI now has a direct SQLite import slice for copied/checkpointed Room v getter --data-dir legacy import-room-db ``` -The direct importer opens the DB read-only, requires `PRAGMA user_version = 17`, reads legacy `app` and `extra_app` rows, maps known app-id keys to `android/` or `magisk/`, writes getter tracked package state plus the `legacy-room-v17` migration record in one transaction, and emits sanitized report counts/warnings. Current `hub` and `extra_hub` rows are not imported as top-level objects; they are counted/dropped with warnings until a later accepted mapping exists. Android/platform code is still responsible for producing a WAL/SHM-consistent copied DB file before invoking getter. +The direct importer opens the DB read-only, requires `PRAGMA user_version = 17`, reads legacy `app` and `extra_app` rows, maps known app-id keys to `android/` or `magisk/`, writes getter tracked package state plus the `legacy-room-v17` migration record in one transaction, and emits sanitized report counts/warnings. Current `hub` and `extra_hub` rows are not imported as top-level objects; they are counted/dropped with warnings until a later accepted mapping exists. + +The first Flutter/Android migration UX slice adds a no-UI Android platform adapter that locates `app_metadata_database.db`, copies the SQLite triplet (`.db`, `-wal`, `-shm`) into an app-private getter-import path, checkpoints/canonicalizes the copy, and returns that copied DB path to Flutter. Flutter starts the flow and renders getter-owned reports. Getter still owns the actual import operation; the default product APK keeps the action disabled until the production getter import bridge is connected. The host-side CLI also keeps the deterministic JSON bridge bundle for tests and non-Android fixtures: diff --git a/docs/architecture/adr/0007-flutter-getter-bridge-contract.md b/docs/architecture/adr/0007-flutter-getter-bridge-contract.md index 2d9bbf60..c9ec5bf3 100644 --- a/docs/architecture/adr/0007-flutter-getter-bridge-contract.md +++ b/docs/architecture/adr/0007-flutter-getter-bridge-contract.md @@ -37,6 +37,14 @@ listDownloadTasks() listTaskEvents(after, limit) ``` +The third accepted API surface adds the first legacy migration action boundary: + +```text +importLegacyRoomDatabase(databasePath) +``` + +The Android platform adapter may prepare a copied/checkpointed legacy Room SQLite file and return its path to Flutter, but getter still owns the actual `legacy import-room-db` import semantics. Flutter starts the flow and renders getter reports; it must not inspect or map Room tables directly. + `loadSnapshot()` composes the smaller getter-owned operations into the UI shell's first snapshot DTO. It must not perform repository resolution, version comparison, migration mapping, or update selection in Dart. `readMigrationReports()` must go through a getter operation such as `legacy report-list`; Flutter must not inspect getter's data-directory layout directly. `listDownloadTasks()` and `listTaskEvents()` render getter-owned task/event DTOs; Flutter must not synthesize task states, retry policy, installer behavior, or update decisions. ## Flutter DTOs @@ -50,6 +58,9 @@ RepositorySummary TrackedPackageSummary PackageEvaluation MigrationReportSummary +LegacyMigrationImportResult +MigrationWarningSummary +MigrationSourceCounts DownloadTaskSummary TaskEventPage TaskEventSummary @@ -97,6 +108,26 @@ The bridge maps getter errors into `GetterError`: Flutter may choose presentation, but the source classification belongs to getter or a documented platform adapter. +## Legacy migration platform adapter + +Flutter owns the migration screen and user-visible flow. Android-native code exposes a no-UI platform adapter over `net.xzos.upgradeall/legacy_migration` with `prepareLegacyRoomImport`. + +That adapter may: + +- locate `app_metadata_database.db` in the app database directory; +- copy the SQLite triplet (`.db`, `-wal`, `-shm`) into an app-private getter-import path; +- checkpoint/canonicalize the copied database into a standalone SQLite file; +- return `{ found, database_path, message }` to Flutter. + +That adapter must not: + +- show Android-native UI; +- map legacy rows into package IDs; +- decide what fields are dropped/imported; +- write getter storage directly. + +Flutter then calls a getter bridge operation equivalent to `legacy import-room-db ` and renders getter-owned reports. + ## Event model The first bridge slice is snapshot-only. Streaming events, progress, cancellation, backpressure, foreground services, notification lifecycle, and installer handoff are explicitly deferred to the update/download/install lifecycle ADR/work. @@ -146,13 +177,14 @@ Costs: The first implementation slice must provide: - Flutter widget tests that continue to use `FakeGetterAdapter`. -- A Flutter/Dart integration test that builds or receives a real `getter-cli` binary, initializes a real getter data directory, and reads repositories, tracked packages, package evaluation output, migration reports, and task lifecycle DTOs through `CliGetterAdapter`. +- Flutter widget tests for the migration flow using fake platform/getter adapters. +- A Flutter/Dart integration test that builds or receives a real `getter-cli` binary, initializes a real getter data directory, and reads repositories, tracked packages, package evaluation output, migration reports, direct Room import output, and task lifecycle DTOs through `CliGetterAdapter`. - `just verify` coverage for the bridge integration test. ## Non-goals - No full FFI/native bridge implementation in this ADR. - No update/download/install event stream. -- No direct Android Room DB reader. +- No Android-owned legacy Room mapping/import semantics; Android only prepares a copied DB file for getter. - No product-complete Flutter UI. - No product/domain decisions in Dart. diff --git a/docs/migration/legacy-room-mapping.md b/docs/migration/legacy-room-mapping.md index 97065401..a3e0ee5e 100644 --- a/docs/migration/legacy-room-mapping.md +++ b/docs/migration/legacy-room-mapping.md @@ -97,7 +97,7 @@ Currently dropped with warnings: - hub auth/API keys/provider credentials; - app regex/cloud config fields whose new package equivalent is not accepted yet. -The direct CLI reader expects Android/platform code to provide a WAL/SHM-consistent DB copy; it does not perform Android Room checkpointing itself. +The direct CLI reader expects Android/platform code to provide a WAL/SHM-consistent DB copy; it does not perform Android Room checkpointing itself. The first Flutter APK migration-adapter slice prepares that input with a no-UI Android MethodChannel adapter that copies the SQLite triplet (`.db`, `-wal`, `-shm`), checkpoints/canonicalizes the copy in app-private storage, and returns the copied DB path for Flutter to pass to getter. The default product migration action remains disabled until the production getter import bridge is connected. ## Current CLI bridge bundle diff --git a/todo.md b/todo.md index 0aa10fb3..c082127c 100644 --- a/todo.md +++ b/todo.md @@ -380,7 +380,7 @@ Remaining follow-up: Goal: replace bridge-only JSON import with the Android upgrade path. -Status: first getter-owned direct DB slice completed. The getter CLI now supports `legacy import-room-db ` for copied/checkpointed Room v17 SQLite files. It reads `app` and `extra_app`, maps known legacy app-id keys, writes `tracked_packages` plus `legacy-room-v17` in one transaction, prevents rerun, emits sanitized reports, and documents dropped hub/extra_hub fields. Android-side WAL/SHM copy/checkpoint and Flutter migration UX remain future work. +Status: first getter-owned direct DB slice completed, and the first Flutter/platform UX slice is in progress. The getter CLI now supports `legacy import-room-db ` for copied/checkpointed Room v17 SQLite files. It reads `app` and `extra_app`, maps known legacy app-id keys, writes `tracked_packages` plus `legacy-room-v17` in one transaction, prevents rerun, emits sanitized reports, and documents dropped hub/extra_hub fields. Android-side code now exposes a no-UI MethodChannel adapter that locates, copies, and checkpoints the legacy SQLite triplet for Flutter to pass into getter. Flutter has a migration page that starts this adapter flow and renders getter reports, but the default product APK keeps the action disabled until the production getter import bridge replaces the dev CLI adapter. Completed tasks: @@ -393,10 +393,9 @@ Completed tasks: Remaining tasks: -1. Android migrator copies old DB plus `-wal` and `-shm` safely before invoking getter. -2. Android/platform adapter opens/checkpoints/canonicalizes old Room schema to latest supported legacy version. +1. Wire a production getter bridge for `importLegacyRoomDatabase` in the Flutter APK; until then the default product migration action stays disabled with an explicit bridge-unavailable state. +2. Add focused native adapter coverage for SQLite triplet copy/checkpoint behavior if practical. 3. Extend accepted mapping if future ADR accepts direct `hub`/`extra_hub` semantics. -4. Flutter migration page starts the adapter flow and renders getter reports. Acceptance progress: @@ -407,7 +406,7 @@ Acceptance progress: - Mixed valid/invalid app rows import valid rows and warn: done. - DBs with app rows but zero importable rows fail with recovery report: done. - Report sanitization for dropped `hub`/`extra_hub` secrets and URL rewrite data: done. -- WAL/SHM pending writes: pending Android adapter slice. +- WAL/SHM pending writes: first Android adapter copy/checkpoint slice implemented; focused native adapter test still pending. - Per-app failures become warnings; global unreadable DB becomes recovery state, not crash: done for the getter-owned direct importer. ### Phase B: `local_autogen` generation From 39af4c702850082e32e1848ede43d240a6b2ac6f Mon Sep 17 00:00:00 2001 From: Xiangzhe Date: Tue, 23 Jun 2026 19:18:03 +0800 Subject: [PATCH 22/85] feat(app): add Android platform adapter spine --- .../upgradeall-rewrite-validation.yml | 3 + app_flutter/README.md | 3 +- .../android/app/src/main/AndroidManifest.xml | 4 + .../src/main/rust/platform_adapter/Cargo.toml | 16 ++ .../main/rust/platform_adapter/src/android.rs | 144 +++++++++++ .../src/main/rust/platform_adapter/src/lib.rs | 227 ++++++++++++++++++ docs/README.md | 9 +- .../flutter-ui-feature-parity-and-testing.md | 1 + docs/architecture/README.md | 1 + .../0002-getter-flutter-platform-boundary.md | 18 +- .../0007-flutter-getter-bridge-contract.md | 11 + .../adr/0008-flutter-product-apk-entry.md | 2 +- ...platform-adapter-and-package-visibility.md | 145 +++++++++++ .../upgradeall-getter-rewrite-wiki.md | 45 ++-- docs/lua-api/templates.md | 18 +- justfile | 5 + todo.md | 7 +- 17 files changed, 615 insertions(+), 44 deletions(-) create mode 100644 core-getter/src/main/rust/platform_adapter/Cargo.toml create mode 100644 core-getter/src/main/rust/platform_adapter/src/android.rs create mode 100644 core-getter/src/main/rust/platform_adapter/src/lib.rs create mode 100644 docs/architecture/adr/0009-android-platform-adapter-and-package-visibility.md diff --git a/.github/workflows/upgradeall-rewrite-validation.yml b/.github/workflows/upgradeall-rewrite-validation.yml index 2e76e5ff..ff9ea4bd 100644 --- a/.github/workflows/upgradeall-rewrite-validation.yml +++ b/.github/workflows/upgradeall-rewrite-validation.yml @@ -41,6 +41,9 @@ jobs: with: components: clippy, rustfmt + - name: Install Android Rust target + run: rustup target add aarch64-linux-android + - name: Install just run: | if ! command -v just >/dev/null 2>&1; then diff --git a/app_flutter/README.md b/app_flutter/README.md index cb6fef91..0bc55fa1 100644 --- a/app_flutter/README.md +++ b/app_flutter/README.md @@ -10,8 +10,9 @@ This is the new Flutter shell and product APK entry for the UpgradeAll rewrite. - Placeholder routes for apps, repositories, downloads, logs, settings, and legacy migration - `FakeGetterAdapter` for deterministic widget tests - `CliGetterAdapter` as a development/integration bridge against the real `getter-cli` JSON envelope +- Product manifest permissions include `QUERY_ALL_PACKAGES` per ADR-0009 so the future Rust-active Android platform adapter can provide complete installed package inventory facts to getter. -`CliGetterAdapter` is not the final Android production bridge. It exists to keep the getter-owned DTO and error contract executable while the native bridge is designed in ADR-0007. +`CliGetterAdapter` is not the final Android production bridge. It exists to keep the getter-owned DTO and error contract executable while the native bridge is designed in ADR-0007. Installed-autogen product flows must use getter/native bridge operations backed by the Rust-active platform adapter from ADR-0009, not a Flutter-led MethodChannel scanner. ## Verification diff --git a/app_flutter/android/app/src/main/AndroidManifest.xml b/app_flutter/android/app/src/main/AndroidManifest.xml index 7beddf2f..561ef820 100644 --- a/app_flutter/android/app/src/main/AndroidManifest.xml +++ b/app_flutter/android/app/src/main/AndroidManifest.xml @@ -3,6 +3,10 @@ checks once live provider runtime is approved. The current shell does not implement those semantics yet, but the product APK owns the permission. --> + + = OnceCell::new(); + +struct AndroidRuntime { + java_vm: JavaVM, + #[allow(dead_code)] + application_context: GlobalRef, + class_loader: GlobalRef, +} + +/// Initialize Android platform access from a JNI entrypoint. +/// +/// `context` should be an Android `Context`. The function stores +/// `context.getApplicationContext()` and its class loader as global refs. It is +/// idempotent for the lifetime of the process. +pub fn init_with_env( + env: &mut JNIEnv<'_>, + context: JObject<'_>, +) -> Result<(), PlatformAdapterError> { + RUNTIME + .get_or_try_init(|| runtime_from_env(env, context)) + .map(|_| ()) +} + +fn runtime_from_env( + env: &mut JNIEnv<'_>, + context: JObject<'_>, +) -> Result { + let java_vm = env + .get_java_vm() + .map_err(|error| PlatformAdapterError::Jni(error.to_string()))?; + + let application_context = env + .call_method( + &context, + "getApplicationContext", + "()Landroid/content/Context;", + &[], + ) + .and_then(|value| value.l()) + .map_err(|error| PlatformAdapterError::Jni(error.to_string()))?; + let application_context = if application_context.is_null() { + env.new_global_ref(&context) + } else { + env.new_global_ref(&application_context) + } + .map_err(|error| PlatformAdapterError::Jni(error.to_string()))?; + + let class_loader = env + .call_method( + application_context.as_obj(), + "getClassLoader", + "()Ljava/lang/ClassLoader;", + &[], + ) + .and_then(|value| value.l()) + .map_err(|error| PlatformAdapterError::Jni(error.to_string()))?; + let class_loader = env + .new_global_ref(&class_loader) + .map_err(|error| PlatformAdapterError::Jni(error.to_string()))?; + + Ok(AndroidRuntime { + java_vm, + application_context, + class_loader, + }) +} + +/// Android implementation placeholder for platform capabilities. +#[derive(Debug, Default)] +pub struct AndroidPlatformAdapter; + +impl PlatformAdapter for AndroidPlatformAdapter { + fn scan_installed_inventory( + &self, + _options: InstalledInventoryScanOptions, + ) -> Result { + with_attached_env(|_env, _runtime| Ok(()))?; + Err(PlatformAdapterError::Unsupported { + capability: "installed_inventory.android_jni_provider", + }) + } +} + +fn with_attached_env( + f: impl FnOnce(&mut JNIEnv<'_>, &AndroidRuntime) -> Result, +) -> Result { + let runtime = RUNTIME.get().ok_or(PlatformAdapterError::NotInitialized)?; + let mut env = runtime + .java_vm + .attach_current_thread() + .map_err(|error| PlatformAdapterError::Jni(error.to_string()))?; + + f(&mut env, runtime) +} + +/// Load an application class with the app classloader instead of `FindClass`. +#[allow(dead_code)] +fn load_class<'local>( + env: &mut JNIEnv<'local>, + runtime: &AndroidRuntime, + binary_name: &str, +) -> Result, PlatformAdapterError> { + let name = env + .new_string(binary_name) + .map_err(|error| PlatformAdapterError::Jni(error.to_string()))?; + let class = env + .call_method( + runtime.class_loader.as_obj(), + "loadClass", + "(Ljava/lang/String;)Ljava/lang/Class;", + &[jni::objects::JValue::Object(&JObject::from(name))], + ) + .and_then(|value| value.l()) + .map_err(|error| PlatformAdapterError::Jni(error.to_string()))?; + Ok(class) +} + +/// Convert a Java string into a Rust string. +#[allow(dead_code)] +fn java_string(env: &mut JNIEnv<'_>, value: JObject<'_>) -> Result { + if value.is_null() { + return Ok(String::new()); + } + let value = JString::from(value); + env.get_string(&value) + .map(|value| value.into()) + .map_err(|error| PlatformAdapterError::Jni(error.to_string())) +} diff --git a/core-getter/src/main/rust/platform_adapter/src/lib.rs b/core-getter/src/main/rust/platform_adapter/src/lib.rs new file mode 100644 index 00000000..9c146331 --- /dev/null +++ b/core-getter/src/main/rust/platform_adapter/src/lib.rs @@ -0,0 +1,227 @@ +//! Rust-active platform capability adapter for the UpgradeAll Android product. +//! +//! This crate intentionally lives outside the reusable getter submodule. It +//! defines platform facts and Android runtime plumbing for the product/native +//! bridge layer. getter still owns domain decisions such as package ids, +//! repository coverage, Lua generation, and storage writes. + +use serde::{Deserialize, Serialize}; + +#[cfg(target_os = "android")] +pub mod android; + +pub const INSTALLED_INVENTORY_FORMAT: &str = "upgradeall-installed-inventory"; +pub const INSTALLED_INVENTORY_VERSION: u32 = 1; + +/// A small Rust-owned interface for platform capabilities. +/// +/// Implementations return platform facts only. Callers must not infer getter +/// product decisions from this interface; the native bridge/getter operation is +/// responsible for converting facts into getter-owned workflows. +pub trait PlatformAdapter: Send + Sync { + fn scan_installed_inventory( + &self, + options: InstalledInventoryScanOptions, + ) -> Result; +} + +/// Host/test adapter used when no platform implementation is available. +#[derive(Debug, Default)] +pub struct NoopPlatformAdapter; + +impl PlatformAdapter for NoopPlatformAdapter { + fn scan_installed_inventory( + &self, + _options: InstalledInventoryScanOptions, + ) -> Result { + Err(PlatformAdapterError::Unsupported { + capability: "installed_inventory", + }) + } +} + +#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)] +pub struct InstalledInventoryScanOptions { + #[serde(default)] + pub include_system_apps: bool, + #[serde(default)] + pub include_self: bool, +} + +impl Default for InstalledInventoryScanOptions { + fn default() -> Self { + Self { + include_system_apps: false, + include_self: false, + } + } +} + +#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)] +pub struct InstalledInventoryScanResult { + pub inventory: InstalledInventory, + pub stats: InstalledInventoryScanStats, + #[serde(default)] + pub diagnostics: Vec, +} + +#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)] +pub struct InstalledInventory { + pub format: String, + pub version: u32, + #[serde(default)] + pub items: Vec, +} + +impl InstalledInventory { + pub fn new(items: Vec) -> Self { + Self { + format: INSTALLED_INVENTORY_FORMAT.to_owned(), + version: INSTALLED_INVENTORY_VERSION, + items, + } + } +} + +/// Getter-compatible installed inventory facts produced by platform code. +/// +/// Android platform adapters emit raw package names and metadata only. They do +/// not normalize to `android/` package ids. Magisk facts are excluded +/// from this PackageManager adapter surface and need a separate capability +/// decision. +#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)] +#[serde(tag = "kind", rename_all = "snake_case")] +pub enum InstalledInventoryItem { + AndroidPackage { + package_name: String, + #[serde(default)] + label: Option, + #[serde(default)] + version_name: Option, + #[serde(default)] + version_code: Option, + }, +} + +#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)] +pub struct InstalledInventoryScanStats { + pub total_seen: u32, + pub returned: u32, + pub filtered_system: u32, + pub filtered_self: u32, +} + +#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)] +pub struct PlatformDiagnostic { + pub code: String, + pub message: String, + #[serde(default)] + pub detail: Option, +} + +#[derive(Debug, thiserror::Error)] +pub enum PlatformAdapterError { + #[error("platform capability '{capability}' is unsupported")] + Unsupported { capability: &'static str }, + #[error("platform adapter is not initialized")] + NotInitialized, + #[error("platform adapter JNI error: {0}")] + Jni(String), + #[error("platform adapter response is malformed: {0}")] + MalformedResponse(String), +} + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn scan_options_default_to_privacy_preserving_user_inventory() { + let options = InstalledInventoryScanOptions::default(); + + assert!(!options.include_system_apps); + assert!(!options.include_self); + + let json = serde_json::to_value(options).expect("serialize options"); + assert_eq!(json["include_system_apps"], false); + assert_eq!(json["include_self"], false); + } + + #[test] + fn inventory_serializes_to_getter_compatible_android_package_facts() { + let inventory = InstalledInventory::new(vec![InstalledInventoryItem::AndroidPackage { + package_name: "org.fdroid.fdroid".to_owned(), + label: Some("F-Droid".to_owned()), + version_name: Some("1.20.0".to_owned()), + version_code: Some(1_020_000), + }]); + + let json = serde_json::to_value(&inventory).expect("serialize inventory"); + + assert_eq!(json["format"], INSTALLED_INVENTORY_FORMAT); + assert_eq!(json["version"], INSTALLED_INVENTORY_VERSION); + assert_eq!(json["items"][0]["kind"], "android_package"); + assert_eq!(json["items"][0]["package_name"], "org.fdroid.fdroid"); + assert!(json["items"][0].get("package_id").is_none()); + } + + #[test] + fn scan_result_deserializes_with_default_diagnostics() { + let json = r#" + { + "inventory": { + "format": "upgradeall-installed-inventory", + "version": 1, + "items": [] + }, + "stats": { + "total_seen": 3, + "returned": 1, + "filtered_system": 1, + "filtered_self": 1 + } + } + "#; + + let result: InstalledInventoryScanResult = + serde_json::from_str(json).expect("deserialize scan result"); + + assert!(result.diagnostics.is_empty()); + assert_eq!(result.stats.total_seen, 3); + assert_eq!(result.inventory.items, Vec::new()); + } + + #[test] + fn noop_adapter_reports_unsupported_installed_inventory() { + let adapter = NoopPlatformAdapter; + + let error = adapter + .scan_installed_inventory(InstalledInventoryScanOptions::default()) + .expect_err("noop adapter should not scan"); + + assert!(matches!( + error, + PlatformAdapterError::Unsupported { + capability: "installed_inventory" + } + )); + } + + #[test] + fn platform_inventory_json_is_accepted_by_getter_core_autogen_schema() { + let inventory = InstalledInventory::new(vec![InstalledInventoryItem::AndroidPackage { + package_name: "org.fdroid.fdroid".to_owned(), + label: Some("F-Droid".to_owned()), + version_name: Some("1.20.0".to_owned()), + version_code: Some(1_020_000), + }]); + let json = serde_json::to_string(&inventory).expect("serialize platform inventory"); + + let getter_inventory: getter_core::autogen::InstalledInventory = + serde_json::from_str(&json).expect("getter-core should accept platform inventory"); + + getter_core::autogen::validate_installed_inventory(&getter_inventory) + .expect("inventory format/version should match getter core"); + assert_eq!(getter_inventory.items.len(), 1); + } +} diff --git a/docs/README.md b/docs/README.md index eff8bd2f..fdad3af6 100644 --- a/docs/README.md +++ b/docs/README.md @@ -17,10 +17,11 @@ Start here: 7. `architecture/adr/0006-package-centric-cli-command-contract.md` — getter CLI automation contract. 8. `architecture/adr/0007-flutter-getter-bridge-contract.md` — Flutter/getter DTO and bridge contract. 9. `architecture/adr/0008-flutter-product-apk-entry.md` — Flutter app as the sole product APK entry. -10. `lua-api/` — practical Lua package authoring docs, including offline `repo validate` diagnostics. -11. `migration/legacy-room-mapping.md` — old data mapping rules. -12. `app/flutter-ui-feature-parity-and-testing.md` — Flutter feature parity and BDD/TDD test boundary. -13. `implementation/coding-agent-handoff.md` — coding-agent / pi-agent handoff instructions. +10. `architecture/adr/0009-android-platform-adapter-and-package-visibility.md` — Rust-active Android platform adapter and package visibility policy. +11. `lua-api/` — practical Lua package authoring docs, including offline `repo validate` diagnostics. +12. `migration/legacy-room-mapping.md` — old data mapping rules. +13. `app/flutter-ui-feature-parity-and-testing.md` — Flutter feature parity and BDD/TDD test boundary. +14. `implementation/coding-agent-handoff.md` — coding-agent / pi-agent handoff instructions. Canonical architecture ADRs live in `docs/architecture/adr/*`. The `docs/adr/*` directory is kept for historical/refactor-phase ADRs and transition notes. diff --git a/docs/app/flutter-ui-feature-parity-and-testing.md b/docs/app/flutter-ui-feature-parity-and-testing.md index 8fecf5d0..e3e91c32 100644 --- a/docs/app/flutter-ui-feature-parity-and-testing.md +++ b/docs/app/flutter-ui-feature-parity-and-testing.md @@ -76,6 +76,7 @@ The first Flutter implementation slice is intentionally a shell, not product log - `CliGetterAdapter` exercises a real getter data directory through the `getter-cli` JSON envelope for development/integration tests. - ADR-0007 documents the bridge contract and explicitly treats the CLI adapter as a test/development bridge, not the final Android production path. - Product decisions such as repository resolution, updates, migrations, storage, and downloads still belong in Rust getter. +- Installed-autogen product flows must call getter/native bridge operations that use the Rust-active Android platform adapter from ADR-0009; Flutter should not lead PackageManager inventory scanning through a Dart MethodChannel API. - CI/release APK artifacts must be built from `app_flutter`, not from the legacy `:app` module. - The downloads route may render getter task/event DTOs read-only, but it must not implement a Dart download task state machine, retry policy, or installer semantics. diff --git a/docs/architecture/README.md b/docs/architecture/README.md index f841c893..8b4ff1b7 100644 --- a/docs/architecture/README.md +++ b/docs/architecture/README.md @@ -16,6 +16,7 @@ Planned / active ADRs: - `adr/0006-package-centric-cli-command-contract.md` - `adr/0007-flutter-getter-bridge-contract.md` - `adr/0008-flutter-product-apk-entry.md` +- `adr/0009-android-platform-adapter-and-package-visibility.md` Documentation policy: diff --git a/docs/architecture/adr/0002-getter-flutter-platform-boundary.md b/docs/architecture/adr/0002-getter-flutter-platform-boundary.md index a07b70d6..6f17496b 100644 --- a/docs/architecture/adr/0002-getter-flutter-platform-boundary.md +++ b/docs/architecture/adr/0002-getter-flutter-platform-boundary.md @@ -12,7 +12,7 @@ Getter remains a separate reusable git submodule at `core-getter/src/main/rust/g The Flutter Android app embeds getter as a Rust library / FFI-style core. The app does not use a standalone getter daemon as the primary path. -Platform-specific APIs are exposed to getter through RPC/callback-style boundaries so that thread management and platform complexity remain isolated. +Platform-specific APIs are exposed to getter through documented platform adapter seams so that thread management and platform complexity remain isolated. For Android installed inventory, ADR-0009 supersedes the earlier MethodChannel-led scan idea: Rust/native bridge code is the active caller, initializes JVM/context/classloader handles, and calls Android implementation classes for raw PackageManager facts. ## getter owns @@ -32,13 +32,19 @@ Platform-specific APIs are exposed to getter through RPC/callback-style boundari ## Flutter APP owns - UI rendering and navigation. -- Android permission prompts. -- Android PackageManager inventory scanning. +- Android permission prompts and user-facing permission explanations. +- User confirmation flows. +- Rendering getter-owned DTOs, platform diagnostics, and recovery states. + +## Platform adapters own + +- Raw Android PackageManager installed-package facts exposed through the Rust-active platform adapter accepted in ADR-0009. - Installed version lookup through platform APIs. -- APK install / Shizuku/root/system installer adapters. -- Notifications / foreground service integration. +- APK install / Shizuku/root/system installer adapters after installer semantics are accepted. +- Notifications / foreground service integration after background-runtime semantics are accepted. - SAF/file picker and URI permissions. -- User confirmation flows. + +Platform adapters expose facts/capabilities to Rust getter/native bridge code. They must not perform package-id normalization, repository resolution, Lua validation, autogen candidate selection, migration mapping, download retry policy, or storage writes. ## Boundary rule diff --git a/docs/architecture/adr/0007-flutter-getter-bridge-contract.md b/docs/architecture/adr/0007-flutter-getter-bridge-contract.md index c9ec5bf3..b788df2a 100644 --- a/docs/architecture/adr/0007-flutter-getter-bridge-contract.md +++ b/docs/architecture/adr/0007-flutter-getter-bridge-contract.md @@ -45,6 +45,15 @@ importLegacyRoomDatabase(databasePath) The Android platform adapter may prepare a copied/checkpointed legacy Room SQLite file and return its path to Flutter, but getter still owns the actual `legacy import-room-db` import semantics. Flutter starts the flow and renders getter reports; it must not inspect or map Room tables directly. +The installed-autogen product bridge must follow ADR-0009's Rust-active platform adapter direction rather than a Flutter-led inventory scan. The future bridge shape is a getter-owned operation such as: + +```text +previewInstalledAutogen(scanOptions) +applyInstalledAutogen(preview, acceptedPackages) +``` + +Internally, Rust/native bridge code scans Android inventory through the platform adapter, then asks getter to plan/apply `local_autogen`. Flutter renders getter-owned preview/apply DTOs and scan diagnostics; it must not expose a product Dart `InstalledInventoryPlatform` scanner or convert Android package names into package ids. + `loadSnapshot()` composes the smaller getter-owned operations into the UI shell's first snapshot DTO. It must not perform repository resolution, version comparison, migration mapping, or update selection in Dart. `readMigrationReports()` must go through a getter operation such as `legacy report-list`; Flutter must not inspect getter's data-directory layout directly. `listDownloadTasks()` and `listTaskEvents()` render getter-owned task/event DTOs; Flutter must not synthesize task states, retry policy, installer behavior, or update decisions. ## Flutter DTOs @@ -98,6 +107,8 @@ and structured error envelopes: Flutter adapter code may parse and display these fields, but it must not infer missing domain state from them. If the UI needs a richer field, add it to getter output first and cover it with getter tests. +For installed-autogen flows, CLI/dev tests may continue to pass fixture inventory JSON to `getter autogen installed preview/apply`. The Android product bridge should not expose that fixture boundary as a Flutter-owned scanning API; it should wrap scan + getter autogen planning behind a getter/native bridge operation. + ## Error model The bridge maps getter errors into `GetterError`: diff --git a/docs/architecture/adr/0008-flutter-product-apk-entry.md b/docs/architecture/adr/0008-flutter-product-apk-entry.md index 63bd5e47..be99739e 100644 --- a/docs/architecture/adr/0008-flutter-product-apk-entry.md +++ b/docs/architecture/adr/0008-flutter-product-apk-entry.md @@ -13,7 +13,7 @@ The legacy Android `:app` module and its native Activity/Fragment/XML UI are kep All user-visible flows migrate into Flutter. Android-native code remains allowed only for non-UI platform adapter responsibilities such as: - legacy Room database copy/checkpoint handoff; -- installed package inventory collection; +- installed package inventory facts exposed through the Rust-active platform adapter from ADR-0009; - Android permission prompts and capability adapters; - SAF/file picker and URI permission plumbing; - installer handoff adapters; diff --git a/docs/architecture/adr/0009-android-platform-adapter-and-package-visibility.md b/docs/architecture/adr/0009-android-platform-adapter-and-package-visibility.md new file mode 100644 index 00000000..022848e1 --- /dev/null +++ b/docs/architecture/adr/0009-android-platform-adapter-and-package-visibility.md @@ -0,0 +1,145 @@ +# ADR-0009: Android platform adapter and package visibility + +> Status: Accepted for first implementation slice +> Date: 2026-06-23 +> Project: UpgradeAll rewrite — Flutter APP + Rust getter core + Lua package repository model + +## Decision + +UpgradeAll will use a Rust-active platform adapter for Android platform capabilities. + +Rust/getter-side native code defines the platform interface and actively calls the Android implementation. Android/Kotlin code supplies raw platform facts only. Flutter remains the product UI and renders getter-owned DTOs; it does not lead installed-app inventory scanning or turn Android package names into UpgradeAll package ids. + +The first accepted platform capability is installed Android package inventory for `local_autogen` preview/apply workflows. The product Flutter APK declares: + +```xml + +``` + +This is an explicit product/distribution decision: UpgradeAll is an app updater and installed-app tracker, so broad installed-package visibility is core functionality rather than incidental implementation convenience. + +## Rust-active adapter pattern + +The Android implementation follows the same architectural pattern as `rustls-platform-verifier`: + +1. A JNI entrypoint initializes platform access with the current JVM, application `Context`, and app `ClassLoader`. +2. Rust stores process-lifetime global references. +3. When Rust needs a platform capability, it attaches the current thread to the JVM, loads Android implementation classes through the app classloader, and calls static Kotlin/Java methods. +4. Kotlin/Android code returns data facts in a stable transport shape. +5. Rust validates/deserializes those facts before passing them to getter-owned workflows. + +Rust must not use Android `FindClass` from arbitrary background threads for app classes. App classes are loaded through the stored app classloader. + +## Installed inventory contract + +The platform adapter returns a wrapper result: + +```json +{ + "inventory": { + "format": "upgradeall-installed-inventory", + "version": 1, + "items": [ + { + "kind": "android_package", + "package_name": "org.fdroid.fdroid", + "label": "F-Droid", + "version_name": "1.20.0", + "version_code": 1020000 + } + ] + }, + "stats": { + "total_seen": 123, + "returned": 42, + "filtered_system": 80, + "filtered_self": 1 + }, + "diagnostics": [] +} +``` + +The installed inventory is getter-compatible, but it remains raw platform fact data: + +- Android supplies `package_name`, label, version name, and version code. +- Android/Flutter must not generate `android/` package ids. +- Android/Flutter must not decide repository coverage, autogen candidates, generated Lua file paths, or tracking-state writes. +- Magisk modules are not part of this PackageManager capability. They require a separate root/Shizuku/Magisk capability decision. + +## Scan options + +The first scan options are: + +```json +{ + "include_system_apps": false, + "include_self": false +} +``` + +Defaults exclude system apps and the UpgradeAll application itself. Disabled-app filtering is not part of the first Rust interface; disabled packages are treated as installed PackageManager facts until a later product decision defines user-facing semantics. + +## Getter and Flutter responsibilities + +The product operation shape is: + +```text +Flutter UI + -> getter/native bridge: preview installed autogen + -> Rust platform adapter: scan installed inventory facts + -> getter core: plan local_autogen candidates/skips + <- getter-owned preview DTO +``` + +Flutter may ask getter for preview/apply operations and render scan stats/diagnostics returned by getter. Flutter must not implement a separate Dart `InstalledInventoryPlatform` scanner or MethodChannel-led inventory flow for the product path. + +CLI/dev workflows remain fixture-based: + +```text +getter autogen installed preview --inventory installed.json +getter autogen installed apply --preview preview.json --accept-all +``` + +The CLI has no Android PackageManager, so fixtures remain the headless oracle for getter domain behavior. + +## Permission policy + +`QUERY_ALL_PACKAGES` is declared only in the Flutter product APK manifest path (`app_flutter`). The legacy native `:app` module remains reference-only and is not the rewrite product APK path. + +The permission may have distribution-policy implications on app stores. The project accepts that trade-off for the rewrite product because full installed-app visibility is necessary for UpgradeAll's app-updater inventory and autogen workflows. + +If lint/build tooling flags `QUERY_ALL_PACKAGES`, the manifest may suppress that lint with an inline comment and `tools:ignore="QueryAllPackagesPermission"`; this suppression must remain documented as policy, not treated as a generic lint cleanup. + +## First implementation slice + +The first slice is intentionally narrow: + +- document this ADR and update existing boundary docs; +- add `QUERY_ALL_PACKAGES` to the Flutter product manifest; +- add a superproject Rust crate for platform adapter DTOs, errors, a `NoopPlatformAdapter`, and Android runtime/JNI initialization skeleton; +- add validation for that crate. + +The first slice does not: + +- make the reusable getter submodule depend on superproject-only crates; +- add Kotlin PackageManager scanner behavior; +- wire product native bridge operations; +- add Flutter installed-autogen UX; +- add Magisk scanning; +- add live downloads, background worker policy, installer URI/SAF semantics, or notification behavior. + +## Consequences + +Positive: + +- Rust remains the active caller and owner of platform interface shape. +- The platform seam is testable with host DTO tests before device integration exists. +- Flutter cannot accidentally become the owner of inventory/autogen decisions. +- The model can later support other Android capabilities using the same runtime initialization pattern. + +Costs and risks: + +- The first slice is not yet product-complete; production bridge packaging into the Flutter APK still needs a later accepted implementation. +- The Rust platform DTOs must stay compatible with getter's installed inventory contract. +- JNI/runtime bugs require Android build/device validation beyond host unit tests. +- Broad package visibility is now an explicit product policy with distribution implications. diff --git a/docs/architecture/upgradeall-getter-rewrite-wiki.md b/docs/architecture/upgradeall-getter-rewrite-wiki.md index 609e05bb..078d73f5 100644 --- a/docs/architecture/upgradeall-getter-rewrite-wiki.md +++ b/docs/architecture/upgradeall-getter-rewrite-wiki.md @@ -177,7 +177,7 @@ Hub 3. 所有 product/domain logic 都放在 getter。 4. Android App 只是 Flutter UI + platform adapter。 5. App 内 getter 形态采用嵌入式 Rust library / FFI 风格，不以 daemon 作为主路径。 -6. 平台专用 API 通过 RPC/callback 暴露给 getter，用于隐藏平台线程/API 复杂度。 +6. 平台专用 API 通过 Rust-active platform adapter 暴露给 getter/native bridge；Rust 定义接口并主动调用 Android 实现，Android/Kotlin 只提供平台事实。 7. 后端存储使用 SQLite。 8. 用户通过非标准方式改坏 backend storage 时，getter fail fast 报错，不提供复杂恢复引导。 9. 用户二次开发采用 patch stack/source fork 模式，不设计复杂 runtime customization/plugin 系统。 @@ -191,16 +191,18 @@ Hub Flutter APP - UI rendering - navigation - - Android permissions - - Android PackageManager inventory - - installer adapter - - notification adapter - - SAF/file picker adapter - - platform RPC/callback server + - Android permission prompts + - user confirmation flows + - render getter/platform DTOs | - | FFI / RPC-like boundary + | FFI / native bridge boundary v -Rust getter core +Rust getter core + native bridge + - Rust-active platform adapter interface + - Android PackageManager inventory calls through platform adapter + - installer adapter handoff + - notification adapter handoff + - SAF/file picker/URI permission handoff - app/package model - repository/overlay resolution - Lua package evaluation @@ -238,7 +240,7 @@ Rust getter core 保留在 Flutter/Android adapter： -- Android PackageManager installed app scanning。 +- Android PackageManager installed app scanning exposed as raw facts through the Rust-active platform adapter (ADR-0009)。 - Android installed version lookup。 - APK install / package installer / Shizuku/root installer。 - Android permission request。 @@ -1036,21 +1038,24 @@ Room DB 信息： 用户点击“从已安装应用生成”： -1. Android adapter 扫描 installed inventory。 -2. getter 找出可生成的候选列表。 -3. UI 展示列表。 -4. 用户 yes/no 确认。 -5. getter 写入 `local_autogen` repo。 -6. 生成后不会自动消失。 +1. Flutter 调用 getter/native bridge 的 installed-autogen preview 操作。 +2. Rust platform adapter 主动调用 Android PackageManager adapter，取得 installed inventory 原始事实。 +3. getter 找出可生成的候选列表。 +4. UI 展示 getter-owned preview DTO。 +5. 用户 yes/no 确认。 +6. getter 写入 `local_autogen` repo。 +7. 生成后不会自动消失。 ### 14.2 清理流程 用户点击“清除不存在的应用”： -1. getter 计算将删除列表。 -2. UI 展示列表。 -3. 用户 yes/no 确认。 -4. getter 删除 `local_autogen` 中不再安装的记录/文件。 +1. Flutter 调用 getter/native bridge 的 installed-autogen cleanup preview 操作。 +2. Rust platform adapter 主动调用 Android PackageManager adapter，取得当前 installed inventory 原始事实。 +3. getter 计算将删除列表。 +4. UI 展示 getter-owned preview DTO。 +5. 用户 yes/no 确认。 +6. getter 删除 `local_autogen` 中不再安装的记录/文件。 普通清理按钮只作用于 `local_autogen`，不删除 `local`。 diff --git a/docs/lua-api/templates.md b/docs/lua-api/templates.md index 0ca5b925..468638ad 100644 --- a/docs/lua-api/templates.md +++ b/docs/lua-api/templates.md @@ -46,20 +46,20 @@ return android.local_app { Generation flow: -1. Android/platform adapter writes an installed-inventory DTO. -2. User clicks generate. -3. getter computes candidate list through `autogen installed preview --inventory `. -4. Flutter shows preview list. +1. User clicks generate in Flutter. +2. Flutter calls a getter/native bridge operation for installed-autogen preview. +3. Rust calls the Android platform adapter for installed-inventory facts, then getter computes the candidate list. CLI/dev tests may still exercise this with `autogen installed preview --inventory ` fixtures. +4. Flutter shows the getter-owned preview list. 5. User confirms yes/no. -6. getter applies the accepted preview through `autogen installed apply --preview --accept-all` or repeated `--accept `. +6. getter applies the accepted preview through the native bridge operation; CLI/dev tests may still use `autogen installed apply --preview --accept-all` or repeated `--accept `. 7. getter writes files under `/repositories/local_autogen`, registers the repo, records `autogen-manifest.json`, and tracks accepted packages in `main.db`. Cleanup flow: -1. Android/platform adapter writes the current installed-inventory DTO. -2. User clicks clear missing generated apps. -3. getter computes deletion list through `autogen cleanup preview --inventory `. -4. Flutter shows preview list. +1. User clicks clear missing generated apps. +2. Flutter calls a getter/native bridge operation; Rust obtains the current installed-inventory facts through the Android platform adapter. +3. getter computes the deletion list. CLI/dev tests may still exercise this with `autogen cleanup preview --inventory ` fixtures. +4. Flutter shows the getter-owned preview list. 5. User confirms yes/no. 6. getter deletes only accepted manifest-managed `local_autogen` files/state. diff --git a/justfile b/justfile index da3dddd1..cea0161d 100644 --- a/justfile +++ b/justfile @@ -2,6 +2,7 @@ set shell := ["bash", "-eu", "-o", "pipefail", "-c"] GETTER_MANIFEST := "core-getter/src/main/rust/getter/Cargo.toml" API_PROXY_MANIFEST := "core-getter/src/main/rust/api_proxy/Cargo.toml" +PLATFORM_ADAPTER_MANIFEST := "core-getter/src/main/rust/platform_adapter/Cargo.toml" verify: just test-getter-unit @@ -36,8 +37,12 @@ verify-workspace-skeleton: test "$(git ls-files -s core-getter/src/main/rust/getter | awk '{print $1}')" = "160000" cargo metadata --manifest-path {{ GETTER_MANIFEST }} --no-deps --format-version 1 >/tmp/upgradeall-getter-metadata.json cargo metadata --manifest-path {{ API_PROXY_MANIFEST }} --no-deps --format-version 1 >/tmp/upgradeall-api-proxy-metadata.json + cargo metadata --manifest-path {{ PLATFORM_ADAPTER_MANIFEST }} --no-deps --format-version 1 >/tmp/upgradeall-platform-adapter-metadata.json cargo fmt --manifest-path {{ GETTER_MANIFEST }} --all --check + cargo fmt --manifest-path {{ PLATFORM_ADAPTER_MANIFEST }} --all --check cargo check --manifest-path {{ GETTER_MANIFEST }} --workspace --all-targets cargo check --manifest-path {{ API_PROXY_MANIFEST }} + cargo test --manifest-path {{ PLATFORM_ADAPTER_MANIFEST }} + cargo check --manifest-path {{ PLATFORM_ADAPTER_MANIFEST }} --target aarch64-linux-android cd app_flutter && flutter analyze ./gradlew --no-daemon projects diff --git a/todo.md b/todo.md index c082127c..b3a9edea 100644 --- a/todo.md +++ b/todo.md @@ -441,9 +441,10 @@ Completed tasks: Remaining tasks: -1. Android adapter supplies real installed inventory DTO. -2. Flutter confirmation UX consumes getter preview/apply DTOs. -3. Cache invalidation hooks beyond file-hash-based repository reload need to be expanded when evaluated/provider caches become active. +1. Finish the Rust-active Android platform adapter path from ADR-0009: Kotlin PackageManager provider/scanner, native bridge packaging, and Rust calls that return installed inventory facts. +2. Add getter/native bridge operations that combine platform scan + getter `local_autogen` preview/apply; Flutter must consume these getter-owned DTOs rather than leading a MethodChannel inventory scan. +3. Flutter confirmation UX consumes getter preview/apply DTOs. +4. Cache invalidation hooks beyond file-hash-based repository reload need to be expanded when evaluated/provider caches become active. Acceptance progress: From 3a5a602c03e43984eef904ad3bb8f12458df4824 Mon Sep 17 00:00:00 2001 From: Xiangzhe Date: Tue, 23 Jun 2026 20:40:35 +0800 Subject: [PATCH 23/85] feat(app): add Android installed inventory provider --- .../upgradeall-rewrite-validation.yml | 13 +- core-getter/consumer-rules.pro | 2 + core-getter/rpc/build.gradle.kts | 8 + .../platform/InstalledInventoryProvider.kt | 261 ++++++++++++++++++ .../src/main/rust/api_proxy/Cargo.toml | 1 + .../src/main/rust/api_proxy/src/lib.rs | 34 ++- .../main/rust/platform_adapter/src/android.rs | 52 +++- .../InstalledInventoryCollectorTest.kt | 110 ++++++++ core-websdk/data/build.gradle.kts | 8 + ...platform-adapter-and-package-visibility.md | 17 +- justfile | 5 + todo.md | 5 +- 12 files changed, 491 insertions(+), 25 deletions(-) create mode 100644 core-getter/src/main/java/net/xzos/upgradeall/getter/platform/InstalledInventoryProvider.kt create mode 100644 core-getter/src/test/java/net/xzos/upgradeall/getter/platform/InstalledInventoryCollectorTest.kt diff --git a/.github/workflows/upgradeall-rewrite-validation.yml b/.github/workflows/upgradeall-rewrite-validation.yml index ff9ea4bd..f3908391 100644 --- a/.github/workflows/upgradeall-rewrite-validation.yml +++ b/.github/workflows/upgradeall-rewrite-validation.yml @@ -11,6 +11,8 @@ jobs: rewrite-validation: name: Rewrite validation runs-on: ubuntu-latest + env: + NDK_VERSION: 29.0.14206865 steps: - name: Checkout repository uses: actions/checkout@v6 @@ -28,7 +30,12 @@ jobs: uses: android-actions/setup-android@v3 - name: Install Android SDK packages - run: sdkmanager --install "platforms;android-36" "build-tools;36.0.0" "platform-tools" + run: sdkmanager --install "platforms;android-36" "build-tools;36.0.0" "platform-tools" "ndk;${{ env.NDK_VERSION }}" + + - name: Add Android NDK to environment + run: | + echo "ANDROID_NDK_HOME=$ANDROID_HOME/ndk/$NDK_VERSION" >> "$GITHUB_ENV" + echo "ANDROID_NDK_ROOT=$ANDROID_HOME/ndk/$NDK_VERSION" >> "$GITHUB_ENV" - name: Install Flutter uses: subosito/flutter-action@v2 @@ -41,8 +48,8 @@ jobs: with: components: clippy, rustfmt - - name: Install Android Rust target - run: rustup target add aarch64-linux-android + - name: Install Android Rust targets + run: rustup target add aarch64-linux-android armv7-linux-androideabi x86_64-linux-android - name: Install just run: | diff --git a/core-getter/consumer-rules.pro b/core-getter/consumer-rules.pro index e69de29b..ed547b69 100644 --- a/core-getter/consumer-rules.pro +++ b/core-getter/consumer-rules.pro @@ -0,0 +1,2 @@ +# Rust JNI loads this provider reflectively through the app classloader. +-keep class net.xzos.upgradeall.getter.platform.** { *; } diff --git a/core-getter/rpc/build.gradle.kts b/core-getter/rpc/build.gradle.kts index 601180c1..4517a654 100644 --- a/core-getter/rpc/build.gradle.kts +++ b/core-getter/rpc/build.gradle.kts @@ -1,3 +1,5 @@ +import org.jetbrains.kotlin.gradle.dsl.JvmTarget + plugins { id("java-library") alias(libs.plugins.kotlin.jvm) @@ -8,6 +10,12 @@ java { targetCompatibility = JavaVersion.VERSION_21 } +kotlin { + compilerOptions { + jvmTarget.set(JvmTarget.JVM_21) + } +} + dependencies { api(project(":core-websdk:data")) diff --git a/core-getter/src/main/java/net/xzos/upgradeall/getter/platform/InstalledInventoryProvider.kt b/core-getter/src/main/java/net/xzos/upgradeall/getter/platform/InstalledInventoryProvider.kt new file mode 100644 index 00000000..544f2d75 --- /dev/null +++ b/core-getter/src/main/java/net/xzos/upgradeall/getter/platform/InstalledInventoryProvider.kt @@ -0,0 +1,261 @@ +package net.xzos.upgradeall.getter.platform + +import android.Manifest +import android.content.Context +import android.content.pm.ApplicationInfo +import android.content.pm.PackageInfo +import android.content.pm.PackageManager +import android.os.Build +import org.json.JSONArray +import org.json.JSONObject + +private const val INSTALLED_INVENTORY_FORMAT = "upgradeall-installed-inventory" +private const val INSTALLED_INVENTORY_VERSION = 1 + +/** + * JNI entrypoint used by Rust's platform adapter. + * + * Kotlin returns raw Android PackageManager facts only. It does not construct + * UpgradeAll package ids, decide repository coverage, generate Lua, or write + * getter storage. + */ +@Suppress("unused") +object InstalledInventoryProvider { + @JvmStatic + fun scanInstalledInventory(context: Context, optionsJson: String): String { + val options = InstalledInventoryJson.decodeOptions(optionsJson) + val result = InstalledInventoryScanner.scan(context.applicationContext ?: context, options) + return InstalledInventoryJson.encodeResult(result) + } +} + +object InstalledInventoryScanner { + fun scan( + context: Context, + options: InstalledInventoryScanOptions = InstalledInventoryScanOptions(), + ): InstalledInventoryScanResult { + val packageManager = context.packageManager + val rawPackages = getInstalledPackages(packageManager).map { packageInfo -> + packageInfo.toRawInstalledPackage(packageManager) + } + val result = InstalledInventoryCollector.collect( + selfPackageName = context.packageName, + packages = rawPackages, + options = options, + ) + val diagnostics = result.diagnostics.toMutableList() + if (!declaresQueryAllPackages(context)) { + diagnostics += PlatformDiagnostic( + code = "package_visibility.query_all_packages_missing", + message = "QUERY_ALL_PACKAGES is not declared; installed app inventory may be incomplete.", + ) + } + return result.copy(diagnostics = diagnostics) + } + + @Suppress("DEPRECATION") + private fun getInstalledPackages(packageManager: PackageManager): List { + return if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.TIRAMISU) { + packageManager.getInstalledPackages(PackageManager.PackageInfoFlags.of(0)) + } else { + packageManager.getInstalledPackages(0) + } + } + + private fun PackageInfo.toRawInstalledPackage(packageManager: PackageManager): RawInstalledPackage { + val appInfo = applicationInfo + return RawInstalledPackage( + packageName = packageName.orEmpty(), + label = appInfo.safeLabel(packageManager), + versionName = versionName?.takeIf { it.isNotBlank() }, + versionCode = packageVersionCode(), + isSystem = appInfo.isSystemPackage(), + ) + } + + private fun ApplicationInfo?.safeLabel(packageManager: PackageManager): String? { + return try { + this?.loadLabel(packageManager)?.toString()?.takeIf { it.isNotBlank() } + } catch (_: RuntimeException) { + null + } + } + + private fun ApplicationInfo?.isSystemPackage(): Boolean { + val flags = this?.flags ?: return false + return flags and ApplicationInfo.FLAG_SYSTEM != 0 || + flags and ApplicationInfo.FLAG_UPDATED_SYSTEM_APP != 0 + } + + @Suppress("DEPRECATION") + private fun PackageInfo.packageVersionCode(): Long { + return if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.P) { + longVersionCode + } else { + versionCode.toLong() + } + } + + private fun declaresQueryAllPackages(context: Context): Boolean { + return try { + val packageInfo = if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.TIRAMISU) { + context.packageManager.getPackageInfo( + context.packageName, + PackageManager.PackageInfoFlags.of(PackageManager.GET_PERMISSIONS.toLong()), + ) + } else { + @Suppress("DEPRECATION") + context.packageManager.getPackageInfo(context.packageName, PackageManager.GET_PERMISSIONS) + } + packageInfo.requestedPermissions?.contains(Manifest.permission.QUERY_ALL_PACKAGES) == true + } catch (_: RuntimeException) { + false + } + } +} + +object InstalledInventoryCollector { + fun collect( + selfPackageName: String, + packages: List, + options: InstalledInventoryScanOptions, + ): InstalledInventoryScanResult { + var filteredSystem = 0 + var filteredSelf = 0 + val itemsByPackageName = linkedMapOf() + + for (rawPackage in packages) { + val packageName = rawPackage.packageName.trim() + if (packageName.isEmpty()) { + continue + } + if (!options.includeSelf && packageName == selfPackageName) { + filteredSelf++ + continue + } + if (!options.includeSystemApps && rawPackage.isSystem) { + filteredSystem++ + continue + } + itemsByPackageName[packageName] = InstalledInventoryItem( + packageName = packageName, + label = rawPackage.label?.takeIf { it.isNotBlank() }, + versionName = rawPackage.versionName?.takeIf { it.isNotBlank() }, + versionCode = rawPackage.versionCode, + ) + } + + val items = itemsByPackageName.values.sortedBy { it.packageName } + return InstalledInventoryScanResult( + inventory = InstalledInventory(items = items), + stats = InstalledInventoryScanStats( + totalSeen = packages.size, + returned = items.size, + filteredSystem = filteredSystem, + filteredSelf = filteredSelf, + ), + ) + } +} + +data class InstalledInventoryScanOptions( + val includeSystemApps: Boolean = false, + val includeSelf: Boolean = false, +) + +data class RawInstalledPackage( + val packageName: String, + val label: String? = null, + val versionName: String? = null, + val versionCode: Long? = null, + val isSystem: Boolean = false, +) + +data class InstalledInventoryScanResult( + val inventory: InstalledInventory, + val stats: InstalledInventoryScanStats, + val diagnostics: List = emptyList(), +) + +data class InstalledInventory( + val format: String = INSTALLED_INVENTORY_FORMAT, + val version: Int = INSTALLED_INVENTORY_VERSION, + val items: List = emptyList(), +) + +data class InstalledInventoryItem( + val packageName: String, + val label: String? = null, + val versionName: String? = null, + val versionCode: Long? = null, +) + +data class InstalledInventoryScanStats( + val totalSeen: Int, + val returned: Int, + val filteredSystem: Int, + val filteredSelf: Int, +) + +data class PlatformDiagnostic( + val code: String, + val message: String, + val detail: String? = null, +) + +object InstalledInventoryJson { + fun decodeOptions(json: String): InstalledInventoryScanOptions { + val value = if (json.isBlank()) JSONObject() else JSONObject(json) + return InstalledInventoryScanOptions( + includeSystemApps = value.optBoolean("include_system_apps", false), + includeSelf = value.optBoolean("include_self", false), + ) + } + + fun encodeResult(result: InstalledInventoryScanResult): String { + return JSONObject() + .put("inventory", encodeInventory(result.inventory)) + .put("stats", encodeStats(result.stats)) + .put("diagnostics", JSONArray().also { diagnostics -> + result.diagnostics.forEach { diagnostics.put(encodeDiagnostic(it)) } + }) + .toString() + } + + private fun encodeInventory(inventory: InstalledInventory): JSONObject { + return JSONObject() + .put("format", inventory.format) + .put("version", inventory.version) + .put("items", JSONArray().also { items -> + inventory.items.forEach { items.put(encodeItem(it)) } + }) + } + + private fun encodeItem(item: InstalledInventoryItem): JSONObject { + return JSONObject() + .put("kind", "android_package") + .put("package_name", item.packageName) + .putNullable("label", item.label) + .putNullable("version_name", item.versionName) + .putNullable("version_code", item.versionCode) + } + + private fun encodeStats(stats: InstalledInventoryScanStats): JSONObject { + return JSONObject() + .put("total_seen", stats.totalSeen) + .put("returned", stats.returned) + .put("filtered_system", stats.filteredSystem) + .put("filtered_self", stats.filteredSelf) + } + + private fun encodeDiagnostic(diagnostic: PlatformDiagnostic): JSONObject { + return JSONObject() + .put("code", diagnostic.code) + .put("message", diagnostic.message) + .putNullable("detail", diagnostic.detail) + } + + private fun JSONObject.putNullable(name: String, value: Any?): JSONObject { + return put(name, value ?: JSONObject.NULL) + } +} diff --git a/core-getter/src/main/rust/api_proxy/Cargo.toml b/core-getter/src/main/rust/api_proxy/Cargo.toml index 0dfb56f6..95d3c08c 100644 --- a/core-getter/src/main/rust/api_proxy/Cargo.toml +++ b/core-getter/src/main/rust/api_proxy/Cargo.toml @@ -8,6 +8,7 @@ edition = "2021" [dependencies] jni = "0.21" # from rustls-platform-verifier-android, sync version getter = { path = "../getter", default-features = false, features = ["native-tokio", "rustls-platform-verifier-android"] } +upgradeall-platform-adapter = { path = "../platform_adapter" } serde = { version = "1.0", features = ["derive"] } serde_json = "1.0.145" tokio = { version = "1.48.0", features = ["rt-multi-thread"] } diff --git a/core-getter/src/main/rust/api_proxy/src/lib.rs b/core-getter/src/main/rust/api_proxy/src/lib.rs index 8006bd74..3f2dc3a4 100644 --- a/core-getter/src/main/rust/api_proxy/src/lib.rs +++ b/core-getter/src/main/rust/api_proxy/src/lib.rs @@ -15,16 +15,42 @@ pub extern "C" fn Java_net_xzos_upgradeall_getter_NativeLib_runServer<'local>( _context: JObject, callback: JObject<'local>, ) -> JString<'local> { - // Initialize the certificate verifier for future use. + // Initialize Android-hosted Rust platform integrations for future use. // https://github.com/rustls/rustls-platform-verifier/tree/3edb4d278215a8603020351b8b519d907a26041f?tab=readme-ov-file#crate-initialization #[cfg(target_os = "android")] - match rustls_platform_verifier::android::init_hosted(&mut env, _context) { - Ok(_) => {} - Err(e) => { + { + let rustls_context = match env.new_local_ref(&_context) { + Ok(context) => context, + Err(e) => { + return env + .new_string(format!("Error creating rustls context ref: {}", e)) + .expect("Failed to create Java string"); + } + }; + if let Err(e) = rustls_platform_verifier::android::init_hosted(&mut env, rustls_context) { return env .new_string(format!("Error initializing certificate verifier: {}", e)) .expect("Failed to create Java string"); } + + let platform_context = match env.new_local_ref(&_context) { + Ok(context) => context, + Err(e) => { + return env + .new_string(format!( + "Error creating platform adapter context ref: {}", + e + )) + .expect("Failed to create Java string"); + } + }; + if let Err(e) = + upgradeall_platform_adapter::android::init_with_env(&mut env, platform_context) + { + return env + .new_string(format!("Error initializing platform adapter: {}", e)) + .expect("Failed to create Java string"); + } } let (startup_tx, startup_rx) = channel::>(); thread::spawn(move || { diff --git a/core-getter/src/main/rust/platform_adapter/src/android.rs b/core-getter/src/main/rust/platform_adapter/src/android.rs index e8543021..acd211d9 100644 --- a/core-getter/src/main/rust/platform_adapter/src/android.rs +++ b/core-getter/src/main/rust/platform_adapter/src/android.rs @@ -2,23 +2,22 @@ //! //! This follows the same shape as rustls-platform-verifier: the Rust native //! entrypoint initializes JVM/context/classloader handles once, then Rust code -//! can attach a thread and call app classes through the app classloader. The -//! PackageManager scanner/provider is intentionally not implemented in this -//! skeleton slice. +//! can attach a thread and call app classes through the app classloader. use crate::{ InstalledInventoryScanOptions, InstalledInventoryScanResult, PlatformAdapter, PlatformAdapterError, }; -use jni::objects::{GlobalRef, JObject, JString}; +use jni::objects::{GlobalRef, JClass, JObject, JString, JValue}; use jni::{JNIEnv, JavaVM}; use once_cell::sync::OnceCell; static RUNTIME: OnceCell = OnceCell::new(); +const INSTALLED_INVENTORY_PROVIDER_CLASS: &str = + "net.xzos.upgradeall.getter.platform.InstalledInventoryProvider"; struct AndroidRuntime { java_vm: JavaVM, - #[allow(dead_code)] application_context: GlobalRef, class_loader: GlobalRef, } @@ -88,11 +87,44 @@ pub struct AndroidPlatformAdapter; impl PlatformAdapter for AndroidPlatformAdapter { fn scan_installed_inventory( &self, - _options: InstalledInventoryScanOptions, + options: InstalledInventoryScanOptions, ) -> Result { - with_attached_env(|_env, _runtime| Ok(()))?; - Err(PlatformAdapterError::Unsupported { - capability: "installed_inventory.android_jni_provider", + let options_json = serde_json::to_string(&options).map_err(|error| { + PlatformAdapterError::MalformedResponse(format!( + "failed to encode scan options for Android provider: {error}" + )) + })?; + + with_attached_env(|env, runtime| { + let provider_class = JClass::from(load_class( + env, + runtime, + INSTALLED_INVENTORY_PROVIDER_CLASS, + )?); + let context = env + .new_local_ref(runtime.application_context.as_obj()) + .map_err(|error| PlatformAdapterError::Jni(error.to_string()))?; + let options_json = env + .new_string(options_json) + .map_err(|error| PlatformAdapterError::Jni(error.to_string()))?; + let options_json = JObject::from(options_json); + + let result = env + .call_static_method( + provider_class, + "scanInstalledInventory", + "(Landroid/content/Context;Ljava/lang/String;)Ljava/lang/String;", + &[JValue::Object(&context), JValue::Object(&options_json)], + ) + .and_then(|value| value.l()) + .map_err(|error| PlatformAdapterError::Jni(error.to_string()))?; + let result_json = java_string(env, result)?; + + serde_json::from_str(&result_json).map_err(|error| { + PlatformAdapterError::MalformedResponse(format!( + "Android installed inventory provider returned invalid JSON: {error}" + )) + }) }) } } @@ -110,7 +142,6 @@ fn with_attached_env( } /// Load an application class with the app classloader instead of `FindClass`. -#[allow(dead_code)] fn load_class<'local>( env: &mut JNIEnv<'local>, runtime: &AndroidRuntime, @@ -132,7 +163,6 @@ fn load_class<'local>( } /// Convert a Java string into a Rust string. -#[allow(dead_code)] fn java_string(env: &mut JNIEnv<'_>, value: JObject<'_>) -> Result { if value.is_null() { return Ok(String::new()); diff --git a/core-getter/src/test/java/net/xzos/upgradeall/getter/platform/InstalledInventoryCollectorTest.kt b/core-getter/src/test/java/net/xzos/upgradeall/getter/platform/InstalledInventoryCollectorTest.kt new file mode 100644 index 00000000..c4c6757d --- /dev/null +++ b/core-getter/src/test/java/net/xzos/upgradeall/getter/platform/InstalledInventoryCollectorTest.kt @@ -0,0 +1,110 @@ +package net.xzos.upgradeall.getter.platform + +import org.junit.Assert.assertEquals +import org.junit.Assert.assertFalse +import org.junit.Assert.assertTrue +import org.junit.Test + +class InstalledInventoryCollectorTest { + @Test + fun defaultOptionsFilterSelfAndSystemPackagesAndSortResults() { + val result = InstalledInventoryCollector.collect( + selfPackageName = "net.xzos.upgradeall", + packages = listOf( + rawPackage("org.fdroid.fdroid", label = "F-Droid"), + rawPackage("android", label = "Android System", isSystem = true), + rawPackage("net.xzos.upgradeall", label = "UpgradeAll"), + rawPackage("com.termux", label = "Termux"), + ), + options = InstalledInventoryScanOptions(), + ) + + assertEquals(4, result.stats.totalSeen) + assertEquals(2, result.stats.returned) + assertEquals(1, result.stats.filteredSystem) + assertEquals(1, result.stats.filteredSelf) + assertEquals(listOf("com.termux", "org.fdroid.fdroid"), result.inventory.items.map { it.packageName }) + assertFalse(result.inventory.items.any { it.packageName.startsWith("android/") }) + } + + @Test + fun optionsCanIncludeSelfAndSystemPackages() { + val result = InstalledInventoryCollector.collect( + selfPackageName = "net.xzos.upgradeall", + packages = listOf( + rawPackage("android", isSystem = true), + rawPackage("net.xzos.upgradeall"), + ), + options = InstalledInventoryScanOptions( + includeSystemApps = true, + includeSelf = true, + ), + ) + + assertEquals(2, result.stats.totalSeen) + assertEquals(2, result.stats.returned) + assertEquals(0, result.stats.filteredSystem) + assertEquals(0, result.stats.filteredSelf) + assertEquals(listOf("android", "net.xzos.upgradeall"), result.inventory.items.map { it.packageName }) + } + + @Test + fun duplicatePackageNamesKeepTheLastFactDeterministically() { + val result = InstalledInventoryCollector.collect( + selfPackageName = "net.xzos.upgradeall", + packages = listOf( + rawPackage("org.fdroid.fdroid", label = "Old Label", versionCode = 1), + rawPackage("org.fdroid.fdroid", label = "New Label", versionCode = 2), + ), + options = InstalledInventoryScanOptions(), + ) + + assertEquals(2, result.stats.totalSeen) + assertEquals(1, result.stats.returned) + assertEquals("New Label", result.inventory.items.single().label) + assertEquals(2L, result.inventory.items.single().versionCode) + } + + @Test + fun blankPackageNamesAreSkippedWithoutCreatingPackageIds() { + val result = InstalledInventoryCollector.collect( + selfPackageName = "net.xzos.upgradeall", + packages = listOf( + rawPackage(" ", label = "Blank"), + rawPackage("com.example.valid", label = "Valid"), + ), + options = InstalledInventoryScanOptions(), + ) + + assertEquals(2, result.stats.totalSeen) + assertEquals(1, result.stats.returned) + assertEquals("com.example.valid", result.inventory.items.single().packageName) + } + + @Test + fun inventoryContractMatchesGetterInstalledInventoryFormat() { + val result = InstalledInventoryCollector.collect( + selfPackageName = "net.xzos.upgradeall", + packages = listOf(rawPackage("org.fdroid.fdroid")), + options = InstalledInventoryScanOptions(), + ) + + assertEquals("upgradeall-installed-inventory", result.inventory.format) + assertEquals(1, result.inventory.version) + assertTrue(result.diagnostics.isEmpty()) + } + + private fun rawPackage( + packageName: String, + label: String? = null, + versionName: String? = null, + versionCode: Long? = null, + isSystem: Boolean = false, + ) = RawInstalledPackage( + packageName = packageName, + label = label, + versionName = versionName, + versionCode = versionCode, + isSystem = isSystem, + ) +} diff --git a/core-websdk/data/build.gradle.kts b/core-websdk/data/build.gradle.kts index 0ee8d3dc..1d00a186 100644 --- a/core-websdk/data/build.gradle.kts +++ b/core-websdk/data/build.gradle.kts @@ -1,3 +1,5 @@ +import org.jetbrains.kotlin.gradle.dsl.JvmTarget + plugins { id("java-library") alias(libs.plugins.kotlin.jvm) @@ -8,6 +10,12 @@ java { targetCompatibility = JavaVersion.VERSION_21 } +kotlin { + compilerOptions { + jvmTarget.set(JvmTarget.JVM_21) + } +} + dependencies { implementation(libs.gson) implementation(libs.jackson.databind) diff --git a/docs/architecture/adr/0009-android-platform-adapter-and-package-visibility.md b/docs/architecture/adr/0009-android-platform-adapter-and-package-visibility.md index 022848e1..e047d8b2 100644 --- a/docs/architecture/adr/0009-android-platform-adapter-and-package-visibility.md +++ b/docs/architecture/adr/0009-android-platform-adapter-and-package-visibility.md @@ -110,21 +110,28 @@ The permission may have distribution-policy implications on app stores. The proj If lint/build tooling flags `QUERY_ALL_PACKAGES`, the manifest may suppress that lint with an inline comment and `tools:ignore="QueryAllPackagesPermission"`; this suppression must remain documented as policy, not treated as a generic lint cleanup. -## First implementation slice +## Implementation slices -The first slice is intentionally narrow: +The first slice was intentionally narrow: - document this ADR and update existing boundary docs; - add `QUERY_ALL_PACKAGES` to the Flutter product manifest; - add a superproject Rust crate for platform adapter DTOs, errors, a `NoopPlatformAdapter`, and Android runtime/JNI initialization skeleton; - add validation for that crate. -The first slice does not: +The second slice adds the first Android facts provider while preserving the same boundary: + +- `net.xzos.upgradeall.getter.platform.InstalledInventoryProvider` is a no-UI Kotlin provider called by Rust JNI through the app classloader; +- Kotlin `InstalledInventoryScanner` collects raw PackageManager facts and encodes the getter-compatible installed inventory JSON; +- Kotlin collector tests cover filtering, sorting, duplicate handling, and contract format without constructing package ids; +- Rust `AndroidPlatformAdapter::scan_installed_inventory` serializes scan options, calls the provider, and deserializes the JSON into platform DTOs; +- `api_proxy` initializes the platform adapter runtime alongside `rustls-platform-verifier`, using separate JNI local refs for each initializer. + +These slices still do not: - make the reusable getter submodule depend on superproject-only crates; -- add Kotlin PackageManager scanner behavior; -- wire product native bridge operations; - add Flutter installed-autogen UX; +- add product native bridge operations that combine platform scan + getter autogen preview/apply; - add Magisk scanning; - add live downloads, background worker policy, installer URI/SAF semantics, or notification behavior. diff --git a/justfile b/justfile index cea0161d..c6b55e1c 100644 --- a/justfile +++ b/justfile @@ -9,6 +9,7 @@ verify: just test-getter-bdd just test-flutter-widget just verify-workspace-skeleton + just test-android-platform-adapter just test-flutter-getter-cli-integration just build-flutter-android-debug @@ -30,6 +31,9 @@ test-flutter-getter-cli-integration: cargo build --manifest-path {{ GETTER_MANIFEST }} -p getter-cli --bin getter-cli cd app_flutter && GETTER_CLI_BIN="../core-getter/src/main/rust/getter/target/debug/getter-cli" flutter test dev_test/cli_getter_adapter_test.dart +test-android-platform-adapter: + ./gradlew --no-daemon ':core-getter:buildDebugApi_proxyRust[arm64-v8a]' ':core-getter:buildDebugApi_proxyRust[armeabi-v7a]' ':core-getter:buildDebugApi_proxyRust[x86_64]' :core-getter:testDebugUnitTest --tests 'net.xzos.upgradeall.getter.platform.InstalledInventoryCollectorTest' :core-getter:assembleDebug + build-flutter-android-debug: cd app_flutter && flutter build apk --debug @@ -42,6 +46,7 @@ verify-workspace-skeleton: cargo fmt --manifest-path {{ PLATFORM_ADAPTER_MANIFEST }} --all --check cargo check --manifest-path {{ GETTER_MANIFEST }} --workspace --all-targets cargo check --manifest-path {{ API_PROXY_MANIFEST }} + cargo check --manifest-path {{ API_PROXY_MANIFEST }} --target aarch64-linux-android cargo test --manifest-path {{ PLATFORM_ADAPTER_MANIFEST }} cargo check --manifest-path {{ PLATFORM_ADAPTER_MANIFEST }} --target aarch64-linux-android cd app_flutter && flutter analyze diff --git a/todo.md b/todo.md index b3a9edea..d182fc35 100644 --- a/todo.md +++ b/todo.md @@ -438,11 +438,12 @@ Completed tasks: 5. Track accepted generated packages in getter storage without clobbering existing user state. 6. Preserve edited generated files into `local` before autogen rewrite/delete. 7. Guard cleanup deletion by current autogen manifest, repository id, and generated-package resolution. +8. Add Rust-active Android installed inventory provider/scanner path: Kotlin PackageManager facts provider, Rust JNI call/deserialization, and `api_proxy` runtime initialization. Remaining tasks: -1. Finish the Rust-active Android platform adapter path from ADR-0009: Kotlin PackageManager provider/scanner, native bridge packaging, and Rust calls that return installed inventory facts. -2. Add getter/native bridge operations that combine platform scan + getter `local_autogen` preview/apply; Flutter must consume these getter-owned DTOs rather than leading a MethodChannel inventory scan. +1. Add getter/native bridge operations that combine platform scan + getter `local_autogen` preview/apply; Flutter must consume these getter-owned DTOs rather than leading a MethodChannel inventory scan. +2. Wire/package the production bridge into `app_flutter` so the Flutter APK can exercise the Rust-active installed inventory provider path. 3. Flutter confirmation UX consumes getter preview/apply DTOs. 4. Cache invalidation hooks beyond file-hash-based repository reload need to be expanded when evaluated/provider caches become active. From dd9415462dc27e5cd5f0f6724f785bccd08d1c00 Mon Sep 17 00:00:00 2001 From: Xiangzhe Date: Wed, 24 Jun 2026 00:03:40 +0800 Subject: [PATCH 24/85] feat(app): bridge installed autogen to Flutter --- .github/workflows/android.yml | 8 +- .../upgradeall-rewrite-validation.yml | 6 +- app_flutter/README.md | 8 +- app_flutter/android/app/build.gradle | 4 +- .../net/xzos/upgradeall/MainActivity.kt | 101 ++++ app_flutter/android/build.gradle | 16 + .../android/getter_bridge/build.gradle | 72 +++ .../src/main/AndroidManifest.xml | 1 + .../net/xzos/upgradeall/getter/NativeLib.kt | 26 ++ app_flutter/android/settings.gradle | 4 + app_flutter/lib/cli_getter_adapter.dart | 28 ++ app_flutter/lib/getter_adapter.dart | 413 +++++++++++++++++ app_flutter/lib/main.dart | 277 +++++++++++ app_flutter/lib/native_getter_adapter.dart | 111 +++++ .../test/native_getter_adapter_test.dart | 161 +++++++ app_flutter/test/widget_test.dart | 81 ++++ .../net/xzos/upgradeall/getter/NativeLib.kt | 6 +- .../src/main/rust/api_proxy/Cargo.toml | 3 +- .../src/main/rust/api_proxy/src/lib.rs | 437 ++++++++++++++++-- core-getter/src/main/rust/getter | 2 +- .../0007-flutter-getter-bridge-contract.md | 15 +- ...platform-adapter-and-package-visibility.md | 16 +- .../upgradeall-getter-rewrite-wiki.md | 2 + justfile | 3 +- todo.md | 12 +- tools/verify_flutter_apk_bridge.py | 45 ++ 26 files changed, 1783 insertions(+), 75 deletions(-) create mode 100644 app_flutter/android/getter_bridge/build.gradle create mode 100644 app_flutter/android/getter_bridge/src/main/AndroidManifest.xml create mode 100644 app_flutter/android/getter_bridge/src/main/kotlin/net/xzos/upgradeall/getter/NativeLib.kt create mode 100644 app_flutter/lib/native_getter_adapter.dart create mode 100644 app_flutter/test/native_getter_adapter_test.dart create mode 100644 tools/verify_flutter_apk_bridge.py diff --git a/.github/workflows/android.yml b/.github/workflows/android.yml index 48048de8..8c01b562 100644 --- a/.github/workflows/android.yml +++ b/.github/workflows/android.yml @@ -39,10 +39,12 @@ jobs: BUILD_TOOL_VERSION=$(ls "$ANDROID_HOME/build-tools" | sort -V | tail -n 1) echo "BUILD_TOOL_VERSION=$BUILD_TOOL_VERSION" >> "$GITHUB_ENV" echo "$ANDROID_HOME/build-tools/$BUILD_TOOL_VERSION" >> "$GITHUB_PATH" - echo "ANDROID_NDK_HOME=$ANDROID_HOME/ndk/$NDK_VERSION" >> "$GITHUB_ENV" - echo "ANDROID_NDK_ROOT=$ANDROID_HOME/ndk/$NDK_VERSION" >> "$GITHUB_ENV" + NDK_HOME="$ANDROID_HOME/ndk/$NDK_VERSION" + echo "ANDROID_NDK_HOME=$NDK_HOME" >> "$GITHUB_ENV" + echo "ANDROID_NDK_ROOT=$NDK_HOME" >> "$GITHUB_ENV" + echo "CC_aarch64_linux_android=$NDK_HOME/toolchains/llvm/prebuilt/linux-x86_64/bin/aarch64-linux-android23-clang" >> "$GITHUB_ENV" echo "Android build tools: $BUILD_TOOL_VERSION" - echo "Android NDK: $ANDROID_HOME/ndk/$NDK_VERSION" + echo "Android NDK: $NDK_HOME" - name: Install Flutter uses: subosito/flutter-action@v2 diff --git a/.github/workflows/upgradeall-rewrite-validation.yml b/.github/workflows/upgradeall-rewrite-validation.yml index f3908391..d05375d4 100644 --- a/.github/workflows/upgradeall-rewrite-validation.yml +++ b/.github/workflows/upgradeall-rewrite-validation.yml @@ -34,8 +34,10 @@ jobs: - name: Add Android NDK to environment run: | - echo "ANDROID_NDK_HOME=$ANDROID_HOME/ndk/$NDK_VERSION" >> "$GITHUB_ENV" - echo "ANDROID_NDK_ROOT=$ANDROID_HOME/ndk/$NDK_VERSION" >> "$GITHUB_ENV" + NDK_HOME="$ANDROID_HOME/ndk/$NDK_VERSION" + echo "ANDROID_NDK_HOME=$NDK_HOME" >> "$GITHUB_ENV" + echo "ANDROID_NDK_ROOT=$NDK_HOME" >> "$GITHUB_ENV" + echo "CC_aarch64_linux_android=$NDK_HOME/toolchains/llvm/prebuilt/linux-x86_64/bin/aarch64-linux-android23-clang" >> "$GITHUB_ENV" - name: Install Flutter uses: subosito/flutter-action@v2 diff --git a/app_flutter/README.md b/app_flutter/README.md index 0bc55fa1..78efbafb 100644 --- a/app_flutter/README.md +++ b/app_flutter/README.md @@ -10,9 +10,11 @@ This is the new Flutter shell and product APK entry for the UpgradeAll rewrite. - Placeholder routes for apps, repositories, downloads, logs, settings, and legacy migration - `FakeGetterAdapter` for deterministic widget tests - `CliGetterAdapter` as a development/integration bridge against the real `getter-cli` JSON envelope -- Product manifest permissions include `QUERY_ALL_PACKAGES` per ADR-0009 so the future Rust-active Android platform adapter can provide complete installed package inventory facts to getter. +- A slim Android `:getter_bridge` library inside `app_flutter/android/getter_bridge` packages the Rust `api_proxy` native library and the no-UI installed-inventory provider classes into the Flutter product APK without depending on the legacy native `:app` UI or old `GetterPort` RPC wrapper surface. +- `MainActivity` exposes a no-UI `net.xzos.upgradeall/getter_bridge` MethodChannel for native bridge plumbing. The installed-autogen methods derive the app-private getter data directory on Android, call Rust JNI entrypoints, and return getter-style JSON envelopes consumed by `MethodChannelGetterAdapter`. +- Product manifest permissions include `QUERY_ALL_PACKAGES` per ADR-0009 so the Rust-active Android platform adapter can provide complete installed package inventory facts to getter. -`CliGetterAdapter` is not the final Android production bridge. It exists to keep the getter-owned DTO and error contract executable while the native bridge is designed in ADR-0007. Installed-autogen product flows must use getter/native bridge operations backed by the Rust-active platform adapter from ADR-0009, not a Flutter-led MethodChannel scanner. +`CliGetterAdapter` is not the final Android production bridge. It exists to keep the getter-owned DTO and error contract executable. `MethodChannelGetterAdapter` is the current production bridge slice for installed-autogen preview/apply: Flutter renders getter-owned DTOs and passes accepted package ids back to getter, but PackageManager scanning, package-id decisions, and `local_autogen` writes remain in Rust/native getter code. ## Verification @@ -22,6 +24,6 @@ flutter test GETTER_CLI_BIN=/path/to/getter-cli flutter test dev_test/cli_getter_adapter_test.dart ``` -From the repository root, `just verify` also runs the Flutter analyzer, widget tests, getter CLI integration/dev test, and Android debug build. +From the repository root, `just verify` also runs the Flutter analyzer, widget tests, getter CLI integration/dev test, Android debug build, and an APK inspection that verifies the Flutter APK contains `libapi_proxy.so`, `NativeLib`, and `InstalledInventoryProvider`. Android CI/release artifacts are built from this Flutter project with `flutter build apk`; the root Gradle `:app` module is no longer the rewrite product APK path. diff --git a/app_flutter/android/app/build.gradle b/app_flutter/android/app/build.gradle index d1cc0deb..5bc415de 100644 --- a/app_flutter/android/app/build.gradle +++ b/app_flutter/android/app/build.gradle @@ -86,4 +86,6 @@ flutter { source '../..' } -dependencies {} +dependencies { + implementation project(':getter_bridge') +} diff --git a/app_flutter/android/app/src/main/kotlin/net/xzos/upgradeall/MainActivity.kt b/app_flutter/android/app/src/main/kotlin/net/xzos/upgradeall/MainActivity.kt index babaf554..ce20e44d 100644 --- a/app_flutter/android/app/src/main/kotlin/net/xzos/upgradeall/MainActivity.kt +++ b/app_flutter/android/app/src/main/kotlin/net/xzos/upgradeall/MainActivity.kt @@ -5,16 +5,46 @@ import android.os.Handler import android.os.Looper import io.flutter.embedding.android.FlutterActivity import io.flutter.embedding.engine.FlutterEngine +import io.flutter.plugin.common.MethodCall import io.flutter.plugin.common.MethodChannel import java.io.File import java.util.concurrent.Executors +import net.xzos.upgradeall.getter.NativeLib +import org.json.JSONArray +import org.json.JSONObject class MainActivity : FlutterActivity() { private val legacyMigrationExecutor = Executors.newSingleThreadExecutor() + private val getterBridgeExecutor = Executors.newSingleThreadExecutor() private val mainHandler = Handler(Looper.getMainLooper()) + private val nativeLib by lazy { NativeLib() } override fun configureFlutterEngine(flutterEngine: FlutterEngine) { super.configureFlutterEngine(flutterEngine) + MethodChannel( + flutterEngine.dartExecutor.binaryMessenger, + GETTER_BRIDGE_CHANNEL, + ).setMethodCallHandler { call, result -> + when (call.method) { + "initializeBridge" -> runGetterBridge(result) { + nativeLib.initializeBridge(applicationContext) + } + + "previewInstalledAutogen" -> runGetterBridge(result) { + nativeLib.previewInstalledAutogen( + applicationContext, + previewInstalledAutogenRequest(call), + ) + } + + "applyInstalledAutogen" -> runGetterBridge(result) { + nativeLib.applyInstalledAutogen(applyInstalledAutogenRequest(call)) + } + + else -> result.notImplemented() + } + } + MethodChannel( flutterEngine.dartExecutor.binaryMessenger, LEGACY_MIGRATION_CHANNEL, @@ -44,9 +74,79 @@ class MainActivity : FlutterActivity() { override fun onDestroy() { legacyMigrationExecutor.shutdown() + getterBridgeExecutor.shutdown() super.onDestroy() } + private fun runGetterBridge(result: MethodChannel.Result, operation: () -> String) { + getterBridgeExecutor.execute { + try { + val response = operation() + mainHandler.post { result.success(response) } + } catch (error: UnsatisfiedLinkError) { + mainHandler.post { + result.error( + "bridge.native_unavailable", + error.message ?: "Getter native bridge is unavailable", + null, + ) + } + } catch (error: Exception) { + mainHandler.post { + result.error( + "bridge.call_failed", + error.message ?: "Getter native bridge call failed", + null, + ) + } + } + } + } + + private fun previewInstalledAutogenRequest(call: MethodCall): String { + val args = call.arguments as? Map<*, *> ?: emptyMap() + val scanOptions = args["scan_options"] as? Map<*, *> ?: args + return JSONObject() + .put("data_dir", getterDataDir().absolutePath) + .put( + "scan_options", + JSONObject() + .put( + "include_system_apps", + scanOptions["include_system_apps"] as? Boolean ?: false, + ) + .put( + "include_self", + scanOptions["include_self"] as? Boolean ?: false, + ), + ) + .toString() + } + + private fun applyInstalledAutogenRequest(call: MethodCall): String { + val args = call.arguments as? Map<*, *> ?: emptyMap() + val previewJson = args["preview_json"] as? String + ?: throw IllegalArgumentException("preview_json is required") + val acceptance = args["acceptance"] as? Map<*, *> + val packageIds = acceptance + ?.get("package_ids") + ?.let { value -> value as? Collection<*> } + ?.map { value -> value.toString() } + ?: emptyList() + return JSONObject() + .put("data_dir", getterDataDir().absolutePath) + .put("preview", JSONObject(previewJson)) + .put( + "acceptance", + JSONObject() + .put("mode", acceptance?.get("mode") as? String ?: "all") + .put("package_ids", JSONArray(packageIds)), + ) + .toString() + } + + private fun getterDataDir(): File = File(filesDir, "getter") + private fun prepareLegacyRoomImport(): Map { val source = getDatabasePath(LEGACY_ROOM_DB_NAME) if (!source.exists()) { @@ -109,6 +209,7 @@ class MainActivity : FlutterActivity() { } private companion object { + const val GETTER_BRIDGE_CHANNEL = "net.xzos.upgradeall/getter_bridge" const val LEGACY_MIGRATION_CHANNEL = "net.xzos.upgradeall/legacy_migration" const val LEGACY_ROOM_DB_NAME = "app_metadata_database.db" val SQLITE_SUFFIXES = listOf("", "-wal", "-shm") diff --git a/app_flutter/android/build.gradle b/app_flutter/android/build.gradle index cc71bc70..3eeea9b3 100644 --- a/app_flutter/android/build.gradle +++ b/app_flutter/android/build.gradle @@ -1,3 +1,5 @@ +import groovy.json.JsonSlurper + buildscript { ext.kotlin_version = '2.0.0' repositories { @@ -10,10 +12,24 @@ buildscript { } } +String findRustlsPlatformVerifierProject() { + def apiProxyManifest = file('../../core-getter/src/main/rust/api_proxy/Cargo.toml') + def dependencyText = providers.exec { + commandLine('cargo', 'metadata', '--format-version', '1', '--manifest-path', apiProxyManifest.path) + }.standardOutput.asText.get() + def dependencyJson = new JsonSlurper().parseText(dependencyText) + def manifestPath = file(dependencyJson.packages.find { it.name == 'rustls-platform-verifier-android' }.manifest_path) + return new File(manifestPath.parentFile, 'maven').path +} + allprojects { repositories { google() mavenCentral() + maven { + url = findRustlsPlatformVerifierProject() + metadataSources.artifact() + } } } diff --git a/app_flutter/android/getter_bridge/build.gradle b/app_flutter/android/getter_bridge/build.gradle new file mode 100644 index 00000000..594c2d47 --- /dev/null +++ b/app_flutter/android/getter_bridge/build.gradle @@ -0,0 +1,72 @@ +plugins { + id "com.android.library" + id "kotlin-android" + id "io.github.MatrixDev.android-rust" +} + +def resolveAndroidNdkPath() { + def envNdk = System.getenv("ANDROID_NDK_HOME") ?: System.getenv("ANDROID_NDK_ROOT") + if (envNdk != null && !envNdk.isBlank()) { + return envNdk + } + def localProperties = new Properties() + def localPropertiesFile = rootProject.file("local.properties") + if (localPropertiesFile.exists()) { + localPropertiesFile.withInputStream { localProperties.load(it) } + def sdkDir = localProperties.getProperty("sdk.dir") + if (sdkDir != null && !sdkDir.isBlank()) { + return new File(sdkDir, "ndk/29.0.14206865").path + } + } + return null +} + +android { + namespace "net.xzos.upgradeall.getter.bridge" + compileSdkVersion 36 + ndkVersion "29.0.14206865" + def configuredNdkPath = resolveAndroidNdkPath() + if (configuredNdkPath != null) { + ndkPath configuredNdkPath + } + + defaultConfig { + minSdkVersion 23 + consumerProguardFiles file("../../../core-getter/consumer-rules.pro") + } + + sourceSets { + main.java.srcDirs += file("../../../core-getter/src/main/java/net/xzos/upgradeall/getter/platform") + } + + compileOptions { + sourceCompatibility JavaVersion.VERSION_17 + targetCompatibility JavaVersion.VERSION_17 + } + + kotlinOptions { + jvmTarget = "17" + } +} + +androidRust { + minimumSupportedRustVersion = "1.62.1" + + module("api_proxy") { moduleConfig -> + moduleConfig.path = file("../../../core-getter/src/main/rust/api_proxy") + moduleConfig.targets = ["x86_64", "arm", "arm64"] + + moduleConfig.buildType("debug") { + it.profile = "dev" + } + + moduleConfig.buildType("release") { + it.profile = "release" + it.runTests = true + } + } +} + +dependencies { + implementation "rustls:rustls-platform-verifier:latest.release" +} diff --git a/app_flutter/android/getter_bridge/src/main/AndroidManifest.xml b/app_flutter/android/getter_bridge/src/main/AndroidManifest.xml new file mode 100644 index 00000000..94cbbcfc --- /dev/null +++ b/app_flutter/android/getter_bridge/src/main/AndroidManifest.xml @@ -0,0 +1 @@ + diff --git a/app_flutter/android/getter_bridge/src/main/kotlin/net/xzos/upgradeall/getter/NativeLib.kt b/app_flutter/android/getter_bridge/src/main/kotlin/net/xzos/upgradeall/getter/NativeLib.kt new file mode 100644 index 00000000..7921aed0 --- /dev/null +++ b/app_flutter/android/getter_bridge/src/main/kotlin/net/xzos/upgradeall/getter/NativeLib.kt @@ -0,0 +1,26 @@ +package net.xzos.upgradeall.getter + +import android.content.Context + +class RunServerCallback(private val callback: (String) -> Unit) { + fun callback(url: String) { + callback.invoke(url) + } +} + +class NativeLib { + external fun runServer(context: Context, callback: RunServerCallback): String + external fun initializeBridge(context: Context): String + external fun previewInstalledAutogen(context: Context, requestJson: String): String + external fun applyInstalledAutogen(requestJson: String): String + + fun runServerLambda(context: Context, callback: (String) -> Unit): String { + return runServer(context, RunServerCallback(callback)) + } + + companion object { + init { + System.loadLibrary("api_proxy") + } + } +} diff --git a/app_flutter/android/settings.gradle b/app_flutter/android/settings.gradle index 4dedb24d..a79f894b 100644 --- a/app_flutter/android/settings.gradle +++ b/app_flutter/android/settings.gradle @@ -18,6 +18,8 @@ pluginManagement { plugins { id "dev.flutter.flutter-gradle-plugin" version "1.0.0" apply false + id "com.android.library" version "8.6.0" apply false + id "io.github.MatrixDev.android-rust" version "0.6.0" apply false } } @@ -27,3 +29,5 @@ plugins { } include ":app" +include ":getter_bridge" +project(":getter_bridge").projectDir = file("getter_bridge") diff --git a/app_flutter/lib/cli_getter_adapter.dart b/app_flutter/lib/cli_getter_adapter.dart index d6c75c0e..5aa6ddae 100644 --- a/app_flutter/lib/cli_getter_adapter.dart +++ b/app_flutter/lib/cli_getter_adapter.dart @@ -17,6 +17,9 @@ class CliGetterAdapter implements GetterAdapter { @override bool get supportsLegacyRoomImport => true; + @override + bool get supportsInstalledAutogen => false; + @override void initialize() { _runGetter(const ['init']); @@ -88,6 +91,31 @@ class CliGetterAdapter implements GetterAdapter { return _taskEventPageFromJson(_data(json)); } + @override + Future previewInstalledAutogen({ + InstalledAutogenScanOptions options = const InstalledAutogenScanOptions(), + }) async { + throw const GetterBridgeException( + GetterError( + code: 'bridge.unsupported', + message: 'CLI adapter cannot scan Android installed inventory', + ), + ); + } + + @override + Future applyInstalledAutogen( + InstalledAutogenPreview preview, { + List? acceptedPackageIds, + }) async { + throw const GetterBridgeException( + GetterError( + code: 'bridge.unsupported', + message: 'CLI adapter cannot apply Android installed autogen previews', + ), + ); + } + @override GetterSnapshot loadSnapshot() { initialize(); diff --git a/app_flutter/lib/getter_adapter.dart b/app_flutter/lib/getter_adapter.dart index 8d1a47fc..7b9a7855 100644 --- a/app_flutter/lib/getter_adapter.dart +++ b/app_flutter/lib/getter_adapter.dart @@ -6,6 +6,8 @@ abstract interface class GetterAdapter { bool get supportsLegacyRoomImport; + bool get supportsInstalledAutogen; + void initialize(); List listRepositories(); @@ -22,6 +24,15 @@ abstract interface class GetterAdapter { TaskEventPage listTaskEvents({required int after, required int limit}); + Future previewInstalledAutogen({ + InstalledAutogenScanOptions options = const InstalledAutogenScanOptions(), + }); + + Future applyInstalledAutogen( + InstalledAutogenPreview preview, { + List? acceptedPackageIds, + }); + GetterSnapshot loadSnapshot(); } @@ -50,6 +61,9 @@ class FakeGetterAdapter implements GetterAdapter { @override bool get supportsLegacyRoomImport => false; + @override + bool get supportsInstalledAutogen => true; + @override void initialize() {} @@ -173,6 +187,74 @@ class FakeGetterAdapter implements GetterAdapter { ); } + @override + Future previewInstalledAutogen({ + InstalledAutogenScanOptions options = const InstalledAutogenScanOptions(), + }) async { + return InstalledAutogenPreview.fromJson(const { + 'operation': 'installed.preview', + 'target_repo_id': 'local_autogen', + 'target_repo_path': '/fake/getter/repositories/local_autogen', + 'scan': { + 'stats': { + 'total_seen': 3, + 'returned': 1, + 'filtered_system': 1, + 'filtered_self': 1, + }, + 'diagnostics': [], + }, + 'summary': { + 'candidate_count': 1, + 'skipped_count': 1, + 'write_count': 1, + 'delete_count': 0, + }, + 'candidates': [ + { + 'package_id': 'android/com.example.autogen', + 'kind': 'android', + 'display_name': 'Example Autogen', + 'installed_target': { + 'kind': 'android_package', + 'package_name': 'com.example.autogen', + }, + 'action': 'create', + 'output_relative_path': 'packages/android/com.example.autogen.lua', + 'content_hash': 'fnv1a64:fake', + 'content': '-- fake generated content', + }, + ], + 'skipped': [ + { + 'package_id': 'android/org.fdroid.fdroid', + 'reason': 'covered_by_higher_priority_repo', + 'covering_repo_id': 'official', + }, + ], + 'diagnostics': [], + }); + } + + @override + Future applyInstalledAutogen( + InstalledAutogenPreview preview, { + List? acceptedPackageIds, + }) async { + return InstalledAutogenApplyResult.fromJson(const { + 'target_repo_id': 'local_autogen', + 'target_repo_path': '/fake/getter/repositories/local_autogen', + 'applied_count': 1, + 'applied': [ + { + 'package_id': 'android/com.example.autogen', + 'output_relative_path': 'packages/android/com.example.autogen.lua', + }, + ], + 'preserved_to_local': [], + }); + } + @override GetterSnapshot loadSnapshot() => _snapshot; } @@ -351,6 +433,305 @@ class TaskEventSummary { final String? message; } +class InstalledAutogenScanOptions { + const InstalledAutogenScanOptions({ + this.includeSystemApps = false, + this.includeSelf = false, + }); + + final bool includeSystemApps; + final bool includeSelf; + + Map toJson() => { + 'include_system_apps': includeSystemApps, + 'include_self': includeSelf, + }; +} + +class InstalledAutogenPreview { + InstalledAutogenPreview({ + required this.operation, + required this.targetRepoId, + required this.targetRepoPath, + required this.summary, + required this.candidates, + required this.skipped, + required this.diagnostics, + required this.scanStats, + required this.rawJson, + }); + + factory InstalledAutogenPreview.fromJson(Map json) { + final scan = _jsonMapOrNull(json['scan'], 'autogen.scan'); + return InstalledAutogenPreview( + operation: _jsonString(json['operation'], 'autogen.operation'), + targetRepoId: + _jsonString(json['target_repo_id'], 'autogen.target_repo_id'), + targetRepoPath: _jsonOptionalString( + json['target_repo_path'], + 'autogen.target_repo_path', + ), + summary: AutogenSummary.fromJson( + _jsonMap(json['summary'], 'autogen.summary'), + ), + candidates: _jsonList(json['candidates'], 'autogen.candidates') + .map((candidate) => InstalledAutogenCandidate.fromJson( + _jsonMap(candidate, 'autogen.candidate'), + )) + .toList(growable: false), + skipped: _jsonList(json['skipped'], 'autogen.skipped') + .map((skip) => InstalledAutogenSkip.fromJson( + _jsonMap(skip, 'autogen.skip'), + )) + .toList(growable: false), + diagnostics: _jsonList( + scan?['diagnostics'] ?? json['diagnostics'], + 'autogen.diagnostics', + ) + .map((diagnostic) => PlatformDiagnosticSummary.fromJson( + _jsonMap(diagnostic, 'autogen.diagnostic'), + )) + .toList(growable: false), + scanStats: scan == null || scan['stats'] == null + ? null + : InstalledAutogenScanStats.fromJson( + _jsonMap(scan['stats'], 'autogen.scan.stats'), + ), + rawJson: Map.unmodifiable(json), + ); + } + + final String operation; + final String targetRepoId; + final String? targetRepoPath; + final AutogenSummary summary; + final List candidates; + final List skipped; + final List diagnostics; + final InstalledAutogenScanStats? scanStats; + final Map rawJson; +} + +class AutogenSummary { + const AutogenSummary({ + required this.candidateCount, + required this.skippedCount, + required this.writeCount, + required this.deleteCount, + }); + + factory AutogenSummary.fromJson(Map json) { + return AutogenSummary( + candidateCount: + _jsonInt(json['candidate_count'], 'autogen.summary.candidate_count'), + skippedCount: + _jsonInt(json['skipped_count'], 'autogen.summary.skipped_count'), + writeCount: _jsonInt(json['write_count'], 'autogen.summary.write_count'), + deleteCount: + _jsonInt(json['delete_count'], 'autogen.summary.delete_count'), + ); + } + + final int candidateCount; + final int skippedCount; + final int writeCount; + final int deleteCount; +} + +class InstalledAutogenCandidate { + const InstalledAutogenCandidate({ + required this.packageId, + required this.kind, + required this.displayName, + required this.action, + required this.outputRelativePath, + required this.contentHash, + required this.installedTarget, + }); + + factory InstalledAutogenCandidate.fromJson(Map json) { + return InstalledAutogenCandidate( + packageId: + _jsonString(json['package_id'], 'autogen.candidate.package_id'), + kind: _jsonString(json['kind'], 'autogen.candidate.kind'), + displayName: + _jsonString(json['display_name'], 'autogen.candidate.display_name'), + action: _jsonString(json['action'], 'autogen.candidate.action'), + outputRelativePath: _jsonString( + json['output_relative_path'], + 'autogen.candidate.output_relative_path', + ), + contentHash: + _jsonString(json['content_hash'], 'autogen.candidate.content_hash'), + installedTarget: _jsonMap( + json['installed_target'], + 'autogen.candidate.installed_target', + ), + ); + } + + final String packageId; + final String kind; + final String displayName; + final String action; + final String outputRelativePath; + final String contentHash; + final Map installedTarget; +} + +class InstalledAutogenSkip { + const InstalledAutogenSkip({ + required this.packageId, + required this.reason, + required this.coveringRepoId, + }); + + factory InstalledAutogenSkip.fromJson(Map json) { + return InstalledAutogenSkip( + packageId: _jsonString(json['package_id'], 'autogen.skip.package_id'), + reason: _jsonString(json['reason'], 'autogen.skip.reason'), + coveringRepoId: _jsonOptionalString( + json['covering_repo_id'], + 'autogen.skip.covering_repo_id', + ), + ); + } + + final String packageId; + final String reason; + final String? coveringRepoId; +} + +class InstalledAutogenScanStats { + const InstalledAutogenScanStats({ + required this.totalSeen, + required this.returned, + required this.filteredSystem, + required this.filteredSelf, + }); + + factory InstalledAutogenScanStats.fromJson(Map json) { + return InstalledAutogenScanStats( + totalSeen: _jsonInt(json['total_seen'], 'autogen.scan.total_seen'), + returned: _jsonInt(json['returned'], 'autogen.scan.returned'), + filteredSystem: + _jsonInt(json['filtered_system'], 'autogen.scan.filtered_system'), + filteredSelf: + _jsonInt(json['filtered_self'], 'autogen.scan.filtered_self'), + ); + } + + final int totalSeen; + final int returned; + final int filteredSystem; + final int filteredSelf; +} + +class PlatformDiagnosticSummary { + const PlatformDiagnosticSummary({ + required this.code, + required this.message, + required this.detail, + }); + + factory PlatformDiagnosticSummary.fromJson(Map json) { + return PlatformDiagnosticSummary( + code: _jsonString(json['code'], 'autogen.diagnostic.code'), + message: _jsonString(json['message'], 'autogen.diagnostic.message'), + detail: _jsonOptionalString(json['detail'], 'autogen.diagnostic.detail'), + ); + } + + final String code; + final String message; + final String? detail; +} + +class InstalledAutogenApplyResult { + InstalledAutogenApplyResult({ + required this.targetRepoId, + required this.targetRepoPath, + required this.appliedCount, + required this.applied, + required this.preservedToLocal, + }); + + factory InstalledAutogenApplyResult.fromJson(Map json) { + return InstalledAutogenApplyResult( + targetRepoId: + _jsonString(json['target_repo_id'], 'autogen.apply.target_repo_id'), + targetRepoPath: _jsonOptionalString( + json['target_repo_path'], + 'autogen.apply.target_repo_path', + ), + appliedCount: + _jsonInt(json['applied_count'], 'autogen.apply.applied_count'), + applied: _jsonList(json['applied'], 'autogen.apply.applied') + .map((applied) => InstalledAutogenAppliedPackage.fromJson( + _jsonMap(applied, 'autogen.apply.applied_item'), + )) + .toList(growable: false), + preservedToLocal: _jsonList( + json['preserved_to_local'], + 'autogen.apply.preserved_to_local', + ) + .map((preserved) => InstalledAutogenPreservedPackage.fromJson( + _jsonMap(preserved, 'autogen.apply.preserved_item'), + )) + .toList(growable: false), + ); + } + + final String targetRepoId; + final String? targetRepoPath; + final int appliedCount; + final List applied; + final List preservedToLocal; +} + +class InstalledAutogenAppliedPackage { + const InstalledAutogenAppliedPackage({ + required this.packageId, + required this.outputRelativePath, + }); + + factory InstalledAutogenAppliedPackage.fromJson(Map json) { + return InstalledAutogenAppliedPackage( + packageId: _jsonString(json['package_id'], 'autogen.apply.package_id'), + outputRelativePath: _jsonString( + json['output_relative_path'], + 'autogen.apply.output_relative_path', + ), + ); + } + + final String packageId; + final String outputRelativePath; +} + +class InstalledAutogenPreservedPackage { + const InstalledAutogenPreservedPackage({ + required this.packageId, + required this.repositoryId, + required this.relativePath, + }); + + factory InstalledAutogenPreservedPackage.fromJson(Map json) { + return InstalledAutogenPreservedPackage( + packageId: + _jsonString(json['package_id'], 'autogen.preserved.package_id'), + repositoryId: + _jsonString(json['repository_id'], 'autogen.preserved.repository_id'), + relativePath: + _jsonString(json['relative_path'], 'autogen.preserved.relative_path'), + ); + } + + final String packageId; + final String repositoryId; + final String relativePath; +} + class GetterError { const GetterError({required this.code, required this.message, this.detail}); @@ -372,3 +753,35 @@ class GetterBridgeException implements Exception { return 'GetterBridgeException$exit: ${error.code}: ${error.message}$detail'; } } + +Map _jsonMap(Object? value, String name) { + if (value is Map) return value; + if (value is Map) return value.cast(); + throw FormatException('$name should be a JSON object'); +} + +Map? _jsonMapOrNull(Object? value, String name) { + if (value == null) return null; + return _jsonMap(value, name); +} + +List _jsonList(Object? value, String name) { + if (value is List) return value; + if (value is List) return value.cast(); + throw FormatException('$name should be a JSON array'); +} + +String _jsonString(Object? value, String name) { + if (value is String) return value; + throw FormatException('$name should be a string'); +} + +String? _jsonOptionalString(Object? value, String name) { + if (value == null || value is String) return value as String?; + throw FormatException('$name should be a string or null'); +} + +int _jsonInt(Object? value, String name) { + if (value is int) return value; + throw FormatException('$name should be an integer'); +} diff --git a/app_flutter/lib/main.dart b/app_flutter/lib/main.dart index 9d3dced6..d7666a7d 100644 --- a/app_flutter/lib/main.dart +++ b/app_flutter/lib/main.dart @@ -2,10 +2,12 @@ import 'package:flutter/material.dart'; import 'getter_adapter.dart'; import 'legacy_migration_platform.dart'; +import 'native_getter_adapter.dart'; void main() { runApp( const UpgradeAllApp( + getter: MethodChannelGetterAdapter(), legacyMigrationPlatform: MethodChannelLegacyMigrationPlatform(), ), ); @@ -21,6 +23,8 @@ class AppKeys { static const logsRoute = ValueKey('route.logs'); static const settingsRoute = ValueKey('route.settings'); static const migrationRoute = ValueKey('route.migration'); + static const installedAutogenRoute = + ValueKey('route.installed_autogen'); static const openApps = ValueKey('action.open_apps'); static const openRepositories = ValueKey('action.open_repositories'); @@ -28,9 +32,15 @@ class AppKeys { static const openLogs = ValueKey('action.open_logs'); static const openSettings = ValueKey('action.open_settings'); static const openMigration = ValueKey('action.open_migration'); + static const openInstalledAutogen = + ValueKey('action.open_installed_autogen'); static const openFirstApp = ValueKey('action.open_first_app'); static const startLegacyMigration = ValueKey('action.start_legacy_migration'); + static const previewInstalledAutogen = + ValueKey('action.preview_installed_autogen'); + static const applyInstalledAutogen = + ValueKey('action.apply_installed_autogen'); static const updateSummary = ValueKey('state.update_summary'); static const getterStatus = ValueKey('state.getter_status'); @@ -49,6 +59,24 @@ class AppKeys { static const migrationError = ValueKey('state.migration_error'); static const migrationReportsList = ValueKey('state.migration_reports_list'); + static const installedAutogenReady = + ValueKey('state.installed_autogen_ready'); + static const installedAutogenBridgeUnavailable = + ValueKey('state.installed_autogen_bridge_unavailable'); + static const installedAutogenPreview = + ValueKey('state.installed_autogen_preview'); + static const installedAutogenCandidatesList = + ValueKey('state.installed_autogen_candidates_list'); + static const installedAutogenSkipsList = + ValueKey('state.installed_autogen_skips_list'); + static const installedAutogenDiagnosticsList = + ValueKey('state.installed_autogen_diagnostics_list'); + static const installedAutogenScanStats = + ValueKey('state.installed_autogen_scan_stats'); + static const installedAutogenApplied = + ValueKey('state.installed_autogen_applied'); + static const installedAutogenError = + ValueKey('state.installed_autogen_error'); static ValueKey appRow(String packageId) => ValueKey('state.app.$packageId'); @@ -58,6 +86,14 @@ class AppKeys { ValueKey('state.download_task.$taskId'); static ValueKey taskEventRow(int cursor) => ValueKey('state.task_event.$cursor'); + static ValueKey autogenCandidateRow(String packageId) => + ValueKey('state.autogen_candidate.$packageId'); + static ValueKey autogenSkipRow(String packageId) => + ValueKey('state.autogen_skip.$packageId'); + static ValueKey autogenDiagnosticRow(int index) => + ValueKey('state.autogen_diagnostic.$index'); + static ValueKey autogenAppliedRow(String packageId) => + ValueKey('state.autogen_applied.$packageId'); } class UpgradeAllApp extends StatelessWidget { @@ -89,6 +125,7 @@ class UpgradeAllApp extends StatelessWidget { getter: getter, legacyMigrationPlatform: legacyMigrationPlatform, ), + '/autogen': (context) => InstalledAutogenPage(getter: getter), }, onGenerateRoute: (settings) { if (settings.name == '/apps/detail') { @@ -169,6 +206,12 @@ class HomePage extends StatelessWidget { label: 'Legacy migration', routeName: '/migration', ), + const _RouteButton( + key: AppKeys.openInstalledAutogen, + icon: Icons.auto_fix_high, + label: 'Installed autogen', + routeName: '/autogen', + ), ], ), ); @@ -336,6 +379,240 @@ class DownloadsPage extends StatelessWidget { } } +class InstalledAutogenPage extends StatefulWidget { + const InstalledAutogenPage({super.key, required this.getter}); + + final GetterAdapter getter; + + @override + State createState() => _InstalledAutogenPageState(); +} + +class _InstalledAutogenPageState extends State { + InstalledAutogenPreview? _preview; + InstalledAutogenApplyResult? _applyResult; + GetterError? _error; + bool _running = false; + + Future _previewInstalledAutogen() async { + setState(() { + _running = true; + _error = null; + _applyResult = null; + }); + try { + final preview = await widget.getter.previewInstalledAutogen(); + if (!mounted) return; + setState(() { + _preview = preview; + _running = false; + }); + } on GetterBridgeException catch (error) { + if (!mounted) return; + setState(() { + _error = error.error; + _running = false; + }); + } catch (error) { + if (!mounted) return; + setState(() { + _error = GetterError( + code: 'bridge.installed_autogen_error', + message: 'Installed autogen bridge failed', + detail: error.toString(), + ); + _running = false; + }); + } + } + + Future _applyInstalledAutogen() async { + final preview = _preview; + if (preview == null) return; + setState(() { + _running = true; + _error = null; + }); + try { + final result = await widget.getter.applyInstalledAutogen( + preview, + acceptedPackageIds: preview.candidates + .map((candidate) => candidate.packageId) + .toList(growable: false), + ); + if (!mounted) return; + setState(() { + _applyResult = result; + _running = false; + }); + } on GetterBridgeException catch (error) { + if (!mounted) return; + setState(() { + _error = error.error; + _running = false; + }); + } catch (error) { + if (!mounted) return; + setState(() { + _error = GetterError( + code: 'bridge.installed_autogen_error', + message: 'Installed autogen bridge failed', + detail: error.toString(), + ); + _running = false; + }); + } + } + + @override + Widget build(BuildContext context) { + final preview = _preview; + final applyResult = _applyResult; + final canUseBridge = widget.getter.supportsInstalledAutogen; + return Scaffold( + key: AppKeys.installedAutogenRoute, + appBar: AppBar(title: const Text('Installed autogen')), + body: ListView( + padding: const EdgeInsets.all(16), + children: [ + ElevatedButton.icon( + key: AppKeys.previewInstalledAutogen, + onPressed: + _running || !canUseBridge ? null : _previewInstalledAutogen, + icon: const Icon(Icons.manage_search), + label: Text(_running ? 'Working…' : 'Preview installed autogen'), + ), + if (!canUseBridge) + const Padding( + padding: EdgeInsets.only(top: 12), + child: Text( + key: AppKeys.installedAutogenBridgeUnavailable, + 'Getter installed-autogen bridge is not connected', + ), + ), + if (preview == null && _error == null) + const Padding( + padding: EdgeInsets.only(top: 16), + child: Text( + key: AppKeys.installedAutogenReady, + 'Ready to preview installed app fallback packages', + ), + ), + if (_error != null) + Padding( + padding: const EdgeInsets.only(top: 16), + child: Text( + key: AppKeys.installedAutogenError, + '${_error!.code}: ${_error!.message}', + ), + ), + if (preview != null) ...[ + const SizedBox(height: 16), + Text( + key: AppKeys.installedAutogenPreview, + '${preview.summary.candidateCount} candidates, ${preview.summary.skippedCount} skipped', + ), + if (preview.scanStats != null) + Padding( + padding: const EdgeInsets.only(top: 8), + child: Text( + key: AppKeys.installedAutogenScanStats, + 'Seen ${preview.scanStats!.totalSeen}, returned ${preview.scanStats!.returned}, filtered system ${preview.scanStats!.filteredSystem}, filtered self ${preview.scanStats!.filteredSelf}', + ), + ), + const SizedBox(height: 16), + Text('Candidates', style: Theme.of(context).textTheme.titleMedium), + ListView.builder( + key: AppKeys.installedAutogenCandidatesList, + shrinkWrap: true, + physics: const NeverScrollableScrollPhysics(), + itemCount: preview.candidates.length, + itemBuilder: (context, index) { + final candidate = preview.candidates[index]; + return ListTile( + key: AppKeys.autogenCandidateRow(candidate.packageId), + title: Text(candidate.displayName), + subtitle: Text( + '${candidate.packageId} • ${candidate.outputRelativePath}', + ), + ); + }, + ), + if (preview.skipped.isNotEmpty) ...[ + const SizedBox(height: 16), + Text('Skipped', style: Theme.of(context).textTheme.titleMedium), + ListView.builder( + key: AppKeys.installedAutogenSkipsList, + shrinkWrap: true, + physics: const NeverScrollableScrollPhysics(), + itemCount: preview.skipped.length, + itemBuilder: (context, index) { + final skipped = preview.skipped[index]; + return ListTile( + key: AppKeys.autogenSkipRow(skipped.packageId), + title: Text(skipped.packageId), + subtitle: Text( + '${skipped.reason}${skipped.coveringRepoId == null ? '' : ' • ${skipped.coveringRepoId}'}', + ), + ); + }, + ), + ], + if (preview.diagnostics.isNotEmpty) ...[ + const SizedBox(height: 16), + Text('Diagnostics', + style: Theme.of(context).textTheme.titleMedium), + ListView.builder( + key: AppKeys.installedAutogenDiagnosticsList, + shrinkWrap: true, + physics: const NeverScrollableScrollPhysics(), + itemCount: preview.diagnostics.length, + itemBuilder: (context, index) { + final diagnostic = preview.diagnostics[index]; + return ListTile( + key: AppKeys.autogenDiagnosticRow(index), + title: Text(diagnostic.code), + subtitle: Text(diagnostic.message), + ); + }, + ), + ], + const SizedBox(height: 16), + ElevatedButton.icon( + key: AppKeys.applyInstalledAutogen, + onPressed: _running || preview.candidates.isEmpty + ? null + : _applyInstalledAutogen, + icon: const Icon(Icons.check), + label: const Text('Apply all candidates'), + ), + ], + if (applyResult != null) ...[ + const SizedBox(height: 16), + Text( + key: AppKeys.installedAutogenApplied, + 'Applied ${applyResult.appliedCount} packages', + ), + ListView.builder( + shrinkWrap: true, + physics: const NeverScrollableScrollPhysics(), + itemCount: applyResult.applied.length, + itemBuilder: (context, index) { + final applied = applyResult.applied[index]; + return ListTile( + key: AppKeys.autogenAppliedRow(applied.packageId), + title: Text(applied.packageId), + subtitle: Text(applied.outputRelativePath), + ); + }, + ), + ], + ], + ), + ); + } +} + class LogsPage extends StatelessWidget { const LogsPage({super.key}); diff --git a/app_flutter/lib/native_getter_adapter.dart b/app_flutter/lib/native_getter_adapter.dart new file mode 100644 index 00000000..558cf8d1 --- /dev/null +++ b/app_flutter/lib/native_getter_adapter.dart @@ -0,0 +1,111 @@ +import 'dart:convert'; + +import 'package:flutter/services.dart'; + +import 'getter_adapter.dart'; + +/// First Android production bridge slice. +/// +/// Until the full native getter bridge replaces every CLI/fake surface, this +/// adapter inherits the deterministic shell data from [FakeGetterAdapter] and +/// overrides only installed-autogen operations with the Rust/native bridge. +/// The bridge returns getter-owned JSON envelopes; Dart parses and renders them +/// but does not scan PackageManager or make autogen/package decisions. +class MethodChannelGetterAdapter extends FakeGetterAdapter { + const MethodChannelGetterAdapter({ + MethodChannel channel = const MethodChannel( + 'net.xzos.upgradeall/getter_bridge', + ), + }) : _channel = channel; + + final MethodChannel _channel; + + @override + bool get supportsInstalledAutogen => true; + + @override + void initialize() { + // The installed-autogen bridge initializes lazily when preview is called. + } + + @override + Future previewInstalledAutogen({ + InstalledAutogenScanOptions options = const InstalledAutogenScanOptions(), + }) async { + final data = await _invokeGetterData( + 'previewInstalledAutogen', + {'scan_options': options.toJson()}, + ); + return InstalledAutogenPreview.fromJson(data); + } + + @override + Future applyInstalledAutogen( + InstalledAutogenPreview preview, { + List? acceptedPackageIds, + }) async { + final data = await _invokeGetterData( + 'applyInstalledAutogen', + { + 'preview_json': jsonEncode(preview.rawJson), + 'acceptance': acceptedPackageIds == null + ? const {'mode': 'all'} + : { + 'mode': 'packages', + 'package_ids': acceptedPackageIds, + }, + }, + ); + return InstalledAutogenApplyResult.fromJson(data); + } + + Future> _invokeGetterData( + String method, + Map arguments, + ) async { + try { + final response = await _channel.invokeMethod(method, arguments); + if (response == null || response.isEmpty) { + throw const GetterBridgeException( + GetterError( + code: 'bridge.empty_response', + message: 'Getter native bridge returned an empty response', + ), + ); + } + final envelope = _asMap(jsonDecode(response), 'getter bridge response'); + if (envelope['ok'] != true) { + throw GetterBridgeException(_errorFromEnvelope(envelope)); + } + return _asMap(envelope['data'], 'getter bridge data'); + } on PlatformException catch (error) { + throw GetterBridgeException( + GetterError( + code: error.code, + message: error.message ?? 'Getter native bridge call failed', + detail: error.details?.toString(), + ), + ); + } + } +} + +GetterError _errorFromEnvelope(Map envelope) { + final error = _asMap(envelope['error'], 'getter bridge error'); + return GetterError( + code: _asString(error['code'], 'getter bridge error.code'), + message: _asString(error['message'], 'getter bridge error.message'), + detail: error['detail']?.toString(), + ); +} + +Map _asMap(Object? value, String name) { + if (value is Map) return value; + if (value is Map) return value.cast(); + throw FormatException('$name should be a JSON object'); +} + +String _asString(Object? value, String name) { + if (value is String) return value; + throw FormatException('$name should be a string'); +} diff --git a/app_flutter/test/native_getter_adapter_test.dart b/app_flutter/test/native_getter_adapter_test.dart new file mode 100644 index 00000000..f8305f27 --- /dev/null +++ b/app_flutter/test/native_getter_adapter_test.dart @@ -0,0 +1,161 @@ +import 'dart:convert'; + +import 'package:flutter/services.dart'; +import 'package:flutter_test/flutter_test.dart'; +import 'package:upgradeall/getter_adapter.dart'; +import 'package:upgradeall/native_getter_adapter.dart'; + +void main() { + TestWidgetsFlutterBinding.ensureInitialized(); + + const channel = MethodChannel('test/getter_bridge'); + + tearDown(() { + TestDefaultBinaryMessengerBinding.instance.defaultBinaryMessenger + .setMockMethodCallHandler(channel, null); + }); + + test('native preview sends scan options and parses getter envelope', + () async { + MethodCall? captured; + TestDefaultBinaryMessengerBinding.instance.defaultBinaryMessenger + .setMockMethodCallHandler(channel, (call) async { + captured = call; + return jsonEncode({ + 'ok': true, + 'command': 'autogen installed preview', + 'data': _previewJson(), + 'warnings': [], + }); + }); + + const adapter = MethodChannelGetterAdapter(channel: channel); + final preview = await adapter.previewInstalledAutogen( + options: const InstalledAutogenScanOptions( + includeSystemApps: true, + includeSelf: true, + ), + ); + + expect(captured!.method, 'previewInstalledAutogen'); + expect(captured!.arguments, { + 'scan_options': { + 'include_system_apps': true, + 'include_self': true, + }, + }); + expect(preview.summary.candidateCount, 1); + expect(preview.scanStats!.returned, 1); + expect(preview.candidates.single.packageId, 'android/com.example.autogen'); + }); + + test('native apply forwards preview JSON and package acceptance', () async { + MethodCall? captured; + TestDefaultBinaryMessengerBinding.instance.defaultBinaryMessenger + .setMockMethodCallHandler(channel, (call) async { + captured = call; + return jsonEncode({ + 'ok': true, + 'command': 'autogen installed apply', + 'data': { + 'target_repo_id': 'local_autogen', + 'target_repo_path': '/getter/repositories/local_autogen', + 'applied_count': 1, + 'applied': [ + { + 'package_id': 'android/com.example.autogen', + 'output_relative_path': + 'packages/android/com.example.autogen.lua', + }, + ], + 'preserved_to_local': [], + }, + 'warnings': [], + }); + }); + + const adapter = MethodChannelGetterAdapter(channel: channel); + final preview = InstalledAutogenPreview.fromJson(_previewJson()); + final result = await adapter.applyInstalledAutogen( + preview, + acceptedPackageIds: const ['android/com.example.autogen'], + ); + + expect(captured!.method, 'applyInstalledAutogen'); + final args = + (captured!.arguments as Map).cast(); + expect(jsonDecode(args['preview_json']! as String), preview.rawJson); + expect(args['acceptance'], { + 'mode': 'packages', + 'package_ids': ['android/com.example.autogen'], + }); + expect(result.applied.single.packageId, 'android/com.example.autogen'); + }); + + test('native adapter maps getter error envelope to bridge exception', + () async { + TestDefaultBinaryMessengerBinding.instance.defaultBinaryMessenger + .setMockMethodCallHandler(channel, (call) async { + return jsonEncode({ + 'ok': false, + 'command': call.method, + 'error': { + 'code': 'autogen.preview_error', + 'message': 'Preview failed', + 'detail': 'bad inventory', + }, + }); + }); + + const adapter = MethodChannelGetterAdapter(channel: channel); + + await expectLater( + adapter.previewInstalledAutogen(), + throwsA( + isA().having( + (error) => error.error.code, + 'code', + 'autogen.preview_error', + ), + ), + ); + }); +} + +Map _previewJson() => { + 'operation': 'installed.preview', + 'target_repo_id': 'local_autogen', + 'target_repo_path': '/getter/repositories/local_autogen', + 'scan': { + 'stats': { + 'total_seen': 2, + 'returned': 1, + 'filtered_system': 1, + 'filtered_self': 0, + }, + 'diagnostics': [], + }, + 'summary': { + 'candidate_count': 1, + 'skipped_count': 0, + 'write_count': 1, + 'delete_count': 0, + }, + 'candidates': [ + { + 'package_id': 'android/com.example.autogen', + 'kind': 'android', + 'display_name': 'Example Autogen', + 'installed_target': { + 'kind': 'android_package', + 'package_name': 'com.example.autogen', + }, + 'action': 'create', + 'output_relative_path': 'packages/android/com.example.autogen.lua', + 'content_hash': 'fnv1a64:fake', + 'content': '-- fake generated content', + }, + ], + 'skipped': [], + 'diagnostics': [], + }; diff --git a/app_flutter/test/widget_test.dart b/app_flutter/test/widget_test.dart index cc1d6c87..83b9716f 100644 --- a/app_flutter/test/widget_test.dart +++ b/app_flutter/test/widget_test.dart @@ -125,6 +125,64 @@ void main() { expect(find.byKey(AppKeys.migrationImported), findsNothing); }); + testWidgets('installed autogen route previews and applies getter DTOs', + (tester) async { + final getter = _AutogenRecordingGetterAdapter(); + await tester.pumpWidget(UpgradeAllApp(getter: getter)); + + await tester.drag(find.byType(ListView).first, const Offset(0, -240)); + await tester.pumpAndSettle(); + await tester.tap(find.byKey(AppKeys.openInstalledAutogen)); + await tester.pumpAndSettle(); + expect(find.byKey(AppKeys.installedAutogenRoute), findsOneWidget); + expect(find.byKey(AppKeys.installedAutogenReady), findsOneWidget); + + await tester.tap(find.byKey(AppKeys.previewInstalledAutogen)); + await tester.pumpAndSettle(); + + expect(find.byKey(AppKeys.installedAutogenPreview), findsOneWidget); + expect(find.byKey(AppKeys.installedAutogenScanStats), findsOneWidget); + expect( + find.byKey(AppKeys.autogenCandidateRow('android/com.example.autogen')), + findsOneWidget, + ); + expect( + find.byKey(AppKeys.autogenSkipRow('android/org.fdroid.fdroid')), + findsOneWidget, + ); + + await tester.tap(find.byKey(AppKeys.applyInstalledAutogen)); + await tester.pumpAndSettle(); + + expect(find.byKey(AppKeys.installedAutogenApplied), findsOneWidget); + expect( + find.byKey(AppKeys.autogenAppliedRow('android/com.example.autogen')), + findsOneWidget, + ); + expect(getter.acceptedPackageIds, ['android/com.example.autogen']); + }); + + testWidgets('installed autogen route disables actions without bridge', + (tester) async { + await tester.pumpWidget( + const UpgradeAllApp(getter: _NoInstalledAutogenGetterAdapter()), + ); + + await tester.drag(find.byType(ListView).first, const Offset(0, -240)); + await tester.pumpAndSettle(); + await tester.tap(find.byKey(AppKeys.openInstalledAutogen)); + await tester.pumpAndSettle(); + + final button = tester.widget( + find.byKey(AppKeys.previewInstalledAutogen), + ); + expect(button.onPressed, isNull); + expect( + find.byKey(AppKeys.installedAutogenBridgeUnavailable), + findsOneWidget, + ); + }); + testWidgets('migration route disables import when getter bridge is absent', (tester) async { await tester.pumpWidget( @@ -194,6 +252,29 @@ class _LegacyMigrationCapableGetterAdapter extends FakeGetterAdapter { bool get supportsLegacyRoomImport => true; } +class _NoInstalledAutogenGetterAdapter extends FakeGetterAdapter { + const _NoInstalledAutogenGetterAdapter(); + + @override + bool get supportsInstalledAutogen => false; +} + +class _AutogenRecordingGetterAdapter extends FakeGetterAdapter { + List? acceptedPackageIds; + + @override + Future applyInstalledAutogen( + InstalledAutogenPreview preview, { + List? acceptedPackageIds, + }) { + this.acceptedPackageIds = acceptedPackageIds; + return super.applyInstalledAutogen( + preview, + acceptedPackageIds: acceptedPackageIds, + ); + } +} + class _MigrationGetterAdapter extends FakeGetterAdapter { String? importedDatabasePath; @override diff --git a/core-getter/src/main/java/net/xzos/upgradeall/getter/NativeLib.kt b/core-getter/src/main/java/net/xzos/upgradeall/getter/NativeLib.kt index 8223222a..adc71ceb 100644 --- a/core-getter/src/main/java/net/xzos/upgradeall/getter/NativeLib.kt +++ b/core-getter/src/main/java/net/xzos/upgradeall/getter/NativeLib.kt @@ -14,7 +14,11 @@ class NativeLib { * A native method that is implemented by the 'getter' native library, * which is packaged with this application. */ - external fun runServer(context:Context, callback: RunServerCallback): String + external fun runServer(context: Context, callback: RunServerCallback): String + external fun initializeBridge(context: Context): String + external fun previewInstalledAutogen(context: Context, requestJson: String): String + external fun applyInstalledAutogen(requestJson: String): String + fun runServerLambda(context: Context, callback: (String) -> Unit): String { return runServer(context, RunServerCallback(callback)) } diff --git a/core-getter/src/main/rust/api_proxy/Cargo.toml b/core-getter/src/main/rust/api_proxy/Cargo.toml index 95d3c08c..46a63539 100644 --- a/core-getter/src/main/rust/api_proxy/Cargo.toml +++ b/core-getter/src/main/rust/api_proxy/Cargo.toml @@ -7,10 +7,11 @@ edition = "2021" [dependencies] jni = "0.21" # from rustls-platform-verifier-android, sync version -getter = { path = "../getter", default-features = false, features = ["native-tokio", "rustls-platform-verifier-android"] } +getter = { path = "../getter", default-features = false, features = ["domain", "native-tokio", "rustls-platform-verifier-android"] } upgradeall-platform-adapter = { path = "../platform_adapter" } serde = { version = "1.0", features = ["derive"] } serde_json = "1.0.145" +thiserror = "1" tokio = { version = "1.48.0", features = ["rt-multi-thread"] } [lib] diff --git a/core-getter/src/main/rust/api_proxy/src/lib.rs b/core-getter/src/main/rust/api_proxy/src/lib.rs index 3f2dc3a4..8aad40ae 100644 --- a/core-getter/src/main/rust/api_proxy/src/lib.rs +++ b/core-getter/src/main/rust/api_proxy/src/lib.rs @@ -1,57 +1,57 @@ extern crate jni; +use getter::operations::autogen::{self, AutogenAcceptance, AutogenOperationError}; use getter::rpc::server::run_server_hanging; #[cfg(target_os = "android")] use getter::rustls_platform_verifier; -use jni::objects::{JClass, JObject, JString, JValue}; +use jni::objects::{JObject, JString, JValue}; use jni::JNIEnv; +use serde::Deserialize; +use serde_json::{json, Value}; +use std::path::{Path, PathBuf}; use std::sync::mpsc::channel; use std::thread; +use upgradeall_platform_adapter::InstalledInventoryScanOptions; +#[cfg(target_os = "android")] +use upgradeall_platform_adapter::PlatformAdapter; + +const MAIN_DB_FILE: &str = "main.db"; +const CACHE_DB_FILE: &str = "cache.db"; + +#[derive(Debug, Deserialize)] +struct PreviewInstalledAutogenRequest { + data_dir: PathBuf, + #[serde(default)] + scan_options: InstalledInventoryScanOptions, +} + +#[derive(Debug, Deserialize)] +struct ApplyInstalledAutogenRequest { + data_dir: PathBuf, + preview: Value, + #[serde(default)] + acceptance: ApplyInstalledAutogenAcceptance, +} + +#[derive(Debug, Default, Deserialize)] +struct ApplyInstalledAutogenAcceptance { + #[serde(default)] + mode: Option, + #[serde(default)] + package_ids: Vec, +} #[no_mangle] pub extern "C" fn Java_net_xzos_upgradeall_getter_NativeLib_runServer<'local>( mut env: JNIEnv<'local>, - _: JClass<'local>, - _context: JObject, + _: JObject<'local>, + context: JObject<'local>, callback: JObject<'local>, ) -> JString<'local> { - // Initialize Android-hosted Rust platform integrations for future use. - // https://github.com/rustls/rustls-platform-verifier/tree/3edb4d278215a8603020351b8b519d907a26041f?tab=readme-ov-file#crate-initialization - #[cfg(target_os = "android")] - { - let rustls_context = match env.new_local_ref(&_context) { - Ok(context) => context, - Err(e) => { - return env - .new_string(format!("Error creating rustls context ref: {}", e)) - .expect("Failed to create Java string"); - } - }; - if let Err(e) = rustls_platform_verifier::android::init_hosted(&mut env, rustls_context) { - return env - .new_string(format!("Error initializing certificate verifier: {}", e)) - .expect("Failed to create Java string"); - } - - let platform_context = match env.new_local_ref(&_context) { - Ok(context) => context, - Err(e) => { - return env - .new_string(format!( - "Error creating platform adapter context ref: {}", - e - )) - .expect("Failed to create Java string"); - } - }; - if let Err(e) = - upgradeall_platform_adapter::android::init_with_env(&mut env, platform_context) - { - return env - .new_string(format!("Error initializing platform adapter: {}", e)) - .expect("Failed to create Java string"); - } + if let Err(error) = init_android_integrations(&mut env, &context) { + return java_string_or_fallback(&mut env, error); } + let (startup_tx, startup_rx) = channel::>(); thread::spawn(move || { let runtime = match tokio::runtime::Runtime::new() { @@ -83,20 +83,22 @@ pub extern "C" fn Java_net_xzos_upgradeall_getter_NativeLib_runServer<'local>( let url = match startup_rx.recv() { Ok(Ok(url)) => url, Ok(Err(error)) => { - return env.new_string(error).expect("Failed to create Java string"); + return java_string_or_fallback(&mut env, error); } Err(e) => { - return env - .new_string(format!("Error receiving URL from server thread: {}", e)) - .expect("Failed to create Java string"); + return java_string_or_fallback( + &mut env, + format!("Error receiving URL from server thread: {}", e), + ); } }; let jurl = match env.new_string(url) { Ok(jurl) => jurl, Err(e) => { - return env - .new_string(format!("Error creating URL Java string: {}", e)) - .expect("Failed to create Java string"); + return java_string_or_fallback( + &mut env, + format!("Error creating URL Java string: {e}"), + ); } }; let call_result = env.call_method( @@ -107,10 +109,345 @@ pub extern "C" fn Java_net_xzos_upgradeall_getter_NativeLib_runServer<'local>( ); if let Err(e) = call_result { - return env - .new_string(format!("JNI call error: {}", e)) - .expect("Failed to create Java string"); + return java_string_or_fallback(&mut env, format!("JNI call error: {e}")); } - env.new_string("").expect("Failed to create Java string") + java_string_or_fallback(&mut env, "") +} + +#[no_mangle] +pub extern "C" fn Java_net_xzos_upgradeall_getter_NativeLib_initializeBridge<'local>( + mut env: JNIEnv<'local>, + _: JObject<'local>, + context: JObject<'local>, +) -> JString<'local> { + let response = match init_android_integrations(&mut env, &context) { + Ok(()) => success_envelope("bridge initialize", json!({ "initialized": true })), + Err(error) => error_envelope( + "bridge initialize", + "bridge.initialize_error", + "Getter native bridge initialization failed", + Some(error), + ), + }; + java_string_or_fallback(&mut env, response) +} + +#[no_mangle] +pub extern "C" fn Java_net_xzos_upgradeall_getter_NativeLib_previewInstalledAutogen<'local>( + mut env: JNIEnv<'local>, + _: JObject<'local>, + context: JObject<'local>, + request_json: JString<'local>, +) -> JString<'local> { + let command = "autogen installed preview"; + let response = match jstring_to_string(&mut env, &request_json) + .and_then(|raw| preview_installed_autogen(&mut env, &context, &raw)) + { + Ok(data) => success_envelope(command, data), + Err(error) => operation_error_envelope(command, error), + }; + java_string_or_fallback(&mut env, response) +} + +#[no_mangle] +pub extern "C" fn Java_net_xzos_upgradeall_getter_NativeLib_applyInstalledAutogen<'local>( + mut env: JNIEnv<'local>, + _: JObject<'local>, + request_json: JString<'local>, +) -> JString<'local> { + let command = "autogen installed apply"; + let response = match jstring_to_string(&mut env, &request_json) + .and_then(|raw| apply_installed_autogen(&raw)) + { + Ok(data) => success_envelope(command, data), + Err(error) => operation_error_envelope(command, error), + }; + java_string_or_fallback(&mut env, response) +} + +fn preview_installed_autogen( + env: &mut JNIEnv<'_>, + context: &JObject<'_>, + request_json: &str, +) -> Result { + init_android_integrations(env, context).map_err(BridgeOperationError::Initialize)?; + let request: PreviewInstalledAutogenRequest = serde_json::from_str(request_json) + .map_err(|source| BridgeOperationError::InvalidRequest(source.to_string()))?; + let db = open_main_db(&request.data_dir)?; + let scan = scan_installed_inventory(request.scan_options)?; + let inventory: getter::core::autogen::InstalledInventory = + serde_json::to_value(&scan.inventory) + .and_then(serde_json::from_value) + .map_err(|source| BridgeOperationError::PlatformMalformed(source.to_string()))?; + let plan = autogen::build_local_autogen_plan(&db, &inventory)?; + let mut preview = autogen::installed_preview_json(&request.data_dir, &plan); + if let Some(object) = preview.as_object_mut() { + object.insert( + "scan".to_owned(), + json!({ + "stats": scan.stats, + "diagnostics": scan.diagnostics, + }), + ); + } + Ok(preview) +} + +fn apply_installed_autogen(request_json: &str) -> Result { + let request: ApplyInstalledAutogenRequest = serde_json::from_str(request_json) + .map_err(|source| BridgeOperationError::InvalidRequest(source.to_string()))?; + let db = open_main_db(&request.data_dir)?; + let preview = autogen::unwrap_preview_payload(request.preview, "installed.preview")?; + let acceptance = request.acceptance.into_autogen_acceptance()?; + Ok(autogen::apply_installed_preview( + &request.data_dir, + &db, + &preview, + &acceptance, + )?) +} + +impl ApplyInstalledAutogenAcceptance { + fn into_autogen_acceptance(self) -> Result { + match self.mode.as_deref().unwrap_or("all") { + "all" => Ok(AutogenAcceptance::AcceptAll), + "packages" => Ok(AutogenAcceptance::Accept(self.package_ids)), + other => Err(BridgeOperationError::InvalidRequest(format!( + "unsupported installed autogen acceptance mode '{other}'" + ))), + } + } +} + +fn open_main_db(data_dir: &Path) -> Result { + std::fs::create_dir_all(data_dir) + .map_err(|source| BridgeOperationError::Storage(source.to_string()))?; + getter::storage::CacheDb::open(data_dir.join(CACHE_DB_FILE))?; + Ok(getter::storage::MainDb::open(data_dir.join(MAIN_DB_FILE))?) +} + +fn scan_installed_inventory( + options: InstalledInventoryScanOptions, +) -> Result { + #[cfg(target_os = "android")] + { + upgradeall_platform_adapter::android::AndroidPlatformAdapter + .scan_installed_inventory(options) + .map_err(BridgeOperationError::Platform) + } + #[cfg(not(target_os = "android"))] + { + let _ = options; + Err(BridgeOperationError::Platform( + upgradeall_platform_adapter::PlatformAdapterError::Unsupported { + capability: "installed_inventory.android", + }, + )) + } +} + +fn init_android_integrations(env: &mut JNIEnv<'_>, context: &JObject<'_>) -> Result<(), String> { + // Initialize Android-hosted Rust platform integrations for future use. + // https://github.com/rustls/rustls-platform-verifier/tree/3edb4d278215a8603020351b8b519d907a26041f?tab=readme-ov-file#crate-initialization + #[cfg(target_os = "android")] + { + let rustls_context = env + .new_local_ref(context) + .map_err(|e| format!("Error creating rustls context ref: {e}"))?; + rustls_platform_verifier::android::init_hosted(env, rustls_context) + .map_err(|e| format!("Error initializing certificate verifier: {e}"))?; + + let platform_context = env + .new_local_ref(context) + .map_err(|e| format!("Error creating platform adapter context ref: {e}"))?; + upgradeall_platform_adapter::android::init_with_env(env, platform_context) + .map_err(|e| format!("Error initializing platform adapter: {e}"))?; + } + #[cfg(not(target_os = "android"))] + { + let _ = env; + let _ = context; + } + Ok(()) +} + +fn jstring_to_string( + env: &mut JNIEnv<'_>, + value: &JString<'_>, +) -> Result { + env.get_string(value) + .map(|value| value.into()) + .map_err(|source| BridgeOperationError::Jni(source.to_string())) +} + +fn java_string_or_fallback<'local>( + env: &mut JNIEnv<'local>, + value: impl AsRef, +) -> JString<'local> { + env.new_string(value.as_ref()).unwrap_or_else(|_| { + env.new_string("JNI string allocation failed") + .expect("fallback string") + }) +} + +fn success_envelope(command: &str, data: Value) -> String { + json!({ + "ok": true, + "command": command, + "data": data, + "warnings": [], + }) + .to_string() +} + +fn operation_error_envelope(command: &str, error: BridgeOperationError) -> String { + let (code, message, detail) = error.parts(); + error_envelope(command, code, message, detail) +} + +fn error_envelope(command: &str, code: &str, message: &str, detail: Option) -> String { + json!({ + "ok": false, + "command": command, + "error": { + "code": code, + "message": message, + "detail": detail, + }, + }) + .to_string() +} + +#[derive(Debug, thiserror::Error)] +enum BridgeOperationError { + #[error("invalid bridge request: {0}")] + InvalidRequest(String), + #[error("JNI error: {0}")] + Jni(String), + #[error("bridge initialization failed: {0}")] + Initialize(String), + #[error("platform error: {0}")] + Platform(#[from] upgradeall_platform_adapter::PlatformAdapterError), + #[error("platform inventory response is malformed: {0}")] + PlatformMalformed(String), + #[error("storage error: {0}")] + Storage(String), + #[error("repository error: {0}")] + Repository(String), + #[error("autogen error: {0}")] + Autogen(String), +} + +impl BridgeOperationError { + fn parts(self) -> (&'static str, &'static str, Option) { + match self { + Self::InvalidRequest(detail) => ( + "bridge.invalid_request", + "Getter native bridge request is invalid", + Some(detail), + ), + Self::Jni(detail) => ( + "bridge.jni_error", + "Getter native bridge JNI operation failed", + Some(detail), + ), + Self::Initialize(detail) => ( + "bridge.initialize_error", + "Getter native bridge initialization failed", + Some(detail), + ), + Self::Platform(upgradeall_platform_adapter::PlatformAdapterError::Unsupported { + capability, + }) => ( + "platform.unsupported", + "Android platform capability is unsupported", + Some(capability.to_owned()), + ), + Self::Platform(upgradeall_platform_adapter::PlatformAdapterError::NotInitialized) => ( + "platform.not_initialized", + "Android platform adapter is not initialized", + None, + ), + Self::Platform(upgradeall_platform_adapter::PlatformAdapterError::Jni(detail)) => ( + "platform.jni_error", + "Android platform adapter JNI operation failed", + Some(detail), + ), + Self::Platform( + upgradeall_platform_adapter::PlatformAdapterError::MalformedResponse(detail), + ) => ( + "platform.malformed_response", + "Android platform adapter response is malformed", + Some(detail), + ), + Self::PlatformMalformed(detail) => ( + "platform.malformed_response", + "Android platform inventory response is malformed", + Some(detail), + ), + Self::Storage(detail) => ( + "storage.error", + "Getter storage operation failed", + Some(detail), + ), + Self::Repository(detail) => ( + "repository.error", + "Getter repository operation failed", + Some(detail), + ), + Self::Autogen(detail) => ( + "autogen.error", + "Getter autogen operation failed", + Some(detail), + ), + } + } +} + +impl From for BridgeOperationError { + fn from(value: getter::storage::StorageError) -> Self { + Self::Storage(value.to_string()) + } +} + +impl From for BridgeOperationError { + fn from(value: AutogenOperationError) -> Self { + match value { + AutogenOperationError::Storage(source) => Self::Storage(source.to_string()), + AutogenOperationError::Repository(detail) => Self::Repository(detail), + AutogenOperationError::Autogen(detail) => Self::Autogen(detail), + } + } +} + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn packages_acceptance_defaults_to_all() { + let acceptance = ApplyInstalledAutogenAcceptance::default() + .into_autogen_acceptance() + .expect("acceptance"); + + assert!(matches!(acceptance, AutogenAcceptance::AcceptAll)); + } + + #[test] + fn packages_acceptance_preserves_getter_package_ids() { + let acceptance = ApplyInstalledAutogenAcceptance { + mode: Some("packages".to_owned()), + package_ids: vec!["android/org.fdroid.fdroid".parse().expect("package id")], + } + .into_autogen_acceptance() + .expect("acceptance"); + + match acceptance { + AutogenAcceptance::Accept(ids) => { + assert_eq!(ids[0].to_string(), "android/org.fdroid.fdroid") + } + AutogenAcceptance::AcceptAll => panic!("expected explicit package acceptance"), + } + } } diff --git a/core-getter/src/main/rust/getter b/core-getter/src/main/rust/getter index cb4cb655..cee3b007 160000 --- a/core-getter/src/main/rust/getter +++ b/core-getter/src/main/rust/getter @@ -1 +1 @@ -Subproject commit cb4cb655f3aefa69200ac23cfcc1e770d678d664 +Subproject commit cee3b007d5e03d9035f5319a582f452dfbc7c911 diff --git a/docs/architecture/adr/0007-flutter-getter-bridge-contract.md b/docs/architecture/adr/0007-flutter-getter-bridge-contract.md index b788df2a..c0361f88 100644 --- a/docs/architecture/adr/0007-flutter-getter-bridge-contract.md +++ b/docs/architecture/adr/0007-flutter-getter-bridge-contract.md @@ -45,14 +45,16 @@ importLegacyRoomDatabase(databasePath) The Android platform adapter may prepare a copied/checkpointed legacy Room SQLite file and return its path to Flutter, but getter still owns the actual `legacy import-room-db` import semantics. Flutter starts the flow and renders getter reports; it must not inspect or map Room tables directly. -The installed-autogen product bridge must follow ADR-0009's Rust-active platform adapter direction rather than a Flutter-led inventory scan. The future bridge shape is a getter-owned operation such as: +The fourth accepted API surface adds the production installed-autogen bridge boundary and must follow ADR-0009's Rust-active platform adapter direction rather than a Flutter-led inventory scan: ```text previewInstalledAutogen(scanOptions) applyInstalledAutogen(preview, acceptedPackages) ``` -Internally, Rust/native bridge code scans Android inventory through the platform adapter, then asks getter to plan/apply `local_autogen`. Flutter renders getter-owned preview/apply DTOs and scan diagnostics; it must not expose a product Dart `InstalledInventoryPlatform` scanner or convert Android package names into package ids. +The Android product APK packages a slim `:getter_bridge` library under `app_flutter/android/getter_bridge`. It builds the Rust `api_proxy` cdylib and includes only the no-UI native bridge / installed-inventory provider classes needed by the Flutter product path, avoiding the legacy native `:app` UI and old `GetterPort` hub/RPC wrapper surface. `MainActivity` exposes a no-UI `net.xzos.upgradeall/getter_bridge` MethodChannel that derives the app-private getter data directory and forwards preview/apply requests to JNI entrypoints returning getter-style JSON envelopes. + +Internally, Rust/native bridge code scans Android inventory through the platform adapter, then asks getter-owned shared autogen operations to plan/apply `local_autogen`. `MethodChannelGetterAdapter` consumes the returned getter-style JSON envelopes. Flutter renders getter-owned preview/apply DTOs and scan diagnostics, then passes the package ids from displayed accepted preview candidates back to getter on apply; it must not expose a product Dart `InstalledInventoryPlatform` scanner or convert Android package names into package ids. `loadSnapshot()` composes the smaller getter-owned operations into the UI shell's first snapshot DTO. It must not perform repository resolution, version comparison, migration mapping, or update selection in Dart. `readMigrationReports()` must go through a getter operation such as `legacy report-list`; Flutter must not inspect getter's data-directory layout directly. `listDownloadTasks()` and `listTaskEvents()` render getter-owned task/event DTOs; Flutter must not synthesize task states, retry policy, installer behavior, or update decisions. @@ -74,6 +76,11 @@ DownloadTaskSummary TaskEventPage TaskEventSummary GetterError +InstalledAutogenPreview +InstalledAutogenCandidate +InstalledAutogenSkip +InstalledAutogenScanStats +InstalledAutogenApplyResult ``` DTOs are a UI transport shape, not a new product model. Any field whose value requires domain interpretation must be supplied by getter or by a platform capability explicitly documented in a later ADR. @@ -107,7 +114,7 @@ and structured error envelopes: Flutter adapter code may parse and display these fields, but it must not infer missing domain state from them. If the UI needs a richer field, add it to getter output first and cover it with getter tests. -For installed-autogen flows, CLI/dev tests may continue to pass fixture inventory JSON to `getter autogen installed preview/apply`. The Android product bridge should not expose that fixture boundary as a Flutter-owned scanning API; it should wrap scan + getter autogen planning behind a getter/native bridge operation. +For installed-autogen flows, CLI/dev tests may continue to pass fixture inventory JSON to `getter autogen installed preview/apply`. The Android product bridge does not expose that fixture boundary as a Flutter-owned scanning API; it wraps scan + getter autogen planning behind a getter/native bridge operation. ## Error model @@ -194,7 +201,7 @@ The first implementation slice must provide: ## Non-goals -- No full FFI/native bridge implementation in this ADR. +- No full FFI/native bridge implementation beyond the first installed-autogen preview/apply JNI/MethodChannel operation slice. - No update/download/install event stream. - No Android-owned legacy Room mapping/import semantics; Android only prepares a copied DB file for getter. - No product-complete Flutter UI. diff --git a/docs/architecture/adr/0009-android-platform-adapter-and-package-visibility.md b/docs/architecture/adr/0009-android-platform-adapter-and-package-visibility.md index e047d8b2..97a777de 100644 --- a/docs/architecture/adr/0009-android-platform-adapter-and-package-visibility.md +++ b/docs/architecture/adr/0009-android-platform-adapter-and-package-visibility.md @@ -127,14 +127,24 @@ The second slice adds the first Android facts provider while preserving the same - Rust `AndroidPlatformAdapter::scan_installed_inventory` serializes scan options, calls the provider, and deserializes the JSON into platform DTOs; - `api_proxy` initializes the platform adapter runtime alongside `rustls-platform-verifier`, using separate JNI local refs for each initializer. +The third slice wires the first product bridge operation without changing ownership boundaries: + +- installed-autogen preview/apply semantics are extracted into reusable getter-owned `getter-operations` code so CLI and native bridge use the same `local_autogen` rules; +- `getter-core` Lua support is feature-gated so the Android native bridge can use autogen/storage operations without pulling Lua evaluation into `api_proxy`; +- `api_proxy` exposes JNI entrypoints for bridge initialization, installed-autogen preview, and installed-autogen apply; +- preview initializes the Rust-active Android platform adapter, scans PackageManager facts, and passes getter-compatible inventory into getter-owned autogen planning; +- apply validates the preview/acceptance and uses the same getter-owned apply code as the CLI; +- `app_flutter/android/getter_bridge` is a slim Android library that packages `libapi_proxy.so`, `NativeLib`, and the facts provider classes into the Flutter product APK without depending on the legacy native `:app` UI or old `GetterPort` hub/RPC wrapper surface; +- `just verify` now inspects the Flutter debug APK for the native bridge library and provider classes. + These slices still do not: - make the reusable getter submodule depend on superproject-only crates; -- add Flutter installed-autogen UX; -- add product native bridge operations that combine platform scan + getter autogen preview/apply; - add Magisk scanning; - add live downloads, background worker policy, installer URI/SAF semantics, or notification behavior. +A follow-up slice added the first Flutter installed-autogen preview/apply confirmation UI. It consumes getter-owned DTOs from `MethodChannelGetterAdapter` and passes displayed accepted package ids back to getter; it still does not scan PackageManager or generate package ids in Dart/Flutter. + ## Consequences Positive: @@ -146,7 +156,7 @@ Positive: Costs and risks: -- The first slice is not yet product-complete; production bridge packaging into the Flutter APK still needs a later accepted implementation. +- The current bridge slice is not yet product-complete because device/instrumented runtime verification is still pending. - The Rust platform DTOs must stay compatible with getter's installed inventory contract. - JNI/runtime bugs require Android build/device validation beyond host unit tests. - Broad package visibility is now an explicit product policy with distribution implications. diff --git a/docs/architecture/upgradeall-getter-rewrite-wiki.md b/docs/architecture/upgradeall-getter-rewrite-wiki.md index 078d73f5..4e195fde 100644 --- a/docs/architecture/upgradeall-getter-rewrite-wiki.md +++ b/docs/architecture/upgradeall-getter-rewrite-wiki.md @@ -1046,6 +1046,8 @@ Room DB 信息： 6. getter 写入 `local_autogen` repo。 7. 生成后不会自动消失。 +实现进展：Flutter 产品 APK 通过 `app_flutter/android/getter_bridge` 打包一个 slim native bridge library，包含 Rust `api_proxy`、`NativeLib` 和 Android installed-inventory facts provider。`api_proxy` 已提供 installed-autogen preview/apply JNI entrypoints；它们调用 Rust-active platform adapter 扫描 Android PackageManager 原始事实，再调用 getter-owned `getter-operations` 执行 `local_autogen` preview/apply。Flutter 已新增 installed-autogen 页面和 `MethodChannelGetterAdapter`，只渲染 getter-owned preview/apply DTO 并把用户接受的包 id 传回 getter；不能引入 Dart-led installed inventory scanner 或在 Dart/Kotlin 中生成 package id。 + ### 14.2 清理流程 用户点击“清除不存在的应用”： diff --git a/justfile b/justfile index c6b55e1c..3c6cdf19 100644 --- a/justfile +++ b/justfile @@ -36,6 +36,7 @@ test-android-platform-adapter: build-flutter-android-debug: cd app_flutter && flutter build apk --debug + python3 tools/verify_flutter_apk_bridge.py app_flutter/build/app/outputs/flutter-apk/app-debug.apk verify-workspace-skeleton: test "$(git ls-files -s core-getter/src/main/rust/getter | awk '{print $1}')" = "160000" @@ -46,7 +47,7 @@ verify-workspace-skeleton: cargo fmt --manifest-path {{ PLATFORM_ADAPTER_MANIFEST }} --all --check cargo check --manifest-path {{ GETTER_MANIFEST }} --workspace --all-targets cargo check --manifest-path {{ API_PROXY_MANIFEST }} - cargo check --manifest-path {{ API_PROXY_MANIFEST }} --target aarch64-linux-android + if [ -n "${ANDROID_NDK_HOME:-}" ]; then export CC_aarch64_linux_android="$ANDROID_NDK_HOME/toolchains/llvm/prebuilt/linux-x86_64/bin/aarch64-linux-android23-clang"; fi; cargo check --manifest-path {{ API_PROXY_MANIFEST }} --target aarch64-linux-android cargo test --manifest-path {{ PLATFORM_ADAPTER_MANIFEST }} cargo check --manifest-path {{ PLATFORM_ADAPTER_MANIFEST }} --target aarch64-linux-android cd app_flutter && flutter analyze diff --git a/todo.md b/todo.md index d182fc35..6eb50290 100644 --- a/todo.md +++ b/todo.md @@ -427,7 +427,7 @@ User-confirmed decisions: - autogen apply/cleanup are getter-managed; if a generated file has been edited, getter preserves it into `local` before regenerating/deleting. - applying installed autogen also tracks accepted packages because user confirmation means the user wants update tracking. -Status: first getter-owned CLI/core slice in progress. Implemented pure autogen planning, installed preview/apply, cleanup preview/apply, deterministic package Lua generation, manifest-managed cleanup, higher-priority coverage skips, local preservation for edited autogen files, guarded cleanup against stale/tampered previews, and preservation of existing tracked user state during autogen apply. Flutter/Android inventory collection and UX remain future adapter work. +Status: getter-owned CLI/core and first production bridge slices are in progress. Implemented pure autogen planning, installed preview/apply, cleanup preview/apply, deterministic package Lua generation, manifest-managed cleanup, higher-priority coverage skips, local preservation for edited autogen files, guarded cleanup against stale/tampered previews, and preservation of existing tracked user state during autogen apply. Added Rust-active Android PackageManager inventory collection and first native bridge preview/apply operations packaged into the Flutter product APK. Added a Flutter installed-autogen preview/apply UI that renders getter-owned DTOs and calls the native bridge without Dart-led package-id or autogen decisions. Completed tasks: @@ -439,13 +439,15 @@ Completed tasks: 6. Preserve edited generated files into `local` before autogen rewrite/delete. 7. Guard cleanup deletion by current autogen manifest, repository id, and generated-package resolution. 8. Add Rust-active Android installed inventory provider/scanner path: Kotlin PackageManager facts provider, Rust JNI call/deserialization, and `api_proxy` runtime initialization. +9. Extract installed-autogen preview/apply semantics into reusable getter-owned `getter-operations` code so CLI and native bridge share the same `local_autogen` rules. +10. Add native bridge operations that combine platform scan + getter `local_autogen` preview/apply while returning getter-style JSON envelopes. +11. Wire/package a slim production bridge into `app_flutter` so the Flutter APK contains `libapi_proxy.so`, `NativeLib`, and the installed-inventory provider classes without depending on the legacy native `:app` UI or old `GetterPort` hub/RPC wrapper surface. +12. Add Flutter confirmation UX that consumes getter preview/apply DTOs and passes displayed accepted package ids back to getter/native bridge. Remaining tasks: -1. Add getter/native bridge operations that combine platform scan + getter `local_autogen` preview/apply; Flutter must consume these getter-owned DTOs rather than leading a MethodChannel inventory scan. -2. Wire/package the production bridge into `app_flutter` so the Flutter APK can exercise the Rust-active installed inventory provider path. -3. Flutter confirmation UX consumes getter preview/apply DTOs. -4. Cache invalidation hooks beyond file-hash-based repository reload need to be expanded when evaluated/provider caches become active. +1. Add device/instrumented validation for the full Flutter MethodChannel -> JNI -> Rust platform scan -> getter autogen preview/apply path if practical. +2. Cache invalidation hooks beyond file-hash-based repository reload need to be expanded when evaluated/provider caches become active. Acceptance progress: diff --git a/tools/verify_flutter_apk_bridge.py b/tools/verify_flutter_apk_bridge.py new file mode 100644 index 00000000..81e12bb3 --- /dev/null +++ b/tools/verify_flutter_apk_bridge.py @@ -0,0 +1,45 @@ +#!/usr/bin/env python3 +"""Verify that a Flutter APK packages the getter native bridge.""" + +from __future__ import annotations + +import sys +import zipfile +from pathlib import Path + +EXPECTED_ABIS = ("arm64-v8a", "armeabi-v7a", "x86_64") +DEX_MARKERS = ( + b"net/xzos/upgradeall/getter/NativeLib", + b"net/xzos/upgradeall/getter/platform/InstalledInventoryProvider", +) + + +def main() -> int: + if len(sys.argv) != 2: + print("usage: verify_flutter_apk_bridge.py ", file=sys.stderr) + return 2 + + apk = Path(sys.argv[1]) + missing: list[str] = [] + with zipfile.ZipFile(apk) as archive: + names = set(archive.namelist()) + missing.extend( + f"lib/{abi}/libapi_proxy.so" + for abi in EXPECTED_ABIS + if f"lib/{abi}/libapi_proxy.so" not in names + ) + dex = b"".join( + archive.read(name) + for name in names + if name.startswith("classes") and name.endswith(".dex") + ) + + missing.extend(marker.decode() for marker in DEX_MARKERS if marker not in dex) + if missing: + print(f"missing from {apk}: {', '.join(missing)}", file=sys.stderr) + return 1 + return 0 + + +if __name__ == "__main__": + raise SystemExit(main()) From 8a6ac213465d51f5d18f47ce1d4078152dfbf87f Mon Sep 17 00:00:00 2001 From: Xiangzhe Date: Wed, 24 Jun 2026 10:19:53 +0800 Subject: [PATCH 25/85] feat(app): bridge legacy room import --- app_flutter/README.md | 4 +- .../net/xzos/upgradeall/MainActivity.kt | 24 +++++ .../net/xzos/upgradeall/getter/NativeLib.kt | 2 + .../dev_test/cli_getter_adapter_test.dart | 14 +-- app_flutter/lib/cli_getter_adapter.dart | 82 ++------------ app_flutter/lib/getter_adapter.dart | 100 +++++++++++++++++- app_flutter/lib/main.dart | 29 ++++- app_flutter/lib/native_getter_adapter.dart | 34 ++++++ .../test/native_getter_adapter_test.dart | 71 +++++++++++++ app_flutter/test/widget_test.dart | 6 +- core-getter/consumer-rules.pro | 3 + .../net/xzos/upgradeall/getter/NativeLib.kt | 2 + .../src/main/rust/api_proxy/src/lib.rs | 65 ++++++++++++ core-getter/src/main/rust/getter | 2 +- .../adr/0003-legacy-room-migration.md | 2 +- .../0007-flutter-getter-bridge-contract.md | 6 +- .../upgradeall-getter-rewrite-wiki.md | 2 + todo.md | 12 ++- 18 files changed, 358 insertions(+), 102 deletions(-) diff --git a/app_flutter/README.md b/app_flutter/README.md index 78efbafb..0f6e48c4 100644 --- a/app_flutter/README.md +++ b/app_flutter/README.md @@ -11,10 +11,10 @@ This is the new Flutter shell and product APK entry for the UpgradeAll rewrite. - `FakeGetterAdapter` for deterministic widget tests - `CliGetterAdapter` as a development/integration bridge against the real `getter-cli` JSON envelope - A slim Android `:getter_bridge` library inside `app_flutter/android/getter_bridge` packages the Rust `api_proxy` native library and the no-UI installed-inventory provider classes into the Flutter product APK without depending on the legacy native `:app` UI or old `GetterPort` RPC wrapper surface. -- `MainActivity` exposes a no-UI `net.xzos.upgradeall/getter_bridge` MethodChannel for native bridge plumbing. The installed-autogen methods derive the app-private getter data directory on Android, call Rust JNI entrypoints, and return getter-style JSON envelopes consumed by `MethodChannelGetterAdapter`. +- `MainActivity` exposes a no-UI `net.xzos.upgradeall/getter_bridge` MethodChannel for native bridge plumbing. The legacy migration and installed-autogen methods derive the app-private getter data directory on Android, call Rust JNI entrypoints, and return getter-style JSON envelopes consumed by `MethodChannelGetterAdapter`. - Product manifest permissions include `QUERY_ALL_PACKAGES` per ADR-0009 so the Rust-active Android platform adapter can provide complete installed package inventory facts to getter. -`CliGetterAdapter` is not the final Android production bridge. It exists to keep the getter-owned DTO and error contract executable. `MethodChannelGetterAdapter` is the current production bridge slice for installed-autogen preview/apply: Flutter renders getter-owned DTOs and passes accepted package ids back to getter, but PackageManager scanning, package-id decisions, and `local_autogen` writes remain in Rust/native getter code. +`CliGetterAdapter` is not the final Android production bridge. It exists to keep the getter-owned DTO and error contract executable for dev tests. `MethodChannelGetterAdapter` is the current production bridge slice for direct legacy Room import/report-list and installed-autogen preview/apply: Flutter renders getter-owned DTOs and passes user choices/paths back to getter, but Room mapping, PackageManager scanning, package-id decisions, and `local_autogen` writes remain in Rust/native getter code. ## Verification diff --git a/app_flutter/android/app/src/main/kotlin/net/xzos/upgradeall/MainActivity.kt b/app_flutter/android/app/src/main/kotlin/net/xzos/upgradeall/MainActivity.kt index ce20e44d..b3858252 100644 --- a/app_flutter/android/app/src/main/kotlin/net/xzos/upgradeall/MainActivity.kt +++ b/app_flutter/android/app/src/main/kotlin/net/xzos/upgradeall/MainActivity.kt @@ -41,6 +41,14 @@ class MainActivity : FlutterActivity() { nativeLib.applyInstalledAutogen(applyInstalledAutogenRequest(call)) } + "importLegacyRoomDatabase" -> runGetterBridge(result) { + nativeLib.importLegacyRoomDatabase(importLegacyRoomDatabaseRequest(call)) + } + + "legacyReportList" -> runGetterBridge(result) { + nativeLib.legacyReportList(legacyReportListRequest()) + } + else -> result.notImplemented() } } @@ -145,6 +153,22 @@ class MainActivity : FlutterActivity() { .toString() } + private fun importLegacyRoomDatabaseRequest(call: MethodCall): String { + val args = call.arguments as? Map<*, *> ?: emptyMap() + val databasePath = args["database_path"] as? String + ?: throw IllegalArgumentException("database_path is required") + return JSONObject() + .put("data_dir", getterDataDir().absolutePath) + .put("database_path", databasePath) + .toString() + } + + private fun legacyReportListRequest(): String { + return JSONObject() + .put("data_dir", getterDataDir().absolutePath) + .toString() + } + private fun getterDataDir(): File = File(filesDir, "getter") private fun prepareLegacyRoomImport(): Map { diff --git a/app_flutter/android/getter_bridge/src/main/kotlin/net/xzos/upgradeall/getter/NativeLib.kt b/app_flutter/android/getter_bridge/src/main/kotlin/net/xzos/upgradeall/getter/NativeLib.kt index 7921aed0..66686143 100644 --- a/app_flutter/android/getter_bridge/src/main/kotlin/net/xzos/upgradeall/getter/NativeLib.kt +++ b/app_flutter/android/getter_bridge/src/main/kotlin/net/xzos/upgradeall/getter/NativeLib.kt @@ -13,6 +13,8 @@ class NativeLib { external fun initializeBridge(context: Context): String external fun previewInstalledAutogen(context: Context, requestJson: String): String external fun applyInstalledAutogen(requestJson: String): String + external fun importLegacyRoomDatabase(requestJson: String): String + external fun legacyReportList(requestJson: String): String fun runServerLambda(context: Context, callback: (String) -> Unit): String { return runServer(context, RunServerCallback(callback)) diff --git a/app_flutter/dev_test/cli_getter_adapter_test.dart b/app_flutter/dev_test/cli_getter_adapter_test.dart index 8b57ada5..df30f023 100644 --- a/app_flutter/dev_test/cli_getter_adapter_test.dart +++ b/app_flutter/dev_test/cli_getter_adapter_test.dart @@ -4,7 +4,7 @@ import 'package:flutter_test/flutter_test.dart'; import 'package:upgradeall/cli_getter_adapter.dart'; void main() { - test('CliGetterAdapter imports a direct legacy Room database', () { + test('CliGetterAdapter imports a direct legacy Room database', () async { final getterCli = Platform.environment['GETTER_CLI_BIN']; if (getterCli == null || getterCli.isEmpty) { fail('GETTER_CLI_BIN must point to the built getter-cli binary'); @@ -19,7 +19,7 @@ void main() { CliGetterAdapter(executable: getterCli, dataDir: dataDir.path); adapter.initialize(); - final result = adapter.importLegacyRoomDatabase(legacyDb.path); + final result = await adapter.importLegacyRoomDatabase(legacyDb.path); expect(result.alreadyImported, isFalse); expect(result.importedRecords, 1); @@ -32,14 +32,15 @@ void main() { expect(tracked.ignoredVersion, '1.20.0'); expect(tracked.packageResolution, 'missing_package_definition'); - final reports = adapter.readMigrationReports(); + final reports = await adapter.readMigrationReports(); expect( reports.singleWhere((report) => report.code == 'migration.imported').ok, isTrue, ); }); - test('CliGetterAdapter reads real getter repository and tracked state', () { + test('CliGetterAdapter reads real getter repository and tracked state', + () async { final getterCli = Platform.environment['GETTER_CLI_BIN']; if (getterCli == null || getterCli.isEmpty) { fail('GETTER_CLI_BIN must point to the built getter-cli binary'); @@ -99,12 +100,13 @@ void main() { expect(evaluated.repositoryId, 'official'); expect(evaluated.hasFreeNetworkWarning, isTrue); - final reports = adapter.readMigrationReports(); + final reports = await adapter.readMigrationReports(); expect( reports.singleWhere((report) => report.code == 'migration.imported').ok, isTrue); - final alreadyImported = adapter.importLegacyRoomDatabase(legacyDb.path); + final alreadyImported = + await adapter.importLegacyRoomDatabase(legacyDb.path); expect(alreadyImported.alreadyImported, isTrue); expect(alreadyImported.importedRecords, 0); expect( diff --git a/app_flutter/lib/cli_getter_adapter.dart b/app_flutter/lib/cli_getter_adapter.dart index 5aa6ddae..f83041f0 100644 --- a/app_flutter/lib/cli_getter_adapter.dart +++ b/app_flutter/lib/cli_getter_adapter.dart @@ -51,22 +51,25 @@ class CliGetterAdapter implements GetterAdapter { } @override - List readMigrationReports() { + Future> readMigrationReports() async { final json = _runGetter(const ['legacy', 'report-list']); final reports = _asList(_data(json)['reports'], 'reports'); return reports - .map((report) => _migrationReportFromJson(_asMap(report, 'report'))) + .map((report) => + MigrationReportSummary.fromJson(_asMap(report, 'report'))) .toList(growable: false); } @override - LegacyMigrationImportResult importLegacyRoomDatabase(String databasePath) { + Future importLegacyRoomDatabase( + String databasePath, + ) async { final json = _runGetter([ 'legacy', 'import-room-db', databasePath, ]); - return _legacyMigrationImportResultFromJson(_data(json)); + return LegacyMigrationImportResult.fromJson(_data(json)); } @override @@ -211,70 +214,6 @@ PackageEvaluation _packageEvaluationFromJson(Object? value) { ); } -MigrationReportSummary _migrationReportFromJson(Map json) { - return MigrationReportSummary( - ok: _asBool(json['ok'], 'migration.ok'), - code: _asString(json['code'], 'migration.code'), - message: _asString(json['message'], 'migration.message'), - importedRecords: _asInt(json['imported_records'], 'migration.imported'), - trackedRecords: _asInt(json['tracked_records'], 'migration.tracked'), - ); -} - -LegacyMigrationImportResult _legacyMigrationImportResultFromJson( - Map json, -) { - final warningsValue = json['warnings']; - final sourceCountsValue = json['source_counts']; - return LegacyMigrationImportResult( - alreadyImported: _asOptionalBool( - json['already_imported'], - 'migration.already_imported', - ) ?? - false, - importedRecords: _asInt(json['imported_records'], 'migration.imported'), - trackedPackages: _asList(json['apps'], 'migration.apps') - .map(_trackedPackageFromJson) - .toList(growable: false), - warnings: warningsValue == null - ? const [] - : _asList(warningsValue, 'migration.warnings') - .map((warning) => _migrationWarningFromJson( - _asMap(warning, 'migration.warning'), - )) - .toList(growable: false), - sourceCounts: sourceCountsValue == null - ? null - : _migrationSourceCountsFromJson( - _asMap(sourceCountsValue, 'migration.source_counts'), - ), - ); -} - -MigrationWarningSummary _migrationWarningFromJson(Map json) { - return MigrationWarningSummary( - code: _asString(json['code'], 'migration.warning.code'), - message: _asString(json['message'], 'migration.warning.message'), - ); -} - -MigrationSourceCounts _migrationSourceCountsFromJson( - Map json, -) { - return MigrationSourceCounts( - appRows: _asInt(json['app_rows'], 'migration.source_counts.app_rows'), - extraAppRows: _asInt( - json['extra_app_rows'], - 'migration.source_counts.extra_app_rows', - ), - hubRows: _asInt(json['hub_rows'], 'migration.source_counts.hub_rows'), - extraHubRows: _asInt( - json['extra_hub_rows'], - 'migration.source_counts.extra_hub_rows', - ), - ); -} - DownloadTaskSummary _downloadTaskFromJson(Map json) { return DownloadTaskSummary( id: _asString(json['id'], 'task.id'), @@ -370,10 +309,3 @@ bool _asBool(Object? value, String name) { } throw FormatException('$name should be a boolean'); } - -bool? _asOptionalBool(Object? value, String name) { - if (value == null || value is bool) { - return value as bool?; - } - throw FormatException('$name should be a boolean or null'); -} diff --git a/app_flutter/lib/getter_adapter.dart b/app_flutter/lib/getter_adapter.dart index 7b9a7855..cbf7891d 100644 --- a/app_flutter/lib/getter_adapter.dart +++ b/app_flutter/lib/getter_adapter.dart @@ -16,9 +16,10 @@ abstract interface class GetterAdapter { PackageEvaluation evaluatePackage(String packageId, {String? repositoryId}); - List readMigrationReports(); + Future> readMigrationReports(); - LegacyMigrationImportResult importLegacyRoomDatabase(String databasePath); + Future importLegacyRoomDatabase( + String databasePath); List listDownloadTasks(); @@ -156,12 +157,13 @@ class FakeGetterAdapter implements GetterAdapter { ); @override - List readMigrationReports() { + Future> readMigrationReports() async { return const []; } @override - LegacyMigrationImportResult importLegacyRoomDatabase(String databasePath) { + Future importLegacyRoomDatabase( + String databasePath) async { throw const GetterBridgeException( GetterError( code: 'bridge.not_connected', @@ -306,6 +308,24 @@ class TrackedPackageSummary { required this.packageResolution, }); + factory TrackedPackageSummary.fromJson(Map json) { + return TrackedPackageSummary( + id: _jsonString(json['id'], 'tracked.id'), + enabled: _jsonBool(json['enabled'], 'tracked.enabled'), + favorite: _jsonBool(json['favorite'], 'tracked.favorite'), + ignoredVersion: _jsonOptionalString( + json['ignored_version'], + 'tracked.ignored_version', + ), + repositoryId: + _jsonOptionalString(json['repository_id'], 'tracked.repository_id'), + packageResolution: _jsonString( + json['package_resolution'], + 'tracked.package_resolution', + ), + ); + } + final String id; final bool enabled; final bool favorite; @@ -337,6 +357,16 @@ class MigrationReportSummary { required this.trackedRecords, }); + factory MigrationReportSummary.fromJson(Map json) { + return MigrationReportSummary( + ok: _jsonBool(json['ok'], 'migration.ok'), + code: _jsonString(json['code'], 'migration.code'), + message: _jsonString(json['message'], 'migration.message'), + importedRecords: _jsonInt(json['imported_records'], 'migration.imported'), + trackedRecords: _jsonInt(json['tracked_records'], 'migration.tracked'), + ); + } + final bool ok; final String code; final String message; @@ -353,6 +383,36 @@ class LegacyMigrationImportResult { required this.sourceCounts, }); + factory LegacyMigrationImportResult.fromJson(Map json) { + final warningsValue = json['warnings']; + final sourceCountsValue = json['source_counts']; + return LegacyMigrationImportResult( + alreadyImported: _jsonOptionalBool( + json['already_imported'], + 'migration.already_imported', + ) ?? + false, + importedRecords: _jsonInt(json['imported_records'], 'migration.imported'), + trackedPackages: _jsonList(json['apps'], 'migration.apps') + .map((tracked) => TrackedPackageSummary.fromJson( + _jsonMap(tracked, 'migration.tracked_package'), + )) + .toList(growable: false), + warnings: warningsValue == null + ? const [] + : _jsonList(warningsValue, 'migration.warnings') + .map((warning) => MigrationWarningSummary.fromJson( + _jsonMap(warning, 'migration.warning'), + )) + .toList(growable: false), + sourceCounts: sourceCountsValue == null + ? null + : MigrationSourceCounts.fromJson( + _jsonMap(sourceCountsValue, 'migration.source_counts'), + ), + ); + } + final bool alreadyImported; final int importedRecords; final List trackedPackages; @@ -363,6 +423,13 @@ class LegacyMigrationImportResult { class MigrationWarningSummary { const MigrationWarningSummary({required this.code, required this.message}); + factory MigrationWarningSummary.fromJson(Map json) { + return MigrationWarningSummary( + code: _jsonString(json['code'], 'migration.warning.code'), + message: _jsonString(json['message'], 'migration.warning.message'), + ); + } + final String code; final String message; } @@ -375,6 +442,21 @@ class MigrationSourceCounts { required this.extraHubRows, }); + factory MigrationSourceCounts.fromJson(Map json) { + return MigrationSourceCounts( + appRows: _jsonInt(json['app_rows'], 'migration.source_counts.app_rows'), + extraAppRows: _jsonInt( + json['extra_app_rows'], + 'migration.source_counts.extra_app_rows', + ), + hubRows: _jsonInt(json['hub_rows'], 'migration.source_counts.hub_rows'), + extraHubRows: _jsonInt( + json['extra_hub_rows'], + 'migration.source_counts.extra_hub_rows', + ), + ); + } + final int appRows; final int extraAppRows; final int hubRows; @@ -785,3 +867,13 @@ int _jsonInt(Object? value, String name) { if (value is int) return value; throw FormatException('$name should be an integer'); } + +bool _jsonBool(Object? value, String name) { + if (value is bool) return value; + throw FormatException('$name should be a boolean'); +} + +bool? _jsonOptionalBool(Object? value, String name) { + if (value == null || value is bool) return value as bool?; + throw FormatException('$name should be a boolean or null'); +} diff --git a/app_flutter/lib/main.dart b/app_flutter/lib/main.dart index d7666a7d..5787070c 100644 --- a/app_flutter/lib/main.dart +++ b/app_flutter/lib/main.dart @@ -656,7 +656,7 @@ class MigrationPage extends StatefulWidget { } class _MigrationPageState extends State { - late List _reports; + List _reports = const []; LegacyMigrationImportResult? _importResult; String? _status; GetterError? _error; @@ -665,7 +665,20 @@ class _MigrationPageState extends State { @override void initState() { super.initState(); - _reports = widget.getter.readMigrationReports(); + _loadMigrationReports(); + } + + Future _loadMigrationReports() async { + try { + final reports = await widget.getter.readMigrationReports(); + if (!mounted) return; + setState(() { + _reports = reports; + }); + } on GetterBridgeException { + // Reports are best-effort on page open. The explicit migration action + // surfaces bridge errors to the user. + } } Future _startMigration() async { @@ -688,8 +701,8 @@ class _MigrationPageState extends State { } final importResult = - widget.getter.importLegacyRoomDatabase(candidate.databasePath!); - final reports = widget.getter.readMigrationReports(); + await widget.getter.importLegacyRoomDatabase(candidate.databasePath!); + final reports = await widget.getter.readMigrationReports(); if (!mounted) return; setState(() { _importResult = importResult; @@ -700,11 +713,17 @@ class _MigrationPageState extends State { _running = false; }); } on GetterBridgeException catch (error) { + var reports = _reports; + try { + reports = await widget.getter.readMigrationReports(); + } on GetterBridgeException { + // Keep the reports already on screen if the bridge cannot list them. + } if (!mounted) return; setState(() { _error = error.error; _status = error.error.message; - _reports = widget.getter.readMigrationReports(); + _reports = reports; _running = false; }); } catch (error) { diff --git a/app_flutter/lib/native_getter_adapter.dart b/app_flutter/lib/native_getter_adapter.dart index 558cf8d1..14d00b25 100644 --- a/app_flutter/lib/native_getter_adapter.dart +++ b/app_flutter/lib/native_getter_adapter.dart @@ -20,6 +20,9 @@ class MethodChannelGetterAdapter extends FakeGetterAdapter { final MethodChannel _channel; + @override + bool get supportsLegacyRoomImport => true; + @override bool get supportsInstalledAutogen => true; @@ -28,6 +31,31 @@ class MethodChannelGetterAdapter extends FakeGetterAdapter { // The installed-autogen bridge initializes lazily when preview is called. } + @override + Future> readMigrationReports() async { + final data = await _invokeGetterData( + 'legacyReportList', + const {}, + ); + final reports = _asList(data['reports'], 'legacy reports'); + return reports + .map((report) => MigrationReportSummary.fromJson( + _asMap(report, 'legacy report'), + )) + .toList(growable: false); + } + + @override + Future importLegacyRoomDatabase( + String databasePath, + ) async { + final data = await _invokeGetterData( + 'importLegacyRoomDatabase', + {'database_path': databasePath}, + ); + return LegacyMigrationImportResult.fromJson(data); + } + @override Future previewInstalledAutogen({ InstalledAutogenScanOptions options = const InstalledAutogenScanOptions(), @@ -105,6 +133,12 @@ Map _asMap(Object? value, String name) { throw FormatException('$name should be a JSON object'); } +List _asList(Object? value, String name) { + if (value is List) return value; + if (value is List) return value.cast(); + throw FormatException('$name should be a JSON array'); +} + String _asString(Object? value, String name) { if (value is String) return value; throw FormatException('$name should be a string'); diff --git a/app_flutter/test/native_getter_adapter_test.dart b/app_flutter/test/native_getter_adapter_test.dart index f8305f27..f1cc5ffe 100644 --- a/app_flutter/test/native_getter_adapter_test.dart +++ b/app_flutter/test/native_getter_adapter_test.dart @@ -92,6 +92,77 @@ void main() { expect(result.applied.single.packageId, 'android/com.example.autogen'); }); + test('native legacy import and reports parse getter envelopes', () async { + final calls = []; + TestDefaultBinaryMessengerBinding.instance.defaultBinaryMessenger + .setMockMethodCallHandler(channel, (call) async { + calls.add(call); + switch (call.method) { + case 'importLegacyRoomDatabase': + return jsonEncode({ + 'ok': true, + 'command': 'legacy import-room-db', + 'data': { + 'imported_records': 1, + 'apps': [ + { + 'id': 'android/org.fdroid.fdroid', + 'enabled': true, + 'favorite': true, + 'ignored_version': '1.20.0', + 'repository_id': null, + 'package_resolution': 'missing_package_definition', + }, + ], + 'warnings': [], + 'source_counts': { + 'app_rows': 1, + 'extra_app_rows': 1, + 'hub_rows': 0, + 'extra_hub_rows': 0, + }, + }, + 'warnings': [], + }); + case 'legacyReportList': + return jsonEncode({ + 'ok': true, + 'command': 'legacy report-list', + 'data': { + 'reports': [ + { + 'ok': true, + 'code': 'migration.imported', + 'message': 'Legacy Room data imported', + 'imported_records': 1, + 'tracked_records': 1, + }, + ], + }, + 'warnings': [], + }); + default: + fail('unexpected method ${call.method}'); + } + }); + + const adapter = MethodChannelGetterAdapter(channel: channel); + final importResult = + await adapter.importLegacyRoomDatabase('/tmp/legacy.db'); + final reports = await adapter.readMigrationReports(); + + expect(calls.map((call) => call.method), [ + 'importLegacyRoomDatabase', + 'legacyReportList', + ]); + expect(calls.first.arguments, { + 'database_path': '/tmp/legacy.db', + }); + expect(importResult.importedRecords, 1); + expect(importResult.trackedPackages.single.id, 'android/org.fdroid.fdroid'); + expect(reports.single.code, 'migration.imported'); + }); + test('native adapter maps getter error envelope to bridge exception', () async { TestDefaultBinaryMessengerBinding.instance.defaultBinaryMessenger diff --git a/app_flutter/test/widget_test.dart b/app_flutter/test/widget_test.dart index 83b9716f..d9dbdf51 100644 --- a/app_flutter/test/widget_test.dart +++ b/app_flutter/test/widget_test.dart @@ -282,7 +282,9 @@ class _MigrationGetterAdapter extends FakeGetterAdapter { var _reports = const []; @override - LegacyMigrationImportResult importLegacyRoomDatabase(String databasePath) { + Future importLegacyRoomDatabase( + String databasePath, + ) async { importedDatabasePath = databasePath; _reports = const [ MigrationReportSummary( @@ -317,7 +319,7 @@ class _MigrationGetterAdapter extends FakeGetterAdapter { } @override - List readMigrationReports() => _reports; + Future> readMigrationReports() async => _reports; } class _PreparedLegacyMigrationPlatform implements LegacyMigrationPlatform { diff --git a/core-getter/consumer-rules.pro b/core-getter/consumer-rules.pro index ed547b69..56e59b64 100644 --- a/core-getter/consumer-rules.pro +++ b/core-getter/consumer-rules.pro @@ -1,2 +1,5 @@ +# Flutter/Kotlin calls these JNI entrypoints by their native method names. +-keep class net.xzos.upgradeall.getter.NativeLib { *; } + # Rust JNI loads this provider reflectively through the app classloader. -keep class net.xzos.upgradeall.getter.platform.** { *; } diff --git a/core-getter/src/main/java/net/xzos/upgradeall/getter/NativeLib.kt b/core-getter/src/main/java/net/xzos/upgradeall/getter/NativeLib.kt index adc71ceb..e064a726 100644 --- a/core-getter/src/main/java/net/xzos/upgradeall/getter/NativeLib.kt +++ b/core-getter/src/main/java/net/xzos/upgradeall/getter/NativeLib.kt @@ -18,6 +18,8 @@ class NativeLib { external fun initializeBridge(context: Context): String external fun previewInstalledAutogen(context: Context, requestJson: String): String external fun applyInstalledAutogen(requestJson: String): String + external fun importLegacyRoomDatabase(requestJson: String): String + external fun legacyReportList(requestJson: String): String fun runServerLambda(context: Context, callback: (String) -> Unit): String { return runServer(context, RunServerCallback(callback)) diff --git a/core-getter/src/main/rust/api_proxy/src/lib.rs b/core-getter/src/main/rust/api_proxy/src/lib.rs index 8aad40ae..7c1df8a1 100644 --- a/core-getter/src/main/rust/api_proxy/src/lib.rs +++ b/core-getter/src/main/rust/api_proxy/src/lib.rs @@ -1,6 +1,7 @@ extern crate jni; use getter::operations::autogen::{self, AutogenAcceptance, AutogenOperationError}; +use getter::operations::legacy_room::{self, LegacyRoomOperationError}; use getter::rpc::server::run_server_hanging; #[cfg(target_os = "android")] use getter::rustls_platform_verifier; @@ -33,6 +34,17 @@ struct ApplyInstalledAutogenRequest { acceptance: ApplyInstalledAutogenAcceptance, } +#[derive(Debug, Deserialize)] +struct ImportLegacyRoomDatabaseRequest { + data_dir: PathBuf, + database_path: PathBuf, +} + +#[derive(Debug, Deserialize)] +struct LegacyReportListRequest { + data_dir: PathBuf, +} + #[derive(Debug, Default, Deserialize)] struct ApplyInstalledAutogenAcceptance { #[serde(default)] @@ -166,6 +178,37 @@ pub extern "C" fn Java_net_xzos_upgradeall_getter_NativeLib_applyInstalledAutoge java_string_or_fallback(&mut env, response) } +#[no_mangle] +pub extern "C" fn Java_net_xzos_upgradeall_getter_NativeLib_importLegacyRoomDatabase<'local>( + mut env: JNIEnv<'local>, + _: JObject<'local>, + request_json: JString<'local>, +) -> JString<'local> { + let command = "legacy import-room-db"; + let response = match jstring_to_string(&mut env, &request_json) + .and_then(|raw| import_legacy_room_database(&raw)) + { + Ok(data) => success_envelope(command, data), + Err(error) => operation_error_envelope(command, error), + }; + java_string_or_fallback(&mut env, response) +} + +#[no_mangle] +pub extern "C" fn Java_net_xzos_upgradeall_getter_NativeLib_legacyReportList<'local>( + mut env: JNIEnv<'local>, + _: JObject<'local>, + request_json: JString<'local>, +) -> JString<'local> { + let command = "legacy report-list"; + let response = + match jstring_to_string(&mut env, &request_json).and_then(|raw| legacy_report_list(&raw)) { + Ok(data) => success_envelope(command, data), + Err(error) => operation_error_envelope(command, error), + }; + java_string_or_fallback(&mut env, response) +} + fn preview_installed_autogen( env: &mut JNIEnv<'_>, context: &JObject<'_>, @@ -208,6 +251,19 @@ fn apply_installed_autogen(request_json: &str) -> Result Result { + let request: ImportLegacyRoomDatabaseRequest = serde_json::from_str(request_json) + .map_err(|source| BridgeOperationError::InvalidRequest(source.to_string()))?; + legacy_room::import_room_db_json(&request.data_dir, &request.database_path) + .map_err(BridgeOperationError::from) +} + +fn legacy_report_list(request_json: &str) -> Result { + let request: LegacyReportListRequest = serde_json::from_str(request_json) + .map_err(|source| BridgeOperationError::InvalidRequest(source.to_string()))?; + legacy_room::report_list_json(&request.data_dir).map_err(BridgeOperationError::from) +} + impl ApplyInstalledAutogenAcceptance { fn into_autogen_acceptance(self) -> Result { match self.mode.as_deref().unwrap_or("all") { @@ -337,6 +393,8 @@ enum BridgeOperationError { Repository(String), #[error("autogen error: {0}")] Autogen(String), + #[error("migration error: {0}")] + Migration(#[from] LegacyRoomOperationError), } impl BridgeOperationError { @@ -401,6 +459,13 @@ impl BridgeOperationError { "Getter autogen operation failed", Some(detail), ), + Self::Migration(error) => ( + error.code(), + error.message(), + error + .detail() + .or_else(|| error.report_path().map(|path| path.display().to_string())), + ), } } } diff --git a/core-getter/src/main/rust/getter b/core-getter/src/main/rust/getter index cee3b007..21dc994f 160000 --- a/core-getter/src/main/rust/getter +++ b/core-getter/src/main/rust/getter @@ -1 +1 @@ -Subproject commit cee3b007d5e03d9035f5319a582f452dfbc7c911 +Subproject commit 21dc994f5d452238a0b5fda3ecf9cd09e541155a diff --git a/docs/architecture/adr/0003-legacy-room-migration.md b/docs/architecture/adr/0003-legacy-room-migration.md index 48c7c208..347b2509 100644 --- a/docs/architecture/adr/0003-legacy-room-migration.md +++ b/docs/architecture/adr/0003-legacy-room-migration.md @@ -65,7 +65,7 @@ getter --data-dir legacy import-room-db The direct importer opens the DB read-only, requires `PRAGMA user_version = 17`, reads legacy `app` and `extra_app` rows, maps known app-id keys to `android/` or `magisk/`, writes getter tracked package state plus the `legacy-room-v17` migration record in one transaction, and emits sanitized report counts/warnings. Current `hub` and `extra_hub` rows are not imported as top-level objects; they are counted/dropped with warnings until a later accepted mapping exists. -The first Flutter/Android migration UX slice adds a no-UI Android platform adapter that locates `app_metadata_database.db`, copies the SQLite triplet (`.db`, `-wal`, `-shm`) into an app-private getter-import path, checkpoints/canonicalizes the copy, and returns that copied DB path to Flutter. Flutter starts the flow and renders getter-owned reports. Getter still owns the actual import operation; the default product APK keeps the action disabled until the production getter import bridge is connected. +The first Flutter/Android migration UX slice adds a no-UI Android platform adapter that locates `app_metadata_database.db`, copies the SQLite triplet (`.db`, `-wal`, `-shm`) into an app-private getter-import path, checkpoints/canonicalizes the copy, and returns that copied DB path to Flutter. A follow-up production bridge slice wires Flutter to the native getter bridge for `importLegacyRoomDatabase` and `legacyReportList`; Flutter starts the flow and renders getter-owned reports, while Rust getter owns the actual import operation and Room-row mapping. The host-side CLI also keeps the deterministic JSON bridge bundle for tests and non-Android fixtures: diff --git a/docs/architecture/adr/0007-flutter-getter-bridge-contract.md b/docs/architecture/adr/0007-flutter-getter-bridge-contract.md index c0361f88..66d49ad7 100644 --- a/docs/architecture/adr/0007-flutter-getter-bridge-contract.md +++ b/docs/architecture/adr/0007-flutter-getter-bridge-contract.md @@ -43,7 +43,7 @@ The third accepted API surface adds the first legacy migration action boundary: importLegacyRoomDatabase(databasePath) ``` -The Android platform adapter may prepare a copied/checkpointed legacy Room SQLite file and return its path to Flutter, but getter still owns the actual `legacy import-room-db` import semantics. Flutter starts the flow and renders getter reports; it must not inspect or map Room tables directly. +The Android platform adapter may prepare a copied/checkpointed legacy Room SQLite file and return its path to Flutter, but getter still owns the actual `legacy import-room-db` import semantics. The production Android bridge exposes `importLegacyRoomDatabase` and `legacyReportList` through JNI/MethodChannel by delegating to getter-owned `getter-operations` legacy Room code. Flutter starts the flow and renders getter reports; it must not inspect or map Room tables directly. The fourth accepted API surface adds the production installed-autogen bridge boundary and must follow ADR-0009's Rust-active platform adapter direction rather than a Flutter-led inventory scan: @@ -52,7 +52,7 @@ previewInstalledAutogen(scanOptions) applyInstalledAutogen(preview, acceptedPackages) ``` -The Android product APK packages a slim `:getter_bridge` library under `app_flutter/android/getter_bridge`. It builds the Rust `api_proxy` cdylib and includes only the no-UI native bridge / installed-inventory provider classes needed by the Flutter product path, avoiding the legacy native `:app` UI and old `GetterPort` hub/RPC wrapper surface. `MainActivity` exposes a no-UI `net.xzos.upgradeall/getter_bridge` MethodChannel that derives the app-private getter data directory and forwards preview/apply requests to JNI entrypoints returning getter-style JSON envelopes. +The Android product APK packages a slim `:getter_bridge` library under `app_flutter/android/getter_bridge`. It builds the Rust `api_proxy` cdylib and includes only the no-UI native bridge / installed-inventory provider classes needed by the Flutter product path, avoiding the legacy native `:app` UI and old `GetterPort` hub/RPC wrapper surface. `MainActivity` exposes a no-UI `net.xzos.upgradeall/getter_bridge` MethodChannel that derives the app-private getter data directory and forwards migration and installed-autogen requests to JNI entrypoints returning getter-style JSON envelopes. Internally, Rust/native bridge code scans Android inventory through the platform adapter, then asks getter-owned shared autogen operations to plan/apply `local_autogen`. `MethodChannelGetterAdapter` consumes the returned getter-style JSON envelopes. Flutter renders getter-owned preview/apply DTOs and scan diagnostics, then passes the package ids from displayed accepted preview candidates back to getter on apply; it must not expose a product Dart `InstalledInventoryPlatform` scanner or convert Android package names into package ids. @@ -201,7 +201,7 @@ The first implementation slice must provide: ## Non-goals -- No full FFI/native bridge implementation beyond the first installed-autogen preview/apply JNI/MethodChannel operation slice. +- No full FFI/native bridge implementation beyond the first direct legacy Room import/report-list and installed-autogen preview/apply JNI/MethodChannel operation slices. - No update/download/install event stream. - No Android-owned legacy Room mapping/import semantics; Android only prepares a copied DB file for getter. - No product-complete Flutter UI. diff --git a/docs/architecture/upgradeall-getter-rewrite-wiki.md b/docs/architecture/upgradeall-getter-rewrite-wiki.md index 4e195fde..c397b39e 100644 --- a/docs/architecture/upgradeall-getter-rewrite-wiki.md +++ b/docs/architecture/upgradeall-getter-rewrite-wiki.md @@ -1030,6 +1030,8 @@ Room DB 信息： - 单个 package 无法匹配不应阻塞整个 App。 - 该 package 显示 missing/needs package script 状态。 +实现进展：Android/Flutter 侧已有 no-UI legacy migration adapter 负责定位、复制并 checkpoint 旧 Room SQLite triplet；Flutter 产品 APK 通过 slim getter/native bridge 调用 Rust `importLegacyRoomDatabase` / `legacyReportList`。Room 表读取、字段映射、migration record、tracked package 写入和 sanitized report 仍由 getter-owned Rust code 完成，Flutter/Kotlin 不解析 Room 行。 + --- ## 14. Installed autogen UX diff --git a/todo.md b/todo.md index 6eb50290..f9ab0e5e 100644 --- a/todo.md +++ b/todo.md @@ -380,7 +380,7 @@ Remaining follow-up: Goal: replace bridge-only JSON import with the Android upgrade path. -Status: first getter-owned direct DB slice completed, and the first Flutter/platform UX slice is in progress. The getter CLI now supports `legacy import-room-db ` for copied/checkpointed Room v17 SQLite files. It reads `app` and `extra_app`, maps known legacy app-id keys, writes `tracked_packages` plus `legacy-room-v17` in one transaction, prevents rerun, emits sanitized reports, and documents dropped hub/extra_hub fields. Android-side code now exposes a no-UI MethodChannel adapter that locates, copies, and checkpoints the legacy SQLite triplet for Flutter to pass into getter. Flutter has a migration page that starts this adapter flow and renders getter reports, but the default product APK keeps the action disabled until the production getter import bridge replaces the dev CLI adapter. +Status: getter-owned direct DB and production bridge slices are implemented. The getter CLI supports `legacy import-room-db ` for copied/checkpointed Room v17 SQLite files. It reads `app` and `extra_app`, maps known legacy app-id keys, writes `tracked_packages` plus `legacy-room-v17` in one transaction, prevents rerun, emits sanitized reports, and documents dropped hub/extra_hub fields. Android-side code exposes a no-UI MethodChannel adapter that locates, copies, and checkpoints the legacy SQLite triplet. The Flutter product APK now uses the native getter bridge for `importLegacyRoomDatabase` and `legacyReportList`; Flutter starts the adapter flow and renders getter-owned results/reports without mapping Room rows in Dart/Kotlin. Completed tasks: @@ -391,10 +391,16 @@ Completed tasks: 5. Reports are sanitized and visible through `legacy report-list`. 6. Dropped `hub`/`extra_hub` fields are documented in `docs/migration/legacy-room-mapping.md`. +Completed additional bridge tasks: + +7. Extract direct Room DB import/report-list behavior into reusable getter-owned `getter-operations` code shared by CLI and native bridge. +8. Wire production native bridge operations for `importLegacyRoomDatabase` and `legacyReportList` into the Flutter APK. +9. Enable the product migration page through `MethodChannelGetterAdapter` while keeping Flutter as DTO/rendering glue. + Remaining tasks: -1. Wire a production getter bridge for `importLegacyRoomDatabase` in the Flutter APK; until then the default product migration action stays disabled with an explicit bridge-unavailable state. -2. Add focused native adapter coverage for SQLite triplet copy/checkpoint behavior if practical. +1. Add focused native adapter coverage for SQLite triplet copy/checkpoint behavior if practical. +2. Add device/instrumented validation for the full Flutter MethodChannel -> Android copy/checkpoint -> JNI -> Rust getter import path if practical. 3. Extend accepted mapping if future ADR accepts direct `hub`/`extra_hub` semantics. Acceptance progress: From 044ca723434a7e64ddd6c0561024d7e46ca9f733 Mon Sep 17 00:00:00 2001 From: Xiangzhe Date: Wed, 24 Jun 2026 10:29:32 +0800 Subject: [PATCH 26/85] ci: set Android ar for cargo checks --- .github/workflows/android.yml | 1 + .github/workflows/upgradeall-rewrite-validation.yml | 1 + justfile | 2 +- 3 files changed, 3 insertions(+), 1 deletion(-) diff --git a/.github/workflows/android.yml b/.github/workflows/android.yml index 8c01b562..a712b6f7 100644 --- a/.github/workflows/android.yml +++ b/.github/workflows/android.yml @@ -43,6 +43,7 @@ jobs: echo "ANDROID_NDK_HOME=$NDK_HOME" >> "$GITHUB_ENV" echo "ANDROID_NDK_ROOT=$NDK_HOME" >> "$GITHUB_ENV" echo "CC_aarch64_linux_android=$NDK_HOME/toolchains/llvm/prebuilt/linux-x86_64/bin/aarch64-linux-android23-clang" >> "$GITHUB_ENV" + echo "AR_aarch64_linux_android=$NDK_HOME/toolchains/llvm/prebuilt/linux-x86_64/bin/llvm-ar" >> "$GITHUB_ENV" echo "Android build tools: $BUILD_TOOL_VERSION" echo "Android NDK: $NDK_HOME" diff --git a/.github/workflows/upgradeall-rewrite-validation.yml b/.github/workflows/upgradeall-rewrite-validation.yml index d05375d4..288b81d7 100644 --- a/.github/workflows/upgradeall-rewrite-validation.yml +++ b/.github/workflows/upgradeall-rewrite-validation.yml @@ -38,6 +38,7 @@ jobs: echo "ANDROID_NDK_HOME=$NDK_HOME" >> "$GITHUB_ENV" echo "ANDROID_NDK_ROOT=$NDK_HOME" >> "$GITHUB_ENV" echo "CC_aarch64_linux_android=$NDK_HOME/toolchains/llvm/prebuilt/linux-x86_64/bin/aarch64-linux-android23-clang" >> "$GITHUB_ENV" + echo "AR_aarch64_linux_android=$NDK_HOME/toolchains/llvm/prebuilt/linux-x86_64/bin/llvm-ar" >> "$GITHUB_ENV" - name: Install Flutter uses: subosito/flutter-action@v2 diff --git a/justfile b/justfile index 3c6cdf19..c370c3e3 100644 --- a/justfile +++ b/justfile @@ -47,7 +47,7 @@ verify-workspace-skeleton: cargo fmt --manifest-path {{ PLATFORM_ADAPTER_MANIFEST }} --all --check cargo check --manifest-path {{ GETTER_MANIFEST }} --workspace --all-targets cargo check --manifest-path {{ API_PROXY_MANIFEST }} - if [ -n "${ANDROID_NDK_HOME:-}" ]; then export CC_aarch64_linux_android="$ANDROID_NDK_HOME/toolchains/llvm/prebuilt/linux-x86_64/bin/aarch64-linux-android23-clang"; fi; cargo check --manifest-path {{ API_PROXY_MANIFEST }} --target aarch64-linux-android + if [ -n "${ANDROID_NDK_HOME:-}" ]; then export CC_aarch64_linux_android="$ANDROID_NDK_HOME/toolchains/llvm/prebuilt/linux-x86_64/bin/aarch64-linux-android23-clang"; export AR_aarch64_linux_android="$ANDROID_NDK_HOME/toolchains/llvm/prebuilt/linux-x86_64/bin/llvm-ar"; fi; cargo check --manifest-path {{ API_PROXY_MANIFEST }} --target aarch64-linux-android cargo test --manifest-path {{ PLATFORM_ADAPTER_MANIFEST }} cargo check --manifest-path {{ PLATFORM_ADAPTER_MANIFEST }} --target aarch64-linux-android cd app_flutter && flutter analyze From 65df6ab9131cee65fe06203b6d49832a5b2bf16d Mon Sep 17 00:00:00 2001 From: Xiangzhe Date: Wed, 24 Jun 2026 11:19:19 +0800 Subject: [PATCH 27/85] test(app): cover legacy room preparation --- app_flutter/android/app/build.gradle | 1 + .../upgradeall/LegacyRoomImportPreparer.kt | 86 +++++++++++++++++++ .../net/xzos/upgradeall/MainActivity.kt | 59 +------------ .../LegacyRoomImportPreparerTest.kt | 81 +++++++++++++++++ justfile | 4 + todo.md | 7 +- 6 files changed, 178 insertions(+), 60 deletions(-) create mode 100644 app_flutter/android/app/src/main/kotlin/net/xzos/upgradeall/LegacyRoomImportPreparer.kt create mode 100644 app_flutter/android/app/src/test/kotlin/net/xzos/upgradeall/LegacyRoomImportPreparerTest.kt diff --git a/app_flutter/android/app/build.gradle b/app_flutter/android/app/build.gradle index 5bc415de..5adaebb5 100644 --- a/app_flutter/android/app/build.gradle +++ b/app_flutter/android/app/build.gradle @@ -88,4 +88,5 @@ flutter { dependencies { implementation project(':getter_bridge') + testImplementation "junit:junit:4.13.2" } diff --git a/app_flutter/android/app/src/main/kotlin/net/xzos/upgradeall/LegacyRoomImportPreparer.kt b/app_flutter/android/app/src/main/kotlin/net/xzos/upgradeall/LegacyRoomImportPreparer.kt new file mode 100644 index 00000000..079cf2d4 --- /dev/null +++ b/app_flutter/android/app/src/main/kotlin/net/xzos/upgradeall/LegacyRoomImportPreparer.kt @@ -0,0 +1,86 @@ +package net.xzos.upgradeall + +import android.database.sqlite.SQLiteDatabase +import java.io.File + +internal data class PreparedLegacyRoomImport( + val found: Boolean, + val databasePath: String?, + val message: String, +) { + fun toMethodChannelResult(): Map = mapOf( + "found" to found, + "database_path" to databasePath, + "message" to message, + ) +} + +internal fun interface CopiedDatabaseCheckpointer { + fun checkpoint(database: File) +} + +internal class LegacyRoomImportPreparer( + private val checkpointer: CopiedDatabaseCheckpointer = AndroidSqliteCopiedDatabaseCheckpointer(), +) { + fun prepare(source: File, destination: File): PreparedLegacyRoomImport { + if (!source.exists()) { + return PreparedLegacyRoomImport( + found = false, + databasePath = null, + message = "No legacy Room database found", + ) + } + + copySqliteTriplet(source, destination) + checkpointer.checkpoint(destination) + + return PreparedLegacyRoomImport( + found = true, + databasePath = destination.absolutePath, + message = "Legacy Room database prepared", + ) + } + + private fun copySqliteTriplet(source: File, destination: File) { + destination.parentFile?.mkdirs() + SQLITE_SUFFIXES.forEach { suffix -> + val sourceFile = File(source.path + suffix) + val destinationFile = File(destination.path + suffix) + if (sourceFile.exists()) { + sourceFile.copyTo(destinationFile, overwrite = true) + } else if (destinationFile.exists()) { + destinationFile.delete() + } + } + } + + private companion object { + val SQLITE_SUFFIXES = listOf("", "-wal", "-shm") + } +} + +internal class AndroidSqliteCopiedDatabaseCheckpointer : CopiedDatabaseCheckpointer { + override fun checkpoint(database: File) { + val db = SQLiteDatabase.openDatabase( + database.path, + null, + SQLiteDatabase.OPEN_READWRITE, + ) + try { + db.rawQuery("PRAGMA wal_checkpoint(FULL)", null).use { cursor -> + while (cursor.moveToNext()) { + // Drain the pragma result so SQLite performs the checkpoint. + } + } + db.rawQuery("PRAGMA journal_mode=DELETE", null).use { cursor -> + while (cursor.moveToNext()) { + // Drain the pragma result and leave a standalone import DB. + } + } + } finally { + db.close() + } + File(database.path + "-wal").delete() + File(database.path + "-shm").delete() + } +} diff --git a/app_flutter/android/app/src/main/kotlin/net/xzos/upgradeall/MainActivity.kt b/app_flutter/android/app/src/main/kotlin/net/xzos/upgradeall/MainActivity.kt index b3858252..23f35a4d 100644 --- a/app_flutter/android/app/src/main/kotlin/net/xzos/upgradeall/MainActivity.kt +++ b/app_flutter/android/app/src/main/kotlin/net/xzos/upgradeall/MainActivity.kt @@ -1,6 +1,5 @@ package net.xzos.upgradeall -import android.database.sqlite.SQLiteDatabase import android.os.Handler import android.os.Looper import io.flutter.embedding.android.FlutterActivity @@ -172,70 +171,18 @@ class MainActivity : FlutterActivity() { private fun getterDataDir(): File = File(filesDir, "getter") private fun prepareLegacyRoomImport(): Map { - val source = getDatabasePath(LEGACY_ROOM_DB_NAME) - if (!source.exists()) { - return mapOf( - "found" to false, - "database_path" to null, - "message" to "No legacy Room database found", - ) - } - val destination = File( File(filesDir, "getter-imports/legacy-room"), LEGACY_ROOM_DB_NAME, ) - copySqliteTriplet(source, destination) - checkpointCopiedDatabase(destination) - - return mapOf( - "found" to true, - "database_path" to destination.absolutePath, - "message" to "Legacy Room database prepared", - ) - } - - private fun copySqliteTriplet(source: File, destination: File) { - destination.parentFile?.mkdirs() - SQLITE_SUFFIXES.forEach { suffix -> - val sourceFile = File(source.path + suffix) - val destinationFile = File(destination.path + suffix) - if (sourceFile.exists()) { - sourceFile.copyTo(destinationFile, overwrite = true) - } else if (destinationFile.exists()) { - destinationFile.delete() - } - } - } - - private fun checkpointCopiedDatabase(database: File) { - val db = SQLiteDatabase.openDatabase( - database.path, - null, - SQLiteDatabase.OPEN_READWRITE, - ) - try { - db.rawQuery("PRAGMA wal_checkpoint(FULL)", null).use { cursor -> - while (cursor.moveToNext()) { - // Drain the pragma result so SQLite performs the checkpoint. - } - } - db.rawQuery("PRAGMA journal_mode=DELETE", null).use { cursor -> - while (cursor.moveToNext()) { - // Drain the pragma result and leave a standalone import DB. - } - } - } finally { - db.close() - } - File(database.path + "-wal").delete() - File(database.path + "-shm").delete() + return LegacyRoomImportPreparer() + .prepare(getDatabasePath(LEGACY_ROOM_DB_NAME), destination) + .toMethodChannelResult() } private companion object { const val GETTER_BRIDGE_CHANNEL = "net.xzos.upgradeall/getter_bridge" const val LEGACY_MIGRATION_CHANNEL = "net.xzos.upgradeall/legacy_migration" const val LEGACY_ROOM_DB_NAME = "app_metadata_database.db" - val SQLITE_SUFFIXES = listOf("", "-wal", "-shm") } } diff --git a/app_flutter/android/app/src/test/kotlin/net/xzos/upgradeall/LegacyRoomImportPreparerTest.kt b/app_flutter/android/app/src/test/kotlin/net/xzos/upgradeall/LegacyRoomImportPreparerTest.kt new file mode 100644 index 00000000..1cb03d64 --- /dev/null +++ b/app_flutter/android/app/src/test/kotlin/net/xzos/upgradeall/LegacyRoomImportPreparerTest.kt @@ -0,0 +1,81 @@ +package net.xzos.upgradeall + +import java.io.File +import org.junit.Assert.assertEquals +import org.junit.Assert.assertFalse +import org.junit.Assert.assertNull +import org.junit.Assert.assertTrue +import org.junit.Rule +import org.junit.Test +import org.junit.rules.TemporaryFolder + +class LegacyRoomImportPreparerTest { + @get:Rule + val temp = TemporaryFolder() + + @Test + fun missingSourceReturnsNotFoundAndDoesNotCheckpoint() { + val source = File(temp.root, "source/app_metadata_database.db") + val destination = File(temp.root, "destination/app_metadata_database.db") + val checkpointer = RecordingCheckpointer() + + val result = LegacyRoomImportPreparer(checkpointer).prepare(source, destination) + + assertFalse(result.found) + assertNull(result.databasePath) + assertEquals("No legacy Room database found", result.message) + assertTrue(checkpointer.databases.isEmpty()) + assertFalse(destination.exists()) + } + + @Test + fun copiesExistingSqliteTripletAndCallsCheckpoint() { + val source = File(temp.root, "source/app_metadata_database.db") + val destination = File(temp.root, "destination/app_metadata_database.db") + source.writeTextWithParents("db") + File(source.path + "-wal").writeTextWithParents("wal") + File(source.path + "-shm").writeTextWithParents("shm") + val checkpointer = RecordingCheckpointer() + + val result = LegacyRoomImportPreparer(checkpointer).prepare(source, destination) + + assertTrue(result.found) + assertEquals(destination.absolutePath, result.databasePath) + assertEquals("Legacy Room database prepared", result.message) + assertEquals("db", destination.readText()) + assertEquals("wal", File(destination.path + "-wal").readText()) + assertEquals("shm", File(destination.path + "-shm").readText()) + assertEquals(listOf(destination), checkpointer.databases) + } + + @Test + fun removesStaleDestinationSidecarsWhenSourceSidecarsAreAbsent() { + val source = File(temp.root, "source/app_metadata_database.db") + val destination = File(temp.root, "destination/app_metadata_database.db") + source.writeTextWithParents("fresh-db") + destination.writeTextWithParents("old-db") + File(destination.path + "-wal").writeTextWithParents("stale-wal") + File(destination.path + "-shm").writeTextWithParents("stale-shm") + val checkpointer = RecordingCheckpointer() + + LegacyRoomImportPreparer(checkpointer).prepare(source, destination) + + assertEquals("fresh-db", destination.readText()) + assertFalse(File(destination.path + "-wal").exists()) + assertFalse(File(destination.path + "-shm").exists()) + assertEquals(listOf(destination), checkpointer.databases) + } + + private fun File.writeTextWithParents(text: String) { + parentFile?.mkdirs() + writeText(text) + } + + private class RecordingCheckpointer : CopiedDatabaseCheckpointer { + val databases = mutableListOf() + + override fun checkpoint(database: File) { + databases.add(database) + } + } +} diff --git a/justfile b/justfile index c370c3e3..34b5e9f7 100644 --- a/justfile +++ b/justfile @@ -10,6 +10,7 @@ verify: just test-flutter-widget just verify-workspace-skeleton just test-android-platform-adapter + just test-flutter-android-platform-adapter just test-flutter-getter-cli-integration just build-flutter-android-debug @@ -34,6 +35,9 @@ test-flutter-getter-cli-integration: test-android-platform-adapter: ./gradlew --no-daemon ':core-getter:buildDebugApi_proxyRust[arm64-v8a]' ':core-getter:buildDebugApi_proxyRust[armeabi-v7a]' ':core-getter:buildDebugApi_proxyRust[x86_64]' :core-getter:testDebugUnitTest --tests 'net.xzos.upgradeall.getter.platform.InstalledInventoryCollectorTest' :core-getter:assembleDebug +test-flutter-android-platform-adapter: + cd app_flutter/android && ./gradlew --no-daemon :app:testDebugUnitTest --tests 'net.xzos.upgradeall.LegacyRoomImportPreparerTest' + build-flutter-android-debug: cd app_flutter && flutter build apk --debug python3 tools/verify_flutter_apk_bridge.py app_flutter/build/app/outputs/flutter-apk/app-debug.apk diff --git a/todo.md b/todo.md index f9ab0e5e..f1b3b522 100644 --- a/todo.md +++ b/todo.md @@ -399,9 +399,8 @@ Completed additional bridge tasks: Remaining tasks: -1. Add focused native adapter coverage for SQLite triplet copy/checkpoint behavior if practical. -2. Add device/instrumented validation for the full Flutter MethodChannel -> Android copy/checkpoint -> JNI -> Rust getter import path if practical. -3. Extend accepted mapping if future ADR accepts direct `hub`/`extra_hub` semantics. +1. Add device/instrumented validation for the full Flutter MethodChannel -> Android copy/checkpoint -> JNI -> Rust getter import path if practical. +2. Extend accepted mapping if future ADR accepts direct `hub`/`extra_hub` semantics. Acceptance progress: @@ -412,7 +411,7 @@ Acceptance progress: - Mixed valid/invalid app rows import valid rows and warn: done. - DBs with app rows but zero importable rows fail with recovery report: done. - Report sanitization for dropped `hub`/`extra_hub` secrets and URL rewrite data: done. -- WAL/SHM pending writes: first Android adapter copy/checkpoint slice implemented; focused native adapter test still pending. +- WAL/SHM pending writes: first Android adapter copy/checkpoint slice implemented; focused JVM native-adapter tests now cover triplet copy, stale sidecar cleanup, missing DB behavior, and checkpointer invocation. Real Android SQLite checkpoint behavior still needs device/instrumented coverage. - Per-app failures become warnings; global unreadable DB becomes recovery state, not crash: done for the getter-owned direct importer. ### Phase B: `local_autogen` generation From c0f67522c220bcf0c0e570e392569257b74f61ab Mon Sep 17 00:00:00 2001 From: Xiangzhe Date: Wed, 24 Jun 2026 13:55:58 +0800 Subject: [PATCH 28/85] test(app): validate native bridge on device --- app_flutter/README.md | 12 +- app_flutter/android/app/build.gradle | 2 +- .../android/app/src/debug/AndroidManifest.xml | 14 +++ .../app_metadata_database.db | Bin 0 -> 4096 bytes .../app_metadata_database.db-shm | Bin 0 -> 32768 bytes .../app_metadata_database.db-wal | Bin 0 -> 45352 bytes .../integration_test/native_bridge_test.dart | 114 ++++++++++++++++++ app_flutter/pubspec.lock | 63 ++++++++++ app_flutter/pubspec.yaml | 7 ++ justfile | 3 + todo.md | 9 +- 11 files changed, 217 insertions(+), 7 deletions(-) create mode 100644 app_flutter/android/app/src/debug/AndroidManifest.xml create mode 100644 app_flutter/integration_test/fixtures/legacy_room_v17_wal/app_metadata_database.db create mode 100644 app_flutter/integration_test/fixtures/legacy_room_v17_wal/app_metadata_database.db-shm create mode 100644 app_flutter/integration_test/fixtures/legacy_room_v17_wal/app_metadata_database.db-wal create mode 100644 app_flutter/integration_test/native_bridge_test.dart diff --git a/app_flutter/README.md b/app_flutter/README.md index 0f6e48c4..109a046a 100644 --- a/app_flutter/README.md +++ b/app_flutter/README.md @@ -24,6 +24,16 @@ flutter test GETTER_CLI_BIN=/path/to/getter-cli flutter test dev_test/cli_getter_adapter_test.dart ``` -From the repository root, `just verify` also runs the Flutter analyzer, widget tests, getter CLI integration/dev test, Android debug build, and an APK inspection that verifies the Flutter APK contains `libapi_proxy.so`, `NativeLib`, and `InstalledInventoryProvider`. +Device/emulator bridge validation is available when an Android device is attached: + +```bash +flutter test integration_test/native_bridge_test.dart -d emulator-5554 +# or, from the repository root: +just test-flutter-device-bridge emulator-5554 +``` + +The device bridge test exercises the production MethodChannel/JNI path for copied legacy Room import/report-list and installed-autogen preview/apply. If using the local `Pixel_9a` AVD, start it with enough memory (for example `-memory 4096`) so the Flutter debug VM is not killed by Android low-memory pressure. + +From the repository root, `just verify` also runs the Flutter analyzer, widget tests, getter CLI integration/dev test, Android debug build, and an APK inspection that verifies the Flutter APK contains `libapi_proxy.so`, `NativeLib`, and `InstalledInventoryProvider`. `just verify` intentionally does not require an attached device; use `just test-flutter-device-bridge` for the emulator-only path. Android CI/release artifacts are built from this Flutter project with `flutter build apk`; the root Gradle `:app` module is no longer the rewrite product APK path. diff --git a/app_flutter/android/app/build.gradle b/app_flutter/android/app/build.gradle index 5adaebb5..c7501bb0 100644 --- a/app_flutter/android/app/build.gradle +++ b/app_flutter/android/app/build.gradle @@ -33,7 +33,7 @@ if (keystorePropertiesFile.exists()) { android { namespace "net.xzos.upgradeall" compileSdkVersion 36 - ndkVersion flutter.ndkVersion + ndkVersion "29.0.14206865" compileOptions { sourceCompatibility JavaVersion.VERSION_17 diff --git a/app_flutter/android/app/src/debug/AndroidManifest.xml b/app_flutter/android/app/src/debug/AndroidManifest.xml new file mode 100644 index 00000000..457b5089 --- /dev/null +++ b/app_flutter/android/app/src/debug/AndroidManifest.xml @@ -0,0 +1,14 @@ + + + + + + + diff --git a/app_flutter/integration_test/fixtures/legacy_room_v17_wal/app_metadata_database.db b/app_flutter/integration_test/fixtures/legacy_room_v17_wal/app_metadata_database.db new file mode 100644 index 0000000000000000000000000000000000000000..4a3492ca9ee32d110ce3484c72386f9ff8651a2c GIT binary patch literal 4096 zcmWFz^vNtqRY=P(%1ta$FlG>7U}9o$P*7lCU|@t|AVoG{WYC*>iWj6^fNV2HHI9bB nXb6mkz-S1JhQMeDjE2By2#kinXb6mkz-S1JhQMeDP#6LLGPMVA literal 0 HcmV?d00001 diff --git a/app_flutter/integration_test/fixtures/legacy_room_v17_wal/app_metadata_database.db-shm b/app_flutter/integration_test/fixtures/legacy_room_v17_wal/app_metadata_database.db-shm new file mode 100644 index 0000000000000000000000000000000000000000..7a91c4378852f840b058d7bf31e75236ae5268fe GIT binary patch literal 32768 zcmeI)KMnyw6bIn(-_j_R3Z1x;mP#j4D3lTxZ~#Ye4ObvsLxP#jMwFD+_e*A9GCSGX z_Z{H%uj7!?jH?ntuN!M4u1j`1wO9M)d2^iJN88AV7cs0RjXF5FkK+009C72oNAZfB*pk1PBlyK!5-N z0t5&UAV7cs0RjXF5FkK+009C72oNAZfB*pk1PBlyK!5-N0%Z}X#F-NHIQybqmWK2} z;A4SW=zM(EK?3CxXoNKDNC^C+KOpOr*jrVVeVQh=RipceS<<#D=}2TiRm#U_SIFha*GjQD!w4XN00IagfB*sr zAbBxx=G(b%{x>aylwq4$W1$zuuvV4sC<4RjX}q zIcgkE?Xa`1_5G z;f>0)?x1d%?^P<$BTlk+=M)C*YSkYrrpkLv~0d{xv7h(9z05I_I{1Q0*~0R#|0009IRMc{&( zQ&?PV3Dt%3^`5?PGuQ`TNpEuNRQl_*S!C;Pdd+Y47d1 z8tMfWBv^#X&FvtxJH-k(ZQFA!fJAma-0>uLSI zfHm(?xL!caXb2#H00IagfB*srAb-wlk_3%r}p{m?hrZ;!+UW|aJyC2`< report.code), contains('migration.imported')); + }); + + testWidgets('native bridge previews and applies installed autogen for self', + (tester) async { + await _resetAppData(); + + const adapter = MethodChannelGetterAdapter(); + final preview = await adapter.previewInstalledAutogen( + options: const InstalledAutogenScanOptions(includeSelf: true), + ); + + expect(preview.scanStats, isNotNull); + expect(preview.scanStats!.totalSeen, greaterThan(0)); + expect( + preview.candidates.map((candidate) => candidate.packageId), + contains(_selfPackageId), + ); + + final result = await adapter.applyInstalledAutogen( + preview, + acceptedPackageIds: const [_selfPackageId], + ); + + expect(result.appliedCount, 1); + expect( + result.applied.map((package) => package.packageId), + contains(_selfPackageId), + ); + }); +} + +const _debugPackageName = 'net.xzos.upgradeall.debug'; +const _selfPackageId = 'android/$_debugPackageName'; +const _legacyDbName = 'app_metadata_database.db'; +const _legacyFixtureDir = 'integration_test/fixtures/legacy_room_v17_wal'; + +Directory get _packageDataDir => Directory('/data/user/0/$_debugPackageName'); +Directory get _databasesDir => Directory('${_packageDataDir.path}/databases'); +Directory get _filesDir => Directory('${_packageDataDir.path}/files'); +File get _legacyDatabase => File('${_databasesDir.path}/$_legacyDbName'); + +Future _resetAppData() async { + for (final path in [ + '${_filesDir.path}/getter', + '${_filesDir.path}/getter-imports', + ]) { + final dir = Directory(path); + if (await dir.exists()) { + await dir.delete(recursive: true); + } + } + for (final file in [ + _legacyDatabase, + File('${_legacyDatabase.path}-wal'), + File('${_legacyDatabase.path}-shm'), + ]) { + if (await file.exists()) { + await file.delete(); + } + } +} + +Future _installLegacyRoomFixture() async { + await _databasesDir.create(recursive: true); + for (final suffix in ['', '-wal', '-shm']) { + final asset = + await rootBundle.load('$_legacyFixtureDir/$_legacyDbName$suffix'); + await File('${_legacyDatabase.path}$suffix').writeAsBytes( + asset.buffer.asUint8List(asset.offsetInBytes, asset.lengthInBytes), + flush: true, + ); + } +} diff --git a/app_flutter/pubspec.lock b/app_flutter/pubspec.lock index 81c00a98..81104f36 100644 --- a/app_flutter/pubspec.lock +++ b/app_flutter/pubspec.lock @@ -57,11 +57,24 @@ packages: url: "https://pub.dev" source: hosted version: "1.3.1" + file: + dependency: transitive + description: + name: file + sha256: "1b92bec4fc2a72f59a8e15af5f52cd441e4a7860b49499d69dfa817af20e925d" + url: "https://pub.dev" + source: hosted + version: "6.1.4" flutter: dependency: "direct main" description: flutter source: sdk version: "0.0.0" + flutter_driver: + dependency: transitive + description: flutter + source: sdk + version: "0.0.0" flutter_lints: dependency: "direct dev" description: @@ -75,6 +88,16 @@ packages: description: flutter source: sdk version: "0.0.0" + fuchsia_remote_debug_protocol: + dependency: transitive + description: flutter + source: sdk + version: "0.0.0" + integration_test: + dependency: "direct dev" + description: flutter + source: sdk + version: "0.0.0" lints: dependency: transitive description: @@ -115,6 +138,22 @@ packages: url: "https://pub.dev" source: hosted version: "1.8.3" + platform: + dependency: transitive + description: + name: platform + sha256: ae68c7bfcd7383af3629daafb32fb4e8681c7154428da4febcff06200585f102 + url: "https://pub.dev" + source: hosted + version: "3.1.2" + process: + dependency: transitive + description: + name: process + sha256: "53fd8db9cec1d37b0574e12f07520d582019cb6c44abf5479a01505099a34a09" + url: "https://pub.dev" + source: hosted + version: "4.2.4" sky_engine: dependency: transitive description: flutter @@ -152,6 +191,14 @@ packages: url: "https://pub.dev" source: hosted version: "1.2.0" + sync_http: + dependency: transitive + description: + name: sync_http + sha256: "7f0cd72eca000d2e026bcd6f990b81d0ca06022ef4e32fb257b30d3d1014a961" + url: "https://pub.dev" + source: hosted + version: "0.3.1" term_glyph: dependency: transitive description: @@ -176,6 +223,14 @@ packages: url: "https://pub.dev" source: hosted version: "2.1.4" + vm_service: + dependency: transitive + description: + name: vm_service + sha256: c538be99af830f478718b51630ec1b6bee5e74e52c8a802d328d9e71d35d2583 + url: "https://pub.dev" + source: hosted + version: "11.10.0" web: dependency: transitive description: @@ -184,5 +239,13 @@ packages: url: "https://pub.dev" source: hosted version: "0.3.0" + webdriver: + dependency: transitive + description: + name: webdriver + sha256: "3c923e918918feeb90c4c9fdf1fe39220fa4c0e8e2c0fffaded174498ef86c49" + url: "https://pub.dev" + source: hosted + version: "3.0.2" sdks: dart: ">=3.2.3 <4.0.0" diff --git a/app_flutter/pubspec.yaml b/app_flutter/pubspec.yaml index a222b7a8..95bb0810 100644 --- a/app_flutter/pubspec.yaml +++ b/app_flutter/pubspec.yaml @@ -39,6 +39,8 @@ dependencies: dev_dependencies: flutter_test: sdk: flutter + integration_test: + sdk: flutter # The "flutter_lints" package below contains a set of recommended lints to # encourage good coding practices. The lint set provided by the package is @@ -58,6 +60,11 @@ flutter: # the material Icons class. uses-material-design: true + assets: + - integration_test/fixtures/legacy_room_v17_wal/app_metadata_database.db + - integration_test/fixtures/legacy_room_v17_wal/app_metadata_database.db-wal + - integration_test/fixtures/legacy_room_v17_wal/app_metadata_database.db-shm + # To add assets to your application, add an assets section, like this: # assets: # - images/a_dot_burr.jpeg diff --git a/justfile b/justfile index 34b5e9f7..0dccbb6f 100644 --- a/justfile +++ b/justfile @@ -38,6 +38,9 @@ test-android-platform-adapter: test-flutter-android-platform-adapter: cd app_flutter/android && ./gradlew --no-daemon :app:testDebugUnitTest --tests 'net.xzos.upgradeall.LegacyRoomImportPreparerTest' +test-flutter-device-bridge device="emulator-5554": + cd app_flutter && flutter test integration_test/native_bridge_test.dart -d {{ device }} + build-flutter-android-debug: cd app_flutter && flutter build apk --debug python3 tools/verify_flutter_apk_bridge.py app_flutter/build/app/outputs/flutter-apk/app-debug.apk diff --git a/todo.md b/todo.md index f1b3b522..dc116dd4 100644 --- a/todo.md +++ b/todo.md @@ -399,8 +399,7 @@ Completed additional bridge tasks: Remaining tasks: -1. Add device/instrumented validation for the full Flutter MethodChannel -> Android copy/checkpoint -> JNI -> Rust getter import path if practical. -2. Extend accepted mapping if future ADR accepts direct `hub`/`extra_hub` semantics. +1. Extend accepted mapping if future ADR accepts direct `hub`/`extra_hub` semantics. Acceptance progress: @@ -411,7 +410,7 @@ Acceptance progress: - Mixed valid/invalid app rows import valid rows and warn: done. - DBs with app rows but zero importable rows fail with recovery report: done. - Report sanitization for dropped `hub`/`extra_hub` secrets and URL rewrite data: done. -- WAL/SHM pending writes: first Android adapter copy/checkpoint slice implemented; focused JVM native-adapter tests now cover triplet copy, stale sidecar cleanup, missing DB behavior, and checkpointer invocation. Real Android SQLite checkpoint behavior still needs device/instrumented coverage. +- WAL/SHM pending writes: first Android adapter copy/checkpoint slice implemented; focused JVM native-adapter tests cover triplet copy, stale sidecar cleanup, missing DB behavior, and checkpointer invocation. Device integration validation on the `Pixel_9a` emulator covers Flutter MethodChannel -> Android copy/checkpoint -> JNI -> Rust getter import/report-list using a Room v17 fixture whose committed rows remain in the WAL sidecar before Android checkpointing. - Per-app failures become warnings; global unreadable DB becomes recovery state, not crash: done for the getter-owned direct importer. ### Phase B: `local_autogen` generation @@ -451,13 +450,13 @@ Completed tasks: Remaining tasks: -1. Add device/instrumented validation for the full Flutter MethodChannel -> JNI -> Rust platform scan -> getter autogen preview/apply path if practical. -2. Cache invalidation hooks beyond file-hash-based repository reload need to be expanded when evaluated/provider caches become active. +1. Cache invalidation hooks beyond file-hash-based repository reload need to be expanded when evaluated/provider caches become active. Acceptance progress: - BDD for preview/confirm cleanup UX: done for CLI slice. - TDD for deterministic Lua generation and no overwrite of `local`: done for core/CLI slice. +- Device integration validation on the `Pixel_9a` emulator covers Flutter MethodChannel -> JNI -> Rust platform scan -> getter autogen preview/apply for the app's own installed package. - Yellow/free-network warning tagging remains getter-driven metadata, not hardcoded UI behavior: not needed for installed-target-only stubs in this slice. ### Phase C: repository tooling and diagnostics From d392b5c7c01aea7ab6e3c19a22b7805ebe819836 Mon Sep 17 00:00:00 2001 From: Xiangzhe Date: Thu, 25 Jun 2026 10:21:14 +0800 Subject: [PATCH 29/85] docs(architecture): accept runtime task ADRs --- CONTEXT.md | 51 ++++++ core-getter/src/main/rust/getter | 2 +- docs/README.md | 10 +- docs/architecture/README.md | 4 +- .../adr/0004-sqlite-main-db-and-cache-db.md | 2 +- ...age-metadata-cache-and-version-baseline.md | 155 ++++++++++++++++++ ...-update-runtime-side-effects-and-events.md | 78 +++++++++ .../upgradeall-getter-rewrite-wiki.md | 2 +- docs/lua-api/package-lifecycle.md | 2 +- todo.md | 23 +-- 10 files changed, 310 insertions(+), 19 deletions(-) create mode 100644 CONTEXT.md create mode 100644 docs/architecture/adr/0010-package-metadata-cache-and-version-baseline.md create mode 100644 docs/architecture/adr/0011-lua-update-runtime-side-effects-and-events.md diff --git a/CONTEXT.md b/CONTEXT.md new file mode 100644 index 00000000..40f67c8e --- /dev/null +++ b/CONTEXT.md @@ -0,0 +1,51 @@ +# Domain Context + +## Glossary + +### Lua update runtime + +The getter-owned runtime that evaluates a package's complete Lua lifecycle from capability checks through update action resolution. It is the center of UpgradeAll's update behavior: package Lua supplies a fully materialized lifecycle contract, getter supplies host APIs and validation, and the runtime produces getter-owned update/download/install DTOs for the app to render or execute through platform adapters. + +### Complete lifecycle contract + +The package shape consumed by getter's Lua update runtime after Lua templates/base classes have filled defaults. For getter runtime purposes, lifecycle functions are not missing or optional: every package has the full supported lifecycle surface, even if some functions come from a template default rather than package-specific Lua code. + +### Lifecycle entrypoint + +A scenario-specific Lua function that getter invokes after loading a complete package into memory. Getter chooses the entrypoint for the scenario, such as matching, update checking, action resolution, or post-update handling. Getter does not hard-code the internal Lua call graph; once the entrypoint is invoked, Lua/template code may call other lifecycle functions or helpers as needed within the validated contract. + +### Installed version entrypoint + +A Lua lifecycle entrypoint/template method that resolves the currently installed/local version for a package. This function exists as part of the complete lifecycle contract. For normal non-live update checks, the effective local baseline is `pin_version` when the user has set one; otherwise getter uses this Lua entrypoint as the baseline source. The DTO should keep the observed/local version, local-version status, and the effective comparison baseline separate so UI/CLI can display both when `pin_version` overrides local version. The installed version entrypoint returns a structured value such as `{ status = "present", version = "1.2.3", extra = { version_code = 123 } }` or `{ status = "not_installed" }`; platform/API failures use Lua errors such as `error("reason")`, not not-installed values. Without a `pin_version` override, getter must have a `present` local version to compare; if the entrypoint reports `not_installed`, there is no local baseline to display or compare, and if it raises an error getter reports the Lua/platform version-source error. With a `pin_version` override, getter may still call the installed version entrypoint for display; if that call fails, getter reports a local-version diagnostic but continues comparison against `pin_version`; if it reports `not_installed`, UI omits the local version row and still shows/uses `pin_version`. For Android apps with a standard version source, the default Lua template can simply call the getter/platform host API that reads platform-specific package facts such as version name/code, return `not_installed` when the app is absent, and raise a Lua error if the platform call itself fails; special packages can override or inherit a different Lua implementation. For live packages, getter uses the `present` result as the local baseline; if it returns `not_installed`, that means no local baseline is semantically available, and getter falls back to the last successfully installed/accepted live version recorded in getter state. If the live package's installed version entrypoint raises an error, getter must not fall back unless a `pin_version` override supplies the effective baseline for that check. + +### Getter operation + +A product-level getter API such as update check, task submission, task cancellation, or installed-autogen preview/apply. Flutter and stable CLI commands should call getter operations rather than individual Lua lifecycle functions. Direct lifecycle entrypoint calls are diagnostic/test tooling, not the product bridge contract. + +### Version behavior model + +The rewrite does not preserve the old Kotlin version-number stack wholesale. Lua packages/templates own local-version acquisition through lifecycle inheritance/override. Getter supplies host/platform APIs and small helper tools for common version extraction/comparison tasks, such as regex-based extraction and platform facts like Android version name/code, but Lua/template code decides when and how to use them. New rewrite domain language uses `pin_version`, not `ignore_version`: `pin_version` is a persisted user-selected local version override stored in `main.db` tracked package state, not a transient update-check parameter and not cache data. In the first implementation, `pin_version` is a scalar UTF-8 string so CLI usage stays simple, e.g. `getter version pin ` rather than hand-written JSON; pin/unpin commands mutate durable getter state. When set, getter compares upstream candidates against `pin_version` as the effective local version instead of the platform/Lua-installed version result. Other version comparison behavior remains the normal package/version comparison behavior. UI/CLI display should still show both observed local version and `pin_version` when an observed local version exists: Flutter shows local version above and bold pin version below, with latest version on the right; CLI compact display uses `version: (~~)` where the tilde-marked value is the pin override. If local version acquisition errors while `pin_version` is set, the check may still proceed using `pin_version`, but the error must be visible as a diagnostic. If the package is explicitly not installed/no-local, UI omits the local version row instead of showing an error. Legacy Room `ignore_version_number` / transitional `ignored_version` inputs map into rewrite `pin_version`, and legacy migration reports must emit an informational rename note so reviewers/users can see that the setting was preserved under the new name. Legacy invalid/include regex fields are migration inputs or Lua-template helper parameters, not global getter-owned version behavior. If structured pin metadata or extra fields are needed later, CLI must expose ergonomic flags or a separate advanced command rather than requiring users to type raw JSON. + +### Lua template class + +A Lua-side reusable template/base abstraction that fills default lifecycle function implementations before getter validates and runs a package. Template defaults are authoring convenience, not getter runtime optionality; getter receives the completed lifecycle contract. + +### Lua dependency closure + +The set of Lua package files, template classes, helper modules, parent package imports, and runtime/API versions that shape a completed package lifecycle contract and its package metadata. Package metadata cache entries are valid only for the same Lua dependency closure and operation context. + +### Side-effect executor + +A runtime boundary that performs effects requested by resolved update actions, such as network fetches, downloads, installer handoffs, Android system notifications, or platform callbacks. The Lua update runtime may be implemented before every side-effect executor is real, as long as executor boundaries and events are shaped like the future product behavior. The first mock download executor simulates progress and task state only; it does not write real files, validate real artifacts, or perform artifact handoff. The first mock install executor also simulates state only and does not trigger a real Android installer/handoff, but it must exercise a fake user-waiting handoff state using `status = running` and `phase = { category = "waiting_user", reason = "install_handoff" }`. Fake waiting-user install handoff does not auto-complete; it is completed through a generic task-level `user-result` method so future real installer, permission, SAF, confirmation, or other user-mediated callbacks and tests share the same task continuation boundary instead of special-casing install. `user-result` uses user-facing semantic outcomes `accepted` and `rejected` rather than raw task terminal statuses; getter maps those outcomes to the next task state or continuation behavior. `user-result` does not include a `canceled` outcome: canceling the whole task remains the separate `task cancel` method. `accepted` represents the successful/accepted outcome of the user-mediated step at the granularity the platform adapter can observe; Android does not provide a stable separate boundary for "user just agreed but install is not complete", so the rewrite does not introduce a separate `completed` user-result outcome. For fake install handoff, `accepted` continues the mock install and completes the task as `completed`; `rejected` maps to `failed`, not `canceled`, so an accidental rejection remains retryable on the same task. `rejected` may include an optional reason; if omitted, getter supplies a default current diagnostic such as `user.rejected`. `user-result` is valid only while the task is in a user-waiting phase such as `{ category = "waiting_user", reason = "install_handoff" }`; calling it in any other phase/status is an API error and must not mutate task state. For Android installer callbacks, the installer UI's "cancel" result is treated as this user-mediated `rejected` outcome rather than task cancellation. Platform-specific installation is a side-effect executor/handoff, not Flutter-owned product logic. + +### Runtime notification callback + +A getter-owned notification boundary used by native/Flutter UI to learn that runtime state changed and what it changed to. It is a callback/notification mechanism, not the source of truth and not a persisted event log. Task state is current getter-runtime process state only: it is never persisted to `main.db` or `cache.db`, and UpgradeAll does not support cross-process, app-restart, device-sleep, or downloader-style resume/recovery semantics because UpgradeAll is not a general-purpose downloader. After Flutter/Android/getter runtime process restart, the in-memory task registry is empty unless new tasks have been created in that process; UI must not present old tasks as recoverable or show a downloader-style interrupted-task recovery prompt. If the user still wants to update, they start again from the package/update action. Any residual temporary-file cleanup belongs to the specific executor implementation and is not task recovery. Getter is a monolithic package-manager runtime object held by the product app/native bridge process, not a task daemon. In the native bridge product path, getter runtime is a process-lifetime singleton: native bridge initialization creates or retrieves the runtime, Flutter route/page rebuilds do not recreate task state, and app process death drops runtime/task state. First implementation supports a single main app engine callback binding and exposes runtime notifications to Flutter as a push stream, e.g. Android/Kotlin bridge `EventChannel` or equivalent. Product bridge is not polling-only: `RuntimeNotification.task_changed` snapshots are pushed to Flutter, while `task get`/`task list` remain authoritative query operations when UI/CLI needs current state. The native/Rust bridge product path supports one main Flutter subscription; if multiple Flutter pages/components need the stream, Dart owns broadcast/state-management fanout inside the app rather than requiring Rust/native multi-subscriber semantics. The stream is best-effort push: it should push notifications whenever it can, but it is not a reliable message queue and does not guarantee delivery of every intermediate progress snapshot. The bridge must avoid unbounded backpressure queues; it may coalesce/drop intermediate progress notifications, while authoritative current state remains available through `task list`/`task get`. The stream does not replay notifications missed while Flutter is unsubscribed or disconnected; reconnecting Flutter should resubscribe and immediately query current task state with `task list`/`task get`, then process newly pushed notifications. CLI/debug tooling may use query/polling/scripted commands instead of product stream semantics. Multi-engine or multi-isolate sharing semantics require a later ADR if needed. Product task submission uses a getter-issued opaque `action_id`. Update/check operations may send Flutter rich display DTOs for package/version/action presentation, but Flutter acts by returning only the `action_id`; it must not construct tasks by assembling or echoing raw URLs, checksums, installer types, package IDs, versions, or full action payloads itself. The getter runtime's internal action registry may and should hold a sealed action plan with full execution details such as package id, target version/live revision, artifact descriptors, checksum/signature expectations, installer/executor plan, and the bound Lua/package execution context; those details are internal getter execution data, not product bridge input. The sealed plan must be bound to the Lua/package context that produced the action and must not re-read current Lua files during submit or retry. The runtime should load, validate, and materialize the package/template/helper context into a package-version Lua object in memory before issuing the action, then execute that bound in-memory object/plan rather than behaving like a shell that reads and executes one line at a time. If task execution later needs Lua hooks or helper functions, it calls the bound package-version Lua object; the entire Lua call chain used by that object is already loaded/materialized in memory and is not resolved again from the filesystem. The package-version Lua object lives with the action/task that needs it: the action registry holds it until the action is consumed or expires; successful submit consumes the action and transfers the sealed plan/object to the task; failed tasks keep enough of the object to support retry; completed/canceled tasks may release it when the task is cleared from the current runtime registry; expired unsubmitted actions release it when the action registry cleans them up. `action_id` is scoped to the current getter runtime process and is not a persisted cross-process handle. `action_id` is single-use: once task submission successfully creates a task, getter consumes/removes that action from the runtime action registry. A consumed action is permanently gone within that runtime: it is not restored if the created task later fails, and it must not be reused because reuse would blur action lifecycle and task lifecycle. Reusing the same consumed action, such as from a UI double-submit, returns `action.not_found` instead of creating another task. If task submission references an expired or unknown `action_id`, getter returns `action.not_found` and must not automatically re-run update check or attempt to match a fresh package/version candidate, because that could change the candidate the user saw. Flutter should prompt the user to refresh package/update state and submit a new current-runtime action. CLI/debug tooling may use fixtures/scripts or full request JSON for tests, but the product bridge must stay anchored to getter-owned update/package operations. Runtime task scheduling does not impose a global serial queue or task-registry-level cross-task lock: concurrent downloads and concurrent installs are allowed, and tasks behave like independent branches in a task tree/forest. This tree/forest wording is only a mental model for independence, not an exposed parent/child task API. The first runtime does not need `parent_task_id` or visible download/install subtasks unless a later batch-update ADR introduces them. Tasks do not know about, wait for, or coordinate with sibling tasks through the runtime scheduler. However, package installation/state mutation for the same package is a package-scoped resource and must be protected by a package-level lock inside the relevant executor/operation. Installing the same package, whether for the same version or different versions, must not run its package mutation critical section concurrently. This package lock must not be implemented as task creation rejection, task deduplication, task merge, or a global task lock; if two tasks for the same package are created due to user action, race, or bug, the tasks may both exist. The package lock is non-waiting: when a task reaches the package mutation/install critical section and the same package is already locked by another task, this is treated as incorrect usage and the later task fails immediately with a user-visible diagnostic instead of waiting for the lock or entering a resource-waiting phase. The diagnostic code is `package.locked`, and task phase reason can use `package_locked` to make clear that the failing resource is the package mutation boundary, not the task scheduler. Download execution is task-local and does not use the package-level lock; even if two tasks download the same artifact, they are independent task-internal effects until a later package mutation/install boundary is reached. CLI task commands do not promise state across separate CLI invocations and must not create a task DB or daemon just to make pause/resume/user-result work across commands; tests should exercise the Rust runtime library directly or through a single-process scripted/debug command when CLI coverage is useful. The top-level payload is a generic `RuntimeNotification` with a `kind` discriminator; the first product kind is `task_changed`, carrying a lightweight but sufficiently complete current task snapshot. Callback payloads must include enough task snapshot data so Flutter can update UI without querying getter after every notification; this avoids unnecessary backend pressure. The current task snapshot includes task-state fields such as `task_id`, `package_id`, `status`, structured `phase`, `progress`, current control `capabilities`, optional `current_diagnostic`, and an `updated_at`/snapshot timestamp for UI ordering or stale-update handling. The snapshot is task state only, not package metadata or duplicated display metadata; external UI/callers are expected to know or query package metadata through the proper package/update APIs. Task progress supports at least `percent` and `bit` units; when bit-level current/total values are available, callbacks should prefer `bit` because it carries more information and percent can be derived from it. Task control supports cancel, retry, pause, and resume from the first runtime implementation. These controls are methods on an existing task object, not factories for new tasks. `task cancel` is valid for active `queued`, `running`, and `paused` tasks; a paused task can be canceled directly without first resuming. It is not valid for `failed`, `completed`, or already `canceled` tasks. Failed tasks should use retry or remove, while completed/canceled tasks are terminal. Task status values are `queued`, `running`, `paused`, `failed`, `completed`, and `canceled`; the rewrite uses `completed` and does not keep a `succeeded` alias. Waiting for user action is not a status: the task remains `running`, and the phase is structured as `{ category, reason? }`, e.g. `{ category: "waiting_user", reason: "install_handoff" }`. The first phase schema intentionally avoids extra fields such as detail/executor/localized message; progress, diagnostics, and UI text are separate concerns. In particular, `retry` retries the same task identified by the same `task_id`; it must not automatically create a new task. Only `completed` and `canceled` are truly terminal task states. Failed tasks are not terminal: if the current task state permits retry, retry reuses the same `task_id` and transitions the existing task back into a runnable state. Retry is a task method over the sealed, already-consumed action plan, not action reuse and not an implicit update refresh. It does not revalidate against the action registry, re-run update check, or match a fresh candidate; if the sealed artifact/package plan has become invalid, the retry fails naturally in the relevant execution phase such as download, validation, package lock, or install. It should resume from the failed phase when the runtime has enough task-local state: download failures retry download, fake install `rejected` failures retry the install handoff, and `package.locked` failures retry entering the package mutation boundary. If the runtime lacks enough intermediate state for a precise phase retry, it may restart from the task-internal action plan beginning, but still as the same task and without creating/restoring an `action_id`. If retry itself fails, getter returns an error and the caller may invoke retry again on the same task when the task state still permits retry. Completed, canceled, and failed tasks remain queryable task objects in the current runtime process indefinitely until an explicit manual cleanup operation removes them; there is no automatic TTL/capacity retention cleanup and no persistent storage. CLI should expose both `task remove ` for removing one in-memory task and `task clean` for explicit bulk cleanup of non-active in-memory tasks. `task remove ` is an explicit single-task operation and may remove `failed`, `completed`, or `canceled` tasks; removing a failed task also discards its retry capability. By default, `task clean` removes `completed` and `canceled` tasks only; it does not remove `failed` tasks because failed tasks may still be retryable. Removing failed tasks requires an explicit option such as `task clean --failed` or `task clean --all-inactive`, where all inactive means `completed`, `canceled`, and `failed`. Removal/cleanup must not remove active tasks such as queued/running/paused tasks; callers must cancel them first. After removal or cleanup, `task get` returns `task.not_found` for the removed task, and associated in-memory sealed action plan/Lua object may be released. The retry method/interface still exists, but retrying a completed or canceled task must immediately return an error and must not resurrect the task or create a replacement task. If the user wants a fresh task object after completion/cancellation, they must use the original task creation/submission path again. Pause/resume are task-level APIs, not download-executor-only public APIs, but whether they are currently allowed is phase/executor-specific and process-specific. Some phases, including `waiting_user`, are state snapshots rather than pausable processes, so they must expose `pause = false` and `resume = false` even while the task status remains `running`. `task resume` is valid only for a `paused` task whose current executor/phase supports resume; calling resume for `running`, `waiting_user`, `queued`, `failed`, `completed`, or `canceled` tasks returns an unsupported-control error and does not mutate state. Failed tasks use retry rather than resume. The first implementation must support pause/resume for download-phase tasks; other phases can expose false capabilities. Task snapshots include current capability flags for these controls so Flutter does not infer executor-specific behavior from status/phase; a capability may be false for the current task/phase, but the API/control model exists. Calling an unsupported control for the current state/phase, such as pausing a `waiting_user` phase or resuming a non-paused task, returns an explicit error such as `task.pause_not_supported`, `task.resume_not_supported`, `task.retry_not_supported`, or `task.cancel_not_supported` and must not mutate task state; unsupported controls are never silent no-ops. Task snapshots include at most the current diagnostic summary needed for UI display, not a diagnostic history or event log; detailed diagnostics/logs use separate query operations. Every field included in a task snapshot, including capabilities and current diagnostic summary, must also be obtainable by active query operations or a combination of query operations within the current runtime process; the callback exists to reduce getter/UI polling pressure, not to become the only way to learn state. If UI/CLI explicitly wants authoritative task state, it calls separate getter operations that query/recompute current internal task state, including at least single-task lookup and task listing/summary for active tasks or package-scoped tasks. This is not the same as an Android system notification. In the first Phase D runtime slice, downloads and installers may be mock side-effect executors, but runtime callbacks must be shaped like product notifications so Flutter can refresh task/progress UI without owning the task state machine. + +### Provider host API + +A getter-owned API exposed to package Lua for requesting provider/source data. Lua packages should call provider host APIs rather than performing arbitrary direct HTTP by default. The provider executor behind the host API can be fake during early runtime development and live later, but caching, diagnostics, permissions, and output validation belong to the Lua update runtime boundary. + +### Package metadata cache + +The cache of software metadata produced by running package Lua/provider logic, analogous in spirit to Gentoo eix's binary cache over package metadata. It stores reusable metadata such as package identity, descriptions, homepage/source information, available versions/candidates, changelog/release notes when provided by sources, artifact descriptors, licenses/tags, and source/provider diagnostics. It is persisted in `cache.db` from the first runtime implementation. It is keyed by the getter-tracked Lua dependency closure plus runtime context that can affect metadata. Freshness is determined by provider/source freshness tokens such as ETag, Last-Modified, source cursor, index revision, or response digest when available; TTL is only a fallback revalidation hint. If the Lua dependency closure changes, the runtime may reuse unchanged provider/source cache as input, but it must rerun Lua normalization and create/update the current PackageMetadata entry for the new closure digest. A forced refresh bypasses existing cached reads and updates/replaces the relevant cache entries on success so `cache.db` reflects the newly observed source facts. If forced refresh fails, the runtime must not delete still-usable old cache entries, but it must report refresh failure/staleness explicitly and must not present old cache as a successful fresh synchronization. Cache consistency is a design invariant for later Phase D decisions. Cache is not an audit log; product semantics only require the current effective cache entry, and old entries may be garbage-collected. Artifact descriptors inside PackageMetadata are package-management contracts, not mutable cache truth: a versioned artifact's URL/locator, size, checksum, signature, and content identity describe the expected file. If upstream changes the file or a downloaded file does not match declared metadata/hash, getter treats that as an invalid artifact/download failure rather than silently accepting or refreshing the artifact identity. Live/floating behavior is a package/Lua-level flag, analogous to Gentoo `9999` live ebuilds, not an artifact-level flag. Getter/UI must surface live versions before download/task submission because artifact-stage detection is too late for user awareness. Live version checks are opt-in and require a separate live flag such as `--live`; live packages do not participate in the ordinary versioned update check by default. The live update rule is simple: run the live Lua path to obtain the current live version string, compare it with the local baseline, and report an available update when they differ. The local baseline comes from the package's installed version entrypoint when it returns `{ status = "present", ... }`; if it returns `{ status = "not_installed" }`, live checks fall back to getter's last successfully installed/accepted live version. The entrypoint still exists in the complete lifecycle contract. If the entrypoint raises a Lua error because a platform/API call failed, getter must report that error and must not fall back. A live version is an arbitrary valid UTF-8 string; getter does not parse, order, or validate it as a semantic version. A live package may delegate arbitrary/latest download resolution to Lua, but those results are not cacheable as stable artifact metadata because upstream may change at any time. The cache is not the authoritative user state and should not cache final user-state-dependent decisions such as pin_version override state, task state, or installer results. diff --git a/core-getter/src/main/rust/getter b/core-getter/src/main/rust/getter index 21dc994f..8e3e3c12 160000 --- a/core-getter/src/main/rust/getter +++ b/core-getter/src/main/rust/getter @@ -1 +1 @@ -Subproject commit 21dc994f5d452238a0b5fda3ecf9cd09e541155a +Subproject commit 8e3e3c12803f1f8c2659361bfeba78c666917772 diff --git a/docs/README.md b/docs/README.md index fdad3af6..c46ba306 100644 --- a/docs/README.md +++ b/docs/README.md @@ -18,10 +18,12 @@ Start here: 8. `architecture/adr/0007-flutter-getter-bridge-contract.md` — Flutter/getter DTO and bridge contract. 9. `architecture/adr/0008-flutter-product-apk-entry.md` — Flutter app as the sole product APK entry. 10. `architecture/adr/0009-android-platform-adapter-and-package-visibility.md` — Rust-active Android platform adapter and package visibility policy. -11. `lua-api/` — practical Lua package authoring docs, including offline `repo validate` diagnostics. -12. `migration/legacy-room-mapping.md` — old data mapping rules. -13. `app/flutter-ui-feature-parity-and-testing.md` — Flutter feature parity and BDD/TDD test boundary. -14. `implementation/coding-agent-handoff.md` — coding-agent / pi-agent handoff instructions. +11. `architecture/adr/0010-package-metadata-cache-and-version-baseline.md` — accepted package metadata cache, live-version, installed-version, and `pin_version` rules. +12. `architecture/adr/0011-lua-update-runtime-side-effects-and-events.md` — accepted Phase D Lua runtime, task/action lifecycle, mock side-effect executor, and RuntimeNotification bridge rules. +13. `lua-api/` — practical Lua package authoring docs, including offline `repo validate` diagnostics. +14. `migration/legacy-room-mapping.md` — old data mapping rules. +15. `app/flutter-ui-feature-parity-and-testing.md` — Flutter feature parity and BDD/TDD test boundary. +16. `implementation/coding-agent-handoff.md` — coding-agent / pi-agent handoff instructions. Canonical architecture ADRs live in `docs/architecture/adr/*`. The `docs/adr/*` directory is kept for historical/refactor-phase ADRs and transition notes. diff --git a/docs/architecture/README.md b/docs/architecture/README.md index 8b4ff1b7..0104215d 100644 --- a/docs/architecture/README.md +++ b/docs/architecture/README.md @@ -6,7 +6,7 @@ Start here: - `upgradeall-getter-rewrite-wiki.md` — main living wiki for the Flutter + Rust getter + Lua package repository redesign. -Planned / active ADRs: +Canonical ADRs: - `adr/0001-app-centric-lua-package-repository-model.md` - `adr/0002-getter-flutter-platform-boundary.md` @@ -17,6 +17,8 @@ Planned / active ADRs: - `adr/0007-flutter-getter-bridge-contract.md` - `adr/0008-flutter-product-apk-entry.md` - `adr/0009-android-platform-adapter-and-package-visibility.md` +- `adr/0010-package-metadata-cache-and-version-baseline.md` +- `adr/0011-lua-update-runtime-side-effects-and-events.md` Documentation policy: diff --git a/docs/architecture/adr/0004-sqlite-main-db-and-cache-db.md b/docs/architecture/adr/0004-sqlite-main-db-and-cache-db.md index 426d0bc3..efc5ce60 100644 --- a/docs/architecture/adr/0004-sqlite-main-db-and-cache-db.md +++ b/docs/architecture/adr/0004-sqlite-main-db-and-cache-db.md @@ -23,7 +23,7 @@ Users manually corrupting backend storage is considered non-standard usage. gett - Ignored versions, pins, favorites. - Migration records. - Settings and credential references. -- Download task persistent state. +- Operation-specific durable records accepted by later ADRs. ADR-0011 explicitly excludes runtime task state from main/cache DB persistence. ## Cache DB stores diff --git a/docs/architecture/adr/0010-package-metadata-cache-and-version-baseline.md b/docs/architecture/adr/0010-package-metadata-cache-and-version-baseline.md new file mode 100644 index 00000000..1073b27d --- /dev/null +++ b/docs/architecture/adr/0010-package-metadata-cache-and-version-baseline.md @@ -0,0 +1,155 @@ +# ADR-0010: Package metadata cache and version baseline + +> Status: Accepted +> Date: 2026-06-24 +> Project: UpgradeAll rewrite — Flutter APP + Rust getter core + Lua package repository model + +## Decision + +UpgradeAll's rewrite uses getter-owned package metadata caching and getter-owned version-baseline semantics. The cache is persisted in `cache.db`; user version override state is persisted in `main.db`; package Lua/templates own local-version acquisition and normalization through the complete lifecycle contract. + +## Package metadata cache + +The runtime caches software metadata produced by running package Lua/provider logic. This cache is analogous in spirit to Gentoo `eix` package metadata caching: it supports fast query/display/update planning over reusable package metadata such as identity, description, homepage/source information, available versions/candidates, changelog or release notes when supplied by sources, artifact descriptors, licenses/tags, and source/provider diagnostics. + +The cache model has two layers: + +1. **Provider/source cache**: provider host API responses keyed by provider id, request parameters, executor/cache policy, auth/permission mode, and other provider-context inputs. +2. **Package metadata cache**: normalized package metadata produced by Lua/package logic from provider/source data. + +Package metadata cache entries are persisted in `cache.db` from the first runtime implementation. + +### Cache key and Lua dependency closure + +Package metadata cache entries must be keyed by the Lua dependency closure and runtime context that can affect metadata, including at least: + +- package Lua file hash; +- loaded template/base class hashes; +- loaded helper module hashes; +- parent package imports and their dependency closure digests; +- Lua API/schema/runtime version; +- platform target and permission/network mode when they can affect produced metadata; +- provider/source cache keys or content digests used to produce the metadata. + +The runtime should automatically track the Lua dependency closure from actual loaded modules/templates/package imports. Explicit dependency declarations may exist only as a supplement or escape hatch for dependencies that the loader cannot otherwise observe. + +If provider/source validation proves data unchanged, the runtime may update checked-at/freshness metadata without replacing the provider body. Package metadata normalization may be skipped only when the Lua dependency closure digest and other package metadata key inputs are unchanged. If the Lua package/template/helper dependency closure changes, the runtime may reuse unchanged provider/source cache as input, but it must rerun Lua normalization and create/update the current PackageMetadata entry for the new closure digest. If provider/source data changes, the runtime updates provider/source cache and reruns package metadata normalization for affected packages. + +### Freshness and refresh + +Freshness should be determined by provider/source freshness tokens when available, with TTL as a fallback revalidation hint. Examples include ETag, Last-Modified, source cursor, upstream index revision, or response digest. TTL expiry means the entry should be revalidated; it does not by itself mean the old cache must be deleted. + +A forced refresh bypasses cached reads for the refreshed scope and, on success, updates or replaces the relevant `cache.db` entries with newly observed source facts. `--refresh` is not a read-only cache bypass mode. If the runtime has successfully observed newer actual provider/package metadata, keeping stale cache entries as the effective cache value is a consistency bug. + +If forced refresh fails, the runtime must not delete still-usable old cache entries merely because the refresh failed. Instead, the operation must report refresh failure and staleness explicitly. If an operation elects to fall back to old cache, the result must make that fallback visible through diagnostics such as `cache.refresh_failed`, `used_stale_cache`, and stale age/cursor metadata. Old cache must not be presented as a successful fresh synchronization. + +`cache.db` is not an audit log. Product semantics only require the current effective cache entry for a package/context. Old provider or package metadata entries may be retained temporarily for debugging or transaction safety, but they can be garbage-collected without preserving a product-visible history. Future metadata history/diff features require a separate design. + +## Artifact descriptors and live versions + +Artifact descriptors inside PackageMetadata are package-management contracts, not mutable cache truth. For a normal versioned release, the artifact URL/locator, size, checksum, signature, and content identity describe the expected file. If upstream changes the file behind the same declared release, or if the downloaded file's metadata/hash/signature does not match, getter must treat it as an invalid artifact/download failure rather than silently accepting the new file or treating the mismatch as a cache refresh. Refreshing metadata may discover a new valid release/artifact descriptor, but it must not launder a mismatched downloaded file into correctness. + +Explicitly live/floating packages, analogous to Gentoo `9999` live ebuilds, are different. Live/floating behavior is a package/Lua-level flag, not an artifact-level flag. Getter/UI must surface live versions before download/task submission because artifact-stage detection is too late for user awareness. + +Live version checks are opt-in and require a separate live flag such as `--live`; live packages do not participate in the ordinary versioned update check by default. The live update rule is intentionally simple: run the live Lua path to obtain the current live version string, compare it with the local baseline, and report an available update when they differ. A live version is an arbitrary valid UTF-8 string; getter does not parse, order, or validate it as a semantic version. A live package may allow Lua to resolve arbitrary/latest upstream artifacts at execution time, but those results are not cacheable as stable artifact metadata because upstream may change at any time and downstream cannot continuously refresh. + +## Installed version entrypoint + +The installed/local version source is part of the completed Lua lifecycle contract. Getter uses an installed version entrypoint/template method to resolve the current baseline and to produce display data. + +For non-live update checks, the effective local baseline is `pin_version` when the user has set one; otherwise getter uses this entrypoint. + +The installed version entrypoint returns a structured value such as: + +```lua +return { + status = "present", + version = "1.2.3", + extra = { + version_code = 123, + }, +} +``` + +or: + +```lua +return { + status = "not_installed", +} +``` + +Platform/API failures use Lua errors such as `error("reason")`, not `not_installed` values. + +Without a `pin_version` override, getter must have a `present` local version to compare. If the entrypoint reports `not_installed`, there is no local baseline to display or compare. If it raises an error, getter reports the Lua/platform version-source error. + +With a `pin_version` override, getter may still call the installed version entrypoint for display. If that call fails, getter reports a local-version diagnostic but continues comparison against `pin_version`. If it reports `not_installed`, UI omits the local version row and still shows/uses `pin_version`. + +For Android apps with a standard version source, the default Lua template can call the getter/platform host API that reads platform-specific package facts such as version name/code, return `not_installed` when the app is absent, and raise a Lua error if the platform call itself fails. Special packages can override or inherit a different Lua implementation. + +For live checks, the installed version entrypoint may return `{ status = "not_installed" }` to mean no local baseline is semantically available; getter then falls back to the last successfully installed/accepted live version recorded in getter state. If the live package's installed version entrypoint raises a Lua error because a platform/API call failed, getter must report that error and must not fall back unless a `pin_version` override supplies the effective baseline for that check. + +## Version model and `pin_version` + +The rewrite does not preserve the old Kotlin version-number stack wholesale. Lua packages/templates own local-version acquisition and normalization through lifecycle inheritance/override. Getter supplies host/platform APIs and small helper tools for common extraction/comparison tasks, such as regex-based extraction and platform facts like Android version name/code, but Lua/template code decides when and how to use them. Legacy invalid/include regex fields are migration inputs or Lua-template helper parameters, not global getter-owned version behavior. + +New rewrite domain language uses `pin_version`, not `ignore_version`: `pin_version` is a persisted user-selected local version override stored in `main.db` tracked package state, not a transient update-check parameter and not cache data. + +In the first implementation, `pin_version` is a scalar UTF-8 string so CLI usage stays simple, e.g.: + +```bash +getter version pin +getter version unpin +``` + +Pin/unpin commands mutate durable getter state. When set, getter compares upstream candidates against `pin_version` as the effective local version instead of the platform/Lua-installed version result. Other version comparison behavior remains the normal package/version comparison behavior. + +UI/CLI display should still show both observed local version and `pin_version` when an observed local version exists: + +- Flutter shows local version above and bold pin version below, with latest version on the right. +- CLI compact display uses `version: (~~)` where the tilde-marked value is the pin override. + +If local version acquisition errors while `pin_version` is set, the check may still proceed using `pin_version`, but the error must be visible as a diagnostic. If the package is explicitly not installed/no-local, UI omits the local version row instead of showing an error. + +Legacy Room `ignore_version_number` and transitional `ignored_version` inputs map into rewrite `pin_version`; new rewrite storage, DTOs, and Flutter UI should emit/use `pin_version`. Legacy migration reports must emit an informational rename note such as `migration.renamed_ignored_version_to_pin_version` so reviewers/users can see that the setting was preserved under the new name. + +If structured pin metadata or extra fields are needed later, CLI must expose ergonomic flags or a separate advanced command rather than requiring users to type raw JSON. + +## Non-authoritative cache boundary + +The package metadata cache must not be the authoritative store for: + +- tracked/enabled packages; +- favorites; +- pin_version override state; +- final user-state-dependent selected update status; +- download task state; +- installer handoff results; +- Flutter UI state. + +Those remain main DB or operation-specific state. + +## Consequences + +Positive: + +- Cache invalidation follows the effective Lua dependency closure instead of fragile manual cache-clearing. +- Forced refresh semantics preserve cache consistency while keeping stale cache available after failed refreshes. +- Version baseline logic becomes explicit and user-visible. +- `pin_version` naming better matches the actual behavior than legacy `ignore_version` terminology. +- CLI remains ergonomic for pin/unpin workflows. + +Costs: + +- The runtime loader must track actual Lua/template/module/package imports. +- Cache keys become more complex than a package file hash. +- Tests must distinguish provider/source cache behavior from package metadata cache behavior. +- Existing rewrite code using `ignored_version` must be renamed before these DTOs/storage names become stable product API. + +## Follow-up implementation notes + +- Rename rewrite-facing `ignored_version` fields, storage columns, fixtures, and DTOs to `pin_version`. +- Keep accepting legacy names only at migration/import boundaries when needed. +- Add migration report notice `migration.renamed_ignored_version_to_pin_version`. +- Add getter CLI commands for durable pin/unpin. +- Add Lua/helper API for version extraction/comparison without preserving the old Kotlin version stack wholesale. diff --git a/docs/architecture/adr/0011-lua-update-runtime-side-effects-and-events.md b/docs/architecture/adr/0011-lua-update-runtime-side-effects-and-events.md new file mode 100644 index 00000000..018dff49 --- /dev/null +++ b/docs/architecture/adr/0011-lua-update-runtime-side-effects-and-events.md @@ -0,0 +1,78 @@ +# ADR-0011: Lua update runtime, side effects, and runtime events + +> Status: Accepted +> Date: 2026-06-24 +> Project: UpgradeAll rewrite — Flutter APP + Rust getter core + Lua package repository model + +## Context + +ADR-0010 accepts the package metadata cache, version baseline, `pin_version`, installed version entrypoint, and live-version rules. + +This ADR accepts the first Phase D runtime architecture for the getter-owned Lua update runtime: + +- runtime notification callback and Flutter push-stream bridge shape; +- side-effect executor boundaries; +- mock provider/download/install executors for the first implementation slice; +- operation DTO boundaries for CLI/native bridge/Flutter; +- in-memory task/action lifecycle, controls, user-result, retry, and cleanup semantics. + +Future Android download/install/background/system-notification semantics remain deferred to later ADRs. + +## Current settled boundaries carried forward + +UpgradeAll's Phase D runtime remains getter-owned and cross-platform. Flutter subscribes/renders getter-owned DTOs/events and must not own provider selection, version comparison, cache invalidation, download task state machines, retry policy, installer semantics, or local-autogen/package-id decisions. + +The Lua update runtime is not merely `mlua` file evaluation. It loads a package and its Lua template/base/helper dependency closure, materializes a complete lifecycle contract, validates it, invokes scenario-specific lifecycle entrypoints, exposes getter-owned host APIs, and emits getter-owned task/runtime events for UI subscribers. + +External side effects may remain mocked in the first implementation slice. Mock side effects are a development/implementation scaffold, not a product architecture decision. The architecture decision is the runtime shape and its boundaries. The first mock download executor simulates progress and task state only; it does not write real files, validate real artifacts, or perform artifact handoff. The first mock install executor also simulates state only and does not trigger a real Android installer/handoff, but it must exercise a fake user-waiting handoff state using `status = running` and `phase = { category = "waiting_user", reason = "install_handoff" }`. Fake waiting-user install handoff does not auto-complete; it is completed through a generic task-level `user-result` method so future real installer, permission, SAF, confirmation, or other user-mediated callbacks and tests share the same task continuation boundary instead of special-casing install. `user-result` uses user-facing semantic outcomes `accepted` and `rejected` rather than raw task terminal statuses; getter maps those outcomes to the next task state or continuation behavior. `user-result` does not include a `canceled` outcome: canceling the whole task remains the separate `task cancel` method. `accepted` represents the successful/accepted outcome of the user-mediated step at the granularity the platform adapter can observe; Android does not provide a stable separate boundary for "user just agreed but install is not complete", so the rewrite does not introduce a separate `completed` user-result outcome. For fake install handoff, `accepted` continues the mock install and completes the task as `completed`; `rejected` maps to `failed`, not `canceled`, so an accidental rejection remains retryable on the same task. `rejected` may include an optional reason; if omitted, getter supplies a default current diagnostic such as `user.rejected`. `user-result` is valid only while the task is in a user-waiting phase such as `{ category = "waiting_user", reason = "install_handoff" }`; calling it in any other phase/status is an API error and must not mutate task state. For Android installer callbacks, the installer UI's "cancel" result is treated as this user-mediated `rejected` outcome rather than task cancellation. + +### Complete lifecycle contract + +For getter runtime purposes, lifecycle functions are not optional. A package consumed by the runtime has a complete lifecycle contract after Lua templates/base classes fill default implementations. + +Template defaults are authoring convenience. Getter should validate and run the completed contract rather than treating absent lifecycle functions as normal runtime state. + +### Entrypoint-oriented execution + +Getter invokes scenario-specific lifecycle entrypoints, not a hard-coded global lifecycle sequence on every operation. + +Examples: + +- installed matching may invoke a matching entrypoint; +- update checking may invoke the update/check entrypoint that internally calls discovery/prepare/select/resolve helpers as its Lua/template contract defines; +- post-update behavior runs only after an update/install result exists. + +Getter should not hard-code the internal Lua call graph. Once getter invokes the selected entrypoint, Lua/template code may call other lifecycle functions or helpers within the validated contract. + +Stable Flutter/product APIs should be getter operations such as update check, task submission, cancellation, installed-autogen preview/apply, and task/event retrieval. Direct calls to individual Lua lifecycle functions are diagnostic/test tooling, not the product bridge contract. + +### Provider host API, not default raw HTTP + +Package Lua should call getter-owned provider/source host APIs by default rather than performing arbitrary direct HTTP. + +The provider executor behind the host API may be fake/mock during the first runtime implementation and live later. Caching, diagnostics, permissions, and output validation remain part of the getter-owned runtime boundary. + +Direct/free-network Lua remains a separately declared permission path as described in ADR-0005. It is not the default provider model for normal packages. + +### Runtime event callbacks + +The first Phase D runtime slice should define a getter-owned runtime callback boundary so native/Flutter UI can learn that task/runtime state changed and what it changed to, without owning task state machines. + +The callback is a notification mechanism, not the source of truth and not a persisted event log. Task state is current getter-runtime process state only: it is never persisted to `main.db` or `cache.db`, and UpgradeAll does not support cross-process, app-restart, device-sleep, or downloader-style resume/recovery semantics because UpgradeAll is not a general-purpose downloader. After Flutter/Android/getter runtime process restart, the in-memory task registry is empty unless new tasks have been created in that process; UI must not present old tasks as recoverable or show a downloader-style interrupted-task recovery prompt. If the user still wants to update, they start again from the package/update action. Any residual temporary-file cleanup belongs to the specific executor implementation and is not task recovery. Getter is a monolithic package-manager runtime object held by the product app/native bridge process, not a task daemon. In the native bridge product path, getter runtime is a process-lifetime singleton: native bridge initialization creates or retrieves the runtime, Flutter route/page rebuilds do not recreate task state, and app process death drops runtime/task state. First implementation supports a single main app engine callback binding and exposes runtime notifications to Flutter as a push stream, e.g. Android/Kotlin bridge `EventChannel` or equivalent. Product bridge is not polling-only: `RuntimeNotification.task_changed` snapshots are pushed to Flutter, while `task get`/`task list` remain authoritative query operations when UI/CLI needs current state. The native/Rust bridge product path supports one main Flutter subscription; if multiple Flutter pages/components need the stream, Dart owns broadcast/state-management fanout inside the app rather than requiring Rust/native multi-subscriber semantics. The stream is best-effort push: it should push notifications whenever it can, but it is not a reliable message queue and does not guarantee delivery of every intermediate progress snapshot. The bridge must avoid unbounded backpressure queues; it may coalesce/drop intermediate progress notifications, while authoritative current state remains available through `task list`/`task get`. The stream does not replay notifications missed while Flutter is unsubscribed or disconnected; reconnecting Flutter should resubscribe and immediately query current task state with `task list`/`task get`, then process newly pushed notifications. CLI/debug tooling may use query/polling/scripted commands instead of product stream semantics. Multi-engine or multi-isolate sharing semantics require a later ADR if needed. Product task submission uses a getter-issued opaque `action_id`. Update/check operations may send Flutter rich display DTOs for package/version/action presentation, but Flutter acts by returning only the `action_id`; it must not construct tasks by assembling or echoing raw URLs, checksums, installer types, package IDs, versions, or full action payloads itself. The getter runtime's internal action registry may and should hold a sealed action plan with full execution details such as package id, target version/live revision, artifact descriptors, checksum/signature expectations, installer/executor plan, and the bound Lua/package execution context; those details are internal getter execution data, not product bridge input. The sealed plan must be bound to the Lua/package context that produced the action and must not re-read current Lua files during submit or retry. The runtime should load, validate, and materialize the package/template/helper context into a package-version Lua object in memory before issuing the action, then execute that bound in-memory object/plan rather than behaving like a shell that reads and executes one line at a time. If task execution later needs Lua hooks or helper functions, it calls the bound package-version Lua object; the entire Lua call chain used by that object is already loaded/materialized in memory and is not resolved again from the filesystem. The package-version Lua object lives with the action/task that needs it: the action registry holds it until the action is consumed or expires; successful submit consumes the action and transfers the sealed plan/object to the task; failed tasks keep enough of the object to support retry; completed/canceled tasks may release it when the task is cleared from the current runtime registry; expired unsubmitted actions release it when the action registry cleans them up. `action_id` is scoped to the current getter runtime process and is not a persisted cross-process handle. `action_id` is single-use: once task submission successfully creates a task, getter consumes/removes that action from the runtime action registry. A consumed action is permanently gone within that runtime: it is not restored if the created task later fails, and it must not be reused because reuse would blur action lifecycle and task lifecycle. Reusing the same consumed action, such as from a UI double-submit, returns `action.not_found` instead of creating another task. If task submission references an expired or unknown `action_id`, getter returns `action.not_found` and must not automatically re-run update check or attempt to match a fresh package/version candidate, because that could change the candidate the user saw. Flutter should prompt the user to refresh package/update state and submit a new current-runtime action. CLI/debug tooling may use fixtures/scripts or full request JSON for tests, but the product bridge must stay anchored to getter-owned update/package operations. Runtime task scheduling does not impose a global serial queue or task-registry-level cross-task lock: concurrent downloads and concurrent installs are allowed, and tasks behave like independent branches in a task tree/forest. This tree/forest wording is only a mental model for independence, not an exposed parent/child task API. The first runtime does not need `parent_task_id` or visible download/install subtasks unless a later batch-update ADR introduces them. Tasks do not know about, wait for, or coordinate with sibling tasks through the runtime scheduler. However, package installation/state mutation for the same package is a package-scoped resource and must be protected by a package-level lock inside the relevant executor/operation. Installing the same package, whether for the same version or different versions, must not run its package mutation critical section concurrently. This package lock must not be implemented as task creation rejection, task deduplication, task merge, or a global task lock; if two tasks for the same package are created due to user action, race, or bug, the tasks may both exist. The package lock is non-waiting: when a task reaches the package mutation/install critical section and the same package is already locked by another task, this is treated as incorrect usage and the later task fails immediately with a user-visible diagnostic instead of waiting for the lock or entering a resource-waiting phase. The diagnostic code is `package.locked`, and task phase reason can use `package_locked` to make clear that the failing resource is the package mutation boundary, not the task scheduler. Download execution is task-local and does not use the package-level lock; even if two tasks download the same artifact, they are independent task-internal effects until a later package mutation/install boundary is reached. CLI task commands do not promise state across separate CLI invocations and must not create a task DB or daemon just to make pause/resume/user-result work across commands; tests should exercise the Rust runtime library directly or through a single-process scripted/debug command when CLI coverage is useful. The top-level payload is a generic `RuntimeNotification` with a `kind` discriminator. The first product kind is `task_changed`, carrying a lightweight but sufficiently complete current task snapshot. Callback payloads must include enough task snapshot data so Flutter can update UI without querying getter after every notification; this avoids unnecessary backend pressure. The current task snapshot includes task-state fields such as `task_id`, `package_id`, `status`, structured `phase`, `progress`, current control `capabilities`, optional `current_diagnostic`, and an `updated_at`/snapshot timestamp for UI ordering or stale-update handling. The snapshot is task state only, not package metadata or duplicated display metadata; external UI/callers are expected to know or query package metadata through the proper package/update APIs. Task progress supports at least `percent` and `bit` units; when bit-level current/total values are available, callbacks should prefer `bit` because it carries more information and percent can be derived from it. Task control supports cancel, retry, pause, and resume from the first runtime implementation. These controls are methods on an existing task object, not factories for new tasks. `task cancel` is valid for active `queued`, `running`, and `paused` tasks; a paused task can be canceled directly without first resuming. It is not valid for `failed`, `completed`, or already `canceled` tasks. Failed tasks should use retry or remove, while completed/canceled tasks are terminal. Task status values are `queued`, `running`, `paused`, `failed`, `completed`, and `canceled`; the rewrite uses `completed` and does not keep a `succeeded` alias. Waiting for user action is not a status: the task remains `running`, and the phase is structured as `{ category, reason? }`, e.g. `{ category: "waiting_user", reason: "install_handoff" }`. The first phase schema intentionally avoids extra fields such as detail/executor/localized message; progress, diagnostics, and UI text are separate concerns. In particular, `retry` retries the same task identified by the same `task_id`; it must not automatically create a new task. Only `completed` and `canceled` are truly terminal task states. Failed tasks are not terminal: if the current task state permits retry, retry reuses the same `task_id` and transitions the existing task back into a runnable state. Retry is a task method over the sealed, already-consumed action plan, not action reuse and not an implicit update refresh. It does not revalidate against the action registry, re-run update check, or match a fresh candidate; if the sealed artifact/package plan has become invalid, the retry fails naturally in the relevant execution phase such as download, validation, package lock, or install. It should resume from the failed phase when the runtime has enough task-local state: download failures retry download, fake install `rejected` failures retry the install handoff, and `package.locked` failures retry entering the package mutation boundary. If the runtime lacks enough intermediate state for a precise phase retry, it may restart from the task-internal action plan beginning, but still as the same task and without creating/restoring an `action_id`. If retry itself fails, getter returns an error and the caller may invoke retry again on the same task when the task state still permits retry. Completed, canceled, and failed tasks remain queryable task objects in the current runtime process indefinitely until an explicit manual cleanup operation removes them; there is no automatic TTL/capacity retention cleanup and no persistent storage. CLI should expose both `task remove ` for removing one in-memory task and `task clean` for explicit bulk cleanup of non-active in-memory tasks. `task remove ` is an explicit single-task operation and may remove `failed`, `completed`, or `canceled` tasks; removing a failed task also discards its retry capability. By default, `task clean` removes `completed` and `canceled` tasks only; it does not remove `failed` tasks because failed tasks may still be retryable. Removing failed tasks requires an explicit option such as `task clean --failed` or `task clean --all-inactive`, where all inactive means `completed`, `canceled`, and `failed`. Removal/cleanup must not remove active tasks such as queued/running/paused tasks; callers must cancel them first. After removal or cleanup, `task get` returns `task.not_found` for the removed task, and associated in-memory sealed action plan/Lua object may be released. The retry method/interface still exists, but retrying a completed or canceled task must immediately return an error and must not resurrect the task or create a replacement task. If the user wants a fresh task object after completion/cancellation, they must use the original task creation/submission path again. Pause/resume are task-level APIs, not download-executor-only public APIs, but whether they are currently allowed is phase/executor-specific and process-specific. Some phases, including `waiting_user`, are state snapshots rather than pausable processes, so they must expose `pause = false` and `resume = false` even while the task status remains `running`. `task resume` is valid only for a `paused` task whose current executor/phase supports resume; calling resume for `running`, `waiting_user`, `queued`, `failed`, `completed`, or `canceled` tasks returns an unsupported-control error and does not mutate state. Failed tasks use retry rather than resume. The first implementation must support pause/resume for download-phase tasks; other phases can expose false capabilities. Task snapshots include current capability flags for these controls so Flutter does not infer executor-specific behavior from status/phase; a capability may be false for the current task/phase, but the API/control model exists. Calling an unsupported control for the current state/phase, such as pausing a `waiting_user` phase or resuming a non-paused task, returns an explicit error such as `task.pause_not_supported`, `task.resume_not_supported`, `task.retry_not_supported`, or `task.cancel_not_supported` and must not mutate task state; unsupported controls are never silent no-ops. Task snapshots include at most the current diagnostic summary needed for UI display, not a diagnostic history or event log; detailed diagnostics/logs use separate query operations. Every field included in a task snapshot, including capabilities and current diagnostic summary, must also be obtainable by active query operations or a combination of query operations within the current runtime process; the callback exists to reduce getter/UI polling pressure, not to become the only way to learn state. If UI/CLI explicitly wants authoritative task state, it calls separate getter operations that query/recompute current internal task state, including at least single-task lookup and task listing/summary for active tasks or package-scoped tasks. Runtime notifications must not be inflated into a log/cursor/replay system merely to answer current-state queries. + +This is not the same as Android system notifications. Android notification/foreground-service behavior remains a later platform side-effect decision. + +Downloads and installers may be mock side-effect executors in the first implementation, but their task/runtime callbacks should be shaped like future product notifications. + +## Deferred to later ADRs + +This ADR intentionally accepts the first runtime/task/notification architecture while leaving real platform side effects for later design work: + +- live HTTP/provider execution as product default; +- real Android download/background worker semantics; +- Android PackageInstaller/intent/URI/SAF/FileProvider/Shizuku/root installer execution semantics; +- Android system notification/foreground-service policy; +- multi-engine or multi-isolate runtime notification sharing; +- batch-update parent/child task APIs, if ever needed. + +The accepted boundary remains: no Flutter-owned provider selection, version comparison, package metadata caching, download task state machine, or installer semantics. diff --git a/docs/architecture/upgradeall-getter-rewrite-wiki.md b/docs/architecture/upgradeall-getter-rewrite-wiki.md index c397b39e..6d0f94ef 100644 --- a/docs/architecture/upgradeall-getter-rewrite-wiki.md +++ b/docs/architecture/upgradeall-getter-rewrite-wiki.md @@ -891,7 +891,7 @@ v1 暂不做 repo/script/artifact 强校验。 - migration records。 - settings。 - credentials references。 -- download task persistent state。 +- later ADR-accepted operation-specific durable records; ADR-0011 keeps runtime task state process-memory only and excludes it from main/cache DB persistence。 ### 11.2 Cache DB diff --git a/docs/lua-api/package-lifecycle.md b/docs/lua-api/package-lifecycle.md index b58f836b..ede75bbb 100644 --- a/docs/lua-api/package-lifecycle.md +++ b/docs/lua-api/package-lifecycle.md @@ -63,7 +63,7 @@ return { The first Phase D implementation slice exposes this boundary only through an offline CLI fixture command: `getter --data-dir update check --fixture `. The fixture is normalized JSON, not live provider output, and the command returns `network_required = false`, update-check status, selected candidate/artifact, and generated download/install action DTOs. It does not execute network providers, download files, persist download tasks, stream progress events, or invoke Android installers. -The second Phase D slice consumes those generated actions through an explicitly offline/fake task lifecycle: `task submit --request `, `task run `, `task list`, `task cancel `, `task events --after --limit `, and `task install-result --status `. This proves getter-owned persistent task state, cancellation, pollable event DTOs, and abstract install handoff recording without live network I/O, background runners, native streaming, Flutter task-state logic, or Android installer calls. +ADR-0011 supersedes the earlier persisted fake task scaffold. The accepted Phase D runtime consumes getter-issued actions through an in-memory process-lifetime runtime: task state is not stored in SQLite, `action_id` is single-use, task submission binds a sealed action plan plus package-version Lua object, mock download/install executors simulate task state, and `RuntimeNotification.task_changed` is pushed to Flutter as a best-effort current snapshot. CLI coverage for this model should use Rust runtime tests or a single-process scripted/debug command rather than pretending separate CLI invocations share task memory. ## post_update diff --git a/todo.md b/todo.md index dc116dd4..ffde96d6 100644 --- a/todo.md +++ b/todo.md @@ -481,7 +481,7 @@ Acceptance: Goal: move from static app/repo display to real update workflows. -Status: second getter-owned offline lifecycle slice in progress. The accepted minimal Phase D work remains intentionally offline/fake: it defines normalized offline update-check DTOs, reuses Rust getter update selection/version comparison, adds `getter --data-dir update check --fixture `, and now adds a command-driven fake task lifecycle for persisted task state, cancellation, pollable task events, and abstract install handoff result recording. It still does not run live providers, perform network downloads, run background workers, invoke Android installers, or add Flutter product task state. +Status: ADR-0011 is accepted for the first in-memory Lua runtime/task/action/RuntimeNotification architecture. Earlier offline CLI/dev scaffolding proved update-check and fake task DTOs, but ADR-0011 supersedes the persisted fake task model: task state is process-memory only, `action_id` is single-use, task submission binds a sealed action plan plus in-memory package-version Lua object, and Flutter receives best-effort push `RuntimeNotification.task_changed` snapshots. The first implementation of this accepted runtime has begun in `getter-core`; live providers, real downloads, Android installers, background workers, and Android system notifications remain deferred to later ADRs. Completed tasks: @@ -494,6 +494,8 @@ Completed tasks: 7. Add main DB task/event/install-handoff tables and storage APIs with TDD coverage. 8. Implement deterministic fake/offline downloader behavior beyond the previous placeholder crate: submit, run, cancel, list, poll events, and record install result. 9. Add CLI commands and BDD coverage for `task submit`, `task run`, `task list`, `task cancel`, `task events`, and `task install-result`. +10. Accept ADR-0011 for the in-memory runtime/task/action/RuntimeNotification model. +11. Add `getter-core::runtime` with in-memory `GetterRuntime`, single-use `action_id`, sealed action plans, package-version Lua object binding, generic `user-result`, mock download/install state, package-level non-waiting lock, task controls, remove/clean, and RuntimeNotification DTOs with TDD coverage. Completed additional UI/bridge slice: @@ -503,19 +505,20 @@ Completed additional UI/bridge slice: Remaining tasks: -1. Implement live provider/downloader behavior beyond the fake/offline proof. -2. Add native stream/backpressure runtime beyond the current pollable CLI/dev event contract. -3. Decide and implement background worker/restart/retry/resume policy for real downloads. -4. Define Android production install handoff URI/SAF/permission/notification details and wire platform adapter execution. -5. Add product-level Flutter BDD for update/download user flows after live/provider/background/installer decisions are accepted; the current slice only covers read-only DTO rendering. +1. Wire `getter-core::runtime` into getter operations/CLI single-process debug tooling without reintroducing persisted task state. +2. Wire the native bridge process-lifetime runtime singleton and Flutter push stream/EventChannel for `RuntimeNotification.task_changed` snapshots. +3. Replace or retire the older persisted fake CLI task scaffold so public task status/control language uses ADR-0011 (`completed`, `user-result`, remove/clean, no cross-invocation task state). +4. Implement live provider/downloader behavior beyond the fake/offline proof after a later ADR accepts real side-effect details. +5. Define Android production install handoff URI/SAF/permission/notification details and wire platform adapter execution after later ADRs. +6. Add product-level Flutter BDD for update/download user flows after live/provider/background/installer decisions are accepted; the current slice only covers read-only DTO rendering. Acceptance progress: - CLI can run an offline fixture update check: done. -- Getter can persist and list fake/offline task state: done for CLI/dev slice. -- Getter can cancel queued/running fake tasks and reject invalid terminal cancellation: done. -- Getter can expose pollable task events with cursor/limit: done for CLI/dev slice; native streaming remains deferred. -- Getter can record abstract install handoff requests/results: done for CLI/dev slice; Android installer execution remains deferred. +- Older CLI/dev fake task scaffold can persist/list task state: done, but superseded by ADR-0011 and slated for replacement/retirement. +- `getter-core::runtime` can manage in-memory tasks, controls, `user-result`, retry, package lock, remove/clean, and notifications: first TDD slice done. +- Getter can expose pollable task events with cursor/limit in the older CLI/dev scaffold; ADR-0011 native push stream wiring remains pending. +- Getter can record abstract install handoff requests/results in the older scaffold; ADR-0011 uses generic `user-result` and mock install waiting-user state, Android installer execution remains deferred. - Flutter displays getter task/event DTOs rather than calculating status itself: done for read-only CLI/dev bridge slice. - Android platform adapter owns permissions/notifications/installer handoff: documented/deferred; no Android execution added in this slice. From dba34ec7e4cf9a927658552b4624f4ca01ea8f75 Mon Sep 17 00:00:00 2001 From: Xiangzhe Date: Thu, 25 Jun 2026 10:37:50 +0800 Subject: [PATCH 30/85] feat(app): bridge runtime task operations --- app_flutter/android/app/build.gradle | 1 + .../upgradeall/GetterBridgeRequestBuilder.kt | 15 ++ .../net/xzos/upgradeall/MainActivity.kt | 10 ++ .../GetterBridgeRequestBuilderTest.kt | 44 +++++ .../net/xzos/upgradeall/getter/NativeLib.kt | 1 + app_flutter/lib/native_getter_adapter.dart | 18 ++ .../test/native_getter_adapter_test.dart | 39 ++++ .../net/xzos/upgradeall/getter/NativeLib.kt | 1 + .../src/main/rust/api_proxy/src/lib.rs | 169 +++++++++++++++++- core-getter/src/main/rust/getter | 2 +- 10 files changed, 298 insertions(+), 2 deletions(-) create mode 100644 app_flutter/android/app/src/main/kotlin/net/xzos/upgradeall/GetterBridgeRequestBuilder.kt create mode 100644 app_flutter/android/app/src/test/kotlin/net/xzos/upgradeall/GetterBridgeRequestBuilderTest.kt diff --git a/app_flutter/android/app/build.gradle b/app_flutter/android/app/build.gradle index c7501bb0..880b5c76 100644 --- a/app_flutter/android/app/build.gradle +++ b/app_flutter/android/app/build.gradle @@ -89,4 +89,5 @@ flutter { dependencies { implementation project(':getter_bridge') testImplementation "junit:junit:4.13.2" + testImplementation "org.json:json:20250517" } diff --git a/app_flutter/android/app/src/main/kotlin/net/xzos/upgradeall/GetterBridgeRequestBuilder.kt b/app_flutter/android/app/src/main/kotlin/net/xzos/upgradeall/GetterBridgeRequestBuilder.kt new file mode 100644 index 00000000..45a3eaa2 --- /dev/null +++ b/app_flutter/android/app/src/main/kotlin/net/xzos/upgradeall/GetterBridgeRequestBuilder.kt @@ -0,0 +1,15 @@ +package net.xzos.upgradeall + +import org.json.JSONObject + +object GetterBridgeRequestBuilder { + fun runtimeOperationRequest(args: Map<*, *>): String { + val operation = args["operation"] as? String + ?: throw IllegalArgumentException("operation is required") + val payload = args["payload"] as? Map<*, *> ?: emptyMap() + return JSONObject() + .put("operation", operation) + .put("payload", JSONObject(payload)) + .toString() + } +} diff --git a/app_flutter/android/app/src/main/kotlin/net/xzos/upgradeall/MainActivity.kt b/app_flutter/android/app/src/main/kotlin/net/xzos/upgradeall/MainActivity.kt index 23f35a4d..01623097 100644 --- a/app_flutter/android/app/src/main/kotlin/net/xzos/upgradeall/MainActivity.kt +++ b/app_flutter/android/app/src/main/kotlin/net/xzos/upgradeall/MainActivity.kt @@ -48,6 +48,10 @@ class MainActivity : FlutterActivity() { nativeLib.legacyReportList(legacyReportListRequest()) } + "runtimeOperation" -> runGetterBridge(result) { + nativeLib.runtimeOperation(runtimeOperationRequest(call)) + } + else -> result.notImplemented() } } @@ -168,6 +172,12 @@ class MainActivity : FlutterActivity() { .toString() } + private fun runtimeOperationRequest(call: MethodCall): String { + val args = call.arguments as? Map<*, *> + ?: throw IllegalArgumentException("runtime operation arguments are required") + return GetterBridgeRequestBuilder.runtimeOperationRequest(args) + } + private fun getterDataDir(): File = File(filesDir, "getter") private fun prepareLegacyRoomImport(): Map { diff --git a/app_flutter/android/app/src/test/kotlin/net/xzos/upgradeall/GetterBridgeRequestBuilderTest.kt b/app_flutter/android/app/src/test/kotlin/net/xzos/upgradeall/GetterBridgeRequestBuilderTest.kt new file mode 100644 index 00000000..67ccba2b --- /dev/null +++ b/app_flutter/android/app/src/test/kotlin/net/xzos/upgradeall/GetterBridgeRequestBuilderTest.kt @@ -0,0 +1,44 @@ +package net.xzos.upgradeall + +import org.json.JSONObject +import org.junit.Assert.assertEquals +import org.junit.Assert.assertThrows +import org.junit.Test + +class GetterBridgeRequestBuilderTest { + @Test + fun runtimeOperationRequestPreservesOperationAndPayload() { + val json = JSONObject( + GetterBridgeRequestBuilder.runtimeOperationRequest( + mapOf( + "operation" to "task_get", + "payload" to mapOf("task_id" to "task-1"), + ), + ), + ) + + assertEquals("task_get", json.getString("operation")) + assertEquals("task-1", json.getJSONObject("payload").getString("task_id")) + } + + @Test + fun runtimeOperationRequestDefaultsMissingPayloadToEmptyObject() { + val json = JSONObject( + GetterBridgeRequestBuilder.runtimeOperationRequest( + mapOf("operation" to "task_list"), + ), + ) + + assertEquals("task_list", json.getString("operation")) + assertEquals(0, json.getJSONObject("payload").length()) + } + + @Test + fun runtimeOperationRequestRequiresOperation() { + val error = assertThrows(IllegalArgumentException::class.java) { + GetterBridgeRequestBuilder.runtimeOperationRequest(mapOf("payload" to emptyMap())) + } + + assertEquals("operation is required", error.message) + } +} diff --git a/app_flutter/android/getter_bridge/src/main/kotlin/net/xzos/upgradeall/getter/NativeLib.kt b/app_flutter/android/getter_bridge/src/main/kotlin/net/xzos/upgradeall/getter/NativeLib.kt index 66686143..550ccefc 100644 --- a/app_flutter/android/getter_bridge/src/main/kotlin/net/xzos/upgradeall/getter/NativeLib.kt +++ b/app_flutter/android/getter_bridge/src/main/kotlin/net/xzos/upgradeall/getter/NativeLib.kt @@ -15,6 +15,7 @@ class NativeLib { external fun applyInstalledAutogen(requestJson: String): String external fun importLegacyRoomDatabase(requestJson: String): String external fun legacyReportList(requestJson: String): String + external fun runtimeOperation(requestJson: String): String fun runServerLambda(context: Context, callback: (String) -> Unit): String { return runServer(context, RunServerCallback(callback)) diff --git a/app_flutter/lib/native_getter_adapter.dart b/app_flutter/lib/native_getter_adapter.dart index 14d00b25..b9b70502 100644 --- a/app_flutter/lib/native_getter_adapter.dart +++ b/app_flutter/lib/native_getter_adapter.dart @@ -87,6 +87,24 @@ class MethodChannelGetterAdapter extends FakeGetterAdapter { return InstalledAutogenApplyResult.fromJson(data); } + /// Invoke a getter runtime operation through the native bridge. + /// + /// This is an internal/debug bridge primitive for ADR-0011 wiring. Product UI + /// should use typed getter operations and getter-issued `action_id`s rather + /// than assembling runtime action plans in Dart. + Future> invokeRuntimeOperation( + String operation, { + Map payload = const {}, + }) { + return _invokeGetterData( + 'runtimeOperation', + { + 'operation': operation, + 'payload': payload, + }, + ); + } + Future> _invokeGetterData( String method, Map arguments, diff --git a/app_flutter/test/native_getter_adapter_test.dart b/app_flutter/test/native_getter_adapter_test.dart index f1cc5ffe..73594af4 100644 --- a/app_flutter/test/native_getter_adapter_test.dart +++ b/app_flutter/test/native_getter_adapter_test.dart @@ -163,6 +163,45 @@ void main() { expect(reports.single.code, 'migration.imported'); }); + test('native runtime operation forwards operation and payload', () async { + MethodCall? captured; + TestDefaultBinaryMessengerBinding.instance.defaultBinaryMessenger + .setMockMethodCallHandler(channel, (call) async { + captured = call; + return jsonEncode({ + 'ok': true, + 'command': 'runtime operation', + 'data': { + 'task_id': 'task-1', + 'package_id': 'android/org.fdroid.fdroid', + 'status': 'completed', + 'phase': {'category': 'completed'}, + 'capabilities': { + 'cancel': false, + 'pause': false, + 'resume': false, + 'retry': false, + }, + 'updated_at': 1, + }, + 'warnings': [], + }); + }); + + const adapter = MethodChannelGetterAdapter(channel: channel); + final data = await adapter.invokeRuntimeOperation( + 'task_get', + payload: const {'task_id': 'task-1'}, + ); + + expect(captured!.method, 'runtimeOperation'); + expect(captured!.arguments, { + 'operation': 'task_get', + 'payload': {'task_id': 'task-1'}, + }); + expect(data['status'], 'completed'); + }); + test('native adapter maps getter error envelope to bridge exception', () async { TestDefaultBinaryMessengerBinding.instance.defaultBinaryMessenger diff --git a/core-getter/src/main/java/net/xzos/upgradeall/getter/NativeLib.kt b/core-getter/src/main/java/net/xzos/upgradeall/getter/NativeLib.kt index e064a726..9765557f 100644 --- a/core-getter/src/main/java/net/xzos/upgradeall/getter/NativeLib.kt +++ b/core-getter/src/main/java/net/xzos/upgradeall/getter/NativeLib.kt @@ -20,6 +20,7 @@ class NativeLib { external fun applyInstalledAutogen(requestJson: String): String external fun importLegacyRoomDatabase(requestJson: String): String external fun legacyReportList(requestJson: String): String + external fun runtimeOperation(requestJson: String): String fun runServerLambda(context: Context, callback: (String) -> Unit): String { return runServer(context, RunServerCallback(callback)) diff --git a/core-getter/src/main/rust/api_proxy/src/lib.rs b/core-getter/src/main/rust/api_proxy/src/lib.rs index 7c1df8a1..0ada7651 100644 --- a/core-getter/src/main/rust/api_proxy/src/lib.rs +++ b/core-getter/src/main/rust/api_proxy/src/lib.rs @@ -2,6 +2,7 @@ extern crate jni; use getter::operations::autogen::{self, AutogenAcceptance, AutogenOperationError}; use getter::operations::legacy_room::{self, LegacyRoomOperationError}; +use getter::operations::runtime as runtime_operations; use getter::rpc::server::run_server_hanging; #[cfg(target_os = "android")] use getter::rustls_platform_verifier; @@ -11,6 +12,7 @@ use serde::Deserialize; use serde_json::{json, Value}; use std::path::{Path, PathBuf}; use std::sync::mpsc::channel; +use std::sync::{Mutex, OnceLock}; use std::thread; use upgradeall_platform_adapter::InstalledInventoryScanOptions; #[cfg(target_os = "android")] @@ -19,6 +21,8 @@ use upgradeall_platform_adapter::PlatformAdapter; const MAIN_DB_FILE: &str = "main.db"; const CACHE_DB_FILE: &str = "cache.db"; +static GETTER_RUNTIME: OnceLock> = OnceLock::new(); + #[derive(Debug, Deserialize)] struct PreviewInstalledAutogenRequest { data_dir: PathBuf, @@ -45,6 +49,13 @@ struct LegacyReportListRequest { data_dir: PathBuf, } +#[derive(Debug, Deserialize)] +struct RuntimeOperationRequest { + operation: String, + #[serde(default)] + payload: Value, +} + #[derive(Debug, Default, Deserialize)] struct ApplyInstalledAutogenAcceptance { #[serde(default)] @@ -134,7 +145,10 @@ pub extern "C" fn Java_net_xzos_upgradeall_getter_NativeLib_initializeBridge<'lo context: JObject<'local>, ) -> JString<'local> { let response = match init_android_integrations(&mut env, &context) { - Ok(()) => success_envelope("bridge initialize", json!({ "initialized": true })), + Ok(()) => { + init_getter_runtime(); + success_envelope("bridge initialize", json!({ "initialized": true })) + } Err(error) => error_envelope( "bridge initialize", "bridge.initialize_error", @@ -178,6 +192,20 @@ pub extern "C" fn Java_net_xzos_upgradeall_getter_NativeLib_applyInstalledAutoge java_string_or_fallback(&mut env, response) } +#[no_mangle] +pub extern "C" fn Java_net_xzos_upgradeall_getter_NativeLib_runtimeOperation<'local>( + mut env: JNIEnv<'local>, + _: JObject<'local>, + request_json: JString<'local>, +) -> JString<'local> { + let command = "runtime operation"; + let response = match jstring_to_string(&mut env, &request_json).and_then(runtime_operation) { + Ok(data) => success_envelope(command, data), + Err(error) => operation_error_envelope(command, error), + }; + java_string_or_fallback(&mut env, response) +} + #[no_mangle] pub extern "C" fn Java_net_xzos_upgradeall_getter_NativeLib_importLegacyRoomDatabase<'local>( mut env: JNIEnv<'local>, @@ -264,6 +292,54 @@ fn legacy_report_list(request_json: &str) -> Result legacy_room::report_list_json(&request.data_dir).map_err(BridgeOperationError::from) } +fn init_getter_runtime() -> &'static Mutex { + GETTER_RUNTIME.get_or_init(|| Mutex::new(getter::core::runtime::GetterRuntime::new())) +} + +fn runtime_operation(request_json: String) -> Result { + let runtime = init_getter_runtime(); + let mut runtime = runtime + .lock() + .map_err(|_| BridgeOperationError::RuntimePoisoned)?; + runtime_operation_with_runtime(&mut runtime, &request_json) +} + +fn runtime_operation_with_runtime( + runtime: &mut getter::core::runtime::GetterRuntime, + request_json: &str, +) -> Result { + let request: RuntimeOperationRequest = serde_json::from_str(request_json) + .map_err(|source| BridgeOperationError::InvalidRequest(source.to_string()))?; + let payload = if request.payload.is_null() { + "{}".to_owned() + } else { + request.payload.to_string() + }; + match request.operation.as_str() { + "task_submit" => runtime_operations::submit_action_json(runtime, &payload), + "task_get" => runtime_operations::task_get_json(runtime, &payload), + "task_list" => runtime_operations::task_list_json(runtime, &payload), + "task_start" => runtime_operations::task_start_json(runtime, &payload), + "task_download_progress" => { + runtime_operations::task_download_progress_json(runtime, &payload) + } + "task_complete_download" => { + runtime_operations::task_complete_download_json(runtime, &payload) + } + "task_pause" => runtime_operations::task_pause_json(runtime, &payload), + "task_resume" => runtime_operations::task_resume_json(runtime, &payload), + "task_user_result" => runtime_operations::task_user_result_json(runtime, &payload), + "task_cancel" => runtime_operations::task_cancel_json(runtime, &payload), + "task_retry" => runtime_operations::task_retry_json(runtime, &payload), + "task_remove" => runtime_operations::task_remove_json(runtime, &payload), + "task_clean" => runtime_operations::task_clean_json(runtime, &payload), + other => Err(runtime_operations::RuntimeOperationError::InvalidRequest( + format!("unsupported runtime operation '{other}'"), + )), + } + .map_err(BridgeOperationError::Runtime) +} + impl ApplyInstalledAutogenAcceptance { fn into_autogen_acceptance(self) -> Result { match self.mode.as_deref().unwrap_or("all") { @@ -395,6 +471,10 @@ enum BridgeOperationError { Autogen(String), #[error("migration error: {0}")] Migration(#[from] LegacyRoomOperationError), + #[error("runtime error: {0}")] + Runtime(#[from] runtime_operations::RuntimeOperationError), + #[error("runtime lock is poisoned")] + RuntimePoisoned, } impl BridgeOperationError { @@ -466,6 +546,8 @@ impl BridgeOperationError { .detail() .or_else(|| error.report_path().map(|path| path.display().to_string())), ), + Self::Runtime(error) => (error.code(), error.message(), error.detail()), + Self::RuntimePoisoned => ("runtime.poisoned", "Getter runtime lock is poisoned", None), } } } @@ -489,6 +571,10 @@ impl From for BridgeOperationError { #[cfg(test)] mod tests { use super::*; + use getter::core::{ + runtime::{PackageVersionLuaObject, SealedActionPlan}, + UpdateAction, + }; #[test] fn packages_acceptance_defaults_to_all() { @@ -515,4 +601,85 @@ mod tests { AutogenAcceptance::AcceptAll => panic!("expected explicit package acceptance"), } } + + #[test] + fn runtime_dispatcher_uses_in_memory_runtime_controls() { + let mut runtime = getter::core::runtime::GetterRuntime::new(); + let action = runtime_operations::issue_action( + &mut runtime, + SealedActionPlan { + package_id: "android/org.fdroid.fdroid".parse().expect("package id"), + actions: vec![ + UpdateAction::Download { + url: "https://example.invalid/app.apk".to_owned(), + file_name: "app.apk".to_owned(), + }, + UpdateAction::Install { + installer: "android_package".to_owned(), + file: "app.apk".to_owned(), + }, + ], + lua_object: PackageVersionLuaObject { + object_id: "lua:android/org.fdroid.fdroid".to_owned(), + dependency_digest: "sha256:test".to_owned(), + }, + }, + ); + let action_id = action["action_id"].as_str().expect("action id"); + + let submitted = runtime_operation_with_runtime( + &mut runtime, + &json!({ + "operation": "task_submit", + "payload": { "action_id": action_id } + }) + .to_string(), + ) + .expect("submit"); + let task_id = submitted["task_id"].as_str().expect("task id"); + assert_eq!(submitted["status"], "queued"); + + runtime_operation_with_runtime( + &mut runtime, + &json!({ "operation": "task_start", "payload": { "task_id": task_id } }).to_string(), + ) + .expect("start"); + let waiting = runtime_operation_with_runtime( + &mut runtime, + &json!({ + "operation": "task_complete_download", + "payload": { "task_id": task_id } + }) + .to_string(), + ) + .expect("complete download"); + assert_eq!(waiting["status"], "running"); + assert_eq!(waiting["phase"]["category"], "waiting_user"); + + let completed = runtime_operation_with_runtime( + &mut runtime, + &json!({ + "operation": "task_user_result", + "payload": { "task_id": task_id, "result": "accepted" } + }) + .to_string(), + ) + .expect("user result"); + assert_eq!(completed["status"], "completed"); + } + + #[test] + fn runtime_dispatcher_rejects_unknown_operation() { + let mut runtime = getter::core::runtime::GetterRuntime::new(); + + let error = runtime_operation_with_runtime( + &mut runtime, + &json!({ "operation": "task_install_result", "payload": {} }).to_string(), + ) + .unwrap_err(); + + let (code, _, detail) = error.parts(); + assert_eq!(code, "runtime.invalid_request"); + assert!(detail.unwrap().contains("unsupported runtime operation")); + } } diff --git a/core-getter/src/main/rust/getter b/core-getter/src/main/rust/getter index 8e3e3c12..aa0b75ee 160000 --- a/core-getter/src/main/rust/getter +++ b/core-getter/src/main/rust/getter @@ -1 +1 @@ -Subproject commit 8e3e3c12803f1f8c2659361bfeba78c666917772 +Subproject commit aa0b75eed55f5be2614ec40623c13aa1982d4559 From 108625a898f4e44d71ae388d3931ae9c49d3ce32 Mon Sep 17 00:00:00 2001 From: Xiangzhe Date: Thu, 25 Jun 2026 10:43:10 +0800 Subject: [PATCH 31/85] feat(app): stream runtime notifications --- .../net/xzos/upgradeall/MainActivity.kt | 69 ++++++++++++++- .../net/xzos/upgradeall/getter/NativeLib.kt | 1 + app_flutter/lib/native_getter_adapter.dart | 16 +++- .../test/native_getter_adapter_test.dart | 41 +++++++++ .../net/xzos/upgradeall/getter/NativeLib.kt | 1 + .../src/main/rust/api_proxy/src/lib.rs | 86 ++++++++++++++++++- 6 files changed, 209 insertions(+), 5 deletions(-) diff --git a/app_flutter/android/app/src/main/kotlin/net/xzos/upgradeall/MainActivity.kt b/app_flutter/android/app/src/main/kotlin/net/xzos/upgradeall/MainActivity.kt index 01623097..3cb29a86 100644 --- a/app_flutter/android/app/src/main/kotlin/net/xzos/upgradeall/MainActivity.kt +++ b/app_flutter/android/app/src/main/kotlin/net/xzos/upgradeall/MainActivity.kt @@ -4,6 +4,7 @@ import android.os.Handler import android.os.Looper import io.flutter.embedding.android.FlutterActivity import io.flutter.embedding.engine.FlutterEngine +import io.flutter.plugin.common.EventChannel import io.flutter.plugin.common.MethodCall import io.flutter.plugin.common.MethodChannel import java.io.File @@ -16,10 +17,28 @@ class MainActivity : FlutterActivity() { private val legacyMigrationExecutor = Executors.newSingleThreadExecutor() private val getterBridgeExecutor = Executors.newSingleThreadExecutor() private val mainHandler = Handler(Looper.getMainLooper()) + @Volatile + private var runtimeEventSink: EventChannel.EventSink? = null private val nativeLib by lazy { NativeLib() } override fun configureFlutterEngine(flutterEngine: FlutterEngine) { super.configureFlutterEngine(flutterEngine) + EventChannel( + flutterEngine.dartExecutor.binaryMessenger, + RUNTIME_NOTIFICATION_CHANNEL, + ).setStreamHandler( + object : EventChannel.StreamHandler { + override fun onListen(arguments: Any?, events: EventChannel.EventSink?) { + runtimeEventSink = events + emitRuntimeNotifications() + } + + override fun onCancel(arguments: Any?) { + runtimeEventSink = null + } + }, + ) + MethodChannel( flutterEngine.dartExecutor.binaryMessenger, GETTER_BRIDGE_CHANNEL, @@ -48,7 +67,7 @@ class MainActivity : FlutterActivity() { nativeLib.legacyReportList(legacyReportListRequest()) } - "runtimeOperation" -> runGetterBridge(result) { + "runtimeOperation" -> runGetterBridge(result, emitRuntimeNotifications = true) { nativeLib.runtimeOperation(runtimeOperationRequest(call)) } @@ -89,11 +108,23 @@ class MainActivity : FlutterActivity() { super.onDestroy() } - private fun runGetterBridge(result: MethodChannel.Result, operation: () -> String) { + private fun runGetterBridge( + result: MethodChannel.Result, + emitRuntimeNotifications: Boolean = false, + operation: () -> String, + ) { getterBridgeExecutor.execute { try { val response = operation() - mainHandler.post { result.success(response) } + val notifications = if (emitRuntimeNotifications) { + drainRuntimeNotificationEvents() + } else { + emptyList() + } + mainHandler.post { + result.success(response) + emitRuntimeNotifications(notifications) + } } catch (error: UnsatisfiedLinkError) { mainHandler.post { result.error( @@ -178,6 +209,37 @@ class MainActivity : FlutterActivity() { return GetterBridgeRequestBuilder.runtimeOperationRequest(args) } + private fun emitRuntimeNotifications() { + getterBridgeExecutor.execute { + val notifications = drainRuntimeNotificationEvents() + mainHandler.post { emitRuntimeNotifications(notifications) } + } + } + + private fun emitRuntimeNotifications(notifications: List) { + val sink = runtimeEventSink ?: return + for (notification in notifications) { + sink.success(notification) + } + } + + private fun drainRuntimeNotificationEvents(): List { + return try { + val envelope = JSONObject(nativeLib.drainRuntimeNotifications()) + if (!envelope.optBoolean("ok", false)) { + return emptyList() + } + val notifications = envelope + .getJSONObject("data") + .getJSONArray("notifications") + List(notifications.length()) { index -> notifications.getJSONObject(index).toString() } + } catch (_: UnsatisfiedLinkError) { + emptyList() + } catch (_: Exception) { + emptyList() + } + } + private fun getterDataDir(): File = File(filesDir, "getter") private fun prepareLegacyRoomImport(): Map { @@ -192,6 +254,7 @@ class MainActivity : FlutterActivity() { private companion object { const val GETTER_BRIDGE_CHANNEL = "net.xzos.upgradeall/getter_bridge" + const val RUNTIME_NOTIFICATION_CHANNEL = "net.xzos.upgradeall/runtime_notifications" const val LEGACY_MIGRATION_CHANNEL = "net.xzos.upgradeall/legacy_migration" const val LEGACY_ROOM_DB_NAME = "app_metadata_database.db" } diff --git a/app_flutter/android/getter_bridge/src/main/kotlin/net/xzos/upgradeall/getter/NativeLib.kt b/app_flutter/android/getter_bridge/src/main/kotlin/net/xzos/upgradeall/getter/NativeLib.kt index 550ccefc..f0165f8c 100644 --- a/app_flutter/android/getter_bridge/src/main/kotlin/net/xzos/upgradeall/getter/NativeLib.kt +++ b/app_flutter/android/getter_bridge/src/main/kotlin/net/xzos/upgradeall/getter/NativeLib.kt @@ -16,6 +16,7 @@ class NativeLib { external fun importLegacyRoomDatabase(requestJson: String): String external fun legacyReportList(requestJson: String): String external fun runtimeOperation(requestJson: String): String + external fun drainRuntimeNotifications(): String fun runServerLambda(context: Context, callback: (String) -> Unit): String { return runServer(context, RunServerCallback(callback)) diff --git a/app_flutter/lib/native_getter_adapter.dart b/app_flutter/lib/native_getter_adapter.dart index b9b70502..ffd3101f 100644 --- a/app_flutter/lib/native_getter_adapter.dart +++ b/app_flutter/lib/native_getter_adapter.dart @@ -16,9 +16,14 @@ class MethodChannelGetterAdapter extends FakeGetterAdapter { MethodChannel channel = const MethodChannel( 'net.xzos.upgradeall/getter_bridge', ), - }) : _channel = channel; + EventChannel runtimeNotificationChannel = const EventChannel( + 'net.xzos.upgradeall/runtime_notifications', + ), + }) : _channel = channel, + _runtimeNotificationChannel = runtimeNotificationChannel; final MethodChannel _channel; + final EventChannel _runtimeNotificationChannel; @override bool get supportsLegacyRoomImport => true; @@ -92,6 +97,15 @@ class MethodChannelGetterAdapter extends FakeGetterAdapter { /// This is an internal/debug bridge primitive for ADR-0011 wiring. Product UI /// should use typed getter operations and getter-issued `action_id`s rather /// than assembling runtime action plans in Dart. + Stream> runtimeNotifications() { + return _runtimeNotificationChannel.receiveBroadcastStream().map((event) { + if (event is String) { + return _asMap(jsonDecode(event), 'runtime notification'); + } + return _asMap(event, 'runtime notification'); + }); + } + Future> invokeRuntimeOperation( String operation, { Map payload = const {}, diff --git a/app_flutter/test/native_getter_adapter_test.dart b/app_flutter/test/native_getter_adapter_test.dart index 73594af4..af3afde1 100644 --- a/app_flutter/test/native_getter_adapter_test.dart +++ b/app_flutter/test/native_getter_adapter_test.dart @@ -9,10 +9,14 @@ void main() { TestWidgetsFlutterBinding.ensureInitialized(); const channel = MethodChannel('test/getter_bridge'); + const eventChannel = EventChannel('test/runtime_notifications'); + const eventMethodChannel = MethodChannel('test/runtime_notifications'); tearDown(() { TestDefaultBinaryMessengerBinding.instance.defaultBinaryMessenger .setMockMethodCallHandler(channel, null); + TestDefaultBinaryMessengerBinding.instance.defaultBinaryMessenger + .setMockMethodCallHandler(eventMethodChannel, null); }); test('native preview sends scan options and parses getter envelope', @@ -163,6 +167,43 @@ void main() { expect(reports.single.code, 'migration.imported'); }); + test('runtime notification stream decodes pushed JSON events', () async { + TestDefaultBinaryMessengerBinding.instance.defaultBinaryMessenger + .setMockMethodCallHandler(eventMethodChannel, (call) async { + if (call.method == 'listen') { + await TestDefaultBinaryMessengerBinding.instance.defaultBinaryMessenger + .handlePlatformMessage( + 'test/runtime_notifications', + const StandardMethodCodec().encodeSuccessEnvelope( + jsonEncode({ + 'kind': 'task_changed', + 'task': { + 'task_id': 'task-1', + 'package_id': 'android/org.fdroid.fdroid', + 'status': 'completed', + }, + }), + ), + (_) {}, + ); + } + return null; + }); + + const adapter = MethodChannelGetterAdapter( + channel: channel, + runtimeNotificationChannel: eventChannel, + ); + + final notification = await adapter.runtimeNotifications().first; + + expect(notification['kind'], 'task_changed'); + expect( + (notification['task'] as Map)['task_id'], + 'task-1', + ); + }); + test('native runtime operation forwards operation and payload', () async { MethodCall? captured; TestDefaultBinaryMessengerBinding.instance.defaultBinaryMessenger diff --git a/core-getter/src/main/java/net/xzos/upgradeall/getter/NativeLib.kt b/core-getter/src/main/java/net/xzos/upgradeall/getter/NativeLib.kt index 9765557f..6cea7ace 100644 --- a/core-getter/src/main/java/net/xzos/upgradeall/getter/NativeLib.kt +++ b/core-getter/src/main/java/net/xzos/upgradeall/getter/NativeLib.kt @@ -21,6 +21,7 @@ class NativeLib { external fun importLegacyRoomDatabase(requestJson: String): String external fun legacyReportList(requestJson: String): String external fun runtimeOperation(requestJson: String): String + external fun drainRuntimeNotifications(): String fun runServerLambda(context: Context, callback: (String) -> Unit): String { return runServer(context, RunServerCallback(callback)) diff --git a/core-getter/src/main/rust/api_proxy/src/lib.rs b/core-getter/src/main/rust/api_proxy/src/lib.rs index 0ada7651..e968ca2f 100644 --- a/core-getter/src/main/rust/api_proxy/src/lib.rs +++ b/core-getter/src/main/rust/api_proxy/src/lib.rs @@ -10,6 +10,7 @@ use jni::objects::{JObject, JString, JValue}; use jni::JNIEnv; use serde::Deserialize; use serde_json::{json, Value}; +use std::collections::VecDeque; use std::path::{Path, PathBuf}; use std::sync::mpsc::channel; use std::sync::{Mutex, OnceLock}; @@ -20,8 +21,10 @@ use upgradeall_platform_adapter::PlatformAdapter; const MAIN_DB_FILE: &str = "main.db"; const CACHE_DB_FILE: &str = "cache.db"; +const MAX_RUNTIME_NOTIFICATION_QUEUE: usize = 64; static GETTER_RUNTIME: OnceLock> = OnceLock::new(); +static RUNTIME_NOTIFICATIONS: OnceLock>> = OnceLock::new(); #[derive(Debug, Deserialize)] struct PreviewInstalledAutogenRequest { @@ -206,6 +209,19 @@ pub extern "C" fn Java_net_xzos_upgradeall_getter_NativeLib_runtimeOperation<'lo java_string_or_fallback(&mut env, response) } +#[no_mangle] +pub extern "C" fn Java_net_xzos_upgradeall_getter_NativeLib_drainRuntimeNotifications<'local>( + mut env: JNIEnv<'local>, + _: JObject<'local>, +) -> JString<'local> { + let command = "runtime notifications drain"; + let response = match drain_runtime_notifications() { + Ok(data) => success_envelope(command, data), + Err(error) => operation_error_envelope(command, error), + }; + java_string_or_fallback(&mut env, response) +} + #[no_mangle] pub extern "C" fn Java_net_xzos_upgradeall_getter_NativeLib_importLegacyRoomDatabase<'local>( mut env: JNIEnv<'local>, @@ -293,7 +309,38 @@ fn legacy_report_list(request_json: &str) -> Result } fn init_getter_runtime() -> &'static Mutex { - GETTER_RUNTIME.get_or_init(|| Mutex::new(getter::core::runtime::GetterRuntime::new())) + GETTER_RUNTIME.get_or_init(|| { + let mut runtime = getter::core::runtime::GetterRuntime::new(); + runtime.set_notification_sink(|notification| { + enqueue_runtime_notification(notification); + }); + Mutex::new(runtime) + }) +} + +fn runtime_notification_queue() -> &'static Mutex> { + RUNTIME_NOTIFICATIONS.get_or_init(|| Mutex::new(VecDeque::new())) +} + +fn enqueue_runtime_notification(notification: getter::core::runtime::RuntimeNotification) { + let Ok(value) = serde_json::to_value(notification) else { + return; + }; + let Ok(mut queue) = runtime_notification_queue().lock() else { + return; + }; + if queue.len() >= MAX_RUNTIME_NOTIFICATION_QUEUE { + queue.pop_front(); + } + queue.push_back(value); +} + +fn drain_runtime_notifications() -> Result { + let mut queue = runtime_notification_queue() + .lock() + .map_err(|_| BridgeOperationError::RuntimeNotificationQueuePoisoned)?; + let notifications: Vec = queue.drain(..).collect(); + Ok(json!({ "notifications": notifications })) } fn runtime_operation(request_json: String) -> Result { @@ -475,6 +522,8 @@ enum BridgeOperationError { Runtime(#[from] runtime_operations::RuntimeOperationError), #[error("runtime lock is poisoned")] RuntimePoisoned, + #[error("runtime notification queue is poisoned")] + RuntimeNotificationQueuePoisoned, } impl BridgeOperationError { @@ -548,6 +597,11 @@ impl BridgeOperationError { ), Self::Runtime(error) => (error.code(), error.message(), error.detail()), Self::RuntimePoisoned => ("runtime.poisoned", "Getter runtime lock is poisoned", None), + Self::RuntimeNotificationQueuePoisoned => ( + "runtime.notification_queue_poisoned", + "Getter runtime notification queue is poisoned", + None, + ), } } } @@ -682,4 +736,34 @@ mod tests { assert_eq!(code, "runtime.invalid_request"); assert!(detail.unwrap().contains("unsupported runtime operation")); } + + #[test] + fn runtime_notification_queue_is_bounded_and_drained() { + drain_runtime_notifications().expect("clear queue"); + for index in 0..(MAX_RUNTIME_NOTIFICATION_QUEUE + 1) { + enqueue_runtime_notification(getter::core::runtime::RuntimeNotification::TaskChanged { + task: getter::core::runtime::TaskSnapshot { + task_id: format!("task-{index}"), + package_id: "android/org.fdroid.fdroid".parse().expect("package id"), + status: getter::core::runtime::RuntimeTaskStatus::Running, + phase: getter::core::runtime::TaskPhase::new( + getter::core::runtime::TaskPhaseCategory::Download, + ), + progress: None, + capabilities: getter::core::runtime::TaskCapabilities::default(), + current_diagnostic: None, + updated_at: index as u64, + }, + }); + } + + let drained = drain_runtime_notifications().expect("drain notifications"); + let notifications = drained["notifications"].as_array().expect("notifications"); + + assert_eq!(notifications.len(), MAX_RUNTIME_NOTIFICATION_QUEUE); + assert_eq!(notifications[0]["task"]["task_id"], "task-1"); + assert_eq!(notifications.last().unwrap()["task"]["task_id"], "task-64"); + let empty = drain_runtime_notifications().expect("drain empty queue"); + assert_eq!(empty["notifications"].as_array().unwrap().len(), 0); + } } From 2b98939d6f1fffc46f6c68eae7eb1bf21826f30c Mon Sep 17 00:00:00 2001 From: Xiangzhe Date: Thu, 25 Jun 2026 10:53:35 +0800 Subject: [PATCH 32/85] docs(todo): update runtime bridge progress --- todo.md | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-) diff --git a/todo.md b/todo.md index ffde96d6..d054275b 100644 --- a/todo.md +++ b/todo.md @@ -481,7 +481,7 @@ Acceptance: Goal: move from static app/repo display to real update workflows. -Status: ADR-0011 is accepted for the first in-memory Lua runtime/task/action/RuntimeNotification architecture. Earlier offline CLI/dev scaffolding proved update-check and fake task DTOs, but ADR-0011 supersedes the persisted fake task model: task state is process-memory only, `action_id` is single-use, task submission binds a sealed action plan plus in-memory package-version Lua object, and Flutter receives best-effort push `RuntimeNotification.task_changed` snapshots. The first implementation of this accepted runtime has begun in `getter-core`; live providers, real downloads, Android installers, background workers, and Android system notifications remain deferred to later ADRs. +Status: ADR-0011 is accepted for the first in-memory Lua runtime/task/action/RuntimeNotification architecture. Earlier offline CLI/dev scaffolding proved update-check and fake task DTOs, but ADR-0011 supersedes the persisted fake task model: task state is process-memory only, `action_id` is single-use, task submission binds a sealed action plan plus in-memory package-version Lua object, and Flutter receives best-effort push `RuntimeNotification.task_changed` snapshots. The first runtime core plus shared JSON operation seam and native bridge/EventChannel skeleton are implemented. Live provider action issuance, real downloads, Android installers, background workers, and Android system notifications remain deferred to later ADRs. Completed tasks: @@ -496,17 +496,19 @@ Completed tasks: 9. Add CLI commands and BDD coverage for `task submit`, `task run`, `task list`, `task cancel`, `task events`, and `task install-result`. 10. Accept ADR-0011 for the in-memory runtime/task/action/RuntimeNotification model. 11. Add `getter-core::runtime` with in-memory `GetterRuntime`, single-use `action_id`, sealed action plans, package-version Lua object binding, generic `user-result`, mock download/install state, package-level non-waiting lock, task controls, remove/clean, and RuntimeNotification DTOs with TDD coverage. +12. Add shared `getter-operations::runtime` JSON controls for submit/get/list/start/progress/complete-download/pause/resume/user-result/cancel/retry/remove/clean without persisted task state. Completed additional UI/bridge slice: 10. Extend Flutter getter bridge DTOs/adapters with read-only task list and event page APIs backed by existing getter CLI `task list` and `task events`. 11. Render getter-owned task/event DTOs on the Flutter Downloads route without adding a Dart task state machine. 12. Add Flutter widget/dev integration coverage for reading and rendering getter task lifecycle DTOs. +13. Add native bridge process-lifetime runtime singleton, runtime operation dispatcher, bounded best-effort notification drain, Kotlin EventChannel, and Dart runtime notification stream primitive. Remaining tasks: -1. Wire `getter-core::runtime` into getter operations/CLI single-process debug tooling without reintroducing persisted task state. -2. Wire the native bridge process-lifetime runtime singleton and Flutter push stream/EventChannel for `RuntimeNotification.task_changed` snapshots. +1. Add a getter-owned update/action issuance operation that materializes real sealed action plans and returns opaque `action_id` to Flutter; keep Dart from constructing action payloads. +2. Optionally add CLI single-process scripted/debug tooling over `getter-operations::runtime` without pretending separate invocations share memory. 3. Replace or retire the older persisted fake CLI task scaffold so public task status/control language uses ADR-0011 (`completed`, `user-result`, remove/clean, no cross-invocation task state). 4. Implement live provider/downloader behavior beyond the fake/offline proof after a later ADR accepts real side-effect details. 5. Define Android production install handoff URI/SAF/permission/notification details and wire platform adapter execution after later ADRs. @@ -517,7 +519,7 @@ Acceptance progress: - CLI can run an offline fixture update check: done. - Older CLI/dev fake task scaffold can persist/list task state: done, but superseded by ADR-0011 and slated for replacement/retirement. - `getter-core::runtime` can manage in-memory tasks, controls, `user-result`, retry, package lock, remove/clean, and notifications: first TDD slice done. -- Getter can expose pollable task events with cursor/limit in the older CLI/dev scaffold; ADR-0011 native push stream wiring remains pending. +- Getter can expose pollable task events with cursor/limit in the older CLI/dev scaffold; ADR-0011 native push stream skeleton is done with bounded best-effort EventChannel delivery and current-state query operations still pending typed Flutter UI use. - Getter can record abstract install handoff requests/results in the older scaffold; ADR-0011 uses generic `user-result` and mock install waiting-user state, Android installer execution remains deferred. - Flutter displays getter task/event DTOs rather than calculating status itself: done for read-only CLI/dev bridge slice. - Android platform adapter owns permissions/notifications/installer handoff: documented/deferred; no Android execution added in this slice. From ced857650e71c286889407bf2198dc79ddc32d3c Mon Sep 17 00:00:00 2001 From: Xiangzhe Date: Thu, 25 Jun 2026 11:09:47 +0800 Subject: [PATCH 33/85] feat(app): issue runtime actions from update checks --- .../src/main/rust/api_proxy/src/lib.rs | 50 +++++++++++++++++++ core-getter/src/main/rust/getter | 2 +- 2 files changed, 51 insertions(+), 1 deletion(-) diff --git a/core-getter/src/main/rust/api_proxy/src/lib.rs b/core-getter/src/main/rust/api_proxy/src/lib.rs index e968ca2f..0c4f5d62 100644 --- a/core-getter/src/main/rust/api_proxy/src/lib.rs +++ b/core-getter/src/main/rust/api_proxy/src/lib.rs @@ -363,6 +363,9 @@ fn runtime_operation_with_runtime( request.payload.to_string() }; match request.operation.as_str() { + "update_check_offline_issue_action" => { + runtime_operations::issue_action_from_offline_update_check_json(runtime, &payload) + } "task_submit" => runtime_operations::submit_action_json(runtime, &payload), "task_get" => runtime_operations::task_get_json(runtime, &payload), "task_list" => runtime_operations::task_list_json(runtime, &payload), @@ -656,6 +659,53 @@ mod tests { } } + #[test] + fn runtime_dispatcher_issues_action_from_offline_update_check() { + let mut runtime = getter::core::runtime::GetterRuntime::new(); + + let issued = runtime_operation_with_runtime( + &mut runtime, + &json!({ + "operation": "update_check_offline_issue_action", + "payload": { + "fixture": { + "format": "getter-offline-update-check", + "version": 1, + "package_id": "android/org.fdroid.fdroid", + "installed_version": "1.0.0", + "candidates": [ + { + "version": "1.2.0", + "artifacts": [ + { + "name": "app.apk", + "url": "https://example.invalid/app.apk", + "file_name": "app.apk" + } + ] + } + ] + } + } + }) + .to_string(), + ) + .expect("issue action"); + + assert_eq!(issued["update"]["status"], "update_available"); + let action_id = issued["action"]["action_id"].as_str().expect("action id"); + let submitted = runtime_operation_with_runtime( + &mut runtime, + &json!({ + "operation": "task_submit", + "payload": { "action_id": action_id } + }) + .to_string(), + ) + .expect("submit issued action"); + assert_eq!(submitted["package_id"], "android/org.fdroid.fdroid"); + } + #[test] fn runtime_dispatcher_uses_in_memory_runtime_controls() { let mut runtime = getter::core::runtime::GetterRuntime::new(); diff --git a/core-getter/src/main/rust/getter b/core-getter/src/main/rust/getter index aa0b75ee..c4f2c281 160000 --- a/core-getter/src/main/rust/getter +++ b/core-getter/src/main/rust/getter @@ -1 +1 @@ -Subproject commit aa0b75eed55f5be2614ec40623c13aa1982d4559 +Subproject commit c4f2c281611d64ba4af41ceafd76142246ae847c From 5061d2025075131a3305b4721b199f8a6b412e82 Mon Sep 17 00:00:00 2001 From: Xiangzhe Date: Thu, 25 Jun 2026 11:10:46 +0800 Subject: [PATCH 34/85] docs(todo): note offline action issuance --- todo.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/todo.md b/todo.md index d054275b..ea83ea63 100644 --- a/todo.md +++ b/todo.md @@ -481,7 +481,7 @@ Acceptance: Goal: move from static app/repo display to real update workflows. -Status: ADR-0011 is accepted for the first in-memory Lua runtime/task/action/RuntimeNotification architecture. Earlier offline CLI/dev scaffolding proved update-check and fake task DTOs, but ADR-0011 supersedes the persisted fake task model: task state is process-memory only, `action_id` is single-use, task submission binds a sealed action plan plus in-memory package-version Lua object, and Flutter receives best-effort push `RuntimeNotification.task_changed` snapshots. The first runtime core plus shared JSON operation seam and native bridge/EventChannel skeleton are implemented. Live provider action issuance, real downloads, Android installers, background workers, and Android system notifications remain deferred to later ADRs. +Status: ADR-0011 is accepted for the first in-memory Lua runtime/task/action/RuntimeNotification architecture. Earlier offline CLI/dev scaffolding proved update-check and fake task DTOs, but ADR-0011 supersedes the persisted fake task model: task state is process-memory only, `action_id` is single-use, task submission binds a sealed action plan plus in-memory package-version Lua object, and Flutter receives best-effort push `RuntimeNotification.task_changed` snapshots. The first runtime core plus shared JSON operation seam, offline update-check action issuance, and native bridge/EventChannel skeleton are implemented. Live provider action issuance, real downloads, Android installers, background workers, and Android system notifications remain deferred to later ADRs. Completed tasks: @@ -496,7 +496,7 @@ Completed tasks: 9. Add CLI commands and BDD coverage for `task submit`, `task run`, `task list`, `task cancel`, `task events`, and `task install-result`. 10. Accept ADR-0011 for the in-memory runtime/task/action/RuntimeNotification model. 11. Add `getter-core::runtime` with in-memory `GetterRuntime`, single-use `action_id`, sealed action plans, package-version Lua object binding, generic `user-result`, mock download/install state, package-level non-waiting lock, task controls, remove/clean, and RuntimeNotification DTOs with TDD coverage. -12. Add shared `getter-operations::runtime` JSON controls for submit/get/list/start/progress/complete-download/pause/resume/user-result/cancel/retry/remove/clean without persisted task state. +12. Add shared `getter-operations::runtime` JSON controls for offline update-check action issuance plus submit/get/list/start/progress/complete-download/pause/resume/user-result/cancel/retry/remove/clean without persisted task state. Completed additional UI/bridge slice: @@ -507,7 +507,7 @@ Completed additional UI/bridge slice: Remaining tasks: -1. Add a getter-owned update/action issuance operation that materializes real sealed action plans and returns opaque `action_id` to Flutter; keep Dart from constructing action payloads. +1. Replace the offline-fixture action issuance seam with real package/repository/provider update-check action issuance that materializes sealed action plans and returns opaque `action_id` to Flutter; keep Dart from constructing action payloads. 2. Optionally add CLI single-process scripted/debug tooling over `getter-operations::runtime` without pretending separate invocations share memory. 3. Replace or retire the older persisted fake CLI task scaffold so public task status/control language uses ADR-0011 (`completed`, `user-result`, remove/clean, no cross-invocation task state). 4. Implement live provider/downloader behavior beyond the fake/offline proof after a later ADR accepts real side-effect details. From 71c71c97d6ead73119b919a148301dfd4edc3fb9 Mon Sep 17 00:00:00 2001 From: Xiangzhe Date: Thu, 25 Jun 2026 11:20:00 +0800 Subject: [PATCH 35/85] docs(lua): document static update candidates --- core-getter/src/main/rust/getter | 2 +- docs/lua-api/package-lifecycle.md | 23 +++++++++++++++++++++++ 2 files changed, 24 insertions(+), 1 deletion(-) diff --git a/core-getter/src/main/rust/getter b/core-getter/src/main/rust/getter index c4f2c281..604a5f6a 160000 --- a/core-getter/src/main/rust/getter +++ b/core-getter/src/main/rust/getter @@ -1 +1 @@ -Subproject commit c4f2c281611d64ba4af41ceafd76142246ae847c +Subproject commit 604a5f6a49e26ca895f26dceae655ffb404e5033 diff --git a/docs/lua-api/package-lifecycle.md b/docs/lua-api/package-lifecycle.md index ede75bbb..3f695c92 100644 --- a/docs/lua-api/package-lifecycle.md +++ b/docs/lua-api/package-lifecycle.md @@ -61,6 +61,29 @@ return { } ``` +As the first offline/mock-provider bridge toward this lifecycle, package Lua may also declare static `updates` candidates in the package table. Getter validates this table, performs Rust-owned selection/version comparison, and issues opaque runtime `action_id`s from the selected candidate; Flutter must still return only the getter-issued `action_id` and must not assemble download/install action payloads. + +```lua +return package_def { + id = "android/org.fdroid.fdroid", + name = "F-Droid", + updates = { + { + version = "1.2.0", + channel = "stable", + source = "fixture", + artifacts = { + { + name = "app.apk", + url = "https://example.invalid/app.apk", + file_name = "fdroid.apk", + }, + }, + }, + }, +} +``` + The first Phase D implementation slice exposes this boundary only through an offline CLI fixture command: `getter --data-dir update check --fixture `. The fixture is normalized JSON, not live provider output, and the command returns `network_required = false`, update-check status, selected candidate/artifact, and generated download/install action DTOs. It does not execute network providers, download files, persist download tasks, stream progress events, or invoke Android installers. ADR-0011 supersedes the earlier persisted fake task scaffold. The accepted Phase D runtime consumes getter-issued actions through an in-memory process-lifetime runtime: task state is not stored in SQLite, `action_id` is single-use, task submission binds a sealed action plan plus package-version Lua object, mock download/install executors simulate task state, and `RuntimeNotification.task_changed` is pushed to Flutter as a best-effort current snapshot. CLI coverage for this model should use Rust runtime tests or a single-process scripted/debug command rather than pretending separate CLI invocations share task memory. From 75b9f6522fa34f1974b9e3c0f33d225804c87369 Mon Sep 17 00:00:00 2001 From: Xiangzhe Date: Thu, 25 Jun 2026 11:49:43 +0800 Subject: [PATCH 36/85] feat(app): issue actions from registered packages --- .../net/xzos/upgradeall/MainActivity.kt | 4 +- .../src/main/rust/api_proxy/Cargo.toml | 5 +- .../src/main/rust/api_proxy/src/lib.rs | 91 ++++++++++++++++++- core-getter/src/main/rust/getter | 2 +- 4 files changed, 98 insertions(+), 4 deletions(-) diff --git a/app_flutter/android/app/src/main/kotlin/net/xzos/upgradeall/MainActivity.kt b/app_flutter/android/app/src/main/kotlin/net/xzos/upgradeall/MainActivity.kt index 3cb29a86..e5520540 100644 --- a/app_flutter/android/app/src/main/kotlin/net/xzos/upgradeall/MainActivity.kt +++ b/app_flutter/android/app/src/main/kotlin/net/xzos/upgradeall/MainActivity.kt @@ -206,7 +206,9 @@ class MainActivity : FlutterActivity() { private fun runtimeOperationRequest(call: MethodCall): String { val args = call.arguments as? Map<*, *> ?: throw IllegalArgumentException("runtime operation arguments are required") - return GetterBridgeRequestBuilder.runtimeOperationRequest(args) + return JSONObject(GetterBridgeRequestBuilder.runtimeOperationRequest(args)) + .put("data_dir", getterDataDir().absolutePath) + .toString() } private fun emitRuntimeNotifications() { diff --git a/core-getter/src/main/rust/api_proxy/Cargo.toml b/core-getter/src/main/rust/api_proxy/Cargo.toml index 46a63539..b941d6b0 100644 --- a/core-getter/src/main/rust/api_proxy/Cargo.toml +++ b/core-getter/src/main/rust/api_proxy/Cargo.toml @@ -7,13 +7,16 @@ edition = "2021" [dependencies] jni = "0.21" # from rustls-platform-verifier-android, sync version -getter = { path = "../getter", default-features = false, features = ["domain", "native-tokio", "rustls-platform-verifier-android"] } +getter = { path = "../getter", default-features = false, features = ["domain", "lua", "native-tokio", "rustls-platform-verifier-android"] } upgradeall-platform-adapter = { path = "../platform_adapter" } serde = { version = "1.0", features = ["derive"] } serde_json = "1.0.145" thiserror = "1" tokio = { version = "1.48.0", features = ["rt-multi-thread"] } +[dev-dependencies] +tempfile = "3" + [lib] crate-type = ["cdylib"] diff --git a/core-getter/src/main/rust/api_proxy/src/lib.rs b/core-getter/src/main/rust/api_proxy/src/lib.rs index 0c4f5d62..01f2ee91 100644 --- a/core-getter/src/main/rust/api_proxy/src/lib.rs +++ b/core-getter/src/main/rust/api_proxy/src/lib.rs @@ -57,6 +57,8 @@ struct RuntimeOperationRequest { operation: String, #[serde(default)] payload: Value, + #[serde(default)] + data_dir: Option, } #[derive(Debug, Default, Deserialize)] @@ -366,6 +368,15 @@ fn runtime_operation_with_runtime( "update_check_offline_issue_action" => { runtime_operations::issue_action_from_offline_update_check_json(runtime, &payload) } + "update_check_package_issue_action" => { + let data_dir = request.data_dir.as_ref().ok_or_else(|| { + BridgeOperationError::InvalidRequest( + "data_dir is required for package update checks".to_owned(), + ) + })?; + let db = open_main_db(data_dir)?; + runtime_operations::issue_action_from_registered_package_json(runtime, &db, &payload) + } "task_submit" => runtime_operations::submit_action_json(runtime, &payload), "task_get" => runtime_operations::task_get_json(runtime, &payload), "task_list" => runtime_operations::task_list_json(runtime, &payload), @@ -629,9 +640,11 @@ impl From for BridgeOperationError { mod tests { use super::*; use getter::core::{ + repository::{RepositoryMetadata, REPO_API_VERSION_V1}, runtime::{PackageVersionLuaObject, SealedActionPlan}, - UpdateAction, + RepositoryPriority, UpdateAction, }; + use std::fs; #[test] fn packages_acceptance_defaults_to_all() { @@ -659,6 +672,45 @@ mod tests { } } + #[test] + fn runtime_dispatcher_issues_action_from_registered_package_update_check() { + let temp = tempfile::tempdir().unwrap(); + let data_dir = temp.path().join("data"); + let repo_root = temp.path().join("repo"); + write_static_update_repo(&repo_root); + let db = open_main_db(&data_dir).unwrap(); + db.upsert_repository( + &RepositoryMetadata { + id: "official".parse().unwrap(), + name: "Official".to_owned(), + priority: RepositoryPriority::new(0), + api_version: REPO_API_VERSION_V1.to_owned(), + }, + Some(&repo_root), + None, + ) + .unwrap(); + let mut runtime = getter::core::runtime::GetterRuntime::new(); + + let issued = runtime_operation_with_runtime( + &mut runtime, + &json!({ + "operation": "update_check_package_issue_action", + "data_dir": data_dir, + "payload": { + "package_id": "android/org.fdroid.fdroid", + "installed_version": "1.0.0" + } + }) + .to_string(), + ) + .expect("issue action"); + + assert_eq!(issued["package"]["repository"], "official"); + assert_eq!(issued["update"]["status"], "update_available"); + assert!(issued["action"]["action_id"].as_str().is_some()); + } + #[test] fn runtime_dispatcher_issues_action_from_offline_update_check() { let mut runtime = getter::core::runtime::GetterRuntime::new(); @@ -787,6 +839,43 @@ mod tests { assert!(detail.unwrap().contains("unsupported runtime operation")); } + fn write_static_update_repo(root: &std::path::Path) { + fs::create_dir_all(root.join("packages/android")).unwrap(); + fs::create_dir(root.join("lib")).unwrap(); + fs::create_dir(root.join("templates")).unwrap(); + fs::write( + root.join("repo.toml"), + r#"id = "official" +name = "Official" +priority = 0 +api_version = "getter.repo.v1" +"#, + ) + .unwrap(); + fs::write( + root.join("packages/android/org.fdroid.fdroid.lua"), + r#" +return package_def { + id = "android/org.fdroid.fdroid", + name = "F-Droid", + updates = { + { + version = "1.2.0", + artifacts = { + { + name = "app.apk", + url = "https://example.invalid/app.apk", + file_name = "app.apk", + }, + }, + }, + }, +} +"#, + ) + .unwrap(); + } + #[test] fn runtime_notification_queue_is_bounded_and_drained() { drain_runtime_notifications().expect("clear queue"); diff --git a/core-getter/src/main/rust/getter b/core-getter/src/main/rust/getter index 604a5f6a..1738c03f 160000 --- a/core-getter/src/main/rust/getter +++ b/core-getter/src/main/rust/getter @@ -1 +1 @@ -Subproject commit 604a5f6a49e26ca895f26dceae655ffb404e5033 +Subproject commit 1738c03f58450095bc075fb7adf75e4a57262854 From 2aad93fd55d05ddb8c7d5bffe46fe466fde76d86 Mon Sep 17 00:00:00 2001 From: Xiangzhe Date: Thu, 25 Jun 2026 11:54:50 +0800 Subject: [PATCH 37/85] docs(todo): note registered package action issuance --- todo.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/todo.md b/todo.md index ea83ea63..1ccea206 100644 --- a/todo.md +++ b/todo.md @@ -481,7 +481,7 @@ Acceptance: Goal: move from static app/repo display to real update workflows. -Status: ADR-0011 is accepted for the first in-memory Lua runtime/task/action/RuntimeNotification architecture. Earlier offline CLI/dev scaffolding proved update-check and fake task DTOs, but ADR-0011 supersedes the persisted fake task model: task state is process-memory only, `action_id` is single-use, task submission binds a sealed action plan plus in-memory package-version Lua object, and Flutter receives best-effort push `RuntimeNotification.task_changed` snapshots. The first runtime core plus shared JSON operation seam, offline update-check action issuance, and native bridge/EventChannel skeleton are implemented. Live provider action issuance, real downloads, Android installers, background workers, and Android system notifications remain deferred to later ADRs. +Status: ADR-0011 is accepted for the first in-memory Lua runtime/task/action/RuntimeNotification architecture. Earlier offline CLI/dev scaffolding proved update-check and fake task DTOs, but ADR-0011 supersedes the persisted fake task model: task state is process-memory only, `action_id` is single-use, task submission binds a sealed action plan plus in-memory package-version Lua object, and Flutter receives best-effort push `RuntimeNotification.task_changed` snapshots. The first runtime core plus shared JSON operation seam, offline update-check action issuance, registered-package/static-update action issuance, and native bridge/EventChannel skeleton are implemented. Live provider action issuance, real downloads, Android installers, background workers, and Android system notifications remain deferred to later ADRs. Completed tasks: @@ -507,7 +507,7 @@ Completed additional UI/bridge slice: Remaining tasks: -1. Replace the offline-fixture action issuance seam with real package/repository/provider update-check action issuance that materializes sealed action plans and returns opaque `action_id` to Flutter; keep Dart from constructing action payloads. +1. Replace the current static `updates` package seam with live provider update-check action issuance that materializes sealed action plans and returns opaque `action_id` to Flutter; keep Dart from constructing action payloads. 2. Optionally add CLI single-process scripted/debug tooling over `getter-operations::runtime` without pretending separate invocations share memory. 3. Replace or retire the older persisted fake CLI task scaffold so public task status/control language uses ADR-0011 (`completed`, `user-result`, remove/clean, no cross-invocation task state). 4. Implement live provider/downloader behavior beyond the fake/offline proof after a later ADR accepts real side-effect details. From 7ebf9bee2d6e75c78b41f070f52e2388dc00c869 Mon Sep 17 00:00:00 2001 From: Xiangzhe Date: Thu, 25 Jun 2026 12:43:30 +0800 Subject: [PATCH 38/85] feat(app): adopt pin version DTOs --- app_flutter/dev_test/cli_getter_adapter_test.dart | 6 +++--- app_flutter/integration_test/native_bridge_test.dart | 2 +- app_flutter/lib/cli_getter_adapter.dart | 2 +- app_flutter/lib/getter_adapter.dart | 12 ++++++------ app_flutter/test/native_getter_adapter_test.dart | 2 +- app_flutter/test/widget_test.dart | 2 +- core-getter/src/main/rust/getter | 2 +- docs/architecture/adr/0003-legacy-room-migration.md | 2 +- .../adr/0006-package-centric-cli-command-contract.md | 6 +++--- docs/architecture/upgradeall-getter-rewrite-wiki.md | 6 +++--- docs/lua-api/package-lifecycle.md | 2 +- docs/lua-api/templates.md | 2 +- docs/migration/legacy-room-mapping.md | 4 ++-- todo.md | 4 ++-- 14 files changed, 27 insertions(+), 27 deletions(-) diff --git a/app_flutter/dev_test/cli_getter_adapter_test.dart b/app_flutter/dev_test/cli_getter_adapter_test.dart index df30f023..241f35dd 100644 --- a/app_flutter/dev_test/cli_getter_adapter_test.dart +++ b/app_flutter/dev_test/cli_getter_adapter_test.dart @@ -29,7 +29,7 @@ void main() { (package) => package.id == 'android/org.fdroid.fdroid', ); expect(tracked.favorite, isTrue); - expect(tracked.ignoredVersion, '1.20.0'); + expect(tracked.pinVersion, '1.20.0'); expect(tracked.packageResolution, 'missing_package_definition'); final reports = await adapter.readMigrationReports(); @@ -89,7 +89,7 @@ void main() { (package) => package.id == 'android/org.fdroid.fdroid', ); expect(tracked.favorite, isTrue); - expect(tracked.ignoredVersion, '1.20.0'); + expect(tracked.pinVersion, '1.20.0'); expect(tracked.packageResolution, 'official_repository_package'); final evaluated = adapter.evaluatePackage( @@ -181,7 +181,7 @@ File _createLegacyBundle(Directory temp) { "kind": "android", "installed_id": "org.fdroid.fdroid", "official_package_available": true, - "ignored_version": "1.20.0", + "pin_version": "1.20.0", "favorite": true } ] diff --git a/app_flutter/integration_test/native_bridge_test.dart b/app_flutter/integration_test/native_bridge_test.dart index b0bce474..941ae88e 100644 --- a/app_flutter/integration_test/native_bridge_test.dart +++ b/app_flutter/integration_test/native_bridge_test.dart @@ -32,7 +32,7 @@ void main() { expect(result.trackedPackages, hasLength(1)); expect(result.trackedPackages.single.id, 'android/org.fdroid.fdroid'); expect(result.trackedPackages.single.favorite, isTrue); - expect(result.trackedPackages.single.ignoredVersion, '1.20.0'); + expect(result.trackedPackages.single.pinVersion, '1.20.0'); expect(result.sourceCounts?.appRows, 1); expect(result.sourceCounts?.extraAppRows, 1); diff --git a/app_flutter/lib/cli_getter_adapter.dart b/app_flutter/lib/cli_getter_adapter.dart index f83041f0..0f460663 100644 --- a/app_flutter/lib/cli_getter_adapter.dart +++ b/app_flutter/lib/cli_getter_adapter.dart @@ -191,7 +191,7 @@ TrackedPackageSummary _trackedPackageFromJson(Object? value) { id: _asString(json['id'], 'tracked.id'), enabled: _asBool(json['enabled'], 'tracked.enabled'), favorite: _asBool(json['favorite'], 'tracked.favorite'), - ignoredVersion: json['ignored_version'] as String?, + pinVersion: json['pin_version'] as String?, repositoryId: json['repository_id'] as String?, packageResolution: _asString( json['package_resolution'], diff --git a/app_flutter/lib/getter_adapter.dart b/app_flutter/lib/getter_adapter.dart index cbf7891d..07c88aaa 100644 --- a/app_flutter/lib/getter_adapter.dart +++ b/app_flutter/lib/getter_adapter.dart @@ -78,7 +78,7 @@ class FakeGetterAdapter implements GetterAdapter { id: 'android/org.fdroid.fdroid', enabled: true, favorite: false, - ignoredVersion: null, + pinVersion: null, repositoryId: 'official', packageResolution: 'official_repository_package', ), @@ -303,7 +303,7 @@ class TrackedPackageSummary { required this.id, required this.enabled, required this.favorite, - required this.ignoredVersion, + required this.pinVersion, required this.repositoryId, required this.packageResolution, }); @@ -313,9 +313,9 @@ class TrackedPackageSummary { id: _jsonString(json['id'], 'tracked.id'), enabled: _jsonBool(json['enabled'], 'tracked.enabled'), favorite: _jsonBool(json['favorite'], 'tracked.favorite'), - ignoredVersion: _jsonOptionalString( - json['ignored_version'], - 'tracked.ignored_version', + pinVersion: _jsonOptionalString( + json['pin_version'], + 'tracked.pin_version', ), repositoryId: _jsonOptionalString(json['repository_id'], 'tracked.repository_id'), @@ -329,7 +329,7 @@ class TrackedPackageSummary { final String id; final bool enabled; final bool favorite; - final String? ignoredVersion; + final String? pinVersion; final String? repositoryId; final String packageResolution; } diff --git a/app_flutter/test/native_getter_adapter_test.dart b/app_flutter/test/native_getter_adapter_test.dart index af3afde1..fa194d90 100644 --- a/app_flutter/test/native_getter_adapter_test.dart +++ b/app_flutter/test/native_getter_adapter_test.dart @@ -113,7 +113,7 @@ void main() { 'id': 'android/org.fdroid.fdroid', 'enabled': true, 'favorite': true, - 'ignored_version': '1.20.0', + 'pin_version': '1.20.0', 'repository_id': null, 'package_resolution': 'missing_package_definition', }, diff --git a/app_flutter/test/widget_test.dart b/app_flutter/test/widget_test.dart index d9dbdf51..5b72c9fb 100644 --- a/app_flutter/test/widget_test.dart +++ b/app_flutter/test/widget_test.dart @@ -303,7 +303,7 @@ class _MigrationGetterAdapter extends FakeGetterAdapter { id: 'android/org.fdroid.fdroid', enabled: true, favorite: true, - ignoredVersion: '1.20.0', + pinVersion: '1.20.0', repositoryId: null, packageResolution: 'missing_package_definition', ), diff --git a/core-getter/src/main/rust/getter b/core-getter/src/main/rust/getter index 1738c03f..9ae7e568 160000 --- a/core-getter/src/main/rust/getter +++ b/core-getter/src/main/rust/getter @@ -1 +1 @@ -Subproject commit 1738c03f58450095bc075fb7adf75e4a57262854 +Subproject commit 9ae7e5680188b2ef87bc052d2e55d5375116c0ce diff --git a/docs/architecture/adr/0003-legacy-room-migration.md b/docs/architecture/adr/0003-legacy-room-migration.md index 347b2509..3dde893b 100644 --- a/docs/architecture/adr/0003-legacy-room-migration.md +++ b/docs/architecture/adr/0003-legacy-room-migration.md @@ -79,7 +79,7 @@ The host-side CLI also keeps the deterministic JSON bridge bundle for tests and "installed_id": "org.fdroid.fdroid", "official_package_available": true, "common_conversion_available": false, - "ignored_version": "1.20.0", + "pin_version": "1.20.0", "favorite": true } ] diff --git a/docs/architecture/adr/0006-package-centric-cli-command-contract.md b/docs/architecture/adr/0006-package-centric-cli-command-contract.md index 5ee2d85e..23a59e3b 100644 --- a/docs/architecture/adr/0006-package-centric-cli-command-contract.md +++ b/docs/architecture/adr/0006-package-centric-cli-command-contract.md @@ -86,7 +86,7 @@ The first supported `legacy import-room-bundle` slice accepts a JSON bridge bund "installed_id": "org.fdroid.fdroid", "official_package_available": true, "common_conversion_available": false, - "ignored_version": "1.20.0", + "pin_version": "1.20.0", "favorite": true } ] @@ -99,9 +99,9 @@ It maps `apps[]` into getter tracked package state in `main.db`, writes a saniti `legacy report-list` returns sanitized migration report summaries through the same JSON envelope so app/test adapters do not need to inspect getter's data-directory layout directly. -The first installed-app autogen slice accepts an Android/platform-provided inventory DTO, computes generated fallback packages in Rust, previews before writing, and applies only after explicit `--accept-all` or `--accept ` confirmation. Getter owns the canonical repository path `/repositories/local_autogen`, fixed repo id `local_autogen`, default priority `-1`, deterministic package file path `packages//.lua`, and `autogen-manifest.json`. Candidates are skipped when any registered repository with priority higher than `local_autogen` already provides that package id. Applying installed autogen writes/registers `local_autogen` package files and tracks accepted packages in `main.db` when they are not already resolved; existing user state (`enabled`, `favorite`, `ignored_version`) and existing non-missing resolution metadata are preserved. Cleanup preview/apply only targets manifest-managed `local_autogen` packages missing from the current installed inventory. Cleanup refuses stale/tampered previews that do not match the current manifest and deletes tracked state only for rows still owned by `local_autogen` generated packages. If an existing autogen file was modified, getter preserves that content into the user-authored `local` repo before regenerating or deleting the managed autogen file. +The first installed-app autogen slice accepts an Android/platform-provided inventory DTO, computes generated fallback packages in Rust, previews before writing, and applies only after explicit `--accept-all` or `--accept ` confirmation. Getter owns the canonical repository path `/repositories/local_autogen`, fixed repo id `local_autogen`, default priority `-1`, deterministic package file path `packages//.lua`, and `autogen-manifest.json`. Candidates are skipped when any registered repository with priority higher than `local_autogen` already provides that package id. Applying installed autogen writes/registers `local_autogen` package files and tracks accepted packages in `main.db` when they are not already resolved; existing user state (`enabled`, `favorite`, `pin_version`) and existing non-missing resolution metadata are preserved. Cleanup preview/apply only targets manifest-managed `local_autogen` packages missing from the current installed inventory. Cleanup refuses stale/tampered previews that do not match the current manifest and deletes tracked state only for rows still owned by `local_autogen` generated packages. If an existing autogen file was modified, getter preserves that content into the user-authored `local` repo before regenerating or deleting the managed autogen file. -`update check --fixture ` is the first Phase D offline update-check slice. The fixture contract is explicitly offline and uses `format = "getter-offline-update-check"`, `version = 1`, `package_id`, optional `installed_version`, optional `ignored_version`, and normalized candidate/artifact DTOs. The command returns `network_required = false`, a getter-owned status (`update_available`, `up_to_date`, `no_candidates`, or `ignored`), the selected update when one exists, and generated download/install action DTOs. An update-selected result must have an actionable artifact; a selected candidate without artifacts is a structured update-check error rather than `update_available` with no actions. It reuses Rust getter update selection and version comparison; it does not run providers, perform downloads, persist download tasks, stream events, or call Android installers. +`update check --fixture ` is the first Phase D offline update-check slice. The fixture contract is explicitly offline and uses `format = "getter-offline-update-check"`, `version = 1`, `package_id`, optional `installed_version`, optional `pin_version` (with transitional `ignored_version` accepted as an input alias), and normalized candidate/artifact DTOs. The command returns `network_required = false`, observed `installed_version`, `effective_local_version`, a getter-owned status (`update_available`, `up_to_date`, or `no_candidates`), the selected update when one exists, and generated download/install action DTOs. An update-selected result must have an actionable artifact; a selected candidate without artifacts is a structured update-check error rather than `update_available` with no actions. It reuses Rust getter update selection and version comparison; it does not run providers, perform downloads, persist download tasks, stream events, or call Android installers. The first getter-owned task lifecycle slice is also explicitly offline/fake and command-driven. `task submit --request ` accepts `format = "getter-download-request"`, `version = 1`, `package_id`, `executor = "fake"`, and update actions containing at least one `download` action; an optional `install` action creates an abstract install handoff after a successful fake run. `task run ` deterministically advances the fake task to `succeeded`; it performs no network I/O and writes no downloaded bytes. `task list` returns persisted task summaries from `main.db`. `task cancel ` persists cancellation for `queued`/`running` tasks, is idempotent for already-canceled tasks, and rejects terminal success/failure with a structured download error. `task events --after --limit ` is a pollable CLI/dev event contract with a positive `limit`; it is not a native streaming API, and native streaming/backpressure remains deferred. `task install-result --status ` records the platform-side result of an abstract handoff; the getter-created `requested` handoff state is not accepted as a platform result. Getter records handoff requests/results but does not call Android installers, request permissions, create notifications, or decide Android URI/SAF semantics. diff --git a/docs/architecture/upgradeall-getter-rewrite-wiki.md b/docs/architecture/upgradeall-getter-rewrite-wiki.md index 6d0f94ef..9b856926 100644 --- a/docs/architecture/upgradeall-getter-rewrite-wiki.md +++ b/docs/architecture/upgradeall-getter-rewrite-wiki.md @@ -885,8 +885,8 @@ v1 暂不做 repo/script/artifact 强校验。 - repo priority。 - enabled apps/packages。 - user source priority override。 -- ignore versions。 -- pins。 +- legacy ignore/mark version state mapped into `pin_version`. +- pins / version baselines。 - favorites/star。 - migration records。 - settings。 @@ -983,7 +983,7 @@ end - saved apps 的基本 identity。 - Android package / Magisk module installed id。 -- ignore version / mark version 能力，如果可映射。 +- legacy ignore version / mark version 能力映射为 `pin_version`，如果可映射。 - user-visible tracked app 列表。 - 常见 source/cloud config 能力，如果可内置转换。 diff --git a/docs/lua-api/package-lifecycle.md b/docs/lua-api/package-lifecycle.md index 3f695c92..97533012 100644 --- a/docs/lua-api/package-lifecycle.md +++ b/docs/lua-api/package-lifecycle.md @@ -45,7 +45,7 @@ Normalize, filter and enrich release candidates. Choose the candidate/artifact to update to, using installed version and user state. -The first getter-core selection helper uses deterministic tokenized version comparison: digit runs compare numerically, text suffixes compare case-insensitively, separators are ignored, and a prerelease-like text suffix (for example `beta`/`rc`) sorts before the final release with the same numeric prefix. The selector skips the user's ignored version and returns the highest candidate newer than the installed version. +The first getter-core selection helper uses deterministic tokenized version comparison: digit runs compare numerically, text suffixes compare case-insensitively, separators are ignored, and a prerelease-like text suffix (for example `beta`/`rc`) sorts before the final release with the same numeric prefix. The selector returns the highest candidate newer than the effective local baseline. The effective baseline is normally the observed installed version; when the user has set `pin_version`, getter compares candidates against that pin override instead while still keeping the observed installed version available for display/diagnostics. ## resolve diff --git a/docs/lua-api/templates.md b/docs/lua-api/templates.md index 468638ad..d0b028c3 100644 --- a/docs/lua-api/templates.md +++ b/docs/lua-api/templates.md @@ -63,7 +63,7 @@ Cleanup flow: 5. User confirms yes/no. 6. getter deletes only accepted manifest-managed `local_autogen` files/state. -Cleanup apply refuses stale/tampered previews that do not match the current manifest, and guarded tracked-state deletion only removes rows still owned by `local_autogen` generated packages. Installed apply preserves existing user state (`enabled`, `favorite`, `ignored_version`) and existing non-missing resolution metadata when a package is already tracked. If a managed autogen file has been edited, getter preserves that content into the user-authored `local` repo before regenerating or deleting the generated file. Ordinary autogen cleanup never deletes `local`. +Cleanup apply refuses stale/tampered previews that do not match the current manifest, and guarded tracked-state deletion only removes rows still owned by `local_autogen` generated packages. Installed apply preserves existing user state (`enabled`, `favorite`, `pin_version`) and existing non-missing resolution metadata when a package is already tracked. If a managed autogen file has been edited, getter preserves that content into the user-authored `local` repo before regenerating or deleting the generated file. Ordinary autogen cleanup never deletes `local`. ## Repositories diff --git a/docs/migration/legacy-room-mapping.md b/docs/migration/legacy-room-mapping.md index a3e0ee5e..19903224 100644 --- a/docs/migration/legacy-room-mapping.md +++ b/docs/migration/legacy-room-mapping.md @@ -64,7 +64,7 @@ Complex auth may be dropped. ## ExtraApp mapping -Map mark/ignore version state when possible. In the direct Room DB importer, `extra_app.mark_version_number` wins over `app.ignore_version_number` when both exist for the same package id because it is the more specific extra-app state. +Map legacy mark/ignore version state into rewrite `pin_version` when possible. In the direct Room DB importer, `extra_app.mark_version_number` wins over `app.ignore_version_number` when both exist for the same package id because it is the more specific extra-app state. ## ExtraHub mapping @@ -113,7 +113,7 @@ The host-side CLI implementation also accepts a deterministic JSON bridge bundle "installed_id": "org.fdroid.fdroid", "official_package_available": true, "common_conversion_available": false, - "ignored_version": "1.20.0", + "pin_version": "1.20.0", "favorite": true } ] diff --git a/todo.md b/todo.md index 1ccea206..445e995e 100644 --- a/todo.md +++ b/todo.md @@ -157,7 +157,7 @@ Do not jump to AGP 9 as part of the immediate fix unless the minimal Kotlin fix | SQLite storage | Use main DB + cache DB, not JSONL product store | `MainDb` and `CacheDb` implemented; `init` creates `main.db` and `cache.db` | Aligned | | Lua package repositories | Lua files return JSON-like tables; Rust validates | `getter-core/src/lua.rs` and repository loader implemented; hardened lib search path | Aligned | | Legacy migration | Automatic migration eventually; initial slice may be JSON bridge | JSON bridge bundle import exists; direct Room reader deferred | Partial but acceptable | -| ExtraApp preservation | Do not repeat old bug of skipping `extra_app` state | Current mapping preserves `ignored_version` and `favorite` from extra app slice | Aligned for current slice | +| ExtraApp preservation | Do not repeat old bug of skipping `extra_app` state | Current mapping preserves legacy version override as `pin_version` plus `favorite` from extra app slice | Aligned for current slice | | Flutter UI | Flutter owns UI/platform only | `FakeGetterAdapter`, route keys, placeholder pages; no real product logic | Acceptable shell; freeze scope until bridge | | Mixed TDD/BDD | TDD for Rust/domain, BDD for user-facing/integration | Rust unit tests + CLI BDD + Flutter widget tests | Aligned | | Verification | `just verify` should be the main gate | `just verify` exists, passes locally, and passes in the rewrite validation workflow | Aligned | @@ -489,7 +489,7 @@ Completed tasks: 2. Reuse existing getter-core update selection for update availability. 3. Generate minimal download/install action DTOs for the selected artifact. 4. Add CLI command `update check --fixture `. -5. Add BDD coverage for update available, up to date, ignored latest fallback, ignored-only, unknown installed version, and malformed fixture. +5. Add BDD coverage for update available, up to date, `pin_version` baseline override, unknown installed version, and malformed fixture. 6. Add getter-core task/event/install-handoff DTOs for the first offline lifecycle proof. 7. Add main DB task/event/install-handoff tables and storage APIs with TDD coverage. 8. Implement deterministic fake/offline downloader behavior beyond the previous placeholder crate: submit, run, cancel, list, poll events, and record install result. From c1f712674c744cf307dc262fad03f214558d3a95 Mon Sep 17 00:00:00 2001 From: Xiangzhe Date: Thu, 25 Jun 2026 13:10:47 +0800 Subject: [PATCH 39/85] feat(app): add typed runtime task APIs --- app_flutter/lib/cli_getter_adapter.dart | 95 ++++ app_flutter/lib/getter_adapter.dart | 448 ++++++++++++++++++ app_flutter/lib/main.dart | 113 +++-- app_flutter/lib/native_getter_adapter.dart | 127 +++++ .../test/native_getter_adapter_test.dart | 140 ++++++ app_flutter/test/widget_test.dart | 14 +- todo.md | 3 +- 7 files changed, 883 insertions(+), 57 deletions(-) diff --git a/app_flutter/lib/cli_getter_adapter.dart b/app_flutter/lib/cli_getter_adapter.dart index 0f460663..9d6a1c69 100644 --- a/app_flutter/lib/cli_getter_adapter.dart +++ b/app_flutter/lib/cli_getter_adapter.dart @@ -119,6 +119,101 @@ class CliGetterAdapter implements GetterAdapter { ); } + @override + Future checkPackageForUpdate( + String packageId, { + String? repositoryId, + String? installedVersion, + String? pinVersion, + }) async { + throw const GetterBridgeException( + GetterError( + code: 'bridge.unsupported', + message: 'CLI adapter does not host a process-lifetime runtime', + ), + ); + } + + @override + Future submitRuntimeAction(String actionId) { + throw const GetterBridgeException( + GetterError( + code: 'bridge.unsupported', + message: 'CLI adapter does not host a process-lifetime runtime', + ), + ); + } + + @override + Future> listRuntimeTasks({ + bool active = false, + String? packageId, + }) { + throw const GetterBridgeException( + GetterError( + code: 'bridge.unsupported', + message: 'CLI adapter does not host a process-lifetime runtime', + ), + ); + } + + @override + Future getRuntimeTask(String taskId) => + _unsupportedRuntimeTask(); + + @override + Future startRuntimeTask(String taskId) => + _unsupportedRuntimeTask(); + + @override + Future pauseRuntimeTask(String taskId) => + _unsupportedRuntimeTask(); + + @override + Future resumeRuntimeTask(String taskId) => + _unsupportedRuntimeTask(); + + @override + Future cancelRuntimeTask(String taskId) => + _unsupportedRuntimeTask(); + + @override + Future retryRuntimeTask(String taskId) => + _unsupportedRuntimeTask(); + + @override + Future removeRuntimeTask(String taskId) => + _unsupportedRuntimeTask(); + + @override + Future sendRuntimeUserResult( + String taskId, + RuntimeUserResult result, { + String? reason, + }) => + _unsupportedRuntimeTask(); + + @override + Future> cleanRuntimeTasks({ + RuntimeTaskCleanMode mode = RuntimeTaskCleanMode.defaultMode, + }) { + throw const GetterBridgeException( + GetterError( + code: 'bridge.unsupported', + message: 'CLI adapter does not host a process-lifetime runtime', + ), + ); + } + + Future _unsupportedRuntimeTask() { + throw const GetterBridgeException( + GetterError( + code: 'bridge.unsupported', + message: 'CLI adapter does not host a process-lifetime runtime', + ), + ); + } + @override GetterSnapshot loadSnapshot() { initialize(); diff --git a/app_flutter/lib/getter_adapter.dart b/app_flutter/lib/getter_adapter.dart index 07c88aaa..d7234667 100644 --- a/app_flutter/lib/getter_adapter.dart +++ b/app_flutter/lib/getter_adapter.dart @@ -34,6 +34,44 @@ abstract interface class GetterAdapter { List? acceptedPackageIds, }); + Future checkPackageForUpdate( + String packageId, { + String? repositoryId, + String? installedVersion, + String? pinVersion, + }); + + Future submitRuntimeAction(String actionId); + + Future> listRuntimeTasks({ + bool active = false, + String? packageId, + }); + + Future getRuntimeTask(String taskId); + + Future startRuntimeTask(String taskId); + + Future pauseRuntimeTask(String taskId); + + Future resumeRuntimeTask(String taskId); + + Future cancelRuntimeTask(String taskId); + + Future retryRuntimeTask(String taskId); + + Future removeRuntimeTask(String taskId); + + Future sendRuntimeUserResult( + String taskId, + RuntimeUserResult result, { + String? reason, + }); + + Future> cleanRuntimeTasks({ + RuntimeTaskCleanMode mode = RuntimeTaskCleanMode.defaultMode, + }); + GetterSnapshot loadSnapshot(); } @@ -257,6 +295,137 @@ class FakeGetterAdapter implements GetterAdapter { }); } + @override + Future checkPackageForUpdate( + String packageId, { + String? repositoryId, + String? installedVersion, + String? pinVersion, + }) async { + return RuntimeUpdateCheckResult.fromJson({ + 'package': { + 'id': packageId, + 'name': 'F-Droid', + 'repository': repositoryId ?? 'official', + 'permissions': {'free_network': false}, + }, + 'update': { + 'network_required': false, + 'package_id': packageId, + 'installed_version': installedVersion, + 'effective_local_version': pinVersion ?? installedVersion, + 'policy': {'pin_version': pinVersion}, + 'status': 'update_available', + 'selected': { + 'package_id': packageId, + 'candidate': { + 'version': '1.2.0', + 'artifacts': [ + { + 'name': 'app.apk', + 'url': 'https://example.invalid/app.apk', + 'file_name': 'app.apk', + }, + ], + }, + 'artifact': { + 'name': 'app.apk', + 'url': 'https://example.invalid/app.apk', + 'file_name': 'app.apk', + }, + }, + 'actions': [ + { + 'type': 'download', + 'url': 'https://example.invalid/app.apk', + 'file_name': 'app.apk', + }, + ], + }, + 'action': { + 'action_id': 'action-fake', + 'package_id': packageId, + }, + }); + } + + @override + Future submitRuntimeAction(String actionId) async { + return RuntimeTaskSnapshot.fromJson(_runtimeTaskJson('task-1')); + } + + @override + Future> listRuntimeTasks({ + bool active = false, + String? packageId, + }) async { + return [ + RuntimeTaskSnapshot.fromJson(_runtimeTaskJson('task-1')), + ]; + } + + @override + Future getRuntimeTask(String taskId) async { + return RuntimeTaskSnapshot.fromJson(_runtimeTaskJson(taskId)); + } + + @override + Future startRuntimeTask(String taskId) => + getRuntimeTask(taskId); + + @override + Future pauseRuntimeTask(String taskId) => + getRuntimeTask(taskId); + + @override + Future resumeRuntimeTask(String taskId) => + getRuntimeTask(taskId); + + @override + Future cancelRuntimeTask(String taskId) => + getRuntimeTask(taskId); + + @override + Future retryRuntimeTask(String taskId) => + getRuntimeTask(taskId); + + @override + Future removeRuntimeTask(String taskId) => + getRuntimeTask(taskId); + + @override + Future sendRuntimeUserResult( + String taskId, + RuntimeUserResult result, { + String? reason, + }) => + getRuntimeTask(taskId); + + @override + Future> cleanRuntimeTasks({ + RuntimeTaskCleanMode mode = RuntimeTaskCleanMode.defaultMode, + }) async { + return []; + } + + static Map _runtimeTaskJson(String taskId) { + return { + 'task_id': taskId, + 'package_id': 'android/org.fdroid.fdroid', + 'status': 'queued', + 'phase': {'category': 'queued'}, + 'progress': null, + 'capabilities': { + 'cancel': true, + 'pause': false, + 'resume': false, + 'retry': false, + }, + 'current_diagnostic': null, + 'updated_at': 1, + }; + } + @override GetterSnapshot loadSnapshot() => _snapshot; } @@ -515,6 +684,285 @@ class TaskEventSummary { final String? message; } +class RuntimeUpdateCheckResult { + const RuntimeUpdateCheckResult({ + required this.package, + required this.update, + required this.action, + }); + + factory RuntimeUpdateCheckResult.fromJson(Map json) { + return RuntimeUpdateCheckResult( + package: RuntimePackageSummary.fromJson( + _jsonMap(json['package'], 'runtime.package'), + ), + update: RuntimeUpdateSummary.fromJson( + _jsonMap(json['update'], 'runtime.update'), + ), + action: json['action'] == null + ? null + : RuntimeIssuedAction.fromJson( + _jsonMap(json['action'], 'runtime.action'), + ), + ); + } + + final RuntimePackageSummary package; + final RuntimeUpdateSummary update; + final RuntimeIssuedAction? action; +} + +class RuntimePackageSummary { + const RuntimePackageSummary({ + required this.id, + required this.name, + required this.repositoryId, + }); + + factory RuntimePackageSummary.fromJson(Map json) { + return RuntimePackageSummary( + id: _jsonString(json['id'], 'runtime.package.id'), + name: _jsonString(json['name'], 'runtime.package.name'), + repositoryId: + _jsonString(json['repository'], 'runtime.package.repository'), + ); + } + + final String id; + final String name; + final String repositoryId; +} + +class RuntimeUpdateSummary { + const RuntimeUpdateSummary({ + required this.packageId, + required this.status, + required this.installedVersion, + required this.effectiveLocalVersion, + required this.selectedVersion, + required this.actions, + }); + + factory RuntimeUpdateSummary.fromJson(Map json) { + final selected = + _jsonMapOrNull(json['selected'], 'runtime.update.selected'); + final candidate = selected == null + ? null + : _jsonMap(selected['candidate'], 'runtime.update.selected.candidate'); + return RuntimeUpdateSummary( + packageId: _jsonString(json['package_id'], 'runtime.update.package_id'), + status: _jsonString(json['status'], 'runtime.update.status'), + installedVersion: _jsonOptionalString( + json['installed_version'], + 'runtime.update.installed_version', + ), + effectiveLocalVersion: _jsonOptionalString( + json['effective_local_version'], + 'runtime.update.effective_local_version', + ), + selectedVersion: candidate == null + ? null + : _jsonString( + candidate['version'], 'runtime.update.selected.version'), + actions: _jsonList(json['actions'], 'runtime.update.actions') + .map((action) => _jsonMap(action, 'runtime.update.action')) + .toList(growable: false), + ); + } + + final String packageId; + final String status; + final String? installedVersion; + final String? effectiveLocalVersion; + final String? selectedVersion; + final List> actions; +} + +class RuntimeIssuedAction { + const RuntimeIssuedAction({required this.actionId, required this.packageId}); + + factory RuntimeIssuedAction.fromJson(Map json) { + return RuntimeIssuedAction( + actionId: _jsonString(json['action_id'], 'runtime.action.action_id'), + packageId: _jsonString(json['package_id'], 'runtime.action.package_id'), + ); + } + + final String actionId; + final String packageId; +} + +class RuntimeTaskSnapshot { + const RuntimeTaskSnapshot({ + required this.taskId, + required this.packageId, + required this.status, + required this.phase, + required this.progress, + required this.capabilities, + required this.currentDiagnostic, + required this.updatedAt, + }); + + factory RuntimeTaskSnapshot.fromJson(Map json) { + return RuntimeTaskSnapshot( + taskId: _jsonString(json['task_id'], 'runtime.task.task_id'), + packageId: _jsonString(json['package_id'], 'runtime.task.package_id'), + status: _jsonString(json['status'], 'runtime.task.status'), + phase: RuntimeTaskPhase.fromJson( + _jsonMap(json['phase'], 'runtime.task.phase'), + ), + progress: json['progress'] == null + ? null + : RuntimeTaskProgress.fromJson( + _jsonMap(json['progress'], 'runtime.task.progress'), + ), + capabilities: RuntimeTaskCapabilities.fromJson( + _jsonMap(json['capabilities'], 'runtime.task.capabilities'), + ), + currentDiagnostic: json['current_diagnostic'] == null + ? null + : RuntimeTaskDiagnostic.fromJson( + _jsonMap( + json['current_diagnostic'], + 'runtime.task.current_diagnostic', + ), + ), + updatedAt: _jsonInt(json['updated_at'], 'runtime.task.updated_at'), + ); + } + + final String taskId; + final String packageId; + final String status; + final RuntimeTaskPhase phase; + final RuntimeTaskProgress? progress; + final RuntimeTaskCapabilities capabilities; + final RuntimeTaskDiagnostic? currentDiagnostic; + final int updatedAt; +} + +class RuntimeTaskPhase { + const RuntimeTaskPhase({required this.category, required this.reason}); + + factory RuntimeTaskPhase.fromJson(Map json) { + return RuntimeTaskPhase( + category: _jsonString(json['category'], 'runtime.task.phase.category'), + reason: _jsonOptionalString(json['reason'], 'runtime.task.phase.reason'), + ); + } + + final String category; + final String? reason; +} + +class RuntimeTaskProgress { + const RuntimeTaskProgress({ + required this.unit, + required this.current, + required this.total, + }); + + factory RuntimeTaskProgress.fromJson(Map json) { + return RuntimeTaskProgress( + unit: _jsonString(json['unit'], 'runtime.task.progress.unit'), + current: _jsonInt(json['current'], 'runtime.task.progress.current'), + total: json['total'] == null + ? null + : _jsonInt(json['total'], 'runtime.task.progress.total'), + ); + } + + final String unit; + final int current; + final int? total; +} + +class RuntimeTaskCapabilities { + const RuntimeTaskCapabilities({ + required this.cancel, + required this.pause, + required this.resume, + required this.retry, + }); + + factory RuntimeTaskCapabilities.fromJson(Map json) { + return RuntimeTaskCapabilities( + cancel: _jsonBool(json['cancel'], 'runtime.task.capabilities.cancel'), + pause: _jsonBool(json['pause'], 'runtime.task.capabilities.pause'), + resume: _jsonBool(json['resume'], 'runtime.task.capabilities.resume'), + retry: _jsonBool(json['retry'], 'runtime.task.capabilities.retry'), + ); + } + + final bool cancel; + final bool pause; + final bool resume; + final bool retry; +} + +class RuntimeTaskDiagnostic { + const RuntimeTaskDiagnostic({ + required this.code, + required this.message, + required this.severity, + }); + + factory RuntimeTaskDiagnostic.fromJson(Map json) { + return RuntimeTaskDiagnostic( + code: _jsonString(json['code'], 'runtime.task.diagnostic.code'), + message: _jsonString(json['message'], 'runtime.task.diagnostic.message'), + severity: + _jsonString(json['severity'], 'runtime.task.diagnostic.severity'), + ); + } + + final String code; + final String message; + final String severity; +} + +enum RuntimeUserResult { + accepted, + rejected; + + String get wireName => switch (this) { + RuntimeUserResult.accepted => 'accepted', + RuntimeUserResult.rejected => 'rejected', + }; +} + +enum RuntimeTaskCleanMode { + defaultMode, + failed, + allInactive; + + String get wireName => switch (this) { + RuntimeTaskCleanMode.defaultMode => 'default', + RuntimeTaskCleanMode.failed => 'failed', + RuntimeTaskCleanMode.allInactive => 'all_inactive', + }; +} + +class RuntimeNotificationEnvelope { + const RuntimeNotificationEnvelope({required this.kind, required this.task}); + + factory RuntimeNotificationEnvelope.fromJson(Map json) { + final kind = _jsonString(json['kind'], 'runtime.notification.kind'); + return RuntimeNotificationEnvelope( + kind: kind, + task: kind == 'task_changed' + ? RuntimeTaskSnapshot.fromJson( + _jsonMap(json['task'], 'runtime.notification.task'), + ) + : null, + ); + } + + final String kind; + final RuntimeTaskSnapshot? task; +} + class InstalledAutogenScanOptions { const InstalledAutogenScanOptions({ this.includeSystemApps = false, diff --git a/app_flutter/lib/main.dart b/app_flutter/lib/main.dart index 5787070c..db9b2573 100644 --- a/app_flutter/lib/main.dart +++ b/app_flutter/lib/main.dart @@ -319,62 +319,71 @@ class DownloadsPage extends StatelessWidget { @override Widget build(BuildContext context) { - final tasks = getter.listDownloadTasks(); - final events = getter.listTaskEvents(after: 0, limit: 20).events; return Scaffold( key: AppKeys.downloadsRoute, appBar: AppBar(title: const Text('Downloads')), - body: tasks.isEmpty - ? const Center( - child: Text(key: AppKeys.downloadsEmpty, 'No download tasks yet'), - ) - : ListView( - padding: const EdgeInsets.all(16), - children: [ - Text('Tasks', style: Theme.of(context).textTheme.titleMedium), - const SizedBox(height: 8), - ListView.builder( - key: AppKeys.downloadsList, - shrinkWrap: true, - physics: const NeverScrollableScrollPhysics(), - itemCount: tasks.length, - itemBuilder: (context, index) { - final task = tasks[index]; - return Card( - child: ListTile( - key: AppKeys.downloadTaskRow(task.id), - title: Text(task.packageId), - subtitle: Text( - '${task.status} • ${task.downloadFileName}', - ), - trailing: task.installHandoffId == null - ? null - : const Chip(label: Text('Install handoff')), - ), - ); - }, - ), - const SizedBox(height: 16), - Text('Events', style: Theme.of(context).textTheme.titleMedium), - const SizedBox(height: 8), - ListView.builder( - key: AppKeys.taskEventsList, - shrinkWrap: true, - physics: const NeverScrollableScrollPhysics(), - itemCount: events.length, - itemBuilder: (context, index) { - final event = events[index]; - return ListTile( - key: AppKeys.taskEventRow(event.cursor), - title: Text(event.kind), - subtitle: Text( - '${event.taskId} • ${event.status ?? 'no status'}', - ), - ); - }, + body: FutureBuilder>( + future: getter.listRuntimeTasks(), + builder: (context, snapshot) { + if (snapshot.connectionState != ConnectionState.done) { + return const Center(child: CircularProgressIndicator()); + } + if (snapshot.hasError) { + return const Center( + child: Text( + key: AppKeys.downloadsEmpty, + 'Runtime tasks unavailable', + ), + ); + } + final tasks = snapshot.data ?? const []; + if (tasks.isEmpty) { + return const Center( + child: Text(key: AppKeys.downloadsEmpty, 'No runtime tasks yet'), + ); + } + return ListView.builder( + key: AppKeys.downloadsList, + padding: const EdgeInsets.all(16), + itemCount: tasks.length, + itemBuilder: (context, index) { + final task = tasks[index]; + return Card( + child: ListTile( + key: AppKeys.downloadTaskRow(task.taskId), + title: Text(task.packageId), + subtitle: Text( + '${task.status} • ${task.phase.category}', + ), + trailing: + _TaskCapabilitiesChips(capabilities: task.capabilities), ), - ], - ), + ); + }, + ); + }, + ), + ); + } +} + +class _TaskCapabilitiesChips extends StatelessWidget { + const _TaskCapabilitiesChips({required this.capabilities}); + + final RuntimeTaskCapabilities capabilities; + + @override + Widget build(BuildContext context) { + final labels = [ + if (capabilities.cancel) 'Cancel', + if (capabilities.pause) 'Pause', + if (capabilities.resume) 'Resume', + if (capabilities.retry) 'Retry', + ]; + if (labels.isEmpty) return const SizedBox.shrink(); + return Wrap( + spacing: 4, + children: labels.map((label) => Chip(label: Text(label))).toList(), ); } } diff --git a/app_flutter/lib/native_getter_adapter.dart b/app_flutter/lib/native_getter_adapter.dart index ffd3101f..9fa0f9ff 100644 --- a/app_flutter/lib/native_getter_adapter.dart +++ b/app_flutter/lib/native_getter_adapter.dart @@ -106,6 +106,10 @@ class MethodChannelGetterAdapter extends FakeGetterAdapter { }); } + Stream runtimeNotificationEnvelopes() { + return runtimeNotifications().map(RuntimeNotificationEnvelope.fromJson); + } + Future> invokeRuntimeOperation( String operation, { Map payload = const {}, @@ -119,6 +123,129 @@ class MethodChannelGetterAdapter extends FakeGetterAdapter { ); } + @override + Future checkPackageForUpdate( + String packageId, { + String? repositoryId, + String? installedVersion, + String? pinVersion, + }) async { + final payload = { + 'package_id': packageId, + if (repositoryId != null) 'repository_id': repositoryId, + if (installedVersion != null) 'installed_version': installedVersion, + if (pinVersion != null) 'pin_version': pinVersion, + }; + final data = await invokeRuntimeOperation( + 'update_check_package_issue_action', + payload: payload, + ); + return RuntimeUpdateCheckResult.fromJson(data); + } + + @override + Future submitRuntimeAction(String actionId) { + return _runtimeTaskOperation( + 'task_submit', + {'action_id': actionId}, + ); + } + + @override + Future> listRuntimeTasks({ + bool active = false, + String? packageId, + }) async { + final data = await invokeRuntimeOperation( + 'task_list', + payload: { + 'active': active, + if (packageId != null) 'package_id': packageId, + }, + ); + return _runtimeTasksFromData(data); + } + + @override + Future getRuntimeTask(String taskId) { + return _runtimeTaskOperation('task_get', _taskIdPayload(taskId)); + } + + @override + Future startRuntimeTask(String taskId) { + return _runtimeTaskOperation('task_start', _taskIdPayload(taskId)); + } + + @override + Future pauseRuntimeTask(String taskId) { + return _runtimeTaskOperation('task_pause', _taskIdPayload(taskId)); + } + + @override + Future resumeRuntimeTask(String taskId) { + return _runtimeTaskOperation('task_resume', _taskIdPayload(taskId)); + } + + @override + Future cancelRuntimeTask(String taskId) { + return _runtimeTaskOperation('task_cancel', _taskIdPayload(taskId)); + } + + @override + Future retryRuntimeTask(String taskId) { + return _runtimeTaskOperation('task_retry', _taskIdPayload(taskId)); + } + + @override + Future removeRuntimeTask(String taskId) { + return _runtimeTaskOperation('task_remove', _taskIdPayload(taskId)); + } + + @override + Future sendRuntimeUserResult( + String taskId, + RuntimeUserResult result, { + String? reason, + }) { + return _runtimeTaskOperation( + 'task_user_result', + { + 'task_id': taskId, + 'result': result.wireName, + if (reason != null) 'reason': reason, + }, + ); + } + + @override + Future> cleanRuntimeTasks({ + RuntimeTaskCleanMode mode = RuntimeTaskCleanMode.defaultMode, + }) async { + final data = await invokeRuntimeOperation( + 'task_clean', + payload: {'mode': mode.wireName}, + ); + return _runtimeTasksFromData(data); + } + + Future _runtimeTaskOperation( + String operation, + Map payload, + ) async { + final data = await invokeRuntimeOperation(operation, payload: payload); + return RuntimeTaskSnapshot.fromJson(data); + } + + List _runtimeTasksFromData(Map data) { + return _asList(data['tasks'], 'runtime tasks') + .map((task) => RuntimeTaskSnapshot.fromJson(_asMap(task, 'task'))) + .toList(growable: false); + } + + Map _taskIdPayload(String taskId) { + return {'task_id': taskId}; + } + Future> _invokeGetterData( String method, Map arguments, diff --git a/app_flutter/test/native_getter_adapter_test.dart b/app_flutter/test/native_getter_adapter_test.dart index fa194d90..52279395 100644 --- a/app_flutter/test/native_getter_adapter_test.dart +++ b/app_flutter/test/native_getter_adapter_test.dart @@ -204,6 +204,126 @@ void main() { ); }); + test('typed runtime update check returns getter-issued action id', () async { + final calls = []; + TestDefaultBinaryMessengerBinding.instance.defaultBinaryMessenger + .setMockMethodCallHandler(channel, (call) async { + calls.add(call); + if (call.method != 'runtimeOperation') { + fail('unexpected method ${call.method}'); + } + final args = + (call.arguments as Map).cast(); + if (args['operation'] == 'update_check_package_issue_action') { + return jsonEncode({ + 'ok': true, + 'command': 'runtime operation', + 'data': { + 'package': { + 'id': 'android/org.fdroid.fdroid', + 'name': 'F-Droid', + 'repository': 'official', + 'permissions': {'free_network': false}, + }, + 'update': { + 'network_required': false, + 'package_id': 'android/org.fdroid.fdroid', + 'installed_version': '1.0.0', + 'effective_local_version': '1.0.0', + 'policy': {'pin_version': null}, + 'status': 'update_available', + 'selected': { + 'package_id': 'android/org.fdroid.fdroid', + 'candidate': { + 'version': '1.2.0', + 'artifacts': [], + }, + }, + 'actions': [ + { + 'type': 'download', + 'url': 'https://example.invalid/app.apk', + 'file_name': 'app.apk', + }, + ], + }, + 'action': { + 'action_id': 'action-1', + 'package_id': 'android/org.fdroid.fdroid', + }, + }, + 'warnings': [], + }); + } + if (args['operation'] == 'task_submit') { + return jsonEncode({ + 'ok': true, + 'command': 'runtime operation', + 'data': _runtimeTaskJson('task-1', status: 'queued'), + 'warnings': [], + }); + } + fail('unexpected runtime operation ${args['operation']}'); + }); + + const adapter = MethodChannelGetterAdapter(channel: channel); + final update = await adapter.checkPackageForUpdate( + 'android/org.fdroid.fdroid', + repositoryId: 'official', + installedVersion: '1.0.0', + ); + final task = await adapter.submitRuntimeAction(update.action!.actionId); + + expect(update.action!.actionId, 'action-1'); + expect(update.update.selectedVersion, '1.2.0'); + expect(task.taskId, 'task-1'); + expect(calls.first.arguments, { + 'operation': 'update_check_package_issue_action', + 'payload': { + 'package_id': 'android/org.fdroid.fdroid', + 'repository_id': 'official', + 'installed_version': '1.0.0', + }, + }); + expect(calls.last.arguments, { + 'operation': 'task_submit', + 'payload': {'action_id': 'action-1'}, + }); + }); + + test('typed runtime task controls parse task snapshots', () async { + final operations = []; + TestDefaultBinaryMessengerBinding.instance.defaultBinaryMessenger + .setMockMethodCallHandler(channel, (call) async { + final args = + (call.arguments as Map).cast(); + operations.add(args['operation']! as String); + return jsonEncode({ + 'ok': true, + 'command': 'runtime operation', + 'data': args['operation'] == 'task_list' || + args['operation'] == 'task_clean' + ? { + 'tasks': [ + _runtimeTaskJson('task-1', status: 'running') + ], + } + : _runtimeTaskJson('task-1', status: 'running'), + 'warnings': [], + }); + }); + + const adapter = MethodChannelGetterAdapter(channel: channel); + final tasks = await adapter.listRuntimeTasks(active: true); + final canceled = await adapter.cancelRuntimeTask('task-1'); + final cleaned = await adapter.cleanRuntimeTasks(); + + expect(tasks.single.status, 'running'); + expect(canceled.taskId, 'task-1'); + expect(cleaned.single.taskId, 'task-1'); + expect(operations, ['task_list', 'task_cancel', 'task_clean']); + }); + test('native runtime operation forwards operation and payload', () async { MethodCall? captured; TestDefaultBinaryMessengerBinding.instance.defaultBinaryMessenger @@ -273,6 +393,26 @@ void main() { }); } +Map _runtimeTaskJson( + String taskId, { + required String status, +}) => + { + 'task_id': taskId, + 'package_id': 'android/org.fdroid.fdroid', + 'status': status, + 'phase': {'category': status}, + 'progress': null, + 'capabilities': { + 'cancel': true, + 'pause': false, + 'resume': false, + 'retry': false, + }, + 'current_diagnostic': null, + 'updated_at': 1, + }; + Map _previewJson() => { 'operation': 'installed.preview', 'target_repo_id': 'local_autogen', diff --git a/app_flutter/test/widget_test.dart b/app_flutter/test/widget_test.dart index 5b72c9fb..6fdf8e08 100644 --- a/app_flutter/test/widget_test.dart +++ b/app_flutter/test/widget_test.dart @@ -53,7 +53,7 @@ void main() { expect(find.byKey(AppKeys.repoRow('local_autogen')), findsOneWidget); }); - testWidgets('downloads route renders getter task DTOs read-only', + testWidgets('downloads route renders runtime task snapshots read-only', (tester) async { await tester.pumpWidget(const UpgradeAllApp()); @@ -63,9 +63,8 @@ void main() { expect(find.byKey(AppKeys.downloadsRoute), findsOneWidget); expect(find.byKey(AppKeys.downloadsList), findsOneWidget); expect(find.byKey(AppKeys.downloadTaskRow('task-1')), findsOneWidget); - expect(find.byKey(AppKeys.taskEventsList), findsOneWidget); - expect(find.byKey(AppKeys.taskEventRow(3)), findsOneWidget); - expect(find.text('Install handoff'), findsOneWidget); + expect(find.text('queued • queued'), findsOneWidget); + expect(find.text('Cancel'), findsOneWidget); }); testWidgets('downloads route exposes getter empty task state', @@ -243,6 +242,13 @@ class _NoTaskGetterAdapter extends FakeGetterAdapter { hasMore: false, ); } + + @override + Future> listRuntimeTasks({ + bool active = false, + String? packageId, + }) async => + const []; } class _LegacyMigrationCapableGetterAdapter extends FakeGetterAdapter { diff --git a/todo.md b/todo.md index 445e995e..a189d65a 100644 --- a/todo.md +++ b/todo.md @@ -504,6 +504,7 @@ Completed additional UI/bridge slice: 11. Render getter-owned task/event DTOs on the Flutter Downloads route without adding a Dart task state machine. 12. Add Flutter widget/dev integration coverage for reading and rendering getter task lifecycle DTOs. 13. Add native bridge process-lifetime runtime singleton, runtime operation dispatcher, bounded best-effort notification drain, Kotlin EventChannel, and Dart runtime notification stream primitive. +14. Add typed Dart runtime/update methods for package update-check action issuance, action-id submission, task query/control/user-result/remove/clean, typed runtime notifications, and runtime task snapshot rendering on the Downloads route. Remaining tasks: @@ -512,7 +513,7 @@ Remaining tasks: 3. Replace or retire the older persisted fake CLI task scaffold so public task status/control language uses ADR-0011 (`completed`, `user-result`, remove/clean, no cross-invocation task state). 4. Implement live provider/downloader behavior beyond the fake/offline proof after a later ADR accepts real side-effect details. 5. Define Android production install handoff URI/SAF/permission/notification details and wire platform adapter execution after later ADRs. -6. Add product-level Flutter BDD for update/download user flows after live/provider/background/installer decisions are accepted; the current slice only covers read-only DTO rendering. +6. Add product-level Flutter BDD for full update/download/install user flows after live/provider/background/installer decisions are accepted; the current slice covers typed runtime DTO methods and read-only runtime task snapshot rendering. Acceptance progress: From b2c03b76b21e32d4203fa5a707faf8678bb90d70 Mon Sep 17 00:00:00 2001 From: Xiangzhe Date: Thu, 25 Jun 2026 13:42:05 +0800 Subject: [PATCH 40/85] chore(app): point CLI debug tasks at runtime script --- .../dev_test/cli_getter_adapter_test.dart | 10 +++++-- app_flutter/lib/cli_getter_adapter.dart | 5 ++-- core-getter/src/main/rust/getter | 2 +- ...06-package-centric-cli-command-contract.md | 17 +++++++----- .../0007-flutter-getter-bridge-contract.md | 4 +-- todo.md | 26 +++++++++---------- 6 files changed, 37 insertions(+), 27 deletions(-) diff --git a/app_flutter/dev_test/cli_getter_adapter_test.dart b/app_flutter/dev_test/cli_getter_adapter_test.dart index 241f35dd..18bd3ee7 100644 --- a/app_flutter/dev_test/cli_getter_adapter_test.dart +++ b/app_flutter/dev_test/cli_getter_adapter_test.dart @@ -72,12 +72,18 @@ void main() { bundle.path, ]); _runGetter(getterCli, dataDir.path, [ - 'task', + 'debug', + 'fake-task', 'submit', '--request', taskRequest.path, ]); - _runGetter(getterCli, dataDir.path, ['task', 'run', 'task-1']); + _runGetter(getterCli, dataDir.path, [ + 'debug', + 'fake-task', + 'run', + 'task-1', + ]); final repositories = adapter.listRepositories(); expect(repositories.map((repo) => repo.id), contains('official')); diff --git a/app_flutter/lib/cli_getter_adapter.dart b/app_flutter/lib/cli_getter_adapter.dart index 9d6a1c69..6bd39b37 100644 --- a/app_flutter/lib/cli_getter_adapter.dart +++ b/app_flutter/lib/cli_getter_adapter.dart @@ -74,7 +74,7 @@ class CliGetterAdapter implements GetterAdapter { @override List listDownloadTasks() { - final json = _runGetter(const ['task', 'list']); + final json = _runGetter(const ['debug', 'fake-task', 'list']); final tasks = _asList(_data(json)['tasks'], 'tasks'); return tasks .map((task) => _downloadTaskFromJson(_asMap(task, 'task'))) @@ -84,7 +84,8 @@ class CliGetterAdapter implements GetterAdapter { @override TaskEventPage listTaskEvents({required int after, required int limit}) { final json = _runGetter([ - 'task', + 'debug', + 'fake-task', 'events', '--after', after.toString(), diff --git a/core-getter/src/main/rust/getter b/core-getter/src/main/rust/getter index 9ae7e568..ae5b00cc 160000 --- a/core-getter/src/main/rust/getter +++ b/core-getter/src/main/rust/getter @@ -1 +1 @@ -Subproject commit 9ae7e5680188b2ef87bc052d2e55d5375116c0ce +Subproject commit ae5b00ccf0989ee1b0b69521bb508d4136c2bcb4 diff --git a/docs/architecture/adr/0006-package-centric-cli-command-contract.md b/docs/architecture/adr/0006-package-centric-cli-command-contract.md index 23a59e3b..bb232ab0 100644 --- a/docs/architecture/adr/0006-package-centric-cli-command-contract.md +++ b/docs/architecture/adr/0006-package-centric-cli-command-contract.md @@ -22,12 +22,13 @@ getter --data-dir repo validate getter --data-dir package eval [--repo ] getter --data-dir storage validate getter --data-dir update check --fixture -getter --data-dir task submit --request -getter --data-dir task run -getter --data-dir task list -getter --data-dir task cancel -getter --data-dir task events --after --limit -getter --data-dir task install-result --status +getter --data-dir runtime script --script +getter --data-dir debug fake-task submit --request +getter --data-dir debug fake-task run +getter --data-dir debug fake-task list +getter --data-dir debug fake-task cancel +getter --data-dir debug fake-task events --after --limit +getter --data-dir debug fake-task install-result --status getter --data-dir autogen installed preview --inventory getter --data-dir autogen installed apply --preview (--accept-all|--accept ...) getter --data-dir autogen cleanup preview --inventory @@ -103,7 +104,9 @@ The first installed-app autogen slice accepts an Android/platform-provided inven `update check --fixture ` is the first Phase D offline update-check slice. The fixture contract is explicitly offline and uses `format = "getter-offline-update-check"`, `version = 1`, `package_id`, optional `installed_version`, optional `pin_version` (with transitional `ignored_version` accepted as an input alias), and normalized candidate/artifact DTOs. The command returns `network_required = false`, observed `installed_version`, `effective_local_version`, a getter-owned status (`update_available`, `up_to_date`, or `no_candidates`), the selected update when one exists, and generated download/install action DTOs. An update-selected result must have an actionable artifact; a selected candidate without artifacts is a structured update-check error rather than `update_available` with no actions. It reuses Rust getter update selection and version comparison; it does not run providers, perform downloads, persist download tasks, stream events, or call Android installers. -The first getter-owned task lifecycle slice is also explicitly offline/fake and command-driven. `task submit --request ` accepts `format = "getter-download-request"`, `version = 1`, `package_id`, `executor = "fake"`, and update actions containing at least one `download` action; an optional `install` action creates an abstract install handoff after a successful fake run. `task run ` deterministically advances the fake task to `succeeded`; it performs no network I/O and writes no downloaded bytes. `task list` returns persisted task summaries from `main.db`. `task cancel ` persists cancellation for `queued`/`running` tasks, is idempotent for already-canceled tasks, and rejects terminal success/failure with a structured download error. `task events --after --limit ` is a pollable CLI/dev event contract with a positive `limit`; it is not a native streaming API, and native streaming/backpressure remains deferred. `task install-result --status ` records the platform-side result of an abstract handoff; the getter-created `requested` handoff state is not accepted as a platform result. Getter records handoff requests/results but does not call Android installers, request permissions, create notifications, or decide Android URI/SAF semantics. +The old persisted fake downloader slice is retained only as debug scaffolding under `debug fake-task ...`. `debug fake-task submit --request ` accepts `format = "getter-download-request"`, `version = 1`, `package_id`, `executor = "fake"`, and update actions containing at least one `download` action; an optional `install` action creates an abstract install handoff after a successful fake run. `debug fake-task run ` deterministically advances the fake task to `succeeded`; it performs no network I/O and writes no downloaded bytes. `debug fake-task list` returns persisted fake-task summaries from `main.db`. `debug fake-task cancel ` persists cancellation for `queued`/`running` fake tasks, is idempotent for already-canceled tasks, and rejects terminal success/failure with a structured download error. `debug fake-task events --after --limit ` is a pollable debug event contract with a positive `limit`; it is not the ADR-0011 runtime event model. `debug fake-task install-result --status ` records the platform-side result of an abstract debug handoff; the getter-created `requested` handoff state is not accepted as a platform result. This scaffold is not a product task API. + +ADR-0011 runtime task debugging uses `runtime script --script `. The script command creates one in-memory `GetterRuntime` for that single CLI process, executes scripted operations such as `issue_action`, `submit_action`, `task_start`, `task_complete_download`, `task_user_result`, `task_remove`, and `task_clean`, then drops all runtime task state when the process exits. It exists so CLI tests can cover runtime remove/clean/control semantics without introducing a task database, daemon, or cross-invocation task promise. Product task submission remains getter-issued opaque `action_id` only through the native bridge/runtime operation path. `repo validate ` validates a repository path offline without requiring it to be registered first. It returns `valid`, `diagnostics`, `package_count`, and `network_required = false`; diagnostics are getter-owned structured records with stable codes, message, severity, source path, and optional package id/field. diff --git a/docs/architecture/adr/0007-flutter-getter-bridge-contract.md b/docs/architecture/adr/0007-flutter-getter-bridge-contract.md index 66d49ad7..277cc2df 100644 --- a/docs/architecture/adr/0007-flutter-getter-bridge-contract.md +++ b/docs/architecture/adr/0007-flutter-getter-bridge-contract.md @@ -148,9 +148,9 @@ Flutter then calls a getter bridge operation equivalent to `legacy import-room-d ## Event model -The first bridge slice is snapshot-only. Streaming events, progress, cancellation, backpressure, foreground services, notification lifecycle, and installer handoff are explicitly deferred to the update/download/install lifecycle ADR/work. +The initial bridge slice was snapshot-only. ADR-0011 supersedes the old persisted fake-task CLI scaffold for product task flow: runtime task state is process-memory only in the native getter singleton, `RuntimeNotification.task_changed` is pushed over the bridge, and current-state task query operations remain authoritative. The remaining `debug fake-task ...` CLI commands are development scaffolding, not a Flutter/product task API. CLI runtime task coverage uses `runtime script --script `, which executes within one process and intentionally drops runtime task state after the command exits. -The first Phase D lifecycle slice defines getter-owned task/event/handoff DTOs through the CLI only: task state is persisted in getter `main.db`, task events are pollable with `after` cursor plus `limit`, and fake executor progress is command-driven rather than background-streamed. This pollable CLI/dev contract is not the final native stream API. Flutter should not maintain its own task state machine; future Flutter/bridge work must render getter task/event DTOs or ask getter for richer fields. +Flutter should not maintain its own task state machine; it renders getter-owned runtime task snapshots and invokes getter-owned task controls/update operations using opaque `action_id`s. Android platform install remains a handoff boundary. Getter may request/record an abstract install handoff, but Android permissions, notifications, PackageInstaller/Shizuku/root execution, and path-versus-URI/SAF semantics belong to platform adapter work and remain outside this bridge slice. diff --git a/todo.md b/todo.md index a189d65a..cd3e85d3 100644 --- a/todo.md +++ b/todo.md @@ -493,36 +493,36 @@ Completed tasks: 6. Add getter-core task/event/install-handoff DTOs for the first offline lifecycle proof. 7. Add main DB task/event/install-handoff tables and storage APIs with TDD coverage. 8. Implement deterministic fake/offline downloader behavior beyond the previous placeholder crate: submit, run, cancel, list, poll events, and record install result. -9. Add CLI commands and BDD coverage for `task submit`, `task run`, `task list`, `task cancel`, `task events`, and `task install-result`. +9. Add debug fake-task CLI commands and BDD coverage for persisted offline scaffold operations: submit, run, list, cancel, events, and install-result. 10. Accept ADR-0011 for the in-memory runtime/task/action/RuntimeNotification model. 11. Add `getter-core::runtime` with in-memory `GetterRuntime`, single-use `action_id`, sealed action plans, package-version Lua object binding, generic `user-result`, mock download/install state, package-level non-waiting lock, task controls, remove/clean, and RuntimeNotification DTOs with TDD coverage. 12. Add shared `getter-operations::runtime` JSON controls for offline update-check action issuance plus submit/get/list/start/progress/complete-download/pause/resume/user-result/cancel/retry/remove/clean without persisted task state. Completed additional UI/bridge slice: -10. Extend Flutter getter bridge DTOs/adapters with read-only task list and event page APIs backed by existing getter CLI `task list` and `task events`. -11. Render getter-owned task/event DTOs on the Flutter Downloads route without adding a Dart task state machine. -12. Add Flutter widget/dev integration coverage for reading and rendering getter task lifecycle DTOs. +10. Extend Flutter getter bridge DTOs/adapters with read-only debug fake-task list and event page APIs backed by getter CLI `debug fake-task list/events`. +11. Render getter-owned task/event DTOs on the initial Flutter Downloads route without adding a Dart task state machine. +12. Add Flutter widget/dev integration coverage for reading and rendering getter debug fake-task lifecycle DTOs. 13. Add native bridge process-lifetime runtime singleton, runtime operation dispatcher, bounded best-effort notification drain, Kotlin EventChannel, and Dart runtime notification stream primitive. 14. Add typed Dart runtime/update methods for package update-check action issuance, action-id submission, task query/control/user-result/remove/clean, typed runtime notifications, and runtime task snapshot rendering on the Downloads route. +15. Add `runtime script --script ` as a single-process CLI debug harness over `getter-operations::runtime`, including task remove/clean coverage, and move the old persisted fake downloader scaffold out of the public `task` namespace to `debug fake-task ...`. Remaining tasks: 1. Replace the current static `updates` package seam with live provider update-check action issuance that materializes sealed action plans and returns opaque `action_id` to Flutter; keep Dart from constructing action payloads. -2. Optionally add CLI single-process scripted/debug tooling over `getter-operations::runtime` without pretending separate invocations share memory. -3. Replace or retire the older persisted fake CLI task scaffold so public task status/control language uses ADR-0011 (`completed`, `user-result`, remove/clean, no cross-invocation task state). -4. Implement live provider/downloader behavior beyond the fake/offline proof after a later ADR accepts real side-effect details. -5. Define Android production install handoff URI/SAF/permission/notification details and wire platform adapter execution after later ADRs. -6. Add product-level Flutter BDD for full update/download/install user flows after live/provider/background/installer decisions are accepted; the current slice covers typed runtime DTO methods and read-only runtime task snapshot rendering. +2. Continue retiring old fake-task scaffolding from product-facing adapters/docs as newer runtime/native flows cover those cases; the remaining `debug fake-task ...` commands are development-only. +3. Implement live provider/downloader behavior beyond the fake/offline proof after a later ADR accepts real side-effect details. +4. Define Android production install handoff URI/SAF/permission/notification details and wire platform adapter execution after later ADRs. +5. Add product-level Flutter BDD for full update/download/install user flows after live/provider/background/installer decisions are accepted; the current slice covers typed runtime DTO methods and read-only runtime task snapshot rendering. Acceptance progress: - CLI can run an offline fixture update check: done. -- Older CLI/dev fake task scaffold can persist/list task state: done, but superseded by ADR-0011 and slated for replacement/retirement. +- Older CLI/dev fake task scaffold can persist/list task state: done under `debug fake-task ...`; it is development-only and superseded by ADR-0011 for product runtime tasks. - `getter-core::runtime` can manage in-memory tasks, controls, `user-result`, retry, package lock, remove/clean, and notifications: first TDD slice done. -- Getter can expose pollable task events with cursor/limit in the older CLI/dev scaffold; ADR-0011 native push stream skeleton is done with bounded best-effort EventChannel delivery and current-state query operations still pending typed Flutter UI use. -- Getter can record abstract install handoff requests/results in the older scaffold; ADR-0011 uses generic `user-result` and mock install waiting-user state, Android installer execution remains deferred. -- Flutter displays getter task/event DTOs rather than calculating status itself: done for read-only CLI/dev bridge slice. +- Getter can expose pollable task events with cursor/limit only in the `debug fake-task` scaffold; ADR-0011 native push stream skeleton is done with bounded best-effort EventChannel delivery and typed current-state query operations in Flutter. +- Getter can record abstract install handoff requests/results in the debug fake-task scaffold; ADR-0011 uses generic `user-result` and mock install waiting-user state, Android installer execution remains deferred. +- Flutter displays getter runtime task snapshots rather than calculating status itself: done for read-only typed runtime snapshot rendering. - Android platform adapter owns permissions/notifications/installer handoff: documented/deferred; no Android execution added in this slice. ## 10. Do-not-do list for the next agent From 3897038a88dce323e5d315c2e5e1fae3279d0dfe Mon Sep 17 00:00:00 2001 From: Xiangzhe Date: Thu, 25 Jun 2026 13:48:36 +0800 Subject: [PATCH 41/85] refactor(app): drop fake task DTOs from product adapter --- app_flutter/lib/cli_getter_adapter.dart | 56 ++++++++++- app_flutter/lib/getter_adapter.dart | 126 ------------------------ app_flutter/test/widget_test.dart | 13 --- 3 files changed, 54 insertions(+), 141 deletions(-) diff --git a/app_flutter/lib/cli_getter_adapter.dart b/app_flutter/lib/cli_getter_adapter.dart index 6bd39b37..e0aebd01 100644 --- a/app_flutter/lib/cli_getter_adapter.dart +++ b/app_flutter/lib/cli_getter_adapter.dart @@ -72,7 +72,7 @@ class CliGetterAdapter implements GetterAdapter { return LegacyMigrationImportResult.fromJson(_data(json)); } - @override + /// Development-only accessor for the persisted `debug fake-task` scaffold. List listDownloadTasks() { final json = _runGetter(const ['debug', 'fake-task', 'list']); final tasks = _asList(_data(json)['tasks'], 'tasks'); @@ -81,7 +81,7 @@ class CliGetterAdapter implements GetterAdapter { .toList(growable: false); } - @override + /// Development-only accessor for the persisted `debug fake-task` scaffold. TaskEventPage listTaskEvents({required int after, required int limit}) { final json = _runGetter([ 'debug', @@ -260,6 +260,58 @@ class CliGetterAdapter implements GetterAdapter { } } +class DownloadTaskSummary { + const DownloadTaskSummary({ + required this.id, + required this.packageId, + required this.status, + required this.executor, + required this.actions, + required this.downloadFileName, + required this.downloadedFile, + required this.failureMessage, + required this.installHandoffId, + }); + + final String id; + final String packageId; + final String status; + final String executor; + final List> actions; + final String downloadFileName; + final String? downloadedFile; + final String? failureMessage; + final String? installHandoffId; +} + +class TaskEventPage { + const TaskEventPage({ + required this.events, + required this.nextCursor, + required this.hasMore, + }); + + final List events; + final int nextCursor; + final bool hasMore; +} + +class TaskEventSummary { + const TaskEventSummary({ + required this.cursor, + required this.taskId, + required this.kind, + required this.status, + required this.message, + }); + + final int cursor; + final String taskId; + final String kind; + final String? status; + final String? message; +} + Map _data(Map envelope) { return _asMap(envelope['data'], 'data'); } diff --git a/app_flutter/lib/getter_adapter.dart b/app_flutter/lib/getter_adapter.dart index d7234667..a9310724 100644 --- a/app_flutter/lib/getter_adapter.dart +++ b/app_flutter/lib/getter_adapter.dart @@ -21,10 +21,6 @@ abstract interface class GetterAdapter { Future importLegacyRoomDatabase( String databasePath); - List listDownloadTasks(); - - TaskEventPage listTaskEvents({required int after, required int limit}); - Future previewInstalledAutogen({ InstalledAutogenScanOptions options = const InstalledAutogenScanOptions(), }); @@ -141,59 +137,6 @@ class FakeGetterAdapter implements GetterAdapter { ); } - static const _downloadTasks = [ - DownloadTaskSummary( - id: 'task-1', - packageId: 'android/org.fdroid.fdroid', - status: 'succeeded', - executor: 'fake', - actions: >[ - { - 'type': 'download', - 'url': 'https://example.invalid/app.apk', - 'file_name': 'app.apk', - }, - { - 'type': 'install', - 'installer': 'android_package', - 'file': 'app.apk', - }, - ], - downloadFileName: 'app.apk', - downloadedFile: 'app.apk', - failureMessage: null, - installHandoffId: 'handoff-1', - ), - ]; - - static const _taskEvents = TaskEventPage( - events: [ - TaskEventSummary( - cursor: 1, - taskId: 'task-1', - kind: 'task_created', - status: 'queued', - message: 'Task created', - ), - TaskEventSummary( - cursor: 2, - taskId: 'task-1', - kind: 'task_succeeded', - status: 'succeeded', - message: 'Task succeeded', - ), - TaskEventSummary( - cursor: 3, - taskId: 'task-1', - kind: 'install_handoff_requested', - status: 'succeeded', - message: 'Install handoff requested', - ), - ], - nextCursor: 3, - hasMore: false, - ); - @override Future> readMigrationReports() async { return const []; @@ -210,23 +153,6 @@ class FakeGetterAdapter implements GetterAdapter { ); } - @override - List listDownloadTasks() => _downloadTasks; - - @override - TaskEventPage listTaskEvents({required int after, required int limit}) { - final events = _taskEvents.events - .where((event) => event.cursor > after) - .take(limit) - .toList(growable: false); - final nextCursor = events.isEmpty ? after : events.last.cursor; - return TaskEventPage( - events: events, - nextCursor: nextCursor, - hasMore: _taskEvents.events.any((event) => event.cursor > nextCursor), - ); - } - @override Future previewInstalledAutogen({ InstalledAutogenScanOptions options = const InstalledAutogenScanOptions(), @@ -632,58 +558,6 @@ class MigrationSourceCounts { final int extraHubRows; } -class DownloadTaskSummary { - const DownloadTaskSummary({ - required this.id, - required this.packageId, - required this.status, - required this.executor, - required this.actions, - required this.downloadFileName, - required this.downloadedFile, - required this.failureMessage, - required this.installHandoffId, - }); - - final String id; - final String packageId; - final String status; - final String executor; - final List> actions; - final String downloadFileName; - final String? downloadedFile; - final String? failureMessage; - final String? installHandoffId; -} - -class TaskEventPage { - const TaskEventPage({ - required this.events, - required this.nextCursor, - required this.hasMore, - }); - - final List events; - final int nextCursor; - final bool hasMore; -} - -class TaskEventSummary { - const TaskEventSummary({ - required this.cursor, - required this.taskId, - required this.kind, - required this.status, - required this.message, - }); - - final int cursor; - final String taskId; - final String kind; - final String? status; - final String? message; -} - class RuntimeUpdateCheckResult { const RuntimeUpdateCheckResult({ required this.package, diff --git a/app_flutter/test/widget_test.dart b/app_flutter/test/widget_test.dart index 6fdf8e08..24cbe1c1 100644 --- a/app_flutter/test/widget_test.dart +++ b/app_flutter/test/widget_test.dart @@ -230,19 +230,6 @@ void main() { class _NoTaskGetterAdapter extends FakeGetterAdapter { const _NoTaskGetterAdapter(); - @override - List listDownloadTasks() => - const []; - - @override - TaskEventPage listTaskEvents({required int after, required int limit}) { - return const TaskEventPage( - events: [], - nextCursor: 0, - hasMore: false, - ); - } - @override Future> listRuntimeTasks({ bool active = false, From dd9115c08c570347ce44d5d9258e507cd59bac7e Mon Sep 17 00:00:00 2001 From: Xiangzhe Date: Thu, 25 Jun 2026 13:56:05 +0800 Subject: [PATCH 42/85] docs(lua): describe static mock provider boundary --- core-getter/src/main/rust/getter | 2 +- docs/lua-api/package-lifecycle.md | 4 ++-- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/core-getter/src/main/rust/getter b/core-getter/src/main/rust/getter index ae5b00cc..1cde4f4b 160000 --- a/core-getter/src/main/rust/getter +++ b/core-getter/src/main/rust/getter @@ -1 +1 @@ -Subproject commit ae5b00ccf0989ee1b0b69521bb508d4136c2bcb4 +Subproject commit 1cde4f4bf01c5f703724f4ca62cc89496886eedd diff --git a/docs/lua-api/package-lifecycle.md b/docs/lua-api/package-lifecycle.md index 97533012..445637ec 100644 --- a/docs/lua-api/package-lifecycle.md +++ b/docs/lua-api/package-lifecycle.md @@ -61,7 +61,7 @@ return { } ``` -As the first offline/mock-provider bridge toward this lifecycle, package Lua may also declare static `updates` candidates in the package table. Getter validates this table, performs Rust-owned selection/version comparison, and issues opaque runtime `action_id`s from the selected candidate; Flutter must still return only the getter-issued `action_id` and must not assemble download/install action payloads. +As the first offline/mock-provider bridge toward this lifecycle, package Lua may also declare static `updates` candidates in the package table. Getter validates this table, routes it through a mock provider boundary (`StaticPackageUpdatesProvider`), performs Rust-owned selection/version comparison, and issues opaque runtime `action_id`s from the selected candidate; Flutter must still return only the getter-issued `action_id` and must not assemble download/install action payloads. ```lua return package_def { @@ -84,7 +84,7 @@ return package_def { } ``` -The first Phase D implementation slice exposes this boundary only through an offline CLI fixture command: `getter --data-dir update check --fixture `. The fixture is normalized JSON, not live provider output, and the command returns `network_required = false`, update-check status, selected candidate/artifact, and generated download/install action DTOs. It does not execute network providers, download files, persist download tasks, stream progress events, or invoke Android installers. +The first Phase D implementation exposes offline update checks through both a normalized CLI fixture command (`getter --data-dir update check --fixture `) and registered-package native/runtime action issuance over static Lua `updates`. These are mock-provider paths, not live provider output. They return `network_required = false`, update-check status, selected candidate/artifact, and getter-owned action issuance data. They do not execute network providers, download files, persist download tasks, stream progress events, or invoke Android installers. ADR-0011 supersedes the earlier persisted fake task scaffold. The accepted Phase D runtime consumes getter-issued actions through an in-memory process-lifetime runtime: task state is not stored in SQLite, `action_id` is single-use, task submission binds a sealed action plan plus package-version Lua object, mock download/install executors simulate task state, and `RuntimeNotification.task_changed` is pushed to Flutter as a best-effort current snapshot. CLI coverage for this model should use Rust runtime tests or a single-process scripted/debug command rather than pretending separate CLI invocations share task memory. From 1530a425ec9b9e02cc2662891ee3241a37d407c7 Mon Sep 17 00:00:00 2001 From: Xiangzhe Date: Thu, 25 Jun 2026 14:13:23 +0800 Subject: [PATCH 43/85] refactor(app): drop CLI fake task adapter APIs --- .../dev_test/cli_getter_adapter_test.dart | 58 -------- app_flutter/lib/cli_getter_adapter.dart | 130 ------------------ .../0007-flutter-getter-bridge-contract.md | 42 +++--- todo.md | 8 +- 4 files changed, 25 insertions(+), 213 deletions(-) diff --git a/app_flutter/dev_test/cli_getter_adapter_test.dart b/app_flutter/dev_test/cli_getter_adapter_test.dart index 18bd3ee7..ddb60c0f 100644 --- a/app_flutter/dev_test/cli_getter_adapter_test.dart +++ b/app_flutter/dev_test/cli_getter_adapter_test.dart @@ -53,7 +53,6 @@ void main() { final repoDir = _createFixtureRepository(temp, 'official'); final bundle = _createLegacyBundle(temp); final legacyDb = _createLegacyRoomDatabase(temp); - final taskRequest = _createDownloadTaskRequest(temp); final adapter = CliGetterAdapter(executable: getterCli, dataDir: dataDir.path); @@ -71,20 +70,6 @@ void main() { 'import-room-bundle', bundle.path, ]); - _runGetter(getterCli, dataDir.path, [ - 'debug', - 'fake-task', - 'submit', - '--request', - taskRequest.path, - ]); - _runGetter(getterCli, dataDir.path, [ - 'debug', - 'fake-task', - 'run', - 'task-1', - ]); - final repositories = adapter.listRepositories(); expect(repositories.map((repo) => repo.id), contains('official')); expect( @@ -120,26 +105,6 @@ void main() { contains('android/org.fdroid.fdroid'), ); - final tasks = adapter.listDownloadTasks(); - final task = tasks.singleWhere((task) => task.id == 'task-1'); - expect(task.packageId, 'android/org.fdroid.fdroid'); - expect(task.status, 'succeeded'); - expect(task.downloadFileName, 'app.apk'); - expect(task.installHandoffId, 'handoff-1'); - - final eventPage = adapter.listTaskEvents(after: 0, limit: 10); - expect(eventPage.hasMore, isFalse); - expect(eventPage.nextCursor, greaterThanOrEqualTo(4)); - expect( - eventPage.events.map((event) => event.kind), - containsAll([ - 'task_created', - 'task_started', - 'task_succeeded', - 'install_handoff_requested', - ]), - ); - final snapshot = adapter.loadSnapshot(); expect(snapshot.status, 'Getter CLI ready'); expect(snapshot.repositories.map((repo) => repo.id), contains('official')); @@ -229,29 +194,6 @@ conn.close() return db; } -File _createDownloadTaskRequest(Directory temp) { - return File('${temp.path}/download-request.json')..writeAsStringSync(''' -{ - "format": "getter-download-request", - "version": 1, - "package_id": "android/org.fdroid.fdroid", - "executor": "fake", - "actions": [ - { - "type": "download", - "url": "https://example.invalid/app.apk", - "file_name": "app.apk" - }, - { - "type": "install", - "installer": "android_package", - "file": "app.apk" - } - ] -} -'''); -} - void _runGetter(String getterCli, String dataDir, List args) { final result = Process.runSync( getterCli, diff --git a/app_flutter/lib/cli_getter_adapter.dart b/app_flutter/lib/cli_getter_adapter.dart index e0aebd01..8e50d461 100644 --- a/app_flutter/lib/cli_getter_adapter.dart +++ b/app_flutter/lib/cli_getter_adapter.dart @@ -72,29 +72,6 @@ class CliGetterAdapter implements GetterAdapter { return LegacyMigrationImportResult.fromJson(_data(json)); } - /// Development-only accessor for the persisted `debug fake-task` scaffold. - List listDownloadTasks() { - final json = _runGetter(const ['debug', 'fake-task', 'list']); - final tasks = _asList(_data(json)['tasks'], 'tasks'); - return tasks - .map((task) => _downloadTaskFromJson(_asMap(task, 'task'))) - .toList(growable: false); - } - - /// Development-only accessor for the persisted `debug fake-task` scaffold. - TaskEventPage listTaskEvents({required int after, required int limit}) { - final json = _runGetter([ - 'debug', - 'fake-task', - 'events', - '--after', - after.toString(), - '--limit', - limit.toString(), - ]); - return _taskEventPageFromJson(_data(json)); - } - @override Future previewInstalledAutogen({ InstalledAutogenScanOptions options = const InstalledAutogenScanOptions(), @@ -260,58 +237,6 @@ class CliGetterAdapter implements GetterAdapter { } } -class DownloadTaskSummary { - const DownloadTaskSummary({ - required this.id, - required this.packageId, - required this.status, - required this.executor, - required this.actions, - required this.downloadFileName, - required this.downloadedFile, - required this.failureMessage, - required this.installHandoffId, - }); - - final String id; - final String packageId; - final String status; - final String executor; - final List> actions; - final String downloadFileName; - final String? downloadedFile; - final String? failureMessage; - final String? installHandoffId; -} - -class TaskEventPage { - const TaskEventPage({ - required this.events, - required this.nextCursor, - required this.hasMore, - }); - - final List events; - final int nextCursor; - final bool hasMore; -} - -class TaskEventSummary { - const TaskEventSummary({ - required this.cursor, - required this.taskId, - required this.kind, - required this.status, - required this.message, - }); - - final int cursor; - final String taskId; - final String kind; - final String? status; - final String? message; -} - Map _data(Map envelope) { return _asMap(envelope['data'], 'data'); } @@ -362,54 +287,6 @@ PackageEvaluation _packageEvaluationFromJson(Object? value) { ); } -DownloadTaskSummary _downloadTaskFromJson(Map json) { - return DownloadTaskSummary( - id: _asString(json['id'], 'task.id'), - packageId: _asString(json['package_id'], 'task.package_id'), - status: _asString(json['status'], 'task.status'), - executor: _asString(json['executor'], 'task.executor'), - actions: _asList(json['actions'], 'task.actions') - .map((action) => _asMap(action, 'task.action')) - .toList(growable: false), - downloadFileName: _asString( - json['download_file_name'], - 'task.download_file_name', - ), - downloadedFile: _asOptionalString( - json['downloaded_file'], - 'task.downloaded_file', - ), - failureMessage: _asOptionalString( - json['failure_message'], - 'task.failure_message', - ), - installHandoffId: _asOptionalString( - json['install_handoff_id'], - 'task.install_handoff_id', - ), - ); -} - -TaskEventPage _taskEventPageFromJson(Map json) { - return TaskEventPage( - events: _asList(json['events'], 'task.events') - .map((event) => _taskEventFromJson(_asMap(event, 'task.event'))) - .toList(growable: false), - nextCursor: _asInt(json['next_cursor'], 'task.next_cursor'), - hasMore: _asBool(json['has_more'], 'task.has_more'), - ); -} - -TaskEventSummary _taskEventFromJson(Map json) { - return TaskEventSummary( - cursor: _asInt(json['cursor'], 'task.event.cursor'), - taskId: _asString(json['task_id'], 'task.event.task_id'), - kind: _asString(json['kind'], 'task.event.kind'), - status: _asOptionalString(json['status'], 'task.event.status'), - message: _asOptionalString(json['message'], 'task.event.message'), - ); -} - Map _asMap(Object? value, String name) { if (value is Map) { return value; @@ -437,13 +314,6 @@ String _asString(Object? value, String name) { throw FormatException('$name should be a string'); } -String? _asOptionalString(Object? value, String name) { - if (value == null || value is String) { - return value as String?; - } - throw FormatException('$name should be a string or null'); -} - int _asInt(Object? value, String name) { if (value is int) { return value; diff --git a/docs/architecture/adr/0007-flutter-getter-bridge-contract.md b/docs/architecture/adr/0007-flutter-getter-bridge-contract.md index 277cc2df..9c1253eb 100644 --- a/docs/architecture/adr/0007-flutter-getter-bridge-contract.md +++ b/docs/architecture/adr/0007-flutter-getter-bridge-contract.md @@ -30,14 +30,7 @@ readMigrationReports() loadSnapshot() ``` -The second accepted API surface adds read-only task lifecycle DTO consumption for the already accepted offline/fake getter lifecycle: - -```text -listDownloadTasks() -listTaskEvents(after, limit) -``` - -The third accepted API surface adds the first legacy migration action boundary: +The second accepted API surface adds the first legacy migration action boundary: ```text importLegacyRoomDatabase(databasePath) @@ -45,7 +38,7 @@ importLegacyRoomDatabase(databasePath) The Android platform adapter may prepare a copied/checkpointed legacy Room SQLite file and return its path to Flutter, but getter still owns the actual `legacy import-room-db` import semantics. The production Android bridge exposes `importLegacyRoomDatabase` and `legacyReportList` through JNI/MethodChannel by delegating to getter-owned `getter-operations` legacy Room code. Flutter starts the flow and renders getter reports; it must not inspect or map Room tables directly. -The fourth accepted API surface adds the production installed-autogen bridge boundary and must follow ADR-0009's Rust-active platform adapter direction rather than a Flutter-led inventory scan: +The third accepted API surface adds the production installed-autogen bridge boundary and must follow ADR-0009's Rust-active platform adapter direction rather than a Flutter-led inventory scan: ```text previewInstalledAutogen(scanOptions) @@ -56,7 +49,7 @@ The Android product APK packages a slim `:getter_bridge` library under `app_flut Internally, Rust/native bridge code scans Android inventory through the platform adapter, then asks getter-owned shared autogen operations to plan/apply `local_autogen`. `MethodChannelGetterAdapter` consumes the returned getter-style JSON envelopes. Flutter renders getter-owned preview/apply DTOs and scan diagnostics, then passes the package ids from displayed accepted preview candidates back to getter on apply; it must not expose a product Dart `InstalledInventoryPlatform` scanner or convert Android package names into package ids. -`loadSnapshot()` composes the smaller getter-owned operations into the UI shell's first snapshot DTO. It must not perform repository resolution, version comparison, migration mapping, or update selection in Dart. `readMigrationReports()` must go through a getter operation such as `legacy report-list`; Flutter must not inspect getter's data-directory layout directly. `listDownloadTasks()` and `listTaskEvents()` render getter-owned task/event DTOs; Flutter must not synthesize task states, retry policy, installer behavior, or update decisions. +`loadSnapshot()` composes the smaller getter-owned operations into the UI shell's first snapshot DTO. It must not perform repository resolution, version comparison, migration mapping, or update selection in Dart. `readMigrationReports()` must go through a getter operation such as `legacy report-list`; Flutter must not inspect getter's data-directory layout directly. Runtime task rendering must use ADR-0011 runtime task snapshot APIs and opaque action/task controls; Flutter must not synthesize task states, retry policy, installer behavior, or update decisions. ## Flutter DTOs @@ -72,9 +65,16 @@ MigrationReportSummary LegacyMigrationImportResult MigrationWarningSummary MigrationSourceCounts -DownloadTaskSummary -TaskEventPage -TaskEventSummary +RuntimeUpdateCheckResult +RuntimePackageSummary +RuntimeUpdateSummary +RuntimeIssuedAction +RuntimeTaskSnapshot +RuntimeTaskPhase +RuntimeTaskProgress +RuntimeTaskCapabilities +RuntimeTaskDiagnostic +RuntimeNotificationEnvelope GetterError InstalledAutogenPreview InstalledAutogenCandidate @@ -179,15 +179,15 @@ If a feature requires one of these decisions, add or extend a getter operation i Positive: -- The first bridge is executable in CI without waiting for full mobile FFI. -- CLI output remains the headless test oracle. -- Flutter can start consuming real getter data while preserving the Rust-owned domain boundary. -- Future native bridge work has a concrete DTO/error contract to preserve. +- The early bridge was executable in CI before the native bridge stabilized. +- CLI output remains a headless oracle for storage/repository/migration coverage. +- Flutter can consume real getter data while preserving the Rust-owned domain boundary. +- The native bridge now has a concrete DTO/error/runtime notification contract to preserve. Costs: - The CLI adapter is development/test infrastructure, not the final mobile path. -- Snapshot-only UI cannot yet represent long-running update/download/install flows. +- Runtime task UI still exposes only the first read-only snapshot rendering slice until live provider/downloader/installer ADRs are accepted. - Getter output schemas must evolve carefully because they are now a cross-boundary contract. ## Validation @@ -196,13 +196,13 @@ The first implementation slice must provide: - Flutter widget tests that continue to use `FakeGetterAdapter`. - Flutter widget tests for the migration flow using fake platform/getter adapters. -- A Flutter/Dart integration test that builds or receives a real `getter-cli` binary, initializes a real getter data directory, and reads repositories, tracked packages, package evaluation output, migration reports, direct Room import output, and task lifecycle DTOs through `CliGetterAdapter`. +- A Flutter/Dart integration test that builds or receives a real `getter-cli` binary, initializes a real getter data directory, and reads repositories, tracked packages, package evaluation output, migration reports, and direct Room import output through `CliGetterAdapter`. - `just verify` coverage for the bridge integration test. ## Non-goals -- No full FFI/native bridge implementation beyond the first direct legacy Room import/report-list and installed-autogen preview/apply JNI/MethodChannel operation slices. -- No update/download/install event stream. +- No product-complete live provider/downloader/installer execution beyond the ADR-0011 in-memory runtime operation and notification skeleton. +- No durable update/download/install event log or cross-process task recovery. - No Android-owned legacy Room mapping/import semantics; Android only prepares a copied DB file for getter. - No product-complete Flutter UI. - No product/domain decisions in Dart. diff --git a/todo.md b/todo.md index cd3e85d3..ab642a2e 100644 --- a/todo.md +++ b/todo.md @@ -506,14 +506,14 @@ Completed additional UI/bridge slice: 13. Add native bridge process-lifetime runtime singleton, runtime operation dispatcher, bounded best-effort notification drain, Kotlin EventChannel, and Dart runtime notification stream primitive. 14. Add typed Dart runtime/update methods for package update-check action issuance, action-id submission, task query/control/user-result/remove/clean, typed runtime notifications, and runtime task snapshot rendering on the Downloads route. 15. Add `runtime script --script ` as a single-process CLI debug harness over `getter-operations::runtime`, including task remove/clean coverage, and move the old persisted fake downloader scaffold out of the public `task` namespace to `debug fake-task ...`. +16. Remove remaining `debug fake-task` DTO accessors from `CliGetterAdapter` and keep fake-task coverage inside getter CLI BDD/dev scaffolding only, so Flutter product/development adapters no longer expose the old persisted task/event DTO surface. Remaining tasks: 1. Replace the current static `updates` package seam with live provider update-check action issuance that materializes sealed action plans and returns opaque `action_id` to Flutter; keep Dart from constructing action payloads. -2. Continue retiring old fake-task scaffolding from product-facing adapters/docs as newer runtime/native flows cover those cases; the remaining `debug fake-task ...` commands are development-only. -3. Implement live provider/downloader behavior beyond the fake/offline proof after a later ADR accepts real side-effect details. -4. Define Android production install handoff URI/SAF/permission/notification details and wire platform adapter execution after later ADRs. -5. Add product-level Flutter BDD for full update/download/install user flows after live/provider/background/installer decisions are accepted; the current slice covers typed runtime DTO methods and read-only runtime task snapshot rendering. +2. Implement live provider/downloader behavior beyond the fake/offline proof after a later ADR accepts real side-effect details. +3. Define Android production install handoff URI/SAF/permission/notification details and wire platform adapter execution after later ADRs. +4. Add product-level Flutter BDD for full update/download/install user flows after live/provider/background/installer decisions are accepted; the current slice covers typed runtime DTO methods and read-only runtime task snapshot rendering. Acceptance progress: From 7bcaa0b9ddca4a9ea817815074ba25ee61cee44f Mon Sep 17 00:00:00 2001 From: Xiangzhe Date: Thu, 25 Jun 2026 15:43:05 +0800 Subject: [PATCH 44/85] ci: install multilib for Android LuaJIT builds --- .github/workflows/android.yml | 5 +++++ .github/workflows/upgradeall-rewrite-validation.yml | 5 +++++ 2 files changed, 10 insertions(+) diff --git a/.github/workflows/android.yml b/.github/workflows/android.yml index a712b6f7..37ee7141 100644 --- a/.github/workflows/android.yml +++ b/.github/workflows/android.yml @@ -27,6 +27,11 @@ jobs: distribution: 'temurin' java-version: 21 + - name: Install Linux host build dependencies + run: | + sudo apt-get update + sudo apt-get install -y gcc-multilib g++-multilib + - name: Setup Android SDK uses: android-actions/setup-android@v3 diff --git a/.github/workflows/upgradeall-rewrite-validation.yml b/.github/workflows/upgradeall-rewrite-validation.yml index 288b81d7..6469b016 100644 --- a/.github/workflows/upgradeall-rewrite-validation.yml +++ b/.github/workflows/upgradeall-rewrite-validation.yml @@ -26,6 +26,11 @@ jobs: distribution: temurin java-version: 21 + - name: Install Linux host build dependencies + run: | + sudo apt-get update + sudo apt-get install -y gcc-multilib g++-multilib + - name: Set up Android SDK uses: android-actions/setup-android@v3 From 4e6274aa19e4caed7eeb2b3319115eb6e3c1af0b Mon Sep 17 00:00:00 2001 From: Xiangzhe Date: Thu, 25 Jun 2026 17:52:41 +0800 Subject: [PATCH 45/85] chore(app): update Flutter toolchain baseline --- app_flutter/README.md | 9 ++ app_flutter/android/app/build.gradle | 3 +- app_flutter/android/build.gradle | 2 +- .../android/getter_bridge/build.gradle | 2 +- app_flutter/android/gradle.properties | 5 + .../gradle/wrapper/gradle-wrapper.properties | 2 +- app_flutter/android/settings.gradle | 25 +++- .../lib/legacy_migration_platform.dart | 3 + app_flutter/lib/native_getter_adapter.dart | 56 ++++--- app_flutter/pubspec.lock | 137 ++++++++++-------- app_flutter/pubspec.yaml | 5 +- docs/README.md | 9 ++ .../flutter-ui-feature-parity-and-testing.md | 4 + gradle/libs.versions.toml | 2 +- 14 files changed, 163 insertions(+), 101 deletions(-) diff --git a/app_flutter/README.md b/app_flutter/README.md index 109a046a..3460eceb 100644 --- a/app_flutter/README.md +++ b/app_flutter/README.md @@ -2,6 +2,15 @@ This is the new Flutter shell and product APK entry for the UpgradeAll rewrite. It must remain a UI and platform adapter around the Rust getter core; product logic, repository resolution, storage, and migration behavior belong in getter. The legacy Android `:app` UI is kept only as reference code during migration. +## Toolchain baseline + +- Flutter stable `>=3.44.4` +- Dart SDK `>=3.12.2 <4.0.0` +- Gradle `9.3.1`, Android Gradle Plugin `9.0.1`, Kotlin Gradle Plugin `2.3.20` +- Android product APK `minSdkVersion` follows `flutter.minSdkVersion` from the active stable Flutter SDK (Flutter 3.44 currently uses Android API 24). + +Do not validate the rewrite with an older local Flutter SDK; older Flutter tester/Impeller builds can crash in widget tests and do not match CI. Do not pin the Flutter product APK to an Android API level below the active stable Flutter SDK baseline just to preserve old local compatibility. + ## Current slice - Android release application identity: `net.xzos.upgradeall` diff --git a/app_flutter/android/app/build.gradle b/app_flutter/android/app/build.gradle index 880b5c76..4fce1811 100644 --- a/app_flutter/android/app/build.gradle +++ b/app_flutter/android/app/build.gradle @@ -1,6 +1,5 @@ plugins { id "com.android.application" - id "kotlin-android" id "dev.flutter.flutter-gradle-plugin" } @@ -50,7 +49,7 @@ android { defaultConfig { applicationId "net.xzos.upgradeall" - minSdkVersion 23 + minSdkVersion flutter.minSdkVersion targetSdkVersion 36 versionCode flutterVersionCode.toInteger() versionName flutterVersionName diff --git a/app_flutter/android/build.gradle b/app_flutter/android/build.gradle index 3eeea9b3..775ef008 100644 --- a/app_flutter/android/build.gradle +++ b/app_flutter/android/build.gradle @@ -1,7 +1,7 @@ import groovy.json.JsonSlurper buildscript { - ext.kotlin_version = '2.0.0' + ext.kotlin_version = '2.3.20' repositories { google() mavenCentral() diff --git a/app_flutter/android/getter_bridge/build.gradle b/app_flutter/android/getter_bridge/build.gradle index 594c2d47..631a3699 100644 --- a/app_flutter/android/getter_bridge/build.gradle +++ b/app_flutter/android/getter_bridge/build.gradle @@ -1,6 +1,6 @@ plugins { id "com.android.library" - id "kotlin-android" + id "org.jetbrains.kotlin.android" id "io.github.MatrixDev.android-rust" } diff --git a/app_flutter/android/gradle.properties b/app_flutter/android/gradle.properties index 5d46147a..f8a16ee9 100644 --- a/app_flutter/android/gradle.properties +++ b/app_flutter/android/gradle.properties @@ -1,4 +1,9 @@ org.gradle.jvmargs=-Xmx4G android.useAndroidX=true android.enableJetifier=true +# Flutter's Gradle plugin still expects AGP's legacy Android extension while +# Flutter 3.44 templates use AGP 9, so keep the template compatibility flags +# until Flutter removes them upstream. +android.newDsl=false +android.builtInKotlin=false android.suppressUnsupportedCompileSdk=36 diff --git a/app_flutter/android/gradle/wrapper/gradle-wrapper.properties b/app_flutter/android/gradle/wrapper/gradle-wrapper.properties index 45181329..bdc0141f 100644 --- a/app_flutter/android/gradle/wrapper/gradle-wrapper.properties +++ b/app_flutter/android/gradle/wrapper/gradle-wrapper.properties @@ -2,4 +2,4 @@ distributionBase=GRADLE_USER_HOME distributionPath=wrapper/dists zipStoreBase=GRADLE_USER_HOME zipStorePath=wrapper/dists -distributionUrl=https\://services.gradle.org/distributions/gradle-8.7-bin.zip +distributionUrl=https\://services.gradle.org/distributions/gradle-9.3.1-bin.zip diff --git a/app_flutter/android/settings.gradle b/app_flutter/android/settings.gradle index a79f894b..8d03a57a 100644 --- a/app_flutter/android/settings.gradle +++ b/app_flutter/android/settings.gradle @@ -8,6 +8,26 @@ pluginManagement { } settings.ext.flutterSdkPath = flutterSdkPath() + def ensureCargoBinPath = { + def localPropertiesFile = file("local.properties") + def properties = new Properties() + if (localPropertiesFile.exists()) { + localPropertiesFile.withInputStream { properties.load(it) } + } + def configuredCargoBin = properties.getProperty("cargo.bin") + if (configuredCargoBin != null && !configuredCargoBin.isBlank()) { + return + } + def pathEntries = (System.getenv("PATH") ?: "").split(File.pathSeparator) + def cargoFile = pathEntries.collect { new File(it, "cargo") }.find { it.isFile() && it.canExecute() } + if (cargoFile == null) { + return + } + properties.setProperty("cargo.bin", cargoFile.parentFile.absolutePath) + localPropertiesFile.withOutputStream { properties.store(it, "Generated by Gradle so android-rust can find Cargo binaries") } + } + ensureCargoBinPath() + includeBuild("${settings.ext.flutterSdkPath}/packages/flutter_tools/gradle") repositories { @@ -18,14 +38,15 @@ pluginManagement { plugins { id "dev.flutter.flutter-gradle-plugin" version "1.0.0" apply false - id "com.android.library" version "8.6.0" apply false + id "com.android.library" version "9.0.1" apply false + id "org.jetbrains.kotlin.android" version "2.3.20" apply false id "io.github.MatrixDev.android-rust" version "0.6.0" apply false } } plugins { id "dev.flutter.flutter-plugin-loader" version "1.0.0" - id "com.android.application" version "8.6.0" apply false + id "com.android.application" version "9.0.1" apply false } include ":app" diff --git a/app_flutter/lib/legacy_migration_platform.dart b/app_flutter/lib/legacy_migration_platform.dart index 9b1a70a4..f6535289 100644 --- a/app_flutter/lib/legacy_migration_platform.dart +++ b/app_flutter/lib/legacy_migration_platform.dart @@ -1,3 +1,5 @@ +// ignore_for_file: prefer_initializing_formals + import 'package:flutter/services.dart'; import 'getter_adapter.dart'; @@ -24,6 +26,7 @@ class LegacyRoomImportCandidate { } class MethodChannelLegacyMigrationPlatform implements LegacyMigrationPlatform { + // Keep the public `channel` parameter name for tests/callers. const MethodChannelLegacyMigrationPlatform({ MethodChannel channel = const MethodChannel( 'net.xzos.upgradeall/legacy_migration', diff --git a/app_flutter/lib/native_getter_adapter.dart b/app_flutter/lib/native_getter_adapter.dart index 9fa0f9ff..0d791ffd 100644 --- a/app_flutter/lib/native_getter_adapter.dart +++ b/app_flutter/lib/native_getter_adapter.dart @@ -1,3 +1,5 @@ +// ignore_for_file: prefer_initializing_formals + import 'dart:convert'; import 'package:flutter/services.dart'; @@ -12,6 +14,7 @@ import 'getter_adapter.dart'; /// The bridge returns getter-owned JSON envelopes; Dart parses and renders them /// but does not scan PackageManager or make autogen/package decisions. class MethodChannelGetterAdapter extends FakeGetterAdapter { + // Keep public parameter names stable for tests and injected bridges. const MethodChannelGetterAdapter({ MethodChannel channel = const MethodChannel( 'net.xzos.upgradeall/getter_bridge', @@ -19,8 +22,8 @@ class MethodChannelGetterAdapter extends FakeGetterAdapter { EventChannel runtimeNotificationChannel = const EventChannel( 'net.xzos.upgradeall/runtime_notifications', ), - }) : _channel = channel, - _runtimeNotificationChannel = runtimeNotificationChannel; + }) : _channel = channel, + _runtimeNotificationChannel = runtimeNotificationChannel; final MethodChannel _channel; final EventChannel _runtimeNotificationChannel; @@ -44,9 +47,10 @@ class MethodChannelGetterAdapter extends FakeGetterAdapter { ); final reports = _asList(data['reports'], 'legacy reports'); return reports - .map((report) => MigrationReportSummary.fromJson( - _asMap(report, 'legacy report'), - )) + .map( + (report) => + MigrationReportSummary.fromJson(_asMap(report, 'legacy report')), + ) .toList(growable: false); } @@ -114,13 +118,10 @@ class MethodChannelGetterAdapter extends FakeGetterAdapter { String operation, { Map payload = const {}, }) { - return _invokeGetterData( - 'runtimeOperation', - { - 'operation': operation, - 'payload': payload, - }, - ); + return _invokeGetterData('runtimeOperation', { + 'operation': operation, + 'payload': payload, + }); } @override @@ -132,9 +133,9 @@ class MethodChannelGetterAdapter extends FakeGetterAdapter { }) async { final payload = { 'package_id': packageId, - if (repositoryId != null) 'repository_id': repositoryId, - if (installedVersion != null) 'installed_version': installedVersion, - if (pinVersion != null) 'pin_version': pinVersion, + 'repository_id': ?repositoryId, + 'installed_version': ?installedVersion, + 'pin_version': ?pinVersion, }; final data = await invokeRuntimeOperation( 'update_check_package_issue_action', @@ -145,10 +146,9 @@ class MethodChannelGetterAdapter extends FakeGetterAdapter { @override Future submitRuntimeAction(String actionId) { - return _runtimeTaskOperation( - 'task_submit', - {'action_id': actionId}, - ); + return _runtimeTaskOperation('task_submit', { + 'action_id': actionId, + }); } @override @@ -158,10 +158,7 @@ class MethodChannelGetterAdapter extends FakeGetterAdapter { }) async { final data = await invokeRuntimeOperation( 'task_list', - payload: { - 'active': active, - if (packageId != null) 'package_id': packageId, - }, + payload: {'active': active, 'package_id': ?packageId}, ); return _runtimeTasksFromData(data); } @@ -207,14 +204,11 @@ class MethodChannelGetterAdapter extends FakeGetterAdapter { RuntimeUserResult result, { String? reason, }) { - return _runtimeTaskOperation( - 'task_user_result', - { - 'task_id': taskId, - 'result': result.wireName, - if (reason != null) 'reason': reason, - }, - ); + return _runtimeTaskOperation('task_user_result', { + 'task_id': taskId, + 'result': result.wireName, + 'reason': ?reason, + }); } @override diff --git a/app_flutter/pubspec.lock b/app_flutter/pubspec.lock index 81104f36..1f6541fc 100644 --- a/app_flutter/pubspec.lock +++ b/app_flutter/pubspec.lock @@ -5,66 +5,66 @@ packages: dependency: transitive description: name: async - sha256: "947bfcf187f74dbc5e146c9eb9c0f10c9f8b30743e341481c1e2ed3ecc18c20c" + sha256: e2eb0491ba5ddb6177742d2da23904574082139b07c1e33b8503b9f46f3e1a37 url: "https://pub.dev" source: hosted - version: "2.11.0" + version: "2.13.1" boolean_selector: dependency: transitive description: name: boolean_selector - sha256: "6cfb5af12253eaf2b368f07bacc5a80d1301a071c73360d746b7f2e32d762c66" + sha256: "8aab1771e1243a5063b8b0ff68042d67334e3feab9e95b9490f9a6ebf73b42ea" url: "https://pub.dev" source: hosted - version: "2.1.1" + version: "2.1.2" characters: dependency: transitive description: name: characters - sha256: "04a925763edad70e8443c99234dc3328f442e811f1d8fd1a72f1c8ad0f69a605" + sha256: faf38497bda5ead2a8c7615f4f7939df04333478bf32e4173fcb06d428b5716b url: "https://pub.dev" source: hosted - version: "1.3.0" + version: "1.4.1" clock: dependency: transitive description: name: clock - sha256: cb6d7f03e1de671e34607e909a7213e31d7752be4fb66a86d29fe1eb14bfb5cf + sha256: fddb70d9b5277016c77a80201021d40a2247104d9f4aa7bab7157b7e3f05b84b url: "https://pub.dev" source: hosted - version: "1.1.1" + version: "1.1.2" collection: dependency: transitive description: name: collection - sha256: ee67cb0715911d28db6bf4af1026078bd6f0128b07a5f66fb2ed94ec6783c09a + sha256: "2f5709ae4d3d59dd8f7cd309b4e023046b57d8a6c82130785d2b0e5868084e76" url: "https://pub.dev" source: hosted - version: "1.18.0" + version: "1.19.1" cupertino_icons: dependency: "direct main" description: name: cupertino_icons - sha256: ba631d1c7f7bef6b729a622b7b752645a2d076dba9976925b8f25725a30e1ee6 + sha256: "41e005c33bd814be4d3096aff55b1908d419fde52ca656c8c47719ec745873cd" url: "https://pub.dev" source: hosted - version: "1.0.8" + version: "1.0.9" fake_async: dependency: transitive description: name: fake_async - sha256: "511392330127add0b769b75a987850d136345d9227c6b94c96a04cf4a391bf78" + sha256: "5368f224a74523e8d2e7399ea1638b37aecfca824a3cc4dfdf77bf1fa905ac44" url: "https://pub.dev" source: hosted - version: "1.3.1" + version: "1.3.3" file: dependency: transitive description: name: file - sha256: "1b92bec4fc2a72f59a8e15af5f52cd441e4a7860b49499d69dfa817af20e925d" + sha256: a3b4f84adafef897088c160faf7dfffb7696046cb13ae90b508c2cbc95d3b8d4 url: "https://pub.dev" source: hosted - version: "6.1.4" + version: "7.0.1" flutter: dependency: "direct main" description: flutter @@ -79,10 +79,10 @@ packages: dependency: "direct dev" description: name: flutter_lints - sha256: a25a15ebbdfc33ab1cd26c63a6ee519df92338a9c10f122adda92938253bef04 + sha256: "3105dc8492f6183fb076ccf1f351ac3d60564bff92e20bfc4af9cc1651f4e7e1" url: "https://pub.dev" source: hosted - version: "2.0.3" + version: "6.0.0" flutter_test: dependency: "direct dev" description: flutter @@ -98,99 +98,123 @@ packages: description: flutter source: sdk version: "0.0.0" + leak_tracker: + dependency: transitive + description: + name: leak_tracker + sha256: "33e2e26bdd85a0112ec15400c8cbffea70d0f9c3407491f672a2fad47915e2de" + url: "https://pub.dev" + source: hosted + version: "11.0.2" + leak_tracker_flutter_testing: + dependency: transitive + description: + name: leak_tracker_flutter_testing + sha256: "1dbc140bb5a23c75ea9c4811222756104fbcd1a27173f0c34ca01e16bea473c1" + url: "https://pub.dev" + source: hosted + version: "3.0.10" + leak_tracker_testing: + dependency: transitive + description: + name: leak_tracker_testing + sha256: "8d5a2d49f4a66b49744b23b018848400d23e54caf9463f4eb20df3eb8acb2eb1" + url: "https://pub.dev" + source: hosted + version: "3.0.2" lints: dependency: transitive description: name: lints - sha256: "0a217c6c989d21039f1498c3ed9f3ed71b354e69873f13a8dfc3c9fe76f1b452" + sha256: "12f842a479589fea194fe5c5a3095abc7be0c1f2ddfa9a0e76aed1dbd26a87df" url: "https://pub.dev" source: hosted - version: "2.1.1" + version: "6.1.0" matcher: dependency: transitive description: name: matcher - sha256: "1803e76e6653768d64ed8ff2e1e67bea3ad4b923eb5c56a295c3e634bad5960e" + sha256: dc0b7dc7651697ea4ff3e69ef44b0407ea32c487a39fff6a4004fa585e901861 url: "https://pub.dev" source: hosted - version: "0.12.16" + version: "0.12.19" material_color_utilities: dependency: transitive description: name: material_color_utilities - sha256: "9528f2f296073ff54cb9fee677df673ace1218163c3bc7628093e7eed5203d41" + sha256: "9c337007e82b1889149c82ed242ed1cb24a66044e30979c44912381e9be4c48b" url: "https://pub.dev" source: hosted - version: "0.5.0" + version: "0.13.0" meta: dependency: transitive description: name: meta - sha256: a6e590c838b18133bb482a2745ad77c5bb7715fb0451209e1a7567d416678b8e + sha256: "1741988757a65eb6b36abe716829688cf01910bbf91c34354ff7ec1c3de2b349" url: "https://pub.dev" source: hosted - version: "1.10.0" + version: "1.18.0" path: dependency: transitive description: name: path - sha256: "8829d8a55c13fc0e37127c29fedf290c102f4e40ae94ada574091fe0ff96c917" + sha256: "75cca69d1490965be98c73ceaea117e8a04dd21217b37b292c9ddbec0d955bc5" url: "https://pub.dev" source: hosted - version: "1.8.3" + version: "1.9.1" platform: dependency: transitive description: name: platform - sha256: ae68c7bfcd7383af3629daafb32fb4e8681c7154428da4febcff06200585f102 + sha256: "5d6b1b0036a5f331ebc77c850ebc8506cbc1e9416c27e59b439f917a902a4984" url: "https://pub.dev" source: hosted - version: "3.1.2" + version: "3.1.6" process: dependency: transitive description: name: process - sha256: "53fd8db9cec1d37b0574e12f07520d582019cb6c44abf5479a01505099a34a09" + sha256: c6248e4526673988586e8c00bb22a49210c258dc91df5227d5da9748ecf79744 url: "https://pub.dev" source: hosted - version: "4.2.4" + version: "5.0.5" sky_engine: dependency: transitive description: flutter source: sdk - version: "0.0.99" + version: "0.0.0" source_span: dependency: transitive description: name: source_span - sha256: "53e943d4206a5e30df338fd4c6e7a077e02254531b138a15aec3bd143c1a8b3c" + sha256: "56a02f1f4cd1a2d96303c0144c93bd6d909eea6bee6bf5a0e0b685edbd4c47ab" url: "https://pub.dev" source: hosted - version: "1.10.0" + version: "1.10.2" stack_trace: dependency: transitive description: name: stack_trace - sha256: "73713990125a6d93122541237550ee3352a2d84baad52d375a4cad2eb9b7ce0b" + sha256: "8b27215b45d22309b5cddda1aa2b19bdfec9df0e765f2de506401c071d38d1b1" url: "https://pub.dev" source: hosted - version: "1.11.1" + version: "1.12.1" stream_channel: dependency: transitive description: name: stream_channel - sha256: ba2aa5d8cc609d96bbb2899c28934f9e1af5cddbd60a827822ea467161eb54e7 + sha256: "969e04c80b8bcdf826f8f16579c7b14d780458bd97f56d107d3950fdbeef059d" url: "https://pub.dev" source: hosted - version: "2.1.2" + version: "2.1.4" string_scanner: dependency: transitive description: name: string_scanner - sha256: "556692adab6cfa87322a115640c11f13cb77b3f076ddcc5d6ae3c20242bedcde" + sha256: "921cd31725b72fe181906c6a94d987c78e3b98c2e205b397ea399d4054872b43" url: "https://pub.dev" source: hosted - version: "1.2.0" + version: "1.4.1" sync_http: dependency: transitive description: @@ -203,49 +227,42 @@ packages: dependency: transitive description: name: term_glyph - sha256: a29248a84fbb7c79282b40b8c72a1209db169a2e0542bce341da992fe1bc7e84 + sha256: "7f554798625ea768a7518313e58f83891c7f5024f88e46e7182a4558850a4b8e" url: "https://pub.dev" source: hosted - version: "1.2.1" + version: "1.2.2" test_api: dependency: transitive description: name: test_api - sha256: "5c2f730018264d276c20e4f1503fd1308dfbbae39ec8ee63c5236311ac06954b" + sha256: "949a932224383300f01be9221c39180316445ecb8e7547f70a41a35bf421fb9e" url: "https://pub.dev" source: hosted - version: "0.6.1" + version: "0.7.11" vector_math: dependency: transitive description: name: vector_math - sha256: "80b3257d1492ce4d091729e3a67a60407d227c27241d6927be0130c98e741803" + sha256: d530bd74fea330e6e364cda7a85019c434070188383e1cd8d9777ee586914c5b url: "https://pub.dev" source: hosted - version: "2.1.4" + version: "2.2.0" vm_service: dependency: transitive description: name: vm_service - sha256: c538be99af830f478718b51630ec1b6bee5e74e52c8a802d328d9e71d35d2583 - url: "https://pub.dev" - source: hosted - version: "11.10.0" - web: - dependency: transitive - description: - name: web - sha256: afe077240a270dcfd2aafe77602b4113645af95d0ad31128cc02bce5ac5d5152 + sha256: "0016aef94fc66495ac78af5859181e3f3bf2026bd8eecc72b9565601e19ab360" url: "https://pub.dev" source: hosted - version: "0.3.0" + version: "15.2.0" webdriver: dependency: transitive description: name: webdriver - sha256: "3c923e918918feeb90c4c9fdf1fe39220fa4c0e8e2c0fffaded174498ef86c49" + sha256: "2f3a14ca026957870cfd9c635b83507e0e51d8091568e90129fbf805aba7cade" url: "https://pub.dev" source: hosted - version: "3.0.2" + version: "3.1.0" sdks: - dart: ">=3.2.3 <4.0.0" + dart: ">=3.12.2 <4.0.0" + flutter: ">=3.44.4" diff --git a/app_flutter/pubspec.yaml b/app_flutter/pubspec.yaml index 95bb0810..710ac251 100644 --- a/app_flutter/pubspec.yaml +++ b/app_flutter/pubspec.yaml @@ -19,7 +19,8 @@ publish_to: 'none' # Remove this line if you wish to publish to pub.dev version: 0.20.0-alpha.4+105 environment: - sdk: '>=3.2.3 <4.0.0' + sdk: '>=3.12.2 <4.0.0' + flutter: '>=3.44.4' # Dependencies specify other packages that your package needs in order to work. # To automatically upgrade your package dependencies to the latest versions @@ -47,7 +48,7 @@ dev_dependencies: # activated in the `analysis_options.yaml` file located at the root of your # package. See that file for information about deactivating specific lint # rules and activating additional ones. - flutter_lints: ^2.0.0 + flutter_lints: ^6.0.0 # For information on the generic Dart part of this file, see the # following page: https://dart.dev/tools/pub/pubspec diff --git a/docs/README.md b/docs/README.md index c46ba306..23598262 100644 --- a/docs/README.md +++ b/docs/README.md @@ -6,6 +6,15 @@ This documentation set records the design decisions for the UpgradeAll rewrite. It exists so coding agents and human maintainers can trace every major implementation choice back to a written decision. +## Toolchain baseline + +The rewrite should be validated on current stable toolchains, not old local SDKs: + +- Flutter stable `>=3.44.4` with Dart `>=3.12.2 <4.0.0`. +- Rust stable; latest local validated baseline is `rustc 1.96.0` / `cargo 1.96.0`. +- Android Gradle Plugin `9.0.1`, Gradle `9.3.1`, Kotlin Gradle Plugin `2.3.20`. +- Android product APK `minSdkVersion` follows the active stable Flutter SDK's `flutter.minSdkVersion`. + Start here: 1. `architecture/upgradeall-getter-rewrite-wiki.md` — main living architecture wiki. diff --git a/docs/app/flutter-ui-feature-parity-and-testing.md b/docs/app/flutter-ui-feature-parity-and-testing.md index e3e91c32..9a5b4a6e 100644 --- a/docs/app/flutter-ui-feature-parity-and-testing.md +++ b/docs/app/flutter-ui-feature-parity-and-testing.md @@ -4,6 +4,10 @@ > Date: 2026-06-21 > Project: UpgradeAll rewrite — Flutter APP + Rust getter core + Lua package repository model +## Toolchain baseline + +The rewrite's Flutter UI/test baseline is Flutter stable `>=3.44.4` with Dart `>=3.12.2 <4.0.0`. The Android build baseline is Gradle `9.3.1`, Android Gradle Plugin `9.0.1`, and Kotlin Gradle Plugin `2.3.20`. Local validation should use the same current-stable Flutter generation as CI; older Flutter tester/Impeller builds are not an acceptable validation baseline for this rewrite. The Flutter product APK's Android `minSdkVersion` follows the active stable Flutter SDK's `flutter.minSdkVersion` (Flutter 3.44 currently uses API 24), rather than pinning an older product APK baseline below Flutter's supported default. + ## UI feature parity The Flutter UI should preserve these user-visible product capabilities unless explicitly deferred: diff --git a/gradle/libs.versions.toml b/gradle/libs.versions.toml index 292f52a2..19565b05 100644 --- a/gradle/libs.versions.toml +++ b/gradle/libs.versions.toml @@ -1,7 +1,7 @@ [versions] # Build Tools agp = "9.0.1" -kotlin = "2.3.10" +kotlin = "2.3.20" ksp = "2.3.5" androidRust = "0.6.0" From 25af79c09d61367410c1343138aea7f5ef0e43b5 Mon Sep 17 00:00:00 2001 From: Xiangzhe Date: Thu, 25 Jun 2026 18:16:32 +0800 Subject: [PATCH 46/85] feat(app): load snapshots from native getter read model --- .../upgradeall/GetterBridgeRequestBuilder.kt | 6 +- .../net/xzos/upgradeall/MainActivity.kt | 12 + .../GetterBridgeRequestBuilderTest.kt | 18 + .../net/xzos/upgradeall/getter/NativeLib.kt | 1 + .../dev_test/cli_getter_adapter_test.dart | 199 +++--- app_flutter/lib/cli_getter_adapter.dart | 54 +- app_flutter/lib/getter_adapter.dart | 225 +++--- app_flutter/lib/main.dart | 363 ++++++---- app_flutter/lib/native_getter_adapter.dart | 114 ++- .../test/native_getter_adapter_test.dart | 662 ++++++++++-------- app_flutter/test/widget_test.dart | 89 ++- .../net/xzos/upgradeall/getter/NativeLib.kt | 1 + .../src/main/rust/api_proxy/src/lib.rs | 87 +++ core-getter/src/main/rust/getter | 2 +- .../0007-flutter-getter-bridge-contract.md | 2 +- 15 files changed, 1155 insertions(+), 680 deletions(-) diff --git a/app_flutter/android/app/src/main/kotlin/net/xzos/upgradeall/GetterBridgeRequestBuilder.kt b/app_flutter/android/app/src/main/kotlin/net/xzos/upgradeall/GetterBridgeRequestBuilder.kt index 45a3eaa2..56e0716e 100644 --- a/app_flutter/android/app/src/main/kotlin/net/xzos/upgradeall/GetterBridgeRequestBuilder.kt +++ b/app_flutter/android/app/src/main/kotlin/net/xzos/upgradeall/GetterBridgeRequestBuilder.kt @@ -3,7 +3,11 @@ package net.xzos.upgradeall import org.json.JSONObject object GetterBridgeRequestBuilder { - fun runtimeOperationRequest(args: Map<*, *>): String { + fun readOperationRequest(args: Map<*, *>): String = operationRequest(args) + + fun runtimeOperationRequest(args: Map<*, *>): String = operationRequest(args) + + private fun operationRequest(args: Map<*, *>): String { val operation = args["operation"] as? String ?: throw IllegalArgumentException("operation is required") val payload = args["payload"] as? Map<*, *> ?: emptyMap() diff --git a/app_flutter/android/app/src/main/kotlin/net/xzos/upgradeall/MainActivity.kt b/app_flutter/android/app/src/main/kotlin/net/xzos/upgradeall/MainActivity.kt index e5520540..21872479 100644 --- a/app_flutter/android/app/src/main/kotlin/net/xzos/upgradeall/MainActivity.kt +++ b/app_flutter/android/app/src/main/kotlin/net/xzos/upgradeall/MainActivity.kt @@ -67,6 +67,10 @@ class MainActivity : FlutterActivity() { nativeLib.legacyReportList(legacyReportListRequest()) } + "readOperation" -> runGetterBridge(result) { + nativeLib.readOperation(readOperationRequest(call)) + } + "runtimeOperation" -> runGetterBridge(result, emitRuntimeNotifications = true) { nativeLib.runtimeOperation(runtimeOperationRequest(call)) } @@ -203,6 +207,14 @@ class MainActivity : FlutterActivity() { .toString() } + private fun readOperationRequest(call: MethodCall): String { + val args = call.arguments as? Map<*, *> + ?: throw IllegalArgumentException("read operation arguments are required") + return JSONObject(GetterBridgeRequestBuilder.readOperationRequest(args)) + .put("data_dir", getterDataDir().absolutePath) + .toString() + } + private fun runtimeOperationRequest(call: MethodCall): String { val args = call.arguments as? Map<*, *> ?: throw IllegalArgumentException("runtime operation arguments are required") diff --git a/app_flutter/android/app/src/test/kotlin/net/xzos/upgradeall/GetterBridgeRequestBuilderTest.kt b/app_flutter/android/app/src/test/kotlin/net/xzos/upgradeall/GetterBridgeRequestBuilderTest.kt index 67ccba2b..37917868 100644 --- a/app_flutter/android/app/src/test/kotlin/net/xzos/upgradeall/GetterBridgeRequestBuilderTest.kt +++ b/app_flutter/android/app/src/test/kotlin/net/xzos/upgradeall/GetterBridgeRequestBuilderTest.kt @@ -6,6 +6,24 @@ import org.junit.Assert.assertThrows import org.junit.Test class GetterBridgeRequestBuilderTest { + @Test + fun readOperationRequestPreservesOperationAndPayload() { + val json = JSONObject( + GetterBridgeRequestBuilder.readOperationRequest( + mapOf( + "operation" to "package_eval", + "payload" to mapOf("package_id" to "android/org.fdroid.fdroid"), + ), + ), + ) + + assertEquals("package_eval", json.getString("operation")) + assertEquals( + "android/org.fdroid.fdroid", + json.getJSONObject("payload").getString("package_id"), + ) + } + @Test fun runtimeOperationRequestPreservesOperationAndPayload() { val json = JSONObject( diff --git a/app_flutter/android/getter_bridge/src/main/kotlin/net/xzos/upgradeall/getter/NativeLib.kt b/app_flutter/android/getter_bridge/src/main/kotlin/net/xzos/upgradeall/getter/NativeLib.kt index f0165f8c..e5b2ec14 100644 --- a/app_flutter/android/getter_bridge/src/main/kotlin/net/xzos/upgradeall/getter/NativeLib.kt +++ b/app_flutter/android/getter_bridge/src/main/kotlin/net/xzos/upgradeall/getter/NativeLib.kt @@ -15,6 +15,7 @@ class NativeLib { external fun applyInstalledAutogen(requestJson: String): String external fun importLegacyRoomDatabase(requestJson: String): String external fun legacyReportList(requestJson: String): String + external fun readOperation(requestJson: String): String external fun runtimeOperation(requestJson: String): String external fun drainRuntimeNotifications(): String diff --git a/app_flutter/dev_test/cli_getter_adapter_test.dart b/app_flutter/dev_test/cli_getter_adapter_test.dart index ddb60c0f..0525279f 100644 --- a/app_flutter/dev_test/cli_getter_adapter_test.dart +++ b/app_flutter/dev_test/cli_getter_adapter_test.dart @@ -15,8 +15,10 @@ void main() { final dataDir = Directory('${temp.path}/data')..createSync(); final legacyDb = _createLegacyRoomDatabase(temp); - final adapter = - CliGetterAdapter(executable: getterCli, dataDir: dataDir.path); + final adapter = CliGetterAdapter( + executable: getterCli, + dataDir: dataDir.path, + ); adapter.initialize(); final result = await adapter.importLegacyRoomDatabase(legacyDb.path); @@ -39,82 +41,95 @@ void main() { ); }); - test('CliGetterAdapter reads real getter repository and tracked state', - () async { - final getterCli = Platform.environment['GETTER_CLI_BIN']; - if (getterCli == null || getterCli.isEmpty) { - fail('GETTER_CLI_BIN must point to the built getter-cli binary'); - } - - final temp = Directory.systemTemp.createTempSync('upgradeall-getter-cli-'); - addTearDown(() => temp.deleteSync(recursive: true)); - - final dataDir = Directory('${temp.path}/data')..createSync(); - final repoDir = _createFixtureRepository(temp, 'official'); - final bundle = _createLegacyBundle(temp); - final legacyDb = _createLegacyRoomDatabase(temp); - final adapter = - CliGetterAdapter(executable: getterCli, dataDir: dataDir.path); - - adapter.initialize(); - _runGetter(getterCli, dataDir.path, [ - 'repo', - 'add', - 'official', - repoDir.path, - '--priority', - '0', - ]); - _runGetter(getterCli, dataDir.path, [ - 'legacy', - 'import-room-bundle', - bundle.path, - ]); - final repositories = adapter.listRepositories(); - expect(repositories.map((repo) => repo.id), contains('official')); - expect( - repositories.singleWhere((repo) => repo.id == 'official').priority, 0); - - final trackedPackages = adapter.listTrackedPackages(); - final tracked = trackedPackages.singleWhere( - (package) => package.id == 'android/org.fdroid.fdroid', - ); - expect(tracked.favorite, isTrue); - expect(tracked.pinVersion, '1.20.0'); - expect(tracked.packageResolution, 'official_repository_package'); - - final evaluated = adapter.evaluatePackage( - 'android/org.fdroid.fdroid', - repositoryId: 'official', - ); - expect(evaluated.name, 'F-Droid'); - expect(evaluated.repositoryId, 'official'); - expect(evaluated.hasFreeNetworkWarning, isTrue); - - final reports = await adapter.readMigrationReports(); - expect( + test( + 'CliGetterAdapter reads real getter repository and tracked state', + () async { + final getterCli = Platform.environment['GETTER_CLI_BIN']; + if (getterCli == null || getterCli.isEmpty) { + fail('GETTER_CLI_BIN must point to the built getter-cli binary'); + } + + final temp = Directory.systemTemp.createTempSync( + 'upgradeall-getter-cli-', + ); + addTearDown(() => temp.deleteSync(recursive: true)); + + final dataDir = Directory('${temp.path}/data')..createSync(); + final repoDir = _createFixtureRepository(temp, 'official'); + final bundle = _createLegacyBundle(temp); + final legacyDb = _createLegacyRoomDatabase(temp); + final adapter = CliGetterAdapter( + executable: getterCli, + dataDir: dataDir.path, + ); + + adapter.initialize(); + _runGetter(getterCli, dataDir.path, [ + 'repo', + 'add', + 'official', + repoDir.path, + '--priority', + '0', + ]); + _runGetter(getterCli, dataDir.path, [ + 'legacy', + 'import-room-bundle', + bundle.path, + ]); + final repositories = adapter.listRepositories(); + expect(repositories.map((repo) => repo.id), contains('official')); + expect( + repositories.singleWhere((repo) => repo.id == 'official').priority, + 0, + ); + + final trackedPackages = adapter.listTrackedPackages(); + final tracked = trackedPackages.singleWhere( + (package) => package.id == 'android/org.fdroid.fdroid', + ); + expect(tracked.favorite, isTrue); + expect(tracked.pinVersion, '1.20.0'); + expect(tracked.packageResolution, 'official_repository_package'); + + final evaluated = adapter.evaluatePackage( + 'android/org.fdroid.fdroid', + repositoryId: 'official', + ); + expect(evaluated.name, 'F-Droid'); + expect(evaluated.repositoryId, 'official'); + expect(evaluated.hasFreeNetworkWarning, isTrue); + + final reports = await adapter.readMigrationReports(); + expect( reports.singleWhere((report) => report.code == 'migration.imported').ok, - isTrue); - - final alreadyImported = - await adapter.importLegacyRoomDatabase(legacyDb.path); - expect(alreadyImported.alreadyImported, isTrue); - expect(alreadyImported.importedRecords, 0); - expect( - alreadyImported.trackedPackages.map((package) => package.id), - contains('android/org.fdroid.fdroid'), - ); - - final snapshot = adapter.loadSnapshot(); - expect(snapshot.status, 'Getter CLI ready'); - expect(snapshot.repositories.map((repo) => repo.id), contains('official')); - final app = snapshot.apps.singleWhere( - (app) => app.id == 'android/org.fdroid.fdroid', - ); - expect(app.name, 'F-Droid'); - expect(app.installedVersion, 'unknown'); - expect(app.hasFreeNetworkWarning, isTrue); - }); + isTrue, + ); + + final alreadyImported = await adapter.importLegacyRoomDatabase( + legacyDb.path, + ); + expect(alreadyImported.alreadyImported, isTrue); + expect(alreadyImported.importedRecords, 0); + expect( + alreadyImported.trackedPackages.map((package) => package.id), + contains('android/org.fdroid.fdroid'), + ); + + final snapshot = await adapter.loadSnapshot(); + expect(snapshot.status, 'Getter CLI ready'); + expect( + snapshot.repositories.map((repo) => repo.id), + contains('official'), + ); + final app = snapshot.apps.singleWhere( + (app) => app.id == 'android/org.fdroid.fdroid', + ); + expect(app.name, 'F-Droid'); + expect(app.installedVersion, 'unknown'); + expect(app.hasFreeNetworkWarning, isTrue); + }, + ); } Directory _createFixtureRepository(Directory temp, String repoId) { @@ -128,8 +143,9 @@ name = "Fixture $repoId" priority = 0 api_version = "getter.repo.v1" '''); - File('${repoDir.path}/packages/android/org.fdroid.fdroid.lua') - .writeAsStringSync(''' + File( + '${repoDir.path}/packages/android/org.fdroid.fdroid.lua', + ).writeAsStringSync(''' return package_def { id = "android/org.fdroid.fdroid", name = "F-Droid", @@ -187,21 +203,26 @@ conn.close() db.path, ]); if (result.exitCode != 0) { - fail('failed to create legacy Room DB fixture\n' - 'stdout:\n${result.stdout}\n' - 'stderr:\n${result.stderr}'); + fail( + 'failed to create legacy Room DB fixture\n' + 'stdout:\n${result.stdout}\n' + 'stderr:\n${result.stderr}', + ); } return db; } void _runGetter(String getterCli, String dataDir, List args) { - final result = Process.runSync( - getterCli, - ['--data-dir', dataDir, ...args], - ); + final result = Process.runSync(getterCli, [ + '--data-dir', + dataDir, + ...args, + ]); if (result.exitCode != 0) { - fail('getter ${args.join(' ')} failed with ${result.exitCode}\n' - 'stdout:\n${result.stdout}\n' - 'stderr:\n${result.stderr}'); + fail( + 'getter ${args.join(' ')} failed with ${result.exitCode}\n' + 'stdout:\n${result.stdout}\n' + 'stderr:\n${result.stderr}', + ); } } diff --git a/app_flutter/lib/cli_getter_adapter.dart b/app_flutter/lib/cli_getter_adapter.dart index 8e50d461..ea0ee05f 100644 --- a/app_flutter/lib/cli_getter_adapter.dart +++ b/app_flutter/lib/cli_getter_adapter.dart @@ -55,8 +55,9 @@ class CliGetterAdapter implements GetterAdapter { final json = _runGetter(const ['legacy', 'report-list']); final reports = _asList(_data(json)['reports'], 'reports'); return reports - .map((report) => - MigrationReportSummary.fromJson(_asMap(report, 'report'))) + .map( + (report) => MigrationReportSummary.fromJson(_asMap(report, 'report')), + ) .toList(growable: false); } @@ -64,11 +65,7 @@ class CliGetterAdapter implements GetterAdapter { Future importLegacyRoomDatabase( String databasePath, ) async { - final json = _runGetter([ - 'legacy', - 'import-room-db', - databasePath, - ]); + final json = _runGetter(['legacy', 'import-room-db', databasePath]); return LegacyMigrationImportResult.fromJson(_data(json)); } @@ -168,8 +165,7 @@ class CliGetterAdapter implements GetterAdapter { String taskId, RuntimeUserResult result, { String? reason, - }) => - _unsupportedRuntimeTask(); + }) => _unsupportedRuntimeTask(); @override Future> cleanRuntimeTasks({ @@ -193,23 +189,25 @@ class CliGetterAdapter implements GetterAdapter { } @override - GetterSnapshot loadSnapshot() { + Future loadSnapshot() async { initialize(); final repositories = listRepositories(); final trackedPackages = listTrackedPackages(); - final apps = trackedPackages.map((tracked) { - final evaluated = evaluatePackage( - tracked.id, - repositoryId: tracked.repositoryId, - ); - return AppSummary( - id: tracked.id, - name: evaluated.name, - installedVersion: 'unknown', - latestVersion: 'unknown', - hasFreeNetworkWarning: evaluated.hasFreeNetworkWarning, - ); - }).toList(growable: false); + final apps = trackedPackages + .map((tracked) { + final evaluated = evaluatePackage( + tracked.id, + repositoryId: tracked.repositoryId, + ); + return AppSummary( + id: tracked.id, + name: evaluated.name, + installedVersion: 'unknown', + latestVersion: 'unknown', + hasFreeNetworkWarning: evaluated.hasFreeNetworkWarning, + ); + }) + .toList(growable: false); return GetterSnapshot( status: 'Getter CLI ready', @@ -220,11 +218,11 @@ class CliGetterAdapter implements GetterAdapter { } Map _runGetter(List commandArgs) { - final result = Process.runSync( - executable, - ['--data-dir', dataDir, ...commandArgs], - environment: environment.isEmpty ? null : environment, - ); + final result = Process.runSync(executable, [ + '--data-dir', + dataDir, + ...commandArgs, + ], environment: environment.isEmpty ? null : environment); final stdoutText = result.stdout.toString(); final decoded = stdoutText.trim().isEmpty ? {} diff --git a/app_flutter/lib/getter_adapter.dart b/app_flutter/lib/getter_adapter.dart index a9310724..e631dabf 100644 --- a/app_flutter/lib/getter_adapter.dart +++ b/app_flutter/lib/getter_adapter.dart @@ -19,7 +19,8 @@ abstract interface class GetterAdapter { Future> readMigrationReports(); Future importLegacyRoomDatabase( - String databasePath); + String databasePath, + ); Future previewInstalledAutogen({ InstalledAutogenScanOptions options = const InstalledAutogenScanOptions(), @@ -68,7 +69,7 @@ abstract interface class GetterAdapter { RuntimeTaskCleanMode mode = RuntimeTaskCleanMode.defaultMode, }); - GetterSnapshot loadSnapshot(); + Future loadSnapshot(); } class FakeGetterAdapter implements GetterAdapter { @@ -144,7 +145,8 @@ class FakeGetterAdapter implements GetterAdapter { @override Future importLegacyRoomDatabase( - String databasePath) async { + String databasePath, + ) async { throw const GetterBridgeException( GetterError( code: 'bridge.not_connected', @@ -324,8 +326,7 @@ class FakeGetterAdapter implements GetterAdapter { String taskId, RuntimeUserResult result, { String? reason, - }) => - getRuntimeTask(taskId); + }) => getRuntimeTask(taskId); @override Future> cleanRuntimeTasks({ @@ -353,7 +354,7 @@ class FakeGetterAdapter implements GetterAdapter { } @override - GetterSnapshot loadSnapshot() => _snapshot; + Future loadSnapshot() async => _snapshot; } class GetterSnapshot { @@ -412,8 +413,10 @@ class TrackedPackageSummary { json['pin_version'], 'tracked.pin_version', ), - repositoryId: - _jsonOptionalString(json['repository_id'], 'tracked.repository_id'), + repositoryId: _jsonOptionalString( + json['repository_id'], + 'tracked.repository_id', + ), packageResolution: _jsonString( json['package_resolution'], 'tracked.package_resolution', @@ -482,24 +485,29 @@ class LegacyMigrationImportResult { final warningsValue = json['warnings']; final sourceCountsValue = json['source_counts']; return LegacyMigrationImportResult( - alreadyImported: _jsonOptionalBool( + alreadyImported: + _jsonOptionalBool( json['already_imported'], 'migration.already_imported', ) ?? false, importedRecords: _jsonInt(json['imported_records'], 'migration.imported'), trackedPackages: _jsonList(json['apps'], 'migration.apps') - .map((tracked) => TrackedPackageSummary.fromJson( - _jsonMap(tracked, 'migration.tracked_package'), - )) + .map( + (tracked) => TrackedPackageSummary.fromJson( + _jsonMap(tracked, 'migration.tracked_package'), + ), + ) .toList(growable: false), warnings: warningsValue == null ? const [] : _jsonList(warningsValue, 'migration.warnings') - .map((warning) => MigrationWarningSummary.fromJson( + .map( + (warning) => MigrationWarningSummary.fromJson( _jsonMap(warning, 'migration.warning'), - )) - .toList(growable: false), + ), + ) + .toList(growable: false), sourceCounts: sourceCountsValue == null ? null : MigrationSourceCounts.fromJson( @@ -597,8 +605,10 @@ class RuntimePackageSummary { return RuntimePackageSummary( id: _jsonString(json['id'], 'runtime.package.id'), name: _jsonString(json['name'], 'runtime.package.name'), - repositoryId: - _jsonString(json['repository'], 'runtime.package.repository'), + repositoryId: _jsonString( + json['repository'], + 'runtime.package.repository', + ), ); } @@ -618,8 +628,10 @@ class RuntimeUpdateSummary { }); factory RuntimeUpdateSummary.fromJson(Map json) { - final selected = - _jsonMapOrNull(json['selected'], 'runtime.update.selected'); + final selected = _jsonMapOrNull( + json['selected'], + 'runtime.update.selected', + ); final candidate = selected == null ? null : _jsonMap(selected['candidate'], 'runtime.update.selected.candidate'); @@ -637,7 +649,9 @@ class RuntimeUpdateSummary { selectedVersion: candidate == null ? null : _jsonString( - candidate['version'], 'runtime.update.selected.version'), + candidate['version'], + 'runtime.update.selected.version', + ), actions: _jsonList(json['actions'], 'runtime.update.actions') .map((action) => _jsonMap(action, 'runtime.update.action')) .toList(growable: false), @@ -786,8 +800,10 @@ class RuntimeTaskDiagnostic { return RuntimeTaskDiagnostic( code: _jsonString(json['code'], 'runtime.task.diagnostic.code'), message: _jsonString(json['message'], 'runtime.task.diagnostic.message'), - severity: - _jsonString(json['severity'], 'runtime.task.diagnostic.severity'), + severity: _jsonString( + json['severity'], + 'runtime.task.diagnostic.severity', + ), ); } @@ -801,9 +817,9 @@ enum RuntimeUserResult { rejected; String get wireName => switch (this) { - RuntimeUserResult.accepted => 'accepted', - RuntimeUserResult.rejected => 'rejected', - }; + RuntimeUserResult.accepted => 'accepted', + RuntimeUserResult.rejected => 'rejected', + }; } enum RuntimeTaskCleanMode { @@ -812,10 +828,10 @@ enum RuntimeTaskCleanMode { allInactive; String get wireName => switch (this) { - RuntimeTaskCleanMode.defaultMode => 'default', - RuntimeTaskCleanMode.failed => 'failed', - RuntimeTaskCleanMode.allInactive => 'all_inactive', - }; + RuntimeTaskCleanMode.defaultMode => 'default', + RuntimeTaskCleanMode.failed => 'failed', + RuntimeTaskCleanMode.allInactive => 'all_inactive', + }; } class RuntimeNotificationEnvelope { @@ -847,9 +863,9 @@ class InstalledAutogenScanOptions { final bool includeSelf; Map toJson() => { - 'include_system_apps': includeSystemApps, - 'include_self': includeSelf, - }; + 'include_system_apps': includeSystemApps, + 'include_self': includeSelf, + }; } class InstalledAutogenPreview { @@ -869,8 +885,10 @@ class InstalledAutogenPreview { final scan = _jsonMapOrNull(json['scan'], 'autogen.scan'); return InstalledAutogenPreview( operation: _jsonString(json['operation'], 'autogen.operation'), - targetRepoId: - _jsonString(json['target_repo_id'], 'autogen.target_repo_id'), + targetRepoId: _jsonString( + json['target_repo_id'], + 'autogen.target_repo_id', + ), targetRepoPath: _jsonOptionalString( json['target_repo_path'], 'autogen.target_repo_path', @@ -879,23 +897,29 @@ class InstalledAutogenPreview { _jsonMap(json['summary'], 'autogen.summary'), ), candidates: _jsonList(json['candidates'], 'autogen.candidates') - .map((candidate) => InstalledAutogenCandidate.fromJson( - _jsonMap(candidate, 'autogen.candidate'), - )) + .map( + (candidate) => InstalledAutogenCandidate.fromJson( + _jsonMap(candidate, 'autogen.candidate'), + ), + ) .toList(growable: false), skipped: _jsonList(json['skipped'], 'autogen.skipped') - .map((skip) => InstalledAutogenSkip.fromJson( - _jsonMap(skip, 'autogen.skip'), - )) - .toList(growable: false), - diagnostics: _jsonList( - scan?['diagnostics'] ?? json['diagnostics'], - 'autogen.diagnostics', - ) - .map((diagnostic) => PlatformDiagnosticSummary.fromJson( - _jsonMap(diagnostic, 'autogen.diagnostic'), - )) + .map( + (skip) => + InstalledAutogenSkip.fromJson(_jsonMap(skip, 'autogen.skip')), + ) .toList(growable: false), + diagnostics: + _jsonList( + scan?['diagnostics'] ?? json['diagnostics'], + 'autogen.diagnostics', + ) + .map( + (diagnostic) => PlatformDiagnosticSummary.fromJson( + _jsonMap(diagnostic, 'autogen.diagnostic'), + ), + ) + .toList(growable: false), scanStats: scan == null || scan['stats'] == null ? null : InstalledAutogenScanStats.fromJson( @@ -926,13 +950,19 @@ class AutogenSummary { factory AutogenSummary.fromJson(Map json) { return AutogenSummary( - candidateCount: - _jsonInt(json['candidate_count'], 'autogen.summary.candidate_count'), - skippedCount: - _jsonInt(json['skipped_count'], 'autogen.summary.skipped_count'), + candidateCount: _jsonInt( + json['candidate_count'], + 'autogen.summary.candidate_count', + ), + skippedCount: _jsonInt( + json['skipped_count'], + 'autogen.summary.skipped_count', + ), writeCount: _jsonInt(json['write_count'], 'autogen.summary.write_count'), - deleteCount: - _jsonInt(json['delete_count'], 'autogen.summary.delete_count'), + deleteCount: _jsonInt( + json['delete_count'], + 'autogen.summary.delete_count', + ), ); } @@ -955,18 +985,24 @@ class InstalledAutogenCandidate { factory InstalledAutogenCandidate.fromJson(Map json) { return InstalledAutogenCandidate( - packageId: - _jsonString(json['package_id'], 'autogen.candidate.package_id'), + packageId: _jsonString( + json['package_id'], + 'autogen.candidate.package_id', + ), kind: _jsonString(json['kind'], 'autogen.candidate.kind'), - displayName: - _jsonString(json['display_name'], 'autogen.candidate.display_name'), + displayName: _jsonString( + json['display_name'], + 'autogen.candidate.display_name', + ), action: _jsonString(json['action'], 'autogen.candidate.action'), outputRelativePath: _jsonString( json['output_relative_path'], 'autogen.candidate.output_relative_path', ), - contentHash: - _jsonString(json['content_hash'], 'autogen.candidate.content_hash'), + contentHash: _jsonString( + json['content_hash'], + 'autogen.candidate.content_hash', + ), installedTarget: _jsonMap( json['installed_target'], 'autogen.candidate.installed_target', @@ -1018,10 +1054,14 @@ class InstalledAutogenScanStats { return InstalledAutogenScanStats( totalSeen: _jsonInt(json['total_seen'], 'autogen.scan.total_seen'), returned: _jsonInt(json['returned'], 'autogen.scan.returned'), - filteredSystem: - _jsonInt(json['filtered_system'], 'autogen.scan.filtered_system'), - filteredSelf: - _jsonInt(json['filtered_self'], 'autogen.scan.filtered_self'), + filteredSystem: _jsonInt( + json['filtered_system'], + 'autogen.scan.filtered_system', + ), + filteredSelf: _jsonInt( + json['filtered_self'], + 'autogen.scan.filtered_self', + ), ); } @@ -1062,27 +1102,36 @@ class InstalledAutogenApplyResult { factory InstalledAutogenApplyResult.fromJson(Map json) { return InstalledAutogenApplyResult( - targetRepoId: - _jsonString(json['target_repo_id'], 'autogen.apply.target_repo_id'), + targetRepoId: _jsonString( + json['target_repo_id'], + 'autogen.apply.target_repo_id', + ), targetRepoPath: _jsonOptionalString( json['target_repo_path'], 'autogen.apply.target_repo_path', ), - appliedCount: - _jsonInt(json['applied_count'], 'autogen.apply.applied_count'), + appliedCount: _jsonInt( + json['applied_count'], + 'autogen.apply.applied_count', + ), applied: _jsonList(json['applied'], 'autogen.apply.applied') - .map((applied) => InstalledAutogenAppliedPackage.fromJson( - _jsonMap(applied, 'autogen.apply.applied_item'), - )) - .toList(growable: false), - preservedToLocal: _jsonList( - json['preserved_to_local'], - 'autogen.apply.preserved_to_local', - ) - .map((preserved) => InstalledAutogenPreservedPackage.fromJson( - _jsonMap(preserved, 'autogen.apply.preserved_item'), - )) + .map( + (applied) => InstalledAutogenAppliedPackage.fromJson( + _jsonMap(applied, 'autogen.apply.applied_item'), + ), + ) .toList(growable: false), + preservedToLocal: + _jsonList( + json['preserved_to_local'], + 'autogen.apply.preserved_to_local', + ) + .map( + (preserved) => InstalledAutogenPreservedPackage.fromJson( + _jsonMap(preserved, 'autogen.apply.preserved_item'), + ), + ) + .toList(growable: false), ); } @@ -1122,12 +1171,18 @@ class InstalledAutogenPreservedPackage { factory InstalledAutogenPreservedPackage.fromJson(Map json) { return InstalledAutogenPreservedPackage( - packageId: - _jsonString(json['package_id'], 'autogen.preserved.package_id'), - repositoryId: - _jsonString(json['repository_id'], 'autogen.preserved.repository_id'), - relativePath: - _jsonString(json['relative_path'], 'autogen.preserved.relative_path'), + packageId: _jsonString( + json['package_id'], + 'autogen.preserved.package_id', + ), + repositoryId: _jsonString( + json['repository_id'], + 'autogen.preserved.repository_id', + ), + relativePath: _jsonString( + json['relative_path'], + 'autogen.preserved.relative_path', + ), ); } diff --git a/app_flutter/lib/main.dart b/app_flutter/lib/main.dart index db9b2573..97c48322 100644 --- a/app_flutter/lib/main.dart +++ b/app_flutter/lib/main.dart @@ -23,8 +23,9 @@ class AppKeys { static const logsRoute = ValueKey('route.logs'); static const settingsRoute = ValueKey('route.settings'); static const migrationRoute = ValueKey('route.migration'); - static const installedAutogenRoute = - ValueKey('route.installed_autogen'); + static const installedAutogenRoute = ValueKey( + 'route.installed_autogen', + ); static const openApps = ValueKey('action.open_apps'); static const openRepositories = ValueKey('action.open_repositories'); @@ -32,15 +33,19 @@ class AppKeys { static const openLogs = ValueKey('action.open_logs'); static const openSettings = ValueKey('action.open_settings'); static const openMigration = ValueKey('action.open_migration'); - static const openInstalledAutogen = - ValueKey('action.open_installed_autogen'); + static const openInstalledAutogen = ValueKey( + 'action.open_installed_autogen', + ); static const openFirstApp = ValueKey('action.open_first_app'); - static const startLegacyMigration = - ValueKey('action.start_legacy_migration'); - static const previewInstalledAutogen = - ValueKey('action.preview_installed_autogen'); - static const applyInstalledAutogen = - ValueKey('action.apply_installed_autogen'); + static const startLegacyMigration = ValueKey( + 'action.start_legacy_migration', + ); + static const previewInstalledAutogen = ValueKey( + 'action.preview_installed_autogen', + ); + static const applyInstalledAutogen = ValueKey( + 'action.apply_installed_autogen', + ); static const updateSummary = ValueKey('state.update_summary'); static const getterStatus = ValueKey('state.getter_status'); @@ -53,30 +58,41 @@ class AppKeys { static const settingsShell = ValueKey('state.settings_shell'); static const migrationReady = ValueKey('state.migration_ready'); static const migrationStatus = ValueKey('state.migration_status'); - static const migrationBridgeUnavailable = - ValueKey('state.migration_bridge_unavailable'); + static const migrationBridgeUnavailable = ValueKey( + 'state.migration_bridge_unavailable', + ); static const migrationImported = ValueKey('state.migration_imported'); static const migrationError = ValueKey('state.migration_error'); - static const migrationReportsList = - ValueKey('state.migration_reports_list'); - static const installedAutogenReady = - ValueKey('state.installed_autogen_ready'); - static const installedAutogenBridgeUnavailable = - ValueKey('state.installed_autogen_bridge_unavailable'); - static const installedAutogenPreview = - ValueKey('state.installed_autogen_preview'); - static const installedAutogenCandidatesList = - ValueKey('state.installed_autogen_candidates_list'); - static const installedAutogenSkipsList = - ValueKey('state.installed_autogen_skips_list'); - static const installedAutogenDiagnosticsList = - ValueKey('state.installed_autogen_diagnostics_list'); - static const installedAutogenScanStats = - ValueKey('state.installed_autogen_scan_stats'); - static const installedAutogenApplied = - ValueKey('state.installed_autogen_applied'); - static const installedAutogenError = - ValueKey('state.installed_autogen_error'); + static const migrationReportsList = ValueKey( + 'state.migration_reports_list', + ); + static const installedAutogenReady = ValueKey( + 'state.installed_autogen_ready', + ); + static const installedAutogenBridgeUnavailable = ValueKey( + 'state.installed_autogen_bridge_unavailable', + ); + static const installedAutogenPreview = ValueKey( + 'state.installed_autogen_preview', + ); + static const installedAutogenCandidatesList = ValueKey( + 'state.installed_autogen_candidates_list', + ); + static const installedAutogenSkipsList = ValueKey( + 'state.installed_autogen_skips_list', + ); + static const installedAutogenDiagnosticsList = ValueKey( + 'state.installed_autogen_diagnostics_list', + ); + static const installedAutogenScanStats = ValueKey( + 'state.installed_autogen_scan_stats', + ); + static const installedAutogenApplied = ValueKey( + 'state.installed_autogen_applied', + ); + static const installedAutogenError = ValueKey( + 'state.installed_autogen_error', + ); static ValueKey appRow(String packageId) => ValueKey('state.app.$packageId'); @@ -122,9 +138,9 @@ class UpgradeAllApp extends StatelessWidget { '/logs': (context) => const LogsPage(), '/settings': (context) => const SettingsPage(), '/migration': (context) => MigrationPage( - getter: getter, - legacyMigrationPlatform: legacyMigrationPlatform, - ), + getter: getter, + legacyMigrationPlatform: legacyMigrationPlatform, + ), '/autogen': (context) => InstalledAutogenPage(getter: getter), }, onGenerateRoute: (settings) { @@ -141,111 +157,147 @@ class UpgradeAllApp extends StatelessWidget { } } -class HomePage extends StatelessWidget { +class HomePage extends StatefulWidget { const HomePage({super.key, required this.getter}); final GetterAdapter getter; + @override + State createState() => _HomePageState(); +} + +class _HomePageState extends State { + late final Future _snapshot = widget.getter.loadSnapshot(); + @override Widget build(BuildContext context) { - final snapshot = getter.loadSnapshot(); return Scaffold( key: AppKeys.homeRoute, appBar: AppBar(title: const Text('UpgradeAll')), - body: ListView( - padding: const EdgeInsets.all(16), - children: [ - Card( - key: AppKeys.updateSummary, - child: ListTile( - title: const Text('Updates'), - subtitle: Text('${snapshot.updateCount} updates available'), - ), - ), - Card( - key: AppKeys.getterStatus, - child: ListTile( - title: const Text('Getter core'), - subtitle: Text(snapshot.status), - ), - ), - const SizedBox(height: 16), - const _RouteButton( - key: AppKeys.openApps, - icon: Icons.apps, - label: 'Apps', - routeName: '/apps', - ), - const _RouteButton( - key: AppKeys.openRepositories, - icon: Icons.source, - label: 'Repositories', - routeName: '/repositories', - ), - const _RouteButton( - key: AppKeys.openDownloads, - icon: Icons.download, - label: 'Downloads', - routeName: '/downloads', - ), - const _RouteButton( - key: AppKeys.openLogs, - icon: Icons.receipt_long, - label: 'Logs', - routeName: '/logs', - ), - const _RouteButton( - key: AppKeys.openSettings, - icon: Icons.settings, - label: 'Settings', - routeName: '/settings', - ), - const _RouteButton( - key: AppKeys.openMigration, - icon: Icons.move_down, - label: 'Legacy migration', - routeName: '/migration', - ), - const _RouteButton( - key: AppKeys.openInstalledAutogen, - icon: Icons.auto_fix_high, - label: 'Installed autogen', - routeName: '/autogen', - ), - ], + body: FutureBuilder( + future: _snapshot, + builder: (context, snapshot) { + final data = snapshot.data; + return ListView( + padding: const EdgeInsets.all(16), + children: [ + Card( + key: AppKeys.updateSummary, + child: ListTile( + title: const Text('Updates'), + subtitle: Text('${data?.updateCount ?? 0} updates available'), + ), + ), + Card( + key: AppKeys.getterStatus, + child: ListTile( + title: const Text('Getter core'), + subtitle: Text( + snapshot.hasError + ? 'Getter snapshot unavailable' + : data?.status ?? 'Loading getter snapshot...', + ), + ), + ), + const SizedBox(height: 16), + const _RouteButton( + key: AppKeys.openApps, + icon: Icons.apps, + label: 'Apps', + routeName: '/apps', + ), + const _RouteButton( + key: AppKeys.openRepositories, + icon: Icons.source, + label: 'Repositories', + routeName: '/repositories', + ), + const _RouteButton( + key: AppKeys.openDownloads, + icon: Icons.download, + label: 'Downloads', + routeName: '/downloads', + ), + const _RouteButton( + key: AppKeys.openLogs, + icon: Icons.receipt_long, + label: 'Logs', + routeName: '/logs', + ), + const _RouteButton( + key: AppKeys.openSettings, + icon: Icons.settings, + label: 'Settings', + routeName: '/settings', + ), + const _RouteButton( + key: AppKeys.openMigration, + icon: Icons.move_down, + label: 'Legacy migration', + routeName: '/migration', + ), + const _RouteButton( + key: AppKeys.openInstalledAutogen, + icon: Icons.auto_fix_high, + label: 'Installed autogen', + routeName: '/autogen', + ), + ], + ); + }, ), ); } } -class AppsPage extends StatelessWidget { +class AppsPage extends StatefulWidget { const AppsPage({super.key, required this.getter}); final GetterAdapter getter; + @override + State createState() => _AppsPageState(); +} + +class _AppsPageState extends State { + late final Future _snapshot = widget.getter.loadSnapshot(); + @override Widget build(BuildContext context) { - final apps = getter.loadSnapshot().apps; return Scaffold( key: AppKeys.appsRoute, appBar: AppBar(title: const Text('Apps')), - body: ListView.builder( - key: AppKeys.appsList, - itemCount: apps.length, - itemBuilder: (context, index) { - final app = apps[index]; - return ListTile( - key: AppKeys.appRow(app.id), - title: Text(app.name), - subtitle: Text('${app.id} • ${app.installedVersion}'), - trailing: app.hasFreeNetworkWarning - ? const Chip( - label: Text('Network'), - backgroundColor: Colors.amber, - ) - : null, - onTap: () { - Navigator.of(context).pushNamed('/apps/detail', arguments: app); + body: FutureBuilder( + future: _snapshot, + builder: (context, snapshot) { + if (snapshot.connectionState != ConnectionState.done) { + return const Center(child: Text('Loading getter apps...')); + } + if (snapshot.hasError) { + return const Center(child: Text('Getter apps unavailable')); + } + final apps = snapshot.data?.apps ?? const []; + return ListView.builder( + key: AppKeys.appsList, + itemCount: apps.length, + itemBuilder: (context, index) { + final app = apps[index]; + return ListTile( + key: AppKeys.appRow(app.id), + title: Text(app.name), + subtitle: Text('${app.id} • ${app.installedVersion}'), + trailing: app.hasFreeNetworkWarning + ? const Chip( + label: Text('Network'), + backgroundColor: Colors.amber, + ) + : null, + onTap: () { + Navigator.of( + context, + ).pushNamed('/apps/detail', arguments: app); + }, + ); }, ); }, @@ -285,26 +337,45 @@ class AppDetailPage extends StatelessWidget { } } -class RepositoriesPage extends StatelessWidget { +class RepositoriesPage extends StatefulWidget { const RepositoriesPage({super.key, required this.getter}); final GetterAdapter getter; + @override + State createState() => _RepositoriesPageState(); +} + +class _RepositoriesPageState extends State { + late final Future _snapshot = widget.getter.loadSnapshot(); + @override Widget build(BuildContext context) { - final repositories = getter.loadSnapshot().repositories; return Scaffold( key: AppKeys.repositoriesRoute, appBar: AppBar(title: const Text('Repositories')), - body: ListView.builder( - key: AppKeys.repositoriesList, - itemCount: repositories.length, - itemBuilder: (context, index) { - final repository = repositories[index]; - return ListTile( - key: AppKeys.repoRow(repository.id), - title: Text(repository.id), - subtitle: Text('Priority ${repository.priority}'), + body: FutureBuilder( + future: _snapshot, + builder: (context, snapshot) { + if (snapshot.connectionState != ConnectionState.done) { + return const Center(child: Text('Loading getter repositories...')); + } + if (snapshot.hasError) { + return const Center(child: Text('Getter repositories unavailable')); + } + final repositories = + snapshot.data?.repositories ?? const []; + return ListView.builder( + key: AppKeys.repositoriesList, + itemCount: repositories.length, + itemBuilder: (context, index) { + final repository = repositories[index]; + return ListTile( + key: AppKeys.repoRow(repository.id), + title: Text(repository.id), + subtitle: Text('Priority ${repository.priority}'), + ); + }, ); }, ), @@ -352,11 +423,10 @@ class DownloadsPage extends StatelessWidget { child: ListTile( key: AppKeys.downloadTaskRow(task.taskId), title: Text(task.packageId), - subtitle: Text( - '${task.status} • ${task.phase.category}', + subtitle: Text('${task.status} • ${task.phase.category}'), + trailing: _TaskCapabilitiesChips( + capabilities: task.capabilities, ), - trailing: - _TaskCapabilitiesChips(capabilities: task.capabilities), ), ); }, @@ -486,8 +556,9 @@ class _InstalledAutogenPageState extends State { children: [ ElevatedButton.icon( key: AppKeys.previewInstalledAutogen, - onPressed: - _running || !canUseBridge ? null : _previewInstalledAutogen, + onPressed: _running || !canUseBridge + ? null + : _previewInstalledAutogen, icon: const Icon(Icons.manage_search), label: Text(_running ? 'Working…' : 'Preview installed autogen'), ), @@ -569,8 +640,10 @@ class _InstalledAutogenPageState extends State { ], if (preview.diagnostics.isNotEmpty) ...[ const SizedBox(height: 16), - Text('Diagnostics', - style: Theme.of(context).textTheme.titleMedium), + Text( + 'Diagnostics', + style: Theme.of(context).textTheme.titleMedium, + ), ListView.builder( key: AppKeys.installedAutogenDiagnosticsList, shrinkWrap: true, @@ -698,8 +771,8 @@ class _MigrationPageState extends State { }); try { - final candidate = - await widget.legacyMigrationPlatform.prepareLegacyRoomImport(); + final candidate = await widget.legacyMigrationPlatform + .prepareLegacyRoomImport(); if (!mounted) return; if (!candidate.found || candidate.databasePath == null) { setState(() { @@ -709,8 +782,9 @@ class _MigrationPageState extends State { return; } - final importResult = - await widget.getter.importLegacyRoomDatabase(candidate.databasePath!); + final importResult = await widget.getter.importLegacyRoomDatabase( + candidate.databasePath!, + ); final reports = await widget.getter.readMigrationReports(); if (!mounted) return; setState(() { @@ -760,8 +834,9 @@ class _MigrationPageState extends State { children: [ ElevatedButton.icon( key: AppKeys.startLegacyMigration, - onPressed: - _running || !canImportLegacyRoom ? null : _startMigration, + onPressed: _running || !canImportLegacyRoom + ? null + : _startMigration, icon: const Icon(Icons.move_down), label: Text(_running ? 'Migrating…' : 'Start legacy migration'), ), @@ -866,9 +941,7 @@ class _PlaceholderPage extends StatelessWidget { Widget build(BuildContext context) { return Scaffold( appBar: AppBar(title: Text(title)), - body: Center( - child: Text(key: stateKey, message), - ), + body: Center(child: Text(key: stateKey, message)), ); } } diff --git a/app_flutter/lib/native_getter_adapter.dart b/app_flutter/lib/native_getter_adapter.dart index 0d791ffd..e2a731b0 100644 --- a/app_flutter/lib/native_getter_adapter.dart +++ b/app_flutter/lib/native_getter_adapter.dart @@ -6,13 +6,11 @@ import 'package:flutter/services.dart'; import 'getter_adapter.dart'; -/// First Android production bridge slice. +/// Android production getter bridge. /// -/// Until the full native getter bridge replaces every CLI/fake surface, this -/// adapter inherits the deterministic shell data from [FakeGetterAdapter] and -/// overrides only installed-autogen operations with the Rust/native bridge. /// The bridge returns getter-owned JSON envelopes; Dart parses and renders them -/// but does not scan PackageManager or make autogen/package decisions. +/// but does not scan PackageManager, resolve repositories, evaluate Lua, or make +/// autogen/update/runtime decisions. class MethodChannelGetterAdapter extends FakeGetterAdapter { // Keep public parameter names stable for tests and injected bridges. const MethodChannelGetterAdapter({ @@ -96,6 +94,16 @@ class MethodChannelGetterAdapter extends FakeGetterAdapter { return InstalledAutogenApplyResult.fromJson(data); } + Future> invokeReadOperation( + String operation, { + Map payload = const {}, + }) { + return _invokeGetterData('readOperation', { + 'operation': operation, + 'payload': payload, + }); + } + /// Invoke a getter runtime operation through the native bridge. /// /// This is an internal/debug bridge primitive for ADR-0011 wiring. Product UI @@ -124,6 +132,71 @@ class MethodChannelGetterAdapter extends FakeGetterAdapter { }); } + @override + Future loadSnapshot() async { + final repositoriesData = await invokeReadOperation('repository_list'); + final trackedData = await invokeReadOperation('tracked_package_list'); + final repositories = _asList( + repositoriesData['repositories'], + 'repositories', + ).map(_repositoryFromJson).toList(growable: false); + final trackedPackages = _asList(trackedData['packages'], 'tracked packages') + .map( + (tracked) => TrackedPackageSummary.fromJson( + _asMap(tracked, 'tracked package'), + ), + ) + .toList(growable: false); + final apps = []; + for (final tracked in trackedPackages) { + try { + final package = await _evaluatePackageFromGetter( + tracked.id, + repositoryId: tracked.repositoryId, + ); + apps.add( + AppSummary( + id: tracked.id, + name: package.name, + installedVersion: 'unknown', + latestVersion: 'unknown', + hasFreeNetworkWarning: package.hasFreeNetworkWarning, + ), + ); + } catch (_) { + apps.add( + AppSummary( + id: tracked.id, + name: tracked.id, + installedVersion: 'unknown', + latestVersion: 'unknown', + hasFreeNetworkWarning: false, + ), + ); + } + } + return GetterSnapshot( + status: 'Getter native bridge ready', + updateCount: 0, + apps: apps, + repositories: repositories, + ); + } + + Future _evaluatePackageFromGetter( + String packageId, { + String? repositoryId, + }) async { + final data = await invokeReadOperation( + 'package_eval', + payload: { + 'package_id': packageId, + 'repository_id': ?repositoryId, + }, + ); + return _packageEvaluationFromJson(_asMap(data['package'], 'package')); + } + @override Future checkPackageForUpdate( String packageId, { @@ -271,6 +344,27 @@ class MethodChannelGetterAdapter extends FakeGetterAdapter { } } +RepositorySummary _repositoryFromJson(Object? value) { + final json = _asMap(value, 'repository'); + return RepositorySummary( + id: _asString(json['id'], 'repository.id'), + priority: _asInt(json['priority'], 'repository.priority'), + ); +} + +PackageEvaluation _packageEvaluationFromJson(Map json) { + final permissions = _asMap(json['permissions'], 'package.permissions'); + return PackageEvaluation( + id: _asString(json['id'], 'package.id'), + repositoryId: _asString(json['repository'], 'package.repository'), + name: _asString(json['name'], 'package.name'), + hasFreeNetworkWarning: _asBool( + permissions['free_network'], + 'package.permissions.free_network', + ), + ); +} + GetterError _errorFromEnvelope(Map envelope) { final error = _asMap(envelope['error'], 'getter bridge error'); return GetterError( @@ -296,3 +390,13 @@ String _asString(Object? value, String name) { if (value is String) return value; throw FormatException('$name should be a string'); } + +int _asInt(Object? value, String name) { + if (value is int) return value; + throw FormatException('$name should be an integer'); +} + +bool _asBool(Object? value, String name) { + if (value is bool) return value; + throw FormatException('$name should be a boolean'); +} diff --git a/app_flutter/test/native_getter_adapter_test.dart b/app_flutter/test/native_getter_adapter_test.dart index 52279395..51f3e1ce 100644 --- a/app_flutter/test/native_getter_adapter_test.dart +++ b/app_flutter/test/native_getter_adapter_test.dart @@ -19,64 +19,69 @@ void main() { .setMockMethodCallHandler(eventMethodChannel, null); }); - test('native preview sends scan options and parses getter envelope', - () async { - MethodCall? captured; - TestDefaultBinaryMessengerBinding.instance.defaultBinaryMessenger - .setMockMethodCallHandler(channel, (call) async { - captured = call; - return jsonEncode({ - 'ok': true, - 'command': 'autogen installed preview', - 'data': _previewJson(), - 'warnings': [], - }); - }); + test( + 'native preview sends scan options and parses getter envelope', + () async { + MethodCall? captured; + TestDefaultBinaryMessengerBinding.instance.defaultBinaryMessenger + .setMockMethodCallHandler(channel, (call) async { + captured = call; + return jsonEncode({ + 'ok': true, + 'command': 'autogen installed preview', + 'data': _previewJson(), + 'warnings': [], + }); + }); - const adapter = MethodChannelGetterAdapter(channel: channel); - final preview = await adapter.previewInstalledAutogen( - options: const InstalledAutogenScanOptions( - includeSystemApps: true, - includeSelf: true, - ), - ); + const adapter = MethodChannelGetterAdapter(channel: channel); + final preview = await adapter.previewInstalledAutogen( + options: const InstalledAutogenScanOptions( + includeSystemApps: true, + includeSelf: true, + ), + ); - expect(captured!.method, 'previewInstalledAutogen'); - expect(captured!.arguments, { - 'scan_options': { - 'include_system_apps': true, - 'include_self': true, - }, - }); - expect(preview.summary.candidateCount, 1); - expect(preview.scanStats!.returned, 1); - expect(preview.candidates.single.packageId, 'android/com.example.autogen'); - }); + expect(captured!.method, 'previewInstalledAutogen'); + expect(captured!.arguments, { + 'scan_options': { + 'include_system_apps': true, + 'include_self': true, + }, + }); + expect(preview.summary.candidateCount, 1); + expect(preview.scanStats!.returned, 1); + expect( + preview.candidates.single.packageId, + 'android/com.example.autogen', + ); + }, + ); test('native apply forwards preview JSON and package acceptance', () async { MethodCall? captured; TestDefaultBinaryMessengerBinding.instance.defaultBinaryMessenger .setMockMethodCallHandler(channel, (call) async { - captured = call; - return jsonEncode({ - 'ok': true, - 'command': 'autogen installed apply', - 'data': { - 'target_repo_id': 'local_autogen', - 'target_repo_path': '/getter/repositories/local_autogen', - 'applied_count': 1, - 'applied': [ - { - 'package_id': 'android/com.example.autogen', - 'output_relative_path': - 'packages/android/com.example.autogen.lua', + captured = call; + return jsonEncode({ + 'ok': true, + 'command': 'autogen installed apply', + 'data': { + 'target_repo_id': 'local_autogen', + 'target_repo_path': '/getter/repositories/local_autogen', + 'applied_count': 1, + 'applied': [ + { + 'package_id': 'android/com.example.autogen', + 'output_relative_path': + 'packages/android/com.example.autogen.lua', + }, + ], + 'preserved_to_local': [], }, - ], - 'preserved_to_local': [], - }, - 'warnings': [], - }); - }); + 'warnings': [], + }); + }); const adapter = MethodChannelGetterAdapter(channel: channel); final preview = InstalledAutogenPreview.fromJson(_previewJson()); @@ -86,8 +91,8 @@ void main() { ); expect(captured!.method, 'applyInstalledAutogen'); - final args = - (captured!.arguments as Map).cast(); + final args = (captured!.arguments as Map) + .cast(); expect(jsonDecode(args['preview_json']! as String), preview.rawJson); expect(args['acceptance'], { 'mode': 'packages', @@ -100,59 +105,60 @@ void main() { final calls = []; TestDefaultBinaryMessengerBinding.instance.defaultBinaryMessenger .setMockMethodCallHandler(channel, (call) async { - calls.add(call); - switch (call.method) { - case 'importLegacyRoomDatabase': - return jsonEncode({ - 'ok': true, - 'command': 'legacy import-room-db', - 'data': { - 'imported_records': 1, - 'apps': [ - { - 'id': 'android/org.fdroid.fdroid', - 'enabled': true, - 'favorite': true, - 'pin_version': '1.20.0', - 'repository_id': null, - 'package_resolution': 'missing_package_definition', - }, - ], - 'warnings': [], - 'source_counts': { - 'app_rows': 1, - 'extra_app_rows': 1, - 'hub_rows': 0, - 'extra_hub_rows': 0, - }, - }, - 'warnings': [], - }); - case 'legacyReportList': - return jsonEncode({ - 'ok': true, - 'command': 'legacy report-list', - 'data': { - 'reports': [ - { - 'ok': true, - 'code': 'migration.imported', - 'message': 'Legacy Room data imported', + calls.add(call); + switch (call.method) { + case 'importLegacyRoomDatabase': + return jsonEncode({ + 'ok': true, + 'command': 'legacy import-room-db', + 'data': { 'imported_records': 1, - 'tracked_records': 1, + 'apps': [ + { + 'id': 'android/org.fdroid.fdroid', + 'enabled': true, + 'favorite': true, + 'pin_version': '1.20.0', + 'repository_id': null, + 'package_resolution': 'missing_package_definition', + }, + ], + 'warnings': [], + 'source_counts': { + 'app_rows': 1, + 'extra_app_rows': 1, + 'hub_rows': 0, + 'extra_hub_rows': 0, + }, }, - ], - }, - 'warnings': [], - }); - default: - fail('unexpected method ${call.method}'); - } - }); + 'warnings': [], + }); + case 'legacyReportList': + return jsonEncode({ + 'ok': true, + 'command': 'legacy report-list', + 'data': { + 'reports': [ + { + 'ok': true, + 'code': 'migration.imported', + 'message': 'Legacy Room data imported', + 'imported_records': 1, + 'tracked_records': 1, + }, + ], + }, + 'warnings': [], + }); + default: + fail('unexpected method ${call.method}'); + } + }); const adapter = MethodChannelGetterAdapter(channel: channel); - final importResult = - await adapter.importLegacyRoomDatabase('/tmp/legacy.db'); + final importResult = await adapter.importLegacyRoomDatabase( + '/tmp/legacy.db', + ); final reports = await adapter.readMigrationReports(); expect(calls.map((call) => call.method), [ @@ -167,28 +173,108 @@ void main() { expect(reports.single.code, 'migration.imported'); }); + test( + 'native snapshot reads repositories and package data through getter', + () async { + final calls = []; + TestDefaultBinaryMessengerBinding.instance.defaultBinaryMessenger + .setMockMethodCallHandler(channel, (call) async { + calls.add(call); + final args = (call.arguments as Map) + .cast(); + switch (args['operation']) { + case 'repository_list': + return jsonEncode({ + 'ok': true, + 'command': 'read operation', + 'data': { + 'repositories': [ + {'id': 'official', 'priority': 0}, + ], + }, + 'warnings': [], + }); + case 'tracked_package_list': + return jsonEncode({ + 'ok': true, + 'command': 'read operation', + 'data': { + 'packages': [ + { + 'id': 'android/org.fdroid.fdroid', + 'enabled': true, + 'favorite': false, + 'pin_version': null, + 'repository_id': 'official', + 'package_resolution': 'official_repository_package', + }, + ], + }, + 'warnings': [], + }); + case 'package_eval': + expect(args['payload'], { + 'package_id': 'android/org.fdroid.fdroid', + 'repository_id': 'official', + }); + return jsonEncode({ + 'ok': true, + 'command': 'read operation', + 'data': { + 'package': { + 'id': 'android/org.fdroid.fdroid', + 'name': 'F-Droid', + 'repository': 'official', + 'permissions': {'free_network': true}, + }, + }, + 'warnings': [], + }); + default: + fail('unexpected read operation ${args['operation']}'); + } + }); + + const adapter = MethodChannelGetterAdapter(channel: channel); + final snapshot = await adapter.loadSnapshot(); + + expect(calls.map((call) => call.method), [ + 'readOperation', + 'readOperation', + 'readOperation', + ]); + expect(snapshot.status, 'Getter native bridge ready'); + expect(snapshot.repositories.single.id, 'official'); + expect(snapshot.apps.single.id, 'android/org.fdroid.fdroid'); + expect(snapshot.apps.single.name, 'F-Droid'); + expect(snapshot.apps.single.hasFreeNetworkWarning, isTrue); + }, + ); + test('runtime notification stream decodes pushed JSON events', () async { TestDefaultBinaryMessengerBinding.instance.defaultBinaryMessenger .setMockMethodCallHandler(eventMethodChannel, (call) async { - if (call.method == 'listen') { - await TestDefaultBinaryMessengerBinding.instance.defaultBinaryMessenger - .handlePlatformMessage( - 'test/runtime_notifications', - const StandardMethodCodec().encodeSuccessEnvelope( - jsonEncode({ - 'kind': 'task_changed', - 'task': { - 'task_id': 'task-1', - 'package_id': 'android/org.fdroid.fdroid', - 'status': 'completed', - }, - }), - ), - (_) {}, - ); - } - return null; - }); + if (call.method == 'listen') { + await TestDefaultBinaryMessengerBinding + .instance + .defaultBinaryMessenger + .handlePlatformMessage( + 'test/runtime_notifications', + const StandardMethodCodec().encodeSuccessEnvelope( + jsonEncode({ + 'kind': 'task_changed', + 'task': { + 'task_id': 'task-1', + 'package_id': 'android/org.fdroid.fdroid', + 'status': 'completed', + }, + }), + ), + (_) {}, + ); + } + return null; + }); const adapter = MethodChannelGetterAdapter( channel: channel, @@ -208,63 +294,63 @@ void main() { final calls = []; TestDefaultBinaryMessengerBinding.instance.defaultBinaryMessenger .setMockMethodCallHandler(channel, (call) async { - calls.add(call); - if (call.method != 'runtimeOperation') { - fail('unexpected method ${call.method}'); - } - final args = - (call.arguments as Map).cast(); - if (args['operation'] == 'update_check_package_issue_action') { - return jsonEncode({ - 'ok': true, - 'command': 'runtime operation', - 'data': { - 'package': { - 'id': 'android/org.fdroid.fdroid', - 'name': 'F-Droid', - 'repository': 'official', - 'permissions': {'free_network': false}, - }, - 'update': { - 'network_required': false, - 'package_id': 'android/org.fdroid.fdroid', - 'installed_version': '1.0.0', - 'effective_local_version': '1.0.0', - 'policy': {'pin_version': null}, - 'status': 'update_available', - 'selected': { - 'package_id': 'android/org.fdroid.fdroid', - 'candidate': { - 'version': '1.2.0', - 'artifacts': [], + calls.add(call); + if (call.method != 'runtimeOperation') { + fail('unexpected method ${call.method}'); + } + final args = (call.arguments as Map) + .cast(); + if (args['operation'] == 'update_check_package_issue_action') { + return jsonEncode({ + 'ok': true, + 'command': 'runtime operation', + 'data': { + 'package': { + 'id': 'android/org.fdroid.fdroid', + 'name': 'F-Droid', + 'repository': 'official', + 'permissions': {'free_network': false}, }, - }, - 'actions': [ - { - 'type': 'download', - 'url': 'https://example.invalid/app.apk', - 'file_name': 'app.apk', + 'update': { + 'network_required': false, + 'package_id': 'android/org.fdroid.fdroid', + 'installed_version': '1.0.0', + 'effective_local_version': '1.0.0', + 'policy': {'pin_version': null}, + 'status': 'update_available', + 'selected': { + 'package_id': 'android/org.fdroid.fdroid', + 'candidate': { + 'version': '1.2.0', + 'artifacts': [], + }, + }, + 'actions': [ + { + 'type': 'download', + 'url': 'https://example.invalid/app.apk', + 'file_name': 'app.apk', + }, + ], }, - ], - }, - 'action': { - 'action_id': 'action-1', - 'package_id': 'android/org.fdroid.fdroid', - }, - }, - 'warnings': [], - }); - } - if (args['operation'] == 'task_submit') { - return jsonEncode({ - 'ok': true, - 'command': 'runtime operation', - 'data': _runtimeTaskJson('task-1', status: 'queued'), - 'warnings': [], + 'action': { + 'action_id': 'action-1', + 'package_id': 'android/org.fdroid.fdroid', + }, + }, + 'warnings': [], + }); + } + if (args['operation'] == 'task_submit') { + return jsonEncode({ + 'ok': true, + 'command': 'runtime operation', + 'data': _runtimeTaskJson('task-1', status: 'queued'), + 'warnings': [], + }); + } + fail('unexpected runtime operation ${args['operation']}'); }); - } - fail('unexpected runtime operation ${args['operation']}'); - }); const adapter = MethodChannelGetterAdapter(channel: channel); final update = await adapter.checkPackageForUpdate( @@ -295,23 +381,24 @@ void main() { final operations = []; TestDefaultBinaryMessengerBinding.instance.defaultBinaryMessenger .setMockMethodCallHandler(channel, (call) async { - final args = - (call.arguments as Map).cast(); - operations.add(args['operation']! as String); - return jsonEncode({ - 'ok': true, - 'command': 'runtime operation', - 'data': args['operation'] == 'task_list' || - args['operation'] == 'task_clean' - ? { - 'tasks': [ - _runtimeTaskJson('task-1', status: 'running') - ], - } - : _runtimeTaskJson('task-1', status: 'running'), - 'warnings': [], - }); - }); + final args = (call.arguments as Map) + .cast(); + operations.add(args['operation']! as String); + return jsonEncode({ + 'ok': true, + 'command': 'runtime operation', + 'data': + args['operation'] == 'task_list' || + args['operation'] == 'task_clean' + ? { + 'tasks': [ + _runtimeTaskJson('task-1', status: 'running'), + ], + } + : _runtimeTaskJson('task-1', status: 'running'), + 'warnings': [], + }); + }); const adapter = MethodChannelGetterAdapter(channel: channel); final tasks = await adapter.listRuntimeTasks(active: true); @@ -328,26 +415,26 @@ void main() { MethodCall? captured; TestDefaultBinaryMessengerBinding.instance.defaultBinaryMessenger .setMockMethodCallHandler(channel, (call) async { - captured = call; - return jsonEncode({ - 'ok': true, - 'command': 'runtime operation', - 'data': { - 'task_id': 'task-1', - 'package_id': 'android/org.fdroid.fdroid', - 'status': 'completed', - 'phase': {'category': 'completed'}, - 'capabilities': { - 'cancel': false, - 'pause': false, - 'resume': false, - 'retry': false, - }, - 'updated_at': 1, - }, - 'warnings': [], - }); - }); + captured = call; + return jsonEncode({ + 'ok': true, + 'command': 'runtime operation', + 'data': { + 'task_id': 'task-1', + 'package_id': 'android/org.fdroid.fdroid', + 'status': 'completed', + 'phase': {'category': 'completed'}, + 'capabilities': { + 'cancel': false, + 'pause': false, + 'resume': false, + 'retry': false, + }, + 'updated_at': 1, + }, + 'warnings': [], + }); + }); const adapter = MethodChannelGetterAdapter(channel: channel); final data = await adapter.invokeRuntimeOperation( @@ -363,90 +450,91 @@ void main() { expect(data['status'], 'completed'); }); - test('native adapter maps getter error envelope to bridge exception', - () async { - TestDefaultBinaryMessengerBinding.instance.defaultBinaryMessenger - .setMockMethodCallHandler(channel, (call) async { - return jsonEncode({ - 'ok': false, - 'command': call.method, - 'error': { - 'code': 'autogen.preview_error', - 'message': 'Preview failed', - 'detail': 'bad inventory', - }, - }); - }); + test( + 'native adapter maps getter error envelope to bridge exception', + () async { + TestDefaultBinaryMessengerBinding.instance.defaultBinaryMessenger + .setMockMethodCallHandler(channel, (call) async { + return jsonEncode({ + 'ok': false, + 'command': call.method, + 'error': { + 'code': 'autogen.preview_error', + 'message': 'Preview failed', + 'detail': 'bad inventory', + }, + }); + }); - const adapter = MethodChannelGetterAdapter(channel: channel); + const adapter = MethodChannelGetterAdapter(channel: channel); - await expectLater( - adapter.previewInstalledAutogen(), - throwsA( - isA().having( - (error) => error.error.code, - 'code', - 'autogen.preview_error', + await expectLater( + adapter.previewInstalledAutogen(), + throwsA( + isA().having( + (error) => error.error.code, + 'code', + 'autogen.preview_error', + ), ), - ), - ); - }); + ); + }, + ); } Map _runtimeTaskJson( String taskId, { required String status, -}) => - { - 'task_id': taskId, - 'package_id': 'android/org.fdroid.fdroid', - 'status': status, - 'phase': {'category': status}, - 'progress': null, - 'capabilities': { - 'cancel': true, - 'pause': false, - 'resume': false, - 'retry': false, - }, - 'current_diagnostic': null, - 'updated_at': 1, - }; +}) => { + 'task_id': taskId, + 'package_id': 'android/org.fdroid.fdroid', + 'status': status, + 'phase': {'category': status}, + 'progress': null, + 'capabilities': { + 'cancel': true, + 'pause': false, + 'resume': false, + 'retry': false, + }, + 'current_diagnostic': null, + 'updated_at': 1, +}; Map _previewJson() => { - 'operation': 'installed.preview', - 'target_repo_id': 'local_autogen', - 'target_repo_path': '/getter/repositories/local_autogen', - 'scan': { - 'stats': { - 'total_seen': 2, - 'returned': 1, - 'filtered_system': 1, - 'filtered_self': 0, - }, - 'diagnostics': [], - }, - 'summary': { - 'candidate_count': 1, - 'skipped_count': 0, - 'write_count': 1, - 'delete_count': 0, + 'operation': 'installed.preview', + 'target_repo_id': 'local_autogen', + 'target_repo_path': '/getter/repositories/local_autogen', + 'scan': { + 'stats': { + 'total_seen': 2, + 'returned': 1, + 'filtered_system': 1, + 'filtered_self': 0, + }, + 'diagnostics': [], + }, + 'summary': { + 'candidate_count': 1, + 'skipped_count': 0, + 'write_count': 1, + 'delete_count': 0, + }, + 'candidates': [ + { + 'package_id': 'android/com.example.autogen', + 'kind': 'android', + 'display_name': 'Example Autogen', + 'installed_target': { + 'kind': 'android_package', + 'package_name': 'com.example.autogen', }, - 'candidates': [ - { - 'package_id': 'android/com.example.autogen', - 'kind': 'android', - 'display_name': 'Example Autogen', - 'installed_target': { - 'kind': 'android_package', - 'package_name': 'com.example.autogen', - }, - 'action': 'create', - 'output_relative_path': 'packages/android/com.example.autogen.lua', - 'content_hash': 'fnv1a64:fake', - 'content': '-- fake generated content', - }, - ], - 'skipped': [], - 'diagnostics': [], - }; + 'action': 'create', + 'output_relative_path': 'packages/android/com.example.autogen.lua', + 'content_hash': 'fnv1a64:fake', + 'content': '-- fake generated content', + }, + ], + 'skipped': [], + 'diagnostics': [], +}; diff --git a/app_flutter/test/widget_test.dart b/app_flutter/test/widget_test.dart index 24cbe1c1..537b80f5 100644 --- a/app_flutter/test/widget_test.dart +++ b/app_flutter/test/widget_test.dart @@ -6,9 +6,11 @@ import 'package:upgradeall/legacy_migration_platform.dart'; import 'package:upgradeall/main.dart'; void main() { - testWidgets('fresh launch exposes home route and getter state', - (tester) async { + testWidgets('fresh launch exposes home route and getter state', ( + tester, + ) async { await tester.pumpWidget(const UpgradeAllApp()); + await tester.pumpAndSettle(); expect(find.byKey(AppKeys.homeRoute), findsOneWidget); expect(find.byKey(AppKeys.updateSummary), findsOneWidget); @@ -25,8 +27,10 @@ void main() { expect(find.byKey(AppKeys.appsRoute), findsOneWidget); expect(find.byKey(AppKeys.appsList), findsOneWidget); - expect(find.byKey(AppKeys.appRow('android/org.fdroid.fdroid')), - findsOneWidget); + expect( + find.byKey(AppKeys.appRow('android/org.fdroid.fdroid')), + findsOneWidget, + ); expect(find.text('Network'), findsOneWidget); await tester.tap(find.byKey(AppKeys.appRow('android/org.fdroid.fdroid'))); @@ -39,8 +43,9 @@ void main() { expect(find.text('Network access required'), findsOneWidget); }); - testWidgets('repository route lists priority ordered repository IDs', - (tester) async { + testWidgets('repository route lists priority ordered repository IDs', ( + tester, + ) async { await tester.pumpWidget(const UpgradeAllApp()); await tester.tap(find.byKey(AppKeys.openRepositories)); @@ -53,8 +58,9 @@ void main() { expect(find.byKey(AppKeys.repoRow('local_autogen')), findsOneWidget); }); - testWidgets('downloads route renders runtime task snapshots read-only', - (tester) async { + testWidgets('downloads route renders runtime task snapshots read-only', ( + tester, + ) async { await tester.pumpWidget(const UpgradeAllApp()); await tester.tap(find.byKey(AppKeys.openDownloads)); @@ -67,8 +73,9 @@ void main() { expect(find.text('Cancel'), findsOneWidget); }); - testWidgets('downloads route exposes getter empty task state', - (tester) async { + testWidgets('downloads route exposes getter empty task state', ( + tester, + ) async { await tester.pumpWidget( const UpgradeAllApp(getter: _NoTaskGetterAdapter()), ); @@ -80,8 +87,9 @@ void main() { expect(find.byKey(AppKeys.downloadsEmpty), findsOneWidget); }); - testWidgets('migration route imports prepared legacy DB through getter', - (tester) async { + testWidgets('migration route imports prepared legacy DB through getter', ( + tester, + ) async { final getter = _MigrationGetterAdapter(); await tester.pumpWidget( UpgradeAllApp( @@ -105,27 +113,30 @@ void main() { expect(find.text('migration.imported'), findsOneWidget); }); - testWidgets('migration route reports missing legacy DB from platform adapter', - (tester) async { - await tester.pumpWidget( - const UpgradeAllApp( - getter: _LegacyMigrationCapableGetterAdapter(), - legacyMigrationPlatform: _MissingLegacyMigrationPlatform(), - ), - ); + testWidgets( + 'migration route reports missing legacy DB from platform adapter', + (tester) async { + await tester.pumpWidget( + const UpgradeAllApp( + getter: _LegacyMigrationCapableGetterAdapter(), + legacyMigrationPlatform: _MissingLegacyMigrationPlatform(), + ), + ); - await tester.tap(find.byKey(AppKeys.openMigration)); - await tester.pumpAndSettle(); - await tester.tap(find.byKey(AppKeys.startLegacyMigration)); - await tester.pumpAndSettle(); + await tester.tap(find.byKey(AppKeys.openMigration)); + await tester.pumpAndSettle(); + await tester.tap(find.byKey(AppKeys.startLegacyMigration)); + await tester.pumpAndSettle(); - expect(find.byKey(AppKeys.migrationStatus), findsOneWidget); - expect(find.text('No legacy Room database found'), findsOneWidget); - expect(find.byKey(AppKeys.migrationImported), findsNothing); - }); + expect(find.byKey(AppKeys.migrationStatus), findsOneWidget); + expect(find.text('No legacy Room database found'), findsOneWidget); + expect(find.byKey(AppKeys.migrationImported), findsNothing); + }, + ); - testWidgets('installed autogen route previews and applies getter DTOs', - (tester) async { + testWidgets('installed autogen route previews and applies getter DTOs', ( + tester, + ) async { final getter = _AutogenRecordingGetterAdapter(); await tester.pumpWidget(UpgradeAllApp(getter: getter)); @@ -161,8 +172,9 @@ void main() { expect(getter.acceptedPackageIds, ['android/com.example.autogen']); }); - testWidgets('installed autogen route disables actions without bridge', - (tester) async { + testWidgets('installed autogen route disables actions without bridge', ( + tester, + ) async { await tester.pumpWidget( const UpgradeAllApp(getter: _NoInstalledAutogenGetterAdapter()), ); @@ -182,8 +194,9 @@ void main() { ); }); - testWidgets('migration route disables import when getter bridge is absent', - (tester) async { + testWidgets('migration route disables import when getter bridge is absent', ( + tester, + ) async { await tester.pumpWidget( const UpgradeAllApp( legacyMigrationPlatform: _PreparedLegacyMigrationPlatform( @@ -202,8 +215,9 @@ void main() { expect(find.byKey(AppKeys.migrationBridgeUnavailable), findsOneWidget); }); - testWidgets('placeholder routes expose stable empty-state keys', - (tester) async { + testWidgets('placeholder routes expose stable empty-state keys', ( + tester, + ) async { await tester.pumpWidget(const UpgradeAllApp()); await tester.tap(find.byKey(AppKeys.openLogs)); @@ -234,8 +248,7 @@ class _NoTaskGetterAdapter extends FakeGetterAdapter { Future> listRuntimeTasks({ bool active = false, String? packageId, - }) async => - const []; + }) async => const []; } class _LegacyMigrationCapableGetterAdapter extends FakeGetterAdapter { diff --git a/core-getter/src/main/java/net/xzos/upgradeall/getter/NativeLib.kt b/core-getter/src/main/java/net/xzos/upgradeall/getter/NativeLib.kt index 6cea7ace..d9d843be 100644 --- a/core-getter/src/main/java/net/xzos/upgradeall/getter/NativeLib.kt +++ b/core-getter/src/main/java/net/xzos/upgradeall/getter/NativeLib.kt @@ -20,6 +20,7 @@ class NativeLib { external fun applyInstalledAutogen(requestJson: String): String external fun importLegacyRoomDatabase(requestJson: String): String external fun legacyReportList(requestJson: String): String + external fun readOperation(requestJson: String): String external fun runtimeOperation(requestJson: String): String external fun drainRuntimeNotifications(): String diff --git a/core-getter/src/main/rust/api_proxy/src/lib.rs b/core-getter/src/main/rust/api_proxy/src/lib.rs index 01f2ee91..7e2e05c8 100644 --- a/core-getter/src/main/rust/api_proxy/src/lib.rs +++ b/core-getter/src/main/rust/api_proxy/src/lib.rs @@ -2,6 +2,7 @@ extern crate jni; use getter::operations::autogen::{self, AutogenAcceptance, AutogenOperationError}; use getter::operations::legacy_room::{self, LegacyRoomOperationError}; +use getter::operations::read_model::{self, ReadModelOperationError}; use getter::operations::runtime as runtime_operations; use getter::rpc::server::run_server_hanging; #[cfg(target_os = "android")] @@ -52,6 +53,14 @@ struct LegacyReportListRequest { data_dir: PathBuf, } +#[derive(Debug, Deserialize)] +struct ReadOperationRequest { + data_dir: PathBuf, + operation: String, + #[serde(default)] + payload: Value, +} + #[derive(Debug, Deserialize)] struct RuntimeOperationRequest { operation: String, @@ -255,6 +264,20 @@ pub extern "C" fn Java_net_xzos_upgradeall_getter_NativeLib_legacyReportList<'lo java_string_or_fallback(&mut env, response) } +#[no_mangle] +pub extern "C" fn Java_net_xzos_upgradeall_getter_NativeLib_readOperation<'local>( + mut env: JNIEnv<'local>, + _: JObject<'local>, + request_json: JString<'local>, +) -> JString<'local> { + let command = "read operation"; + let response = match jstring_to_string(&mut env, &request_json).and_then(read_operation) { + Ok(data) => success_envelope(command, data), + Err(error) => operation_error_envelope(command, error), + }; + java_string_or_fallback(&mut env, response) +} + fn preview_installed_autogen( env: &mut JNIEnv<'_>, context: &JObject<'_>, @@ -345,6 +368,25 @@ fn drain_runtime_notifications() -> Result { Ok(json!({ "notifications": notifications })) } +fn read_operation(request_json: String) -> Result { + let request: ReadOperationRequest = serde_json::from_str(&request_json) + .map_err(|source| BridgeOperationError::InvalidRequest(source.to_string()))?; + let payload = if request.payload.is_null() { + "{}".to_owned() + } else { + request.payload.to_string() + }; + match request.operation.as_str() { + "repository_list" => read_model::repository_list_json(&request.data_dir), + "tracked_package_list" => read_model::tracked_package_list_json(&request.data_dir), + "package_eval" => read_model::package_eval_json(&request.data_dir, &payload), + other => Err(ReadModelOperationError::InvalidRequest(format!( + "unsupported read operation '{other}'" + ))), + } + .map_err(BridgeOperationError::ReadModel) +} + fn runtime_operation(request_json: String) -> Result { let runtime = init_getter_runtime(); let mut runtime = runtime @@ -532,6 +574,8 @@ enum BridgeOperationError { Autogen(String), #[error("migration error: {0}")] Migration(#[from] LegacyRoomOperationError), + #[error("read model error: {0}")] + ReadModel(#[from] ReadModelOperationError), #[error("runtime error: {0}")] Runtime(#[from] runtime_operations::RuntimeOperationError), #[error("runtime lock is poisoned")] @@ -609,6 +653,7 @@ impl BridgeOperationError { .detail() .or_else(|| error.report_path().map(|path| path.display().to_string())), ), + Self::ReadModel(error) => (error.code(), error.message(), error.detail()), Self::Runtime(error) => (error.code(), error.message(), error.detail()), Self::RuntimePoisoned => ("runtime.poisoned", "Getter runtime lock is poisoned", None), Self::RuntimeNotificationQueuePoisoned => ( @@ -672,6 +717,48 @@ mod tests { } } + #[test] + fn read_operation_lists_repositories_and_evaluates_packages() { + let temp = tempfile::tempdir().unwrap(); + let data_dir = temp.path().join("data"); + let repo_root = temp.path().join("repo"); + write_static_update_repo(&repo_root); + let db = open_main_db(&data_dir).unwrap(); + db.upsert_repository( + &RepositoryMetadata { + id: "official".parse().unwrap(), + name: "Official".to_owned(), + priority: RepositoryPriority::new(0), + api_version: REPO_API_VERSION_V1.to_owned(), + }, + Some(&repo_root), + None, + ) + .unwrap(); + + let repositories = read_operation( + json!({ + "operation": "repository_list", + "data_dir": data_dir, + }) + .to_string(), + ) + .expect("repository list"); + assert_eq!(repositories["repositories"][0]["id"], "official"); + + let package = read_operation( + json!({ + "operation": "package_eval", + "data_dir": data_dir, + "payload": { "package_id": "android/org.fdroid.fdroid" } + }) + .to_string(), + ) + .expect("package eval"); + assert_eq!(package["package"]["id"], "android/org.fdroid.fdroid"); + assert_eq!(package["package"]["repository"], "official"); + } + #[test] fn runtime_dispatcher_issues_action_from_registered_package_update_check() { let temp = tempfile::tempdir().unwrap(); diff --git a/core-getter/src/main/rust/getter b/core-getter/src/main/rust/getter index 1cde4f4b..60a65158 160000 --- a/core-getter/src/main/rust/getter +++ b/core-getter/src/main/rust/getter @@ -1 +1 @@ -Subproject commit 1cde4f4bf01c5f703724f4ca62cc89496886eedd +Subproject commit 60a65158aed40e14f0dea17427b4a92ec1e43818 diff --git a/docs/architecture/adr/0007-flutter-getter-bridge-contract.md b/docs/architecture/adr/0007-flutter-getter-bridge-contract.md index 9c1253eb..242139ba 100644 --- a/docs/architecture/adr/0007-flutter-getter-bridge-contract.md +++ b/docs/architecture/adr/0007-flutter-getter-bridge-contract.md @@ -49,7 +49,7 @@ The Android product APK packages a slim `:getter_bridge` library under `app_flut Internally, Rust/native bridge code scans Android inventory through the platform adapter, then asks getter-owned shared autogen operations to plan/apply `local_autogen`. `MethodChannelGetterAdapter` consumes the returned getter-style JSON envelopes. Flutter renders getter-owned preview/apply DTOs and scan diagnostics, then passes the package ids from displayed accepted preview candidates back to getter on apply; it must not expose a product Dart `InstalledInventoryPlatform` scanner or convert Android package names into package ids. -`loadSnapshot()` composes the smaller getter-owned operations into the UI shell's first snapshot DTO. It must not perform repository resolution, version comparison, migration mapping, or update selection in Dart. `readMigrationReports()` must go through a getter operation such as `legacy report-list`; Flutter must not inspect getter's data-directory layout directly. Runtime task rendering must use ADR-0011 runtime task snapshot APIs and opaque action/task controls; Flutter must not synthesize task states, retry policy, installer behavior, or update decisions. +`loadSnapshot()` composes smaller getter-owned read-model operations into the UI shell's first snapshot DTO. In the Android/native product path, `MethodChannelGetterAdapter` calls `readOperation` for `repository_list`, `tracked_package_list`, and `package_eval`; Rust getter reads SQLite repository/tracked-package state and evaluates registered Lua packages. Flutter may parse and combine those returned DTOs for rendering, but must not perform repository resolution, Lua validation/evaluation, version comparison, migration mapping, or update selection in Dart. `readMigrationReports()` must go through a getter operation such as `legacy report-list`; Flutter must not inspect getter's data-directory layout directly. Runtime task rendering must use ADR-0011 runtime task snapshot APIs and opaque action/task controls; Flutter must not synthesize task states, retry policy, installer behavior, or update decisions. ## Flutter DTOs From 1a5d8226d82ff9ffa16b33b6bdec815441666d30 Mon Sep 17 00:00:00 2001 From: Xiangzhe Date: Thu, 25 Jun 2026 19:20:17 +0800 Subject: [PATCH 47/85] feat(app): submit update checks to runtime --- app_flutter/lib/cli_getter_adapter.dart | 5 + app_flutter/lib/getter_adapter.dart | 7 + app_flutter/lib/main.dart | 130 ++++++++++- app_flutter/lib/native_getter_adapter.dart | 1 + app_flutter/test/widget_test.dart | 214 ++++++++++++++++++ .../flutter-ui-feature-parity-and-testing.md | 3 +- .../0007-flutter-getter-bridge-contract.md | 2 + 7 files changed, 356 insertions(+), 6 deletions(-) diff --git a/app_flutter/lib/cli_getter_adapter.dart b/app_flutter/lib/cli_getter_adapter.dart index ea0ee05f..4042b011 100644 --- a/app_flutter/lib/cli_getter_adapter.dart +++ b/app_flutter/lib/cli_getter_adapter.dart @@ -119,6 +119,11 @@ class CliGetterAdapter implements GetterAdapter { ); } + @override + Stream runtimeNotificationEnvelopes() { + return const Stream.empty(); + } + @override Future> listRuntimeTasks({ bool active = false, diff --git a/app_flutter/lib/getter_adapter.dart b/app_flutter/lib/getter_adapter.dart index e631dabf..8272f402 100644 --- a/app_flutter/lib/getter_adapter.dart +++ b/app_flutter/lib/getter_adapter.dart @@ -40,6 +40,8 @@ abstract interface class GetterAdapter { Future submitRuntimeAction(String actionId); + Stream runtimeNotificationEnvelopes(); + Future> listRuntimeTasks({ bool active = false, String? packageId, @@ -282,6 +284,11 @@ class FakeGetterAdapter implements GetterAdapter { return RuntimeTaskSnapshot.fromJson(_runtimeTaskJson('task-1')); } + @override + Stream runtimeNotificationEnvelopes() { + return const Stream.empty(); + } + @override Future> listRuntimeTasks({ bool active = false, diff --git a/app_flutter/lib/main.dart b/app_flutter/lib/main.dart index 97c48322..794430e7 100644 --- a/app_flutter/lib/main.dart +++ b/app_flutter/lib/main.dart @@ -1,3 +1,5 @@ +import 'dart:async'; + import 'package:flutter/material.dart'; import 'getter_adapter.dart'; @@ -46,6 +48,10 @@ class AppKeys { static const applyInstalledAutogen = ValueKey( 'action.apply_installed_autogen', ); + static const updateCheckStatus = ValueKey( + 'state.update_check_status', + ); + static const updateCheckError = ValueKey('state.update_check_error'); static const updateSummary = ValueKey('state.update_summary'); static const getterStatus = ValueKey('state.getter_status'); @@ -94,6 +100,8 @@ class AppKeys { 'state.installed_autogen_error', ); + static ValueKey checkPackageUpdate(String packageId) => + ValueKey('action.check_update.$packageId'); static ValueKey appRow(String packageId) => ValueKey('state.app.$packageId'); static ValueKey repoRow(String repositoryId) => @@ -147,7 +155,7 @@ class UpgradeAllApp extends StatelessWidget { if (settings.name == '/apps/detail') { final app = settings.arguments! as AppSummary; return MaterialPageRoute( - builder: (context) => AppDetailPage(app: app), + builder: (context) => AppDetailPage(app: app, getter: getter), settings: settings, ); } @@ -306,13 +314,67 @@ class _AppsPageState extends State { } } -class AppDetailPage extends StatelessWidget { - const AppDetailPage({super.key, required this.app}); +class AppDetailPage extends StatefulWidget { + const AppDetailPage({super.key, required this.app, required this.getter}); final AppSummary app; + final GetterAdapter getter; + + @override + State createState() => _AppDetailPageState(); +} + +class _AppDetailPageState extends State { + bool _checkingUpdate = false; + String? _status; + String? _error; + + Future _checkForUpdate() async { + if (_checkingUpdate) return; + setState(() { + _checkingUpdate = true; + _status = 'Checking for updates...'; + _error = null; + }); + + try { + final result = await widget.getter.checkPackageForUpdate( + widget.app.id, + installedVersion: _knownVersion(widget.app.installedVersion), + ); + final action = result.action; + if (action == null) { + if (!mounted) return; + setState(() { + _status = 'No update task available: ${result.update.status}'; + }); + return; + } + + final task = await widget.getter.submitRuntimeAction(action.actionId); + if (!mounted) return; + setState(() { + _status = 'Submitted runtime task ${task.taskId}'; + }); + await Navigator.of(context).pushNamed('/downloads'); + } catch (error) { + if (!mounted) return; + setState(() { + _status = null; + _error = error.toString(); + }); + } finally { + if (mounted) { + setState(() { + _checkingUpdate = false; + }); + } + } + } @override Widget build(BuildContext context) { + final app = widget.app; return Scaffold( key: AppKeys.appDetailRoute, appBar: AppBar(title: Text(app.name)), @@ -323,6 +385,25 @@ class AppDetailPage extends StatelessWidget { const SizedBox(height: 12), Text('Installed: ${app.installedVersion}'), Text('Latest: ${app.latestVersion}'), + const SizedBox(height: 16), + FilledButton.icon( + key: AppKeys.checkPackageUpdate(app.id), + onPressed: _checkingUpdate ? null : _checkForUpdate, + icon: const Icon(Icons.system_update_alt), + label: Text( + _checkingUpdate ? 'Checking update...' : 'Check update', + ), + ), + if (_status != null) + Padding( + padding: const EdgeInsets.only(top: 12), + child: Text(key: AppKeys.updateCheckStatus, _status!), + ), + if (_error != null) + Padding( + padding: const EdgeInsets.only(top: 12), + child: Text(key: AppKeys.updateCheckError, _error!), + ), if (app.hasFreeNetworkWarning) const Padding( padding: EdgeInsets.only(top: 12), @@ -337,6 +418,11 @@ class AppDetailPage extends StatelessWidget { } } +String? _knownVersion(String version) { + final normalized = version.trim(); + return normalized.isEmpty || normalized == 'unknown' ? null : normalized; +} + class RepositoriesPage extends StatefulWidget { const RepositoriesPage({super.key, required this.getter}); @@ -383,18 +469,52 @@ class _RepositoriesPageState extends State { } } -class DownloadsPage extends StatelessWidget { +class DownloadsPage extends StatefulWidget { const DownloadsPage({super.key, required this.getter}); final GetterAdapter getter; + @override + State createState() => _DownloadsPageState(); +} + +class _DownloadsPageState extends State { + late Future> _tasks = widget.getter + .listRuntimeTasks(); + StreamSubscription? _notificationSubscription; + + @override + void initState() { + super.initState(); + _notificationSubscription = widget.getter + .runtimeNotificationEnvelopes() + .listen((notification) { + if (notification.kind == 'task_changed') { + _reloadTasks(); + } + }, onError: (_) {}); + } + + @override + void dispose() { + _notificationSubscription?.cancel(); + super.dispose(); + } + + void _reloadTasks() { + if (!mounted) return; + setState(() { + _tasks = widget.getter.listRuntimeTasks(); + }); + } + @override Widget build(BuildContext context) { return Scaffold( key: AppKeys.downloadsRoute, appBar: AppBar(title: const Text('Downloads')), body: FutureBuilder>( - future: getter.listRuntimeTasks(), + future: _tasks, builder: (context, snapshot) { if (snapshot.connectionState != ConnectionState.done) { return const Center(child: CircularProgressIndicator()); diff --git a/app_flutter/lib/native_getter_adapter.dart b/app_flutter/lib/native_getter_adapter.dart index e2a731b0..bbaa21e5 100644 --- a/app_flutter/lib/native_getter_adapter.dart +++ b/app_flutter/lib/native_getter_adapter.dart @@ -118,6 +118,7 @@ class MethodChannelGetterAdapter extends FakeGetterAdapter { }); } + @override Stream runtimeNotificationEnvelopes() { return runtimeNotifications().map(RuntimeNotificationEnvelope.fromJson); } diff --git a/app_flutter/test/widget_test.dart b/app_flutter/test/widget_test.dart index 537b80f5..b1d56d0f 100644 --- a/app_flutter/test/widget_test.dart +++ b/app_flutter/test/widget_test.dart @@ -1,3 +1,5 @@ +import 'dart:async'; + import 'package:flutter/material.dart'; import 'package:flutter_test/flutter_test.dart'; @@ -43,6 +45,53 @@ void main() { expect(find.text('Network access required'), findsOneWidget); }); + testWidgets('app detail submits getter-issued update action to runtime', ( + tester, + ) async { + final getter = _UpdateCheckRecordingGetterAdapter(); + await tester.pumpWidget(UpgradeAllApp(getter: getter)); + + await tester.tap(find.byKey(AppKeys.openApps)); + await tester.pumpAndSettle(); + await tester.tap(find.byKey(AppKeys.appRow('android/org.fdroid.fdroid'))); + await tester.pumpAndSettle(); + await tester.tap( + find.byKey(AppKeys.checkPackageUpdate('android/org.fdroid.fdroid')), + ); + await tester.pumpAndSettle(); + + expect(getter.checkedPackageId, 'android/org.fdroid.fdroid'); + expect(getter.checkedInstalledVersion, '1.20.0'); + expect(getter.submittedActionId, 'action-from-getter'); + expect(find.byKey(AppKeys.downloadsRoute), findsOneWidget); + expect( + find.byKey(AppKeys.downloadTaskRow('task-from-action')), + findsOneWidget, + ); + }); + + testWidgets('app detail reports update checks without runtime action', ( + tester, + ) async { + await tester.pumpWidget( + UpgradeAllApp(getter: _NoUpdateActionGetterAdapter()), + ); + + await tester.tap(find.byKey(AppKeys.openApps)); + await tester.pumpAndSettle(); + await tester.tap(find.byKey(AppKeys.appRow('android/org.fdroid.fdroid'))); + await tester.pumpAndSettle(); + await tester.tap( + find.byKey(AppKeys.checkPackageUpdate('android/org.fdroid.fdroid')), + ); + await tester.pumpAndSettle(); + + expect(find.byKey(AppKeys.appDetailRoute), findsOneWidget); + expect(find.byKey(AppKeys.updateCheckStatus), findsOneWidget); + expect(find.text('No update task available: up_to_date'), findsOneWidget); + expect(find.byKey(AppKeys.downloadsRoute), findsNothing); + }); + testWidgets('repository route lists priority ordered repository IDs', ( tester, ) async { @@ -73,6 +122,23 @@ void main() { expect(find.text('Cancel'), findsOneWidget); }); + testWidgets('downloads route refreshes after runtime notification', ( + tester, + ) async { + final getter = _NotificationRefreshingGetterAdapter(); + await tester.pumpWidget(UpgradeAllApp(getter: getter)); + + await tester.tap(find.byKey(AppKeys.openDownloads)); + await tester.pumpAndSettle(); + expect(find.text('queued • queued'), findsOneWidget); + + getter.emitRunningTaskNotification(); + await tester.pumpAndSettle(); + + expect(find.text('running • download'), findsOneWidget); + expect(getter.listCallCount, 2); + }); + testWidgets('downloads route exposes getter empty task state', ( tester, ) async { @@ -241,6 +307,154 @@ void main() { }); } +class _UpdateCheckRecordingGetterAdapter extends FakeGetterAdapter { + String? checkedPackageId; + String? checkedInstalledVersion; + String? submittedActionId; + final _tasks = []; + + @override + Future checkPackageForUpdate( + String packageId, { + String? repositoryId, + String? installedVersion, + String? pinVersion, + }) async { + checkedPackageId = packageId; + checkedInstalledVersion = installedVersion; + return RuntimeUpdateCheckResult.fromJson({ + 'package': { + 'id': packageId, + 'name': 'F-Droid', + 'repository': repositoryId ?? 'official', + }, + 'update': { + 'package_id': packageId, + 'status': 'update_available', + 'installed_version': installedVersion, + 'effective_local_version': installedVersion, + 'selected': { + 'candidate': {'version': '1.21.0'}, + }, + 'actions': [ + {'type': 'download'}, + ], + }, + 'action': { + 'action_id': 'action-from-getter', + 'package_id': packageId, + }, + }); + } + + @override + Future submitRuntimeAction(String actionId) async { + submittedActionId = actionId; + final task = RuntimeTaskSnapshot.fromJson(const { + 'task_id': 'task-from-action', + 'package_id': 'android/org.fdroid.fdroid', + 'status': 'queued', + 'phase': {'category': 'queued'}, + 'progress': null, + 'capabilities': { + 'cancel': true, + 'pause': false, + 'resume': false, + 'retry': false, + }, + 'current_diagnostic': null, + 'updated_at': 42, + }); + _tasks + ..clear() + ..add(task); + return task; + } + + @override + Future> listRuntimeTasks({ + bool active = false, + String? packageId, + }) async => List.unmodifiable(_tasks); +} + +class _NoUpdateActionGetterAdapter extends FakeGetterAdapter { + const _NoUpdateActionGetterAdapter(); + + @override + Future checkPackageForUpdate( + String packageId, { + String? repositoryId, + String? installedVersion, + String? pinVersion, + }) async { + return RuntimeUpdateCheckResult.fromJson({ + 'package': { + 'id': packageId, + 'name': 'F-Droid', + 'repository': repositoryId ?? 'official', + }, + 'update': { + 'package_id': packageId, + 'status': 'up_to_date', + 'installed_version': installedVersion, + 'effective_local_version': installedVersion, + 'selected': null, + 'actions': [], + }, + 'action': null, + }); + } +} + +class _NotificationRefreshingGetterAdapter extends FakeGetterAdapter { + final _notifications = + StreamController.broadcast(); + var _running = false; + var listCallCount = 0; + + @override + Future> listRuntimeTasks({ + bool active = false, + String? packageId, + }) async { + listCallCount += 1; + return [_task(_running ? 'running' : 'queued')]; + } + + @override + Stream runtimeNotificationEnvelopes() { + return _notifications.stream; + } + + void emitRunningTaskNotification() { + _running = true; + _notifications.add( + RuntimeNotificationEnvelope(kind: 'task_changed', task: _task('running')), + ); + } + + RuntimeTaskSnapshot _task(String status) { + return RuntimeTaskSnapshot.fromJson({ + 'task_id': 'task-refresh', + 'package_id': 'android/org.fdroid.fdroid', + 'status': status, + 'phase': { + 'category': status == 'running' ? 'download' : 'queued', + }, + 'progress': null, + 'capabilities': { + 'cancel': true, + 'pause': false, + 'resume': false, + 'retry': false, + }, + 'current_diagnostic': null, + 'updated_at': status == 'running' ? 2 : 1, + }); + } +} + class _NoTaskGetterAdapter extends FakeGetterAdapter { const _NoTaskGetterAdapter(); diff --git a/docs/app/flutter-ui-feature-parity-and-testing.md b/docs/app/flutter-ui-feature-parity-and-testing.md index 9a5b4a6e..d0e6cdbd 100644 --- a/docs/app/flutter-ui-feature-parity-and-testing.md +++ b/docs/app/flutter-ui-feature-parity-and-testing.md @@ -82,7 +82,8 @@ The first Flutter implementation slice is intentionally a shell, not product log - Product decisions such as repository resolution, updates, migrations, storage, and downloads still belong in Rust getter. - Installed-autogen product flows must call getter/native bridge operations that use the Rust-active Android platform adapter from ADR-0009; Flutter should not lead PackageManager inventory scanning through a Dart MethodChannel API. - CI/release APK artifacts must be built from `app_flutter`, not from the legacy `:app` module. -- The downloads route may render getter task/event DTOs read-only, but it must not implement a Dart download task state machine, retry policy, or installer semantics. +- The app detail update button may call getter's typed update-check operation, receive a getter-issued opaque `action_id`, submit that `action_id`, and open Downloads. Flutter must not assemble or echo action payloads. +- The downloads route may render getter task/event DTOs read-only and refresh after `RuntimeNotification.task_changed`, but it must not implement a Dart download task state machine, retry policy, or installer semantics. Current-state runtime queries remain authoritative. ## Test pyramid diff --git a/docs/architecture/adr/0007-flutter-getter-bridge-contract.md b/docs/architecture/adr/0007-flutter-getter-bridge-contract.md index 242139ba..05608caa 100644 --- a/docs/architecture/adr/0007-flutter-getter-bridge-contract.md +++ b/docs/architecture/adr/0007-flutter-getter-bridge-contract.md @@ -51,6 +51,8 @@ Internally, Rust/native bridge code scans Android inventory through the platform `loadSnapshot()` composes smaller getter-owned read-model operations into the UI shell's first snapshot DTO. In the Android/native product path, `MethodChannelGetterAdapter` calls `readOperation` for `repository_list`, `tracked_package_list`, and `package_eval`; Rust getter reads SQLite repository/tracked-package state and evaluates registered Lua packages. Flutter may parse and combine those returned DTOs for rendering, but must not perform repository resolution, Lua validation/evaluation, version comparison, migration mapping, or update selection in Dart. `readMigrationReports()` must go through a getter operation such as `legacy report-list`; Flutter must not inspect getter's data-directory layout directly. Runtime task rendering must use ADR-0011 runtime task snapshot APIs and opaque action/task controls; Flutter must not synthesize task states, retry policy, installer behavior, or update decisions. +The first product click-through update flow is: App detail calls a typed getter update-check operation, receives a getter-issued `action_id`, submits only that `action_id` to the process-lifetime runtime, and opens Downloads to query authoritative task snapshots. Flutter may refresh the Downloads page after `RuntimeNotification.task_changed`, but the notification is only a trigger; `task_list`/equivalent runtime queries remain the source of truth. + ## Flutter DTOs The Flutter shell may use DTOs that mirror getter output for rendering: From 239f98fabdb275f357c48ba808b37edfce53306b Mon Sep 17 00:00:00 2001 From: Xiangzhe Date: Fri, 26 Jun 2026 16:14:32 +0800 Subject: [PATCH 48/85] docs(architecture): define provider autogen repository model --- CONTEXT.md | 72 ++- docs/README.md | 9 +- docs/architecture/README.md | 1 + ...pp-centric-lua-package-repository-model.md | 131 +++-- .../adr/0003-legacy-room-migration.md | 2 +- docs/architecture/adr/0005-lua-package-api.md | 24 +- ...06-package-centric-cli-command-contract.md | 2 +- .../0007-flutter-getter-bridge-contract.md | 2 +- ...platform-adapter-and-package-visibility.md | 6 +- ...age-metadata-cache-and-version-baseline.md | 2 +- ...ed-provider-modules-and-autogen-refresh.md | 500 ++++++++++++++++++ .../upgradeall-getter-rewrite-wiki.md | 280 +++++----- docs/lua-api/package-lifecycle.md | 42 +- docs/lua-api/permissions.md | 35 +- docs/lua-api/repository-layout.md | 237 +++++++-- docs/lua-api/templates.md | 96 ++-- todo.md | 27 +- 17 files changed, 1181 insertions(+), 287 deletions(-) create mode 100644 docs/architecture/adr/0012-getter-owned-provider-modules-and-autogen-refresh.md diff --git a/CONTEXT.md b/CONTEXT.md index 40f67c8e..a2566366 100644 --- a/CONTEXT.md +++ b/CONTEXT.md @@ -2,6 +2,74 @@ ## Glossary +### Repository root + +The local directory that contains all enabled UpgradeAll repositories. Its direct child directories are repository aliases, for example `repo/official`, `repo/local`, or `repo/autogen`; `repo/metadata.jsonc` is the only current repo-root reserved entry and is getter-owned repository registry/config state rather than a repository. At the repository root, every direct child directory is a repository alias; future repo-root reserved entries require an explicit design/ADR because they occupy alias namespace. Other non-directory root contents are outside getter domain. `repo/metadata.jsonc` stores repository-related local settings only, currently repository priority/order rules and the autogen output target. If it is missing, getter uses built-in priority defaults: `local` = 100, `autogen` = -1, every other alias = 0, with same-priority aliases resolved in lexicographic order. If present, the priority map is lookup-only: getter discovers actual repository alias directories first, then queries the map by alias; entries for nonexistent aliases are inert and do not warn, create repositories, display repositories, or participate in sorting. The generated repository target defaults to `generated_repository = "autogen"` when omitted; generated starter config should include this default as a comment users may uncomment/change. When autogen runs with target `autogen`, getter creates `repo/autogen/` if needed. If `generated_repository` is set to any other alias, that target directory must already exist or autogen apply reports a configuration error. If `repo/metadata.jsonc` exists but cannot be parsed, getter reports a configuration diagnostic instead of silently falling back. + +### Runtime configuration root + +The getter local runtime/config policy root, named `rc/`. It is a top-level sibling of `repo/` under the getter data directory, for example `/rc/` beside `/repo/`, `main.db`, and `cache.db`. It is not a repository root and does not participate in repository or package discovery. Current defined content is `rc/hook/*.lua` for runtime hook policy. Future runtime/local policy such as environment, credential, or network behavior belongs under `rc/`, not in `repo/metadata.jsonc`, which remains repository-related registry/config. + +### UpgradeAll repository + +A filesystem package repository rooted at `repo/`. The `` directory name is the local repository alias; if a user clones or renames the official repository to `repo/a`, this installation refers to it as `a`. Getter only considers explicit repository entries: reserved repository-root directories such as `.metadata/` and `luaclass/`, plus directory paths that form package paths. Reserved directories are handled only by their own responsibility and never participate in package discovery; package paths cannot begin with reserved names such as `.metadata` or `luaclass`. Future repository-root reserved directories follow the same rule so the repo layout remains organizable. A directory that directly contains `metadata.jsonc` declares a package boundary. `.autogen.jsonc` does not declare a package boundary; it is only a generated-package ownership record inside a package directory that already has `metadata.jsonc`. If `metadata.jsonc` parses correctly as package metadata, the directory is a valid package directory; if parsing fails, getter reports an invalid package metadata diagnostic for that package path. In both cases, that directory is the package path endpoint, and getter does not discover nested packages below it. Other repository-alias contents such as README/docs/random files are outside getter domain entirely, not parsed, validated, displayed, warned about, or modeled as ignored managed objects. A repository contains package directories, shared Lua classes, metadata, autogen definitions, manifests, and package version scripts. The local alias gives users freedom to fork or maintain an intermediate repository layer; it is not the repository's security/trust identity. This term is distinct from upstream provider catalogs such as F-Droid indexes or GitHub APIs. + +### Package path + +A repository-local package identity derived from package directory hierarchy, modeled after Gentoo category/package atoms but allowing multiple category segments, for example `android/app/com.example.app`, `android/magisk/hello`, or `android/f-droid/magisk/hello`. Package Lua does not declare a duplicate package id field; getter derives the package path from the directory. UpgradeAll/getter domain strings, including package paths and aliases, are treated as UTF-8. Getter does not detect or convert other filesystem/text encodings; inputs in other encodings are still interpreted as UTF-8. + +### Qualified package atom + +A package reference in the form `[::repo-name]`, for example `android/f-droid/magisk/hello` or `android/f-droid/magisk/hello::official`. If `::repo-name` is omitted, getter resolves the package path by repository priority from `repo/metadata.jsonc`. If `::repo-name` is present, getter resolves only that local repository alias. Explicit alias references/imports intentionally depend on the local alias: if a user renames `repo/official` to `repo/a`, references to `::official` break, preserving the user's freedom to fork or replace that intermediate layer intentionally. + +### Repository trust identity + +The publishable/signable repository identity declared by repository metadata such as `repo//.metadata/metadata.jsonc`, including origin URL, maintainers, co-maintainers, and signing keys/signatures. Security/trust checks must use this metadata and verified signatures, not the local repository alias. If `.metadata/metadata.jsonc` is missing, the repository may still be used as unverified/local-source content, but repo update, signature, and trust operations are unavailable. If `.metadata/metadata.jsonc` exists but cannot be parsed, getter reports a repository metadata diagnostic. Renaming `repo/official` to `repo/a` changes local references and priority keys but should not make an untrusted repository trusted merely because of its directory name. Repository source files such as metadata, version scripts, package `files/`, `luaclass/`, autogen scripts, `.autogen.jsonc`, and manifests are protected by the repository's Git/signing/maintainer trust model, not by package `Manifest`; package `Manifest` only manages external network/dynamic-download content fetched at package-version execution time. Since `Manifest` and `.autogen.jsonc` are sibling package-directory files, `Manifest` cannot architecturally protect `.autogen.jsonc` or other repository source files. + +### Android package name + +The platform installation identity for an Android application, stored in package metadata as `android.package_name`, for example `com.example.app`. This is distinct from an UpgradeAll package path such as `android/app/com.example.app` and from a qualified package atom such as `android/app/com.example.app::official`. Metadata should avoid using the ambiguous field name `package_id` for this value. + +### Package version script + +A Lua script inside a package directory that describes one concrete version. Getter discovers package version scripts only from direct child files of the package directory whose basename ends with `.lua` and does not start with `.`. Removing the `.lua` suffix yields the literal version string; getter does not require SemVer or otherwise constrain the version syntax at discovery time. Examples include `1.2.3.lua`, `1.2.3-r1.lua`, `v1.2.3.lua`, `2026.06.25.lua`, and `9999.lua`. Version scripts live beside `metadata.jsonc`, `Manifest`, and optional package-local helper files under `files/`; there is no extra `versions/` directory in the accepted repository layout. Every enabled Lua script must start with an explicit interpreter/API-version line such as `#!/bin/upa-lua v1`; the API version cannot be omitted or defaulted. Any Lua file whose basename starts with `.` is excluded from Lua discovery, for example `.9999.lua` or `.10-http-rewrite.lua`: getter does not parse, validate, execute, display, or apply permission metadata to it as Lua. This dot-prefix rule is only for Lua file discovery; non-Lua package files are governed by the explicit getter file whitelist, so `.autogen.jsonc` is managed when present in a generated package. Script permissions are declared per enabled file in package `metadata.jsonc`, not inferred solely from the filename. + +### Live package script + +A special package version script named `9999.lua`, modeled after Gentoo live ebuilds. It represents live/floating upstream behavior and commonly needs free network access, but high-risk/free-network status is declared by package metadata per Lua file rather than by the `9999.lua` name alone. A `9999.lua` file can be marked as not needing free network, and a fixed-version script can be marked high-risk if its metadata grants `allow_free_network`. Getter/UI must surface scripts with free-network permission as higher risk: fixed-version Manifest checks do not prove that freely fetched live/dynamic upstream content was not changed, compromised, or incompletely downloaded. + +### Lua script permission metadata + +Package `metadata.jsonc` declares permissions per enabled Lua version script using a filename-keyed map, for example `lua: { "9999.lua": { permission: ["allow_free_network"] } }`. The map is lookup-only: getter first discovers an enabled Lua file from the filesystem, then queries this map by that file's basename. Getter does not enumerate the map to discover scripts or warnings. `allow_free_network` grants that enabled script free network access and is the source of the user-visible high-risk warning when getter runs Lua or displays enabled version Lua to the user. The permission can apply to `9999.lua` or to any fixed-version script; conversely, `9999.lua` is not high-risk merely by filename if metadata does not grant free-network permission. A version script omitted from the `lua` map defaults to `permission: []`. Permission entries for nonexistent files or dot-prefixed Lua files are inert: they do not enable, display, validate, or otherwise bring those files under getter management. + +### Package Manifest + +A package-directory file named `Manifest` that records allowed hashes for external network/dynamic-download response bodies used by that package's version scripts. Each line is ` [optional-name]`: the hash is authoritative, while the optional name is for humans/debugging because a URL may not reveal the final returned file name. A missing `Manifest` is equivalent to an empty hash set, not an invalid package. For package version scripts without `allow_free_network`, an externally fetched data file or API response body is usable only if its SHA-512 hash appears in that package's `Manifest`; an unlisted or mismatching body fails validation, so a missing/empty `Manifest` means such network fetches cannot succeed. Scripts that do not fetch external network content, or only read package-local `files/`, do not need a `Manifest`. Scripts with `allow_free_network` are not blocked by `Manifest` membership but remain high-risk. `Manifest` belongs only to package directories and package Lua/version-script execution. Repository-level autogen scripts under `.metadata/autogen/` do not have a `Manifest`, but an autogen script that creates package directories must generate correct package `Manifest` files for every generated package it expects to work without `allow_free_network`. `Manifest` is not a repository source manifest: it does not protect `metadata.jsonc`, Lua version scripts, `files/`, sibling `.autogen.jsonc`, `luaclass/`, repository metadata, or autogen scripts. Those source files are protected by the repository Git/signing/maintainer trust model. Since `Manifest` and `.autogen.jsonc` are same-level package files, `Manifest` cannot architecturally protect `.autogen.jsonc`. `Manifest` also does not make `allow_free_network`/live upstream behavior reproducible or safe. + +### Package-local files + +A package directory may contain a `files/` subdirectory for package-local helper data. Package Lua may read files under its own package directory's `files/` subtree through a package-scoped getter host API such as `read_package_file(path)`, where `path` is relative to `files/`. The original built-in implementation, `getter_builtin.read_package_file`, does not expose real filesystem paths or a general `io.open` escape hatch: absolute paths, `..`, directories, cross-package reads, and arbitrary repository reads are rejected. `read_package_file(path)` returns a Lua string; getter does not interpret encoding, MIME type, JSON, or text-vs-binary mode. Hook code may still wrap the public `read_package_file()` name as local user policy; getter core/CLI does not maintain a protective denylist of hookable public functions. File names and formats inside `files/` are package-owned and getter does not assign product semantics to them. Package directory contents outside getter's explicit discovery set (`metadata.jsonc`, optional generated-package `.autogen.jsonc`, `Manifest`, enabled direct-child `*.lua`, and `files/`) are outside getter domain entirely: getter does not parse, validate, display, warn about, or model them as ignored managed objects. The primary reason is clear responsibility boundaries; a smaller getter-core attack surface is a beneficial side effect. This keeps repository layout structured while preserving the repo-trust model: repository source review/signing covers these helper files, and users who do not trust a repository should not use it except by copying/authoring content into a repository they control. + +### Getter hook script + +A user-controlled getter local Lua hook under `rc/hook/*.lua`, analogous to an emerge bashrc-style hook and UpgradeAll's older URL replacement feature. Hooks are runtime/local policy, not repository registry state. Hooks are discovered only from the filesystem: getter lists enabled `rc/hook/*.lua` files, excludes basenames starting with `.`, sorts them deterministically, then loads them before every Lua execution environment. There is no hook registry, metadata map, or persistent disabled-hook state; dot-prefixed Lua files such as `.10-http-rewrite.lua` are excluded from hook Lua discovery and are not hook entries. Hook scripts can wrap exported Lua host entrypoints such as `http_get()` or `read_package_file()` by replacing the visible Lua function and calling the original getter-internal entrypoint from `getter_builtin.`, for example `getter_builtin.http_get()` or `getter_builtin.read_package_file()`, inside the wrapper. Getter core/CLI does not maintain a protective denylist of hookable public functions; if extra protection is needed, it belongs in UI/UX policy rather than the getter core. `getter_builtin.*` is an internal escape hatch for hook code to reach the original unhooked implementation; ordinary package/autogen Lua should depend on the public hooked names instead. The hook layer affects package version scripts, repository-level autogen scripts, and `luaclass/` code through their calls to wrapped host functions, but it is an execution overlay rather than a mechanism for modifying repository source files. Hook loading is fail-closed: any enabled hook parse/load/runtime initialization failure fails the current Lua execution instead of silently continuing without the user's local policy. This intentionally fails “stupidly” rather than pretending success when a hook-dependent proxy/security rule did not load. This preserves user freedom to use mirrors, proxies, local replacement endpoints, or other transparent policy layers. Hook rewriting does not by itself make content trusted: for package version scripts without `allow_free_network`, the fetched response body must still match a hash listed in the package `Manifest`; for scripts with `allow_free_network`, getter/UI surfaces the configured high-risk permission. + +### Provider endpoint/catalog + +An upstream metadata service or structured index, such as an F-Droid provider endpoint/catalog or the GitHub API endpoint for one project. Provider endpoints are sources/backends for package metadata; they are not package identities and not UpgradeAll repositories. + +### Package source + +A declaration inside a package Lua definition that uses a reusable provider module/class to discover provider candidates and artifacts. F-Droid, GitHub, Google Play, and similar systems are package sources/providers, not top-level packages. + +### Reusable Lua provider module/class + +A Lua helper under a repository's `luaclass/` directory that provides common provider behavior and calls getter-owned provider host APIs. F-Droid should have a standard class where the common case specifies only `package_name`; generated F-Droid package metadata/version scripts should not duplicate `name` because display metadata comes from the self-describing F-Droid catalog. F-Droid endpoint names come from endpoint ids/directories, and the endpoint URL defaults to official F-Droid but can be customized. GitHub should have a standard class where the common case specifies typed `owner` and `repo`, with package-authored asset/version rules when needed. + +### Autogen pipeline/template + +A getter-owned preview/apply workflow, backed by repository-level `.metadata/autogen/` metadata and Lua scripts, that writes ordinary package directories/version scripts from structured inputs. F-Droid support is autogen-first: explicit user-selected F-Droid apps and automatically discovered installed F-Droid-covered apps are represented as generated package directories, usually in the configured generated repository alias such as `autogen`, while upstream/local authors may still hand-write F-Droid package directories/version scripts. Each generated package directory stores its own getter-managed generation record, `.autogen.jsonc`, so lookup/cleanup is local to the package directory. Hashes recorded inside `.autogen.jsonc` are ownership/tamper-detection facts for generated output: they answer whether a file is still the file getter generated earlier, and they do not provide security trust, repository signing, or external-download validation. The `.autogen.jsonc` `files` map covers getter-written generated output such as `metadata.jsonc`, `Manifest`, generated Lua scripts, and generated `files/...` helper files; it does not include `.autogen.jsonc` itself, avoiding self-referential hashing. The generated repository is generated output: getter may overwrite package directories it previously generated only when matching `.autogen.jsonc` proves ownership. When refresh/overwrite ownership checks pass, getter clears the existing generated package directory contents, then writes the new generated contents into the same package directory, without preserving old unlisted extra files. If clearing any old file or subdirectory fails, the whole refresh/overwrite fails rather than being ignored. If writing new generated contents fails after clearing, the operation fails directly without rollback; the directory may be empty or partially written, and the next refresh continues by clearing and rewriting again. If a target directory exists without a matching generation record, apply reports a conflict and does not overwrite it. A generated-repo package directory missing `.autogen.jsonc` is a conflict rather than something getter automatically claims; without `.autogen.jsonc`, there is no ownership proof. If `.autogen.jsonc` exists but is malformed or schema-invalid, ordinary package discovery/evaluation is still decided by `metadata.jsonc`, but autogen refresh/apply/cleanup/overwrite reports a generated-ownership conflict and does not auto-fix, overwrite, or delete it. When cleanup ownership checks pass, cleanup clears the generated package directory contents, including `.autogen.jsonc` and any unlisted extra files inside it, but does not delete the package directory itself. If clearing any file or subdirectory fails, the whole cleanup/update fails rather than being ignored. Getter does not classify or preserve unlisted extra files in generated package directories because they are outside getter's domain; generated repositories are generated output. User-authored overrides belong in `repo/local/...`, not by hand-editing `repo/autogen/...`. + ### Lua update runtime The getter-owned runtime that evaluates a package's complete Lua lifecycle from capability checks through update action resolution. It is the center of UpgradeAll's update behavior: package Lua supplies a fully materialized lifecycle contract, getter supplies host APIs and validation, and the runtime produces getter-owned update/download/install DTOs for the app to render or execute through platform adapters. @@ -32,7 +100,7 @@ A Lua-side reusable template/base abstraction that fills default lifecycle funct ### Lua dependency closure -The set of Lua package files, template classes, helper modules, parent package imports, and runtime/API versions that shape a completed package lifecycle contract and its package metadata. Package metadata cache entries are valid only for the same Lua dependency closure and operation context. +The set of package metadata files, version scripts, Lua classes, helper modules, parent package imports, and runtime/API versions that shape a completed package lifecycle contract and its package metadata. Package metadata cache entries are valid only for the same Lua dependency closure and operation context. ### Side-effect executor @@ -44,7 +112,7 @@ A getter-owned notification boundary used by native/Flutter UI to learn that run ### Provider host API -A getter-owned API exposed to package Lua for requesting provider/source data. Lua packages should call provider host APIs rather than performing arbitrary direct HTTP by default. The provider executor behind the host API can be fake during early runtime development and live later, but caching, diagnostics, permissions, and output validation belong to the Lua update runtime boundary. +A getter-owned API exposed to package Lua for requesting provider/source data. Lua packages should call getter host APIs rather than using any Lua-native network library or Flutter/Kotlin HTTP path. HTTP is exposed as a getter-managed host function such as `http_get(url, headers = ..., cache = true|false, ...)`; `cache` defaults to `false`, and Lua/provider modules actively opt a request into HTTP/source caching with `cache = true`. Getter owns execution, cache key construction, persistence, revalidation, stale diagnostics, permissions, secret redaction, and output validation. The provider executor behind the host API can be fake during early runtime development and live later. ### Package metadata cache diff --git a/docs/README.md b/docs/README.md index 23598262..eb18be32 100644 --- a/docs/README.md +++ b/docs/README.md @@ -29,10 +29,11 @@ Start here: 10. `architecture/adr/0009-android-platform-adapter-and-package-visibility.md` — Rust-active Android platform adapter and package visibility policy. 11. `architecture/adr/0010-package-metadata-cache-and-version-baseline.md` — accepted package metadata cache, live-version, installed-version, and `pin_version` rules. 12. `architecture/adr/0011-lua-update-runtime-side-effects-and-events.md` — accepted Phase D Lua runtime, task/action lifecycle, mock side-effect executor, and RuntimeNotification bridge rules. -13. `lua-api/` — practical Lua package authoring docs, including offline `repo validate` diagnostics. -14. `migration/legacy-room-mapping.md` — old data mapping rules. -15. `app/flutter-ui-feature-parity-and-testing.md` — Flutter feature parity and BDD/TDD test boundary. -16. `implementation/coding-agent-handoff.md` — coding-agent / pi-agent handoff instructions. +13. `architecture/adr/0012-getter-owned-provider-modules-and-autogen-refresh.md` — draft live provider design for getter-owned F-Droid autogen, standard GitHub/F-Droid Lua modules, provider cache refresh, and stale-cache semantics. +14. `lua-api/` — practical Lua package authoring docs, including offline `repo validate` diagnostics. +15. `migration/legacy-room-mapping.md` — old data mapping rules. +16. `app/flutter-ui-feature-parity-and-testing.md` — Flutter feature parity and BDD/TDD test boundary. +17. `implementation/coding-agent-handoff.md` — coding-agent / pi-agent handoff instructions. Canonical architecture ADRs live in `docs/architecture/adr/*`. The `docs/adr/*` directory is kept for historical/refactor-phase ADRs and transition notes. diff --git a/docs/architecture/README.md b/docs/architecture/README.md index 0104215d..3e7214fd 100644 --- a/docs/architecture/README.md +++ b/docs/architecture/README.md @@ -19,6 +19,7 @@ Canonical ADRs: - `adr/0009-android-platform-adapter-and-package-visibility.md` - `adr/0010-package-metadata-cache-and-version-baseline.md` - `adr/0011-lua-update-runtime-side-effects-and-events.md` +- `adr/0012-getter-owned-provider-modules-and-autogen-refresh.md` Documentation policy: diff --git a/docs/architecture/adr/0001-app-centric-lua-package-repository-model.md b/docs/architecture/adr/0001-app-centric-lua-package-repository-model.md index 0413572d..d707bc4b 100644 --- a/docs/architecture/adr/0001-app-centric-lua-package-repository-model.md +++ b/docs/architecture/adr/0001-app-centric-lua-package-repository-model.md @@ -9,13 +9,21 @@ UpgradeAll will replace the old hub-app model with an app/package-centric repository model. - The primary user-facing object is an App/package, not a Hub. -- Package IDs are readable UpgradeAll namespaces, not UUIDs. -- Examples: `android/org.fdroid.fdroid`, `android/com.termux`, `magisk/zygisk-next`. -- GitHub, F-Droid, Google Play, CoolApk and similar systems are providers/sources/backends, not package identity. +- Package paths are readable UpgradeAll namespaces, not UUIDs. +- Examples: `android/app/org.fdroid.fdroid`, `android/f-droid/app/org.fdroid.fdroid`, `android/magisk/zygisk-next`. +- Package references use Gentoo-style atoms: `` resolves by repository priority, while `::` selects a local repository alias explicitly. +- The local repository alias is the directory name under `repo/`; if a user clones or renames the official repository to `repo/a`, the local alias is `a`. +- Local aliases are user-controlled reference/priority names, not repository security identities. +- GitHub, F-Droid, Google Play, CoolApk and similar systems are providers/sources/backends, not repository identities. - A single package may have multiple sources. -- Package definitions are Lua files stored in repositories/overlays. +- Package definitions are directories stored in repositories/overlays. +- Package identity is derived from the package directory path, like emerge/ebuild category-package identity, not duplicated as an `id` field inside Lua. +- Package metadata lives in `metadata.jsonc`; Android app install identity uses `android.package_name`, not an ambiguous `package_id` field. +- Package metadata declares Lua script permissions per file, e.g. which script needs `allow_free_network`. +- Package version scripts live directly in the package directory as direct child files named `.lua`; basename must end with `.lua` and must not start with `.`; removing `.lua` yields the literal version string with no SemVer requirement; live/floating scripts use `9999.lua`; every enabled Lua script must start with an explicit `#!/bin/upa-lua v1` API-version line; dot-prefixed Lua files are excluded from Lua discovery. This dot-prefix exclusion is only for Lua files; non-Lua package files are governed by the explicit getter file whitelist, so `.autogen.jsonc` is managed when present in a generated package. +- External network/dynamic-download response-body hashes for package version scripts are recorded in a package `Manifest` as ` [optional-name]`; repository source files are protected by the repository Git/signing/maintainer trust model, not by package `Manifest`. - Repositories have priorities; higher priority wins. -- getter only sees the top-level resolved package for a given package id. +- getter only sees the top-level resolved package for a given package path unless the caller specifies `::repo-name`. ## Context @@ -26,52 +34,107 @@ The new model takes inspiration from Portage/emerge overlays and Funtoo Metatool ## Repository layout ```text -repo/ - repo.toml - packages/ - android/ - org.fdroid.fdroid.lua - magisk/ - zygisk-next.lua - lib/ - github.lua - android.lua - github_android_apk.lua - templates/ - android_installed_app.lua - github_android_apk.lua +/ + main.db + cache.db + repo/ + metadata.jsonc + official/ + .metadata/ + metadata.jsonc + autogen/ + metadata.jsonc + android.lua + luaclass/ + android.lua + github_android_apk.lua + android/ + app/ + org.fdroid.fdroid/ + metadata.jsonc + Manifest + 1.20.0.lua + 9999.lua + files/ + helper-data.json + f-droid/ + app/ + org.fdroid.fdroid/ + metadata.jsonc + Manifest + 1.20.0.lua + local/ + android/ + app/ + org.example.app/ + metadata.jsonc + Manifest + 1.2.3.lua + rc/ + hook/ + http.lua ``` -`packages/` contains final package definitions consumed by getter. +The repository root `repo/metadata.jsonc` contains repository-related local registry/config rules such as repository priority. It is part of getter's local state, not part of any one package repository and not something an upstream repository update should overwrite. Hook/runtime policy is not stored in `repo/metadata.jsonc`; local hooks live under `rc/hook/`. -`lib/` contains reusable Lua modules. These are conceptually like eclasses, but the project does not introduce an `eclass` keyword or syntax. +`rc/` is the getter runtime/local policy root. It is a top-level sibling of `repo/` under the getter data directory, beside storage files such as `main.db` and `cache.db`. It is not a repository root and does not participate in repository or package discovery. Current defined content is `rc/hook/*.lua`; future runtime/local policy such as environment, credential, or network behavior belongs under `rc/`, not in `repo/metadata.jsonc`. -`templates/` contains Lua generators that output new package Lua file content. Templates are for autogen workflows, not runtime package evaluation. +`repo/metadata.jsonc` is the only current repo-root reserved entry. Every direct child directory of `repo/` is an UpgradeAll repository alias. The alias is the local directory name, not necessarily the upstream repository's advertised name. Renaming or cloning a repository to a different child directory intentionally changes its local alias, giving users freedom to fork or maintain an intermediate repository layer. Future repo-root reserved entries require an explicit design/ADR because they occupy alias namespace; no names such as `hook` or `rc` are pre-reserved under `repo/`. -## Repository priority +Package directories are final package definitions consumed by getter. UpgradeAll/getter domain strings, including package paths and aliases, are treated as UTF-8. Getter does not detect or convert other filesystem/text encodings; inputs in other encodings are still interpreted as UTF-8. A directory that directly contains `metadata.jsonc` declares a package boundary. `.autogen.jsonc` does not declare a package boundary; it is only a generated-package ownership record inside a package directory that already has `metadata.jsonc`. If `metadata.jsonc` parses correctly as package metadata, the directory is a valid package directory; if parsing fails, getter reports an invalid package metadata diagnostic for that package path. A package path is the directory path relative to a repository, for example `android/app/org.fdroid.fdroid`. Once a package boundary is found, it is the package path endpoint and getter does not discover nested packages below it. -Default priority convention: +Repository self metadata under `.metadata/metadata.jsonc` stores publishable repository facts such as schema version, upstream URL, description, maintainers/co-maintainers, and signing/trust metadata. Security/trust checks use this verified metadata, not the local alias. If `.metadata/metadata.jsonc` is missing, the repository may still be used as unverified/local-source content, but repo update, signature, and trust operations are unavailable. If `.metadata/metadata.jsonc` exists but cannot be parsed, getter reports a repository metadata diagnostic. Inside a repository alias directory, getter only considers explicit entries: reserved repository-root directories such as `.metadata/` and `luaclass/`, plus directory chains that form package paths. Reserved directories are handled only by their own responsibility and never participate in package discovery; package paths cannot begin with reserved names such as `.metadata` or `luaclass`. Future repository-root reserved directories follow the same rule so the repo layout remains organizable. Other repository contents such as `README.md`, `docs/`, or random helper files are outside getter domain entirely: they are not parsed, validated, displayed, warned about, or modeled as ignored managed objects. -```text -local 100 user-written overrides, default highest priority -community/official 0 normal remote repositories -local_autogen -1 generated fallback packages from installed inventory +Package `metadata.jsonc` stores package-level metadata such as `type`, platform install identity, maintainers, description, homepage, and other user-facing/source facts. Android app install identity is `android.package_name`. + +`Manifest` stores allowed external network/dynamic-download response-body hashes for package version scripts in ` [optional-name]` format. The hash is authoritative; the optional name is for humans/debugging because a URL may not reveal the actual returned filename. A missing `Manifest` is equivalent to an empty hash set, not an invalid package. For package version scripts without `allow_free_network`, getter accepts externally fetched data files or API response bodies only when their SHA-512 hash appears in that package's `Manifest`; with a missing/empty `Manifest`, such network fetches cannot succeed. Scripts that do not fetch external network content, or only read package-local `files/`, do not need a `Manifest`. Scripts with `allow_free_network` are not blocked by `Manifest` membership but remain high-risk. `Manifest` belongs only to package directories; repository-level autogen scripts under `.metadata/autogen/` do not have a Manifest. However, autogen scripts that create package directories must also generate correct package `Manifest` files for generated packages expected to work without `allow_free_network`. It is not a repository source manifest; metadata files, Lua scripts, `files/`, sibling `.autogen.jsonc`, `luaclass/`, autogen scripts, and repository metadata are protected by the repository Git/signing/maintainer trust model. Since `Manifest` and `.autogen.jsonc` are same-level package files, `Manifest` cannot architecturally protect `.autogen.jsonc`. + +`.lua` files are fixed-version package scripts. Getter discovers them only from direct child files of the package directory whose basename ends with `.lua` and does not start with `.`. Removing the `.lua` suffix yields the literal version string; getter does not require SemVer or otherwise constrain the version syntax at discovery time. Examples include `1.2.3.lua`, `1.2.3-r1.lua`, `v1.2.3.lua`, and `2026.06.25.lua`. `9999.lua` is the live/floating package script. Every enabled Lua script must declare its interpreter/API version on the first line, for example `#!/bin/upa-lua v1`; the version is required and has no implicit default. A Lua file whose basename starts with `.`, for example `.9999.lua`, is excluded from Lua discovery: getter does not parse, validate, execute, display, or apply permission metadata to it as Lua. This dot-prefix exclusion is only for Lua files; non-Lua package files are governed by the explicit getter file whitelist, so `.autogen.jsonc` is managed when present in a generated package. + +Package Lua may read package-local helper files under its own package directory's `files/` subtree through a package-scoped getter host API such as `read_package_file(path)`, where `path` is relative to `files/`. The original built-in implementation, `getter_builtin.read_package_file`, does not expose real filesystem paths or a general `io.open` escape hatch; it rejects absolute paths, `..`, directory reads, cross-package reads, and arbitrary repository reads. `read_package_file(path)` returns a Lua string; getter does not interpret encoding, MIME type, JSON, or text-vs-binary mode. Hook code may still wrap the public `read_package_file()` name as local user policy; getter core/CLI does not maintain a protective denylist of hookable public functions. Getter does not assign product semantics to names or formats inside `files/`; the package owns them. Package directory contents outside getter's explicit discovery set (`metadata.jsonc`, optional generated-package `.autogen.jsonc`, `Manifest`, enabled direct-child `*.lua`, and `files/`) are outside getter domain entirely: getter does not parse, validate, display, warn about, or model them as ignored managed objects. The primary reason is clear responsibility boundaries; a smaller getter-core attack surface is a beneficial side effect. This keeps repository layout structured while relying on the repository trust boundary: users should only use repositories they trust, or copy/author content into repositories they control. + +Package `metadata.jsonc` declares permissions per enabled Lua file, for example that `9999.lua` or a fixed-version script needs `allow_free_network`. The permission system is used when getter runs Lua and when getter/UI displays enabled version Lua to the user. The `lua` map is lookup-only: getter first discovers an enabled Lua file from the filesystem, then queries this map by basename. Getter does not enumerate the map to discover scripts or warnings. Dot-prefixed Lua files are excluded from Lua discovery, so permission metadata for them is inert and cannot enable them. Entries for nonexistent files are also inert. `9999.lua` commonly has that permission, but it is not high-risk merely by filename if metadata does not grant it; fixed-version scripts can also be high-risk when metadata grants free network. A version script omitted from the `lua` map defaults to `permission: []`. + +`luaclass/` contains reusable Lua modules. These are conceptually like eclasses, but the project does not introduce an `eclass` keyword or syntax. + +`.metadata/autogen/` contains repository-level autogen metadata and scripts. Autogen scripts may use `luaclass/` helpers and can generate or refresh package directories from structured upstream inputs. Each generated package directory stores its own getter-managed generation record as `.autogen.jsonc`, listing generated files, file hashes, generator/template identity, and input facts needed to decide whether the package can be refreshed or cleaned. Hashes recorded in `.autogen.jsonc` are generated-output ownership/tamper-detection facts only: they answer whether a file is still the file getter generated earlier, not whether it is trusted, repository-signed, or valid as external-download content. The `.autogen.jsonc` `files` map covers getter-written generated output such as `metadata.jsonc`, `Manifest`, generated Lua scripts, and generated `files/...` helper files; it does not include `.autogen.jsonc` itself, avoiding self-referential hashing. The generated repository is generated output: getter may overwrite package directories it previously generated only when matching `.autogen.jsonc` proves ownership. When refresh/overwrite ownership checks pass, getter clears the existing generated package directory contents, then writes the new generated contents into the same package directory, without preserving old unlisted extra files. If clearing any old file or subdirectory fails, the whole refresh/overwrite fails rather than being ignored. If writing new generated contents fails after clearing, the operation fails directly without rollback; the directory may be empty or partially written, and the next refresh continues by clearing and rewriting again. If a target package directory exists without a matching generation record, apply reports a conflict and does not overwrite it. A generated-repo package directory missing `.autogen.jsonc` is a conflict rather than something getter automatically claims. If `.autogen.jsonc` exists but is malformed or schema-invalid, ordinary package discovery/evaluation is still decided by `metadata.jsonc`, but ownership-dependent autogen refresh/apply/cleanup/overwrite reports a conflict and does not auto-fix, overwrite, or delete it. When cleanup ownership checks pass, cleanup clears the generated package directory contents directly, including `.autogen.jsonc` and any unlisted extra files inside it, but does not delete the package directory itself. If clearing any file or subdirectory fails, the whole cleanup/update fails rather than being ignored. Getter does not classify or preserve unlisted extra files in generated package directories because they are outside getter's domain; direct directory-content clearing is simpler and more stable for generated output. User-authored overrides belong in `repo/local/...`, not by hand-editing `repo/autogen/...`. + +Getter preserves user-controlled transparent URL replacement through local hook scripts under `rc/hook/`, analogous to an emerge bashrc-style hook and UpgradeAll's older URL replacement behavior. Hooks are getter-local runtime policy discovered only from the filesystem: list enabled `rc/hook/*.lua`, exclude dot-prefixed basenames, sort deterministically, and load before package version scripts, repository-level autogen scripts, and `luaclass/` code. There is no hook registry, metadata map, or disabled-hook state. Dot-prefixed Lua files are excluded from hook Lua discovery and are not hook entries. Hook scripts can wrap getter-exposed Lua host entrypoints such as `http_get()` or `read_package_file()` and call the original unhooked getter-internal entrypoint through `getter_builtin.`, for example `getter_builtin.http_get()` or `getter_builtin.read_package_file()`, after rewriting the URL or applying local policy. Getter core/CLI does not maintain a protective denylist of hookable public functions; if extra guardrails are needed, they belong in UI/UX policy rather than the getter core. `getter_builtin.*` is an internal escape hatch for hook code; ordinary package/autogen Lua should use the public hooked names instead. Hooks are an execution overlay and must not mutate repository source files. Hook loading is fail-closed for enabled hooks: parse/load/initialization failure fails the current Lua execution instead of silently falling back to unhooked functions. URL rewrites can point requests at mirrors, proxies, or local replacement endpoints, but for package version scripts without `allow_free_network` the returned body still must match a package `Manifest` hash. + +## Repository priority + +Default priority convention in `repo/metadata.jsonc`: + +```jsonc +{ + "version": 1, + // Autogen writes to "autogen" by default. Uncomment and change this + // if generated packages should target another existing repository alias. + // "generated_repository": "autogen", + "priority": { + "local": 100, + "official": 0, + "autogen": -1 + } +} ``` -The user may edit priorities manually. The only hard rule is: higher priority wins. +The user may edit priorities through UI, CLI, or the getter-owned root metadata file. The only hard rule is: higher priority wins when resolving an unqualified package path. If `repo/metadata.jsonc` is missing, getter uses built-in priority defaults: `local` = 100, `autogen` = -1, every other alias = 0, with same-priority aliases resolved in lexicographic order. If present, the priority map is lookup-only: getter discovers actual repository alias directories first, then queries the map by alias; entries for nonexistent aliases are inert and do not warn, create repositories, display repositories, or participate in sorting. The generated repository target defaults to `generated_repository = "autogen"` when omitted, and starter config should show this default as a comment users can uncomment/change. When autogen runs with target `autogen`, getter creates `repo/autogen/` if needed. If `generated_repository` is set to any other alias, that repository directory must already exist or autogen apply reports a configuration error. `generated_repository` only decides autogen output target and normal package resolution still uses repository priority. If `repo/metadata.jsonc` exists but cannot be parsed, getter reports a configuration diagnostic instead of silently falling back. A qualified atom such as `android/app/org.fdroid.fdroid::official` resolves only the named local repository alias. ## Import and override -Reusable Lua modules should use native Lua `require` where practical: +Reusable Lua modules should use Lua import helpers where practical: ```lua -local github_android = require("lib.github_android_apk") +local github_android = require("luaclass.github_android_apk") ``` -Parent package import uses a host helper because package ids contain slashes/dots and repo id must be explicit: +Cross-repository imports may resolve by repository priority when no repository alias is specified, or by explicit local alias when the author writes one. Explicit alias imports intentionally depend on the user's local repository name; if the user renames `repo/official` to `repo/a`, imports that explicitly name `official` are considered intentionally broken by that rename. This is a feature: users can intentionally replace, fork, or interpose repository layers instead of having every package become fully self-maintained. + +Parent package imports should use package atoms rather than raw file paths: ```lua -local base = package_from("official", "android/org.fdroid.fdroid") +local base = package_from("android/app/org.fdroid.fdroid::official") ``` Override is a Lua helper/metatable concern, not a Rust API concern. Rust validates only the final returned data object. @@ -82,7 +145,7 @@ Positive: - App identity is readable and user-supportable. - Multiple sources become package internals rather than top-level user confusion. -- Users can maintain patch stacks by overriding individual package files. +- Users can maintain patch stacks by overriding individual package directories/version scripts. - Autogen can create fallback local package definitions without contaminating user-authored `local` overrides. Costs: diff --git a/docs/architecture/adr/0003-legacy-room-migration.md b/docs/architecture/adr/0003-legacy-room-migration.md index 3dde893b..ee6ad5f5 100644 --- a/docs/architecture/adr/0003-legacy-room-migration.md +++ b/docs/architecture/adr/0003-legacy-room-migration.md @@ -30,7 +30,7 @@ Migration writes to: - `local` repository package Lua files when necessary. - migration records table. -Normal installed-app autogen writes to `local_autogen`, but legacy migration is special: it may generate `local` package files once to preserve explicit old user data. +Normal installed-app autogen writes to the configured generated repository target (`generated_repository`, default `autogen`), but legacy migration is special: it may generate `local` package files once to preserve explicit old user data. ## Package ID mapping diff --git a/docs/architecture/adr/0005-lua-package-api.md b/docs/architecture/adr/0005-lua-package-api.md index 9a5bd087..169a47a7 100644 --- a/docs/architecture/adr/0005-lua-package-api.md +++ b/docs/architecture/adr/0005-lua-package-api.md @@ -6,12 +6,14 @@ ## Decision -getter embeds Lua for package definitions, reusable package helpers and autogen templates. +getter embeds Lua for package version scripts, reusable package helpers, and repository autogen scripts. The Lua/Rust boundary is treated as an RPC/serialization boundary: Lua returns JSON-like tables; Rust validates and deserializes them into typed structs. Lua scripts do not receive mutable Rust domain objects. +Package version Lua may read package-local helper data under its own package directory's `files/` subtree through a package-scoped getter host API such as `read_package_file(path)`, where `path` is relative to `files/`. The original built-in implementation, `getter_builtin.read_package_file`, does not expose real filesystem paths or a general `io.open` escape hatch; it rejects absolute paths, `..`, directory reads, cross-package reads, and arbitrary repository reads. `read_package_file(path)` returns a Lua string; getter does not interpret encoding, MIME type, JSON, or text-vs-binary mode. Hook code may still wrap the public `read_package_file()` name as local user policy; getter core/CLI does not maintain a protective denylist of hookable public functions. Getter does not assign product semantics to file names or formats inside `files/`. Package directory contents outside getter's explicit discovery set are outside getter domain entirely, not ignored managed objects. Repository source trust is handled by repository review/signing and by users choosing trusted repositories or authoring/copying content into their own repositories. + ## Language Use Lua via `mlua` unless implementation evidence later proves a blocker. @@ -20,6 +22,8 @@ Use Lua via `mlua` unless implementation evidence later proves a blocker. Lua can use normal Lua tables/functions/metatables, reusable modules via `require`, package import helper for parent packages, and host-provided provider/network APIs based on permissions. +Package identity is provided by the repository directory path (`repo///` -> ``), not by an `id` field inside returned Lua data. Lua returns package/version content; Rust attaches and validates the path-derived package path and optional repository alias from the selected package atom. + Rust owns schema validation, typed domain model, persistence, event dispatch, download task state and platform callback dispatch. ## Lifecycle phases @@ -41,18 +45,26 @@ post_update ## Network permission model -Lua has no direct network API by default. +Lua has no native or standard-library network API by default. Network access, when allowed by the package/provider mode, goes through getter host APIs such as `http_get(url, headers = ..., cache = true|false, ...)`. + +`cache` defaults to `false`. Passing `cache = true` opts that HTTP request into getter-owned provider/source caching; Lua chooses cache participation per request, but getter owns cache keys, persistence, revalidation, stale diagnostics, and secret redaction. -If a package declares free network permission, getter exposes a direct network host API to that Lua environment and Flutter displays a yellow warning tag in App detail source/version UI. +Free network permission is declared per enabled Lua script in package `metadata.jsonc` using a filename-keyed map, for example `lua: { "9999.lua": { permission: ["allow_free_network"] } }`. The `lua` map is lookup-only: getter first discovers an enabled Lua file from the filesystem, then queries this map by basename. Getter does not enumerate the map to discover scripts or warnings. Entries for nonexistent files or dot-prefixed Lua files are inert. The permission can apply to `9999.lua` or to a fixed-version script. `9999.lua` commonly needs free network, but the filename alone does not grant the permission or force the warning if metadata does not declare it. A version script omitted from the `lua` map defaults to `permission: []`. + +If a package version script does not declare `allow_free_network`, any external network/dynamic-download data file or API response body used by that script must have a SHA-512 hash listed in that package's `Manifest`; an unlisted or mismatched body is rejected. A missing `Manifest` is equivalent to an empty hash set, not an invalid package, so network fetches from scripts without `allow_free_network` cannot succeed when `Manifest` is missing or empty. Scripts that do not fetch external network content, or only read package-local `files/`, do not need a `Manifest`. Scripts with `allow_free_network` are not blocked by `Manifest` membership but remain high-risk. Repository-level autogen scripts under `.metadata/autogen/` do not have a package Manifest, but autogen scripts that create package directories must generate correct package Manifests for generated packages expected to work without `allow_free_network`. Package `Manifest` is not a repository source manifest and cannot protect same-level repository source files such as `.autogen.jsonc`; those are protected by the repository Git/signing/maintainer trust model. + +If a script declares free network permission for arbitrary upstream access beyond Manifest-bound fixed-version content, getter exposes the host HTTP API to that Lua environment and Flutter displays a yellow warning tag in App detail source/version UI. This tag is informative and does not block use. -## Templates +Getter local hook scripts under `rc/hook/` are runtime/local policy, not repository registry state. They are discovered only from the filesystem: getter lists enabled `rc/hook/*.lua`, excludes basenames starting with `.`, sorts deterministically, then loads them before every Lua execution environment. There is no hook registry, metadata map, or disabled-hook state. Enabled hooks may wrap visible Lua host functions such as `http_get()` or `read_package_file()` to implement local policy, then call the original getter-internal entrypoint through `getter_builtin.`, for example `getter_builtin.http_get()` or `getter_builtin.read_package_file()`. Getter core/CLI does not maintain a protective denylist of hookable public functions; extra protection belongs in UI/UX policy rather than the getter core. `getter_builtin.*` is an internal escape hatch for hook code; ordinary package/autogen Lua should use the public hooked names instead. The hook layer affects package version scripts, repository-level autogen scripts, and `luaclass/` code through their calls to wrapped host functions, but it is an execution overlay rather than a mechanism for modifying repository source files. Enabled hook loading is fail-closed. Hooks do not bypass Manifest validation for package version scripts without `allow_free_network`. + +## Autogen scripts and Lua classes -Templates under `templates/` are Lua generators that output Lua package file content. They are distinct from runtime package modules. +Reusable Lua helpers live under repository `luaclass/` directories. Repository-level autogen scripts live under `.metadata/autogen/` and output ordinary package directories/version scripts from structured inputs. They are distinct from runtime package version scripts, although both can import shared `luaclass/` helpers. ## Validation -Rust validates package id/path consistency, known package kind, required fields, installed target schema, phase function presence/type where required, permission schema, action schema and URL/action validity. +Rust derives package path from the package directory path and validates known package kind, package metadata fields, version-script API version line, required fields, installed target schema, phase function presence/type where required, permission schema, action schema and URL/action validity. Returned Lua tables should not contain a duplicate package `id` field. Errors must distinguish Lua runtime errors, schema validation errors and domain validation errors. diff --git a/docs/architecture/adr/0006-package-centric-cli-command-contract.md b/docs/architecture/adr/0006-package-centric-cli-command-contract.md index bb232ab0..7552738f 100644 --- a/docs/architecture/adr/0006-package-centric-cli-command-contract.md +++ b/docs/architecture/adr/0006-package-centric-cli-command-contract.md @@ -100,7 +100,7 @@ It maps `apps[]` into getter tracked package state in `main.db`, writes a saniti `legacy report-list` returns sanitized migration report summaries through the same JSON envelope so app/test adapters do not need to inspect getter's data-directory layout directly. -The first installed-app autogen slice accepts an Android/platform-provided inventory DTO, computes generated fallback packages in Rust, previews before writing, and applies only after explicit `--accept-all` or `--accept ` confirmation. Getter owns the canonical repository path `/repositories/local_autogen`, fixed repo id `local_autogen`, default priority `-1`, deterministic package file path `packages//.lua`, and `autogen-manifest.json`. Candidates are skipped when any registered repository with priority higher than `local_autogen` already provides that package id. Applying installed autogen writes/registers `local_autogen` package files and tracks accepted packages in `main.db` when they are not already resolved; existing user state (`enabled`, `favorite`, `pin_version`) and existing non-missing resolution metadata are preserved. Cleanup preview/apply only targets manifest-managed `local_autogen` packages missing from the current installed inventory. Cleanup refuses stale/tampered previews that do not match the current manifest and deletes tracked state only for rows still owned by `local_autogen` generated packages. If an existing autogen file was modified, getter preserves that content into the user-authored `local` repo before regenerating or deleting the managed autogen file. +The first installed-app autogen slice accepts an Android/platform-provided inventory DTO, computes generated fallback packages in Rust, previews before writing, and applies only after explicit `--accept-all` or `--accept ` confirmation. ADR-0012 supersedes the original Phase 1a fixed generated-repository and flat generated package-file storage model. The current architecture writes ordinary package directories to the configured generated repository target (`generated_repository`, default `autogen`) and records package-local `.autogen.jsonc` ownership state. Cleanup/refresh follows ADR-0012 ownership checks instead of preserving modified generated files into `local`. `update check --fixture ` is the first Phase D offline update-check slice. The fixture contract is explicitly offline and uses `format = "getter-offline-update-check"`, `version = 1`, `package_id`, optional `installed_version`, optional `pin_version` (with transitional `ignored_version` accepted as an input alias), and normalized candidate/artifact DTOs. The command returns `network_required = false`, observed `installed_version`, `effective_local_version`, a getter-owned status (`update_available`, `up_to_date`, or `no_candidates`), the selected update when one exists, and generated download/install action DTOs. An update-selected result must have an actionable artifact; a selected candidate without artifacts is a structured update-check error rather than `update_available` with no actions. It reuses Rust getter update selection and version comparison; it does not run providers, perform downloads, persist download tasks, stream events, or call Android installers. diff --git a/docs/architecture/adr/0007-flutter-getter-bridge-contract.md b/docs/architecture/adr/0007-flutter-getter-bridge-contract.md index 05608caa..d896ef0e 100644 --- a/docs/architecture/adr/0007-flutter-getter-bridge-contract.md +++ b/docs/architecture/adr/0007-flutter-getter-bridge-contract.md @@ -47,7 +47,7 @@ applyInstalledAutogen(preview, acceptedPackages) The Android product APK packages a slim `:getter_bridge` library under `app_flutter/android/getter_bridge`. It builds the Rust `api_proxy` cdylib and includes only the no-UI native bridge / installed-inventory provider classes needed by the Flutter product path, avoiding the legacy native `:app` UI and old `GetterPort` hub/RPC wrapper surface. `MainActivity` exposes a no-UI `net.xzos.upgradeall/getter_bridge` MethodChannel that derives the app-private getter data directory and forwards migration and installed-autogen requests to JNI entrypoints returning getter-style JSON envelopes. -Internally, Rust/native bridge code scans Android inventory through the platform adapter, then asks getter-owned shared autogen operations to plan/apply `local_autogen`. `MethodChannelGetterAdapter` consumes the returned getter-style JSON envelopes. Flutter renders getter-owned preview/apply DTOs and scan diagnostics, then passes the package ids from displayed accepted preview candidates back to getter on apply; it must not expose a product Dart `InstalledInventoryPlatform` scanner or convert Android package names into package ids. +Internally, Rust/native bridge code scans Android inventory through the platform adapter, then asks getter-owned shared autogen operations to plan/apply installed-autogen output in the configured generated repository target (`generated_repository`, default `autogen`). `MethodChannelGetterAdapter` consumes the returned getter-style JSON envelopes. Flutter renders getter-owned preview/apply DTOs and scan diagnostics, then passes the package ids from displayed accepted preview candidates back to getter on apply; it must not expose a product Dart `InstalledInventoryPlatform` scanner or convert Android package names into package ids. `loadSnapshot()` composes smaller getter-owned read-model operations into the UI shell's first snapshot DTO. In the Android/native product path, `MethodChannelGetterAdapter` calls `readOperation` for `repository_list`, `tracked_package_list`, and `package_eval`; Rust getter reads SQLite repository/tracked-package state and evaluates registered Lua packages. Flutter may parse and combine those returned DTOs for rendering, but must not perform repository resolution, Lua validation/evaluation, version comparison, migration mapping, or update selection in Dart. `readMigrationReports()` must go through a getter operation such as `legacy report-list`; Flutter must not inspect getter's data-directory layout directly. Runtime task rendering must use ADR-0011 runtime task snapshot APIs and opaque action/task controls; Flutter must not synthesize task states, retry policy, installer behavior, or update decisions. diff --git a/docs/architecture/adr/0009-android-platform-adapter-and-package-visibility.md b/docs/architecture/adr/0009-android-platform-adapter-and-package-visibility.md index 97a777de..f39b09e5 100644 --- a/docs/architecture/adr/0009-android-platform-adapter-and-package-visibility.md +++ b/docs/architecture/adr/0009-android-platform-adapter-and-package-visibility.md @@ -10,7 +10,7 @@ UpgradeAll will use a Rust-active platform adapter for Android platform capabili Rust/getter-side native code defines the platform interface and actively calls the Android implementation. Android/Kotlin code supplies raw platform facts only. Flutter remains the product UI and renders getter-owned DTOs; it does not lead installed-app inventory scanning or turn Android package names into UpgradeAll package ids. -The first accepted platform capability is installed Android package inventory for `local_autogen` preview/apply workflows. The product Flutter APK declares: +The first accepted platform capability is installed Android package inventory for installed-autogen preview/apply workflows. ADR-0012 supersedes the early fixed generated repository target with the configured generated repository target (`generated_repository`, default `autogen`). The product Flutter APK declares: ```xml @@ -87,7 +87,7 @@ The product operation shape is: Flutter UI -> getter/native bridge: preview installed autogen -> Rust platform adapter: scan installed inventory facts - -> getter core: plan local_autogen candidates/skips + -> getter core: plan installed-autogen candidates/skips <- getter-owned preview DTO ``` @@ -129,7 +129,7 @@ The second slice adds the first Android facts provider while preserving the same The third slice wires the first product bridge operation without changing ownership boundaries: -- installed-autogen preview/apply semantics are extracted into reusable getter-owned `getter-operations` code so CLI and native bridge use the same `local_autogen` rules; +- installed-autogen preview/apply semantics are extracted into reusable getter-owned `getter-operations` code so CLI and native bridge use the same generated-repository ownership rules; - `getter-core` Lua support is feature-gated so the Android native bridge can use autogen/storage operations without pulling Lua evaluation into `api_proxy`; - `api_proxy` exposes JNI entrypoints for bridge initialization, installed-autogen preview, and installed-autogen apply; - preview initializes the Rust-active Android platform adapter, scans PackageManager facts, and passes getter-compatible inventory into getter-owned autogen planning; diff --git a/docs/architecture/adr/0010-package-metadata-cache-and-version-baseline.md b/docs/architecture/adr/0010-package-metadata-cache-and-version-baseline.md index 1073b27d..22d32897 100644 --- a/docs/architecture/adr/0010-package-metadata-cache-and-version-baseline.md +++ b/docs/architecture/adr/0010-package-metadata-cache-and-version-baseline.md @@ -14,7 +14,7 @@ The runtime caches software metadata produced by running package Lua/provider lo The cache model has two layers: -1. **Provider/source cache**: provider host API responses keyed by provider id, request parameters, executor/cache policy, auth/permission mode, and other provider-context inputs. +1. **Provider/source cache**: getter host API responses keyed by provider id, request parameters, executor/cache policy, auth/permission mode, and other provider-context inputs. Lua/provider modules opt individual HTTP host API calls into this cache explicitly, e.g. `http_get(url, headers = ..., cache = true)`, while the default is `cache = false`. 2. **Package metadata cache**: normalized package metadata produced by Lua/package logic from provider/source data. Package metadata cache entries are persisted in `cache.db` from the first runtime implementation. diff --git a/docs/architecture/adr/0012-getter-owned-provider-modules-and-autogen-refresh.md b/docs/architecture/adr/0012-getter-owned-provider-modules-and-autogen-refresh.md new file mode 100644 index 00000000..98eb0c74 --- /dev/null +++ b/docs/architecture/adr/0012-getter-owned-provider-modules-and-autogen-refresh.md @@ -0,0 +1,500 @@ +# ADR-0012: Getter-owned provider modules, autogen, and metadata refresh + +> Status: Draft +> Date: 2026-06-25 +> Project: UpgradeAll rewrite — Flutter APP + Rust getter core + Lua package repository model + +## Context + +ADR-0001 replaces the old hub-app model with package-centric Lua repositories and overlays. ADR-0002 keeps product/domain logic in Rust getter and limits Flutter/Kotlin to UI and platform-adapter roles. ADR-0010 accepts `cache.db` package metadata/provider-source caching, live/floating version rules, and `pin_version` semantics. ADR-0011 accepts getter-owned update runtime actions/tasks and requires package Lua to use getter-owned provider host APIs by default rather than arbitrary raw HTTP. + +The first accepted Phase D runtime implementation uses static Lua `updates` through a mock provider boundary. That path is useful scaffolding but is not the real live-provider design. + +Old-code archaeology shows that the legacy Android/Kotlin implementation had a generic Hub/WebSDK/cloud-config/RPC provider shape rather than local handwritten F-Droid or GitHub parser classes. The old local Kotlin `BaseHub` supported both a batch latest-update shape and a per-app release-list shape: + +```kotlin +getUpdate(hub, appList) // batch-ish, many apps +getReleases(hub, app) // one app/project release list +``` + +The old shared DTOs were similarly generic: + +```text +ReleaseGson(version_number, changelog, assets, extra) +AssetGson(file_name, file_type, download_url) +DownloadItem(name, url, headers, cookies) +``` + +Those historical abstractions are useful evidence, but the rewrite must not revive the old Hub UUID/map model. The rewrite needs a package-centric provider model that supports two first target families: + +- **F-Droid**: a structured catalog/index provider where one provider endpoint can discover metadata for many Android apps. +- **GitHub**: a project/release provider where a package usually names one upstream project and may need package-authored asset/version rules. + +The product goal for F-Droid is autogen-first: F-Droid apps should normally appear as ordinary generated package directories/version scripts, whether the user explicitly chooses one F-Droid app or getter auto-discovers installed apps covered by F-Droid. Upstream repositories and local users may still hand-write F-Droid package directories/version scripts when they need richer behavior or overrides. + +## Decision + +UpgradeAll will implement live provider support through **getter-owned standard provider modules** plus **getter-owned autogen pipelines**. + +F-Droid and GitHub are providers/sources/backends, not package identities and not UpgradeAll repositories. + +### Core decision + +1. Getter owns live provider execution, provider/source caching, package metadata normalization, update selection, action issuance, and autogen/package-path decisions. +2. Reusable Lua provider modules/classes under `luaclass/` provide high-level package-authoring APIs for common provider families. +3. Those Lua modules call getter-owned host APIs by default. HTTP access is exposed as a getter-managed host function such as `http_get(url, headers = ..., cache = true|false, ...)`; `cache` defaults to `false`, and Lua/provider modules opt individual requests into HTTP source caching by passing `cache = true`. This remains getter-owned network/cache execution, not Flutter/Kotlin HTTP and not a Lua standard-library network primitive. +4. F-Droid support is **autogen-first**: + - an F-Droid app is represented as an ordinary package directory with metadata and version scripts; + - explicit user selection of an F-Droid app uses a getter autogen preview/apply operation that generates a package directory/version script; + - automatic installed-app discovery uses the same autogen machinery; + - generated F-Droid package directories normally live in the configured generated repository alias, defaulting to `autogen`, and obey normal repository priority/overlay rules; + - official/community/local repositories may still contain hand-written F-Droid package directories/version scripts. +5. GitHub support is **standard-module-first, hand-authored package by default**: + - package authors normally write one package file per GitHub project; + - a standard GitHub Lua module should require only typed project coordinates for common cases, such as `owner` and `repo`; + - the module provides release, tag/resource, and live latest-commit helper behavior through getter provider host APIs; + - GitHub global search/catalog autogen is not part of the first accepted provider model. +6. Both families normalize into the same getter-owned candidate/artifact metadata model before version comparison and action issuance. +7. Provider/source caches and normalized package metadata caches live in `cache.db`; generated package directories/version scripts, package `files/` helper data, and package-local `.autogen.jsonc` generation records are repository source artifacts, not cache entries. +8. Runtime action/task state remains process-memory only per ADR-0011 and is not stored in `main.db` or `cache.db`. + +## Terminology + +To avoid drifting back into the old hub-app model, ADR-0012 uses these terms strictly: + +- **Repository root**: the local `repo/` directory that contains all enabled UpgradeAll repositories and repository-related local registry/config metadata in the sole current repo-root reserved file, `repo/metadata.jsonc`. Every direct child directory of `repo/` is a repository alias. Future repo-root reserved entries require an explicit design/ADR because they occupy alias namespace. Runtime/local policy lives outside `repo/` under `rc/`. +- **Runtime configuration root**: the local `rc/` directory for getter runtime/local policy. It is a top-level sibling of `repo/` under the getter data directory, beside storage files such as `main.db` and `cache.db`. It is not a repository root and does not participate in package/repository discovery. Current defined content is `rc/hook/*.lua`; future runtime/local policy such as environment, credential, or network behavior belongs under `rc/`, not in `repo/metadata.jsonc`. +- **UpgradeAll repository**: a package repository rooted at `repo/`, where `` is the local repository alias. Getter considers explicit repository entries: reserved repository-root directories such as `.metadata/` and `luaclass/`, plus directory chains that form package paths. Reserved directories are handled only by their own responsibility and never participate in package discovery; future reserved directories follow the same rule so the repo layout remains organizable. Other repository contents are outside getter domain entirely, not ignored managed objects. The local alias is a user-controlled reference/priority name, not the repository's security identity. Repository self metadata lives at `.metadata/metadata.jsonc`; if it is missing, the repository may still be used as unverified/local-source content, but repo update, signature, and trust operations are unavailable; if it exists but cannot be parsed, getter reports a repository metadata diagnostic. Repository source files are protected by the repository Git/signing/maintainer trust model, not by package `Manifest`; package `Manifest` only manages external network/dynamic-download content fetched at package-version execution time. +- **Package path**: a repository-local package identity derived from package directory hierarchy, such as `android/app/org.fdroid.fdroid` or `android/f-droid/magisk/hello`. UpgradeAll/getter domain strings, including package paths and aliases, are treated as UTF-8; getter does not detect or convert other filesystem/text encodings, and inputs in other encodings are still interpreted as UTF-8. +- **Qualified package atom**: `[::repo-name]`; omitting `::repo-name` resolves by repository priority, specifying it resolves only that local alias. If `repo/metadata.jsonc` is missing, built-in priority defaults are `local` = 100, `autogen` = -1, all other aliases = 0, with same-priority aliases resolved in lexicographic order. If present, the priority map is lookup-only: getter discovers actual repository alias directories first, then queries the map by alias; entries for nonexistent aliases are inert and do not warn, create repositories, display repositories, or participate in sorting. `generated_repository` defaults to `autogen` when omitted, and starter config should include that default as a comment users may uncomment/change. When autogen runs with target `autogen`, getter creates `repo/autogen/` if needed; any non-`autogen` target must already exist or autogen apply reports a configuration error. `generated_repository` only decides autogen output target and does not participate in package resolution except through the normal priority map. If `repo/metadata.jsonc` exists but cannot be parsed, getter reports a configuration diagnostic instead of silently falling back. +- **Provider endpoint/catalog**: an upstream service or index, such as the official F-Droid catalog endpoint or the GitHub API endpoint for a repository. It is not an UpgradeAll repository. +- **Package source**: a source declaration inside one package definition/version script that uses a provider module to discover candidates/artifacts. +- **Reusable Lua provider module/class**: a Lua helper under `luaclass/`, such as `luaclass.fdroid_android` or `luaclass.github_android_apk`, that fills common lifecycle behavior and calls getter provider host APIs. +- **Autogen pipeline**: a getter operation and repository-level `.metadata/autogen/` Lua helper that previews and writes package directories/version scripts, Manifests, optional package-local `files/` helper data, and a package-local `.autogen.jsonc` generation record from structured inputs. Autogen output is ordinary repository content. +- **Provider/source cache**: cache.db entries for upstream facts, API responses, indexes, freshness tokens, and parsed provider facts. +- **Package metadata cache**: cache.db entries for normalized package metadata/candidates/artifacts produced by evaluating a package's Lua dependency closure. + +## F-Droid model + +F-Droid is treated as a structured Android catalog provider endpoint. Its product support is autogen-first. + +### F-Droid reusable module + +The common package-authoring API should be intentionally small. The default case should need only the Android/F-Droid package name: + +```lua +local fdroid = require("luaclass.fdroid_android") + +return fdroid.package { + package_name = "org.fdroid.fdroid", +} +``` + +The package path is not duplicated inside the Lua table. Like emerge/ebuilds, getter derives package identity from the package directory path, such as `repo/official/android/f-droid/app/org.fdroid.fdroid` -> `android/f-droid/app/org.fdroid.fdroid`. A directory that directly contains `metadata.jsonc` declares a package boundary. `.autogen.jsonc` does not declare a package boundary; it is only a generated-package ownership record inside a package directory that already has `metadata.jsonc`. If `metadata.jsonc` parses correctly as package metadata, the directory is a valid package directory; if parsing fails, getter reports an invalid package metadata diagnostic for that package path. In both cases that directory is the package path endpoint, so getter does not discover nested packages below it. Package version scripts are discovered from direct child files named `.lua` whose basename does not start with `.`; removing `.lua` yields the literal version string with no SemVer requirement at discovery time. Package version Lua may read helper data under its own package directory's `files/` subtree through a package-scoped host API such as `read_package_file(path)`; the original built-in rejects paths outside that subtree and returns a Lua string without encoding/MIME/JSON/text-vs-binary interpretation, but hook code may still wrap the public `read_package_file()` name because getter core/CLI does not maintain a protective denylist of hookable public functions. File names/formats inside `files/` are package-owned. Package directory contents outside getter's explicit discovery set (`metadata.jsonc`, optional generated-package `.autogen.jsonc`, `Manifest`, enabled direct-child `*.lua`, and `files/`) are outside getter domain entirely, not ignored managed objects; the primary reason is clear responsibility boundaries, with smaller getter-core attack surface as a beneficial side effect. F-Droid display metadata such as name and description comes from the F-Droid catalog; generated metadata/version scripts should not duplicate it unless a hand-written override intentionally does so. Generated content should stay boring. Large provider behavior belongs in reusable Lua classes plus getter host APIs, not in giant generated scripts. + +The default `fdroid.package` class may infer common Android behavior: + +- installed target is Android package `package_name`; +- the autogen generator writes to a package directory such as `android/f-droid/app//`, which derives the package path `android/f-droid/app/`, unless an accepted future schema says otherwise; +- discovery reads F-Droid provider facts through getter host APIs; +- preparation normalizes F-Droid version name/code, changelog/metadata, and APK artifact descriptors; +- selection uses getter-owned version comparison and `pin_version` semantics. + +The common class may allow optional typed fields when needed, for example: + +```lua +return fdroid.package { + package_name = "org.example", + endpoint = "fdroid-official", + channel = "stable", +} +``` + +`package_name` remains the common default because F-Droid is highly structured and self-describing. F-Droid provider endpoint names come from endpoint ids/directories controlled by getter/repository configuration; the endpoint URL defaults to official F-Droid but can be customized. The model must not make third-party F-Droid endpoints, archive variants, signatures, anti-feature metadata, localized metadata, or channel-like preferences impossible to express later. + +### F-Droid provider endpoint/catalog operations + +Getter should expose provider operations equivalent to these conceptual capabilities: + +1. **Catalog/index refresh/cache**: fetch or revalidate F-Droid endpoint facts and store provider/source cache in `cache.db` with freshness metadata. +2. **Catalog query/lookup**: query cached/refreshed F-Droid catalog facts by package name, installed Android package names, or user search input. Getter owns the query semantics and DTOs. +3. **Autogen preview**: turn selected or discovered F-Droid package names into deterministic package-directory/package-output preview DTOs. +4. **Autogen apply**: write accepted package directories/version scripts to the configured generated repository alias (`generated_repository`, default `autogen`) and write a package-local `.autogen.jsonc` generation record. If the target is the default `autogen`, getter creates `repo/autogen/` at autogen runtime if needed; non-`autogen` targets must already exist. +5. **Package update check**: evaluate the generated or hand-written package Lua, call the F-Droid provider host API, normalize candidates/artifacts, compare versions, and issue getter-owned update actions. + +The exact CLI/native operation names may be chosen during implementation, but stable product operations must preserve these boundaries. + +### Explicit user selection flow + +When a user explicitly wants an F-Droid app that is not already covered by a higher-priority package: + +1. Flutter may pass a user search query, a user-entered upstream package name, or a getter-provided catalog item identifier to a getter operation. +2. Getter queries/refreshes the F-Droid catalog as needed and returns getter-owned result/preview DTOs. +3. Flutter renders those DTOs and asks for user confirmation. +4. Flutter submits only the accepted preview selection/package atoms back to getter. +5. Getter writes package directories/version scripts into the generated repository alias and writes a package-local `.autogen.jsonc` generation record. +6. Normal repository priority resolution decides which package definition is active. + +The generated repository is generated output. Getter may overwrite package directories it previously generated only when matching `.autogen.jsonc` proves ownership. If a target package directory exists without a matching generation record, apply reports a conflict and does not overwrite it. A generated-repo package directory missing `.autogen.jsonc` is a conflict rather than something getter automatically claims. If `.autogen.jsonc` exists but is malformed or schema-invalid, ordinary package discovery/evaluation is still decided by `metadata.jsonc`, but ownership-dependent autogen refresh/apply/cleanup/overwrite reports a conflict and does not auto-fix, overwrite, or delete it. Users who want to hand-author or override generated behavior should create or edit `repo/local/...`, not hand-edit `repo/autogen/...`. + +Flutter must not generate Lua, map F-Droid ids to UpgradeAll package paths/atoms, perform provider HTTP, parse F-Droid indexes, or decide whether an existing package should be shadowed. + +If a higher-priority package already provides the target package path, the F-Droid autogen preview should report that the package is already covered or would be shadowed. If the user wants to override upstream behavior, they should create or edit a `local` package directory rather than expecting the generated repository to outrank official/community packages. + +### Installed-app discovery flow + +Installed-app autogen may use the F-Droid catalog as an enrichment source: + +1. Rust/native bridge collects raw installed Android package facts through the platform adapter accepted in ADR-0009. +2. Getter matches installed package names against F-Droid catalog facts. +3. Getter previews generated F-Droid package directories/version scripts for accepted candidates. +4. Flutter renders getter-owned preview DTOs and returns user-accepted package atoms. +5. Getter writes ordinary generated package directories/version scripts and tracks accepted packages in `main.db` as already accepted by ADR-0006/ADR-0007. + +F-Droid catalog matching must not move package-path normalization or autogen decisions into Flutter/Kotlin. + +## GitHub model + +GitHub is treated as a project/release provider, not a catalog-autogen source in the first ADR-0012 scope. + +### GitHub reusable module + +The standard GitHub module should make the common package easy to author with typed project coordinates: + +```lua +local github_android = require("luaclass.github_android_apk") + +return github_android.package { + name = "F-Droid", + android_package = "org.fdroid.fdroid", + owner = "f-droid", + repo = "fdroidclient", + asset = { + include = "%.apk$", + exclude = "debug", + }, +} +``` + +A common shorthand may accept `repo = "owner/name"`, but the normalized schema should keep typed `owner` and `repo` fields internally so validation, diagnostics, cache keys, and auth/rate-limit behavior are explicit. + +The GitHub module should provide common capabilities through getter provider host APIs: + +- release listing and latest release lookup; +- tag/resource lookup when a package chooses tag-based behavior; +- release asset discovery and filtering; +- changelog/release notes extraction; +- optional authenticated API access through getter-managed provider endpoint configuration; +- live latest-commit lookup for packages that explicitly opt into live/floating behavior. + +### Release checks vs latest commit checks + +GitHub release checks are ordinary versioned update checks. They produce stable candidates/artifacts when release metadata and asset descriptors are known. + +GitHub latest-commit checks are **live/floating** behavior under ADR-0010. A latest commit id is not silently treated as an ordinary versioned release unless a package explicitly models it as stable release metadata. Live checks must be opt-in, surfaced to UI/CLI before task submission, and use the live-version semantics from ADR-0010. Free-network/high-risk status is declared per Lua script in package metadata using permissions such as `allow_free_network`; `9999.lua` commonly declares it, but the filename alone is not the permission source. + +### Asset selection + +The standard GitHub class can provide useful defaults, especially for Android APK projects, but it cannot guarantee every GitHub project works with only `owner` and `repo`. + +Package-authored filters/overrides remain valid and expected: + +- include/exclude regexes; +- artifact naming rules; +- prerelease handling; +- ABI/channel/flavor selection; +- fallback from releases to tags; +- checksum/signature sidecar matching when supported. + +Those rules belong in Lua modules/package definitions and getter validation, not Flutter. + +## Normalized provider candidate model + +Provider-specific facts should normalize into a shared candidate/artifact model before selection/action issuance. The exact Rust structs may evolve, but the domain shape should include at least: + +```text +ProviderCandidate + package_path + repo_name? + source_id + provider_kind # fdroid, github, static_mock, ... + upstream_id # package name, owner/repo, etc. + version_name + version_code? # important for Android/F-Droid + revision? # commit id or live revision when applicable + channel? + published_at? + changelog? + artifacts[] + diagnostics[] + provider_metadata_digest? +``` + +```text +ArtifactDescriptor + id/name + file_name? + locator/url + content_type? + size? + sha256?/signature? + headers?/auth_reference? + metadata_digest? +``` + +For Android/F-Droid candidates, version code is a first-class comparison/input fact when available. For GitHub release candidates, version name/tag is normally the primary version input unless the package/module extracts structured Android version facts from assets or metadata. + +Artifact descriptors inside package metadata remain package-management contracts as defined by ADR-0010. For package version scripts without `allow_free_network`, external network/dynamic-download data files or API response bodies must match that package's `Manifest` hashes where applicable. A missing `Manifest` is equivalent to an empty hash set, not an invalid package; scripts that do not fetch external network content, or only read package-local `files/`, do not need a `Manifest`, while network fetches from scripts without `allow_free_network` cannot succeed when `Manifest` is missing or empty. Scripts with `allow_free_network` are not blocked by `Manifest` membership but remain high-risk. Repository-level autogen scripts under `.metadata/autogen/` do not have a package Manifest, but autogen scripts that create package directories must generate correct package Manifests for generated packages expected to work without `allow_free_network`. Package `Manifest` is not a repository source manifest and cannot protect same-level source files such as `.autogen.jsonc`; those are protected by repository trust/signing. Refreshing metadata may discover a new valid release/artifact descriptor, but a downloaded artifact mismatch is a download/validation failure, not a cache refresh shortcut. + +## Cache and freshness model + +ADR-0012 preserves ADR-0010 cache consistency and makes the cache layers explicit. + +### Provider/source cache + +Provider/source cache entries live in `cache.db` and store upstream facts or parsed provider facts. + +Lua/provider modules opt into HTTP source caching per request through getter's host HTTP API, for example: + +```lua +local index = http_get(fdroid_index_url, { + headers = { Accept = "application/json" }, + cache = true, +}) +``` + +`cache = false` is the default so ordinary one-off HTTP calls do not silently become durable provider cache. When `cache = true`, getter owns cache key construction, storage, revalidation, stale diagnostics, and secret redaction; Lua chooses that the request should participate in HTTP/source caching but does not write cache entries itself. + +F-Droid provider cache keys must include inputs such as: + +- provider kind (`fdroid`); +- endpoint id and endpoint URL/config digest; +- auth/permission mode when applicable; +- index format/schema/parser version; +- freshness tokens such as ETag, Last-Modified, index revision, source timestamp, or response digest; +- getter/provider implementation version when it changes interpretation. + +GitHub provider cache keys must include inputs such as: + +- provider kind (`github`); +- endpoint/API base URL; +- owner/repo; +- request type (`releases`, `tags`, `latest_commit`, asset lookup, etc.); +- auth token identity/reference and rate-limit-relevant mode, without storing secrets in cache keys/logs; +- ETag/Last-Modified/API cursor/response digest; +- getter/provider implementation version when it changes interpretation. + +### Generated package records + +Generated F-Droid package directories/version scripts, package-local `files/` helper data, package Manifests, and package-local `.autogen.jsonc` generation records are repository source artifacts, not metadata cache entries. Repository trust/signing protects these source artifacts. Package `Manifest` only constrains external network/dynamic-download response bodies and cannot protect sibling source files such as `.autogen.jsonc`. Hashes inside `.autogen.jsonc` are generated-output ownership/tamper-detection facts only, not security trust, repository signing, or Manifest/download validation. The `.autogen.jsonc` `files` map covers getter-written generated output such as `metadata.jsonc`, `Manifest`, generated Lua scripts, and generated `files/...` helper files; it does not include `.autogen.jsonc` itself, avoiding self-referential hashing. + +Each generated package directory stores its own `.autogen.jsonc`. The generation record should record enough information for safe preview/regeneration/cleanup, such as: + +- generator id and generator/template version; +- provider kind and endpoint id; +- upstream package name; +- generated package path and output directory; +- input catalog revision/digest when known; +- generated file list and file hashes, excluding `.autogen.jsonc` itself and used only to decide whether files still match getter-generated output for refresh/cleanup/overwrite ownership checks; +- ownership state for refresh/cleanup conflict detection. + +If `.autogen.jsonc` is malformed or schema-invalid, it is not accepted as ownership proof; autogen refresh/apply/cleanup/overwrite reports a generated-ownership conflict rather than repairing, replacing, or deleting the directory automatically. Ordinary package discovery/evaluation remains governed by `metadata.jsonc`. + +If a reusable F-Droid module changes, ADR-0010's Lua dependency-closure cache invalidation reruns package metadata normalization. Regenerating the small package file is separate and should be previewed when generator/template identity in `.autogen.jsonc` changes materially. + +### Package metadata cache + +Package metadata cache entries live in `cache.db` and are keyed by: + +- package metadata and version-script hashes; +- loaded Lua provider module/template/helper hashes; +- parent package imports and dependency closure digest; +- Lua API/schema/runtime version; +- platform target and permission/network mode; +- provider/source cache keys or content digests used by the package evaluation; +- relevant user-independent source configuration. + +User state such as tracked/enabled/favorite/pin_version is not package metadata cache authority. Final user-state-dependent update status is computed by getter operations using main DB state plus cached/evaluated metadata. + +### Runtime task state + +Runtime actions/tasks stay process-memory only per ADR-0011. Provider cache entries, generated package source artifacts, and package metadata cache entries must not be used as a hidden task database or downloader resume mechanism. + +## Refresh semantics + +Normal refresh may use fresh provider/source cache. When freshness tokens or TTL indicate revalidation is needed, getter may revalidate provider facts before rerunning or reusing package metadata normalization. + +A forced refresh has stronger semantics: + +- it bypasses cached reads for the requested provider/package scope; +- on success, it updates or replaces relevant provider/source cache entries in `cache.db`; +- if source facts changed, affected package metadata entries must be invalidated or recomputed so later reads do not present old facts as current; +- if source facts are unchanged but freshness metadata is updated, getter may update checked-at/freshness metadata without replacing the provider body; +- it must not leave old provider/package metadata as the effective fresh value after newer facts were successfully observed. + +If forced refresh fails: + +- getter must not delete still-usable old cache merely because refresh failed; +- if old cache is used, the result must report staleness/fallback explicitly; +- diagnostics should include stable codes such as `cache.refresh_failed` and `used_stale_cache`, plus stale age/cursor/freshness details when available; +- old cache must not be presented as a successful fresh synchronization. + +These semantics apply to both F-Droid catalog/index refresh and GitHub per-project/API refresh. + +A bulk F-Droid catalog refresh may update facts for many packages, but that does not create batch download/install tasks and does not imply a parent/child task API. Batch update task semantics remain deferred. + +## Operation contract implications + +### Update check + +A live-provider update check proceeds conceptually as: + +1. Resolve the active package definition through normal repository priority. +2. Load and validate its complete Lua lifecycle contract. +3. Invoke the appropriate update-check lifecycle entrypoint. +4. Lua/provider modules call getter provider host APIs for F-Droid/GitHub facts. +5. Getter validates provider output and normalizes candidates/artifacts. +6. Getter obtains the effective local baseline using installed-version entrypoint plus `pin_version` rules from ADR-0010. +7. Getter selects the update candidate/artifact. +8. Getter issues an opaque `action_id` bound to the loaded package/Lua context and sealed action plan per ADR-0011. +9. Flutter may render the getter DTO and submit only the `action_id`. + +Flutter must not assemble or echo full action payloads, raw URLs, checksums, package paths/atoms, selected versions, source configs, or provider request parameters as task-submission input. + +### Autogen preview/apply + +Autogen operations produce preview DTOs before writing files. Applying a preview writes ordinary package directories/version scripts, Manifests, optional package-local `files/` helper data, and package-local `.autogen.jsonc` generation records through getter. Autogen apply may replace existing generated output only when matching `.autogen.jsonc` proves ownership; target directories without a matching generation record are conflicts. When refresh/overwrite ownership checks pass, getter clears the existing generated package directory contents, then writes the new generated contents into the same package directory, without preserving old unlisted extra files. If clearing any old file or subdirectory fails, the whole refresh/overwrite fails rather than being ignored. If writing new generated contents fails after clearing, the operation fails directly without rollback; the directory may be empty or partially written, and the next refresh continues by clearing and rewriting again. A generated-repo package directory missing `.autogen.jsonc` is a conflict rather than something getter automatically claims. A malformed/schema-invalid `.autogen.jsonc` is also a generated-ownership conflict for refresh/apply/cleanup/overwrite, while package discovery/evaluation remains governed by `metadata.jsonc`. Autogen apply must preserve existing user state as already accepted for installed autogen, and must respect repository priority/ownership rules. + +Autogen cleanup is allowed only after `.autogen.jsonc` ownership checks pass. If generated content no longer matches the package-local generation record, cleanup reports a conflict instead of deleting or copying it into `local`. When ownership checks pass, cleanup clears the generated package directory contents directly, including `.autogen.jsonc` and any unlisted extra files inside it, but does not delete the package directory itself. If clearing any file or subdirectory fails, the whole cleanup/update fails rather than being ignored. Getter does not classify or preserve unlisted extra files in generated package directories because they are outside getter's domain; direct directory-content clearing is simpler and more stable for generated output. Users who want to keep or override edited generated content should move it to `repo/local/...` explicitly. + +## Flutter/native bridge consequences + +Flutter may: + +- request provider catalog search/lookup/autogen preview operations; +- render getter-owned catalog/autogen/update-check DTOs; +- ask users for confirmation; +- pass accepted preview ids/package atoms/action ids back to getter; +- subscribe to runtime notifications and query current task state. + +Flutter and Kotlin must not: + +- parse F-Droid indexes or GitHub API responses; +- decide provider/source selection; +- generate Lua package text; +- map F-Droid/GitHub upstream ids into UpgradeAll package paths/atoms; +- implement version comparison/update selection; +- manage provider/package metadata cache invalidation; +- construct task/action payloads; +- implement downloader/installer task state machines. + +Android/Kotlin remains allowed to expose raw platform facts and platform capabilities through documented platform-adapter seams, such as PackageManager installed inventory or future installer handoffs. + +## CLI consequences + +Getter CLI should expose provider/autogen/update behaviors as getter operations, not as Flutter-only features. Exact command names are implementation details, but the CLI should be able to exercise: + +- F-Droid catalog refresh/query against fixtures or controlled test endpoints; +- F-Droid autogen preview/apply from explicit package names and installed-inventory fixtures; +- GitHub provider module update checks against fixtures or mocked provider host APIs; +- forced refresh success/failure and stale cache diagnostics; +- live latest-commit checks separately from normal release checks. + +CLI tests must not require cross-invocation runtime task persistence. Runtime action/task coverage remains single-process per ADR-0011. + +## Error and diagnostic model + +Provider diagnostics should be getter-owned and stable enough for UI/tests. Examples: + +```text +provider.fdroid.index_unavailable +provider.fdroid.package_not_found +provider.fdroid.parse_error +provider.github.rate_limited +provider.github.repository_not_found +provider.github.asset_not_found +provider.github.auth_required +cache.refresh_failed +used_stale_cache +autogen.already_covered +autogen.preview_stale +autogen.ownership_mismatch +package.source_invalid +``` + +Auth/rate-limit diagnostics must not print secrets. Cache keys and logs may reference credential identities or configured auth labels, but not raw tokens. + +## Testing strategy + +Use TDD for getter/provider behavior: + +- F-Droid provider endpoint cache key/freshness behavior; +- F-Droid catalog lookup from controlled fixtures; +- F-Droid autogen preview/apply output paths, package paths, package-local `.autogen.jsonc` generation records, and repository-priority skip/shadow behavior; +- generated F-Droid package evaluation through `luaclass.fdroid_android` using only `package_name` in the common case; +- GitHub release response normalization from controlled fixtures; +- GitHub asset filter/default behavior; +- GitHub latest-commit live/floating semantics; +- forced refresh success replacing cache and failure preserving stale cache with diagnostics; +- version comparison using Android version code when available and version name fallback otherwise; +- no task persistence across CLI/runtime process boundaries. + +Use BDD for user-visible Flutter/product flows: + +- user explicitly chooses an F-Droid app, sees preview, confirms, and the app appears as a generated package; +- F-Droid catalog/search UI passes only raw user input or getter-issued catalog/preview ids to getter and never interprets provider results in Dart; +- installed-autogen discovers F-Droid-covered installed apps and writes the generated repository only after confirmation; +- a hand-authored GitHub package checks releases and surfaces asset/filter diagnostics; +- stale provider cache warnings are visible without crashing or pretending success; +- update-check action submission still uses getter-issued opaque `action_id` only. + +BDD scenarios should stay focused and not duplicate getter unit coverage. + +## Non-goals + +ADR-0012 does not accept or implement: + +- Flutter-owned F-Droid/GitHub provider logic, HTTP calls, package-path/atom mapping, Lua generation, version comparison, cache invalidation, or action construction. +- A revival of the old Hub/app UUID model. +- Raw arbitrary Lua HTTP as the standard provider path; free network remains a per-script package metadata permission such as `allow_free_network`. +- Removal of user-controlled transparent URL rewrite/mirror/proxy policy; global getter-local hook scripts under `rc/hook/` are runtime/local policy discovered only from the filesystem, dot-prefixed Lua files are excluded from hook Lua discovery, there is no hook registry/metadata/disabled state, enabled hooks can wrap public host functions such as `http_get()` or `read_package_file()`, call original unhooked entrypoints through `getter_builtin.`, and affect package version scripts, repository-level autogen scripts, and `luaclass/` calls, but content trust still depends on package Manifest hashes or explicit free-network permission. +- GitHub global catalog/search/autodiscovery as a first-class F-Droid-like autogen source. +- A guarantee that every GitHub project works with only `owner` and `repo`; package-authored asset/version filters remain valid. +- Real downloader implementation. +- Android installer implementation, PackageInstaller semantics, intent/URI/SAF/FileProvider policy, Shizuku/root behavior, or install-result platform details. +- Android foreground/background service policy. +- Android system notifications. +- Runtime task persistence, app-restart recovery, downloader-style resume/recovery, or daemon behavior. +- Batch parent/child update/download/install task APIs. +- A product UI commitment for full F-Droid catalog browsing/search in the first implementation slice. Getter-owned catalog query can exist as an autogen input; product UI breadth can be staged later. + +## Consequences + +Positive: + +- F-Droid's structured catalog is used where it is strongest: generating ordinary package definitions from package names/catalog facts. +- GitHub remains simple for the common hand-authored project case while still allowing package-specific asset rules. +- Generated packages obey the same repository/overlay model as all other packages. +- Provider execution, cache consistency, version selection, and action issuance remain in getter. +- Flutter can expose useful provider/autogen UX without owning product logic. + +Costs: + +- Getter needs a real provider/source cache layer before live provider behavior is product-complete. +- Package-local `.autogen.jsonc` generation records must be robust enough for regeneration and cleanup. +- F-Droid catalog refresh can affect many packages, requiring careful invalidation/lazy recomputation. +- GitHub rate limits/auth and asset selection need explicit diagnostics. +- Documentation must consistently distinguish UpgradeAll repositories from upstream provider endpoints. + +## Follow-up implementation notes + +Suggested first implementation slices after this ADR is accepted: + +1. Add typed provider/source cache storage primitives in getter `cache.db` with forced-refresh/stale diagnostics tests. +2. Add fixture-backed F-Droid catalog provider and `luaclass.fdroid_android` package evaluation tests. +3. Add F-Droid autogen preview/apply from explicit package names, writing small package directories/version scripts, package Manifests, optional package-local `files/` helper data, and package-local `.autogen.jsonc` generation records without embedded duplicate `id` fields. +4. Update package schema/evaluation so package path is directory-derived rather than a required Lua table field. +5. Integrate F-Droid catalog matching into installed-autogen preview/apply. +6. Add fixture-backed GitHub release provider and standard `luaclass.github_android_apk` tests for release/asset normalization. +7. Add GitHub latest-commit as live/floating check only. +8. Expose native/Flutter DTOs for F-Droid autogen preview and provider diagnostics without moving provider decisions into Dart/Kotlin. diff --git a/docs/architecture/upgradeall-getter-rewrite-wiki.md b/docs/architecture/upgradeall-getter-rewrite-wiki.md index 9b856926..f216b849 100644 --- a/docs/architecture/upgradeall-getter-rewrite-wiki.md +++ b/docs/architecture/upgradeall-getter-rewrite-wiki.md @@ -254,29 +254,29 @@ Rust getter core + native bridge ## 4. Package-centric 模型 -### 4.1 Package ID +### 4.1 Package path -Package 主 ID 使用 UpgradeAll 自己的可读 namespace，不使用 UUID 作为主身份。 +Package 主身份使用 UpgradeAll 自己的可读 repository-local package path，不使用 UUID 作为主身份，也不在 Lua table 里重复声明 `id` 字段。 示例： ```text -android/org.fdroid.fdroid -android/com.termux -magisk/zygisk-next -generic/example-tool +android/app/org.fdroid.fdroid +android/app/com.termux +android/magisk/zygisk-next +generic/tool/example-tool ``` 设计理由： - UUID 对用户无意义。 -- package ID 应可读、可 diff、可手写、可在 issue/文档中引用。 +- package path 应可读、可 diff、可手写、可在 issue/文档中引用。 - Android 和 Magisk 迁移可以自然映射。 旧数据映射： -- 旧 Android app：`android/`。 -- 旧 Magisk module：`magisk/`。 +- 旧 Android app：`android/app/`。 +- 旧 Magisk module：`android/magisk/`。 ### 4.2 APP/package-centric，而不是 hub-centric @@ -303,17 +303,20 @@ CLI/UI 命名建议： 例如： ```lua +#!/bin/upa-lua v1 +-- repo/official/android/app/org.fdroid.fdroid/9999.lua +-- package path: android/app/org.fdroid.fdroid return android_app { - id = "android/org.fdroid.fdroid", - name = "F-Droid", installed = android.package("org.fdroid.fdroid"), sources = { - fdroid.package { package = "org.fdroid.fdroid" }, + fdroid.package { package_name = "org.fdroid.fdroid" }, github.release { repo = "f-droid/fdroidclient" }, }, } ``` +For F-Droid sources, display metadata such as app name/description comes from the self-describing F-Droid catalog rather than duplicated generated Lua fields. + source priority 可以来自 package 默认值，也可以被 user state 覆盖。 --- @@ -328,7 +331,7 @@ Repository 可以是： - official：官方包定义仓库。 - community：社区包定义仓库。 -- local_autogen：自动生成的本地包仓库。 +- autogen：默认的自动生成包仓库。 - local：用户手写/覆盖仓库。 ### 5.2 Priority 规则 @@ -337,20 +340,22 @@ Repository 可以是： - 数字越大优先级越高。 - getter resolved view 只看最高优先级 package。 -- 用户可以手动修改 repo priority。 +- 用户可以通过 `repo/metadata.jsonc`、UI 或 CLI 修改 repo priority。 默认建议： ```text -local 100 用户手写覆盖，默认最高 -official 0 官方仓库 -community 0 或用户配置 -local_autogen -1 根据已安装应用自动生成的 fallback +local 100 用户手写覆盖，默认最高 +official 0 官方仓库 +community 0 或用户配置 +autogen -1 根据已安装应用/显式 autogen 生成的 fallback ``` +`repo/metadata.jsonc` 还可以包含 `generated_repository`，默认值是 `autogen`。初始配置文件应把这个默认值写成注释，用户可取消注释后改成其它已有 alias。实际运行 autogen 时，如果目标是默认 `autogen` 且 `repo/autogen/` 不存在，getter 创建它；如果用户配置的是非 `autogen` alias，则目标目录必须已经存在，否则 autogen apply 报配置错误。`generated_repository` 只决定 autogen 输出目标，package resolution 仍然只看 priority。 + 注意：`local` 只是默认最高，用户可以自己改优先级。 -### 5.3 local 与 local_autogen 的区别 +### 5.3 local 与 autogen 的区别 `local`： @@ -359,14 +364,12 @@ local_autogen -1 根据已安装应用自动生成的 fallback - 默认 priority 最高。 - 普通清理按钮不应删除 `local`。 -`local_autogen`： +`autogen` / configured generated repository： -- 用户点击“从已安装应用生成”后产生。 +- 用户点击“从已安装应用生成”或显式选择 provider autogen 后产生。 - 是低优先级 fallback。 - 上游 official package 出现后，official 会覆盖它。 -- 清理按钮只作用于该 autogen 仓库。 - -仓库名固定为 `local_autogen`。它表达“本地自动生成的 fallback 仓库”。 +- 清理按钮只作用于 configured generated repository。 ### 5.4 首次旧数据迁移与 autogen 的区别 @@ -379,125 +382,116 @@ local_autogen -1 根据已安装应用自动生成的 fallback 普通 installed autogen： - 是用户主动点击按钮触发。 -- 生成到 `local_autogen`。 +- 生成到 `generated_repository` 指定的仓库，默认 `autogen`。 - 不是首启迁移的一部分。 --- ## 6. Repository 文件布局 -建议 layout： +Accepted layout now has a getter data directory with `repo/` and `rc/` as siblings: ```text -repo/ - repo.toml - - packages/ - android/ - org.fdroid.fdroid.lua - com.termux.lua - magisk/ - zygisk-next.lua - - lib/ - std.lua - github.lua - fdroid.lua - google_play.lua - coolapk.lua - android.lua - magisk.lua - github_android_apk.lua - fdroid_android_apk.lua - - templates/ - android_installed_app.lua - magisk_installed_module.lua - github_android_apk.lua - fdroid_android_apk.lua +/ + main.db + cache.db + repo/ + metadata.jsonc + official/ + .metadata/ + metadata.jsonc + autogen/ + metadata.jsonc + android.lua + luaclass/ + github_android_apk.lua + fdroid_android.lua + android/ + app/ + org.fdroid.fdroid/ + metadata.jsonc + Manifest + 1.20.0.lua + 9999.lua + files/ + helper-data.json + autogen/ + android/ + app/ + org.fdroid.fdroid/ + metadata.jsonc + .autogen.jsonc + Manifest + 1.20.0.lua + rc/ + hook/ + 10-http-rewrite.lua ``` -`repo.toml` 示例： - -```toml -id = "official" -name = "UpgradeAll Official" -priority = 0 -api_version = "getter.repo.v1" +`repo/metadata.jsonc` is getter-owned local repository registry/config, not publishable repository metadata: + +```jsonc +{ + "version": 1, + // Autogen writes to "autogen" by default. Uncomment and change this + // if generated packages should target another existing repository alias. + // "generated_repository": "autogen", + "priority": { + "local": 100, + "official": 0, + "autogen": -1 + } +} ``` -### 6.1 packages/ +Repository self metadata lives under `repo//.metadata/metadata.jsonc`. Shared Lua classes/helpers live under `luaclass/`, not `lib/`. Repository-level autogen scripts/metadata live under `.metadata/autogen/`. Runtime/local policy hooks live under top-level `rc/hook/`, not under `repo/`. -`packages/` 里是最终被 getter 解析的 package Lua 文件。 +### 6.1 Package directories -路径建议： +A package is a directory that directly contains `metadata.jsonc`. Package identity is derived from the repository-local directory path, for example: ```text -packages/android/org.fdroid.fdroid.lua -packages/magisk/zygisk-next.lua +repo/official/android/app/org.fdroid.fdroid/ -> android/app/org.fdroid.fdroid +repo/official/android/magisk/zygisk-next/ -> android/magisk/zygisk-next ``` -路径可推导 package id： - -```text -packages/android/org.fdroid.fdroid.lua -> android/org.fdroid.fdroid -``` +A package directory contains `metadata.jsonc`, optional generated-package `.autogen.jsonc`, optional `Manifest`, direct child version scripts such as `1.20.0.lua` or `9999.lua`, and optional package-local helper files under `files/`. There is no `versions/` subdirectory. Lua package files do not declare a duplicate package id; getter derives identity from the package directory path. -文件内也应声明同样 id，getter 校验路径 id 和声明 id 一致。 +### 6.2 luaclass/ -### 6.2 lib/ - -`lib/` 里是 reusable Lua module。 +`luaclass/` contains reusable Lua modules/classes. 注意：这里的角色类似 Gentoo eclass，但项目语法里不需要真的叫 eclass。 原则： -- 不限定 lib 里写什么。 +- 不限定 helper 里写什么。 - 只抽象重复代码。 -- 可以提供高层 helper，例如 `github_android_apk { ... }`。 -- package 文件通过 Lua 原生 `require()` 导入。 +- 可以提供高层 helper，例如 `github_android_apk { ... }` 或 `fdroid.package { ... }`。 +- package 文件通过 Lua `require()` 导入。 示例： ```lua -local github_android = require("lib.github_android_apk") +local github_android = require("luaclass.github_android_apk") ``` -### 6.3 templates/ - -`templates/` 里是 Lua 生成器，用于生成新的 package Lua 文件内容。 +### 6.3 Repository autogen scripts -这参考 Funtoo Metatools/autogen： +`.metadata/autogen/` contains repository-level Lua generators/templates for producing package directories from installed inventory or structured provider/catalog input. Generated package output is ordinary package directories plus a package-local `.autogen.jsonc` ownership record, not a repo-level generation table. -- Funtoo metatools 用 autogen.py/autogen.yaml 查询 upstream 并生成 ebuild。 -- UpgradeAll 的 templates 用 Lua 根据 installed inventory 或用户输入生成 package Lua。 +Example generated package output: -template 直接返回文件路径和文本内容，而不是返回 AST。 - -示例： - -```lua -return template { - id = "android_installed_app", - - generate = function(ctx, input) - return { - path = "packages/android/" .. input.package_name .. ".lua", - content = [[ -local android = require("lib.android") - -return android.local_app { - id = "android/]] .. input.package_name .. [[", - name = "]] .. input.label .. [[", - package_name = "]] .. input.package_name .. [[", -} -]] - } - end -} +```text +repo/autogen/android/app/org.fdroid.fdroid/ + metadata.jsonc + .autogen.jsonc + Manifest + 1.20.0.lua ``` +`.autogen.jsonc` records generator identity, input facts, generated file hashes, and cleanup/refresh ownership state. It is not security trust and does not replace package `Manifest`. + --- ## 7. Lua package API @@ -530,20 +524,20 @@ return android.local_app { 父包导入使用 host helper： ```lua -local base = package_from("official", "android/org.fdroid.fdroid") +local base = package_from("official", "android/app/org.fdroid.fdroid") ``` 理由： -- package id 里有 `/`、`.`、`-` 等字符。 +- package path 里有 `/`、`.`、`-` 等字符。 - Lua 原生 `require()` 会把 `.` 当模块路径分隔。 -- parent package import 需要显式 repo id，避免 priority/递归歧义。 +- parent package import may use an explicit repository alias to avoid priority/recursion ambiguity. - 这是 host function，不是新语法。 Reusable module 仍使用 Lua `require()`： ```lua -local github = require("lib.github") +local github = require("luaclass.github") ``` ### 7.4 Lua/Rust boundary / Lua/Rust 边界 @@ -576,12 +570,13 @@ Lua package scripts 在边界返回 JSON-like object/table。 官方 package： ```lua -local github_android = require("lib.github_android_apk") +#!/bin/upa-lua v1 +-- repo/official/android/app/org.fdroid.fdroid/9999.lua +-- package path: android/app/org.fdroid.fdroid +local github_android = require("luaclass.github_android_apk") return github_android { - id = "android/org.fdroid.fdroid", - name = "F-Droid", - android_package = "org.fdroid.fdroid", + android = { package_name = "org.fdroid.fdroid" }, repo = "f-droid/fdroidclient", asset_pattern = "%.apk$", } @@ -674,7 +669,7 @@ end) - 简单 metadata 修改用 table override。 - 非平凡修改用 function override。 -注意：override helper 是 Lua lib/helper 问题，不是 Rust API 问题。Rust 只关心最终返回的 JSON-like package object 是否符合 schema。 +注意：override helper 是 Lua helper/module 问题，不是 Rust API 问题。Rust 只关心最终返回的 JSON-like package object 是否符合 schema。 --- @@ -834,15 +829,26 @@ return { ## 10. Permissions / network model -### 10.1 默认无直接网络 +### 10.1 默认无 Lua 原生网络 + +默认情况下，Lua package script 不获得 Lua 标准库/第三方库形式的直接网络能力。 -默认情况下，Lua package script 不获得直接网络 API。 +网络请求通过 getter 暴露的 host API 执行，例如： -它可以通过 getter 暴露的 provider/source API 间接获取 release 信息。 +```lua +local body = http_get(url, { + headers = { Accept = "application/json" }, + cache = true, +}) +``` + +`cache` 默认是 `false`。Lua/provider module 通过 `cache = true` 主动把单次 HTTP 请求纳入 getter-owned HTTP/source cache；getter 负责 cache key、持久化、revalidation、stale diagnostics 和 secret redaction，Lua 只表达该请求是否应缓存。 + +标准 provider module/class 可以在声明的 provider/source 语义下使用该 host HTTP API 获取 release/catalog 信息。 ### 10.2 自由网络权限 -如果 package 声明自由网络权限，getter 才向 Lua 环境暴露直接网络接口。 +如果 package 需要超出标准 provider module 的任意 upstream 访问，它必须声明自由网络权限，getter 才向该 Lua 环境暴露对应 host HTTP 能力。 该权限用于类似 live/9999 包或特殊 upstream 逻辑。 @@ -910,9 +916,8 @@ v1 暂不做 repo/script/artifact 强校验。 Cache key 应包含： ```text -repo id -repo revision/hash -package file hash +repository alias and verified repository metadata/revision facts +package path and Lua dependency/file hashes Lua API version getter version or package API version platform target @@ -939,28 +944,21 @@ Android 上 repo sync 可以先采用 archive zip/tar 或 bundled repo snapshot - 可按 package/repository scope 区分。 - 参考 emerge bashrc 的精神：全局 hook 根据上下文做调整。 -建议文件： - -```text -config/hooks/download_rewrite.lua -``` +Accepted hook location is top-level runtime config, `rc/hook/*.lua`. Hooks wrap public getter host functions and call original unhooked entrypoints through `getter_builtin.`. 示例： ```lua -return function(ctx, req) - if ctx.repo_id == "official" and ctx.package_id == "android/com.foo" then - req.url = req.url:gsub("https://github.com/", "https://mirror.example/github/") - end +#!/bin/upa-lua v1 +local upstream_http_get = getter_builtin.http_get - return req +function http_get(url, opts) + local rewritten = url:gsub("https://github.com/", "https://mirror.example/github/") + return upstream_http_get(rewritten, opts) end ``` -执行阶段： - -- `resolve` 生成 DownloadRequest 后。 -- downloader submit 前。 +Hooks are loaded before each Lua execution environment in deterministic filename order. Enabled hook load/init failure fails the current Lua execution. Hooks do not bypass Manifest validation: non-`allow_free_network` package scripts still require response-body SHA-512 membership in the package `Manifest`. --- @@ -1011,7 +1009,7 @@ Room DB 信息： 迁移生成 `local` 是特殊情况，只做一次。 -普通 installed autogen 不写 local，而写 `local_autogen`。 +普通 installed autogen 不写 `local`，而写 `repo/metadata.jsonc` 的 `generated_repository` 目标，默认 `autogen`。 ### 13.4 迁移匹配策略 @@ -1045,10 +1043,10 @@ Room DB 信息： 3. getter 找出可生成的候选列表。 4. UI 展示 getter-owned preview DTO。 5. 用户 yes/no 确认。 -6. getter 写入 `local_autogen` repo。 +6. getter 写入 configured generated repository，默认 `repo/autogen/`。 7. 生成后不会自动消失。 -实现进展：Flutter 产品 APK 通过 `app_flutter/android/getter_bridge` 打包一个 slim native bridge library，包含 Rust `api_proxy`、`NativeLib` 和 Android installed-inventory facts provider。`api_proxy` 已提供 installed-autogen preview/apply JNI entrypoints；它们调用 Rust-active platform adapter 扫描 Android PackageManager 原始事实，再调用 getter-owned `getter-operations` 执行 `local_autogen` preview/apply。Flutter 已新增 installed-autogen 页面和 `MethodChannelGetterAdapter`，只渲染 getter-owned preview/apply DTO 并把用户接受的包 id 传回 getter；不能引入 Dart-led installed inventory scanner 或在 Dart/Kotlin 中生成 package id。 +实现进展：Flutter 产品 APK 通过 `app_flutter/android/getter_bridge` 打包一个 slim native bridge library，包含 Rust `api_proxy`、`NativeLib` 和 Android installed-inventory facts provider。`api_proxy` 已提供 installed-autogen preview/apply JNI entrypoints；它们调用 Rust-active platform adapter 扫描 Android PackageManager 原始事实，再调用 getter-owned `getter-operations` 执行 installed-autogen preview/apply。Flutter 已新增 installed-autogen 页面和 `MethodChannelGetterAdapter`，只渲染 getter-owned preview/apply DTO 并把用户接受的 package path 传回 getter；不能引入 Dart-led installed inventory scanner 或在 Dart/Kotlin 中生成 package path。 ### 14.2 清理流程 @@ -1059,9 +1057,9 @@ Room DB 信息： 3. getter 计算将删除列表。 4. UI 展示 getter-owned preview DTO。 5. 用户 yes/no 确认。 -6. getter 删除 `local_autogen` 中不再安装的记录/文件。 +6. getter 清理 configured generated repository 中不再安装且 ownership checks 通过的 generated package directory contents。 -普通清理按钮只作用于 `local_autogen`，不删除 `local`。 +普通清理按钮只作用于 configured generated repository，不删除 `local`。 --- diff --git a/docs/lua-api/package-lifecycle.md b/docs/lua-api/package-lifecycle.md index 445637ec..4edcb7e4 100644 --- a/docs/lua-api/package-lifecycle.md +++ b/docs/lua-api/package-lifecycle.md @@ -37,6 +37,19 @@ Match installed inventory items to this package. Query sources/providers and return release candidates. +Network access is through getter host APIs. Package version Lua may also read package-local helper data under its own package directory's `files/` subtree through a package-scoped getter host API such as `read_package_file(path)`, where `path` is relative to `files/`; the original built-in does not expose real filesystem paths or freely read arbitrary package/repository directories outside that subtree and returns a Lua string without encoding/MIME/JSON/text-vs-binary interpretation. Hook code may still wrap the public `read_package_file()` name because getter core/CLI does not maintain a protective denylist of hookable public functions. + +Provider modules can opt specific HTTP requests into getter-owned source caching: + +```lua +local body = http_get(url, { + headers = { Accept = "application/json" }, + cache = true, +}) +``` + +`cache` defaults to `false`; Lua chooses cache participation, while getter owns cache keys, storage, revalidation, stale diagnostics, and secret redaction. + ## prepare Normalize, filter and enrich release candidates. @@ -61,12 +74,13 @@ return { } ``` -As the first offline/mock-provider bridge toward this lifecycle, package Lua may also declare static `updates` candidates in the package table. Getter validates this table, routes it through a mock provider boundary (`StaticPackageUpdatesProvider`), performs Rust-owned selection/version comparison, and issues opaque runtime `action_id`s from the selected candidate; Flutter must still return only the getter-issued `action_id` and must not assemble download/install action payloads. +As the first offline/mock-provider bridge toward this lifecycle, a package version script may also declare static `updates` candidates. Getter validates this table, routes it through a mock provider boundary (`StaticPackageUpdatesProvider`), performs Rust-owned selection/version comparison, and issues opaque runtime `action_id`s from the selected candidate; Flutter must still return only the getter-issued `action_id` and must not assemble download/install action payloads. ```lua -return package_def { - id = "android/org.fdroid.fdroid", - name = "F-Droid", +#!/bin/upa-lua v1 +-- repo/official/android/app/org.fdroid.fdroid/1.2.0.lua +-- package path android/app/org.fdroid.fdroid, version 1.2.0 +return package_version { updates = { { version = "1.2.0", @@ -94,7 +108,7 @@ Optional post-update hook. Most persistent state changes should remain in Rust c ## Offline validation -`getter --data-dir repo validate ` validates repository layout and package schema without network access. The command evaluates local package Lua files with the same constrained `lib/` module loading used by `repo eval`/`package eval`, then returns a getter-owned diagnostic report: +`getter --data-dir repo validate ` validates repository layout and package schema without network access. The command evaluates local package metadata and version scripts with the same constrained `luaclass/` module loading used by `repo eval`/`package eval`, then returns a getter-owned diagnostic report: ```json { @@ -105,10 +119,10 @@ Optional post-update hook. Most persistent state changes should remain in Rust c { "severity": "error", "code": "package.schema", - "message": "required string field 'name' is missing", - "package_id": "android/org.fdroid.fdroid", + "message": "required field 'android.package_name' is missing", + "package_path": "android/app/org.fdroid.fdroid", "location": { - "path": "repo/packages/android/org.fdroid.fdroid.lua" + "path": "repo/official/android/app/org.fdroid.fdroid/metadata.jsonc" } } ] @@ -117,18 +131,18 @@ Optional post-update hook. Most persistent state changes should remain in Rust c Initial stable diagnostic codes include: -- `repository.read_repo_toml` -- `repository.parse_repo_toml` +- `repository.read_metadata` +- `repository.parse_metadata` - `repository.invalid_id` - `repository.unsupported_api_version` - `repository.missing_directory` -- `repository.read_packages_dir` - `repository.invalid_package_path` -- `repository.invalid_package_id` -- `repository.hash_package_file` -- `package.read_file` +- `package.read_metadata` +- `package.parse_metadata` +- `package.read_version_script` - `package.lua_runtime` - `package.not_a_table` +- `package.missing_api_version` - `package.unsupported_value` - `package.schema` - `package.domain` diff --git a/docs/lua-api/permissions.md b/docs/lua-api/permissions.md index e30b2e78..a1330be1 100644 --- a/docs/lua-api/permissions.md +++ b/docs/lua-api/permissions.md @@ -6,19 +6,44 @@ ## Default -Lua package scripts do not receive direct network access by default. +Lua package scripts do not receive Lua-native or Flutter/Kotlin-owned network access by default. -They can use getter-provided provider/source APIs. +They can use getter-provided provider/source host APIs. HTTP requests go through getter-managed functions such as: + +```lua +local body = http_get(url, { + headers = { Accept = "application/json" }, + cache = true, +}) +``` + +`cache` defaults to `false`. Passing `cache = true` opts that request into getter-owned provider/source caching; getter owns cache keys, persistence, revalidation, stale diagnostics, permissions, and secret redaction. ## Free network permission -A package may declare free network access for live/9999-like logic or unusual upstreams. +A package declares free network access per Lua script in `metadata.jsonc`, for example: + +```jsonc +{ + "lua": { + "9999.lua": { + "permission": ["allow_free_network"] + } + } +} +``` + +`allow_free_network` may be attached to `9999.lua` or to any fixed-version script. The `lua` map is lookup-only: getter first discovers an enabled Lua file from the filesystem, then queries this map by basename. Getter does not enumerate the map to discover scripts or warnings. `9999.lua` commonly needs it, but the filename alone does not grant free network or force the warning if metadata does not grant that permission. A version script omitted from the `lua` map defaults to `permission: []`. Entries for nonexistent files or dot-prefixed Lua files are inert and do not enable, display, validate, or otherwise bring those files under getter management. + +Without `allow_free_network`, any external network/dynamic-download data file or API response body used by that script must have a SHA-512 hash listed in the package `Manifest`. If the response hash is not listed or does not match, getter rejects the download/use. A missing `Manifest` is equivalent to an empty hash set, not an invalid package, so network fetches from scripts without `allow_free_network` cannot succeed when `Manifest` is missing or empty. Scripts that do not fetch external network content, or only read package-local `files/`, do not need a `Manifest`. Scripts with `allow_free_network` are not blocked by `Manifest` membership but remain high-risk. + +Package-local reads from the script's own package `files/` subtree through `read_package_file(path)` are not free-network access. The path is relative to `files/`; the original built-in does not expose real filesystem paths or a general `io.open` escape hatch and returns a Lua string without encoding/MIME/JSON/text-vs-binary interpretation. Hook code may still wrap the public `read_package_file()` name as local user policy because getter core/CLI does not maintain a protective denylist of hookable public functions. These files are repository source artifacts covered by repository review/signing/trust, and getter does not assign product semantics to their file names or formats. When declared: -- getter exposes a direct Lua network host API; +- getter exposes the relevant host HTTP API to that Lua environment; - Flutter displays a yellow warning tag at App detail source/version level; -- use is not blocked. +- use is not blocked by Manifest membership, but normal diagnostics/download validation still apply. ## Timeouts diff --git a/docs/lua-api/repository-layout.md b/docs/lua-api/repository-layout.md index 9ac55c51..50c113d3 100644 --- a/docs/lua-api/repository-layout.md +++ b/docs/lua-api/repository-layout.md @@ -4,56 +4,233 @@ > Date: 2026-06-21 > Project: UpgradeAll rewrite — Flutter APP + Rust getter core + Lua package repository model -Recommended layout: +Recommended getter data directory layout: ```text -repo/ - repo.toml - packages/ - android/ - org.fdroid.fdroid.lua - magisk/ - zygisk-next.lua - lib/ - github.lua - android.lua - github_android_apk.lua - templates/ - android_installed_app.lua +/ + main.db + cache.db + repo/ + metadata.jsonc + official/ + .metadata/ + metadata.jsonc + autogen/ + metadata.jsonc + android.lua + luaclass/ + android.lua + github_android_apk.lua + android/ + app/ + org.fdroid.fdroid/ + metadata.jsonc + Manifest + 1.20.0.lua + 9999.lua + files/ + helper-data.json + f-droid/ + app/ + org.fdroid.fdroid/ + metadata.jsonc + Manifest + 1.20.0.lua + local/ + android/ + app/ + org.example.app/ + metadata.jsonc + Manifest + 1.2.3.lua + rc/ + hook/ + http.lua ``` -## repo.toml +## Repository root metadata + +`repo/metadata.jsonc` is getter-owned repository registry/config state, not part of any one package repository. It stores repository-related local settings only, currently repository ordering and generated-output target rules: + +```jsonc +{ + "version": 1, + // Autogen writes to "autogen" by default. Uncomment and change this + // if generated packages should target another existing repository alias. + // "generated_repository": "autogen", + "priority": { + "local": 100, + "official": 0, + "autogen": -1 + } +} +``` + +The user may change priority through UI, CLI, or this getter-owned metadata file. Higher priority wins for unqualified package atoms. If `repo/metadata.jsonc` is missing, getter uses built-in priority defaults: `local` = 100, `autogen` = -1, every other alias = 0, with same-priority aliases resolved in lexicographic order. If present, the priority map is lookup-only: getter discovers actual repository alias directories first, then queries the map by alias; entries for nonexistent aliases are inert and do not warn, create repositories, display repositories, or participate in sorting. The generated repository target defaults to `generated_repository = "autogen"` when the field is omitted. When an autogen operation runs with target `autogen`, getter creates `repo/autogen/` if it does not already exist. If `generated_repository` is set to any alias other than `autogen`, that target repository directory must already exist; otherwise autogen apply reports a configuration error instead of creating it. `generated_repository` only decides autogen output target and does not participate in package resolution except through the normal priority map. If `repo/metadata.jsonc` exists but cannot be parsed, getter reports a configuration diagnostic instead of silently falling back. Upstream repository updates must not overwrite this file. Hook/runtime policy is not stored here. + +## Runtime configuration root + +`rc/` is the getter local runtime/config policy root. It is a top-level sibling of `repo/` under the getter data directory, beside storage files such as `main.db` and `cache.db`. It is not a repository root and does not participate in repository or package discovery. Current defined content is `rc/hook/*.lua` for runtime hook policy. Future runtime/local policy such as environment, credential, or network behavior belongs under `rc/`, not in `repo/metadata.jsonc`, which remains repository-related registry/config. + +## Runtime hooks + +Runtime/local hooks live under `rc/hook/`, not under `repo/`. Hooks are runtime policy, while `repo/metadata.jsonc` stays repository-related configuration. Hooks are discovered only from the filesystem: getter lists enabled `rc/hook/*.lua` files, excludes basenames starting with `.`, sorts them deterministically, then loads them before every Lua execution environment. There is no hook registry, metadata map, or persistent disabled-hook state. Enabled hooks can wrap getter-exposed Lua host functions such as `http_get()` for transparent URL replacement or similar local policy. A dot-prefixed Lua file is outside getter management and is not a hook entry. + +## Repository alias + +`repo/metadata.jsonc` is the only current repo-root reserved entry. Every direct child directory of `repo/` is a local repository alias. For example, `repo/official` has alias `official`. If the user clones or renames the official repository to `repo/a`, the local alias is `a`. Renaming is an intentional local rule change that lets users fork, replace, or interpose repository layers. Future repo-root reserved entries require an explicit design/ADR because they occupy alias namespace; no names such as `hook` or `rc` are pre-reserved under `repo/`. + +Repository self-metadata lives at `.metadata/metadata.jsonc` and records schema version, upstream hosting URL, description, maintainers/co-maintainers, and repository signature/trust information. Security/trust checks use this verified repository metadata, not the local alias. If `.metadata/metadata.jsonc` is missing, the repository may still be used as unverified/local-source content, but repo update, signature, and trust operations are unavailable. If `.metadata/metadata.jsonc` exists but cannot be parsed, getter reports a repository metadata diagnostic. A directory named `official` is not trusted merely because of its name. + +Inside a repository alias directory, getter only considers explicit entries: reserved repository-root directories such as `.metadata/` and `luaclass/`, plus directory chains that form package paths. Reserved directories are handled only by their own responsibility and never participate in package discovery; package paths cannot begin with reserved names such as `.metadata` or `luaclass`. Future repository-root reserved directories follow the same rule so the repo layout remains organizable. A directory that directly contains `metadata.jsonc` declares a package boundary. `.autogen.jsonc` does not declare a package boundary; it is only a generated-package ownership record inside a package directory that already has `metadata.jsonc`. If `metadata.jsonc` parses correctly as package metadata, the directory is a valid package directory; if parsing fails, getter reports an invalid package metadata diagnostic for that package path. In both cases, that directory is the package path endpoint and getter does not discover nested packages below it. Other files or directories such as `README.md`, `docs/`, or random helper files are outside getter domain entirely: they are not parsed, validated, displayed, warned about, or modeled as ignored managed objects. The primary reason is clear responsibility boundaries; a smaller getter-core attack surface is a beneficial side effect. + +## Package directories + +Package directories are final package definitions consumed by getter. UpgradeAll/getter domain strings, including package paths and aliases, are treated as UTF-8. Getter does not detect or convert other filesystem/text encodings; inputs in other encodings are still interpreted as UTF-8. + +Package path is derived from the directory path relative to the repository: + +```text +repo/official/android/app/org.fdroid.fdroid -> android/app/org.fdroid.fdroid +repo/official/android/f-droid/magisk/hello -> android/f-droid/magisk/hello +``` + +Package references use qualified atoms: + +```text +android/f-droid/magisk/hello +android/f-droid/magisk/hello::official +``` + +If `::repo-name` is omitted, getter resolves by `repo/metadata.jsonc` priority. If it is present, getter resolves only that local repository alias. + +## Package metadata + +`metadata.jsonc` stores package-level metadata. Android install identity uses `android.package_name`, not an ambiguous `package_id` field. Lua script permissions are declared per file: + +```jsonc +{ + "type": "android:app", + "android": { + "package_name": "com.example.app" + }, + "homepage": "https://example.com", + "description": "...", + "lua": { + "9999.lua": { + "permission": ["allow_free_network"] + } + } +} +``` + +`allow_free_network` is the source of the user-visible high-risk warning. The `lua` map is lookup-only: getter first discovers an enabled Lua file from the filesystem, then queries this map by basename. Getter does not enumerate the map to discover scripts or warnings. `allow_free_network` can apply to `9999.lua` or to fixed-version scripts. `9999.lua` commonly needs it, but the filename alone does not grant free network or force the warning if metadata explicitly does not grant that permission. A version script omitted from the `lua` map defaults to `permission: []`. Entries for nonexistent files or dot-prefixed Lua files are inert and do not enable or display anything. + +Package metadata has no separate schema version field; the repository metadata version covers the repository content schema. + +## Version scripts + +Fixed package versions live directly in the package directory as `.lua`: + +```text +1.2.3.lua +1.2.3-r1.lua +v1.2.3.lua +2026.06.25.lua +``` + +Getter discovers package version scripts only from direct child files of the package directory whose basename ends with `.lua` and does not start with `.`. Removing the `.lua` suffix yields the literal version string; getter does not require SemVer or otherwise constrain the version syntax at discovery time. + +Live/floating package behavior uses the special script name: + +```text +9999.lua +``` + +Every enabled Lua script must specify the interpreter/API version on the first line. This version is required and has no implicit default. Lua file discovery excludes any Lua file whose basename starts with `.`, for example `.9999.lua`: getter does not parse, validate, execute, display, or apply permission metadata to it as Lua. This dot-prefix rule is only for Lua file discovery; non-Lua package files are governed by the explicit getter file whitelist, so `.autogen.jsonc` is managed when present in a generated package. Script permissions still come from package `metadata.jsonc`, not from the shebang: + +```lua +#!/bin/upa-lua v1 +``` + +## Package-local files + +A package directory may contain a `files/` subdirectory for package-local helper data: -```toml -id = "official" -name = "UpgradeAll Official" -priority = 0 -api_version = "getter.repo.v1" +```text +files/ + helper-data.json + patch.diff +``` + +Package Lua may read files under its own package directory's `files/` subtree through a package-scoped getter host API such as: + +```lua +local body = read_package_file("helper-data.json") ``` -## packages/ +The path is relative to `files/`. The original built-in implementation, `getter_builtin.read_package_file`, does not expose real filesystem paths or a general `io.open` escape hatch. It rejects absolute paths, `..`, directory reads, cross-package reads, and arbitrary repository reads. `read_package_file(path)` returns a Lua string; getter does not interpret encoding, MIME type, JSON, or text-vs-binary mode. Hook code may still wrap the public `read_package_file()` name as local user policy; getter core/CLI does not maintain a protective denylist of hookable public functions. Getter does not assign product semantics to file names or formats inside `files/`; the package owns them. + +This keeps repository layout structured while preserving the repo-trust model: repository source review/signing covers these helper files, and users who do not trust a repository should not use it except by copying/authoring content into a repository they control. -Package files are final package definitions consumed by getter. +Package directory contents outside getter's explicit discovery set are outside getter domain entirely. Getter only considers `metadata.jsonc`, optional generated-package `.autogen.jsonc`, `Manifest`, enabled direct-child `*.lua`, and `files/`. Other files or directories are not parsed, validated, displayed, warned about, or modeled as ignored managed objects. The primary reason is clear responsibility boundaries; a smaller getter-core attack surface is a beneficial side effect. -Path-derived package id: +## Manifest + +`Manifest` stores allowed external network/dynamic-download response-body hashes for package version scripts: ```text -packages/android/org.fdroid.fdroid.lua -> android/org.fdroid.fdroid + [optional-name] ``` -The file should declare the same id. getter validates consistency. +The hash is authoritative. The optional name is for humans/debugging only; a URL may not reveal the actual returned file name, so getter validates by response-body SHA-512 membership, not by file name. + +A missing `Manifest` is equivalent to an empty hash set, not an invalid package. For package version scripts without `allow_free_network`, any external network/dynamic-download data file or API response body used by that script must hash to one of the entries in that package's `Manifest`. If the response hash is absent or mismatched, getter rejects the download/use, so a missing/empty `Manifest` means such network fetches cannot succeed. Scripts that do not fetch external network content, or only read package-local `files/`, do not need a `Manifest`. Scripts with `allow_free_network` are not blocked by `Manifest` membership but remain high-risk. -## lib/ +`Manifest` belongs only to package directories and package version Lua execution. Repository-level autogen scripts under `.metadata/autogen/` do not have a `Manifest`, but an autogen script that creates package directories must also generate correct package `Manifest` files for generated packages that are expected to work without `allow_free_network`. + +`Manifest` is not a repository source manifest. It does not protect `metadata.jsonc`, Lua version scripts, package `files/`, sibling `.autogen.jsonc`, `luaclass/`, repository metadata, or autogen scripts; those source files are protected by the repository Git/signing/maintainer trust model. Since `Manifest` and `.autogen.jsonc` are same-level package files, `Manifest` cannot architecturally protect `.autogen.jsonc`. It also does not make `allow_free_network` or `9999.lua` live upstream behavior reproducible or safe. + +## luaclass/ Reusable Lua modules. These are conceptually similar to eclasses but are plain Lua modules. ```lua -local github_android = require("lib.github_android_apk") +local github_android = require("luaclass.github_android_apk") +``` + +Cross-repository imports may resolve by priority when no repository alias is specified, or by explicit local alias when an author intentionally depends on one. Explicit alias imports break if the user renames that alias, preserving the user's ability to replace a repository layer intentionally. + +## Getter hook scripts + +Getter preserves user-controlled transparent URL replacement through local hook scripts under `rc/hook/`, analogous to an emerge bashrc-style hook and UpgradeAll's older URL replacement behavior. + +Hooks are global getter-local runtime policy. Getter discovers hooks only from the filesystem: list enabled `rc/hook/*.lua` files, exclude basenames starting with `.`, sort deterministically, then load before every Lua execution environment. For example, enabled files load as `00-env.lua`, `10-http-rewrite.lua`, then `20-headers.lua`. A Lua file whose basename starts with `.` is excluded from hook Lua discovery, so `.10-http-rewrite.lua` is not parsed, validated, loaded, displayed, or treated as a hook entry. There is no hook registry, metadata map, or disabled-hook state. + +A hook can wrap visible Lua host functions such as `http_get()` or `read_package_file()` and call the original getter-internal entrypoint after rewriting the URL or applying local policy. Getter core/CLI does not maintain a protective denylist of hookable public functions; if extra guardrails are needed, they belong in UI/UX policy rather than the getter core. Original unhooked host entrypoints are available to hook code as `getter_builtin.`, for example: + +```lua +local upstream_http_get = getter_builtin.http_get + +function http_get(url, opts) + local rewritten = rewrite_url(url) + return upstream_http_get(rewritten, opts) +end ``` -## templates/ +Other host functions can use the same pattern when getter exposes a stable hook seam. `getter_builtin.*` is an internal escape hatch for hook code; ordinary package/autogen Lua should use the public hooked names instead. The hook layer affects package version scripts, repository-level autogen scripts, and `luaclass/` code through their calls to wrapped host functions, but it is not a mechanism for modifying repository source files. + +Hook loading is fail-closed for enabled hooks. If any enabled hook fails to parse, load, or initialize, getter fails the current Lua execution instead of warning and continuing with unhooked functions. Dot-prefixed Lua files are excluded from Lua discovery, not modeled as disabled hook objects. This intentionally fails “stupidly” rather than pretending success when a user proxy/security policy did not load. + +Hook rewriting does not by itself trust the returned content. For package version scripts without `allow_free_network`, the fetched response body still must match a `Manifest` hash. For scripts with `allow_free_network`, getter/UI surfaces the configured high-risk permission. + +## .metadata/autogen/ + +Repository-level autogen metadata and Lua scripts live under `.metadata/autogen/`. + +`metadata.jsonc` describes the autogen scripts, for example which file handles Android app discovery or Magisk module discovery. Autogen scripts can use `luaclass/` helpers and can include trust/signature verification logic such as pinned GPG public keys where appropriate. -Lua generators that output package Lua file content. +Each generated package directory stores its own getter-managed generation record as `.autogen.jsonc`. That package-local record lists the generated files, file hashes, generator/template identity, and input facts needed to decide whether the package can be refreshed or cleaned. The recorded file hashes are ownership/tamper-detection facts for generated output: they answer whether a file is still the file getter generated earlier, not whether it is trusted, repository-signed, or valid as external-download content. The `files` map covers getter-written generated output such as `metadata.jsonc`, `Manifest`, generated Lua scripts, and generated `files/...` helper files; it does not include `.autogen.jsonc` itself, avoiding self-referential hashing. Keeping the generation record beside the generated package avoids slow repository-level reverse lookup during cleanup. A generated-repo package directory without `.autogen.jsonc` is treated as a conflict: getter does not automatically claim, overwrite, or delete it; without `.autogen.jsonc`, there is no ownership proof. If `.autogen.jsonc` exists but is malformed or schema-invalid, package discovery/evaluation is still decided by `metadata.jsonc`, but autogen refresh/apply/cleanup/overwrite reports a generated-ownership conflict and does not auto-fix, overwrite, or delete it. When refresh/overwrite ownership checks pass, getter clears the existing generated package directory contents, then writes the new generated contents into the same package directory, without preserving old unlisted extra files. If clearing any old file or subdirectory fails, the whole refresh/overwrite fails rather than being ignored. If writing new generated contents fails after clearing, the operation fails directly without rollback; the directory may be empty or partially written, and the next refresh continues by clearing and rewriting again. When cleanup ownership checks pass, cleanup clears the generated package directory contents, including `.autogen.jsonc` and any unlisted extra files inside it, but does not delete the package directory itself. If clearing any file or subdirectory fails, the whole cleanup/update fails rather than being ignored. Getter does not classify or preserve unlisted extra files in generated package directories because they are outside getter's domain; direct directory-content clearing/replacement is simpler and more stable for generated output. ## Offline validation @@ -63,6 +240,6 @@ Use getter's structured validator before publishing or registering a repository: getter --data-dir /tmp/ua-getter repo validate /path/to/repo ``` -The command does not require the repository to be registered and does not use the network. It checks the local layout, `repo.toml`, package path-derived ids, constrained Lua evaluation, and Rust schema/domain validation. Results are returned as JSON with `valid`, `package_count`, `network_required`, and getter-owned `diagnostics`. +The command does not require the repository to be registered and does not use the network. It checks the local layout, repository metadata, package path derivation, absence of duplicate package `id` declarations, required Lua API-version shebangs, constrained Lua evaluation, manifests, and Rust schema/domain validation. Results are returned as JSON with `valid`, `package_count`, `network_required`, and getter-owned `diagnostics`. Common diagnostic codes include `repository.missing_directory`, `repository.unsupported_api_version`, `package.lua_runtime`, `package.schema`, and `package.domain`. diff --git a/docs/lua-api/templates.md b/docs/lua-api/templates.md index d0b028c3..b5c1ef21 100644 --- a/docs/lua-api/templates.md +++ b/docs/lua-api/templates.md @@ -1,47 +1,81 @@ -# Lua Templates / Autogen +# Lua Autogen > Status: Draft / living design record > Date: 2026-06-21 > Project: UpgradeAll rewrite — Flutter APP + Rust getter core + Lua package repository model -Templates are Lua generators that output package Lua file content. +Autogen scripts are repository-level Lua generators that output package directories, metadata, version scripts, package Manifests, and any package-local helper files needed under `files/`. -They are inspired by Funtoo Metatools/autogen, where autogen code produces ebuilds from upstream or structured inputs. +They are inspired by Funtoo Metatools/autogen, where autogen code produces ebuilds from upstream or structured inputs, but UpgradeAll keeps the model smaller and JSONC/Lua based. -## Template role +## Autogen role -Templates are used for: +Autogen is used for: -- generating package files from installed Android apps; -- generating package files from Magisk modules; +- generating package directories from installed Android apps; +- generating package directories from Magisk modules; - repository maintainer batch generation; - assisted package creation from GitHub/F-Droid metadata. -Templates are not runtime package definitions. +Autogen scripts are not runtime package version scripts. They live under repository `.metadata/autogen/` and may use shared `luaclass/` modules. -## Example +## Example output + +An installed Android app autogen may produce: + +```text +repo/autogen/android/app/com.example.app/ + metadata.jsonc + .autogen.jsonc + Manifest + 1.2.3.lua +``` + +`metadata.jsonc`: + +```jsonc +{ + "type": "android:app", + "android": { + "package_name": "com.example.app" + }, + "homepage": "https://example.com", + "description": "..." +} +``` + +`1.2.3.lua`: ```lua -return template { - id = "android_installed_app", - - generate = function(ctx, input) - return { - path = "packages/android/" .. input.package_name .. ".lua", - content = [[ -local android = require("lib.android") - -return android.local_app { - id = "android/]] .. input.package_name .. [[", - name = "]] .. input.label .. [[", - package_name = "]] .. input.package_name .. [[", +#!/bin/upa-lua v1 +local android = require("luaclass.android") + +return android.package_version { + version = "1.2.3", } -]] - } - end +``` + +`.autogen.jsonc` records the generated output files that getter may later refresh or clean, but it does not list itself in its own `files` map: + +```jsonc +{ + "generator": "installed-android", + "input": { + "package_name": "com.example.app" + }, + "files": { + "metadata.jsonc": "sha512:...", + "Manifest": "sha512:...", + "1.2.3.lua": "sha512:...", + "files/helper.json": "sha512:..." + } } ``` +The `files` map covers getter-written generated output such as `metadata.jsonc`, `Manifest`, generated Lua scripts, and generated `files/...` helper files. It excludes `.autogen.jsonc` itself to avoid self-referential hashing; `.autogen.jsonc` validity is checked by parsing/schema validation and matching ownership fields instead. + +F-Droid autogen should normally keep generated package metadata/version scripts small and rely on F-Droid catalog metadata plus reusable `luaclass/` helpers for provider behavior. If generated package Lua needs local helper data, autogen writes it under that package directory's `files/` subtree; getter does not assign product semantics to file names or formats inside `files/`. + ## UX contract Generation flow: @@ -51,8 +85,8 @@ Generation flow: 3. Rust calls the Android platform adapter for installed-inventory facts, then getter computes the candidate list. CLI/dev tests may still exercise this with `autogen installed preview --inventory ` fixtures. 4. Flutter shows the getter-owned preview list. 5. User confirms yes/no. -6. getter applies the accepted preview through the native bridge operation; CLI/dev tests may still use `autogen installed apply --preview --accept-all` or repeated `--accept `. -7. getter writes files under `/repositories/local_autogen`, registers the repo, records `autogen-manifest.json`, and tracks accepted packages in `main.db`. +6. getter applies the accepted preview through the native bridge operation; CLI/dev tests may still use `autogen installed apply --preview --accept-all` or repeated `--accept `. +7. getter writes package directories/version scripts under the configured generated repository alias, writes a package-local `.autogen.jsonc` generation record, and tracks accepted packages in `main.db`. Cleanup flow: @@ -61,12 +95,12 @@ Cleanup flow: 3. getter computes the deletion list. CLI/dev tests may still exercise this with `autogen cleanup preview --inventory ` fixtures. 4. Flutter shows the getter-owned preview list. 5. User confirms yes/no. -6. getter deletes only accepted manifest-managed `local_autogen` files/state. +6. getter clears the accepted generated package directory contents when `.autogen.jsonc` ownership checks pass; it does not delete the package directory itself. -Cleanup apply refuses stale/tampered previews that do not match the current manifest, and guarded tracked-state deletion only removes rows still owned by `local_autogen` generated packages. Installed apply preserves existing user state (`enabled`, `favorite`, `pin_version`) and existing non-missing resolution metadata when a package is already tracked. If a managed autogen file has been edited, getter preserves that content into the user-authored `local` repo before regenerating or deleting the generated file. Ordinary autogen cleanup never deletes `local`. +Cleanup apply refuses stale/tampered previews that do not match the current package-local `.autogen.jsonc` generation record, and guarded tracked-state deletion only removes rows still owned by generated packages. When cleanup ownership checks pass, cleanup clears the generated package directory contents directly, including `.autogen.jsonc` and any unlisted extra files inside it, but does not delete the package directory itself. If clearing any file or subdirectory fails, the whole cleanup/update fails rather than being ignored. Getter does not classify or preserve unlisted extra files in generated package directories because they are outside getter's domain; direct directory-content clearing is simpler and more stable for generated output. Hashes inside `.autogen.jsonc` are only generated-output ownership/tamper-detection facts, not security trust, repository signing, or Manifest/download validation. Installed apply preserves existing user state (`enabled`, `favorite`, `pin_version`) and existing non-missing resolution metadata when a package is already tracked. The generated repository is generated output: getter may overwrite package directories it previously generated only when matching `.autogen.jsonc` proves ownership. If a target package directory exists without a matching generation record, apply reports a conflict and does not overwrite it; a generated-repo package directory missing `.autogen.jsonc` is a conflict rather than something getter automatically claims. If `.autogen.jsonc` exists but is malformed or schema-invalid, package discovery/evaluation remains governed by `metadata.jsonc`, but ownership-dependent autogen refresh/apply/cleanup/overwrite reports a conflict and does not auto-fix, overwrite, or delete it. Users who want to hand-author or override generated behavior should create or edit `repo/local/...`; ordinary autogen apply/cleanup never overwrites or deletes `local`. ## Repositories -Ordinary installed-app autogen writes to `local_autogen`, using fixed repo id `local_autogen`, default priority `-1`, and deterministic paths such as `packages/android/com.example.app.lua`. Candidates are skipped when any registered repository with priority higher than `local_autogen` already provides the same package id. +Ordinary installed-app autogen writes to the generated repository alias configured by `repo/metadata.jsonc` `generated_repository`, defaulting to `autogen`. Generated starter config should include the default as a comment users may uncomment/change. When the target is the default `autogen`, getter creates `repo/autogen/` at autogen runtime if needed. If the configured target is any other alias, that repository directory must already exist or autogen apply reports a configuration error. The default generated repository priority is `-1`, and deterministic package directories look like `android/app/com.example.app/`. Candidates are skipped when any registered repository with priority higher than the generated repository already provides the same package path. Existing generated package directories may be replaced only when their package-local `.autogen.jsonc` matches the autogen ownership that is applying the refresh; directories without a matching generation record, or with a malformed/schema-invalid `.autogen.jsonc`, are conflicts, not overwrite targets. When refresh/overwrite ownership checks pass, getter clears the existing generated package directory contents, then writes the new generated contents into the same package directory, without preserving old unlisted extra files. If clearing any old file or subdirectory fails, the whole refresh/overwrite fails rather than being ignored. If writing new generated contents fails after clearing, the operation fails directly without rollback; the directory may be empty or partially written, and the next refresh continues by clearing and rewriting again. -Legacy migration may generate `local` files once as a special compatibility path. +Legacy migration may generate `local` package directories once as a special compatibility path. diff --git a/todo.md b/todo.md index ab642a2e..105954fa 100644 --- a/todo.md +++ b/todo.md @@ -296,7 +296,7 @@ UpgradeAll PR #514 now states: - Flutter shell is intentionally fake-adapter only - Gradle/AGP/Kotlin compatibility fixes are included - current CI validation is green -- deferred work includes real bridge, direct Room migration, `local_autogen`, provider/downloader/update lifecycle +- deferred work includes real bridge, direct Room migration, installed-autogen generated repository work, provider/downloader/update lifecycle Getter PR #54 now states: @@ -413,25 +413,26 @@ Acceptance progress: - WAL/SHM pending writes: first Android adapter copy/checkpoint slice implemented; focused JVM native-adapter tests cover triplet copy, stale sidecar cleanup, missing DB behavior, and checkpointer invocation. Device integration validation on the `Pixel_9a` emulator covers Flutter MethodChannel -> Android copy/checkpoint -> JNI -> Rust getter import/report-list using a Room v17 fixture whose committed rows remain in the WAL sidecar before Android checkpointing. - Per-app failures become warnings; global unreadable DB becomes recovery state, not crash: done for the getter-owned direct importer. -### Phase B: `local_autogen` generation +### Phase B: installed-autogen generated repository -Goal: convert installed/legacy state into generated fallback Lua packages without mixing with user-authored overrides. +Goal: convert installed/legacy state into generated fallback package directories without mixing with user-authored overrides. Rules: ```text -local = user-authored, highest priority, never overwritten silently -local_autogen = generated fallback, safe to regenerate/clean after preview +local = user-authored, highest priority, never overwritten silently +autogen = default generated fallback repository, safe to regenerate/clean after preview and ownership checks ``` -User-confirmed decisions: +User-confirmed decisions now live in ADR-0012: -- getter creates/uses canonical `/repositories/local_autogen`. -- any registered repository with priority higher than `local_autogen` suppresses generation for a package id. -- autogen apply/cleanup are getter-managed; if a generated file has been edited, getter preserves it into `local` before regenerating/deleting. +- getter writes installed-autogen output to `repo/metadata.jsonc` `generated_repository`, default `autogen`. +- if the target is default `autogen`, getter creates `repo/autogen/` at autogen runtime when needed; custom generated targets must already exist. +- any registered repository with priority higher than the configured generated repository suppresses generation for a package path. +- autogen apply/cleanup are getter-managed through package-local `.autogen.jsonc` ownership records; modified/missing/malformed ownership records are conflicts, not triggers for copying generated content into `local`. - applying installed autogen also tracks accepted packages because user confirmation means the user wants update tracking. -Status: getter-owned CLI/core and first production bridge slices are in progress. Implemented pure autogen planning, installed preview/apply, cleanup preview/apply, deterministic package Lua generation, manifest-managed cleanup, higher-priority coverage skips, local preservation for edited autogen files, guarded cleanup against stale/tampered previews, and preservation of existing tracked user state during autogen apply. Added Rust-active Android PackageManager inventory collection and first native bridge preview/apply operations packaged into the Flutter product APK. Added a Flutter installed-autogen preview/apply UI that renders getter-owned DTOs and calls the native bridge without Dart-led package-id or autogen decisions. +Status: getter-owned CLI/core and first production bridge slices exist for the earlier installed-autogen implementation. ADR-0012 supersedes the old flat generated-file/manifest storage model with ordinary package directories, package-local `.autogen.jsonc`, generated-repository target config, and clear-then-write cleanup/refresh semantics. Added Rust-active Android PackageManager inventory collection and first native bridge preview/apply operations packaged into the Flutter product APK. Added a Flutter installed-autogen preview/apply UI that renders getter-owned DTOs and calls the native bridge without Dart-led package-path or autogen decisions. Completed tasks: @@ -443,10 +444,10 @@ Completed tasks: 6. Preserve edited generated files into `local` before autogen rewrite/delete. 7. Guard cleanup deletion by current autogen manifest, repository id, and generated-package resolution. 8. Add Rust-active Android installed inventory provider/scanner path: Kotlin PackageManager facts provider, Rust JNI call/deserialization, and `api_proxy` runtime initialization. -9. Extract installed-autogen preview/apply semantics into reusable getter-owned `getter-operations` code so CLI and native bridge share the same `local_autogen` rules. -10. Add native bridge operations that combine platform scan + getter `local_autogen` preview/apply while returning getter-style JSON envelopes. +9. Extract installed-autogen preview/apply semantics into reusable getter-owned `getter-operations` code so CLI and native bridge share the same generated-repository ownership rules. +10. Add native bridge operations that combine platform scan + getter installed-autogen preview/apply while returning getter-style JSON envelopes. 11. Wire/package a slim production bridge into `app_flutter` so the Flutter APK contains `libapi_proxy.so`, `NativeLib`, and the installed-inventory provider classes without depending on the legacy native `:app` UI or old `GetterPort` hub/RPC wrapper surface. -12. Add Flutter confirmation UX that consumes getter preview/apply DTOs and passes displayed accepted package ids back to getter/native bridge. +12. Add Flutter confirmation UX that consumes getter preview/apply DTOs and passes displayed accepted package paths back to getter/native bridge. Remaining tasks: From 4d65323108f4025532aa3bbb7963df86c1d8210e Mon Sep 17 00:00:00 2001 From: Xiangzhe Date: Fri, 26 Jun 2026 18:27:02 +0800 Subject: [PATCH 49/85] feat(app): use ADR-0012 autogen repository defaults --- AGENTS.md | 8 ++++++-- app_flutter/README.md | 2 +- app_flutter/lib/getter_adapter.dart | 10 +++++----- app_flutter/test/native_getter_adapter_test.dart | 8 ++++---- app_flutter/test/widget_test.dart | 2 +- core-getter/src/main/rust/api_proxy/src/lib.rs | 7 +++++-- core-getter/src/main/rust/getter | 2 +- docs/app/flutter-ui-feature-parity-and-testing.md | 2 +- docs/implementation/coding-agent-handoff.md | 4 ++-- 9 files changed, 26 insertions(+), 19 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index 85946e51..60cf2079 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -13,7 +13,11 @@ Before coding, every agent MUST read: 7. `docs/architecture/adr/0005-lua-package-api.md` 8. `docs/architecture/adr/0006-package-centric-cli-command-contract.md` 9. `docs/architecture/adr/0007-flutter-getter-bridge-contract.md` -10. `docs/app/flutter-ui-feature-parity-and-testing.md` +10. `docs/architecture/adr/0012-getter-owned-provider-modules-and-autogen-refresh.md` +11. `docs/lua-api/repository-layout.md` +12. `docs/lua-api/permissions.md` +13. `docs/lua-api/templates.md` +14. `docs/app/flutter-ui-feature-parity-and-testing.md` ## Core architecture rules @@ -26,7 +30,7 @@ Before coding, every agent MUST read: - Backend state uses SQLite main DB plus separate cache DB. - Package Lua source files live in repository folders. - `local` is user-authored override repo. -- `local_autogen` is generated fallback repo. +- `autogen` is the default generated fallback repo; `repo/metadata.jsonc` may configure another existing generated repository alias. - Do not add runtime UI customization/plugin framework unless a later ADR changes this. ## Testing rules diff --git a/app_flutter/README.md b/app_flutter/README.md index 3460eceb..50935fd1 100644 --- a/app_flutter/README.md +++ b/app_flutter/README.md @@ -23,7 +23,7 @@ Do not validate the rewrite with an older local Flutter SDK; older Flutter teste - `MainActivity` exposes a no-UI `net.xzos.upgradeall/getter_bridge` MethodChannel for native bridge plumbing. The legacy migration and installed-autogen methods derive the app-private getter data directory on Android, call Rust JNI entrypoints, and return getter-style JSON envelopes consumed by `MethodChannelGetterAdapter`. - Product manifest permissions include `QUERY_ALL_PACKAGES` per ADR-0009 so the Rust-active Android platform adapter can provide complete installed package inventory facts to getter. -`CliGetterAdapter` is not the final Android production bridge. It exists to keep the getter-owned DTO and error contract executable for dev tests. `MethodChannelGetterAdapter` is the current production bridge slice for direct legacy Room import/report-list and installed-autogen preview/apply: Flutter renders getter-owned DTOs and passes user choices/paths back to getter, but Room mapping, PackageManager scanning, package-id decisions, and `local_autogen` writes remain in Rust/native getter code. +`CliGetterAdapter` is not the final Android production bridge. It exists to keep the getter-owned DTO and error contract executable for dev tests. `MethodChannelGetterAdapter` is the current production bridge slice for direct legacy Room import/report-list and installed-autogen preview/apply: Flutter renders getter-owned DTOs and passes user choices/paths back to getter, but Room mapping, PackageManager scanning, package-id decisions, and `autogen` writes remain in Rust/native getter code. ## Verification diff --git a/app_flutter/lib/getter_adapter.dart b/app_flutter/lib/getter_adapter.dart index 8272f402..4fb76547 100644 --- a/app_flutter/lib/getter_adapter.dart +++ b/app_flutter/lib/getter_adapter.dart @@ -92,7 +92,7 @@ class FakeGetterAdapter implements GetterAdapter { repositories: [ RepositorySummary(id: 'local', priority: 100), RepositorySummary(id: 'official', priority: 0), - RepositorySummary(id: 'local_autogen', priority: -1), + RepositorySummary(id: 'autogen', priority: -1), ], ); @@ -163,8 +163,8 @@ class FakeGetterAdapter implements GetterAdapter { }) async { return InstalledAutogenPreview.fromJson(const { 'operation': 'installed.preview', - 'target_repo_id': 'local_autogen', - 'target_repo_path': '/fake/getter/repositories/local_autogen', + 'target_repo_id': 'autogen', + 'target_repo_path': '/fake/getter/repo/autogen', 'scan': { 'stats': { 'total_seen': 3, @@ -212,8 +212,8 @@ class FakeGetterAdapter implements GetterAdapter { List? acceptedPackageIds, }) async { return InstalledAutogenApplyResult.fromJson(const { - 'target_repo_id': 'local_autogen', - 'target_repo_path': '/fake/getter/repositories/local_autogen', + 'target_repo_id': 'autogen', + 'target_repo_path': '/fake/getter/repo/autogen', 'applied_count': 1, 'applied': [ { diff --git a/app_flutter/test/native_getter_adapter_test.dart b/app_flutter/test/native_getter_adapter_test.dart index 51f3e1ce..c1102de0 100644 --- a/app_flutter/test/native_getter_adapter_test.dart +++ b/app_flutter/test/native_getter_adapter_test.dart @@ -67,8 +67,8 @@ void main() { 'ok': true, 'command': 'autogen installed apply', 'data': { - 'target_repo_id': 'local_autogen', - 'target_repo_path': '/getter/repositories/local_autogen', + 'target_repo_id': 'autogen', + 'target_repo_path': '/getter/repo/autogen', 'applied_count': 1, 'applied': [ { @@ -503,8 +503,8 @@ Map _runtimeTaskJson( Map _previewJson() => { 'operation': 'installed.preview', - 'target_repo_id': 'local_autogen', - 'target_repo_path': '/getter/repositories/local_autogen', + 'target_repo_id': 'autogen', + 'target_repo_path': '/getter/repo/autogen', 'scan': { 'stats': { 'total_seen': 2, diff --git a/app_flutter/test/widget_test.dart b/app_flutter/test/widget_test.dart index b1d56d0f..cee90b79 100644 --- a/app_flutter/test/widget_test.dart +++ b/app_flutter/test/widget_test.dart @@ -104,7 +104,7 @@ void main() { expect(find.byKey(AppKeys.repositoriesList), findsOneWidget); expect(find.byKey(AppKeys.repoRow('local')), findsOneWidget); expect(find.byKey(AppKeys.repoRow('official')), findsOneWidget); - expect(find.byKey(AppKeys.repoRow('local_autogen')), findsOneWidget); + expect(find.byKey(AppKeys.repoRow('autogen')), findsOneWidget); }); testWidgets('downloads route renders runtime task snapshots read-only', ( diff --git a/core-getter/src/main/rust/api_proxy/src/lib.rs b/core-getter/src/main/rust/api_proxy/src/lib.rs index 7e2e05c8..85b5e87b 100644 --- a/core-getter/src/main/rust/api_proxy/src/lib.rs +++ b/core-getter/src/main/rust/api_proxy/src/lib.rs @@ -292,8 +292,8 @@ fn preview_installed_autogen( serde_json::to_value(&scan.inventory) .and_then(serde_json::from_value) .map_err(|source| BridgeOperationError::PlatformMalformed(source.to_string()))?; - let plan = autogen::build_local_autogen_plan(&db, &inventory)?; - let mut preview = autogen::installed_preview_json(&request.data_dir, &plan); + let plan = autogen::build_installed_autogen_plan(&request.data_dir, &db, &inventory)?; + let mut preview = autogen::installed_preview_json(&request.data_dir, &plan)?; if let Some(object) = preview.as_object_mut() { object.insert( "scan".to_owned(), @@ -676,6 +676,9 @@ impl From for BridgeOperationError { match value { AutogenOperationError::Storage(source) => Self::Storage(source.to_string()), AutogenOperationError::Repository(detail) => Self::Repository(detail), + AutogenOperationError::MissingGeneratedRepository { .. } => { + Self::Autogen(value.to_string()) + } AutogenOperationError::Autogen(detail) => Self::Autogen(detail), } } diff --git a/core-getter/src/main/rust/getter b/core-getter/src/main/rust/getter index 60a65158..3d268807 160000 --- a/core-getter/src/main/rust/getter +++ b/core-getter/src/main/rust/getter @@ -1 +1 @@ -Subproject commit 60a65158aed40e14f0dea17427b4a92ec1e43818 +Subproject commit 3d268807a1a23e38789434a7e0a0659692ae8897 diff --git a/docs/app/flutter-ui-feature-parity-and-testing.md b/docs/app/flutter-ui-feature-parity-and-testing.md index d0e6cdbd..fa54800c 100644 --- a/docs/app/flutter-ui-feature-parity-and-testing.md +++ b/docs/app/flutter-ui-feature-parity-and-testing.md @@ -64,7 +64,7 @@ Feature: Installed app autogen Given the device has installed apps not covered by official repository When the user opens Installed Autogen And confirms the generated list - Then getter writes package scripts to local_autogen + Then getter writes package scripts to autogen And the apps appear in the app list as generated fallback packages ``` diff --git a/docs/implementation/coding-agent-handoff.md b/docs/implementation/coding-agent-handoff.md index 54c2cc01..b124937d 100644 --- a/docs/implementation/coding-agent-handoff.md +++ b/docs/implementation/coding-agent-handoff.md @@ -112,9 +112,9 @@ Prefer adding/updating ADRs for decisions rather than burying major changes in c ## Repository naming - `local` is the default highest-priority user-authored override repository. -- `local_autogen` is the generated fallback repository used by ordinary installed-app autogen. +- `autogen` is the default generated fallback repository used by ordinary installed-app autogen; `repo/metadata.jsonc` may configure another existing generated repository alias. - Legacy migration is special and may generate `local` package files once for compatibility. -- Cleanup of missing generated apps only touches `local_autogen`. +- Cleanup of missing generated apps only touches the configured generated repository target. ## Open questions to resolve before implementation hardens From 03cb2068f1d53e3d69fc5b48a8d57bf34736c1d0 Mon Sep 17 00:00:00 2001 From: Xiangzhe Date: Fri, 26 Jun 2026 22:40:22 +0800 Subject: [PATCH 50/85] feat(getter): use package directory discovery --- core-getter/src/main/rust/getter | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/core-getter/src/main/rust/getter b/core-getter/src/main/rust/getter index 3d268807..3388b78d 160000 --- a/core-getter/src/main/rust/getter +++ b/core-getter/src/main/rust/getter @@ -1 +1 @@ -Subproject commit 3d268807a1a23e38789434a7e0a0659692ae8897 +Subproject commit 3388b78dc891dc487bc406d71050e919ce6e4023 From 4dea73112280ff8380d88b687dec41b808a2775a Mon Sep 17 00:00:00 2001 From: Xiangzhe Date: Sat, 27 Jun 2026 00:23:52 +0800 Subject: [PATCH 51/85] feat(app): consume package-directory autogen previews --- app_flutter/lib/getter_adapter.dart | 53 ++----------------- .../test/native_getter_adapter_test.dart | 20 ++++--- app_flutter/test/widget_test.dart | 10 ++-- core-getter/src/main/rust/getter | 2 +- 4 files changed, 22 insertions(+), 63 deletions(-) diff --git a/app_flutter/lib/getter_adapter.dart b/app_flutter/lib/getter_adapter.dart index 4fb76547..d63f328f 100644 --- a/app_flutter/lib/getter_adapter.dart +++ b/app_flutter/lib/getter_adapter.dart @@ -182,7 +182,7 @@ class FakeGetterAdapter implements GetterAdapter { }, 'candidates': [ { - 'package_id': 'android/com.example.autogen', + 'package_id': 'android/app/com.example.autogen', 'kind': 'android', 'display_name': 'Example Autogen', 'installed_target': { @@ -190,8 +190,8 @@ class FakeGetterAdapter implements GetterAdapter { 'package_name': 'com.example.autogen', }, 'action': 'create', - 'output_relative_path': 'packages/android/com.example.autogen.lua', - 'content_hash': 'fnv1a64:fake', + 'output_relative_path': 'android/app/com.example.autogen', + 'content_hash': 'sha512:fake', 'content': '-- fake generated content', }, ], @@ -217,11 +217,10 @@ class FakeGetterAdapter implements GetterAdapter { 'applied_count': 1, 'applied': [ { - 'package_id': 'android/com.example.autogen', - 'output_relative_path': 'packages/android/com.example.autogen.lua', + 'package_id': 'android/app/com.example.autogen', + 'output_relative_path': 'android/app/com.example.autogen', }, ], - 'preserved_to_local': [], }); } @@ -1104,7 +1103,6 @@ class InstalledAutogenApplyResult { required this.targetRepoPath, required this.appliedCount, required this.applied, - required this.preservedToLocal, }); factory InstalledAutogenApplyResult.fromJson(Map json) { @@ -1128,17 +1126,6 @@ class InstalledAutogenApplyResult { ), ) .toList(growable: false), - preservedToLocal: - _jsonList( - json['preserved_to_local'], - 'autogen.apply.preserved_to_local', - ) - .map( - (preserved) => InstalledAutogenPreservedPackage.fromJson( - _jsonMap(preserved, 'autogen.apply.preserved_item'), - ), - ) - .toList(growable: false), ); } @@ -1146,7 +1133,6 @@ class InstalledAutogenApplyResult { final String? targetRepoPath; final int appliedCount; final List applied; - final List preservedToLocal; } class InstalledAutogenAppliedPackage { @@ -1169,35 +1155,6 @@ class InstalledAutogenAppliedPackage { final String outputRelativePath; } -class InstalledAutogenPreservedPackage { - const InstalledAutogenPreservedPackage({ - required this.packageId, - required this.repositoryId, - required this.relativePath, - }); - - factory InstalledAutogenPreservedPackage.fromJson(Map json) { - return InstalledAutogenPreservedPackage( - packageId: _jsonString( - json['package_id'], - 'autogen.preserved.package_id', - ), - repositoryId: _jsonString( - json['repository_id'], - 'autogen.preserved.repository_id', - ), - relativePath: _jsonString( - json['relative_path'], - 'autogen.preserved.relative_path', - ), - ); - } - - final String packageId; - final String repositoryId; - final String relativePath; -} - class GetterError { const GetterError({required this.code, required this.message, this.detail}); diff --git a/app_flutter/test/native_getter_adapter_test.dart b/app_flutter/test/native_getter_adapter_test.dart index c1102de0..7dd5b3c5 100644 --- a/app_flutter/test/native_getter_adapter_test.dart +++ b/app_flutter/test/native_getter_adapter_test.dart @@ -53,7 +53,7 @@ void main() { expect(preview.scanStats!.returned, 1); expect( preview.candidates.single.packageId, - 'android/com.example.autogen', + 'android/app/com.example.autogen', ); }, ); @@ -72,12 +72,10 @@ void main() { 'applied_count': 1, 'applied': [ { - 'package_id': 'android/com.example.autogen', - 'output_relative_path': - 'packages/android/com.example.autogen.lua', + 'package_id': 'android/app/com.example.autogen', + 'output_relative_path': 'android/app/com.example.autogen', }, ], - 'preserved_to_local': [], }, 'warnings': [], }); @@ -87,7 +85,7 @@ void main() { final preview = InstalledAutogenPreview.fromJson(_previewJson()); final result = await adapter.applyInstalledAutogen( preview, - acceptedPackageIds: const ['android/com.example.autogen'], + acceptedPackageIds: const ['android/app/com.example.autogen'], ); expect(captured!.method, 'applyInstalledAutogen'); @@ -96,9 +94,9 @@ void main() { expect(jsonDecode(args['preview_json']! as String), preview.rawJson); expect(args['acceptance'], { 'mode': 'packages', - 'package_ids': ['android/com.example.autogen'], + 'package_ids': ['android/app/com.example.autogen'], }); - expect(result.applied.single.packageId, 'android/com.example.autogen'); + expect(result.applied.single.packageId, 'android/app/com.example.autogen'); }); test('native legacy import and reports parse getter envelopes', () async { @@ -522,7 +520,7 @@ Map _previewJson() => { }, 'candidates': [ { - 'package_id': 'android/com.example.autogen', + 'package_id': 'android/app/com.example.autogen', 'kind': 'android', 'display_name': 'Example Autogen', 'installed_target': { @@ -530,8 +528,8 @@ Map _previewJson() => { 'package_name': 'com.example.autogen', }, 'action': 'create', - 'output_relative_path': 'packages/android/com.example.autogen.lua', - 'content_hash': 'fnv1a64:fake', + 'output_relative_path': 'android/app/com.example.autogen', + 'content_hash': 'sha512:fake', 'content': '-- fake generated content', }, ], diff --git a/app_flutter/test/widget_test.dart b/app_flutter/test/widget_test.dart index cee90b79..1b05d16b 100644 --- a/app_flutter/test/widget_test.dart +++ b/app_flutter/test/widget_test.dart @@ -219,7 +219,9 @@ void main() { expect(find.byKey(AppKeys.installedAutogenPreview), findsOneWidget); expect(find.byKey(AppKeys.installedAutogenScanStats), findsOneWidget); expect( - find.byKey(AppKeys.autogenCandidateRow('android/com.example.autogen')), + find.byKey( + AppKeys.autogenCandidateRow('android/app/com.example.autogen'), + ), findsOneWidget, ); expect( @@ -232,10 +234,12 @@ void main() { expect(find.byKey(AppKeys.installedAutogenApplied), findsOneWidget); expect( - find.byKey(AppKeys.autogenAppliedRow('android/com.example.autogen')), + find.byKey(AppKeys.autogenAppliedRow('android/app/com.example.autogen')), findsOneWidget, ); - expect(getter.acceptedPackageIds, ['android/com.example.autogen']); + expect(getter.acceptedPackageIds, [ + 'android/app/com.example.autogen', + ]); }); testWidgets('installed autogen route disables actions without bridge', ( diff --git a/core-getter/src/main/rust/getter b/core-getter/src/main/rust/getter index 3388b78d..8e880e68 160000 --- a/core-getter/src/main/rust/getter +++ b/core-getter/src/main/rust/getter @@ -1 +1 @@ -Subproject commit 3388b78dc891dc487bc406d71050e919ce6e4023 +Subproject commit 8e880e689457ae48aad3e73ccf7dc12acbaabb0e From 346636faa39893bab839ac8c63609b935ea58d13 Mon Sep 17 00:00:00 2001 From: Xiangzhe Date: Sat, 27 Jun 2026 01:52:08 +0800 Subject: [PATCH 52/85] feat(getter): evaluate package directories --- core-getter/src/main/rust/getter | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/core-getter/src/main/rust/getter b/core-getter/src/main/rust/getter index 8e880e68..367fa9bc 160000 --- a/core-getter/src/main/rust/getter +++ b/core-getter/src/main/rust/getter @@ -1 +1 @@ -Subproject commit 8e880e689457ae48aad3e73ccf7dc12acbaabb0e +Subproject commit 367fa9bc9c3afff4eaf834041cd38f6bc25f577b From aa3fdbb61ff7b26e52aae7985aff18f1204181dc Mon Sep 17 00:00:00 2001 From: Xiangzhe Date: Sat, 27 Jun 2026 02:08:40 +0800 Subject: [PATCH 53/85] fix(getter): update clippy-clean gitlink --- core-getter/src/main/rust/getter | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/core-getter/src/main/rust/getter b/core-getter/src/main/rust/getter index 367fa9bc..15788b42 160000 --- a/core-getter/src/main/rust/getter +++ b/core-getter/src/main/rust/getter @@ -1 +1 @@ -Subproject commit 367fa9bc9c3afff4eaf834041cd38f6bc25f577b +Subproject commit 15788b42a6c786ce86de80af744e72662e1e3e9d From a8739ccaa2eeb4fbd0f7b02973801522d2a03df7 Mon Sep 17 00:00:00 2001 From: Xiangzhe Date: Sat, 27 Jun 2026 02:18:29 +0800 Subject: [PATCH 54/85] feat(getter): expose package-local file reads --- core-getter/src/main/rust/getter | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/core-getter/src/main/rust/getter b/core-getter/src/main/rust/getter index 15788b42..091bb9fd 160000 --- a/core-getter/src/main/rust/getter +++ b/core-getter/src/main/rust/getter @@ -1 +1 @@ -Subproject commit 15788b42a6c786ce86de80af744e72662e1e3e9d +Subproject commit 091bb9fde26187fa01da008a9ce1faf3b2411fcc From 1ce77019dd8a17c5ee81238e5b3f0516127cdc95 Mon Sep 17 00:00:00 2001 From: Xiangzhe Date: Sat, 27 Jun 2026 02:25:33 +0800 Subject: [PATCH 55/85] fix(getter): hash package-local file inputs --- core-getter/src/main/rust/getter | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/core-getter/src/main/rust/getter b/core-getter/src/main/rust/getter index 091bb9fd..139d1ff0 160000 --- a/core-getter/src/main/rust/getter +++ b/core-getter/src/main/rust/getter @@ -1 +1 @@ -Subproject commit 091bb9fde26187fa01da008a9ce1faf3b2411fcc +Subproject commit 139d1ff02b80ce64f753f1b24e8fbe7e4111a96d From 175ae4c8a988db027ab9abcaf112b50943cb2d45 Mon Sep 17 00:00:00 2001 From: Xiangzhe Date: Sat, 27 Jun 2026 02:56:50 +0800 Subject: [PATCH 56/85] feat(getter): add provider cache primitives --- core-getter/src/main/rust/getter | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/core-getter/src/main/rust/getter b/core-getter/src/main/rust/getter index 139d1ff0..edbebcee 160000 --- a/core-getter/src/main/rust/getter +++ b/core-getter/src/main/rust/getter @@ -1 +1 @@ -Subproject commit 139d1ff02b80ce64f753f1b24e8fbe7e4111a96d +Subproject commit edbebceefc960caf20a03eb9a5a9e5210f111949 From e0673f2a9edd252027979243444de0176518b5a8 Mon Sep 17 00:00:00 2001 From: Xiangzhe Date: Sat, 27 Jun 2026 03:34:24 +0800 Subject: [PATCH 57/85] feat(getter): add F-Droid provider fixtures --- core-getter/src/main/rust/getter | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/core-getter/src/main/rust/getter b/core-getter/src/main/rust/getter index edbebcee..bc94040e 160000 --- a/core-getter/src/main/rust/getter +++ b/core-getter/src/main/rust/getter @@ -1 +1 @@ -Subproject commit edbebceefc960caf20a03eb9a5a9e5210f111949 +Subproject commit bc94040e4facb44cd3d5a2a9cd65b83902959b6d From 1a13cbd9d29449a94187651e5bf5279dc975e457 Mon Sep 17 00:00:00 2001 From: Xiangzhe Date: Sat, 27 Jun 2026 04:05:03 +0800 Subject: [PATCH 58/85] feat(getter): cache F-Droid catalog fixtures --- core-getter/src/main/rust/getter | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/core-getter/src/main/rust/getter b/core-getter/src/main/rust/getter index bc94040e..d60cc75f 160000 --- a/core-getter/src/main/rust/getter +++ b/core-getter/src/main/rust/getter @@ -1 +1 @@ -Subproject commit bc94040e4facb44cd3d5a2a9cd65b83902959b6d +Subproject commit d60cc75ff804edab69a1335a9f52bb205d5896f1 From 728cff6bd04b95a73a08adada35ae0ac73e3d2c9 Mon Sep 17 00:00:00 2001 From: Xiangzhe Date: Sat, 27 Jun 2026 10:25:58 +0800 Subject: [PATCH 59/85] feat(getter): add F-Droid autogen CLI slice --- core-getter/src/main/rust/getter | 2 +- .../adr/0006-package-centric-cli-command-contract.md | 4 ++++ 2 files changed, 5 insertions(+), 1 deletion(-) diff --git a/core-getter/src/main/rust/getter b/core-getter/src/main/rust/getter index d60cc75f..de63a6b5 160000 --- a/core-getter/src/main/rust/getter +++ b/core-getter/src/main/rust/getter @@ -1 +1 @@ -Subproject commit d60cc75ff804edab69a1335a9f52bb205d5896f1 +Subproject commit de63a6b515aa8292eeee957db8531b08c4cc467a diff --git a/docs/architecture/adr/0006-package-centric-cli-command-contract.md b/docs/architecture/adr/0006-package-centric-cli-command-contract.md index 7552738f..36012e83 100644 --- a/docs/architecture/adr/0006-package-centric-cli-command-contract.md +++ b/docs/architecture/adr/0006-package-centric-cli-command-contract.md @@ -31,6 +31,8 @@ getter --data-dir debug fake-task events --after --limit getter --data-dir debug fake-task install-result --status getter --data-dir autogen installed preview --inventory getter --data-dir autogen installed apply --preview (--accept-all|--accept ...) +getter --data-dir autogen fdroid preview --index --package [--package ...] +getter --data-dir autogen fdroid apply --preview (--accept-all|--accept ...) getter --data-dir autogen cleanup preview --inventory getter --data-dir autogen cleanup apply --preview (--accept-all|--accept ...) getter --data-dir legacy import-room-bundle @@ -102,6 +104,8 @@ It maps `apps[]` into getter tracked package state in `main.db`, writes a saniti The first installed-app autogen slice accepts an Android/platform-provided inventory DTO, computes generated fallback packages in Rust, previews before writing, and applies only after explicit `--accept-all` or `--accept ` confirmation. ADR-0012 supersedes the original Phase 1a fixed generated-repository and flat generated package-file storage model. The current architecture writes ordinary package directories to the configured generated repository target (`generated_repository`, default `autogen`) and records package-local `.autogen.jsonc` ownership state. Cleanup/refresh follows ADR-0012 ownership checks instead of preserving modified generated files into `local`. +The first explicit F-Droid autogen CLI/dev slice is fixture-backed and offline: `autogen fdroid preview --index --package ` parses the supplied F-Droid index through getter-owned provider/cache code, maps requested upstream package names to package paths such as `android/f-droid/app/org.fdroid.fdroid`, skips packages covered by higher-priority registered repositories, and returns a generated package-directory preview without writing files. `autogen fdroid apply` writes accepted package directories to the configured generated repository and records `.autogen.jsonc` with generator `fdroid-catalog`. This slice does not perform live HTTP, downloads, installer handoff, Flutter/Kotlin provider parsing, or final reusable-provider host API execution; those remain governed by ADR-0012 and later provider/downloader/installer slices. + `update check --fixture ` is the first Phase D offline update-check slice. The fixture contract is explicitly offline and uses `format = "getter-offline-update-check"`, `version = 1`, `package_id`, optional `installed_version`, optional `pin_version` (with transitional `ignored_version` accepted as an input alias), and normalized candidate/artifact DTOs. The command returns `network_required = false`, observed `installed_version`, `effective_local_version`, a getter-owned status (`update_available`, `up_to_date`, or `no_candidates`), the selected update when one exists, and generated download/install action DTOs. An update-selected result must have an actionable artifact; a selected candidate without artifacts is a structured update-check error rather than `update_available` with no actions. It reuses Rust getter update selection and version comparison; it does not run providers, perform downloads, persist download tasks, stream events, or call Android installers. The old persisted fake downloader slice is retained only as debug scaffolding under `debug fake-task ...`. `debug fake-task submit --request ` accepts `format = "getter-download-request"`, `version = 1`, `package_id`, `executor = "fake"`, and update actions containing at least one `download` action; an optional `install` action creates an abstract install handoff after a successful fake run. `debug fake-task run ` deterministically advances the fake task to `succeeded`; it performs no network I/O and writes no downloaded bytes. `debug fake-task list` returns persisted fake-task summaries from `main.db`. `debug fake-task cancel ` persists cancellation for `queued`/`running` fake tasks, is idempotent for already-canceled tasks, and rejects terminal success/failure with a structured download error. `debug fake-task events --after --limit ` is a pollable debug event contract with a positive `limit`; it is not the ADR-0011 runtime event model. `debug fake-task install-result --status ` records the platform-side result of an abstract debug handoff; the getter-created `requested` handoff state is not accepted as a platform result. This scaffold is not a product task API. From dddce326b925d4b0373639cd16df6788a1ab4cc6 Mon Sep 17 00:00:00 2001 From: Xiangzhe Date: Sat, 27 Jun 2026 11:26:14 +0800 Subject: [PATCH 60/85] feat(getter): match F-Droid installed inventory --- core-getter/src/main/rust/getter | 2 +- .../adr/0006-package-centric-cli-command-contract.md | 4 ++-- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/core-getter/src/main/rust/getter b/core-getter/src/main/rust/getter index de63a6b5..6d91ce50 160000 --- a/core-getter/src/main/rust/getter +++ b/core-getter/src/main/rust/getter @@ -1 +1 @@ -Subproject commit de63a6b515aa8292eeee957db8531b08c4cc467a +Subproject commit 6d91ce50f3c5ae41a70633f7a1798cbf64c564e3 diff --git a/docs/architecture/adr/0006-package-centric-cli-command-contract.md b/docs/architecture/adr/0006-package-centric-cli-command-contract.md index 36012e83..bddb7e87 100644 --- a/docs/architecture/adr/0006-package-centric-cli-command-contract.md +++ b/docs/architecture/adr/0006-package-centric-cli-command-contract.md @@ -31,7 +31,7 @@ getter --data-dir debug fake-task events --after --limit getter --data-dir debug fake-task install-result --status getter --data-dir autogen installed preview --inventory getter --data-dir autogen installed apply --preview (--accept-all|--accept ...) -getter --data-dir autogen fdroid preview --index --package [--package ...] +getter --data-dir autogen fdroid preview --index [--package ...] [--inventory ] getter --data-dir autogen fdroid apply --preview (--accept-all|--accept ...) getter --data-dir autogen cleanup preview --inventory getter --data-dir autogen cleanup apply --preview (--accept-all|--accept ...) @@ -104,7 +104,7 @@ It maps `apps[]` into getter tracked package state in `main.db`, writes a saniti The first installed-app autogen slice accepts an Android/platform-provided inventory DTO, computes generated fallback packages in Rust, previews before writing, and applies only after explicit `--accept-all` or `--accept ` confirmation. ADR-0012 supersedes the original Phase 1a fixed generated-repository and flat generated package-file storage model. The current architecture writes ordinary package directories to the configured generated repository target (`generated_repository`, default `autogen`) and records package-local `.autogen.jsonc` ownership state. Cleanup/refresh follows ADR-0012 ownership checks instead of preserving modified generated files into `local`. -The first explicit F-Droid autogen CLI/dev slice is fixture-backed and offline: `autogen fdroid preview --index --package ` parses the supplied F-Droid index through getter-owned provider/cache code, maps requested upstream package names to package paths such as `android/f-droid/app/org.fdroid.fdroid`, skips packages covered by higher-priority registered repositories, and returns a generated package-directory preview without writing files. `autogen fdroid apply` writes accepted package directories to the configured generated repository and records `.autogen.jsonc` with generator `fdroid-catalog`. This slice does not perform live HTTP, downloads, installer handoff, Flutter/Kotlin provider parsing, or final reusable-provider host API execution; those remain governed by ADR-0012 and later provider/downloader/installer slices. +The first F-Droid autogen CLI/dev slice is fixture-backed and offline: `autogen fdroid preview --index --package ` parses the supplied F-Droid index through getter-owned provider/cache code, maps requested upstream package names to package paths such as `android/f-droid/app/org.fdroid.fdroid`, skips packages covered by higher-priority registered repositories, and returns a generated package-directory preview without writing files. The same command may use `--inventory ` to match installed Android package facts from an installed-inventory fixture against the cached F-Droid catalog before producing the same preview shape; at least one `--package` or `--inventory` input is required, and both may be supplied. `autogen fdroid apply` writes accepted package directories to the configured generated repository and records `.autogen.jsonc` with generator `fdroid-catalog`. This slice does not perform live HTTP, downloads, installer handoff, Flutter/Kotlin provider parsing, or final reusable-provider host API execution; those remain governed by ADR-0012 and later provider/downloader/installer slices. `update check --fixture ` is the first Phase D offline update-check slice. The fixture contract is explicitly offline and uses `format = "getter-offline-update-check"`, `version = 1`, `package_id`, optional `installed_version`, optional `pin_version` (with transitional `ignored_version` accepted as an input alias), and normalized candidate/artifact DTOs. The command returns `network_required = false`, observed `installed_version`, `effective_local_version`, a getter-owned status (`update_available`, `up_to_date`, or `no_candidates`), the selected update when one exists, and generated download/install action DTOs. An update-selected result must have an actionable artifact; a selected candidate without artifacts is a structured update-check error rather than `update_available` with no actions. It reuses Rust getter update selection and version comparison; it does not run providers, perform downloads, persist download tasks, stream events, or call Android installers. From b35b7c4c5946987e140e55b6d98ced3fabd18877 Mon Sep 17 00:00:00 2001 From: Xiangzhe Date: Sat, 27 Jun 2026 12:36:00 +0800 Subject: [PATCH 61/85] feat(getter): add GitHub release fixtures --- core-getter/src/main/rust/getter | 2 +- .../adr/0006-package-centric-cli-command-contract.md | 5 ++++- 2 files changed, 5 insertions(+), 2 deletions(-) diff --git a/core-getter/src/main/rust/getter b/core-getter/src/main/rust/getter index 6d91ce50..9001bb93 160000 --- a/core-getter/src/main/rust/getter +++ b/core-getter/src/main/rust/getter @@ -1 +1 @@ -Subproject commit 6d91ce50f3c5ae41a70633f7a1798cbf64c564e3 +Subproject commit 9001bb93b23245c7b2347a722157fffd18f64ca6 diff --git a/docs/architecture/adr/0006-package-centric-cli-command-contract.md b/docs/architecture/adr/0006-package-centric-cli-command-contract.md index bddb7e87..a234bbfa 100644 --- a/docs/architecture/adr/0006-package-centric-cli-command-contract.md +++ b/docs/architecture/adr/0006-package-centric-cli-command-contract.md @@ -33,6 +33,7 @@ getter --data-dir autogen installed preview --inventory getter --data-dir autogen installed apply --preview (--accept-all|--accept ...) getter --data-dir autogen fdroid preview --index [--package ...] [--inventory ] getter --data-dir autogen fdroid apply --preview (--accept-all|--accept ...) +getter --data-dir provider github releases --owner --repo --releases [--asset-include ] [--asset-exclude ] [--include-prereleases] [--refresh] getter --data-dir autogen cleanup preview --inventory getter --data-dir autogen cleanup apply --preview (--accept-all|--accept ...) getter --data-dir legacy import-room-bundle @@ -106,6 +107,8 @@ The first installed-app autogen slice accepts an Android/platform-provided inven The first F-Droid autogen CLI/dev slice is fixture-backed and offline: `autogen fdroid preview --index --package ` parses the supplied F-Droid index through getter-owned provider/cache code, maps requested upstream package names to package paths such as `android/f-droid/app/org.fdroid.fdroid`, skips packages covered by higher-priority registered repositories, and returns a generated package-directory preview without writing files. The same command may use `--inventory ` to match installed Android package facts from an installed-inventory fixture against the cached F-Droid catalog before producing the same preview shape; at least one `--package` or `--inventory` input is required, and both may be supplied. `autogen fdroid apply` writes accepted package directories to the configured generated repository and records `.autogen.jsonc` with generator `fdroid-catalog`. This slice does not perform live HTTP, downloads, installer handoff, Flutter/Kotlin provider parsing, or final reusable-provider host API execution; those remain governed by ADR-0012 and later provider/downloader/installer slices. +The first GitHub provider CLI/dev slice is also fixture-backed and offline: `provider github releases --owner --repo --releases ` parses a controlled GitHub REST releases JSON response through getter-owned provider/cache code, normalizes matching release assets into update candidates, and returns diagnostics such as `provider.github.asset_not_found` when filters match no assets. `--asset-include` and `--asset-exclude` are regex filters over asset names, prereleases are excluded unless `--include-prereleases` is supplied, and `--refresh` exercises forced provider-cache replacement/stale semantics against the fixture. This command does not generate packages, run Lua provider modules, perform live HTTP/auth/rate-limit handling, create runtime actions, download files, invoke installers, or move any GitHub parsing/filtering into Flutter/Kotlin. + `update check --fixture ` is the first Phase D offline update-check slice. The fixture contract is explicitly offline and uses `format = "getter-offline-update-check"`, `version = 1`, `package_id`, optional `installed_version`, optional `pin_version` (with transitional `ignored_version` accepted as an input alias), and normalized candidate/artifact DTOs. The command returns `network_required = false`, observed `installed_version`, `effective_local_version`, a getter-owned status (`update_available`, `up_to_date`, or `no_candidates`), the selected update when one exists, and generated download/install action DTOs. An update-selected result must have an actionable artifact; a selected candidate without artifacts is a structured update-check error rather than `update_available` with no actions. It reuses Rust getter update selection and version comparison; it does not run providers, perform downloads, persist download tasks, stream events, or call Android installers. The old persisted fake downloader slice is retained only as debug scaffolding under `debug fake-task ...`. `debug fake-task submit --request ` accepts `format = "getter-download-request"`, `version = 1`, `package_id`, `executor = "fake"`, and update actions containing at least one `download` action; an optional `install` action creates an abstract install handoff after a successful fake run. `debug fake-task run ` deterministically advances the fake task to `succeeded`; it performs no network I/O and writes no downloaded bytes. `debug fake-task list` returns persisted fake-task summaries from `main.db`. `debug fake-task cancel ` persists cancellation for `queued`/`running` fake tasks, is idempotent for already-canceled tasks, and rejects terminal success/failure with a structured download error. `debug fake-task events --after --limit ` is a pollable debug event contract with a positive `limit`; it is not the ADR-0011 runtime event model. `debug fake-task install-result --status ` records the platform-side result of an abstract debug handoff; the getter-created `requested` handoff state is not accepted as a platform result. This scaffold is not a product task API. @@ -121,7 +124,7 @@ Exit-code classes: - `2`: invalid CLI usage. - `10`: data/storage error. - `20`: migration/import error. -- `30`: future network/provider error. +- `30`: provider operation error. - `40`: download/task lifecycle error. ## Context From 416075e3be6fafcb634726b0a1605013f974e436 Mon Sep 17 00:00:00 2001 From: Xiangzhe Date: Sat, 27 Jun 2026 14:23:23 +0800 Subject: [PATCH 62/85] refactor(getter): remove flat Lua repository layout --- .../dev_test/cli_getter_adapter_test.dart | 34 ++++++++----------- .../src/main/rust/api_proxy/src/lib.rs | 26 +++++++------- core-getter/src/main/rust/getter | 2 +- docs/implementation/coding-agent-handoff.md | 26 ++++++++------ 4 files changed, 44 insertions(+), 44 deletions(-) diff --git a/app_flutter/dev_test/cli_getter_adapter_test.dart b/app_flutter/dev_test/cli_getter_adapter_test.dart index 0525279f..781e48ef 100644 --- a/app_flutter/dev_test/cli_getter_adapter_test.dart +++ b/app_flutter/dev_test/cli_getter_adapter_test.dart @@ -134,26 +134,22 @@ void main() { Directory _createFixtureRepository(Directory temp, String repoId) { final repoDir = Directory('${temp.path}/repo-$repoId')..createSync(); - Directory('${repoDir.path}/packages/android').createSync(recursive: true); - Directory('${repoDir.path}/lib').createSync(); - Directory('${repoDir.path}/templates').createSync(); - File('${repoDir.path}/repo.toml').writeAsStringSync(''' -id = "$repoId" -name = "Fixture $repoId" -priority = 0 -api_version = "getter.repo.v1" -'''); - File( - '${repoDir.path}/packages/android/org.fdroid.fdroid.lua', - ).writeAsStringSync(''' -return package_def { - id = "android/org.fdroid.fdroid", - name = "F-Droid", - installed = { - { kind = "android_package", package_name = "org.fdroid.fdroid" }, - }, - permissions = { free_network = true }, + final packageDir = Directory('${repoDir.path}/android/org.fdroid.fdroid') + ..createSync(recursive: true); + File('${packageDir.path}/metadata.jsonc').writeAsStringSync(''' +{ + "type": "android:app", + "display_name": "F-Droid", + "android": { "package_name": "org.fdroid.fdroid" }, + "lua": { + "9999.lua": { "permission": ["allow_free_network"] } + } } +'''); + File('${packageDir.path}/Manifest').writeAsStringSync(''); + File('${packageDir.path}/9999.lua').writeAsStringSync(''' +#!/bin/upa-lua v1 +return package_version {} '''); return repoDir; } diff --git a/core-getter/src/main/rust/api_proxy/src/lib.rs b/core-getter/src/main/rust/api_proxy/src/lib.rs index 85b5e87b..26aa96ba 100644 --- a/core-getter/src/main/rust/api_proxy/src/lib.rs +++ b/core-getter/src/main/rust/api_proxy/src/lib.rs @@ -930,24 +930,22 @@ mod tests { } fn write_static_update_repo(root: &std::path::Path) { - fs::create_dir_all(root.join("packages/android")).unwrap(); - fs::create_dir(root.join("lib")).unwrap(); - fs::create_dir(root.join("templates")).unwrap(); + let package_dir = root.join("android/org.fdroid.fdroid"); + fs::create_dir_all(&package_dir).unwrap(); fs::write( - root.join("repo.toml"), - r#"id = "official" -name = "Official" -priority = 0 -api_version = "getter.repo.v1" -"#, + package_dir.join("metadata.jsonc"), + r#"{ + "type": "android:app", + "display_name": "F-Droid", + "android": { "package_name": "org.fdroid.fdroid" } +}"#, ) .unwrap(); + fs::write(package_dir.join("Manifest"), "").unwrap(); fs::write( - root.join("packages/android/org.fdroid.fdroid.lua"), - r#" -return package_def { - id = "android/org.fdroid.fdroid", - name = "F-Droid", + package_dir.join("9999.lua"), + r#"#!/bin/upa-lua v1 +return package_version { updates = { { version = "1.2.0", diff --git a/core-getter/src/main/rust/getter b/core-getter/src/main/rust/getter index 9001bb93..32e0fb33 160000 --- a/core-getter/src/main/rust/getter +++ b/core-getter/src/main/rust/getter @@ -1 +1 @@ -Subproject commit 9001bb93b23245c7b2347a722157fffd18f64ca6 +Subproject commit 32e0fb337afa3c773547645b17553deb70282767 diff --git a/docs/implementation/coding-agent-handoff.md b/docs/implementation/coding-agent-handoff.md index b124937d..f1e584ad 100644 --- a/docs/implementation/coding-agent-handoff.md +++ b/docs/implementation/coding-agent-handoff.md @@ -18,7 +18,11 @@ Before coding, read these files in order: 8. `docs/architecture/adr/0005-lua-package-api.md` 9. `docs/architecture/adr/0006-package-centric-cli-command-contract.md` 10. `docs/architecture/adr/0007-flutter-getter-bridge-contract.md` -11. `docs/app/flutter-ui-feature-parity-and-testing.md` +11. `docs/architecture/adr/0012-getter-owned-provider-modules-and-autogen-refresh.md` +12. `docs/lua-api/repository-layout.md` +13. `docs/lua-api/permissions.md` +14. `docs/lua-api/templates.md` +15. `docs/app/flutter-ui-feature-parity-and-testing.md` ## Mission @@ -36,7 +40,7 @@ The old hub-app model must not be reintroduced. - Flutter owns UI and platform adapter only. - getter lives in the reusable `core-getter/src/main/rust/getter` git submodule (`https://github.com/DUpdateSystem/getter`); make getter changes in that submodule and update the superproject gitlink. - getter storage uses SQLite main DB plus separate cache DB. -- Package definitions are Lua files in repositories. +- Package definitions are package directories in repositories, with `metadata.jsonc`, optional `Manifest`, and direct-child version Lua scripts such as `1.2.3.lua` or `9999.lua`. - Lua returns JSON-like tables across the Lua/Rust boundary; Rust validates typed structs. - Package IDs are readable, e.g. `android/org.fdroid.fdroid`, not UUID primary identities. - Legacy Room migration must be automatic for normal users, but it is intentionally limited/simple. @@ -59,16 +63,18 @@ Recommended order: - SelectedUpdate - UpdateAction 3. Implement repository layout loader: - - `repo.toml` - - `packages/` - - `lib/` - - `templates/` + - getter data dir `repo/` plus `rc/` roots; + - `repo/metadata.jsonc` for local repository priority and generated-repository config; + - `repo//` repository aliases; + - package directories that directly contain `metadata.jsonc`; + - optional package `Manifest`, optional generated-package `.autogen.jsonc`, direct-child version scripts, and package-local `files/`; + - repository `luaclass/` helpers and `.metadata/autogen/` generator metadata. 4. Integrate `mlua` minimally: - - load a Lua package file; - - expose `require` search path for repo `lib/`; - - expose `package_from(repo, id)` later; + - load a package-directory version Lua script; + - expose `require` search path for repo `luaclass/`; + - expose `package_from()` later; - return JSON-like Lua table; - - validate into Rust structs. + - validate into Rust structs while deriving package identity from the package directory path. 5. Implement repository priority resolution. 6. Implement main DB and cache DB skeleton. 7. Write migration mapping tests before writing migration implementation. From 80df257d481710fa12bad11289d2dd9559f8be64 Mon Sep 17 00:00:00 2001 From: Xiangzhe Date: Sat, 27 Jun 2026 15:29:44 +0800 Subject: [PATCH 63/85] feat(app): expose F-Droid autogen bridge --- .../upgradeall/GetterBridgeRequestBuilder.kt | 28 +++ .../net/xzos/upgradeall/MainActivity.kt | 40 ++-- .../GetterBridgeRequestBuilderTest.kt | 53 ++++++ .../net/xzos/upgradeall/getter/NativeLib.kt | 2 + app_flutter/lib/cli_getter_adapter.dart | 25 +++ app_flutter/lib/getter_adapter.dart | 64 +++++++ app_flutter/lib/native_getter_adapter.dart | 48 ++++- .../test/native_getter_adapter_test.dart | 102 ++++++++++ .../src/main/rust/api_proxy/src/lib.rs | 175 +++++++++++++++++- .../0007-flutter-getter-bridge-contract.md | 13 +- 10 files changed, 515 insertions(+), 35 deletions(-) diff --git a/app_flutter/android/app/src/main/kotlin/net/xzos/upgradeall/GetterBridgeRequestBuilder.kt b/app_flutter/android/app/src/main/kotlin/net/xzos/upgradeall/GetterBridgeRequestBuilder.kt index 56e0716e..d87a03c3 100644 --- a/app_flutter/android/app/src/main/kotlin/net/xzos/upgradeall/GetterBridgeRequestBuilder.kt +++ b/app_flutter/android/app/src/main/kotlin/net/xzos/upgradeall/GetterBridgeRequestBuilder.kt @@ -1,5 +1,6 @@ package net.xzos.upgradeall +import org.json.JSONArray import org.json.JSONObject object GetterBridgeRequestBuilder { @@ -7,6 +8,33 @@ object GetterBridgeRequestBuilder { fun runtimeOperationRequest(args: Map<*, *>): String = operationRequest(args) + fun fdroidAutogenPreviewRequest(args: Map<*, *>): String { + val payload = args["payload"] as? Map<*, *> ?: emptyMap() + return JSONObject() + .put("payload", JSONObject(payload)) + .toString() + } + + fun autogenApplyRequest(args: Map<*, *>): String { + val previewJson = args["preview_json"] as? String + ?: throw IllegalArgumentException("preview_json is required") + val acceptance = args["acceptance"] as? Map<*, *> + val packageIds = acceptance + ?.get("package_ids") + ?.let { value -> value as? Collection<*> } + ?.map { value -> value.toString() } + ?: emptyList() + return JSONObject() + .put("preview", JSONObject(previewJson)) + .put( + "acceptance", + JSONObject() + .put("mode", acceptance?.get("mode") as? String ?: "all") + .put("package_ids", JSONArray(packageIds)), + ) + .toString() + } + private fun operationRequest(args: Map<*, *>): String { val operation = args["operation"] as? String ?: throw IllegalArgumentException("operation is required") diff --git a/app_flutter/android/app/src/main/kotlin/net/xzos/upgradeall/MainActivity.kt b/app_flutter/android/app/src/main/kotlin/net/xzos/upgradeall/MainActivity.kt index 21872479..a2dde2c7 100644 --- a/app_flutter/android/app/src/main/kotlin/net/xzos/upgradeall/MainActivity.kt +++ b/app_flutter/android/app/src/main/kotlin/net/xzos/upgradeall/MainActivity.kt @@ -10,7 +10,6 @@ import io.flutter.plugin.common.MethodChannel import java.io.File import java.util.concurrent.Executors import net.xzos.upgradeall.getter.NativeLib -import org.json.JSONArray import org.json.JSONObject class MainActivity : FlutterActivity() { @@ -59,6 +58,14 @@ class MainActivity : FlutterActivity() { nativeLib.applyInstalledAutogen(applyInstalledAutogenRequest(call)) } + "previewFdroidAutogen" -> runGetterBridge(result) { + nativeLib.previewFdroidAutogen(previewFdroidAutogenRequest(call)) + } + + "applyFdroidAutogen" -> runGetterBridge(result) { + nativeLib.applyFdroidAutogen(applyFdroidAutogenRequest(call)) + } + "importLegacyRoomDatabase" -> runGetterBridge(result) { nativeLib.importLegacyRoomDatabase(importLegacyRoomDatabaseRequest(call)) } @@ -171,23 +178,22 @@ class MainActivity : FlutterActivity() { private fun applyInstalledAutogenRequest(call: MethodCall): String { val args = call.arguments as? Map<*, *> ?: emptyMap() - val previewJson = args["preview_json"] as? String - ?: throw IllegalArgumentException("preview_json is required") - val acceptance = args["acceptance"] as? Map<*, *> - val packageIds = acceptance - ?.get("package_ids") - ?.let { value -> value as? Collection<*> } - ?.map { value -> value.toString() } - ?: emptyList() - return JSONObject() + return JSONObject(GetterBridgeRequestBuilder.autogenApplyRequest(args)) + .put("data_dir", getterDataDir().absolutePath) + .toString() + } + + private fun previewFdroidAutogenRequest(call: MethodCall): String { + val args = call.arguments as? Map<*, *> ?: emptyMap() + return JSONObject(GetterBridgeRequestBuilder.fdroidAutogenPreviewRequest(args)) + .put("data_dir", getterDataDir().absolutePath) + .toString() + } + + private fun applyFdroidAutogenRequest(call: MethodCall): String { + val args = call.arguments as? Map<*, *> ?: emptyMap() + return JSONObject(GetterBridgeRequestBuilder.autogenApplyRequest(args)) .put("data_dir", getterDataDir().absolutePath) - .put("preview", JSONObject(previewJson)) - .put( - "acceptance", - JSONObject() - .put("mode", acceptance?.get("mode") as? String ?: "all") - .put("package_ids", JSONArray(packageIds)), - ) .toString() } diff --git a/app_flutter/android/app/src/test/kotlin/net/xzos/upgradeall/GetterBridgeRequestBuilderTest.kt b/app_flutter/android/app/src/test/kotlin/net/xzos/upgradeall/GetterBridgeRequestBuilderTest.kt index 37917868..4c4962fc 100644 --- a/app_flutter/android/app/src/test/kotlin/net/xzos/upgradeall/GetterBridgeRequestBuilderTest.kt +++ b/app_flutter/android/app/src/test/kotlin/net/xzos/upgradeall/GetterBridgeRequestBuilderTest.kt @@ -51,6 +51,59 @@ class GetterBridgeRequestBuilderTest { assertEquals(0, json.getJSONObject("payload").length()) } + @Test + fun fdroidAutogenPreviewRequestPreservesGetterPayload() { + val json = JSONObject( + GetterBridgeRequestBuilder.fdroidAutogenPreviewRequest( + mapOf( + "payload" to mapOf( + "index_xml" to "", + "package_names" to listOf("org.fdroid.fdroid"), + ), + ), + ), + ) + + val payload = json.getJSONObject("payload") + assertEquals("", payload.getString("index_xml")) + assertEquals("org.fdroid.fdroid", payload.getJSONArray("package_names").getString(0)) + } + + @Test + fun autogenApplyRequestPreservesPreviewAndAcceptance() { + val json = JSONObject( + GetterBridgeRequestBuilder.autogenApplyRequest( + mapOf( + "preview_json" to "{\"operation\":\"fdroid.autogen.preview\"}", + "acceptance" to mapOf( + "mode" to "packages", + "package_ids" to listOf("android/f-droid/app/org.fdroid.fdroid"), + ), + ), + ), + ) + + assertEquals( + "fdroid.autogen.preview", + json.getJSONObject("preview").getString("operation"), + ) + val acceptance = json.getJSONObject("acceptance") + assertEquals("packages", acceptance.getString("mode")) + assertEquals( + "android/f-droid/app/org.fdroid.fdroid", + acceptance.getJSONArray("package_ids").getString(0), + ) + } + + @Test + fun autogenApplyRequestRequiresPreviewJson() { + val error = assertThrows(IllegalArgumentException::class.java) { + GetterBridgeRequestBuilder.autogenApplyRequest(mapOf("acceptance" to emptyMap())) + } + + assertEquals("preview_json is required", error.message) + } + @Test fun runtimeOperationRequestRequiresOperation() { val error = assertThrows(IllegalArgumentException::class.java) { diff --git a/app_flutter/android/getter_bridge/src/main/kotlin/net/xzos/upgradeall/getter/NativeLib.kt b/app_flutter/android/getter_bridge/src/main/kotlin/net/xzos/upgradeall/getter/NativeLib.kt index e5b2ec14..0ce3921e 100644 --- a/app_flutter/android/getter_bridge/src/main/kotlin/net/xzos/upgradeall/getter/NativeLib.kt +++ b/app_flutter/android/getter_bridge/src/main/kotlin/net/xzos/upgradeall/getter/NativeLib.kt @@ -13,6 +13,8 @@ class NativeLib { external fun initializeBridge(context: Context): String external fun previewInstalledAutogen(context: Context, requestJson: String): String external fun applyInstalledAutogen(requestJson: String): String + external fun previewFdroidAutogen(requestJson: String): String + external fun applyFdroidAutogen(requestJson: String): String external fun importLegacyRoomDatabase(requestJson: String): String external fun legacyReportList(requestJson: String): String external fun readOperation(requestJson: String): String diff --git a/app_flutter/lib/cli_getter_adapter.dart b/app_flutter/lib/cli_getter_adapter.dart index 4042b011..e796de52 100644 --- a/app_flutter/lib/cli_getter_adapter.dart +++ b/app_flutter/lib/cli_getter_adapter.dart @@ -94,6 +94,31 @@ class CliGetterAdapter implements GetterAdapter { ); } + @override + Future previewFdroidAutogen( + Map payload, + ) async { + throw const GetterBridgeException( + GetterError( + code: 'bridge.unsupported', + message: 'CLI adapter does not expose fixture-backed F-Droid autogen', + ), + ); + } + + @override + Future applyFdroidAutogen( + InstalledAutogenPreview preview, { + List? acceptedPackageIds, + }) async { + throw const GetterBridgeException( + GetterError( + code: 'bridge.unsupported', + message: 'CLI adapter does not expose fixture-backed F-Droid autogen', + ), + ); + } + @override Future checkPackageForUpdate( String packageId, { diff --git a/app_flutter/lib/getter_adapter.dart b/app_flutter/lib/getter_adapter.dart index d63f328f..7dced2d3 100644 --- a/app_flutter/lib/getter_adapter.dart +++ b/app_flutter/lib/getter_adapter.dart @@ -31,6 +31,15 @@ abstract interface class GetterAdapter { List? acceptedPackageIds, }); + Future previewFdroidAutogen( + Map payload, + ); + + Future applyFdroidAutogen( + InstalledAutogenPreview preview, { + List? acceptedPackageIds, + }); + Future checkPackageForUpdate( String packageId, { String? repositoryId, @@ -224,6 +233,61 @@ class FakeGetterAdapter implements GetterAdapter { }); } + @override + Future previewFdroidAutogen( + Map payload, + ) async { + return InstalledAutogenPreview.fromJson(const { + 'operation': 'fdroid.autogen.preview', + 'provider': 'fdroid', + 'endpoint_id': 'official', + 'endpoint_url': 'https://f-droid.org/repo', + 'target_repo_id': 'autogen', + 'target_repo_path': '/fake/getter/repo/autogen', + 'summary': { + 'candidate_count': 1, + 'skipped_count': 0, + 'write_count': 1, + 'delete_count': 0, + }, + 'candidates': [ + { + 'package_id': 'android/f-droid/app/org.fdroid.fdroid', + 'kind': 'android', + 'display_name': 'F-Droid', + 'installed_target': { + 'kind': 'android_package', + 'package_name': 'org.fdroid.fdroid', + }, + 'action': 'create', + 'output_relative_path': 'android/f-droid/app/org.fdroid.fdroid', + 'content_hash': 'sha512:fake-fdroid', + 'content': '-- fake generated F-Droid content', + }, + ], + 'skipped': [], + 'diagnostics': [], + }); + } + + @override + Future applyFdroidAutogen( + InstalledAutogenPreview preview, { + List? acceptedPackageIds, + }) async { + return InstalledAutogenApplyResult.fromJson(const { + 'target_repo_id': 'autogen', + 'target_repo_path': '/fake/getter/repo/autogen', + 'applied_count': 1, + 'applied': [ + { + 'package_id': 'android/f-droid/app/org.fdroid.fdroid', + 'output_relative_path': 'android/f-droid/app/org.fdroid.fdroid', + }, + ], + }); + } + @override Future checkPackageForUpdate( String packageId, { diff --git a/app_flutter/lib/native_getter_adapter.dart b/app_flutter/lib/native_getter_adapter.dart index bbaa21e5..17ea0751 100644 --- a/app_flutter/lib/native_getter_adapter.dart +++ b/app_flutter/lib/native_getter_adapter.dart @@ -81,15 +81,30 @@ class MethodChannelGetterAdapter extends FakeGetterAdapter { }) async { final data = await _invokeGetterData( 'applyInstalledAutogen', - { - 'preview_json': jsonEncode(preview.rawJson), - 'acceptance': acceptedPackageIds == null - ? const {'mode': 'all'} - : { - 'mode': 'packages', - 'package_ids': acceptedPackageIds, - }, - }, + _autogenApplyArguments(preview, acceptedPackageIds), + ); + return InstalledAutogenApplyResult.fromJson(data); + } + + @override + Future previewFdroidAutogen( + Map payload, + ) async { + final data = await _invokeGetterData( + 'previewFdroidAutogen', + {'payload': payload}, + ); + return InstalledAutogenPreview.fromJson(data); + } + + @override + Future applyFdroidAutogen( + InstalledAutogenPreview preview, { + List? acceptedPackageIds, + }) async { + final data = await _invokeGetterData( + 'applyFdroidAutogen', + _autogenApplyArguments(preview, acceptedPackageIds), ); return InstalledAutogenApplyResult.fromJson(data); } @@ -314,6 +329,21 @@ class MethodChannelGetterAdapter extends FakeGetterAdapter { return {'task_id': taskId}; } + Map _autogenApplyArguments( + InstalledAutogenPreview preview, + List? acceptedPackageIds, + ) { + return { + 'preview_json': jsonEncode(preview.rawJson), + 'acceptance': acceptedPackageIds == null + ? const {'mode': 'all'} + : { + 'mode': 'packages', + 'package_ids': acceptedPackageIds, + }, + }; + } + Future> _invokeGetterData( String method, Map arguments, diff --git a/app_flutter/test/native_getter_adapter_test.dart b/app_flutter/test/native_getter_adapter_test.dart index 7dd5b3c5..d68c7e79 100644 --- a/app_flutter/test/native_getter_adapter_test.dart +++ b/app_flutter/test/native_getter_adapter_test.dart @@ -99,6 +99,76 @@ void main() { expect(result.applied.single.packageId, 'android/app/com.example.autogen'); }); + test('native F-Droid autogen forwards getter-owned payloads', () async { + final calls = []; + TestDefaultBinaryMessengerBinding.instance.defaultBinaryMessenger + .setMockMethodCallHandler(channel, (call) async { + calls.add(call); + switch (call.method) { + case 'previewFdroidAutogen': + return jsonEncode({ + 'ok': true, + 'command': 'autogen fdroid preview', + 'data': _fdroidPreviewJson(), + 'warnings': [], + }); + case 'applyFdroidAutogen': + return jsonEncode({ + 'ok': true, + 'command': 'autogen fdroid apply', + 'data': { + 'target_repo_id': 'autogen', + 'target_repo_path': '/getter/repo/autogen', + 'applied_count': 1, + 'applied': [ + { + 'package_id': 'android/f-droid/app/org.fdroid.fdroid', + 'output_relative_path': + 'android/f-droid/app/org.fdroid.fdroid', + }, + ], + }, + 'warnings': [], + }); + default: + fail('unexpected method ${call.method}'); + } + }); + + const adapter = MethodChannelGetterAdapter(channel: channel); + final payload = { + 'index_xml': '', + 'package_names': ['org.fdroid.fdroid'], + }; + final preview = await adapter.previewFdroidAutogen(payload); + final result = await adapter.applyFdroidAutogen( + preview, + acceptedPackageIds: const [ + 'android/f-droid/app/org.fdroid.fdroid', + ], + ); + + expect(calls.first.method, 'previewFdroidAutogen'); + expect(calls.first.arguments, {'payload': payload}); + expect(preview.operation, 'fdroid.autogen.preview'); + expect( + preview.candidates.single.packageId, + 'android/f-droid/app/org.fdroid.fdroid', + ); + expect(calls.last.method, 'applyFdroidAutogen'); + final args = (calls.last.arguments as Map) + .cast(); + expect(jsonDecode(args['preview_json']! as String), preview.rawJson); + expect(args['acceptance'], { + 'mode': 'packages', + 'package_ids': ['android/f-droid/app/org.fdroid.fdroid'], + }); + expect( + result.applied.single.packageId, + 'android/f-droid/app/org.fdroid.fdroid', + ); + }); + test('native legacy import and reports parse getter envelopes', () async { final calls = []; TestDefaultBinaryMessengerBinding.instance.defaultBinaryMessenger @@ -499,6 +569,38 @@ Map _runtimeTaskJson( 'updated_at': 1, }; +Map _fdroidPreviewJson() => { + 'operation': 'fdroid.autogen.preview', + 'provider': 'fdroid', + 'endpoint_id': 'official', + 'endpoint_url': 'https://f-droid.org/repo', + 'target_repo_id': 'autogen', + 'target_repo_path': '/getter/repo/autogen', + 'summary': { + 'candidate_count': 1, + 'skipped_count': 0, + 'write_count': 1, + 'delete_count': 0, + }, + 'candidates': [ + { + 'package_id': 'android/f-droid/app/org.fdroid.fdroid', + 'kind': 'android', + 'display_name': 'F-Droid', + 'installed_target': { + 'kind': 'android_package', + 'package_name': 'org.fdroid.fdroid', + }, + 'action': 'create', + 'output_relative_path': 'android/f-droid/app/org.fdroid.fdroid', + 'content_hash': 'sha512:fake-fdroid', + 'content': '-- fake generated F-Droid content', + }, + ], + 'skipped': [], + 'diagnostics': [], +}; + Map _previewJson() => { 'operation': 'installed.preview', 'target_repo_id': 'autogen', diff --git a/core-getter/src/main/rust/api_proxy/src/lib.rs b/core-getter/src/main/rust/api_proxy/src/lib.rs index 26aa96ba..a0070838 100644 --- a/core-getter/src/main/rust/api_proxy/src/lib.rs +++ b/core-getter/src/main/rust/api_proxy/src/lib.rs @@ -1,6 +1,7 @@ extern crate jni; use getter::operations::autogen::{self, AutogenAcceptance, AutogenOperationError}; +use getter::operations::fdroid_autogen; use getter::operations::legacy_room::{self, LegacyRoomOperationError}; use getter::operations::read_model::{self, ReadModelOperationError}; use getter::operations::runtime as runtime_operations; @@ -39,7 +40,22 @@ struct ApplyInstalledAutogenRequest { data_dir: PathBuf, preview: Value, #[serde(default)] - acceptance: ApplyInstalledAutogenAcceptance, + acceptance: ApplyAutogenAcceptance, +} + +#[derive(Debug, Deserialize)] +struct PreviewFdroidAutogenRequest { + data_dir: PathBuf, + #[serde(default)] + payload: Value, +} + +#[derive(Debug, Deserialize)] +struct ApplyFdroidAutogenRequest { + data_dir: PathBuf, + preview: Value, + #[serde(default)] + acceptance: ApplyAutogenAcceptance, } #[derive(Debug, Deserialize)] @@ -71,7 +87,7 @@ struct RuntimeOperationRequest { } #[derive(Debug, Default, Deserialize)] -struct ApplyInstalledAutogenAcceptance { +struct ApplyAutogenAcceptance { #[serde(default)] mode: Option, #[serde(default)] @@ -206,6 +222,38 @@ pub extern "C" fn Java_net_xzos_upgradeall_getter_NativeLib_applyInstalledAutoge java_string_or_fallback(&mut env, response) } +#[no_mangle] +pub extern "C" fn Java_net_xzos_upgradeall_getter_NativeLib_previewFdroidAutogen<'local>( + mut env: JNIEnv<'local>, + _: JObject<'local>, + request_json: JString<'local>, +) -> JString<'local> { + let command = "autogen fdroid preview"; + let response = match jstring_to_string(&mut env, &request_json) + .and_then(|raw| preview_fdroid_autogen(&raw)) + { + Ok(data) => success_envelope(command, data), + Err(error) => operation_error_envelope(command, error), + }; + java_string_or_fallback(&mut env, response) +} + +#[no_mangle] +pub extern "C" fn Java_net_xzos_upgradeall_getter_NativeLib_applyFdroidAutogen<'local>( + mut env: JNIEnv<'local>, + _: JObject<'local>, + request_json: JString<'local>, +) -> JString<'local> { + let command = "autogen fdroid apply"; + let response = match jstring_to_string(&mut env, &request_json) + .and_then(|raw| apply_fdroid_autogen(&raw)) + { + Ok(data) => success_envelope(command, data), + Err(error) => operation_error_envelope(command, error), + }; + java_string_or_fallback(&mut env, response) +} + #[no_mangle] pub extern "C" fn Java_net_xzos_upgradeall_getter_NativeLib_runtimeOperation<'local>( mut env: JNIEnv<'local>, @@ -320,6 +368,38 @@ fn apply_installed_autogen(request_json: &str) -> Result Result { + let request: PreviewFdroidAutogenRequest = serde_json::from_str(request_json) + .map_err(|source| BridgeOperationError::InvalidRequest(source.to_string()))?; + let db = open_main_db(&request.data_dir)?; + let cache_db = open_cache_db(&request.data_dir)?; + let payload = if request.payload.is_null() { + "{}".to_owned() + } else { + request.payload.to_string() + }; + Ok(fdroid_autogen::preview_fdroid_packages_json( + &request.data_dir, + &db, + &cache_db, + &payload, + )?) +} + +fn apply_fdroid_autogen(request_json: &str) -> Result { + let request: ApplyFdroidAutogenRequest = serde_json::from_str(request_json) + .map_err(|source| BridgeOperationError::InvalidRequest(source.to_string()))?; + let db = open_main_db(&request.data_dir)?; + let preview = autogen::unwrap_preview_payload(request.preview, "fdroid.autogen.preview")?; + let acceptance = request.acceptance.into_autogen_acceptance()?; + Ok(fdroid_autogen::apply_fdroid_preview_json( + &request.data_dir, + &db, + &preview, + &acceptance, + )?) +} + fn import_legacy_room_database(request_json: &str) -> Result { let request: ImportLegacyRoomDatabaseRequest = serde_json::from_str(request_json) .map_err(|source| BridgeOperationError::InvalidRequest(source.to_string()))?; @@ -443,13 +523,13 @@ fn runtime_operation_with_runtime( .map_err(BridgeOperationError::Runtime) } -impl ApplyInstalledAutogenAcceptance { +impl ApplyAutogenAcceptance { fn into_autogen_acceptance(self) -> Result { match self.mode.as_deref().unwrap_or("all") { "all" => Ok(AutogenAcceptance::AcceptAll), "packages" => Ok(AutogenAcceptance::Accept(self.package_ids)), other => Err(BridgeOperationError::InvalidRequest(format!( - "unsupported installed autogen acceptance mode '{other}'" + "unsupported autogen acceptance mode '{other}'" ))), } } @@ -462,6 +542,14 @@ fn open_main_db(data_dir: &Path) -> Result Result { + std::fs::create_dir_all(data_dir) + .map_err(|source| BridgeOperationError::Storage(source.to_string()))?; + Ok(getter::storage::CacheDb::open( + data_dir.join(CACHE_DB_FILE), + )?) +} + fn scan_installed_inventory( options: InstalledInventoryScanOptions, ) -> Result { @@ -688,7 +776,8 @@ impl From for BridgeOperationError { mod tests { use super::*; use getter::core::{ - repository::{RepositoryMetadata, REPO_API_VERSION_V1}, + autogen::FDROID_AUTOGEN_GENERATOR, + repository::{RepositoryMetadata, RepositoryPackageDirectoryLayout, REPO_API_VERSION_V1}, runtime::{PackageVersionLuaObject, SealedActionPlan}, RepositoryPriority, UpdateAction, }; @@ -696,7 +785,7 @@ mod tests { #[test] fn packages_acceptance_defaults_to_all() { - let acceptance = ApplyInstalledAutogenAcceptance::default() + let acceptance = ApplyAutogenAcceptance::default() .into_autogen_acceptance() .expect("acceptance"); @@ -705,7 +794,7 @@ mod tests { #[test] fn packages_acceptance_preserves_getter_package_ids() { - let acceptance = ApplyInstalledAutogenAcceptance { + let acceptance = ApplyAutogenAcceptance { mode: Some("packages".to_owned()), package_ids: vec!["android/org.fdroid.fdroid".parse().expect("package id")], } @@ -720,6 +809,59 @@ mod tests { } } + #[test] + fn fdroid_autogen_bridge_preview_and_apply_write_package_directories() { + let temp = tempfile::tempdir().unwrap(); + let data_dir = temp.path().join("data"); + let request = json!({ + "data_dir": data_dir, + "payload": { + "index_xml": fdroid_fixture(), + "package_names": ["org.fdroid.fdroid"] + } + }); + + let preview = preview_fdroid_autogen(&request.to_string()).expect("F-Droid preview"); + + assert_eq!(preview["operation"], "fdroid.autogen.preview"); + assert_eq!( + preview["candidates"][0]["package_id"], + "android/f-droid/app/org.fdroid.fdroid" + ); + + let apply = apply_fdroid_autogen( + &json!({ + "data_dir": data_dir, + "preview": preview, + "acceptance": { "mode": "packages", "package_ids": ["android/f-droid/app/org.fdroid.fdroid"] } + }) + .to_string(), + ) + .expect("F-Droid apply"); + + assert_eq!(apply["applied_count"], 1); + assert_eq!( + apply["applied"][0]["package_id"], + "android/f-droid/app/org.fdroid.fdroid" + ); + let repo_root = temp.path().join("data/repo/autogen"); + let package_dir = repo_root.join("android/f-droid/app/org.fdroid.fdroid"); + assert!(package_dir.join("metadata.jsonc").is_file()); + assert!(package_dir.join("Manifest").is_file()); + assert!(package_dir.join("9999.lua").is_file()); + assert!(package_dir.join(".autogen.jsonc").is_file()); + let record: Value = serde_json::from_str( + &std::fs::read_to_string(package_dir.join(".autogen.jsonc")).unwrap(), + ) + .unwrap(); + assert_eq!(record["generator"], FDROID_AUTOGEN_GENERATOR); + assert_eq!(record["input"]["package_name"], "org.fdroid.fdroid"); + let layout = RepositoryPackageDirectoryLayout::load(&repo_root).unwrap(); + assert!(layout + .package(&"android/f-droid/app/org.fdroid.fdroid".parse().unwrap()) + .is_some()); + } + #[test] fn read_operation_lists_repositories_and_evaluates_packages() { let temp = tempfile::tempdir().unwrap(); @@ -929,6 +1071,25 @@ mod tests { assert!(detail.unwrap().contains("unsupported runtime operation")); } + fn fdroid_fixture() -> &'static str { + r#" + + + + F-Droid + App repository client + + 1.20.0 + 1020000 + org.fdroid.fdroid_1020000.apk + aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa + 1234567 + + + +"# + } + fn write_static_update_repo(root: &std::path::Path) { let package_dir = root.join("android/org.fdroid.fdroid"); fs::create_dir_all(&package_dir).unwrap(); diff --git a/docs/architecture/adr/0007-flutter-getter-bridge-contract.md b/docs/architecture/adr/0007-flutter-getter-bridge-contract.md index d896ef0e..21e1b3a8 100644 --- a/docs/architecture/adr/0007-flutter-getter-bridge-contract.md +++ b/docs/architecture/adr/0007-flutter-getter-bridge-contract.md @@ -45,9 +45,18 @@ previewInstalledAutogen(scanOptions) applyInstalledAutogen(preview, acceptedPackages) ``` -The Android product APK packages a slim `:getter_bridge` library under `app_flutter/android/getter_bridge`. It builds the Rust `api_proxy` cdylib and includes only the no-UI native bridge / installed-inventory provider classes needed by the Flutter product path, avoiding the legacy native `:app` UI and old `GetterPort` hub/RPC wrapper surface. `MainActivity` exposes a no-UI `net.xzos.upgradeall/getter_bridge` MethodChannel that derives the app-private getter data directory and forwards migration and installed-autogen requests to JNI entrypoints returning getter-style JSON envelopes. +The fourth accepted API surface exposes the first F-Droid autogen native bridge slice, reusing getter-owned autogen preview/apply DTOs while keeping F-Droid catalog parsing, package-path mapping, repository coverage, and file generation in Rust getter: -Internally, Rust/native bridge code scans Android inventory through the platform adapter, then asks getter-owned shared autogen operations to plan/apply installed-autogen output in the configured generated repository target (`generated_repository`, default `autogen`). `MethodChannelGetterAdapter` consumes the returned getter-style JSON envelopes. Flutter renders getter-owned preview/apply DTOs and scan diagnostics, then passes the package ids from displayed accepted preview candidates back to getter on apply; it must not expose a product Dart `InstalledInventoryPlatform` scanner or convert Android package names into package ids. +```text +previewFdroidAutogen(payload) +applyFdroidAutogen(preview, acceptedPackages) +``` + +The initial `previewFdroidAutogen` payload is the fixture-backed/offline ADR-0006 F-Droid autogen request shape (`index_xml`, `package_names`, optional `installed_inventory`, endpoint fields, and cache mode) so bridge/widget tests can exercise the cross-boundary contract without live HTTP. Flutter may forward user/test input and render getter-owned candidate/skip/diagnostic DTOs, but it must not parse F-Droid indexes, derive UpgradeAll package paths from F-Droid package names, decide shadowing, or generate Lua. + +The Android product APK packages a slim `:getter_bridge` library under `app_flutter/android/getter_bridge`. It builds the Rust `api_proxy` cdylib and includes only the no-UI native bridge / installed-inventory provider classes needed by the Flutter product path, avoiding the legacy native `:app` UI and old `GetterPort` hub/RPC wrapper surface. `MainActivity` exposes a no-UI `net.xzos.upgradeall/getter_bridge` MethodChannel that derives the app-private getter data directory and forwards migration, installed-autogen, F-Droid autogen, read-model, and runtime requests to JNI entrypoints returning getter-style JSON envelopes. + +Internally, Rust/native bridge code scans Android inventory through the platform adapter for installed-autogen, then asks getter-owned shared autogen operations to plan/apply output in the configured generated repository target (`generated_repository`, default `autogen`). F-Droid autogen bridge calls the same getter-owned F-Droid catalog/autogen operations used by the CLI. `MethodChannelGetterAdapter` consumes the returned getter-style JSON envelopes. Flutter renders getter-owned preview/apply DTOs and scan/provider diagnostics, then passes the package ids from displayed accepted preview candidates back to getter on apply; it must not expose a product Dart `InstalledInventoryPlatform` scanner, convert Android/F-Droid package names into package ids, parse provider data, or generate package content. `loadSnapshot()` composes smaller getter-owned read-model operations into the UI shell's first snapshot DTO. In the Android/native product path, `MethodChannelGetterAdapter` calls `readOperation` for `repository_list`, `tracked_package_list`, and `package_eval`; Rust getter reads SQLite repository/tracked-package state and evaluates registered Lua packages. Flutter may parse and combine those returned DTOs for rendering, but must not perform repository resolution, Lua validation/evaluation, version comparison, migration mapping, or update selection in Dart. `readMigrationReports()` must go through a getter operation such as `legacy report-list`; Flutter must not inspect getter's data-directory layout directly. Runtime task rendering must use ADR-0011 runtime task snapshot APIs and opaque action/task controls; Flutter must not synthesize task states, retry policy, installer behavior, or update decisions. From eb1719d835b0fa28fb9ca2a789eccb96817a1c95 Mon Sep 17 00:00:00 2001 From: Xiangzhe Date: Sat, 27 Jun 2026 16:49:37 +0800 Subject: [PATCH 64/85] feat(getter): add GitHub latest commit fixtures --- core-getter/src/main/rust/getter | 2 +- .../adr/0006-package-centric-cli-command-contract.md | 5 ++++- 2 files changed, 5 insertions(+), 2 deletions(-) diff --git a/core-getter/src/main/rust/getter b/core-getter/src/main/rust/getter index 32e0fb33..a67e8227 160000 --- a/core-getter/src/main/rust/getter +++ b/core-getter/src/main/rust/getter @@ -1 +1 @@ -Subproject commit 32e0fb337afa3c773547645b17553deb70282767 +Subproject commit a67e8227ffefae4b6665ec1a4a2e29e4839fe3e1 diff --git a/docs/architecture/adr/0006-package-centric-cli-command-contract.md b/docs/architecture/adr/0006-package-centric-cli-command-contract.md index a234bbfa..3c61a9a4 100644 --- a/docs/architecture/adr/0006-package-centric-cli-command-contract.md +++ b/docs/architecture/adr/0006-package-centric-cli-command-contract.md @@ -34,6 +34,7 @@ getter --data-dir autogen installed apply --preview (--acc getter --data-dir autogen fdroid preview --index [--package ...] [--inventory ] getter --data-dir autogen fdroid apply --preview (--accept-all|--accept ...) getter --data-dir provider github releases --owner --repo --releases [--asset-include ] [--asset-exclude ] [--include-prereleases] [--refresh] +getter --data-dir provider github latest-commit --owner --repo --commit [--ref ] [--refresh] getter --data-dir autogen cleanup preview --inventory getter --data-dir autogen cleanup apply --preview (--accept-all|--accept ...) getter --data-dir legacy import-room-bundle @@ -107,7 +108,9 @@ The first installed-app autogen slice accepts an Android/platform-provided inven The first F-Droid autogen CLI/dev slice is fixture-backed and offline: `autogen fdroid preview --index --package ` parses the supplied F-Droid index through getter-owned provider/cache code, maps requested upstream package names to package paths such as `android/f-droid/app/org.fdroid.fdroid`, skips packages covered by higher-priority registered repositories, and returns a generated package-directory preview without writing files. The same command may use `--inventory ` to match installed Android package facts from an installed-inventory fixture against the cached F-Droid catalog before producing the same preview shape; at least one `--package` or `--inventory` input is required, and both may be supplied. `autogen fdroid apply` writes accepted package directories to the configured generated repository and records `.autogen.jsonc` with generator `fdroid-catalog`. This slice does not perform live HTTP, downloads, installer handoff, Flutter/Kotlin provider parsing, or final reusable-provider host API execution; those remain governed by ADR-0012 and later provider/downloader/installer slices. -The first GitHub provider CLI/dev slice is also fixture-backed and offline: `provider github releases --owner --repo --releases ` parses a controlled GitHub REST releases JSON response through getter-owned provider/cache code, normalizes matching release assets into update candidates, and returns diagnostics such as `provider.github.asset_not_found` when filters match no assets. `--asset-include` and `--asset-exclude` are regex filters over asset names, prereleases are excluded unless `--include-prereleases` is supplied, and `--refresh` exercises forced provider-cache replacement/stale semantics against the fixture. This command does not generate packages, run Lua provider modules, perform live HTTP/auth/rate-limit handling, create runtime actions, download files, invoke installers, or move any GitHub parsing/filtering into Flutter/Kotlin. +The first GitHub provider CLI/dev slice is fixture-backed and offline: `provider github releases --owner --repo --releases ` parses a controlled GitHub REST releases JSON response through getter-owned provider/cache code, normalizes matching release assets into update candidates, and returns diagnostics such as `provider.github.asset_not_found` when filters match no assets. `--asset-include` and `--asset-exclude` are regex filters over asset names, prereleases are excluded unless `--include-prereleases` is supplied, and `--refresh` exercises forced provider-cache replacement/stale semantics against the fixture. This command does not generate packages, run Lua provider modules, perform live HTTP/auth/rate-limit handling, create runtime actions, download files, invoke installers, or move any GitHub parsing/filtering into Flutter/Kotlin. + +The fixture-backed GitHub latest-commit CLI/dev slice is `provider github latest-commit --owner --repo --commit [--ref ] [--refresh]`. It parses a controlled GitHub commit JSON response through getter-owned provider/cache code and returns `operation = "github.latest_commit"`, `live = true`, `version`, `revision`, and `latest_commit` as a live/floating revision DTO. The omitted `--ref` default is the literal cache-key/query ref `HEAD`; explicit refs produce distinct cache keys. Latest-commit provider cache keys are distinct from release cache keys and include a dedicated latest-commit provider/cache version, request type, API-base digest, owner, repo, and ref. Empty or whitespace commit SHAs are provider refresh failures and are never cached; during forced refresh with an existing valid cache entry, they return stale cache diagnostics. The response is not an ordinary update-check result: it does not contain `candidates`, `selected_update`, artifact lists, runtime actions, or `action_id`. Malformed fixture JSON and invalid commit facts use provider operation exit code `30`; missing or unsupported CLI flags use usage exit code `2`. This slice does not generate packages, run Lua provider modules, perform live HTTP/auth/rate-limit handling, create runtime actions, download files, invoke installers, or move any GitHub parsing/filtering into Flutter/Kotlin. `update check --fixture ` is the first Phase D offline update-check slice. The fixture contract is explicitly offline and uses `format = "getter-offline-update-check"`, `version = 1`, `package_id`, optional `installed_version`, optional `pin_version` (with transitional `ignored_version` accepted as an input alias), and normalized candidate/artifact DTOs. The command returns `network_required = false`, observed `installed_version`, `effective_local_version`, a getter-owned status (`update_available`, `up_to_date`, or `no_candidates`), the selected update when one exists, and generated download/install action DTOs. An update-selected result must have an actionable artifact; a selected candidate without artifacts is a structured update-check error rather than `update_available` with no actions. It reuses Rust getter update selection and version comparison; it does not run providers, perform downloads, persist download tasks, stream events, or call Android installers. From be6f4b37bbc35a9572a8d42eabf1d577eadee696 Mon Sep 17 00:00:00 2001 From: Xiangzhe Date: Sat, 27 Jun 2026 18:00:09 +0800 Subject: [PATCH 65/85] test(getter): cover F-Droid luaclass shape --- core-getter/src/main/rust/getter | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/core-getter/src/main/rust/getter b/core-getter/src/main/rust/getter index a67e8227..496bc08b 160000 --- a/core-getter/src/main/rust/getter +++ b/core-getter/src/main/rust/getter @@ -1 +1 @@ -Subproject commit a67e8227ffefae4b6665ec1a4a2e29e4839fe3e1 +Subproject commit 496bc08b44b20fd02af00490194ec526647e7c7d From 526e5d49b3110b532ed57a79a7aa2f542f4df0c5 Mon Sep 17 00:00:00 2001 From: Xiangzhe Date: Sat, 27 Jun 2026 18:09:58 +0800 Subject: [PATCH 66/85] test(getter): cover GitHub luaclass shape --- core-getter/src/main/rust/getter | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/core-getter/src/main/rust/getter b/core-getter/src/main/rust/getter index 496bc08b..d8cd8c97 160000 --- a/core-getter/src/main/rust/getter +++ b/core-getter/src/main/rust/getter @@ -1 +1 @@ -Subproject commit 496bc08b44b20fd02af00490194ec526647e7c7d +Subproject commit d8cd8c97501c6ff20f49a7b0819eaeb372fde559 From cb74bba1e9c36fae03674aa78973a95c10fae765 Mon Sep 17 00:00:00 2001 From: Xiangzhe Date: Sat, 27 Jun 2026 18:40:03 +0800 Subject: [PATCH 67/85] feat(getter): add Lua host binding seam --- core-getter/src/main/rust/getter | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/core-getter/src/main/rust/getter b/core-getter/src/main/rust/getter index d8cd8c97..7eb6ab2d 160000 --- a/core-getter/src/main/rust/getter +++ b/core-getter/src/main/rust/getter @@ -1 +1 @@ -Subproject commit d8cd8c97501c6ff20f49a7b0819eaeb372fde559 +Subproject commit 7eb6ab2d4b4cfe2515c0fcabf469b91c3ed04f12 From 2f6585e32d0dda2ff534da389b27868e629aea30 Mon Sep 17 00:00:00 2001 From: Xiangzhe Date: Sat, 27 Jun 2026 19:54:14 +0800 Subject: [PATCH 68/85] feat(getter): add dev Lua provider host eval --- core-getter/src/main/rust/getter | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/core-getter/src/main/rust/getter b/core-getter/src/main/rust/getter index 7eb6ab2d..236cfaed 160000 --- a/core-getter/src/main/rust/getter +++ b/core-getter/src/main/rust/getter @@ -1 +1 @@ -Subproject commit 7eb6ab2d4b4cfe2515c0fcabf469b91c3ed04f12 +Subproject commit 236cfaedf9d21833045337962cfcc375a9525c2d From 345ef45c7b80de606d742b6a6ff32818726640ef Mon Sep 17 00:00:00 2001 From: Xiangzhe Date: Sat, 27 Jun 2026 20:40:22 +0800 Subject: [PATCH 69/85] feat(getter): add F-Droid dev Lua provider host eval --- core-getter/src/main/rust/getter | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/core-getter/src/main/rust/getter b/core-getter/src/main/rust/getter index 236cfaed..0f74313b 160000 --- a/core-getter/src/main/rust/getter +++ b/core-getter/src/main/rust/getter @@ -1 +1 @@ -Subproject commit 236cfaedf9d21833045337962cfcc375a9525c2d +Subproject commit 0f74313b52e3cc6083223125376dfa2a3b122a80 From 4e06159dda565d761e6092300c51ca81dd0869bb Mon Sep 17 00:00:00 2001 From: Xiangzhe Date: Sat, 27 Jun 2026 21:56:56 +0800 Subject: [PATCH 70/85] feat(getter): add builtin luaclass fallback --- core-getter/src/main/rust/getter | 2 +- ...tter-owned-provider-modules-and-autogen-refresh.md | 11 ++++++++++- docs/lua-api/repository-layout.md | 6 +++++- 3 files changed, 16 insertions(+), 3 deletions(-) diff --git a/core-getter/src/main/rust/getter b/core-getter/src/main/rust/getter index 0f74313b..517bbcf4 160000 --- a/core-getter/src/main/rust/getter +++ b/core-getter/src/main/rust/getter @@ -1 +1 @@ -Subproject commit 0f74313b52e3cc6083223125376dfa2a3b122a80 +Subproject commit 517bbcf43d4244b93bf8b0f49077905aaf3b630f diff --git a/docs/architecture/adr/0012-getter-owned-provider-modules-and-autogen-refresh.md b/docs/architecture/adr/0012-getter-owned-provider-modules-and-autogen-refresh.md index 98eb0c74..2230dc8e 100644 --- a/docs/architecture/adr/0012-getter-owned-provider-modules-and-autogen-refresh.md +++ b/docs/architecture/adr/0012-getter-owned-provider-modules-and-autogen-refresh.md @@ -69,7 +69,7 @@ To avoid drifting back into the old hub-app model, ADR-0012 uses these terms str - **Qualified package atom**: `[::repo-name]`; omitting `::repo-name` resolves by repository priority, specifying it resolves only that local alias. If `repo/metadata.jsonc` is missing, built-in priority defaults are `local` = 100, `autogen` = -1, all other aliases = 0, with same-priority aliases resolved in lexicographic order. If present, the priority map is lookup-only: getter discovers actual repository alias directories first, then queries the map by alias; entries for nonexistent aliases are inert and do not warn, create repositories, display repositories, or participate in sorting. `generated_repository` defaults to `autogen` when omitted, and starter config should include that default as a comment users may uncomment/change. When autogen runs with target `autogen`, getter creates `repo/autogen/` if needed; any non-`autogen` target must already exist or autogen apply reports a configuration error. `generated_repository` only decides autogen output target and does not participate in package resolution except through the normal priority map. If `repo/metadata.jsonc` exists but cannot be parsed, getter reports a configuration diagnostic instead of silently falling back. - **Provider endpoint/catalog**: an upstream service or index, such as the official F-Droid catalog endpoint or the GitHub API endpoint for a repository. It is not an UpgradeAll repository. - **Package source**: a source declaration inside one package definition/version script that uses a provider module to discover candidates/artifacts. -- **Reusable Lua provider module/class**: a Lua helper under `luaclass/`, such as `luaclass.fdroid_android` or `luaclass.github_android_apk`, that fills common lifecycle behavior and calls getter provider host APIs. +- **Reusable Lua provider module/class**: a Lua helper under `luaclass/`, such as `luaclass.fdroid_android` or `luaclass.github_android_apk`, that fills common lifecycle behavior and calls getter provider host APIs. Getter may ship standard `luaclass.*` modules as a built-in fallback module root; repository-local `repo//luaclass/` modules resolve first so repositories can override or extend the shipped defaults without generated repositories owning shared module files. - **Autogen pipeline**: a getter operation and repository-level `.metadata/autogen/` Lua helper that previews and writes package directories/version scripts, Manifests, optional package-local `files/` helper data, and a package-local `.autogen.jsonc` generation record from structured inputs. Autogen output is ordinary repository content. - **Provider/source cache**: cache.db entries for upstream facts, API responses, indexes, freshness tokens, and parsed provider facts. - **Package metadata cache**: cache.db entries for normalized package metadata/candidates/artifacts produced by evaluating a package's Lua dependency closure. @@ -78,6 +78,15 @@ To avoid drifting back into the old hub-app model, ADR-0012 uses these terms str F-Droid is treated as a structured Android catalog provider endpoint. Its product support is autogen-first. +### Standard `luaclass` module resolution + +Package version Lua resolves `require("luaclass.")` in this order: + +1. the active package repository's `luaclass/` directory, e.g. `repo/official/luaclass/fdroid_android.lua`; then +2. getter-shipped built-in standard modules. + +Repository-local modules deliberately win over getter-shipped modules. This gives trusted repositories and `local` overlays a normal source-level override path while keeping generated repositories boring: generated package directories do not need to copy shared standard modules, and `.autogen.jsonc` remains package-local ownership proof rather than a repository-root ownership system. Built-in modules are part of the getter binary/source distribution and are not repository source files; repository trust/signing still applies only to repository-provided files. Cross-repository module lookup is not accepted in this slice because it would make package behavior depend on unrelated repository priority and trust boundaries. + ### F-Droid reusable module The common package-authoring API should be intentionally small. The default case should need only the Android/F-Droid package name: diff --git a/docs/lua-api/repository-layout.md b/docs/lua-api/repository-layout.md index 50c113d3..02de5707 100644 --- a/docs/lua-api/repository-layout.md +++ b/docs/lua-api/repository-layout.md @@ -84,6 +84,8 @@ Repository self-metadata lives at `.metadata/metadata.jsonc` and records schema Inside a repository alias directory, getter only considers explicit entries: reserved repository-root directories such as `.metadata/` and `luaclass/`, plus directory chains that form package paths. Reserved directories are handled only by their own responsibility and never participate in package discovery; package paths cannot begin with reserved names such as `.metadata` or `luaclass`. Future repository-root reserved directories follow the same rule so the repo layout remains organizable. A directory that directly contains `metadata.jsonc` declares a package boundary. `.autogen.jsonc` does not declare a package boundary; it is only a generated-package ownership record inside a package directory that already has `metadata.jsonc`. If `metadata.jsonc` parses correctly as package metadata, the directory is a valid package directory; if parsing fails, getter reports an invalid package metadata diagnostic for that package path. In both cases, that directory is the package path endpoint and getter does not discover nested packages below it. Other files or directories such as `README.md`, `docs/`, or random helper files are outside getter domain entirely: they are not parsed, validated, displayed, warned about, or modeled as ignored managed objects. The primary reason is clear responsibility boundaries; a smaller getter-core attack surface is a beneficial side effect. +Package version Lua resolves `require("luaclass.")` from the active package repository's `luaclass/` directory first, then from getter-shipped built-in standard modules. Repository-local modules intentionally override built-in standard modules. Generated repositories do not need to copy shared standard modules into repository-root `luaclass/`, and getter does not introduce repo-level autogen ownership records for shared modules; `.autogen.jsonc` remains package-local. + ## Package directories Package directories are final package definitions consumed by getter. UpgradeAll/getter domain strings, including package paths and aliases, are treated as UTF-8. Getter does not detect or convert other filesystem/text encodings; inputs in other encodings are still interpreted as UTF-8. @@ -199,7 +201,9 @@ Reusable Lua modules. These are conceptually similar to eclasses but are plain L local github_android = require("luaclass.github_android_apk") ``` -Cross-repository imports may resolve by priority when no repository alias is specified, or by explicit local alias when an author intentionally depends on one. Explicit alias imports break if the user renames that alias, preserving the user's ability to replace a repository layer intentionally. +Package version Lua resolves `require("luaclass.")` from the active package repository's `luaclass/` directory first, then from getter-shipped built-in standard modules. Repository-local modules intentionally override built-in standard modules, so a trusted repository or `local` overlay can replace the shipped default behavior in normal source form. + +Cross-repository `luaclass` imports are not supported in this model. A package in `repo/official` does not load modules from `repo/local`, `repo/autogen`, or another alias by priority or by explicit alias. If shared behavior is needed for generated packages, it should either live in getter-shipped built-in modules or be copied/authored into the active repository's own `luaclass/` tree. ## Getter hook scripts From b8e929ad67e80c62ffecc8ee49a5cd1b21cc02a3 Mon Sep 17 00:00:00 2001 From: Xiangzhe Date: Sat, 27 Jun 2026 22:40:56 +0800 Subject: [PATCH 71/85] feat(getter): add Lua HTTP host seam --- core-getter/src/main/rust/getter | 2 +- .../adr/0001-app-centric-lua-package-repository-model.md | 2 +- docs/architecture/adr/0005-lua-package-api.md | 6 ++++-- .../adr/0010-package-metadata-cache-and-version-baseline.md | 2 +- ...012-getter-owned-provider-modules-and-autogen-refresh.md | 4 ++-- docs/architecture/upgradeall-getter-rewrite-wiki.md | 4 ++-- docs/lua-api/permissions.md | 6 +++--- docs/lua-api/repository-layout.md | 4 ++-- 8 files changed, 16 insertions(+), 14 deletions(-) diff --git a/core-getter/src/main/rust/getter b/core-getter/src/main/rust/getter index 517bbcf4..759e7b8e 160000 --- a/core-getter/src/main/rust/getter +++ b/core-getter/src/main/rust/getter @@ -1 +1 @@ -Subproject commit 517bbcf43d4244b93bf8b0f49077905aaf3b630f +Subproject commit 759e7b8e1269b851383d57d2fea93916f2d161f7 diff --git a/docs/architecture/adr/0001-app-centric-lua-package-repository-model.md b/docs/architecture/adr/0001-app-centric-lua-package-repository-model.md index d707bc4b..9a2db570 100644 --- a/docs/architecture/adr/0001-app-centric-lua-package-repository-model.md +++ b/docs/architecture/adr/0001-app-centric-lua-package-repository-model.md @@ -99,7 +99,7 @@ Package `metadata.jsonc` declares permissions per enabled Lua file, for example `.metadata/autogen/` contains repository-level autogen metadata and scripts. Autogen scripts may use `luaclass/` helpers and can generate or refresh package directories from structured upstream inputs. Each generated package directory stores its own getter-managed generation record as `.autogen.jsonc`, listing generated files, file hashes, generator/template identity, and input facts needed to decide whether the package can be refreshed or cleaned. Hashes recorded in `.autogen.jsonc` are generated-output ownership/tamper-detection facts only: they answer whether a file is still the file getter generated earlier, not whether it is trusted, repository-signed, or valid as external-download content. The `.autogen.jsonc` `files` map covers getter-written generated output such as `metadata.jsonc`, `Manifest`, generated Lua scripts, and generated `files/...` helper files; it does not include `.autogen.jsonc` itself, avoiding self-referential hashing. The generated repository is generated output: getter may overwrite package directories it previously generated only when matching `.autogen.jsonc` proves ownership. When refresh/overwrite ownership checks pass, getter clears the existing generated package directory contents, then writes the new generated contents into the same package directory, without preserving old unlisted extra files. If clearing any old file or subdirectory fails, the whole refresh/overwrite fails rather than being ignored. If writing new generated contents fails after clearing, the operation fails directly without rollback; the directory may be empty or partially written, and the next refresh continues by clearing and rewriting again. If a target package directory exists without a matching generation record, apply reports a conflict and does not overwrite it. A generated-repo package directory missing `.autogen.jsonc` is a conflict rather than something getter automatically claims. If `.autogen.jsonc` exists but is malformed or schema-invalid, ordinary package discovery/evaluation is still decided by `metadata.jsonc`, but ownership-dependent autogen refresh/apply/cleanup/overwrite reports a conflict and does not auto-fix, overwrite, or delete it. When cleanup ownership checks pass, cleanup clears the generated package directory contents directly, including `.autogen.jsonc` and any unlisted extra files inside it, but does not delete the package directory itself. If clearing any file or subdirectory fails, the whole cleanup/update fails rather than being ignored. Getter does not classify or preserve unlisted extra files in generated package directories because they are outside getter's domain; direct directory-content clearing is simpler and more stable for generated output. User-authored overrides belong in `repo/local/...`, not by hand-editing `repo/autogen/...`. -Getter preserves user-controlled transparent URL replacement through local hook scripts under `rc/hook/`, analogous to an emerge bashrc-style hook and UpgradeAll's older URL replacement behavior. Hooks are getter-local runtime policy discovered only from the filesystem: list enabled `rc/hook/*.lua`, exclude dot-prefixed basenames, sort deterministically, and load before package version scripts, repository-level autogen scripts, and `luaclass/` code. There is no hook registry, metadata map, or disabled-hook state. Dot-prefixed Lua files are excluded from hook Lua discovery and are not hook entries. Hook scripts can wrap getter-exposed Lua host entrypoints such as `http_get()` or `read_package_file()` and call the original unhooked getter-internal entrypoint through `getter_builtin.`, for example `getter_builtin.http_get()` or `getter_builtin.read_package_file()`, after rewriting the URL or applying local policy. Getter core/CLI does not maintain a protective denylist of hookable public functions; if extra guardrails are needed, they belong in UI/UX policy rather than the getter core. `getter_builtin.*` is an internal escape hatch for hook code; ordinary package/autogen Lua should use the public hooked names instead. Hooks are an execution overlay and must not mutate repository source files. Hook loading is fail-closed for enabled hooks: parse/load/initialization failure fails the current Lua execution instead of silently falling back to unhooked functions. URL rewrites can point requests at mirrors, proxies, or local replacement endpoints, but for package version scripts without `allow_free_network` the returned body still must match a package `Manifest` hash. +Getter preserves user-controlled transparent URL replacement through local hook scripts under `rc/hook/`, analogous to an emerge bashrc-style hook and UpgradeAll's older URL replacement behavior. Hooks are getter-local runtime policy discovered only from the filesystem: list enabled `rc/hook/*.lua`, exclude dot-prefixed basenames, sort deterministically, and load before package version scripts, repository-level autogen scripts, and `luaclass/` code. There is no hook registry, metadata map, or disabled-hook state. Dot-prefixed Lua files are excluded from hook Lua discovery and are not hook entries. Hook scripts can wrap getter-exposed Lua host entrypoints such as `http_get()` or `read_package_file()` and call the original unhooked getter-internal entrypoint through `getter_builtin.`, for example `getter_builtin.http_get()` or `getter_builtin.read_package_file()`, after rewriting the URL or applying local policy. Plain package evaluation does not install `http_get`; provider/runtime operations that need network access must deliberately install an HTTP transport and own permission, Manifest, provider, cache, and diagnostic policy for that execution. Getter core/CLI does not maintain a protective denylist of hookable public functions; if extra guardrails are needed, they belong in UI/UX policy rather than the getter core. `getter_builtin.*` is an internal escape hatch for hook code; ordinary package/autogen Lua should use the public hooked names instead. Hooks are an execution overlay and must not mutate repository source files. Hook loading is fail-closed for enabled hooks: parse/load/initialization failure fails the current Lua execution instead of silently falling back to unhooked functions. URL rewrites can point requests at mirrors, proxies, or local replacement endpoints, but for package version scripts without `allow_free_network` the returned body still must match a package `Manifest` hash. ## Repository priority diff --git a/docs/architecture/adr/0005-lua-package-api.md b/docs/architecture/adr/0005-lua-package-api.md index 169a47a7..504bcda8 100644 --- a/docs/architecture/adr/0005-lua-package-api.md +++ b/docs/architecture/adr/0005-lua-package-api.md @@ -45,9 +45,11 @@ post_update ## Network permission model -Lua has no native or standard-library network API by default. Network access, when allowed by the package/provider mode, goes through getter host APIs such as `http_get(url, headers = ..., cache = true|false, ...)`. +Lua has no native or standard-library network API by default. Network access, when allowed by the package/provider mode, goes through getter host APIs such as `http_get(url, { headers = ..., cache = true|false })`. -`cache` defaults to `false`. Passing `cache = true` opts that HTTP request into getter-owned provider/source caching; Lua chooses cache participation per request, but getter owns cache keys, persistence, revalidation, stale diagnostics, and secret redaction. +Plain package evaluation does not install `http_get` by default. The getter operation/runtime that evaluates a provider-backed script must deliberately install an HTTP transport and own permission checks, Manifest validation, provider policy, cache behavior, and diagnostics for that execution. + +`cache` defaults to `false`. Passing `cache = true` opts that HTTP request into getter-owned provider/source caching; Lua chooses cache participation per request, but getter owns cache keys, persistence, revalidation, stale diagnostics, and secret redaction. The initial stable request shape is deliberately small: URL string plus an optional options table containing only string-to-string `headers` and boolean `cache`. Free network permission is declared per enabled Lua script in package `metadata.jsonc` using a filename-keyed map, for example `lua: { "9999.lua": { permission: ["allow_free_network"] } }`. The `lua` map is lookup-only: getter first discovers an enabled Lua file from the filesystem, then queries this map by basename. Getter does not enumerate the map to discover scripts or warnings. Entries for nonexistent files or dot-prefixed Lua files are inert. The permission can apply to `9999.lua` or to a fixed-version script. `9999.lua` commonly needs free network, but the filename alone does not grant the permission or force the warning if metadata does not declare it. A version script omitted from the `lua` map defaults to `permission: []`. diff --git a/docs/architecture/adr/0010-package-metadata-cache-and-version-baseline.md b/docs/architecture/adr/0010-package-metadata-cache-and-version-baseline.md index 22d32897..f5c26be4 100644 --- a/docs/architecture/adr/0010-package-metadata-cache-and-version-baseline.md +++ b/docs/architecture/adr/0010-package-metadata-cache-and-version-baseline.md @@ -14,7 +14,7 @@ The runtime caches software metadata produced by running package Lua/provider lo The cache model has two layers: -1. **Provider/source cache**: getter host API responses keyed by provider id, request parameters, executor/cache policy, auth/permission mode, and other provider-context inputs. Lua/provider modules opt individual HTTP host API calls into this cache explicitly, e.g. `http_get(url, headers = ..., cache = true)`, while the default is `cache = false`. +1. **Provider/source cache**: getter host API responses keyed by provider id, request parameters, executor/cache policy, auth/permission mode, and other provider-context inputs. Lua/provider modules opt individual HTTP host API calls into this cache explicitly, e.g. `http_get(url, { headers = ..., cache = true })`, while the default is `cache = false`. Plain package evaluation does not install HTTP by default; provider/runtime operations deliberately install it when they own the execution policy. 2. **Package metadata cache**: normalized package metadata produced by Lua/package logic from provider/source data. Package metadata cache entries are persisted in `cache.db` from the first runtime implementation. diff --git a/docs/architecture/adr/0012-getter-owned-provider-modules-and-autogen-refresh.md b/docs/architecture/adr/0012-getter-owned-provider-modules-and-autogen-refresh.md index 2230dc8e..098238fa 100644 --- a/docs/architecture/adr/0012-getter-owned-provider-modules-and-autogen-refresh.md +++ b/docs/architecture/adr/0012-getter-owned-provider-modules-and-autogen-refresh.md @@ -42,7 +42,7 @@ F-Droid and GitHub are providers/sources/backends, not package identities and no 1. Getter owns live provider execution, provider/source caching, package metadata normalization, update selection, action issuance, and autogen/package-path decisions. 2. Reusable Lua provider modules/classes under `luaclass/` provide high-level package-authoring APIs for common provider families. -3. Those Lua modules call getter-owned host APIs by default. HTTP access is exposed as a getter-managed host function such as `http_get(url, headers = ..., cache = true|false, ...)`; `cache` defaults to `false`, and Lua/provider modules opt individual requests into HTTP source caching by passing `cache = true`. This remains getter-owned network/cache execution, not Flutter/Kotlin HTTP and not a Lua standard-library network primitive. +3. Those Lua modules call getter-owned host APIs by default. HTTP access is exposed as a getter-managed host function such as `http_get(url, { headers = ..., cache = true|false })`; `cache` defaults to `false`, and Lua/provider modules opt individual requests into HTTP source caching by passing `cache = true`. Plain package evaluation does not install `http_get`; the getter operation/runtime evaluating provider-backed Lua deliberately installs the transport and owns permission, Manifest, provider, cache, and diagnostic policy for that execution. This remains getter-owned network/cache execution, not Flutter/Kotlin HTTP and not a Lua standard-library network primitive. 4. F-Droid support is **autogen-first**: - an F-Droid app is represented as an ordinary package directory with metadata and version scripts; - explicit user selection of an F-Droid app uses a getter autogen preview/apply operation that generates a package directory/version script; @@ -272,7 +272,7 @@ local index = http_get(fdroid_index_url, { }) ``` -`cache = false` is the default so ordinary one-off HTTP calls do not silently become durable provider cache. When `cache = true`, getter owns cache key construction, storage, revalidation, stale diagnostics, and secret redaction; Lua chooses that the request should participate in HTTP/source caching but does not write cache entries itself. +`cache = false` is the default so ordinary one-off HTTP calls do not silently become durable provider cache. The v1 host request shape is intentionally narrow: a URL string plus an optional options table with string-to-string `headers` and boolean `cache`; unsupported options are rejected rather than silently accepted. When `cache = true`, getter owns cache key construction, storage, revalidation, stale diagnostics, and secret redaction; Lua chooses that the request should participate in HTTP/source caching but does not write cache entries itself. F-Droid provider cache keys must include inputs such as: diff --git a/docs/architecture/upgradeall-getter-rewrite-wiki.md b/docs/architecture/upgradeall-getter-rewrite-wiki.md index f216b849..12013856 100644 --- a/docs/architecture/upgradeall-getter-rewrite-wiki.md +++ b/docs/architecture/upgradeall-getter-rewrite-wiki.md @@ -842,7 +842,7 @@ local body = http_get(url, { }) ``` -`cache` 默认是 `false`。Lua/provider module 通过 `cache = true` 主动把单次 HTTP 请求纳入 getter-owned HTTP/source cache；getter 负责 cache key、持久化、revalidation、stale diagnostics 和 secret redaction，Lua 只表达该请求是否应缓存。 +`cache` 默认是 `false`。普通 package evaluation 不默认安装 `http_get`；需要 provider/network 的 getter operation/runtime 必须显式安装 transport，并由 getter 拥有 permission、Manifest、provider、cache、diagnostic policy。Lua/provider module 通过 `cache = true` 主动把单次 HTTP 请求纳入 getter-owned HTTP/source cache；getter 负责 cache key、持久化、revalidation、stale diagnostics 和 secret redaction，Lua 只表达该请求是否应缓存。v1 请求形状保持很小：URL string，加可选 options table，其中只接受 string-to-string `headers` 与 boolean `cache`。 标准 provider module/class 可以在声明的 provider/source 语义下使用该 host HTTP API 获取 release/catalog 信息。 @@ -944,7 +944,7 @@ Android 上 repo sync 可以先采用 archive zip/tar 或 bundled repo snapshot - 可按 package/repository scope 区分。 - 参考 emerge bashrc 的精神：全局 hook 根据上下文做调整。 -Accepted hook location is top-level runtime config, `rc/hook/*.lua`. Hooks wrap public getter host functions and call original unhooked entrypoints through `getter_builtin.`. +Accepted hook location is top-level runtime config, `rc/hook/*.lua`. Hooks wrap public getter host functions and call original unhooked entrypoints through `getter_builtin.`. Plain package evaluation does not install `http_get`; provider/runtime operations that need network install it deliberately and own permission, Manifest, provider, cache, and diagnostic policy. 示例： diff --git a/docs/lua-api/permissions.md b/docs/lua-api/permissions.md index a1330be1..ca641e94 100644 --- a/docs/lua-api/permissions.md +++ b/docs/lua-api/permissions.md @@ -6,9 +6,9 @@ ## Default -Lua package scripts do not receive Lua-native or Flutter/Kotlin-owned network access by default. +Lua package scripts do not receive Lua-native or Flutter/Kotlin-owned network access by default. Plain package evaluation does not install an HTTP function. -They can use getter-provided provider/source host APIs. HTTP requests go through getter-managed functions such as: +Provider-backed getter operations may deliberately install getter-provided provider/source host APIs for that execution. HTTP requests go through getter-managed functions such as: ```lua local body = http_get(url, { @@ -17,7 +17,7 @@ local body = http_get(url, { }) ``` -`cache` defaults to `false`. Passing `cache = true` opts that request into getter-owned provider/source caching; getter owns cache keys, persistence, revalidation, stale diagnostics, permissions, and secret redaction. +`cache` defaults to `false`. Passing `cache = true` opts that request into getter-owned provider/source caching; getter owns cache keys, persistence, revalidation, stale diagnostics, permissions, and secret redaction. The v1 request shape is intentionally narrow: a URL string plus an optional options table with string-to-string `headers` and boolean `cache`; unsupported options should be rejected rather than silently accepted. ## Free network permission diff --git a/docs/lua-api/repository-layout.md b/docs/lua-api/repository-layout.md index 02de5707..179ef030 100644 --- a/docs/lua-api/repository-layout.md +++ b/docs/lua-api/repository-layout.md @@ -74,7 +74,7 @@ The user may change priority through UI, CLI, or this getter-owned metadata file ## Runtime hooks -Runtime/local hooks live under `rc/hook/`, not under `repo/`. Hooks are runtime policy, while `repo/metadata.jsonc` stays repository-related configuration. Hooks are discovered only from the filesystem: getter lists enabled `rc/hook/*.lua` files, excludes basenames starting with `.`, sorts them deterministically, then loads them before every Lua execution environment. There is no hook registry, metadata map, or persistent disabled-hook state. Enabled hooks can wrap getter-exposed Lua host functions such as `http_get()` for transparent URL replacement or similar local policy. A dot-prefixed Lua file is outside getter management and is not a hook entry. +Runtime/local hooks live under `rc/hook/`, not under `repo/`. Hooks are runtime policy, while `repo/metadata.jsonc` stays repository-related configuration. Hooks are discovered only from the filesystem: getter lists enabled `rc/hook/*.lua` files, excludes basenames starting with `.`, sorts them deterministically, then loads them before every Lua execution environment. There is no hook registry, metadata map, or persistent disabled-hook state. Enabled hooks can wrap getter-exposed Lua host functions such as `http_get()` for transparent URL replacement or similar local policy when the active getter operation installs that function; plain package evaluation does not install HTTP by default. A dot-prefixed Lua file is outside getter management and is not a hook entry. ## Repository alias @@ -211,7 +211,7 @@ Getter preserves user-controlled transparent URL replacement through local hook Hooks are global getter-local runtime policy. Getter discovers hooks only from the filesystem: list enabled `rc/hook/*.lua` files, exclude basenames starting with `.`, sort deterministically, then load before every Lua execution environment. For example, enabled files load as `00-env.lua`, `10-http-rewrite.lua`, then `20-headers.lua`. A Lua file whose basename starts with `.` is excluded from hook Lua discovery, so `.10-http-rewrite.lua` is not parsed, validated, loaded, displayed, or treated as a hook entry. There is no hook registry, metadata map, or disabled-hook state. -A hook can wrap visible Lua host functions such as `http_get()` or `read_package_file()` and call the original getter-internal entrypoint after rewriting the URL or applying local policy. Getter core/CLI does not maintain a protective denylist of hookable public functions; if extra guardrails are needed, they belong in UI/UX policy rather than the getter core. Original unhooked host entrypoints are available to hook code as `getter_builtin.`, for example: +A hook can wrap visible Lua host functions such as `http_get()` or `read_package_file()` and call the original getter-internal entrypoint after rewriting the URL or applying local policy. Plain package evaluation does not install `http_get`; provider/runtime operations that need network access deliberately install the HTTP transport and own permission, Manifest, provider, cache, and diagnostic policy for that execution. Getter core/CLI does not maintain a protective denylist of hookable public functions; if extra guardrails are needed, they belong in UI/UX policy rather than the getter core. Original unhooked host entrypoints are available to hook code as `getter_builtin.`, for example: ```lua local upstream_http_get = getter_builtin.http_get From 64e2fb422dbd28b3c65ae5d6ac48ba9e8513608f Mon Sep 17 00:00:00 2001 From: Xiangzhe Date: Sat, 27 Jun 2026 23:25:45 +0800 Subject: [PATCH 72/85] feat(getter): add dev provider luaclass modules --- core-getter/src/main/rust/getter | 2 +- .../adr/0001-app-centric-lua-package-repository-model.md | 2 +- .../0012-getter-owned-provider-modules-and-autogen-refresh.md | 2 ++ 3 files changed, 4 insertions(+), 2 deletions(-) diff --git a/core-getter/src/main/rust/getter b/core-getter/src/main/rust/getter index 759e7b8e..939c6de0 160000 --- a/core-getter/src/main/rust/getter +++ b/core-getter/src/main/rust/getter @@ -1 +1 @@ -Subproject commit 759e7b8e1269b851383d57d2fea93916f2d161f7 +Subproject commit 939c6de0be97b8f0ace143e53e6d3ff9dc76e30e diff --git a/docs/architecture/adr/0001-app-centric-lua-package-repository-model.md b/docs/architecture/adr/0001-app-centric-lua-package-repository-model.md index 9a2db570..fd118e63 100644 --- a/docs/architecture/adr/0001-app-centric-lua-package-repository-model.md +++ b/docs/architecture/adr/0001-app-centric-lua-package-repository-model.md @@ -129,7 +129,7 @@ Reusable Lua modules should use Lua import helpers where practical: local github_android = require("luaclass.github_android_apk") ``` -Cross-repository imports may resolve by repository priority when no repository alias is specified, or by explicit local alias when the author writes one. Explicit alias imports intentionally depend on the user's local repository name; if the user renames `repo/official` to `repo/a`, imports that explicitly name `official` are considered intentionally broken by that rename. This is a feature: users can intentionally replace, fork, or interpose repository layers instead of having every package become fully self-maintained. +`luaclass.*` imports resolve only from the active package repository's own `luaclass/` directory and then from getter-shipped built-in fallback modules. Cross-repository `luaclass` lookup is intentionally unsupported: a package in `repo/official` does not load modules from `repo/local`, `repo/autogen`, or another alias by priority or by explicit alias. Shared behavior needed by generated packages should either live in getter-shipped built-ins or be authored/copied into that package's active repository. Parent package imports should use package atoms rather than raw file paths: diff --git a/docs/architecture/adr/0012-getter-owned-provider-modules-and-autogen-refresh.md b/docs/architecture/adr/0012-getter-owned-provider-modules-and-autogen-refresh.md index 098238fa..dd5be78a 100644 --- a/docs/architecture/adr/0012-getter-owned-provider-modules-and-autogen-refresh.md +++ b/docs/architecture/adr/0012-getter-owned-provider-modules-and-autogen-refresh.md @@ -87,6 +87,8 @@ Package version Lua resolves `require("luaclass.")` in this order: Repository-local modules deliberately win over getter-shipped modules. This gives trusted repositories and `local` overlays a normal source-level override path while keeping generated repositories boring: generated package directories do not need to copy shared standard modules, and `.autogen.jsonc` remains package-local ownership proof rather than a repository-root ownership system. Built-in modules are part of the getter binary/source distribution and are not repository source files; repository trust/signing still applies only to repository-provided files. Cross-repository module lookup is not accepted in this slice because it would make package behavior depend on unrelated repository priority and trust boundaries. +Implementation status: only the neutral `luaclass.android` helper is an always-on getter-shipped builtin today. Provider-named fallback modules such as `luaclass.fdroid_android` and `luaclass.github_android_apk` are currently dev-gated tracer modules over `getter_dev.*` host calls. They exist to validate provider-host seams and builtin fallback distribution, not to publish a stable product Lua provider API. Generated F-Droid output does not depend on them yet; promoting provider modules to an always-on public authoring surface requires a later stable host-API decision. + ### F-Droid reusable module The common package-authoring API should be intentionally small. The default case should need only the Android/F-Droid package name: From dd1c25953b80e1f681e921d6f4b735ceabc3722e Mon Sep 17 00:00:00 2001 From: Xiangzhe Date: Sun, 28 Jun 2026 00:26:15 +0800 Subject: [PATCH 73/85] feat(getter): add Lua HTTP policy harness --- core-getter/src/main/rust/getter | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/core-getter/src/main/rust/getter b/core-getter/src/main/rust/getter index 939c6de0..03e81cc7 160000 --- a/core-getter/src/main/rust/getter +++ b/core-getter/src/main/rust/getter @@ -1 +1 @@ -Subproject commit 939c6de0be97b8f0ace143e53e6d3ff9dc76e30e +Subproject commit 03e81cc7b413b424c486f349f86358c2e19d3ba8 From 8942c7fc723dfe228e408d30c1311f4bc28c97f6 Mon Sep 17 00:00:00 2001 From: Xiangzhe Date: Sun, 28 Jun 2026 01:01:35 +0800 Subject: [PATCH 74/85] feat(getter): add runtime hook harness --- core-getter/src/main/rust/getter | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/core-getter/src/main/rust/getter b/core-getter/src/main/rust/getter index 03e81cc7..25c510ac 160000 --- a/core-getter/src/main/rust/getter +++ b/core-getter/src/main/rust/getter @@ -1 +1 @@ -Subproject commit 03e81cc7b413b424c486f349f86358c2e19d3ba8 +Subproject commit 25c510acdeea043095bb44951b63bdce88bd76e5 From 19f2fd998a8005a95733d041ec2cf916a9de872a Mon Sep 17 00:00:00 2001 From: Xiangzhe Date: Sun, 28 Jun 2026 09:40:05 +0800 Subject: [PATCH 75/85] docs(architecture): design provider host API v1 --- CONTEXT.md | 2 +- PLAN/provider-host-api-v1.md | 477 ++++++++++++++++++ ...ed-provider-modules-and-autogen-refresh.md | 35 +- 3 files changed, 507 insertions(+), 7 deletions(-) create mode 100644 PLAN/provider-host-api-v1.md diff --git a/CONTEXT.md b/CONTEXT.md index a2566366..f95180eb 100644 --- a/CONTEXT.md +++ b/CONTEXT.md @@ -112,7 +112,7 @@ A getter-owned notification boundary used by native/Flutter UI to learn that run ### Provider host API -A getter-owned API exposed to package Lua for requesting provider/source data. Lua packages should call getter host APIs rather than using any Lua-native network library or Flutter/Kotlin HTTP path. HTTP is exposed as a getter-managed host function such as `http_get(url, headers = ..., cache = true|false, ...)`; `cache` defaults to `false`, and Lua/provider modules actively opt a request into HTTP/source caching with `cache = true`. Getter owns execution, cache key construction, persistence, revalidation, stale diagnostics, permissions, secret redaction, and output validation. The provider executor behind the host API can be fake during early runtime development and live later. +A getter-owned API exposed to package Lua for requesting provider/source data. Lua packages should call getter host APIs rather than using any Lua-native network library or Flutter/Kotlin HTTP path. Getter-shipped standard provider modules use provider-specific host functions under `getter.provider.*`, such as `getter.provider.fdroid.update_candidates(...)` and `getter.provider.github.release_candidates(...)`, so Rust owns provider parsing, cache consistency, diagnostics, and candidate normalization. Generic/custom HTTP remains exposed as a getter-managed host function such as `http_get(url, { headers = ..., cache = true|false })`; `cache` defaults to `false`, and Lua actively opts a generic request into HTTP/source caching with `cache = true`. Getter owns execution, cache key construction, persistence, revalidation, stale diagnostics, permissions, secret redaction, and output validation. The provider executor behind the host API can be fake during early runtime development and live later. ### Package metadata cache diff --git a/PLAN/provider-host-api-v1.md b/PLAN/provider-host-api-v1.md new file mode 100644 index 00000000..0eadb309 --- /dev/null +++ b/PLAN/provider-host-api-v1.md @@ -0,0 +1,477 @@ +# PLAN: Stable Lua provider host API v1 + +> Status: reviewed design; ready for the first fixture-backed implementation slice +> Scope: ADR-0012 provider-host API design for F-Droid and GitHub standard Lua modules +> Non-code slice: document and review before promoting dev tracers or changing generated F-Droid output + +## Progress + +- [x] Consume prior-context brief and temporary context-builder handoff. +- [x] Re-check current docs/code seams relevant to provider host APIs. +- [x] Draft v1 API boundary, names, cache/HTTP/hook/Manifest relationships, and non-goals. +- [x] Run read-only oracle/reviewer review of this design. +- [x] Revise this plan and copy durable decisions into ADR-0012 / glossary docs. +- [ ] Next slice: start a narrow TDD implementation of the fixture-backed stable namespace harness. + +## Current evidence and constraints + +- `getter-core` owns constrained Lua evaluation, package/module resolution, JSON-like table conversion, schema/domain validation, `read_package_file`, and the generic `http_get` host seam. It must not depend on provider/cache/storage crates. +- `getter-operations` owns provider/cache orchestration today (`fdroid_catalog`, `github_releases`, `github_latest_commit`, `provider_cache`) and currently hosts the dev-only Lua provider tracer (`lua_provider_host`). +- Plain package evaluation does **not** install `http_get` or provider APIs. Provider-backed operations install host functions deliberately. +- `http_get(url, { headers = ..., cache = true|false })` is the accepted narrow generic HTTP request shape; `cache` defaults false, and unsupported options are rejected. +- Runtime hooks live under `/rc/hook/*.lua`, load after host functions are installed in the current operation harness, and call originals through `getter_builtin.*`. +- Package `Manifest` constrains external network/dynamic response bodies for scripts without `allow_free_network`; missing `Manifest` means an empty allow-list. Hooks and cache must not bypass this. +- Current provider-named modules `luaclass.fdroid_android` and `luaclass.github_android_apk` are dev-gated tracers over `getter_dev.*`. They validate distribution and authoring shape, not a stable product API. +- Repository-local `luaclass/` modules must continue to override getter-shipped built-ins; cross-repository `luaclass` lookup stays unsupported. + +## Design goals + +1. Give standard provider modules a stable host namespace that is not `getter_dev.*`. +2. Keep Rust getter/provider operations as the owner of F-Droid/GitHub parsing, cache refresh, diagnostics, and candidate normalization. +3. Keep `getter-core` provider-agnostic: it may ship Lua source that calls a named host API, but the Rust host implementation is installed by operations. +4. Preserve user hookability without adding protective denylists: stable host functions that are public to package Lua should also have `getter_builtin.*` originals for hooks. +5. Keep v1 small enough to test with fixture-backed transports before live HTTP/auth/UI exposure. +6. Avoid forcing generated F-Droid output to depend on provider modules until the stable host API and validation story are tested. + +## Stable Lua namespace + +### Stable public namespace + +Use one stable table rooted at `getter`: + +```lua +getter.provider.fdroid.update_candidates(spec) +getter.provider.github.release_candidates(spec) +-- reserved for an explicit live/floating operation, not installed by the default release module: +getter.provider.github.latest_commit(spec) +``` + +The first installed v1 candidate APIs are F-Droid `update_candidates` and GitHub `release_candidates`. The GitHub `latest_commit` shape is reserved here because ADR-0012 names live/latest-commit behavior, but it should not be installed by the default release-check module or treated as ordinary versioned update behavior until live package semantics are implemented. + +Rationale: + +- `getter` clearly marks host-owned functionality, unlike free globals such as `fdroid_update_candidates`. +- `provider` groups provider-specific host APIs and leaves room for non-provider getter APIs later. +- Provider and function names are explicit enough for hooks and diagnostics. +- The dev-only `getter_dev.*` namespace remains private and should not appear in stable package docs. + +### Hook originals + +When an operation installs the v1 provider host, it installs both public and original paths: + +```lua +getter.provider.fdroid.update_candidates = +getter_builtin.provider.fdroid.update_candidates = + +getter.provider.github.release_candidates = +getter_builtin.provider.github.release_candidates = + +-- when a live/latest-commit operation installs the reserved function: +getter.provider.github.latest_commit = +getter_builtin.provider.github.latest_commit = +``` + +Hooks may wrap provider functions the same way they wrap `http_get` or `read_package_file`: + +```lua +local upstream = getter_builtin.provider.github.release_candidates + +function getter.provider.github.release_candidates(spec) + spec = shallow_copy(spec) + spec.endpoint_id = spec.endpoint_id or "github-mirror" + return upstream(spec) +end +``` + +Ordinary package/version Lua should call the public `getter.provider.*` functions, usually through `luaclass.*`, and should not depend on `getter_builtin.*`. + +### Generic HTTP remains separate + +`http_get(url, { headers = ..., cache = true|false })` remains the generic HTTP host seam. + +This v1 plan intentionally clarifies ADR-0012's earlier broad wording that Lua/provider modules opt individual HTTP requests into cache with `http_get(cache = true)`: generic/custom Lua HTTP still uses `http_get`, but the getter-shipped standard F-Droid/GitHub modules should use provider-specific Rust host functions. Those host functions opt into provider/source cache through getter operation policy rather than by making Lua parse provider payloads. + +Standard F-Droid/GitHub modules should **not** parse provider payloads in Lua by calling `http_get` directly. They should call provider-specific host functions so Rust owns parsing, cache consistency, diagnostics, and normalization. + +`http_get` hooks affect Lua code that calls `http_get`. Provider-specific host functions are separate hook seams; v1 should not promise that a Lua wrapper around `http_get` intercepts Rust-internal provider requests. If a user wants to rewrite a provider endpoint in v1, they wrap `getter.provider..*` or configure the provider endpoint when that configuration exists. + +## Provider function inputs + +Package Lua passes source coordinates and package-authored selection rules. Operation context supplies cache mode, fixture/live transport, credentials, endpoint config, and refresh policy. + +Package Lua should not pass raw fixture JSON/XML or force-refresh mode in the stable API. Those remain test/operation request inputs. + +### F-Droid + +```lua +local result = getter.provider.fdroid.update_candidates { + package_name = "org.fdroid.fdroid", + endpoint_id = "official", -- optional; default is operation/getter default +} +``` + +Rules: + +- `package_name` is required and non-empty. +- `endpoint_id` is optional and resolves through getter/provider endpoint configuration; omitting it uses the default F-Droid endpoint, initially official F-Droid. +- `endpoint_url` is not a normal package authoring field in v1. Test fixtures may still inject endpoint URL through operation context. +- Channel/archive/anti-feature/localized metadata fields are deferred until their semantics are accepted. + +### GitHub releases + +```lua +local result = getter.provider.github.release_candidates { + owner = "f-droid", + repo = "fdroidclient", + asset = { + include = "[.]apk$", -- Rust regex syntax, not Lua pattern syntax + exclude = "debug", + }, + include_prereleases = false, + endpoint_id = "github", -- optional; default is operation/getter default +} +``` + +Rules: + +- `owner` and `repo` are required non-empty typed fields. +- The stable host input keeps typed `owner`/`repo`; any `"owner/name"` shorthand may exist in a Lua helper, but the host receives normalized typed fields. +- `asset.include` and `asset.exclude` are optional Rust `regex`-syntax filters interpreted by Rust provider code. Examples should use Rust regex forms such as `[.]apk$` or `\\.apk$`, not Lua pattern syntax such as `%.apk$`. +- `include_prereleases` defaults false. +- API base URL, auth identity, rate-limit mode, and live/fixture transport are operation/provider configuration, not package script fixture fields. + +### GitHub latest commit / live revision + +```lua +local result = getter.provider.github.latest_commit { + owner = "DUpdateSystem", + repo = "UpgradeAll", + ref = "HEAD", -- optional; default HEAD + endpoint_id = "github", -- optional +} +``` + +Rules: + +- This is a **live/floating** provider API, not a normal release-candidate API. +- The shape is reserved in the v1 design because ADR-0012 names GitHub latest-commit behavior and Rust already has a fixture-backed provider operation. +- The default v1 release-check host/module should not install or call `latest_commit`. A later explicit live operation/helper/module may install it and map the result into live package behavior only after the live-update UI/CLI semantics are ready. + +## Provider function result envelopes + +Provider host functions return structured envelopes. Standard `luaclass` modules map envelopes into the existing `package_version { updates = ... }` shape, while operations also capture provider call traces/diagnostics outside Lua for DTOs and cache metadata. + +### Candidate result envelope + +F-Droid `update_candidates` and GitHub `release_candidates` return: + +```lua +{ + -- Non-empty candidate sequence. Omit/set nil when there are no candidates; + -- do not return an empty Lua table here until the Lua JSON boundary has + -- explicit array support. + candidates = { + { + version = "1.20.0", + version_code = 1020000, -- optional + channel = "stable", -- optional + source = "fdroid", -- optional, provider id for display/diagnostics + artifacts = { + { + name = "apk", + url = "https://...", + file_name = "app.apk", -- optional + sha256 = "...", -- optional + size = 12345, -- optional bytes + }, + }, + }, + }, + source = "cache" | "refreshed" | "stale", + cache_key = "provider-cache-key", + diagnostics = { + { + code = "cache.refresh_failed", + message = "...", + provider = "fdroid" | "github", + cache_key = "provider-cache-key", -- optional when not cache-related + source = "stale", -- optional + stale_fetched_at_unix = 123, -- optional + -- provider-specific coordinates may be included: + endpoint_id = "official", + package_name = "org.fdroid.fdroid", + owner = "f-droid", + repo = "fdroidclient", + }, + }, +} +``` + +Zero-candidate provider results use `candidates = nil`/omitted plus diagnostics. This mirrors the current dev tracer and avoids the existing Lua JSON ambiguity where an empty Lua table serializes as `{}` rather than `[]`, which would fail the current `updates` array schema if passed through directly. + +The non-empty `candidates[]` item shape is intentionally the current `getter_core::UpdateCandidate` / `UpdateArtifact` shape. Richer provider-candidate fields from ADR-0012 (`published_at`, `changelog`, `metadata_digest`, etc.) are future extension fields, not required to stabilize v1. + +### Latest-commit result envelope + +GitHub `latest_commit` returns: + +```lua +{ + live = true, + version = "0123456", -- display/comparison string for live semantics + revision = "0123456789...", -- full commit id when known + source = "cache" | "refreshed" | "stale", + cache_key = "provider-cache-key", + latest_commit = { -- normalized Rust-owned live revision details + version = "0123456", + revision = "0123456789...", + html_url = "https://github.com/...", -- optional + message = "...", -- optional + }, + diagnostics = { ... }, +} +``` + +It should not be silently converted into ordinary versioned release candidates by the default GitHub APK helper. + +## Standard Lua modules + +### `luaclass.fdroid_android` + +Stable author API should stay the accepted small F-Droid shape: + +```lua +local fdroid = require("luaclass.fdroid_android") + +return fdroid.package { + package_name = "org.fdroid.fdroid", + endpoint_id = "official", -- optional +} +``` + +Implementation sketch: + +```lua +function fdroid.package(spec) + local result = getter.provider.fdroid.update_candidates { + package_name = require_string(spec, "package_name"), + endpoint_id = optional_string(spec, "endpoint_id"), + } + return package_version { + source_priority = { "fdroid" }, + -- nil when there are no candidates, so the final package omits `updates` + -- instead of returning an empty Lua table through the JSON boundary. + updates = result.candidates, + } +end +``` + +It should not require or duplicate display `name`; F-Droid catalog metadata is self-describing, and package identity is path-derived. + +### `luaclass.github_android_apk` + +Stable author API should stay close to the current dev tracer: + +```lua +local github_android = require("luaclass.github_android_apk") + +return github_android.package { + name = "F-Droid", + android_package = "org.fdroid.fdroid", + owner = "f-droid", + repo = "fdroidclient", + asset = { + include = "[.]apk$", -- Rust regex syntax, not Lua pattern syntax + exclude = "debug", + }, + include_prereleases = false, +} +``` + +Implementation sketch: + +```lua +function github_android.package(spec) + local result = getter.provider.github.release_candidates { + owner = require_string(spec, "owner"), + repo = require_string(spec, "repo"), + asset = optional_table(spec, "asset"), + include_prereleases = optional_boolean(spec, "include_prereleases"), + endpoint_id = optional_string(spec, "endpoint_id"), + } + local package = { + name = spec.name, + source_priority = { "github" }, + -- nil when there are no candidates, so the final package omits `updates` + -- instead of returning an empty Lua table through the JSON boundary. + updates = result.candidates, + } + if spec.android_package ~= nil then + package.installed = { + { kind = "android_package", package_name = require_string(spec, "android_package") }, + } + end + return package_version(package) +end +``` + +The module may later grow explicit live helpers, but release checks and latest-commit checks must stay semantically distinct. + +### Promotion path + +Do not promote the dev tracer modules directly as product API. The later implementation should: + +1. Add stable host installation in `getter-operations` while keeping `getter_dev.*` tests private or migrating them. +2. Rewrite the provider modules to call `getter.provider.*`, not `getter_dev.*`. +3. Add tests that default/plain package evaluation either does not resolve provider modules until promoted or fails with a stable host-unavailable diagnostic if a provider module is called without provider host installed. +4. Only then make provider modules always-on getter-shipped built-ins. +5. Only after that, consider changing generated F-Droid output to require `luaclass.fdroid_android`. + +## Cache and refresh semantics + +### Operation-owned cache mode + +The operation evaluating package Lua decides the cache mode: + +- normal update/read: use cached provider facts when valid, refresh on cache miss or stale according to provider policy; +- forced refresh: bypass cached reads for the requested provider/package scope, replace cache on success, and use stale cache only with explicit diagnostics on refresh failure. + +Lua package authors do not pass `mode = "force_refresh"` to stable provider host functions. + +### Provider cache contents + +Provider/source cache remains in `cache.db`. For standard provider functions, cache participation is operation-owned: Lua passes provider coordinates/selection rules to `getter.provider.*`, then Rust provider code decides which upstream request(s), parsed facts, freshness tokens, and cache entries are involved. Generic/custom Lua HTTP remains the path where Lua calls `http_get(..., { cache = true })` directly. + +Provider-specific operations may store parsed facts (current behavior) rather than raw HTTP bodies, but stable v1 must record enough provenance to enforce package Manifest rules for cache hits. At minimum, provider cache/provider-call metadata needs the source response body digest(s), provider/parser/cache schema version, and freshness tokens when available. + +Current fixture cache keys are acceptable for internal tracers, but a stable live implementation must make key inputs explicit: + +- provider id and provider operation/cache schema version; +- endpoint/API base URL digest and endpoint id; +- provider coordinates (`package_name`, `owner/repo`, `ref`, request type); +- parser/provider implementation version when interpretation changes; +- auth/rate-limit identity when auth exists, without storing secrets; +- freshness tokens such as ETag, Last-Modified, source timestamp, index revision, API cursor, or response digest when available. + +Auth/freshness-token design can be implemented incrementally, but the v1 docs must not claim cache keys are final until those inputs exist. + +### Diagnostics + +Stable provider diagnostics should use stable codes and getter-owned fields. Existing codes carry forward: + +- `cache.refresh_failed` +- `used_stale_cache` +- `provider.fdroid.package_not_found` +- `provider.github.asset_not_found` + +Required-field and invalid-shape errors are package authoring/schema errors and may fail Lua evaluation. Provider lookup misses and empty asset matches should return no `candidates` (`nil`/omitted) plus diagnostics, not crash the package evaluation. + +If refresh fails and no usable cache exists, the provider host call fails the operation with a provider diagnostic. If stale cache exists and the operation policy permits stale fallback, the result returns `source = "stale"`, candidates from stale facts, and explicit diagnostics. + +## Manifest and permission semantics + +Provider host calls are still package version script execution. Therefore: + +- If the current script lacks `allow_free_network`, every external response body used by a provider function must hash to a SHA-512 entry in that package's `Manifest`. +- Missing/empty `Manifest` means provider network/cache use cannot succeed for non-free scripts unless no external body is needed. +- A cache hit for parsed provider facts is usable in a non-free script only when the cache/provider-call provenance proves that the underlying source response body digest(s) are allowed by the current package `Manifest`. +- If provenance is missing, the cache entry is not usable for non-free package evaluation; getter must refetch and validate, or fail closed. +- If the script has `allow_free_network`, Manifest membership does not block provider responses, but diagnostics/high-risk UI still apply. + +This keeps provider-specific host calls from bypassing the `http_get`/Manifest policy merely because parsing moved into Rust. + +## Hook semantics + +For v1, hookability is function-level: + +- `http_get` remains a hook seam for Lua-authored generic HTTP. +- `read_package_file` remains a hook seam for package-local file reads. +- `getter.provider.fdroid.update_candidates` and `getter.provider.github.release_candidates` are v1 hook seams for provider-level local policy; the reserved `getter.provider.github.latest_commit` becomes a hook seam only when a later explicit live/latest-commit operation installs it. + +Operation install order should be: + +1. configure constrained Lua/package path; +2. install helper functions and getter host functions (`read_package_file`, `http_get` when applicable, provider APIs when applicable) plus `getter_builtin` originals; +3. load enabled `rc/hook/*.lua` fail-closed in deterministic order; +4. evaluate package/autogen Lua. + +Hooks are trusted local policy for advanced users. Do not add getter-core denylist behavior to prevent users from wrapping provider functions. + +## Error model + +- Invalid Lua host input (missing `package_name`, missing `owner`, invalid `asset` type) is an authoring/schema failure and should fail package evaluation with a stable diagnostic prefix/code when possible. +- Provider HTTP/parse/cache failures are provider operation failures. They should become getter-owned diagnostics; if no fallback exists, the update check fails rather than silently returning no updates. +- Provider miss/no matching artifact is a successful provider call with zero candidates and diagnostics. +- Hook load/runtime errors fail closed and fail the current Lua execution. +- `getter_dev.*` errors are not stable product errors and should not appear in public docs once v1 is implemented. + +## Crate boundary + +- `getter-core` may keep shipping Lua source files under `src/luaclass/*.lua`; those files are strings and can refer to stable host names without Rust linking to provider crates. +- `getter-core` must not implement or know F-Droid/GitHub provider host logic. +- `getter-operations` (or a new operations-owned submodule/crate if needed later) installs `getter.provider.*`, opens `cache.db`, applies provider cache/Manifest/hook policy, and calls `getter-providers` parsers. +- Flutter/Dart and Android/Kotlin remain adapters: they may call getter operations and render DTOs, but they must not parse provider payloads, generate Lua, map upstream IDs to package paths, decide cache invalidation, or perform provider HTTP. + +## Non-goals for v1 + +- No old Hub/app UUID model and no legacy flat Lua layout (`repo.toml`, top-level `packages/`, `lib/`, `templates/`). +- No generated F-Droid rewrite to standard modules until v1 host semantics and tests exist. +- No live product HTTP/auth/rate-limit implementation in this design slice. +- No downloader/installer/background worker/recovery semantics. +- No Flutter/Kotlin provider parsing or domain logic. +- No GitHub global catalog/search/autogen. +- No cross-repository `luaclass` lookup. +- No Lua-native networking or default `http_get` in plain package evaluation. +- No broad protective denylist for local hooks. +- No claim that current fixture cache keys are complete for authenticated/live provider stability. + +## Later implementation slices + +### Slice 1: stable namespace harness (fixture-backed) + +- Add operations-owned installer for `getter.provider.*` using fixture-backed F-Droid/GitHub providers. +- Return result envelopes and record provider call traces. +- Keep `github.latest_commit` reserved/not installed by the default release-check host; it may stay covered by the existing provider operation until live-package semantics are implemented. +- Tests: + - stable namespace exists only in provider-backed operation; + - `getter_dev.*` is not required by stable modules; + - F-Droid and GitHub release functions return `{ candidates, source, cache_key, diagnostics }` for non-empty candidates; + - F-Droid/GitHub zero-candidate cases return `candidates = nil`/omitted plus diagnostics and the standard module does not emit an empty Lua table as `updates`; + - reserved latest-commit shape is not installed/called by the default release module; + - operation cache mode, stale diagnostics, and provider miss diagnostics match this plan. + +### Slice 2: provider cache provenance schema + +- Add provider/source cache provenance storage or a side table that records the source response body digest(s), provider/parser/cache schema version, and freshness tokens used to produce parsed provider facts. +- Teach provider cache reads to report whether provenance is present and Manifest-compatible for the current package script. +- Until this exists, non-free provider cache hits with parsed facts must fail closed or refetch/revalidate; tests must prove missing provenance does not bypass Manifest. +- Keep this in `getter-operations`/storage-facing code, not `getter-core`. + +### Slice 3: hooks + Manifest policy for provider functions + +- Extend the internal policy harness so provider functions are installed before runtime hooks. +- Prove provider hooks can wrap `getter.provider.*` and call `getter_builtin.provider.*`. +- Prove non-free scripts cannot use provider response/cache facts without Manifest-listed source response digests and accepted provenance. +- Keep transport fixture/in-memory; no live HTTP. + +### Slice 4: standard module promotion tests + +- Rewrite `luaclass.fdroid_android` and `luaclass.github_android_apk` to call `getter.provider.*`. +- Keep repository-local override precedence. +- Decide exact user-facing behavior when a provider module is called without provider host installed (stable diagnostic vs module unavailable in plain eval). +- Only after tests, promote modules from dev-gated tracers to always-on built-in fallback modules. + +### Slice 5: docs and generated output migration + +- Update ADR-0012, ADR-0005, `docs/lua-api/permissions.md`, `docs/lua-api/repository-layout.md`, and `CONTEXT.md` with stable names and corrected `http_get` table-options syntax. +- Update generated F-Droid output to use `luaclass.fdroid_android` only after stable module/host tests pass. +- Add generator tests proving generated output does not depend on `getter_dev.*` or repository-local copied modules. + +## Review questions for the read-only reviewer + +1. Are `getter.provider.*` and `getter_builtin.provider.*` consistent with existing hook and Lua-boundary decisions? +2. Does the plan keep provider/cache/storage logic out of `getter-core` while allowing getter-shipped standard Lua modules? +3. Is the envelope shape small enough for v1 while preserving diagnostics/cache/source metadata outside Lua and avoiding empty-Lua-table/JSON-array ambiguity? +4. Is the Manifest/cache provenance rule concrete enough now that provenance storage is a required implementation slice before Manifest-bound provider cache hits become stable? +5. Should latest-commit remain a reserved shape that is not installed by the default release host until live-package semantics are implemented? +6. Are any proposed names or semantics likely to conflict with ADR-0012, ADR-0005, or the existing repository/layout model? diff --git a/docs/architecture/adr/0012-getter-owned-provider-modules-and-autogen-refresh.md b/docs/architecture/adr/0012-getter-owned-provider-modules-and-autogen-refresh.md index dd5be78a..ac25e2be 100644 --- a/docs/architecture/adr/0012-getter-owned-provider-modules-and-autogen-refresh.md +++ b/docs/architecture/adr/0012-getter-owned-provider-modules-and-autogen-refresh.md @@ -42,7 +42,7 @@ F-Droid and GitHub are providers/sources/backends, not package identities and no 1. Getter owns live provider execution, provider/source caching, package metadata normalization, update selection, action issuance, and autogen/package-path decisions. 2. Reusable Lua provider modules/classes under `luaclass/` provide high-level package-authoring APIs for common provider families. -3. Those Lua modules call getter-owned host APIs by default. HTTP access is exposed as a getter-managed host function such as `http_get(url, { headers = ..., cache = true|false })`; `cache` defaults to `false`, and Lua/provider modules opt individual requests into HTTP source caching by passing `cache = true`. Plain package evaluation does not install `http_get`; the getter operation/runtime evaluating provider-backed Lua deliberately installs the transport and owns permission, Manifest, provider, cache, and diagnostic policy for that execution. This remains getter-owned network/cache execution, not Flutter/Kotlin HTTP and not a Lua standard-library network primitive. +3. Those Lua modules call getter-owned host APIs by default. Getter-shipped standard provider modules use provider-specific host functions such as `getter.provider.fdroid.update_candidates(...)` and `getter.provider.github.release_candidates(...)` so Rust owns provider parsing, cache consistency, diagnostics, and candidate normalization. Generic/custom Lua HTTP remains available through a getter-managed host function such as `http_get(url, { headers = ..., cache = true|false })`; `cache` defaults to `false`, and Lua opts individual generic HTTP requests into HTTP source caching by passing `cache = true`. Plain package evaluation does not install `http_get` or provider host APIs; the getter operation/runtime evaluating provider-backed Lua deliberately installs the transport/host functions and owns permission, Manifest, provider, cache, and diagnostic policy for that execution. This remains getter-owned network/cache execution, not Flutter/Kotlin HTTP and not a Lua standard-library network primitive. 4. F-Droid support is **autogen-first**: - an F-Droid app is represented as an ordinary package directory with metadata and version scripts; - explicit user selection of an F-Droid app uses a getter autogen preview/apply operation that generates a package directory/version script; @@ -89,6 +89,27 @@ Repository-local modules deliberately win over getter-shipped modules. This give Implementation status: only the neutral `luaclass.android` helper is an always-on getter-shipped builtin today. Provider-named fallback modules such as `luaclass.fdroid_android` and `luaclass.github_android_apk` are currently dev-gated tracer modules over `getter_dev.*` host calls. They exist to validate provider-host seams and builtin fallback distribution, not to publish a stable product Lua provider API. Generated F-Droid output does not depend on them yet; promoting provider modules to an always-on public authoring surface requires a later stable host-API decision. +### Stable provider host API v1 direction + +The stable provider host namespace is rooted at `getter.provider.*`: + +```lua +getter.provider.fdroid.update_candidates(spec) +getter.provider.github.release_candidates(spec) +-- reserved for an explicit live/floating operation, not installed by the default release module: +getter.provider.github.latest_commit(spec) +``` + +Host functions installed for package Lua are also exposed to runtime hooks through matching originals under `getter_builtin.provider.*`. Ordinary package Lua and `luaclass/` code should call the public `getter.provider.*` functions; `getter_builtin.*` is an escape hatch for local `rc/hook/*.lua` policy. + +F-Droid `update_candidates` takes a required `package_name` and optional `endpoint_id`. GitHub `release_candidates` takes required typed `owner` and `repo`, optional Rust-regex asset filters (`asset.include` / `asset.exclude`), optional `include_prereleases`, and optional `endpoint_id`. The package Lua API keeps typed coordinates even if a Lua helper later accepts shorthand authoring syntax. + +Candidate-returning provider functions return an envelope containing non-empty `candidates`, provider cache `source` (`cache`, `refreshed`, or `stale`), `cache_key`, and getter-owned `diagnostics`. Zero-candidate provider results use `candidates = nil`/omitted plus diagnostics rather than an empty Lua table, because the current Lua-to-JSON boundary serializes empty Lua tables as objects, not arrays. Standard modules pass `result.candidates` through to `package_version { updates = ... }`, so nil/omitted candidates become an omitted `updates` field and validate as no updates. + +The GitHub `latest_commit` host shape is reserved because latest-commit checks are live/floating behavior. The default GitHub release/APK helper must not install or call it by default, and latest-commit results must not be silently treated as ordinary release candidates. A later explicit live operation/helper may install and use it after live-version UI/CLI semantics are implemented. + +Provider functions must not bypass Manifest policy. For package scripts without `allow_free_network`, every external response body used to produce provider facts must match a package `Manifest` SHA-512 entry. Parsed provider cache hits are usable for non-free scripts only when cache provenance records the source response digest(s) and proves Manifest compatibility; missing provenance must fail closed or refetch/revalidate. This provenance storage/test slice is required before Manifest-bound provider cache hits are called stable. + ### F-Droid reusable module The common package-authoring API should be intentionally small. The default case should need only the Android/F-Droid package name: @@ -116,7 +137,7 @@ The common class may allow optional typed fields when needed, for example: ```lua return fdroid.package { package_name = "org.example", - endpoint = "fdroid-official", + endpoint_id = "fdroid-official", channel = "stable", } ``` @@ -181,7 +202,7 @@ return github_android.package { owner = "f-droid", repo = "fdroidclient", asset = { - include = "%.apk$", + include = "[.]apk$", exclude = "debug", }, } @@ -210,7 +231,7 @@ The standard GitHub class can provide useful defaults, especially for Android AP Package-authored filters/overrides remain valid and expected: -- include/exclude regexes; +- include/exclude Rust-regex filters; - artifact naming rules; - prerelease handling; - ABI/channel/flavor selection; @@ -265,7 +286,9 @@ ADR-0012 preserves ADR-0010 cache consistency and makes the cache layers explici Provider/source cache entries live in `cache.db` and store upstream facts or parsed provider facts. -Lua/provider modules opt into HTTP source caching per request through getter's host HTTP API, for example: +Getter-shipped standard provider modules call provider-specific host functions such as `getter.provider.fdroid.update_candidates(...)` and `getter.provider.github.release_candidates(...)`; Rust provider operations decide which upstream request(s), parsed facts, freshness tokens, and provider/source cache entries are involved. + +Generic/custom Lua can still opt into HTTP source caching per request through getter's host HTTP API, for example: ```lua local index = http_get(fdroid_index_url, { @@ -274,7 +297,7 @@ local index = http_get(fdroid_index_url, { }) ``` -`cache = false` is the default so ordinary one-off HTTP calls do not silently become durable provider cache. The v1 host request shape is intentionally narrow: a URL string plus an optional options table with string-to-string `headers` and boolean `cache`; unsupported options are rejected rather than silently accepted. When `cache = true`, getter owns cache key construction, storage, revalidation, stale diagnostics, and secret redaction; Lua chooses that the request should participate in HTTP/source caching but does not write cache entries itself. +`cache = false` is the default so ordinary one-off HTTP calls do not silently become durable provider cache. The v1 host request shape is intentionally narrow: a URL string plus an optional options table with string-to-string `headers` and boolean `cache`; unsupported options are rejected rather than silently accepted. When `cache = true`, getter owns cache key construction, storage, revalidation, stale diagnostics, and secret redaction; Lua chooses that the generic HTTP request should participate in HTTP/source caching but does not write cache entries itself. F-Droid provider cache keys must include inputs such as: From 016f97767568eb7d3c3f20cd4e474bbd71c783d5 Mon Sep 17 00:00:00 2001 From: Xiangzhe Date: Sun, 28 Jun 2026 10:51:28 +0800 Subject: [PATCH 76/85] feat(getter): add stable provider host harness --- core-getter/src/main/rust/getter | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/core-getter/src/main/rust/getter b/core-getter/src/main/rust/getter index 25c510ac..bb5b7032 160000 --- a/core-getter/src/main/rust/getter +++ b/core-getter/src/main/rust/getter @@ -1 +1 @@ -Subproject commit 25c510acdeea043095bb44951b63bdce88bd76e5 +Subproject commit bb5b7032554b76d01d164772e4490698ee315d44 From 561d0e7f04d232086513f80f8bfe6c6b73112a6a Mon Sep 17 00:00:00 2001 From: Xiangzhe Date: Sun, 28 Jun 2026 12:05:16 +0800 Subject: [PATCH 77/85] feat(getter): store provider response provenance --- CONTEXT.md | 6 +++++- PLAN/provider-host-api-v1.md | 4 +++- core-getter/src/main/rust/getter | 2 +- ...012-getter-owned-provider-modules-and-autogen-refresh.md | 6 +++--- 4 files changed, 12 insertions(+), 6 deletions(-) diff --git a/CONTEXT.md b/CONTEXT.md index f95180eb..3983da07 100644 --- a/CONTEXT.md +++ b/CONTEXT.md @@ -112,7 +112,11 @@ A getter-owned notification boundary used by native/Flutter UI to learn that run ### Provider host API -A getter-owned API exposed to package Lua for requesting provider/source data. Lua packages should call getter host APIs rather than using any Lua-native network library or Flutter/Kotlin HTTP path. Getter-shipped standard provider modules use provider-specific host functions under `getter.provider.*`, such as `getter.provider.fdroid.update_candidates(...)` and `getter.provider.github.release_candidates(...)`, so Rust owns provider parsing, cache consistency, diagnostics, and candidate normalization. Generic/custom HTTP remains exposed as a getter-managed host function such as `http_get(url, { headers = ..., cache = true|false })`; `cache` defaults to `false`, and Lua actively opts a generic request into HTTP/source caching with `cache = true`. Getter owns execution, cache key construction, persistence, revalidation, stale diagnostics, permissions, secret redaction, and output validation. The provider executor behind the host API can be fake during early runtime development and live later. +A getter-owned API exposed to package Lua for requesting provider/source data. Lua packages should call getter host APIs rather than using any Lua-native network library or Flutter/Kotlin HTTP path. Getter-shipped standard provider modules use provider-specific host functions under `getter.provider.*`, such as `getter.provider.fdroid.update_candidates(...)` and `getter.provider.github.release_candidates(...)`, so Rust owns provider parsing, cache consistency, diagnostics, and candidate normalization. Generic/custom HTTP remains exposed as a getter-managed host function such as `http_get(url, { headers = ..., cache = true|false })`; `cache` defaults to `false`, and Lua actively opts a generic request into HTTP/source caching with `cache = true`. Getter owns execution, cache key construction, persistence, revalidation, stale diagnostics, permissions, secret redaction, and output validation. For package scripts without `allow_free_network`, provider host calls must not bypass package `Manifest`: refreshed source response bodies must match Manifest SHA-512 entries, and parsed provider cache hits are usable only when their provider cache provenance records source response digest(s) and a compatible provenance schema proving Manifest compatibility. The provider executor behind the host API can be fake during early runtime development and live later. + +### Provider cache provenance + +Source-body evidence stored beside a parsed provider/source cache entry in `cache.db`. It records the SHA-512 digest(s) of external response bodies used to produce parsed provider facts, the provenance schema version, and freshness metadata placeholders/tokens when available. It is not repository trust and not package source integrity; it exists so getter can decide whether a parsed provider cache hit is allowed for the current package version script's Manifest policy. Missing provenance, incompatible provenance schema, or source response digests absent from the package `Manifest` make non-`allow_free_network` provider cache hits fail closed. Scripts with `allow_free_network` are not blocked by Manifest membership, but still surface the high-risk permission. ### Package metadata cache diff --git a/PLAN/provider-host-api-v1.md b/PLAN/provider-host-api-v1.md index 0eadb309..06fcc1f3 100644 --- a/PLAN/provider-host-api-v1.md +++ b/PLAN/provider-host-api-v1.md @@ -11,7 +11,9 @@ - [x] Draft v1 API boundary, names, cache/HTTP/hook/Manifest relationships, and non-goals. - [x] Run read-only oracle/reviewer review of this design. - [x] Revise this plan and copy durable decisions into ADR-0012 / glossary docs. -- [ ] Next slice: start a narrow TDD implementation of the fixture-backed stable namespace harness. +- [x] Implement and validate Slice 1 fixture-backed stable namespace harness. +- [x] Implement and validate Slice 2 provider cache provenance storage and Manifest-compatible cache hits. +- [ ] Next slice: standard provider module promotion tests. ## Current evidence and constraints diff --git a/core-getter/src/main/rust/getter b/core-getter/src/main/rust/getter index bb5b7032..6a83a2ba 160000 --- a/core-getter/src/main/rust/getter +++ b/core-getter/src/main/rust/getter @@ -1 +1 @@ -Subproject commit bb5b7032554b76d01d164772e4490698ee315d44 +Subproject commit 6a83a2baa31efa96d7e7fd9d252b36ef29a9cf0f diff --git a/docs/architecture/adr/0012-getter-owned-provider-modules-and-autogen-refresh.md b/docs/architecture/adr/0012-getter-owned-provider-modules-and-autogen-refresh.md index ac25e2be..5f17df24 100644 --- a/docs/architecture/adr/0012-getter-owned-provider-modules-and-autogen-refresh.md +++ b/docs/architecture/adr/0012-getter-owned-provider-modules-and-autogen-refresh.md @@ -108,7 +108,7 @@ Candidate-returning provider functions return an envelope containing non-empty ` The GitHub `latest_commit` host shape is reserved because latest-commit checks are live/floating behavior. The default GitHub release/APK helper must not install or call it by default, and latest-commit results must not be silently treated as ordinary release candidates. A later explicit live operation/helper may install and use it after live-version UI/CLI semantics are implemented. -Provider functions must not bypass Manifest policy. For package scripts without `allow_free_network`, every external response body used to produce provider facts must match a package `Manifest` SHA-512 entry. Parsed provider cache hits are usable for non-free scripts only when cache provenance records the source response digest(s) and proves Manifest compatibility; missing provenance must fail closed or refetch/revalidate. This provenance storage/test slice is required before Manifest-bound provider cache hits are called stable. +Provider functions must not bypass Manifest policy. For package scripts without `allow_free_network`, every external response body used to produce provider facts must match a package `Manifest` SHA-512 entry. Parsed provider cache hits are usable for non-free scripts only when cache provenance records the source response digest(s) and proves Manifest compatibility; missing provenance must fail closed or refetch/revalidate. The initial fixture-backed provenance slice stores source response SHA-512 digest(s), a provenance schema version, and freshness metadata placeholders with parsed provider cache entries so Manifest-compatible cache hits can be accepted while legacy/missing-provenance rows still fail closed. ### F-Droid reusable module @@ -284,9 +284,9 @@ ADR-0012 preserves ADR-0010 cache consistency and makes the cache layers explici ### Provider/source cache -Provider/source cache entries live in `cache.db` and store upstream facts or parsed provider facts. +Provider/source cache entries live in `cache.db` and store upstream facts or parsed provider facts plus provenance for the source response bodies used to produce those facts. -Getter-shipped standard provider modules call provider-specific host functions such as `getter.provider.fdroid.update_candidates(...)` and `getter.provider.github.release_candidates(...)`; Rust provider operations decide which upstream request(s), parsed facts, freshness tokens, and provider/source cache entries are involved. +Getter-shipped standard provider modules call provider-specific host functions such as `getter.provider.fdroid.update_candidates(...)` and `getter.provider.github.release_candidates(...)`; Rust provider operations decide which upstream request(s), parsed facts, source response digest(s), freshness tokens, and provider/source cache entries are involved. Generic/custom Lua can still opt into HTTP source caching per request through getter's host HTTP API, for example: From b20fb0a9b486b78a72411c6537937f3538b12afb Mon Sep 17 00:00:00 2001 From: Xiangzhe Date: Sun, 28 Jun 2026 14:08:29 +0800 Subject: [PATCH 78/85] feat(getter): promote standard provider luaclasses --- CONTEXT.md | 4 +- PLAN/provider-host-api-v1.md | 41 ++++++++++--------- core-getter/src/main/rust/getter | 2 +- docs/architecture/adr/0005-lua-package-api.md | 4 +- ...ed-provider-modules-and-autogen-refresh.md | 2 +- .../upgradeall-getter-rewrite-wiki.md | 16 +++++--- docs/lua-api/permissions.md | 2 +- docs/lua-api/repository-layout.md | 2 +- 8 files changed, 40 insertions(+), 33 deletions(-) diff --git a/CONTEXT.md b/CONTEXT.md index 3983da07..4f76a753 100644 --- a/CONTEXT.md +++ b/CONTEXT.md @@ -64,7 +64,7 @@ A declaration inside a package Lua definition that uses a reusable provider modu ### Reusable Lua provider module/class -A Lua helper under a repository's `luaclass/` directory that provides common provider behavior and calls getter-owned provider host APIs. F-Droid should have a standard class where the common case specifies only `package_name`; generated F-Droid package metadata/version scripts should not duplicate `name` because display metadata comes from the self-describing F-Droid catalog. F-Droid endpoint names come from endpoint ids/directories, and the endpoint URL defaults to official F-Droid but can be customized. GitHub should have a standard class where the common case specifies typed `owner` and `repo`, with package-authored asset/version rules when needed. +A Lua helper under a repository's `luaclass/` directory or getter-shipped built-in fallback module root that provides common provider behavior and calls getter-owned provider host APIs. F-Droid has a standard built-in class where the common case specifies only `package_name`; generated F-Droid package metadata/version scripts should not duplicate `name` because display metadata comes from the self-describing F-Droid catalog. F-Droid endpoint names come from endpoint ids/directories, and the endpoint URL defaults to official F-Droid but can be customized. GitHub has a standard built-in class where the common case specifies typed `owner` and `repo`, with package-authored asset/version rules when needed. Built-in provider modules call `getter.provider.*` and require a provider-backed operation to install those host functions; plain package evaluation does not install provider host APIs. ### Autogen pipeline/template @@ -112,7 +112,7 @@ A getter-owned notification boundary used by native/Flutter UI to learn that run ### Provider host API -A getter-owned API exposed to package Lua for requesting provider/source data. Lua packages should call getter host APIs rather than using any Lua-native network library or Flutter/Kotlin HTTP path. Getter-shipped standard provider modules use provider-specific host functions under `getter.provider.*`, such as `getter.provider.fdroid.update_candidates(...)` and `getter.provider.github.release_candidates(...)`, so Rust owns provider parsing, cache consistency, diagnostics, and candidate normalization. Generic/custom HTTP remains exposed as a getter-managed host function such as `http_get(url, { headers = ..., cache = true|false })`; `cache` defaults to `false`, and Lua actively opts a generic request into HTTP/source caching with `cache = true`. Getter owns execution, cache key construction, persistence, revalidation, stale diagnostics, permissions, secret redaction, and output validation. For package scripts without `allow_free_network`, provider host calls must not bypass package `Manifest`: refreshed source response bodies must match Manifest SHA-512 entries, and parsed provider cache hits are usable only when their provider cache provenance records source response digest(s) and a compatible provenance schema proving Manifest compatibility. The provider executor behind the host API can be fake during early runtime development and live later. +A getter-owned API exposed to package Lua for requesting provider/source data. Lua packages should call getter host APIs rather than using any Lua-native network library or Flutter/Kotlin HTTP path. Getter-shipped standard provider modules use provider-specific host functions under `getter.provider.*`, such as `getter.provider.fdroid.update_candidates(...)` and `getter.provider.github.release_candidates(...)`, so Rust owns provider parsing, cache consistency, diagnostics, and candidate normalization. Generic/custom HTTP remains exposed as a getter-managed host function such as `http_get(url, { headers = ..., cache = true|false })`; `cache` defaults to `false`, and Lua actively opts a generic request into HTTP/source caching with `cache = true`. Getter owns execution, cache key construction, persistence, revalidation, stale diagnostics, permissions, secret redaction, and output validation. For package scripts without `allow_free_network`, provider host calls must not bypass package `Manifest`: refreshed source response bodies must match Manifest SHA-512 entries, and parsed provider cache hits are usable only when their provider cache provenance records source response digest(s) and a compatible provenance schema proving Manifest compatibility. Getter-shipped provider modules such as `luaclass.fdroid_android` and `luaclass.github_android_apk` are built-in fallback modules over this host API, not independent provider implementations. The provider executor behind the host API can be fake during early runtime development and live later. ### Provider cache provenance diff --git a/PLAN/provider-host-api-v1.md b/PLAN/provider-host-api-v1.md index 06fcc1f3..6599c667 100644 --- a/PLAN/provider-host-api-v1.md +++ b/PLAN/provider-host-api-v1.md @@ -1,8 +1,8 @@ # PLAN: Stable Lua provider host API v1 -> Status: reviewed design; ready for the first fixture-backed implementation slice +> Status: implementation plan with stable host, provenance, and provider module promotion slices validated; generated-output migration pending > Scope: ADR-0012 provider-host API design for F-Droid and GitHub standard Lua modules -> Non-code slice: document and review before promoting dev tracers or changing generated F-Droid output +> Current boundary: provider modules are promoted to built-in fallbacks; generated F-Droid output has not yet been migrated to require them ## Progress @@ -13,17 +13,18 @@ - [x] Revise this plan and copy durable decisions into ADR-0012 / glossary docs. - [x] Implement and validate Slice 1 fixture-backed stable namespace harness. - [x] Implement and validate Slice 2 provider cache provenance storage and Manifest-compatible cache hits. -- [ ] Next slice: standard provider module promotion tests. +- [x] Implement and validate Slice 4 standard provider module promotion tests. +- [ ] Next slice: generated F-Droid provider-module adoption. ## Current evidence and constraints - `getter-core` owns constrained Lua evaluation, package/module resolution, JSON-like table conversion, schema/domain validation, `read_package_file`, and the generic `http_get` host seam. It must not depend on provider/cache/storage crates. -- `getter-operations` owns provider/cache orchestration today (`fdroid_catalog`, `github_releases`, `github_latest_commit`, `provider_cache`) and currently hosts the dev-only Lua provider tracer (`lua_provider_host`). +- `getter-operations` owns provider/cache orchestration today (`fdroid_catalog`, `github_releases`, `github_latest_commit`, `provider_cache`) and hosts the fixture-backed Lua provider operation harness (`lua_provider_host`) plus private `getter_dev.*` compatibility shims for existing development tests. - Plain package evaluation does **not** install `http_get` or provider APIs. Provider-backed operations install host functions deliberately. - `http_get(url, { headers = ..., cache = true|false })` is the accepted narrow generic HTTP request shape; `cache` defaults false, and unsupported options are rejected. - Runtime hooks live under `/rc/hook/*.lua`, load after host functions are installed in the current operation harness, and call originals through `getter_builtin.*`. - Package `Manifest` constrains external network/dynamic response bodies for scripts without `allow_free_network`; missing `Manifest` means an empty allow-list. Hooks and cache must not bypass this. -- Current provider-named modules `luaclass.fdroid_android` and `luaclass.github_android_apk` are dev-gated tracers over `getter_dev.*`. They validate distribution and authoring shape, not a stable product API. +- Current provider-named modules `luaclass.fdroid_android` and `luaclass.github_android_apk` are always-on getter-shipped built-in fallback modules that call operation-installed `getter.provider.*` host functions. Plain package evaluation still does not install provider APIs and fails with a stable host-unavailable error if a package calls a provider module there. - Repository-local `luaclass/` modules must continue to override getter-shipped built-ins; cross-repository `luaclass` lookup stays unsupported. ## Design goals @@ -55,7 +56,7 @@ Rationale: - `getter` clearly marks host-owned functionality, unlike free globals such as `fdroid_update_candidates`. - `provider` groups provider-specific host APIs and leaves room for non-provider getter APIs later. - Provider and function names are explicit enough for hooks and diagnostics. -- The dev-only `getter_dev.*` namespace remains private and should not appear in stable package docs. +- The dev-only `getter_dev.*` compatibility namespace remains private and should not appear in stable package docs. ### Hook originals @@ -209,7 +210,7 @@ F-Droid `update_candidates` and GitHub `release_candidates` return: } ``` -Zero-candidate provider results use `candidates = nil`/omitted plus diagnostics. This mirrors the current dev tracer and avoids the existing Lua JSON ambiguity where an empty Lua table serializes as `{}` rather than `[]`, which would fail the current `updates` array schema if passed through directly. +Zero-candidate provider results use `candidates = nil`/omitted plus diagnostics. This avoids the existing Lua JSON ambiguity where an empty Lua table serializes as `{}` rather than `[]`, which would fail the current `updates` array schema if passed through directly. The non-empty `candidates[]` item shape is intentionally the current `getter_core::UpdateCandidate` / `UpdateArtifact` shape. Richer provider-candidate fields from ADR-0012 (`published_at`, `changelog`, `metadata_digest`, etc.) are future extension fields, not required to stabilize v1. @@ -272,7 +273,7 @@ It should not require or duplicate display `name`; F-Droid catalog metadata is s ### `luaclass.github_android_apk` -Stable author API should stay close to the current dev tracer: +Stable author API shape: ```lua local github_android = require("luaclass.github_android_apk") @@ -321,13 +322,13 @@ The module may later grow explicit live helpers, but release checks and latest-c ### Promotion path -Do not promote the dev tracer modules directly as product API. The later implementation should: +Implemented promotion state: -1. Add stable host installation in `getter-operations` while keeping `getter_dev.*` tests private or migrating them. -2. Rewrite the provider modules to call `getter.provider.*`, not `getter_dev.*`. -3. Add tests that default/plain package evaluation either does not resolve provider modules until promoted or fails with a stable host-unavailable diagnostic if a provider module is called without provider host installed. -4. Only then make provider modules always-on getter-shipped built-ins. -5. Only after that, consider changing generated F-Droid output to require `luaclass.fdroid_android`. +1. Stable host installation exists in `getter-operations`; the old `getter_dev.*` compatibility shims delegate through the stable host functions for development harness callers. +2. `luaclass.fdroid_android` and `luaclass.github_android_apk` call `getter.provider.*`, not `getter_dev.*`. +3. Tests cover provider-backed operation success, repository-local override precedence, and plain package evaluation failing with a stable host-unavailable diagnostic when provider modules are called without installed provider host functions. +4. Provider modules are always-on getter-shipped built-in fallback modules. +5. Generated F-Droid output still does not require `luaclass.fdroid_android`; changing generated output remains a later generator slice. ## Cache and refresh semantics @@ -458,14 +459,16 @@ Hooks are trusted local policy for advanced users. Do not add getter-core denyli ### Slice 4: standard module promotion tests -- Rewrite `luaclass.fdroid_android` and `luaclass.github_android_apk` to call `getter.provider.*`. -- Keep repository-local override precedence. -- Decide exact user-facing behavior when a provider module is called without provider host installed (stable diagnostic vs module unavailable in plain eval). -- Only after tests, promote modules from dev-gated tracers to always-on built-in fallback modules. +Done in the implementation branch: + +- `luaclass.fdroid_android` and `luaclass.github_android_apk` call `getter.provider.*`. +- Repository-local override precedence remains covered. +- Plain package evaluation resolves the built-in provider modules, but calling them without operation-installed provider host functions fails with a stable host-unavailable diagnostic. +- Provider modules are always-on built-in fallback modules; `getter_dev.*` remains only a private compatibility shim in operation tests. ### Slice 5: docs and generated output migration -- Update ADR-0012, ADR-0005, `docs/lua-api/permissions.md`, `docs/lua-api/repository-layout.md`, and `CONTEXT.md` with stable names and corrected `http_get` table-options syntax. +- Update ADR-0012, ADR-0005, `docs/lua-api/permissions.md`, `docs/lua-api/repository-layout.md`, and `CONTEXT.md` with stable provider module status. - Update generated F-Droid output to use `luaclass.fdroid_android` only after stable module/host tests pass. - Add generator tests proving generated output does not depend on `getter_dev.*` or repository-local copied modules. diff --git a/core-getter/src/main/rust/getter b/core-getter/src/main/rust/getter index 6a83a2ba..20f13bfc 160000 --- a/core-getter/src/main/rust/getter +++ b/core-getter/src/main/rust/getter @@ -1 +1 @@ -Subproject commit 6a83a2baa31efa96d7e7fd9d252b36ef29a9cf0f +Subproject commit 20f13bfcb30b9f9cfccb57a88220d9d8dadd0cdb diff --git a/docs/architecture/adr/0005-lua-package-api.md b/docs/architecture/adr/0005-lua-package-api.md index 504bcda8..18d84d0f 100644 --- a/docs/architecture/adr/0005-lua-package-api.md +++ b/docs/architecture/adr/0005-lua-package-api.md @@ -47,9 +47,9 @@ post_update Lua has no native or standard-library network API by default. Network access, when allowed by the package/provider mode, goes through getter host APIs such as `http_get(url, { headers = ..., cache = true|false })`. -Plain package evaluation does not install `http_get` by default. The getter operation/runtime that evaluates a provider-backed script must deliberately install an HTTP transport and own permission checks, Manifest validation, provider policy, cache behavior, and diagnostics for that execution. +Plain package evaluation does not install `http_get` or provider host APIs by default. The getter operation/runtime that evaluates a provider-backed script must deliberately install an HTTP transport and/or provider host functions and own permission checks, Manifest validation, provider policy, cache behavior, and diagnostics for that execution. Getter-shipped standard provider modules such as `luaclass.fdroid_android` and `luaclass.github_android_apk` call stable provider-specific host functions under `getter.provider.*` (for example `getter.provider.fdroid.update_candidates(...)` and `getter.provider.github.release_candidates(...)`) so Rust owns provider parsing and cache policy. Calling those modules without a provider-backed operation fails instead of performing provider work in plain package evaluation. -`cache` defaults to `false`. Passing `cache = true` opts that HTTP request into getter-owned provider/source caching; Lua chooses cache participation per request, but getter owns cache keys, persistence, revalidation, stale diagnostics, and secret redaction. The initial stable request shape is deliberately small: URL string plus an optional options table containing only string-to-string `headers` and boolean `cache`. +`cache` defaults to `false` for generic `http_get`. Passing `cache = true` opts that HTTP request into getter-owned provider/source caching; Lua chooses cache participation per request, but getter owns cache keys, persistence, revalidation, stale diagnostics, and secret redaction. The initial stable request shape is deliberately small: URL string plus an optional options table containing only string-to-string `headers` and boolean `cache`. Free network permission is declared per enabled Lua script in package `metadata.jsonc` using a filename-keyed map, for example `lua: { "9999.lua": { permission: ["allow_free_network"] } }`. The `lua` map is lookup-only: getter first discovers an enabled Lua file from the filesystem, then queries this map by basename. Getter does not enumerate the map to discover scripts or warnings. Entries for nonexistent files or dot-prefixed Lua files are inert. The permission can apply to `9999.lua` or to a fixed-version script. `9999.lua` commonly needs free network, but the filename alone does not grant the permission or force the warning if metadata does not declare it. A version script omitted from the `lua` map defaults to `permission: []`. diff --git a/docs/architecture/adr/0012-getter-owned-provider-modules-and-autogen-refresh.md b/docs/architecture/adr/0012-getter-owned-provider-modules-and-autogen-refresh.md index 5f17df24..06374e62 100644 --- a/docs/architecture/adr/0012-getter-owned-provider-modules-and-autogen-refresh.md +++ b/docs/architecture/adr/0012-getter-owned-provider-modules-and-autogen-refresh.md @@ -87,7 +87,7 @@ Package version Lua resolves `require("luaclass.")` in this order: Repository-local modules deliberately win over getter-shipped modules. This gives trusted repositories and `local` overlays a normal source-level override path while keeping generated repositories boring: generated package directories do not need to copy shared standard modules, and `.autogen.jsonc` remains package-local ownership proof rather than a repository-root ownership system. Built-in modules are part of the getter binary/source distribution and are not repository source files; repository trust/signing still applies only to repository-provided files. Cross-repository module lookup is not accepted in this slice because it would make package behavior depend on unrelated repository priority and trust boundaries. -Implementation status: only the neutral `luaclass.android` helper is an always-on getter-shipped builtin today. Provider-named fallback modules such as `luaclass.fdroid_android` and `luaclass.github_android_apk` are currently dev-gated tracer modules over `getter_dev.*` host calls. They exist to validate provider-host seams and builtin fallback distribution, not to publish a stable product Lua provider API. Generated F-Droid output does not depend on them yet; promoting provider modules to an always-on public authoring surface requires a later stable host-API decision. +Implementation status: getter ships `luaclass.android`, `luaclass.fdroid_android`, and `luaclass.github_android_apk` as built-in fallback modules. The provider-named modules call the stable `getter.provider.*` host namespace and require a provider-backed operation to install those host functions; plain package evaluation does not install them and fails with a stable host-unavailable error if a package calls a provider module there. Generated F-Droid output does not depend on provider modules yet; switching generated output to `luaclass.fdroid_android` remains a later generator migration after this stable host/module behavior is validated. ### Stable provider host API v1 direction diff --git a/docs/architecture/upgradeall-getter-rewrite-wiki.md b/docs/architecture/upgradeall-getter-rewrite-wiki.md index 12013856..19dc45cc 100644 --- a/docs/architecture/upgradeall-getter-rewrite-wiki.md +++ b/docs/architecture/upgradeall-getter-rewrite-wiki.md @@ -575,10 +575,14 @@ Lua package scripts 在边界返回 JSON-like object/table。 -- package path: android/app/org.fdroid.fdroid local github_android = require("luaclass.github_android_apk") -return github_android { - android = { package_name = "org.fdroid.fdroid" }, - repo = "f-droid/fdroidclient", - asset_pattern = "%.apk$", +return github_android.package { + name = "F-Droid", + android_package = "org.fdroid.fdroid", + owner = "f-droid", + repo = "fdroidclient", + asset = { + include = "[.]apk$", + }, } ``` @@ -842,9 +846,9 @@ local body = http_get(url, { }) ``` -`cache` 默认是 `false`。普通 package evaluation 不默认安装 `http_get`；需要 provider/network 的 getter operation/runtime 必须显式安装 transport，并由 getter 拥有 permission、Manifest、provider、cache、diagnostic policy。Lua/provider module 通过 `cache = true` 主动把单次 HTTP 请求纳入 getter-owned HTTP/source cache；getter 负责 cache key、持久化、revalidation、stale diagnostics 和 secret redaction，Lua 只表达该请求是否应缓存。v1 请求形状保持很小：URL string，加可选 options table，其中只接受 string-to-string `headers` 与 boolean `cache`。 +`cache` 默认是 `false`。普通 package evaluation 不默认安装 `http_get` 或 provider host API；需要 provider/network 的 getter operation/runtime 必须显式安装 transport/provider host functions，并由 getter 拥有 permission、Manifest、provider、cache、diagnostic policy。Generic/custom Lua 通过 `cache = true` 主动把单次 HTTP 请求纳入 getter-owned HTTP/source cache；getter 负责 cache key、持久化、revalidation、stale diagnostics 和 secret redaction，Lua 只表达该请求是否应缓存。v1 generic HTTP 请求形状保持很小：URL string，加可选 options table，其中只接受 string-to-string `headers` 与 boolean `cache`。 -标准 provider module/class 可以在声明的 provider/source 语义下使用该 host HTTP API 获取 release/catalog 信息。 +标准 provider module/class 默认调用 provider-specific host API，例如 `getter.provider.fdroid.update_candidates(...)` 和 `getter.provider.github.release_candidates(...)`，由 Rust getter provider operations 负责 release/catalog 获取、解析、cache provenance、diagnostics 和 candidate normalization，而不是在 Lua 中直接用 `http_get` 解析 provider payload。 ### 10.2 自由网络权限 diff --git a/docs/lua-api/permissions.md b/docs/lua-api/permissions.md index ca641e94..55f76fd3 100644 --- a/docs/lua-api/permissions.md +++ b/docs/lua-api/permissions.md @@ -8,7 +8,7 @@ Lua package scripts do not receive Lua-native or Flutter/Kotlin-owned network access by default. Plain package evaluation does not install an HTTP function. -Provider-backed getter operations may deliberately install getter-provided provider/source host APIs for that execution. HTTP requests go through getter-managed functions such as: +Provider-backed getter operations may deliberately install getter-provided provider/source host APIs for that execution. Getter-shipped standard provider modules such as `luaclass.fdroid_android` and `luaclass.github_android_apk` call provider-specific host functions under `getter.provider.*`; plain package evaluation does not install those host functions and therefore cannot run provider-backed modules by itself. Generic/custom HTTP requests go through getter-managed functions such as: ```lua local body = http_get(url, { diff --git a/docs/lua-api/repository-layout.md b/docs/lua-api/repository-layout.md index 179ef030..3a4cf064 100644 --- a/docs/lua-api/repository-layout.md +++ b/docs/lua-api/repository-layout.md @@ -201,7 +201,7 @@ Reusable Lua modules. These are conceptually similar to eclasses but are plain L local github_android = require("luaclass.github_android_apk") ``` -Package version Lua resolves `require("luaclass.")` from the active package repository's `luaclass/` directory first, then from getter-shipped built-in standard modules. Repository-local modules intentionally override built-in standard modules, so a trusted repository or `local` overlay can replace the shipped default behavior in normal source form. +Package version Lua resolves `require("luaclass.")` from the active package repository's `luaclass/` directory first, then from getter-shipped built-in standard modules. Repository-local modules intentionally override built-in standard modules, so a trusted repository or `local` overlay can replace the shipped default behavior in normal source form. Getter-shipped provider modules such as `luaclass.fdroid_android` and `luaclass.github_android_apk` call stable `getter.provider.*` host functions and require a provider-backed operation to install those functions; plain package evaluation does not install provider host APIs. Cross-repository `luaclass` imports are not supported in this model. A package in `repo/official` does not load modules from `repo/local`, `repo/autogen`, or another alias by priority or by explicit alias. If shared behavior is needed for generated packages, it should either live in getter-shipped built-in modules or be copied/authored into the active repository's own `luaclass/` tree. From c86c894728571b0a981558d2c35d4b4492da553f Mon Sep 17 00:00:00 2001 From: Xiangzhe Date: Sun, 28 Jun 2026 14:39:35 +0800 Subject: [PATCH 79/85] fix(getter): validate provider endpoint ids --- core-getter/src/main/rust/getter | 2 +- .../0012-getter-owned-provider-modules-and-autogen-refresh.md | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/core-getter/src/main/rust/getter b/core-getter/src/main/rust/getter index 20f13bfc..05b0a853 160000 --- a/core-getter/src/main/rust/getter +++ b/core-getter/src/main/rust/getter @@ -1 +1 @@ -Subproject commit 20f13bfcb30b9f9cfccb57a88220d9d8dadd0cdb +Subproject commit 05b0a85358b6082c2778736febbfa1a9249fcb14 diff --git a/docs/architecture/adr/0012-getter-owned-provider-modules-and-autogen-refresh.md b/docs/architecture/adr/0012-getter-owned-provider-modules-and-autogen-refresh.md index 06374e62..e6981967 100644 --- a/docs/architecture/adr/0012-getter-owned-provider-modules-and-autogen-refresh.md +++ b/docs/architecture/adr/0012-getter-owned-provider-modules-and-autogen-refresh.md @@ -102,7 +102,7 @@ getter.provider.github.latest_commit(spec) Host functions installed for package Lua are also exposed to runtime hooks through matching originals under `getter_builtin.provider.*`. Ordinary package Lua and `luaclass/` code should call the public `getter.provider.*` functions; `getter_builtin.*` is an escape hatch for local `rc/hook/*.lua` policy. -F-Droid `update_candidates` takes a required `package_name` and optional `endpoint_id`. GitHub `release_candidates` takes required typed `owner` and `repo`, optional Rust-regex asset filters (`asset.include` / `asset.exclude`), optional `include_prereleases`, and optional `endpoint_id`. The package Lua API keeps typed coordinates even if a Lua helper later accepts shorthand authoring syntax. +F-Droid `update_candidates` takes a required `package_name` and optional `endpoint_id`. GitHub `release_candidates` takes required typed `owner` and `repo`, optional Rust-regex asset filters (`asset.include` / `asset.exclude`), optional `include_prereleases`, and optional `endpoint_id`. The package Lua API keeps typed coordinates even if a Lua helper later accepts shorthand authoring syntax. If package Lua supplies an `endpoint_id` that the evaluating operation has not installed or resolved, the provider host call fails instead of silently using a default endpoint. Candidate-returning provider functions return an envelope containing non-empty `candidates`, provider cache `source` (`cache`, `refreshed`, or `stale`), `cache_key`, and getter-owned `diagnostics`. Zero-candidate provider results use `candidates = nil`/omitted plus diagnostics rather than an empty Lua table, because the current Lua-to-JSON boundary serializes empty Lua tables as objects, not arrays. Standard modules pass `result.candidates` through to `package_version { updates = ... }`, so nil/omitted candidates become an omitted `updates` field and validate as no updates. From 9e2be1e787e969e302e465cea614488691b970f7 Mon Sep 17 00:00:00 2001 From: Xiangzhe Date: Sun, 28 Jun 2026 14:56:32 +0800 Subject: [PATCH 80/85] test(getter): lock fdroid generated script boundary --- core-getter/src/main/rust/getter | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/core-getter/src/main/rust/getter b/core-getter/src/main/rust/getter index 05b0a853..15ee5362 160000 --- a/core-getter/src/main/rust/getter +++ b/core-getter/src/main/rust/getter @@ -1 +1 @@ -Subproject commit 05b0a85358b6082c2778736febbfa1a9249fcb14 +Subproject commit 15ee5362e09b89606b9daa34d08b66f321eb8ab3 From 3b70b922e806b2f34fd766be8c72b14fbc8b1365 Mon Sep 17 00:00:00 2001 From: Xiangzhe Date: Sun, 28 Jun 2026 14:57:32 +0800 Subject: [PATCH 81/85] docs(plan): defer fdroid generated provider adoption --- PLAN/provider-host-api-v1.md | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/PLAN/provider-host-api-v1.md b/PLAN/provider-host-api-v1.md index 6599c667..9fcc0910 100644 --- a/PLAN/provider-host-api-v1.md +++ b/PLAN/provider-host-api-v1.md @@ -14,7 +14,8 @@ - [x] Implement and validate Slice 1 fixture-backed stable namespace harness. - [x] Implement and validate Slice 2 provider cache provenance storage and Manifest-compatible cache hits. - [x] Implement and validate Slice 4 standard provider module promotion tests. -- [ ] Next slice: generated F-Droid provider-module adoption. +- [x] Add pre-adoption regression coverage locking generated F-Droid output as self-contained/plain-evaluable. +- [ ] Next slice: product provider-backed package evaluation/update-check operation before generated F-Droid provider-module adoption. ## Current evidence and constraints @@ -469,7 +470,9 @@ Done in the implementation branch: ### Slice 5: docs and generated output migration - Update ADR-0012, ADR-0005, `docs/lua-api/permissions.md`, `docs/lua-api/repository-layout.md`, and `CONTEXT.md` with stable provider module status. -- Update generated F-Droid output to use `luaclass.fdroid_android` only after stable module/host tests pass. +- Add pre-adoption regression coverage that generated F-Droid output remains self-contained, does not require `luaclass.fdroid_android`, and does not call `getter.provider.*` while normal package evaluation remains plain. +- Before changing generated output, add a product-facing provider-backed package evaluation/update-check operation (not the fixture/dev-hidden harness) and route only intended update-check callers through it. +- Update generated F-Droid output to use `luaclass.fdroid_android` only after the product provider-backed operation exists and tests prove generated packages remain usable through that operation. - Add generator tests proving generated output does not depend on `getter_dev.*` or repository-local copied modules. ## Review questions for the read-only reviewer From 0bb0c7759a0eac167c2bc8df42fd5cd9dc9a1b57 Mon Sep 17 00:00:00 2001 From: Xiangzhe Date: Sun, 28 Jun 2026 15:00:19 +0800 Subject: [PATCH 82/85] test(getter): cover configured provider endpoints --- core-getter/src/main/rust/getter | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/core-getter/src/main/rust/getter b/core-getter/src/main/rust/getter index 15ee5362..3996cbec 160000 --- a/core-getter/src/main/rust/getter +++ b/core-getter/src/main/rust/getter @@ -1 +1 @@ -Subproject commit 15ee5362e09b89606b9daa34d08b66f321eb8ab3 +Subproject commit 3996cbec98fbcdb0d9b445e54ba030b8a584e92f From 236794ed58f981add0fe245473519761ce5fc04a Mon Sep 17 00:00:00 2001 From: Xiangzhe Date: Sun, 28 Jun 2026 16:52:18 +0800 Subject: [PATCH 83/85] feat(getter): use provider-backed update checks --- CONTEXT.md | 2 +- PLAN/provider-host-api-v1.md | 26 +++++++++++++------ .../src/main/rust/api_proxy/src/lib.rs | 4 ++- core-getter/src/main/rust/getter | 2 +- .../0007-flutter-getter-bridge-contract.md | 2 +- ...ed-provider-modules-and-autogen-refresh.md | 4 +-- 6 files changed, 26 insertions(+), 14 deletions(-) diff --git a/CONTEXT.md b/CONTEXT.md index 4f76a753..758d8c35 100644 --- a/CONTEXT.md +++ b/CONTEXT.md @@ -112,7 +112,7 @@ A getter-owned notification boundary used by native/Flutter UI to learn that run ### Provider host API -A getter-owned API exposed to package Lua for requesting provider/source data. Lua packages should call getter host APIs rather than using any Lua-native network library or Flutter/Kotlin HTTP path. Getter-shipped standard provider modules use provider-specific host functions under `getter.provider.*`, such as `getter.provider.fdroid.update_candidates(...)` and `getter.provider.github.release_candidates(...)`, so Rust owns provider parsing, cache consistency, diagnostics, and candidate normalization. Generic/custom HTTP remains exposed as a getter-managed host function such as `http_get(url, { headers = ..., cache = true|false })`; `cache` defaults to `false`, and Lua actively opts a generic request into HTTP/source caching with `cache = true`. Getter owns execution, cache key construction, persistence, revalidation, stale diagnostics, permissions, secret redaction, and output validation. For package scripts without `allow_free_network`, provider host calls must not bypass package `Manifest`: refreshed source response bodies must match Manifest SHA-512 entries, and parsed provider cache hits are usable only when their provider cache provenance records source response digest(s) and a compatible provenance schema proving Manifest compatibility. Getter-shipped provider modules such as `luaclass.fdroid_android` and `luaclass.github_android_apk` are built-in fallback modules over this host API, not independent provider implementations. The provider executor behind the host API can be fake during early runtime development and live later. +A getter-owned API exposed to package Lua for requesting provider/source data. Lua packages should call getter host APIs rather than using any Lua-native network library or Flutter/Kotlin HTTP path. Getter-shipped standard provider modules use provider-specific host functions under `getter.provider.*`, such as `getter.provider.fdroid.update_candidates(...)` and `getter.provider.github.release_candidates(...)`, so Rust owns provider parsing, cache consistency, diagnostics, and candidate normalization. Generic/custom HTTP remains exposed as a getter-managed host function such as `http_get(url, { headers = ..., cache = true|false })`; `cache` defaults to `false`, and Lua actively opts a generic request into HTTP/source caching with `cache = true`. Getter owns execution, cache key construction, persistence, revalidation, stale diagnostics, permissions, secret redaction, and output validation. For package scripts without `allow_free_network`, provider host calls must not bypass package `Manifest`: refreshed source response bodies must match Manifest SHA-512 entries, and parsed provider cache hits are usable only when their provider cache provenance records source response digest(s) and a compatible provenance schema proving Manifest compatibility. Getter-shipped provider modules such as `luaclass.fdroid_android` and `luaclass.github_android_apk` are built-in fallback modules over this host API, not independent provider implementations. Plain read-model package evaluation does not install this host API, but the product runtime update-check operation does: `update_check_package_issue_action` installs the stable provider host and runtime hooks, evaluates package Lua, then compares normalized candidates and issues getter-owned opaque actions. The bridge request remains package/update oriented (`package_id`, optional `repository_id`, `installed_version`, and `pin_version`); provider fixture bodies, cache mode, endpoint URLs, and live transport controls are not Flutter/native payload fields. Until live provider transport is accepted and implemented, runtime update-check provider calls use existing provider cache rows and fail on cache miss or Manifest-incompatible provenance. The provider executor behind the host API can be fake during early runtime development and live later. ### Provider cache provenance diff --git a/PLAN/provider-host-api-v1.md b/PLAN/provider-host-api-v1.md index 9fcc0910..6d75d462 100644 --- a/PLAN/provider-host-api-v1.md +++ b/PLAN/provider-host-api-v1.md @@ -1,8 +1,8 @@ # PLAN: Stable Lua provider host API v1 -> Status: implementation plan with stable host, provenance, and provider module promotion slices validated; generated-output migration pending +> Status: stable host, provenance, provider module promotion, and cache-backed product runtime update-check slices validated; generated-output migration pending > Scope: ADR-0012 provider-host API design for F-Droid and GitHub standard Lua modules -> Current boundary: provider modules are promoted to built-in fallbacks; generated F-Droid output has not yet been migrated to require them +> Current boundary: provider modules are promoted to built-in fallbacks, and `update_check_package_issue_action` can evaluate provider-backed packages against Manifest-compatible provider cache entries; generated F-Droid output has not yet been migrated to require them ## Progress @@ -15,7 +15,8 @@ - [x] Implement and validate Slice 2 provider cache provenance storage and Manifest-compatible cache hits. - [x] Implement and validate Slice 4 standard provider module promotion tests. - [x] Add pre-adoption regression coverage locking generated F-Droid output as self-contained/plain-evaluable. -- [ ] Next slice: product provider-backed package evaluation/update-check operation before generated F-Droid provider-module adoption. +- [x] Add product provider-backed package evaluation/update-check operation before generated F-Droid provider-module adoption. +- [ ] Next slice: generated F-Droid provider-module adoption, keeping generated packages usable through the provider-backed update-check path. ## Current evidence and constraints @@ -26,6 +27,8 @@ - Runtime hooks live under `/rc/hook/*.lua`, load after host functions are installed in the current operation harness, and call originals through `getter_builtin.*`. - Package `Manifest` constrains external network/dynamic response bodies for scripts without `allow_free_network`; missing `Manifest` means an empty allow-list. Hooks and cache must not bypass this. - Current provider-named modules `luaclass.fdroid_android` and `luaclass.github_android_apk` are always-on getter-shipped built-in fallback modules that call operation-installed `getter.provider.*` host functions. Plain package evaluation still does not install provider APIs and fails with a stable host-unavailable error if a package calls a provider module there. +- Product runtime update checks deliberately install the stable provider host and runtime hooks before package evaluation. The bridge request shape remains narrow (`package_id`, optional `repository_id`, `installed_version`, and `pin_version`); provider fixture bodies, cache mode, endpoint URLs, and live transport controls are not Flutter/native request fields in this slice. +- Until live provider transport is accepted and implemented, the product runtime path is cache-backed: provider-backed packages can use existing provider cache rows only when Manifest-compatible provenance permits them, while cache miss/refresh remains a later provider operation concern. - Repository-local `luaclass/` modules must continue to override getter-shipped built-ins; cross-repository `luaclass` lookup stays unsupported. ## Design goals @@ -340,7 +343,7 @@ The operation evaluating package Lua decides the cache mode: - normal update/read: use cached provider facts when valid, refresh on cache miss or stale according to provider policy; - forced refresh: bypass cached reads for the requested provider/package scope, replace cache on success, and use stale cache only with explicit diagnostics on refresh failure. -Lua package authors do not pass `mode = "force_refresh"` to stable provider host functions. +Lua package authors do not pass `mode = "force_refresh"` to stable provider host functions. The first product runtime slice keeps `update_check_package_issue_action` in normal cache-backed mode only and does not add provider fixture/cache-mode fields to the Flutter/native bridge payload. Live refresh and user-triggered force-refresh semantics remain separate provider-operation work. ### Provider cache contents @@ -467,11 +470,18 @@ Done in the implementation branch: - Plain package evaluation resolves the built-in provider modules, but calling them without operation-installed provider host functions fails with a stable host-unavailable diagnostic. - Provider modules are always-on built-in fallback modules; `getter_dev.*` remains only a private compatibility shim in operation tests. -### Slice 5: docs and generated output migration +### Slice 5: product runtime adoption and generated output migration + +Done in the implementation branch: + +- Updated ADR-0012, `CONTEXT.md`, and this plan with stable provider module/product runtime status. +- Added pre-adoption regression coverage that generated F-Droid output remains self-contained, does not require `luaclass.fdroid_android`, and does not call `getter.provider.*` while normal package evaluation remains plain. +- Added a product-facing provider-backed package evaluation/update-check operation by routing only `update_check_package_issue_action` through the stable provider host; read-model `package_eval` remains plain. +- Kept the Flutter/native bridge request shape narrow: no provider fixture body, cache mode, endpoint URL, or live transport fields are added to the product payload. +- Added runtime tests proving static packages still work, F-Droid/GitHub provider modules work from Manifest-compatible provider cache entries, Manifest-incompatible cache fails closed, and no-update checks do not issue actions. + +Still pending: -- Update ADR-0012, ADR-0005, `docs/lua-api/permissions.md`, `docs/lua-api/repository-layout.md`, and `CONTEXT.md` with stable provider module status. -- Add pre-adoption regression coverage that generated F-Droid output remains self-contained, does not require `luaclass.fdroid_android`, and does not call `getter.provider.*` while normal package evaluation remains plain. -- Before changing generated output, add a product-facing provider-backed package evaluation/update-check operation (not the fixture/dev-hidden harness) and route only intended update-check callers through it. - Update generated F-Droid output to use `luaclass.fdroid_android` only after the product provider-backed operation exists and tests prove generated packages remain usable through that operation. - Add generator tests proving generated output does not depend on `getter_dev.*` or repository-local copied modules. diff --git a/core-getter/src/main/rust/api_proxy/src/lib.rs b/core-getter/src/main/rust/api_proxy/src/lib.rs index a0070838..f088f6d1 100644 --- a/core-getter/src/main/rust/api_proxy/src/lib.rs +++ b/core-getter/src/main/rust/api_proxy/src/lib.rs @@ -497,7 +497,9 @@ fn runtime_operation_with_runtime( ) })?; let db = open_main_db(data_dir)?; - runtime_operations::issue_action_from_registered_package_json(runtime, &db, &payload) + runtime_operations::issue_action_from_registered_package_json( + runtime, data_dir, &db, &payload, + ) } "task_submit" => runtime_operations::submit_action_json(runtime, &payload), "task_get" => runtime_operations::task_get_json(runtime, &payload), diff --git a/core-getter/src/main/rust/getter b/core-getter/src/main/rust/getter index 3996cbec..44cf664f 160000 --- a/core-getter/src/main/rust/getter +++ b/core-getter/src/main/rust/getter @@ -1 +1 @@ -Subproject commit 3996cbec98fbcdb0d9b445e54ba030b8a584e92f +Subproject commit 44cf664fa2ae3b98a3d762a40fd30c402233f78b diff --git a/docs/architecture/adr/0007-flutter-getter-bridge-contract.md b/docs/architecture/adr/0007-flutter-getter-bridge-contract.md index 21e1b3a8..0c49af6d 100644 --- a/docs/architecture/adr/0007-flutter-getter-bridge-contract.md +++ b/docs/architecture/adr/0007-flutter-getter-bridge-contract.md @@ -60,7 +60,7 @@ Internally, Rust/native bridge code scans Android inventory through the platform `loadSnapshot()` composes smaller getter-owned read-model operations into the UI shell's first snapshot DTO. In the Android/native product path, `MethodChannelGetterAdapter` calls `readOperation` for `repository_list`, `tracked_package_list`, and `package_eval`; Rust getter reads SQLite repository/tracked-package state and evaluates registered Lua packages. Flutter may parse and combine those returned DTOs for rendering, but must not perform repository resolution, Lua validation/evaluation, version comparison, migration mapping, or update selection in Dart. `readMigrationReports()` must go through a getter operation such as `legacy report-list`; Flutter must not inspect getter's data-directory layout directly. Runtime task rendering must use ADR-0011 runtime task snapshot APIs and opaque action/task controls; Flutter must not synthesize task states, retry policy, installer behavior, or update decisions. -The first product click-through update flow is: App detail calls a typed getter update-check operation, receives a getter-issued `action_id`, submits only that `action_id` to the process-lifetime runtime, and opens Downloads to query authoritative task snapshots. Flutter may refresh the Downloads page after `RuntimeNotification.task_changed`, but the notification is only a trigger; `task_list`/equivalent runtime queries remain the source of truth. +The first product click-through update flow is: App detail calls a typed getter update-check operation, receives a getter-issued `action_id`, submits only that `action_id` to the process-lifetime runtime, and opens Downloads to query authoritative task snapshots. The update-check operation may evaluate package Lua through the stable provider host for provider-backed packages, but provider/cache/Manifest logic stays in getter and the Flutter/native request remains package/update oriented: `package_id`, optional `repository_id`, `installed_version`, and `pin_version`. Flutter must not pass provider fixture bodies, cache-mode controls, endpoint URLs, raw provider payloads, or action/download details as product update-check input. Until live provider transport is accepted, provider-backed runtime checks use Manifest-compatible provider cache entries produced by getter provider operations. Flutter may refresh the Downloads page after `RuntimeNotification.task_changed`, but the notification is only a trigger; `task_list`/equivalent runtime queries remain the source of truth. ## Flutter DTOs diff --git a/docs/architecture/adr/0012-getter-owned-provider-modules-and-autogen-refresh.md b/docs/architecture/adr/0012-getter-owned-provider-modules-and-autogen-refresh.md index e6981967..c18c4173 100644 --- a/docs/architecture/adr/0012-getter-owned-provider-modules-and-autogen-refresh.md +++ b/docs/architecture/adr/0012-getter-owned-provider-modules-and-autogen-refresh.md @@ -87,7 +87,7 @@ Package version Lua resolves `require("luaclass.")` in this order: Repository-local modules deliberately win over getter-shipped modules. This gives trusted repositories and `local` overlays a normal source-level override path while keeping generated repositories boring: generated package directories do not need to copy shared standard modules, and `.autogen.jsonc` remains package-local ownership proof rather than a repository-root ownership system. Built-in modules are part of the getter binary/source distribution and are not repository source files; repository trust/signing still applies only to repository-provided files. Cross-repository module lookup is not accepted in this slice because it would make package behavior depend on unrelated repository priority and trust boundaries. -Implementation status: getter ships `luaclass.android`, `luaclass.fdroid_android`, and `luaclass.github_android_apk` as built-in fallback modules. The provider-named modules call the stable `getter.provider.*` host namespace and require a provider-backed operation to install those host functions; plain package evaluation does not install them and fails with a stable host-unavailable error if a package calls a provider module there. Generated F-Droid output does not depend on provider modules yet; switching generated output to `luaclass.fdroid_android` remains a later generator migration after this stable host/module behavior is validated. +Implementation status: getter ships `luaclass.android`, `luaclass.fdroid_android`, and `luaclass.github_android_apk` as built-in fallback modules. The provider-named modules call the stable `getter.provider.*` host namespace and require a provider-backed operation to install those host functions; plain package evaluation does not install them and fails with a stable host-unavailable error if a package calls a provider module there. The product runtime update-check operation `update_check_package_issue_action` installs the stable provider host and runtime hooks, then evaluates package Lua before update selection, so provider-backed packages can issue normal getter-owned update actions from Manifest-compatible provider cache entries. The Flutter/native payload remains package/update oriented (`package_id`, optional `repository_id`, `installed_version`, and `pin_version`); provider fixture bodies, cache mode, endpoint URLs, and live transport controls are not bridge request fields in this slice. Generated F-Droid output does not depend on provider modules yet; switching generated output to `luaclass.fdroid_android` remains a later generator migration after this stable host/module behavior is validated. ### Stable provider host API v1 direction @@ -108,7 +108,7 @@ Candidate-returning provider functions return an envelope containing non-empty ` The GitHub `latest_commit` host shape is reserved because latest-commit checks are live/floating behavior. The default GitHub release/APK helper must not install or call it by default, and latest-commit results must not be silently treated as ordinary release candidates. A later explicit live operation/helper may install and use it after live-version UI/CLI semantics are implemented. -Provider functions must not bypass Manifest policy. For package scripts without `allow_free_network`, every external response body used to produce provider facts must match a package `Manifest` SHA-512 entry. Parsed provider cache hits are usable for non-free scripts only when cache provenance records the source response digest(s) and proves Manifest compatibility; missing provenance must fail closed or refetch/revalidate. The initial fixture-backed provenance slice stores source response SHA-512 digest(s), a provenance schema version, and freshness metadata placeholders with parsed provider cache entries so Manifest-compatible cache hits can be accepted while legacy/missing-provenance rows still fail closed. +Provider functions must not bypass Manifest policy. For package scripts without `allow_free_network`, every external response body used to produce provider facts must match a package `Manifest` SHA-512 entry. Parsed provider cache hits are usable for non-free scripts only when cache provenance records the source response digest(s) and proves Manifest compatibility; missing provenance must fail closed or refetch/revalidate. The initial fixture-backed provenance slice stores source response SHA-512 digest(s), a provenance schema version, and freshness metadata placeholders with parsed provider cache entries so Manifest-compatible cache hits can be accepted while legacy/missing-provenance rows still fail closed. Until live provider transport is accepted and implemented, the product runtime update-check path is cache-backed: provider refresh/cache population remains the responsibility of provider operations rather than Flutter/Kotlin or the runtime update-check bridge payload. ### F-Droid reusable module From e1993f2bd22a367b34e0d44da9aa4ecf4a44037d Mon Sep 17 00:00:00 2001 From: Xiangzhe Date: Sun, 28 Jun 2026 21:00:53 +0800 Subject: [PATCH 84/85] feat(getter): use F-Droid provider module autogen output --- CONTEXT.md | 2 +- PLAN/provider-host-api-v1.md | 26 ++++++++++++------- core-getter/src/main/rust/getter | 2 +- ...06-package-centric-cli-command-contract.md | 4 +-- ...ed-provider-modules-and-autogen-refresh.md | 8 +++--- .../upgradeall-getter-rewrite-wiki.md | 5 ++-- docs/lua-api/templates.md | 14 +++++++++- 7 files changed, 41 insertions(+), 20 deletions(-) diff --git a/CONTEXT.md b/CONTEXT.md index 758d8c35..03144bd3 100644 --- a/CONTEXT.md +++ b/CONTEXT.md @@ -68,7 +68,7 @@ A Lua helper under a repository's `luaclass/` directory or getter-shipped built- ### Autogen pipeline/template -A getter-owned preview/apply workflow, backed by repository-level `.metadata/autogen/` metadata and Lua scripts, that writes ordinary package directories/version scripts from structured inputs. F-Droid support is autogen-first: explicit user-selected F-Droid apps and automatically discovered installed F-Droid-covered apps are represented as generated package directories, usually in the configured generated repository alias such as `autogen`, while upstream/local authors may still hand-write F-Droid package directories/version scripts. Each generated package directory stores its own getter-managed generation record, `.autogen.jsonc`, so lookup/cleanup is local to the package directory. Hashes recorded inside `.autogen.jsonc` are ownership/tamper-detection facts for generated output: they answer whether a file is still the file getter generated earlier, and they do not provide security trust, repository signing, or external-download validation. The `.autogen.jsonc` `files` map covers getter-written generated output such as `metadata.jsonc`, `Manifest`, generated Lua scripts, and generated `files/...` helper files; it does not include `.autogen.jsonc` itself, avoiding self-referential hashing. The generated repository is generated output: getter may overwrite package directories it previously generated only when matching `.autogen.jsonc` proves ownership. When refresh/overwrite ownership checks pass, getter clears the existing generated package directory contents, then writes the new generated contents into the same package directory, without preserving old unlisted extra files. If clearing any old file or subdirectory fails, the whole refresh/overwrite fails rather than being ignored. If writing new generated contents fails after clearing, the operation fails directly without rollback; the directory may be empty or partially written, and the next refresh continues by clearing and rewriting again. If a target directory exists without a matching generation record, apply reports a conflict and does not overwrite it. A generated-repo package directory missing `.autogen.jsonc` is a conflict rather than something getter automatically claims; without `.autogen.jsonc`, there is no ownership proof. If `.autogen.jsonc` exists but is malformed or schema-invalid, ordinary package discovery/evaluation is still decided by `metadata.jsonc`, but autogen refresh/apply/cleanup/overwrite reports a generated-ownership conflict and does not auto-fix, overwrite, or delete it. When cleanup ownership checks pass, cleanup clears the generated package directory contents, including `.autogen.jsonc` and any unlisted extra files inside it, but does not delete the package directory itself. If clearing any file or subdirectory fails, the whole cleanup/update fails rather than being ignored. Getter does not classify or preserve unlisted extra files in generated package directories because they are outside getter's domain; generated repositories are generated output. User-authored overrides belong in `repo/local/...`, not by hand-editing `repo/autogen/...`. +A getter-owned preview/apply workflow, backed by repository-level `.metadata/autogen/` metadata and Lua scripts, that writes ordinary package directories/version scripts from structured inputs. F-Droid support is autogen-first: explicit user-selected F-Droid apps and automatically discovered installed F-Droid-covered apps are represented as generated package directories, usually in the configured generated repository alias such as `autogen`, while upstream/local authors may still hand-write F-Droid package directories/version scripts. Generated F-Droid package directories use minimal metadata, `Manifest` entries derived from provider source response SHA-512 provenance, and a small `9999.lua` that calls `luaclass.fdroid_android` for the default official endpoint; they are validated through provider-backed update-check operations, not plain read-model package evaluation. Each generated package directory stores its own getter-managed generation record, `.autogen.jsonc`, so lookup/cleanup is local to the package directory. Hashes recorded inside `.autogen.jsonc` are ownership/tamper-detection facts for generated output: they answer whether a file is still the file getter generated earlier, and they do not provide security trust, repository signing, or external-download validation. The `.autogen.jsonc` `files` map covers getter-written generated output such as `metadata.jsonc`, `Manifest`, generated Lua scripts, and generated `files/...` helper files; it does not include `.autogen.jsonc` itself, avoiding self-referential hashing. The generated repository is generated output: getter may overwrite package directories it previously generated only when matching `.autogen.jsonc` proves ownership. When refresh/overwrite ownership checks pass, getter clears the existing generated package directory contents, then writes the new generated contents into the same package directory, without preserving old unlisted extra files. If clearing any old file or subdirectory fails, the whole refresh/overwrite fails rather than being ignored. If writing new generated contents fails after clearing, the operation fails directly without rollback; the directory may be empty or partially written, and the next refresh continues by clearing and rewriting again. If a target directory exists without a matching generation record, apply reports a conflict and does not overwrite it. A generated-repo package directory missing `.autogen.jsonc` is a conflict rather than something getter automatically claims; without `.autogen.jsonc`, there is no ownership proof. If `.autogen.jsonc` exists but is malformed or schema-invalid, ordinary package discovery/evaluation is still decided by `metadata.jsonc`, but autogen refresh/apply/cleanup/overwrite reports a generated-ownership conflict and does not auto-fix, overwrite, or delete it. When cleanup ownership checks pass, cleanup clears the generated package directory contents, including `.autogen.jsonc` and any unlisted extra files inside it, but does not delete the package directory itself. If clearing any file or subdirectory fails, the whole cleanup/update fails rather than being ignored. Getter does not classify or preserve unlisted extra files in generated package directories because they are outside getter's domain; generated repositories are generated output. User-authored overrides belong in `repo/local/...`, not by hand-editing `repo/autogen/...`. ### Lua update runtime diff --git a/PLAN/provider-host-api-v1.md b/PLAN/provider-host-api-v1.md index 6d75d462..7323e5fd 100644 --- a/PLAN/provider-host-api-v1.md +++ b/PLAN/provider-host-api-v1.md @@ -1,8 +1,8 @@ # PLAN: Stable Lua provider host API v1 -> Status: stable host, provenance, provider module promotion, and cache-backed product runtime update-check slices validated; generated-output migration pending +> Status: stable host, provenance, provider module promotion, cache-backed product runtime update-check, and generated F-Droid provider-module migration implemented in the current branch > Scope: ADR-0012 provider-host API design for F-Droid and GitHub standard Lua modules -> Current boundary: provider modules are promoted to built-in fallbacks, and `update_check_package_issue_action` can evaluate provider-backed packages against Manifest-compatible provider cache entries; generated F-Droid output has not yet been migrated to require them +> Current boundary: provider modules are promoted to built-in fallbacks, `update_check_package_issue_action` can evaluate provider-backed packages against Manifest-compatible provider cache entries, and generated F-Droid output now uses `luaclass.fdroid_android` while plain read-model package evaluation remains provider-host-free ## Progress @@ -14,9 +14,9 @@ - [x] Implement and validate Slice 1 fixture-backed stable namespace harness. - [x] Implement and validate Slice 2 provider cache provenance storage and Manifest-compatible cache hits. - [x] Implement and validate Slice 4 standard provider module promotion tests. -- [x] Add pre-adoption regression coverage locking generated F-Droid output as self-contained/plain-evaluable. +- [x] Add pre-adoption regression coverage before changing generated F-Droid output. - [x] Add product provider-backed package evaluation/update-check operation before generated F-Droid provider-module adoption. -- [ ] Next slice: generated F-Droid provider-module adoption, keeping generated packages usable through the provider-backed update-check path. +- [x] Migrate generated F-Droid output to `luaclass.fdroid_android`, keeping generated packages usable through the provider-backed update-check path. ## Current evidence and constraints @@ -29,6 +29,7 @@ - Current provider-named modules `luaclass.fdroid_android` and `luaclass.github_android_apk` are always-on getter-shipped built-in fallback modules that call operation-installed `getter.provider.*` host functions. Plain package evaluation still does not install provider APIs and fails with a stable host-unavailable error if a package calls a provider module there. - Product runtime update checks deliberately install the stable provider host and runtime hooks before package evaluation. The bridge request shape remains narrow (`package_id`, optional `repository_id`, `installed_version`, and `pin_version`); provider fixture bodies, cache mode, endpoint URLs, and live transport controls are not Flutter/native request fields in this slice. - Until live provider transport is accepted and implemented, the product runtime path is cache-backed: provider-backed packages can use existing provider cache rows only when Manifest-compatible provenance permits them, while cache miss/refresh remains a later provider operation concern. +- Generated F-Droid package directories now keep `metadata.jsonc` minimal, write `Manifest` entries from provider source response SHA-512 provenance, and generate `9999.lua` as a small `luaclass.fdroid_android` call for the default official F-Droid endpoint. Custom generated F-Droid endpoints remain deferred and are rejected by this generator slice rather than silently emitting unsupported Lua. - Repository-local `luaclass/` modules must continue to override getter-shipped built-ins; cross-repository `luaclass` lookup stays unsupported. ## Design goals @@ -332,7 +333,7 @@ Implemented promotion state: 2. `luaclass.fdroid_android` and `luaclass.github_android_apk` call `getter.provider.*`, not `getter_dev.*`. 3. Tests cover provider-backed operation success, repository-local override precedence, and plain package evaluation failing with a stable host-unavailable diagnostic when provider modules are called without installed provider host functions. 4. Provider modules are always-on getter-shipped built-in fallback modules. -5. Generated F-Droid output still does not require `luaclass.fdroid_android`; changing generated output remains a later generator slice. +5. Generated F-Droid output now requires `luaclass.fdroid_android` and is validated through provider-backed update-check operations; plain read-model package evaluation remains host-free. ## Cache and refresh semantics @@ -475,15 +476,22 @@ Done in the implementation branch: Done in the implementation branch: - Updated ADR-0012, `CONTEXT.md`, and this plan with stable provider module/product runtime status. -- Added pre-adoption regression coverage that generated F-Droid output remains self-contained, does not require `luaclass.fdroid_android`, and does not call `getter.provider.*` while normal package evaluation remains plain. - Added a product-facing provider-backed package evaluation/update-check operation by routing only `update_check_package_issue_action` through the stable provider host; read-model `package_eval` remains plain. - Kept the Flutter/native bridge request shape narrow: no provider fixture body, cache mode, endpoint URL, or live transport fields are added to the product payload. - Added runtime tests proving static packages still work, F-Droid/GitHub provider modules work from Manifest-compatible provider cache entries, Manifest-incompatible cache fails closed, and no-update checks do not issue actions. +- Migrated generated F-Droid output to a small built-in provider module script: -Still pending: + ```lua + local fdroid = require("luaclass.fdroid_android") -- Update generated F-Droid output to use `luaclass.fdroid_android` only after the product provider-backed operation exists and tests prove generated packages remain usable through that operation. -- Add generator tests proving generated output does not depend on `getter_dev.*` or repository-local copied modules. + return fdroid.package { + package_name = "org.fdroid.fdroid", + } + ``` + +- Added generator tests proving generated F-Droid output does not depend on `getter_dev.*`, direct `getter.provider.*` calls, repository-local copied modules, embedded catalog candidates, or embedded artifact URLs. +- Added generated-package runtime proof through `update_check_package_issue_action` plus CLI BDD coverage, while keeping plain `package_eval` provider-host-free. +- Kept generated F-Droid custom endpoint output deferred: this slice emits only the default official endpoint shape and rejects non-default endpoint ids/URLs or missing provider source provenance instead of generating unsupported package Lua. ## Review questions for the read-only reviewer diff --git a/core-getter/src/main/rust/getter b/core-getter/src/main/rust/getter index 44cf664f..2349451a 160000 --- a/core-getter/src/main/rust/getter +++ b/core-getter/src/main/rust/getter @@ -1 +1 @@ -Subproject commit 44cf664fa2ae3b98a3d762a40fd30c402233f78b +Subproject commit 2349451aa7c463e8805a82e87797fd47a1b3c36a diff --git a/docs/architecture/adr/0006-package-centric-cli-command-contract.md b/docs/architecture/adr/0006-package-centric-cli-command-contract.md index 3c61a9a4..a97f3e9c 100644 --- a/docs/architecture/adr/0006-package-centric-cli-command-contract.md +++ b/docs/architecture/adr/0006-package-centric-cli-command-contract.md @@ -106,7 +106,7 @@ It maps `apps[]` into getter tracked package state in `main.db`, writes a saniti The first installed-app autogen slice accepts an Android/platform-provided inventory DTO, computes generated fallback packages in Rust, previews before writing, and applies only after explicit `--accept-all` or `--accept ` confirmation. ADR-0012 supersedes the original Phase 1a fixed generated-repository and flat generated package-file storage model. The current architecture writes ordinary package directories to the configured generated repository target (`generated_repository`, default `autogen`) and records package-local `.autogen.jsonc` ownership state. Cleanup/refresh follows ADR-0012 ownership checks instead of preserving modified generated files into `local`. -The first F-Droid autogen CLI/dev slice is fixture-backed and offline: `autogen fdroid preview --index --package ` parses the supplied F-Droid index through getter-owned provider/cache code, maps requested upstream package names to package paths such as `android/f-droid/app/org.fdroid.fdroid`, skips packages covered by higher-priority registered repositories, and returns a generated package-directory preview without writing files. The same command may use `--inventory ` to match installed Android package facts from an installed-inventory fixture against the cached F-Droid catalog before producing the same preview shape; at least one `--package` or `--inventory` input is required, and both may be supplied. `autogen fdroid apply` writes accepted package directories to the configured generated repository and records `.autogen.jsonc` with generator `fdroid-catalog`. This slice does not perform live HTTP, downloads, installer handoff, Flutter/Kotlin provider parsing, or final reusable-provider host API execution; those remain governed by ADR-0012 and later provider/downloader/installer slices. +The first F-Droid autogen CLI/dev slice is fixture-backed and offline: `autogen fdroid preview --index --package ` parses the supplied F-Droid index through getter-owned provider/cache code, maps requested upstream package names to package paths such as `android/f-droid/app/org.fdroid.fdroid`, skips packages covered by higher-priority registered repositories, and returns a generated package-directory preview without writing files. The same command may use `--inventory ` to match installed Android package facts from an installed-inventory fixture against the cached F-Droid catalog before producing the same preview shape; at least one `--package` or `--inventory` input is required, and both may be supplied. `autogen fdroid apply` writes accepted package directories to the configured generated repository and records `.autogen.jsonc` with generator `fdroid-catalog`. Generated F-Droid package output uses `luaclass.fdroid_android` and is validated through the provider-backed runtime update-check path, not plain package evaluation. This slice does not perform live HTTP, downloads, installer handoff, Flutter/Kotlin provider parsing, or custom F-Droid endpoint generation; those remain governed by ADR-0012 and later provider/downloader/installer slices. The first GitHub provider CLI/dev slice is fixture-backed and offline: `provider github releases --owner --repo --releases ` parses a controlled GitHub REST releases JSON response through getter-owned provider/cache code, normalizes matching release assets into update candidates, and returns diagnostics such as `provider.github.asset_not_found` when filters match no assets. `--asset-include` and `--asset-exclude` are regex filters over asset names, prereleases are excluded unless `--include-prereleases` is supplied, and `--refresh` exercises forced provider-cache replacement/stale semantics against the fixture. This command does not generate packages, run Lua provider modules, perform live HTTP/auth/rate-limit handling, create runtime actions, download files, invoke installers, or move any GitHub parsing/filtering into Flutter/Kotlin. @@ -116,7 +116,7 @@ The fixture-backed GitHub latest-commit CLI/dev slice is `provider github latest The old persisted fake downloader slice is retained only as debug scaffolding under `debug fake-task ...`. `debug fake-task submit --request ` accepts `format = "getter-download-request"`, `version = 1`, `package_id`, `executor = "fake"`, and update actions containing at least one `download` action; an optional `install` action creates an abstract install handoff after a successful fake run. `debug fake-task run ` deterministically advances the fake task to `succeeded`; it performs no network I/O and writes no downloaded bytes. `debug fake-task list` returns persisted fake-task summaries from `main.db`. `debug fake-task cancel ` persists cancellation for `queued`/`running` fake tasks, is idempotent for already-canceled tasks, and rejects terminal success/failure with a structured download error. `debug fake-task events --after --limit ` is a pollable debug event contract with a positive `limit`; it is not the ADR-0011 runtime event model. `debug fake-task install-result --status ` records the platform-side result of an abstract debug handoff; the getter-created `requested` handoff state is not accepted as a platform result. This scaffold is not a product task API. -ADR-0011 runtime task debugging uses `runtime script --script `. The script command creates one in-memory `GetterRuntime` for that single CLI process, executes scripted operations such as `issue_action`, `submit_action`, `task_start`, `task_complete_download`, `task_user_result`, `task_remove`, and `task_clean`, then drops all runtime task state when the process exits. It exists so CLI tests can cover runtime remove/clean/control semantics without introducing a task database, daemon, or cross-invocation task promise. Product task submission remains getter-issued opaque `action_id` only through the native bridge/runtime operation path. +ADR-0011 runtime task debugging uses `runtime script --script `. The script command creates one in-memory `GetterRuntime` for that single CLI process, executes scripted operations such as `issue_action`, `update_check_package_issue_action`, `submit_action`, `task_start`, `task_complete_download`, `task_user_result`, `task_remove`, and `task_clean`, then drops all runtime task state when the process exits. The scripted `update_check_package_issue_action` path uses the same narrow product request fields as the bridge and installs provider host APIs inside getter operations; it is BDD/dev evidence, not a separate provider fixture input surface. It exists so CLI tests can cover runtime remove/clean/control semantics and provider-backed update checks without introducing a task database, daemon, or cross-invocation task promise. Product task submission remains getter-issued opaque `action_id` only through the native bridge/runtime operation path. `repo validate ` validates a repository path offline without requiring it to be registered first. It returns `valid`, `diagnostics`, `package_count`, and `network_required = false`; diagnostics are getter-owned structured records with stable codes, message, severity, source path, and optional package id/field. diff --git a/docs/architecture/adr/0012-getter-owned-provider-modules-and-autogen-refresh.md b/docs/architecture/adr/0012-getter-owned-provider-modules-and-autogen-refresh.md index c18c4173..827ac634 100644 --- a/docs/architecture/adr/0012-getter-owned-provider-modules-and-autogen-refresh.md +++ b/docs/architecture/adr/0012-getter-owned-provider-modules-and-autogen-refresh.md @@ -87,7 +87,7 @@ Package version Lua resolves `require("luaclass.")` in this order: Repository-local modules deliberately win over getter-shipped modules. This gives trusted repositories and `local` overlays a normal source-level override path while keeping generated repositories boring: generated package directories do not need to copy shared standard modules, and `.autogen.jsonc` remains package-local ownership proof rather than a repository-root ownership system. Built-in modules are part of the getter binary/source distribution and are not repository source files; repository trust/signing still applies only to repository-provided files. Cross-repository module lookup is not accepted in this slice because it would make package behavior depend on unrelated repository priority and trust boundaries. -Implementation status: getter ships `luaclass.android`, `luaclass.fdroid_android`, and `luaclass.github_android_apk` as built-in fallback modules. The provider-named modules call the stable `getter.provider.*` host namespace and require a provider-backed operation to install those host functions; plain package evaluation does not install them and fails with a stable host-unavailable error if a package calls a provider module there. The product runtime update-check operation `update_check_package_issue_action` installs the stable provider host and runtime hooks, then evaluates package Lua before update selection, so provider-backed packages can issue normal getter-owned update actions from Manifest-compatible provider cache entries. The Flutter/native payload remains package/update oriented (`package_id`, optional `repository_id`, `installed_version`, and `pin_version`); provider fixture bodies, cache mode, endpoint URLs, and live transport controls are not bridge request fields in this slice. Generated F-Droid output does not depend on provider modules yet; switching generated output to `luaclass.fdroid_android` remains a later generator migration after this stable host/module behavior is validated. +Implementation status: getter ships `luaclass.android`, `luaclass.fdroid_android`, and `luaclass.github_android_apk` as built-in fallback modules. The provider-named modules call the stable `getter.provider.*` host namespace and require a provider-backed operation to install those host functions; plain package evaluation does not install them and fails with a stable host-unavailable error if a package calls a provider module there. The product runtime update-check operation `update_check_package_issue_action` installs the stable provider host and runtime hooks, then evaluates package Lua before update selection, so provider-backed packages can issue normal getter-owned update actions from Manifest-compatible provider cache entries. The Flutter/native payload remains package/update oriented (`package_id`, optional `repository_id`, `installed_version`, and `pin_version`); provider fixture bodies, cache mode, endpoint URLs, and live transport controls are not bridge request fields in this slice. Generated F-Droid output now uses `luaclass.fdroid_android` for the default official endpoint, writes provider source response SHA-512 entries into `Manifest`, and remains usable through provider-backed update checks rather than plain read-model package evaluation. ### Stable provider host API v1 direction @@ -122,7 +122,7 @@ return fdroid.package { } ``` -The package path is not duplicated inside the Lua table. Like emerge/ebuilds, getter derives package identity from the package directory path, such as `repo/official/android/f-droid/app/org.fdroid.fdroid` -> `android/f-droid/app/org.fdroid.fdroid`. A directory that directly contains `metadata.jsonc` declares a package boundary. `.autogen.jsonc` does not declare a package boundary; it is only a generated-package ownership record inside a package directory that already has `metadata.jsonc`. If `metadata.jsonc` parses correctly as package metadata, the directory is a valid package directory; if parsing fails, getter reports an invalid package metadata diagnostic for that package path. In both cases that directory is the package path endpoint, so getter does not discover nested packages below it. Package version scripts are discovered from direct child files named `.lua` whose basename does not start with `.`; removing `.lua` yields the literal version string with no SemVer requirement at discovery time. Package version Lua may read helper data under its own package directory's `files/` subtree through a package-scoped host API such as `read_package_file(path)`; the original built-in rejects paths outside that subtree and returns a Lua string without encoding/MIME/JSON/text-vs-binary interpretation, but hook code may still wrap the public `read_package_file()` name because getter core/CLI does not maintain a protective denylist of hookable public functions. File names/formats inside `files/` are package-owned. Package directory contents outside getter's explicit discovery set (`metadata.jsonc`, optional generated-package `.autogen.jsonc`, `Manifest`, enabled direct-child `*.lua`, and `files/`) are outside getter domain entirely, not ignored managed objects; the primary reason is clear responsibility boundaries, with smaller getter-core attack surface as a beneficial side effect. F-Droid display metadata such as name and description comes from the F-Droid catalog; generated metadata/version scripts should not duplicate it unless a hand-written override intentionally does so. Generated content should stay boring. Large provider behavior belongs in reusable Lua classes plus getter host APIs, not in giant generated scripts. +The package path is not duplicated inside the Lua table. Like emerge/ebuilds, getter derives package identity from the package directory path, such as `repo/official/android/f-droid/app/org.fdroid.fdroid` -> `android/f-droid/app/org.fdroid.fdroid`. A directory that directly contains `metadata.jsonc` declares a package boundary. `.autogen.jsonc` does not declare a package boundary; it is only a generated-package ownership record inside a package directory that already has `metadata.jsonc`. If `metadata.jsonc` parses correctly as package metadata, the directory is a valid package directory; if parsing fails, getter reports an invalid package metadata diagnostic for that package path. In both cases that directory is the package path endpoint, so getter does not discover nested packages below it. Package version scripts are discovered from direct child files named `.lua` whose basename does not start with `.`; removing `.lua` yields the literal version string with no SemVer requirement at discovery time. Package version Lua may read helper data under its own package directory's `files/` subtree through a package-scoped host API such as `read_package_file(path)`; the original built-in rejects paths outside that subtree and returns a Lua string without encoding/MIME/JSON/text-vs-binary interpretation, but hook code may still wrap the public `read_package_file()` name because getter core/CLI does not maintain a protective denylist of hookable public functions. File names/formats inside `files/` are package-owned. Package directory contents outside getter's explicit discovery set (`metadata.jsonc`, optional generated-package `.autogen.jsonc`, `Manifest`, enabled direct-child `*.lua`, and `files/`) are outside getter domain entirely, not ignored managed objects; the primary reason is clear responsibility boundaries, with smaller getter-core attack surface as a beneficial side effect. F-Droid display metadata such as name and description comes from the F-Droid catalog; generated metadata/version scripts should not duplicate it unless a hand-written override intentionally does so. Generated content should stay boring: default generated F-Droid output is a minimal `metadata.jsonc`, `Manifest` entries for the provider source response digest(s), and a `9999.lua` that calls `luaclass.fdroid_android` with `package_name`. Large provider behavior belongs in reusable Lua classes plus getter host APIs, not in giant generated scripts. The default `fdroid.package` class may infer common Android behavior: @@ -142,7 +142,7 @@ return fdroid.package { } ``` -`package_name` remains the common default because F-Droid is highly structured and self-describing. F-Droid provider endpoint names come from endpoint ids/directories controlled by getter/repository configuration; the endpoint URL defaults to official F-Droid but can be customized. The model must not make third-party F-Droid endpoints, archive variants, signatures, anti-feature metadata, localized metadata, or channel-like preferences impossible to express later. +`package_name` remains the common default because F-Droid is highly structured and self-describing. F-Droid provider endpoint names come from endpoint ids/directories controlled by getter/repository configuration; the endpoint URL defaults to official F-Droid but can be customized. The first generated-output migration emits only the default official endpoint shape and rejects custom generated `endpoint_id`/`endpoint_url` output until non-default endpoint configuration is accepted through the provider-backed runtime path. The model must not make third-party F-Droid endpoints, archive variants, signatures, anti-feature metadata, localized metadata, or channel-like preferences impossible to express later. ### F-Droid provider endpoint/catalog operations @@ -465,7 +465,7 @@ Use TDD for getter/provider behavior: - F-Droid provider endpoint cache key/freshness behavior; - F-Droid catalog lookup from controlled fixtures; - F-Droid autogen preview/apply output paths, package paths, package-local `.autogen.jsonc` generation records, and repository-priority skip/shadow behavior; -- generated F-Droid package evaluation through `luaclass.fdroid_android` using only `package_name` in the common case; +- generated F-Droid package update checks through `luaclass.fdroid_android` using only `package_name` in the common case, evaluated by provider-backed runtime operations rather than plain package evaluation; - GitHub release response normalization from controlled fixtures; - GitHub asset filter/default behavior; - GitHub latest-commit live/floating semantics; diff --git a/docs/architecture/upgradeall-getter-rewrite-wiki.md b/docs/architecture/upgradeall-getter-rewrite-wiki.md index 19dc45cc..61dae34b 100644 --- a/docs/architecture/upgradeall-getter-rewrite-wiki.md +++ b/docs/architecture/upgradeall-getter-rewrite-wiki.md @@ -1044,11 +1044,12 @@ Room DB 信息： 1. Flutter 调用 getter/native bridge 的 installed-autogen preview 操作。 2. Rust platform adapter 主动调用 Android PackageManager adapter，取得 installed inventory 原始事实。 -3. getter 找出可生成的候选列表。 +3. getter 找出可生成的候选列表；F-Droid 命中的候选由 getter 生成 minimal package directory：`metadata.jsonc`、带 provider source SHA-512 provenance 的 `Manifest`、以及调用 `luaclass.fdroid_android` 的小型 `9999.lua`。 4. UI 展示 getter-owned preview DTO。 5. 用户 yes/no 确认。 6. getter 写入 configured generated repository，默认 `repo/autogen/`。 -7. 生成后不会自动消失。 +7. 后续 update check 通过 getter/provider-backed runtime 安装 `getter.provider.*` 后执行生成的 F-Droid Lua；普通 read-model package eval 不作为 provider-module 生成包的验证路径。 +8. 生成后不会自动消失。 实现进展：Flutter 产品 APK 通过 `app_flutter/android/getter_bridge` 打包一个 slim native bridge library，包含 Rust `api_proxy`、`NativeLib` 和 Android installed-inventory facts provider。`api_proxy` 已提供 installed-autogen preview/apply JNI entrypoints；它们调用 Rust-active platform adapter 扫描 Android PackageManager 原始事实，再调用 getter-owned `getter-operations` 执行 installed-autogen preview/apply。Flutter 已新增 installed-autogen 页面和 `MethodChannelGetterAdapter`，只渲染 getter-owned preview/apply DTO 并把用户接受的 package path 传回 getter；不能引入 Dart-led installed inventory scanner 或在 Dart/Kotlin 中生成 package path。 diff --git a/docs/lua-api/templates.md b/docs/lua-api/templates.md index b5c1ef21..db4a02ae 100644 --- a/docs/lua-api/templates.md +++ b/docs/lua-api/templates.md @@ -74,7 +74,19 @@ return android.package_version { The `files` map covers getter-written generated output such as `metadata.jsonc`, `Manifest`, generated Lua scripts, and generated `files/...` helper files. It excludes `.autogen.jsonc` itself to avoid self-referential hashing; `.autogen.jsonc` validity is checked by parsing/schema validation and matching ownership fields instead. -F-Droid autogen should normally keep generated package metadata/version scripts small and rely on F-Droid catalog metadata plus reusable `luaclass/` helpers for provider behavior. If generated package Lua needs local helper data, autogen writes it under that package directory's `files/` subtree; getter does not assign product semantics to file names or formats inside `files/`. +F-Droid autogen should normally keep generated package metadata/version scripts small and rely on F-Droid catalog metadata plus reusable `luaclass/` helpers for provider behavior. The default generated F-Droid package uses minimal `metadata.jsonc`, a `Manifest` containing provider source response SHA-512 digest entries, and a small `9999.lua` like: + +```lua +#!/bin/upa-lua v1 +-- @generated by UpgradeAll getter autogen (F-Droid provider module) +local fdroid = require("luaclass.fdroid_android") + +return fdroid.package { + package_name = "org.fdroid.fdroid", +} +``` + +Provider-backed update-check operations install `getter.provider.*` for that module. Plain package evaluation remains host-free and is not the validation path for generated F-Droid provider-module output. If generated package Lua needs local helper data, autogen writes it under that package directory's `files/` subtree; getter does not assign product semantics to file names or formats inside `files/`. ## UX contract From 67e99a8858047a1edaf6b9d6a92473b6878adcbf Mon Sep 17 00:00:00 2001 From: Xiangzhe Date: Mon, 29 Jun 2026 08:52:36 +0800 Subject: [PATCH 85/85] feat(app): add installed F-Droid autogen flow --- PLAN/installed-fdroid-autogen-ui-flow.md | 36 ++++ app_flutter/README.md | 4 +- .../upgradeall/GetterBridgeRequestBuilder.kt | 18 ++ .../net/xzos/upgradeall/MainActivity.kt | 26 ++- .../GetterBridgeRequestBuilderTest.kt | 22 ++ .../net/xzos/upgradeall/getter/NativeLib.kt | 2 + app_flutter/lib/cli_getter_adapter.dart | 25 +++ app_flutter/lib/getter_adapter.dart | 53 ++++- app_flutter/lib/main.dart | 137 ++++++++++++- app_flutter/lib/native_getter_adapter.dart | 23 +++ .../test/native_getter_adapter_test.dart | 80 ++++++++ app_flutter/test/widget_test.dart | 193 ++++++++++++++++++ .../src/main/rust/api_proxy/src/lib.rs | 160 +++++++++++++++ .../flutter-ui-feature-parity-and-testing.md | 4 +- .../0007-flutter-getter-bridge-contract.md | 13 +- 15 files changed, 754 insertions(+), 42 deletions(-) create mode 100644 PLAN/installed-fdroid-autogen-ui-flow.md diff --git a/PLAN/installed-fdroid-autogen-ui-flow.md b/PLAN/installed-fdroid-autogen-ui-flow.md new file mode 100644 index 00000000..26076cd1 --- /dev/null +++ b/PLAN/installed-fdroid-autogen-ui-flow.md @@ -0,0 +1,36 @@ +# PLAN: Installed-app F-Droid autogen Flutter bridge/UI flow + +> Status: completed and reviewed +> Branch: `work/flutter-installed-autogen-ui` +> Scope: one small functional slice for exposing cache-backed installed-app F-Droid autogen preview/apply through getter-owned bridge DTOs + +## Boundary + +Flutter renders getter-owned preview/apply DTOs, asks for user confirmation, and passes accepted package ids from displayed candidates back to getter. + +Rust/native bridge owns Android installed inventory scanning, F-Droid catalog/cache lookup, repository coverage, generated package paths/content, diagnostics, and apply semantics. This slice must not add Dart/Kotlin provider parsing, package-id derivation, fixture bodies, endpoint URLs, cache-mode controls, live transport, downloader, or installer behavior to product UI. + +## Implementation path + +- [x] Add a typed product bridge method for installed F-Droid autogen preview that reuses Rust-active Android inventory scanning and calls getter-owned F-Droid autogen with the scanned installed inventory. +- [x] Add matching Dart/native adapter methods and MethodChannel/Kotlin plumbing; apply uses a typed installed-F-Droid bridge method while preserving getter-owned F-Droid apply semantics. +- [x] Extend the Installed Autogen Flutter page with a separate F-Droid installed preview/confirm/apply action while preserving the existing generic installed-autogen flow and bridge-unavailable state. +- [x] Add focused widget/adapter/Rust bridge tests that verify Flutter forwards only scan options/accepted package ids and renders getter DTOs. +- [x] Update existing architecture/app docs for the new typed bridge surface; no new ADR should be needed unless implementation reveals new product semantics. +- [x] Run changed-area validation and reviewer review of the functional branch. +- [x] Commit, merge into `rewrite/flutter-getter-spine`, push clean unsigned commits, and delete the small-plan branch. + +## Validation and review + +- `git diff --check` +- `cd app_flutter && dart format --set-exit-if-changed lib test` +- `cd app_flutter && flutter analyze` +- `cd app_flutter && flutter test` +- `cd app_flutter/android && ./gradlew --no-daemon :app:testDebugUnitTest --tests 'net.xzos.upgradeall.GetterBridgeRequestBuilderTest'` +- `cd core-getter/src/main/rust/api_proxy && cargo fmt --check && cargo test installed_fdroid_preview` +- `just verify-workspace-skeleton` with the local Android SDK/NDK environment exported. +- Reviewer run `7f996399-7e29-4a8c-8d66-aac95ebddfa6` found no blockers. + +## Stop conditions + +Stop for human collaboration only if this requires live provider transport, provider fixture payloads/cache controls in product Flutter requests, new F-Droid endpoint product semantics, or another architecture/ADR decision not already covered by ADR-0002/0007/0009/0012. diff --git a/app_flutter/README.md b/app_flutter/README.md index 50935fd1..bf3d1322 100644 --- a/app_flutter/README.md +++ b/app_flutter/README.md @@ -20,10 +20,10 @@ Do not validate the rewrite with an older local Flutter SDK; older Flutter teste - `FakeGetterAdapter` for deterministic widget tests - `CliGetterAdapter` as a development/integration bridge against the real `getter-cli` JSON envelope - A slim Android `:getter_bridge` library inside `app_flutter/android/getter_bridge` packages the Rust `api_proxy` native library and the no-UI installed-inventory provider classes into the Flutter product APK without depending on the legacy native `:app` UI or old `GetterPort` RPC wrapper surface. -- `MainActivity` exposes a no-UI `net.xzos.upgradeall/getter_bridge` MethodChannel for native bridge plumbing. The legacy migration and installed-autogen methods derive the app-private getter data directory on Android, call Rust JNI entrypoints, and return getter-style JSON envelopes consumed by `MethodChannelGetterAdapter`. +- `MainActivity` exposes a no-UI `net.xzos.upgradeall/getter_bridge` MethodChannel for native bridge plumbing. The legacy migration, installed-autogen, and installed F-Droid autogen methods derive the app-private getter data directory on Android, call Rust JNI entrypoints, and return getter-style JSON envelopes consumed by `MethodChannelGetterAdapter`. - Product manifest permissions include `QUERY_ALL_PACKAGES` per ADR-0009 so the Rust-active Android platform adapter can provide complete installed package inventory facts to getter. -`CliGetterAdapter` is not the final Android production bridge. It exists to keep the getter-owned DTO and error contract executable for dev tests. `MethodChannelGetterAdapter` is the current production bridge slice for direct legacy Room import/report-list and installed-autogen preview/apply: Flutter renders getter-owned DTOs and passes user choices/paths back to getter, but Room mapping, PackageManager scanning, package-id decisions, and `autogen` writes remain in Rust/native getter code. +`CliGetterAdapter` is not the final Android production bridge. It exists to keep the getter-owned DTO and error contract executable for dev tests. `MethodChannelGetterAdapter` is the current production bridge slice for direct legacy Room import/report-list, installed-autogen preview/apply, and cache-backed installed F-Droid autogen preview/apply: Flutter renders getter-owned DTOs and passes user choices/paths back to getter, but Room mapping, PackageManager scanning, F-Droid catalog matching/cache lookup, package-id decisions, and `autogen` writes remain in Rust/native getter code. ## Verification diff --git a/app_flutter/android/app/src/main/kotlin/net/xzos/upgradeall/GetterBridgeRequestBuilder.kt b/app_flutter/android/app/src/main/kotlin/net/xzos/upgradeall/GetterBridgeRequestBuilder.kt index d87a03c3..edf922a8 100644 --- a/app_flutter/android/app/src/main/kotlin/net/xzos/upgradeall/GetterBridgeRequestBuilder.kt +++ b/app_flutter/android/app/src/main/kotlin/net/xzos/upgradeall/GetterBridgeRequestBuilder.kt @@ -8,6 +8,24 @@ object GetterBridgeRequestBuilder { fun runtimeOperationRequest(args: Map<*, *>): String = operationRequest(args) + fun installedAutogenPreviewRequest(args: Map<*, *>): String { + val scanOptions = args["scan_options"] as? Map<*, *> ?: args + return JSONObject() + .put( + "scan_options", + JSONObject() + .put( + "include_system_apps", + scanOptions["include_system_apps"] as? Boolean ?: false, + ) + .put( + "include_self", + scanOptions["include_self"] as? Boolean ?: false, + ), + ) + .toString() + } + fun fdroidAutogenPreviewRequest(args: Map<*, *>): String { val payload = args["payload"] as? Map<*, *> ?: emptyMap() return JSONObject() diff --git a/app_flutter/android/app/src/main/kotlin/net/xzos/upgradeall/MainActivity.kt b/app_flutter/android/app/src/main/kotlin/net/xzos/upgradeall/MainActivity.kt index a2dde2c7..59dbf978 100644 --- a/app_flutter/android/app/src/main/kotlin/net/xzos/upgradeall/MainActivity.kt +++ b/app_flutter/android/app/src/main/kotlin/net/xzos/upgradeall/MainActivity.kt @@ -58,6 +58,17 @@ class MainActivity : FlutterActivity() { nativeLib.applyInstalledAutogen(applyInstalledAutogenRequest(call)) } + "previewInstalledFdroidAutogen" -> runGetterBridge(result) { + nativeLib.previewInstalledFdroidAutogen( + applicationContext, + previewInstalledAutogenRequest(call), + ) + } + + "applyInstalledFdroidAutogen" -> runGetterBridge(result) { + nativeLib.applyInstalledFdroidAutogen(applyInstalledAutogenRequest(call)) + } + "previewFdroidAutogen" -> runGetterBridge(result) { nativeLib.previewFdroidAutogen(previewFdroidAutogenRequest(call)) } @@ -158,21 +169,8 @@ class MainActivity : FlutterActivity() { private fun previewInstalledAutogenRequest(call: MethodCall): String { val args = call.arguments as? Map<*, *> ?: emptyMap() - val scanOptions = args["scan_options"] as? Map<*, *> ?: args - return JSONObject() + return JSONObject(GetterBridgeRequestBuilder.installedAutogenPreviewRequest(args)) .put("data_dir", getterDataDir().absolutePath) - .put( - "scan_options", - JSONObject() - .put( - "include_system_apps", - scanOptions["include_system_apps"] as? Boolean ?: false, - ) - .put( - "include_self", - scanOptions["include_self"] as? Boolean ?: false, - ), - ) .toString() } diff --git a/app_flutter/android/app/src/test/kotlin/net/xzos/upgradeall/GetterBridgeRequestBuilderTest.kt b/app_flutter/android/app/src/test/kotlin/net/xzos/upgradeall/GetterBridgeRequestBuilderTest.kt index 4c4962fc..73827a86 100644 --- a/app_flutter/android/app/src/test/kotlin/net/xzos/upgradeall/GetterBridgeRequestBuilderTest.kt +++ b/app_flutter/android/app/src/test/kotlin/net/xzos/upgradeall/GetterBridgeRequestBuilderTest.kt @@ -51,6 +51,28 @@ class GetterBridgeRequestBuilderTest { assertEquals(0, json.getJSONObject("payload").length()) } + @Test + fun installedAutogenPreviewRequestPreservesOnlyScanOptions() { + val json = JSONObject( + GetterBridgeRequestBuilder.installedAutogenPreviewRequest( + mapOf( + "scan_options" to mapOf( + "include_system_apps" to true, + "include_self" to false, + ), + "index_xml" to "", + "mode" to "force_refresh", + ), + ), + ) + + val scanOptions = json.getJSONObject("scan_options") + assertEquals(true, scanOptions.getBoolean("include_system_apps")) + assertEquals(false, scanOptions.getBoolean("include_self")) + assertEquals(false, json.has("index_xml")) + assertEquals(false, json.has("mode")) + } + @Test fun fdroidAutogenPreviewRequestPreservesGetterPayload() { val json = JSONObject( diff --git a/app_flutter/android/getter_bridge/src/main/kotlin/net/xzos/upgradeall/getter/NativeLib.kt b/app_flutter/android/getter_bridge/src/main/kotlin/net/xzos/upgradeall/getter/NativeLib.kt index 0ce3921e..35d458e5 100644 --- a/app_flutter/android/getter_bridge/src/main/kotlin/net/xzos/upgradeall/getter/NativeLib.kt +++ b/app_flutter/android/getter_bridge/src/main/kotlin/net/xzos/upgradeall/getter/NativeLib.kt @@ -12,7 +12,9 @@ class NativeLib { external fun runServer(context: Context, callback: RunServerCallback): String external fun initializeBridge(context: Context): String external fun previewInstalledAutogen(context: Context, requestJson: String): String + external fun previewInstalledFdroidAutogen(context: Context, requestJson: String): String external fun applyInstalledAutogen(requestJson: String): String + external fun applyInstalledFdroidAutogen(requestJson: String): String external fun previewFdroidAutogen(requestJson: String): String external fun applyFdroidAutogen(requestJson: String): String external fun importLegacyRoomDatabase(requestJson: String): String diff --git a/app_flutter/lib/cli_getter_adapter.dart b/app_flutter/lib/cli_getter_adapter.dart index e796de52..8d1ab066 100644 --- a/app_flutter/lib/cli_getter_adapter.dart +++ b/app_flutter/lib/cli_getter_adapter.dart @@ -94,6 +94,18 @@ class CliGetterAdapter implements GetterAdapter { ); } + @override + Future previewInstalledFdroidAutogen({ + InstalledAutogenScanOptions options = const InstalledAutogenScanOptions(), + }) async { + throw const GetterBridgeException( + GetterError( + code: 'bridge.unsupported', + message: 'CLI adapter cannot scan Android installed F-Droid autogen', + ), + ); + } + @override Future previewFdroidAutogen( Map payload, @@ -119,6 +131,19 @@ class CliGetterAdapter implements GetterAdapter { ); } + @override + Future applyInstalledFdroidAutogen( + InstalledAutogenPreview preview, { + List? acceptedPackageIds, + }) async { + throw const GetterBridgeException( + GetterError( + code: 'bridge.unsupported', + message: 'CLI adapter does not expose installed F-Droid autogen', + ), + ); + } + @override Future checkPackageForUpdate( String packageId, { diff --git a/app_flutter/lib/getter_adapter.dart b/app_flutter/lib/getter_adapter.dart index 7dced2d3..0a187100 100644 --- a/app_flutter/lib/getter_adapter.dart +++ b/app_flutter/lib/getter_adapter.dart @@ -31,6 +31,10 @@ abstract interface class GetterAdapter { List? acceptedPackageIds, }); + Future previewInstalledFdroidAutogen({ + InstalledAutogenScanOptions options = const InstalledAutogenScanOptions(), + }); + Future previewFdroidAutogen( Map payload, ); @@ -40,6 +44,11 @@ abstract interface class GetterAdapter { List? acceptedPackageIds, }); + Future applyInstalledFdroidAutogen( + InstalledAutogenPreview preview, { + List? acceptedPackageIds, + }); + Future checkPackageForUpdate( String packageId, { String? repositoryId, @@ -233,6 +242,13 @@ class FakeGetterAdapter implements GetterAdapter { }); } + @override + Future previewInstalledFdroidAutogen({ + InstalledAutogenScanOptions options = const InstalledAutogenScanOptions(), + }) async { + return previewFdroidAutogen({}); + } + @override Future previewFdroidAutogen( Map payload, @@ -288,6 +304,14 @@ class FakeGetterAdapter implements GetterAdapter { }); } + @override + Future applyInstalledFdroidAutogen( + InstalledAutogenPreview preview, { + List? acceptedPackageIds, + }) { + return applyFdroidAutogen(preview, acceptedPackageIds: acceptedPackageIds); + } + @override Future checkPackageForUpdate( String packageId, { @@ -953,6 +977,17 @@ class InstalledAutogenPreview { factory InstalledAutogenPreview.fromJson(Map json) { final scan = _jsonMapOrNull(json['scan'], 'autogen.scan'); + final diagnosticsJson = [ + ..._jsonList( + json['diagnostics'] ?? const [], + 'autogen.diagnostics', + ), + if (scan != null) + ..._jsonList( + scan['diagnostics'] ?? const [], + 'autogen.scan.diagnostics', + ), + ]; return InstalledAutogenPreview( operation: _jsonString(json['operation'], 'autogen.operation'), targetRepoId: _jsonString( @@ -979,17 +1014,13 @@ class InstalledAutogenPreview { InstalledAutogenSkip.fromJson(_jsonMap(skip, 'autogen.skip')), ) .toList(growable: false), - diagnostics: - _jsonList( - scan?['diagnostics'] ?? json['diagnostics'], - 'autogen.diagnostics', - ) - .map( - (diagnostic) => PlatformDiagnosticSummary.fromJson( - _jsonMap(diagnostic, 'autogen.diagnostic'), - ), - ) - .toList(growable: false), + diagnostics: diagnosticsJson + .map( + (diagnostic) => PlatformDiagnosticSummary.fromJson( + _jsonMap(diagnostic, 'autogen.diagnostic'), + ), + ) + .toList(growable: false), scanStats: scan == null || scan['stats'] == null ? null : InstalledAutogenScanStats.fromJson( diff --git a/app_flutter/lib/main.dart b/app_flutter/lib/main.dart index 794430e7..45ebacd4 100644 --- a/app_flutter/lib/main.dart +++ b/app_flutter/lib/main.dart @@ -45,9 +45,21 @@ class AppKeys { static const previewInstalledAutogen = ValueKey( 'action.preview_installed_autogen', ); + static const previewInstalledFdroidAutogen = ValueKey( + 'action.preview_installed_fdroid_autogen', + ); static const applyInstalledAutogen = ValueKey( 'action.apply_installed_autogen', ); + static const installedAutogenConfirmDialog = ValueKey( + 'dialog.installed_autogen_confirm', + ); + static const cancelInstalledAutogenApply = ValueKey( + 'action.cancel_installed_autogen_apply', + ); + static const confirmInstalledAutogenApply = ValueKey( + 'action.confirm_installed_autogen_apply', + ); static const updateCheckStatus = ValueKey( 'state.update_check_status', ); @@ -112,6 +124,8 @@ class AppKeys { ValueKey('state.task_event.$cursor'); static ValueKey autogenCandidateRow(String packageId) => ValueKey('state.autogen_candidate.$packageId'); + static ValueKey autogenConfirmCandidateRow(String packageId) => + ValueKey('state.autogen_confirm_candidate.$packageId'); static ValueKey autogenSkipRow(String packageId) => ValueKey('state.autogen_skip.$packageId'); static ValueKey autogenDiagnosticRow(int index) => @@ -592,18 +606,39 @@ class _InstalledAutogenPageState extends State { InstalledAutogenApplyResult? _applyResult; GetterError? _error; bool _running = false; + bool _previewUsesFdroidApply = false; - Future _previewInstalledAutogen() async { + Future _previewInstalledAutogen() { + return _runPreview( + () => widget.getter.previewInstalledAutogen(), + usesFdroidApply: false, + ); + } + + Future _previewInstalledFdroidAutogen() { + return _runPreview( + () => widget.getter.previewInstalledFdroidAutogen(), + usesFdroidApply: true, + ); + } + + Future _runPreview( + Future Function() previewer, { + required bool usesFdroidApply, + }) async { setState(() { _running = true; + _preview = null; + _previewUsesFdroidApply = usesFdroidApply; _error = null; _applyResult = null; }); try { - final preview = await widget.getter.previewInstalledAutogen(); + final preview = await previewer(); if (!mounted) return; setState(() { _preview = preview; + _previewUsesFdroidApply = usesFdroidApply; _running = false; }); } on GetterBridgeException catch (error) { @@ -625,20 +660,35 @@ class _InstalledAutogenPageState extends State { } } - Future _applyInstalledAutogen() async { + Future _confirmAndApplyInstalledAutogen() async { final preview = _preview; if (preview == null) return; + final confirmed = await showDialog( + context: context, + builder: (context) => _InstalledAutogenApplyDialog(preview: preview), + ); + if (confirmed != true || !mounted) return; + await _applyInstalledAutogen(preview); + } + + Future _applyInstalledAutogen(InstalledAutogenPreview preview) async { setState(() { _running = true; _error = null; }); try { - final result = await widget.getter.applyInstalledAutogen( - preview, - acceptedPackageIds: preview.candidates - .map((candidate) => candidate.packageId) - .toList(growable: false), - ); + final acceptedPackageIds = preview.candidates + .map((candidate) => candidate.packageId) + .toList(growable: false); + final result = _previewUsesFdroidApply + ? await widget.getter.applyInstalledFdroidAutogen( + preview, + acceptedPackageIds: acceptedPackageIds, + ) + : await widget.getter.applyInstalledAutogen( + preview, + acceptedPackageIds: acceptedPackageIds, + ); if (!mounted) return; setState(() { _applyResult = result; @@ -682,6 +732,17 @@ class _InstalledAutogenPageState extends State { icon: const Icon(Icons.manage_search), label: Text(_running ? 'Working…' : 'Preview installed autogen'), ), + const SizedBox(height: 8), + ElevatedButton.icon( + key: AppKeys.previewInstalledFdroidAutogen, + onPressed: _running || !canUseBridge + ? null + : _previewInstalledFdroidAutogen, + icon: const Icon(Icons.apps_outage), + label: Text( + _running ? 'Working…' : 'Preview installed F-Droid autogen', + ), + ), if (!canUseBridge) const Padding( padding: EdgeInsets.only(top: 12), @@ -703,7 +764,7 @@ class _InstalledAutogenPageState extends State { padding: const EdgeInsets.only(top: 16), child: Text( key: AppKeys.installedAutogenError, - '${_error!.code}: ${_error!.message}', + _formatGetterError(_error!), ), ), if (preview != null) ...[ @@ -784,7 +845,7 @@ class _InstalledAutogenPageState extends State { key: AppKeys.applyInstalledAutogen, onPressed: _running || preview.candidates.isEmpty ? null - : _applyInstalledAutogen, + : _confirmAndApplyInstalledAutogen, icon: const Icon(Icons.check), label: const Text('Apply all candidates'), ), @@ -815,6 +876,60 @@ class _InstalledAutogenPageState extends State { } } +String _formatGetterError(GetterError error) { + final detail = error.detail; + if (detail == null || detail.trim().isEmpty) { + return '${error.code}: ${error.message}'; + } + return '${error.code}: ${error.message}\n$detail'; +} + +class _InstalledAutogenApplyDialog extends StatelessWidget { + const _InstalledAutogenApplyDialog({required this.preview}); + + final InstalledAutogenPreview preview; + + @override + Widget build(BuildContext context) { + return AlertDialog( + key: AppKeys.installedAutogenConfirmDialog, + title: const Text('Apply generated packages?'), + content: SingleChildScrollView( + child: Column( + mainAxisSize: MainAxisSize.min, + crossAxisAlignment: CrossAxisAlignment.start, + children: [ + Text('Target repository: ${preview.targetRepoId}'), + if (preview.targetRepoPath != null) Text(preview.targetRepoPath!), + const SizedBox(height: 12), + const Text('Packages to write:'), + const SizedBox(height: 8), + ...preview.candidates.map( + (candidate) => Padding( + key: AppKeys.autogenConfirmCandidateRow(candidate.packageId), + padding: const EdgeInsets.only(bottom: 4), + child: Text(candidate.packageId), + ), + ), + ], + ), + ), + actions: [ + TextButton( + key: AppKeys.cancelInstalledAutogenApply, + onPressed: () => Navigator.of(context).pop(false), + child: const Text('Cancel'), + ), + FilledButton( + key: AppKeys.confirmInstalledAutogenApply, + onPressed: () => Navigator.of(context).pop(true), + child: const Text('Apply'), + ), + ], + ); + } +} + class LogsPage extends StatelessWidget { const LogsPage({super.key}); diff --git a/app_flutter/lib/native_getter_adapter.dart b/app_flutter/lib/native_getter_adapter.dart index 17ea0751..5b6c51b7 100644 --- a/app_flutter/lib/native_getter_adapter.dart +++ b/app_flutter/lib/native_getter_adapter.dart @@ -86,6 +86,17 @@ class MethodChannelGetterAdapter extends FakeGetterAdapter { return InstalledAutogenApplyResult.fromJson(data); } + @override + Future previewInstalledFdroidAutogen({ + InstalledAutogenScanOptions options = const InstalledAutogenScanOptions(), + }) async { + final data = await _invokeGetterData( + 'previewInstalledFdroidAutogen', + {'scan_options': options.toJson()}, + ); + return InstalledAutogenPreview.fromJson(data); + } + @override Future previewFdroidAutogen( Map payload, @@ -109,6 +120,18 @@ class MethodChannelGetterAdapter extends FakeGetterAdapter { return InstalledAutogenApplyResult.fromJson(data); } + @override + Future applyInstalledFdroidAutogen( + InstalledAutogenPreview preview, { + List? acceptedPackageIds, + }) async { + final data = await _invokeGetterData( + 'applyInstalledFdroidAutogen', + _autogenApplyArguments(preview, acceptedPackageIds), + ); + return InstalledAutogenApplyResult.fromJson(data); + } + Future> invokeReadOperation( String operation, { Map payload = const {}, diff --git a/app_flutter/test/native_getter_adapter_test.dart b/app_flutter/test/native_getter_adapter_test.dart index d68c7e79..7532aed2 100644 --- a/app_flutter/test/native_getter_adapter_test.dart +++ b/app_flutter/test/native_getter_adapter_test.dart @@ -99,6 +99,86 @@ void main() { expect(result.applied.single.packageId, 'android/app/com.example.autogen'); }); + test('native installed F-Droid preview sends only scan options', () async { + MethodCall? captured; + TestDefaultBinaryMessengerBinding.instance.defaultBinaryMessenger + .setMockMethodCallHandler(channel, (call) async { + captured = call; + return jsonEncode({ + 'ok': true, + 'command': 'autogen installed fdroid preview', + 'data': _fdroidPreviewJson(), + 'warnings': [], + }); + }); + + const adapter = MethodChannelGetterAdapter(channel: channel); + final preview = await adapter.previewInstalledFdroidAutogen( + options: const InstalledAutogenScanOptions(includeSystemApps: true), + ); + + expect(captured!.method, 'previewInstalledFdroidAutogen'); + final args = (captured!.arguments as Map) + .cast(); + expect(args.keys, ['scan_options']); + expect(args['scan_options'], { + 'include_system_apps': true, + 'include_self': false, + }); + expect(preview.operation, 'fdroid.autogen.preview'); + expect( + preview.candidates.single.packageId, + 'android/f-droid/app/org.fdroid.fdroid', + ); + }); + + test('native installed F-Droid apply uses typed bridge method', () async { + MethodCall? captured; + TestDefaultBinaryMessengerBinding.instance.defaultBinaryMessenger + .setMockMethodCallHandler(channel, (call) async { + captured = call; + return jsonEncode({ + 'ok': true, + 'command': 'autogen installed fdroid apply', + 'data': { + 'target_repo_id': 'autogen', + 'target_repo_path': '/getter/repo/autogen', + 'applied_count': 1, + 'applied': [ + { + 'package_id': 'android/f-droid/app/org.fdroid.fdroid', + 'output_relative_path': + 'android/f-droid/app/org.fdroid.fdroid', + }, + ], + }, + 'warnings': [], + }); + }); + + const adapter = MethodChannelGetterAdapter(channel: channel); + final preview = InstalledAutogenPreview.fromJson(_fdroidPreviewJson()); + final result = await adapter.applyInstalledFdroidAutogen( + preview, + acceptedPackageIds: const [ + 'android/f-droid/app/org.fdroid.fdroid', + ], + ); + + expect(captured!.method, 'applyInstalledFdroidAutogen'); + final args = (captured!.arguments as Map) + .cast(); + expect(jsonDecode(args['preview_json']! as String), preview.rawJson); + expect(args['acceptance'], { + 'mode': 'packages', + 'package_ids': ['android/f-droid/app/org.fdroid.fdroid'], + }); + expect( + result.applied.single.packageId, + 'android/f-droid/app/org.fdroid.fdroid', + ); + }); + test('native F-Droid autogen forwards getter-owned payloads', () async { final calls = []; TestDefaultBinaryMessengerBinding.instance.defaultBinaryMessenger diff --git a/app_flutter/test/widget_test.dart b/app_flutter/test/widget_test.dart index 1b05d16b..c37f5176 100644 --- a/app_flutter/test/widget_test.dart +++ b/app_flutter/test/widget_test.dart @@ -232,6 +232,17 @@ void main() { await tester.tap(find.byKey(AppKeys.applyInstalledAutogen)); await tester.pumpAndSettle(); + expect(find.byKey(AppKeys.installedAutogenConfirmDialog), findsOneWidget); + expect( + find.byKey( + AppKeys.autogenConfirmCandidateRow('android/app/com.example.autogen'), + ), + findsOneWidget, + ); + + await tester.tap(find.byKey(AppKeys.confirmInstalledAutogenApply)); + await tester.pumpAndSettle(); + expect(find.byKey(AppKeys.installedAutogenApplied), findsOneWidget); expect( find.byKey(AppKeys.autogenAppliedRow('android/app/com.example.autogen')), @@ -242,6 +253,102 @@ void main() { ]); }); + testWidgets( + 'installed autogen route previews and applies installed F-Droid getter DTOs', + (tester) async { + final getter = _AutogenRecordingGetterAdapter(); + await tester.pumpWidget(UpgradeAllApp(getter: getter)); + + await tester.drag(find.byType(ListView).first, const Offset(0, -240)); + await tester.pumpAndSettle(); + await tester.tap(find.byKey(AppKeys.openInstalledAutogen)); + await tester.pumpAndSettle(); + + await tester.tap(find.byKey(AppKeys.previewInstalledFdroidAutogen)); + await tester.pumpAndSettle(); + + expect(find.byKey(AppKeys.installedAutogenPreview), findsOneWidget); + expect( + find.byKey( + AppKeys.autogenCandidateRow('android/f-droid/app/org.fdroid.fdroid'), + ), + findsOneWidget, + ); + + await tester.tap(find.byKey(AppKeys.applyInstalledAutogen)); + await tester.pumpAndSettle(); + + expect(find.byKey(AppKeys.installedAutogenConfirmDialog), findsOneWidget); + expect(find.text('Target repository: autogen'), findsOneWidget); + expect( + find.byKey( + AppKeys.autogenConfirmCandidateRow( + 'android/f-droid/app/org.fdroid.fdroid', + ), + ), + findsOneWidget, + ); + expect(getter.usedInstalledFdroidApply, isFalse); + + await tester.tap(find.byKey(AppKeys.confirmInstalledAutogenApply)); + await tester.pumpAndSettle(); + + expect(getter.usedInstalledFdroidApply, isTrue); + expect(getter.acceptedPackageIds, [ + 'android/f-droid/app/org.fdroid.fdroid', + ]); + expect( + find.byKey( + AppKeys.autogenAppliedRow('android/f-droid/app/org.fdroid.fdroid'), + ), + findsOneWidget, + ); + }, + ); + + testWidgets('installed autogen confirmation cancel does not apply', ( + tester, + ) async { + final getter = _AutogenRecordingGetterAdapter(); + await tester.pumpWidget(UpgradeAllApp(getter: getter)); + + await tester.drag(find.byType(ListView).first, const Offset(0, -240)); + await tester.pumpAndSettle(); + await tester.tap(find.byKey(AppKeys.openInstalledAutogen)); + await tester.pumpAndSettle(); + + await tester.tap(find.byKey(AppKeys.previewInstalledFdroidAutogen)); + await tester.pumpAndSettle(); + await tester.tap(find.byKey(AppKeys.applyInstalledAutogen)); + await tester.pumpAndSettle(); + await tester.tap(find.byKey(AppKeys.cancelInstalledAutogenApply)); + await tester.pumpAndSettle(); + + expect(getter.usedInstalledFdroidApply, isFalse); + expect(getter.acceptedPackageIds, isNull); + expect(find.byKey(AppKeys.installedAutogenApplied), findsNothing); + }); + + testWidgets('installed autogen route renders bridge error detail', ( + tester, + ) async { + await tester.pumpWidget( + const UpgradeAllApp(getter: _AutogenErrorGetterAdapter()), + ); + + await tester.drag(find.byType(ListView).first, const Offset(0, -240)); + await tester.pumpAndSettle(); + await tester.tap(find.byKey(AppKeys.openInstalledAutogen)); + await tester.pumpAndSettle(); + + await tester.tap(find.byKey(AppKeys.previewInstalledFdroidAutogen)); + await tester.pumpAndSettle(); + + expect(find.byKey(AppKeys.installedAutogenError), findsOneWidget); + expect(find.textContaining('autogen.error'), findsOneWidget); + expect(find.textContaining('refresh provider cache'), findsOneWidget); + }); + testWidgets('installed autogen route disables actions without bridge', ( tester, ) async { @@ -483,9 +590,75 @@ class _NoInstalledAutogenGetterAdapter extends FakeGetterAdapter { bool get supportsInstalledAutogen => false; } +class _AutogenErrorGetterAdapter extends FakeGetterAdapter { + const _AutogenErrorGetterAdapter(); + + @override + Future previewInstalledFdroidAutogen({ + InstalledAutogenScanOptions options = const InstalledAutogenScanOptions(), + }) async { + throw const GetterBridgeException( + GetterError( + code: 'autogen.error', + message: 'Getter autogen operation failed', + detail: + 'F-Droid catalog cache is empty; refresh provider cache before installed F-Droid autogen preview', + ), + ); + } +} + class _AutogenRecordingGetterAdapter extends FakeGetterAdapter { List? acceptedPackageIds; + var usedInstalledFdroidApply = false; + + @override + Future previewInstalledFdroidAutogen({ + InstalledAutogenScanOptions options = const InstalledAutogenScanOptions(), + }) async { + return InstalledAutogenPreview.fromJson(const { + 'operation': 'fdroid.autogen.preview', + 'provider': 'fdroid', + 'endpoint_id': 'official', + 'endpoint_url': 'https://f-droid.org/repo', + 'target_repo_id': 'autogen', + 'target_repo_path': '/fake/getter/repo/autogen', + 'scan': { + 'stats': { + 'total_seen': 2, + 'returned': 1, + 'filtered_system': 1, + 'filtered_self': 0, + }, + 'diagnostics': [], + }, + 'summary': { + 'candidate_count': 1, + 'skipped_count': 0, + 'write_count': 1, + 'delete_count': 0, + }, + 'candidates': [ + { + 'package_id': 'android/f-droid/app/org.fdroid.fdroid', + 'kind': 'android', + 'display_name': 'F-Droid', + 'installed_target': { + 'kind': 'android_package', + 'package_name': 'org.fdroid.fdroid', + }, + 'action': 'create', + 'output_relative_path': 'android/f-droid/app/org.fdroid.fdroid', + 'content_hash': 'sha512:fake-fdroid', + 'content': '-- fake generated F-Droid content', + }, + ], + 'skipped': [], + 'diagnostics': [], + }); + } + @override Future applyInstalledAutogen( InstalledAutogenPreview preview, { @@ -497,6 +670,26 @@ class _AutogenRecordingGetterAdapter extends FakeGetterAdapter { acceptedPackageIds: acceptedPackageIds, ); } + + @override + Future applyInstalledFdroidAutogen( + InstalledAutogenPreview preview, { + List? acceptedPackageIds, + }) async { + usedInstalledFdroidApply = true; + this.acceptedPackageIds = acceptedPackageIds; + return InstalledAutogenApplyResult.fromJson(const { + 'target_repo_id': 'autogen', + 'target_repo_path': '/fake/getter/repo/autogen', + 'applied_count': 1, + 'applied': [ + { + 'package_id': 'android/f-droid/app/org.fdroid.fdroid', + 'output_relative_path': 'android/f-droid/app/org.fdroid.fdroid', + }, + ], + }); + } } class _MigrationGetterAdapter extends FakeGetterAdapter { diff --git a/core-getter/src/main/rust/api_proxy/src/lib.rs b/core-getter/src/main/rust/api_proxy/src/lib.rs index f088f6d1..80c76444 100644 --- a/core-getter/src/main/rust/api_proxy/src/lib.rs +++ b/core-getter/src/main/rust/api_proxy/src/lib.rs @@ -2,7 +2,9 @@ extern crate jni; use getter::operations::autogen::{self, AutogenAcceptance, AutogenOperationError}; use getter::operations::fdroid_autogen; +use getter::operations::fdroid_catalog::{self, FdroidEndpointConfig}; use getter::operations::legacy_room::{self, LegacyRoomOperationError}; +use getter::operations::provider_cache::ProviderCacheMode; use getter::operations::read_model::{self, ReadModelOperationError}; use getter::operations::runtime as runtime_operations; use getter::rpc::server::run_server_hanging; @@ -206,6 +208,25 @@ pub extern "C" fn Java_net_xzos_upgradeall_getter_NativeLib_previewInstalledAuto java_string_or_fallback(&mut env, response) } +#[no_mangle] +pub extern "C" fn Java_net_xzos_upgradeall_getter_NativeLib_previewInstalledFdroidAutogen< + 'local, +>( + mut env: JNIEnv<'local>, + _: JObject<'local>, + context: JObject<'local>, + request_json: JString<'local>, +) -> JString<'local> { + let command = "autogen installed fdroid preview"; + let response = match jstring_to_string(&mut env, &request_json) + .and_then(|raw| preview_installed_fdroid_autogen(&mut env, &context, &raw)) + { + Ok(data) => success_envelope(command, data), + Err(error) => operation_error_envelope(command, error), + }; + java_string_or_fallback(&mut env, response) +} + #[no_mangle] pub extern "C" fn Java_net_xzos_upgradeall_getter_NativeLib_applyInstalledAutogen<'local>( mut env: JNIEnv<'local>, @@ -222,6 +243,22 @@ pub extern "C" fn Java_net_xzos_upgradeall_getter_NativeLib_applyInstalledAutoge java_string_or_fallback(&mut env, response) } +#[no_mangle] +pub extern "C" fn Java_net_xzos_upgradeall_getter_NativeLib_applyInstalledFdroidAutogen<'local>( + mut env: JNIEnv<'local>, + _: JObject<'local>, + request_json: JString<'local>, +) -> JString<'local> { + let command = "autogen installed fdroid apply"; + let response = match jstring_to_string(&mut env, &request_json) + .and_then(|raw| apply_fdroid_autogen(&raw)) + { + Ok(data) => success_envelope(command, data), + Err(error) => operation_error_envelope(command, error), + }; + java_string_or_fallback(&mut env, response) +} + #[no_mangle] pub extern "C" fn Java_net_xzos_upgradeall_getter_NativeLib_previewFdroidAutogen<'local>( mut env: JNIEnv<'local>, @@ -354,6 +391,54 @@ fn preview_installed_autogen( Ok(preview) } +fn preview_installed_fdroid_autogen( + env: &mut JNIEnv<'_>, + context: &JObject<'_>, + request_json: &str, +) -> Result { + init_android_integrations(env, context).map_err(BridgeOperationError::Initialize)?; + let request: PreviewInstalledAutogenRequest = serde_json::from_str(request_json) + .map_err(|source| BridgeOperationError::InvalidRequest(source.to_string()))?; + let scan = scan_installed_inventory(request.scan_options)?; + preview_installed_fdroid_autogen_from_scan(&request.data_dir, scan) +} + +fn preview_installed_fdroid_autogen_from_scan( + data_dir: &Path, + scan: upgradeall_platform_adapter::InstalledInventoryScanResult, +) -> Result { + let db = open_main_db(data_dir)?; + let cache_db = open_cache_db(data_dir)?; + fdroid_catalog::read_or_refresh_fdroid_catalog( + &cache_db, + FdroidEndpointConfig::default(), + ProviderCacheMode::UseCached, + || Err("F-Droid catalog cache is empty; refresh provider cache before installed F-Droid autogen preview".to_owned()), + ) + .map_err(|source| BridgeOperationError::Autogen(source.to_string()))?; + let inventory: getter::core::autogen::InstalledInventory = + serde_json::to_value(&scan.inventory) + .and_then(serde_json::from_value) + .map_err(|source| BridgeOperationError::PlatformMalformed(source.to_string()))?; + let payload = json!({ "installed_inventory": inventory }); + let mut preview = fdroid_autogen::preview_fdroid_packages_json( + data_dir, + &db, + &cache_db, + &payload.to_string(), + )?; + if let Some(object) = preview.as_object_mut() { + object.insert( + "scan".to_owned(), + json!({ + "stats": scan.stats, + "diagnostics": scan.diagnostics, + }), + ); + } + Ok(preview) +} + fn apply_installed_autogen(request_json: &str) -> Result { let request: ApplyInstalledAutogenRequest = serde_json::from_str(request_json) .map_err(|source| BridgeOperationError::InvalidRequest(source.to_string()))?; @@ -864,6 +949,81 @@ mod tests { .is_some()); } + #[test] + fn installed_fdroid_preview_reuses_platform_inventory_as_provider_request() { + let temp = tempfile::tempdir().unwrap(); + let data_dir = temp.path().join("data"); + let cache_db = open_cache_db(&data_dir).unwrap(); + fdroid_catalog::read_or_refresh_fdroid_catalog( + &cache_db, + FdroidEndpointConfig::default(), + ProviderCacheMode::UseCached, + || Ok(fdroid_fixture().to_owned()), + ) + .unwrap(); + let scan = upgradeall_platform_adapter::InstalledInventoryScanResult { + inventory: upgradeall_platform_adapter::InstalledInventory::new(vec![ + upgradeall_platform_adapter::InstalledInventoryItem::AndroidPackage { + package_name: "org.fdroid.fdroid".to_owned(), + label: Some("F-Droid".to_owned()), + version_name: Some("1.20.0".to_owned()), + version_code: Some(1_020_000), + }, + ]), + stats: upgradeall_platform_adapter::InstalledInventoryScanStats { + total_seen: 2, + returned: 1, + filtered_system: 1, + filtered_self: 0, + }, + diagnostics: vec![upgradeall_platform_adapter::PlatformDiagnostic { + code: "platform.note".to_owned(), + message: "scan diagnostic".to_owned(), + detail: None, + }], + }; + + let preview = preview_installed_fdroid_autogen_from_scan(&data_dir, scan).unwrap(); + + assert_eq!(preview["operation"], "fdroid.autogen.preview"); + assert_eq!(preview["source"], "cache"); + assert_eq!(preview["scan"]["stats"]["returned"], 1); + assert_eq!(preview["scan"]["diagnostics"][0]["code"], "platform.note"); + assert_eq!( + preview["candidates"][0]["package_id"], + "android/f-droid/app/org.fdroid.fdroid" + ); + } + + #[test] + fn installed_fdroid_preview_requires_cached_catalog() { + let temp = tempfile::tempdir().unwrap(); + let data_dir = temp.path().join("data"); + let scan = upgradeall_platform_adapter::InstalledInventoryScanResult { + inventory: upgradeall_platform_adapter::InstalledInventory::new(vec![ + upgradeall_platform_adapter::InstalledInventoryItem::AndroidPackage { + package_name: "org.fdroid.fdroid".to_owned(), + label: None, + version_name: None, + version_code: None, + }, + ]), + stats: upgradeall_platform_adapter::InstalledInventoryScanStats { + total_seen: 1, + returned: 1, + filtered_system: 0, + filtered_self: 0, + }, + diagnostics: Vec::new(), + }; + + let error = preview_installed_fdroid_autogen_from_scan(&data_dir, scan).unwrap_err(); + + let detail = error.to_string(); + assert!(detail.contains("F-Droid catalog cache is empty")); + assert!(!detail.contains("index_xml")); + } + #[test] fn read_operation_lists_repositories_and_evaluates_packages() { let temp = tempfile::tempdir().unwrap(); diff --git a/docs/app/flutter-ui-feature-parity-and-testing.md b/docs/app/flutter-ui-feature-parity-and-testing.md index fa54800c..846b7144 100644 --- a/docs/app/flutter-ui-feature-parity-and-testing.md +++ b/docs/app/flutter-ui-feature-parity-and-testing.md @@ -17,7 +17,7 @@ The Flutter UI should preserve these user-visible product capabilities unless ex - App detail with version/source/artifact selection. - App settings/editing. - Repository/source visibility. -- Installed-app autogen preview and confirmation. +- Installed-app autogen preview and confirmation, including the cache-backed installed F-Droid autogen path. - Download task view and controls. - Settings. - Logs. @@ -80,7 +80,7 @@ The first Flutter implementation slice is intentionally a shell, not product log - `CliGetterAdapter` exercises a real getter data directory through the `getter-cli` JSON envelope for development/integration tests. - ADR-0007 documents the bridge contract and explicitly treats the CLI adapter as a test/development bridge, not the final Android production path. - Product decisions such as repository resolution, updates, migrations, storage, and downloads still belong in Rust getter. -- Installed-autogen product flows must call getter/native bridge operations that use the Rust-active Android platform adapter from ADR-0009; Flutter should not lead PackageManager inventory scanning through a Dart MethodChannel API. +- Installed-autogen product flows must call getter/native bridge operations that use the Rust-active Android platform adapter from ADR-0009; Flutter should not lead PackageManager inventory scanning through a Dart MethodChannel API. The installed F-Droid autogen product flow forwards only scan options and accepted package ids; F-Droid catalog cache lookup, package-path derivation, repository coverage, generated content, and cache-miss diagnostics stay in getter/native bridge code. - CI/release APK artifacts must be built from `app_flutter`, not from the legacy `:app` module. - The app detail update button may call getter's typed update-check operation, receive a getter-issued opaque `action_id`, submit that `action_id`, and open Downloads. Flutter must not assemble or echo action payloads. - The downloads route may render getter task/event DTOs read-only and refresh after `RuntimeNotification.task_changed`, but it must not implement a Dart download task state machine, retry policy, or installer semantics. Current-state runtime queries remain authoritative. diff --git a/docs/architecture/adr/0007-flutter-getter-bridge-contract.md b/docs/architecture/adr/0007-flutter-getter-bridge-contract.md index 0c49af6d..848bc4f8 100644 --- a/docs/architecture/adr/0007-flutter-getter-bridge-contract.md +++ b/docs/architecture/adr/0007-flutter-getter-bridge-contract.md @@ -54,9 +54,18 @@ applyFdroidAutogen(preview, acceptedPackages) The initial `previewFdroidAutogen` payload is the fixture-backed/offline ADR-0006 F-Droid autogen request shape (`index_xml`, `package_names`, optional `installed_inventory`, endpoint fields, and cache mode) so bridge/widget tests can exercise the cross-boundary contract without live HTTP. Flutter may forward user/test input and render getter-owned candidate/skip/diagnostic DTOs, but it must not parse F-Droid indexes, derive UpgradeAll package paths from F-Droid package names, decide shadowing, or generate Lua. -The Android product APK packages a slim `:getter_bridge` library under `app_flutter/android/getter_bridge`. It builds the Rust `api_proxy` cdylib and includes only the no-UI native bridge / installed-inventory provider classes needed by the Flutter product path, avoiding the legacy native `:app` UI and old `GetterPort` hub/RPC wrapper surface. `MainActivity` exposes a no-UI `net.xzos.upgradeall/getter_bridge` MethodChannel that derives the app-private getter data directory and forwards migration, installed-autogen, F-Droid autogen, read-model, and runtime requests to JNI entrypoints returning getter-style JSON envelopes. +The fifth accepted API surface adds the product installed-app F-Droid autogen bridge shape. It follows ADR-0009's Rust-active platform adapter direction and deliberately does not expose the fixture/offline F-Droid request payload to product Flutter UI: -Internally, Rust/native bridge code scans Android inventory through the platform adapter for installed-autogen, then asks getter-owned shared autogen operations to plan/apply output in the configured generated repository target (`generated_repository`, default `autogen`). F-Droid autogen bridge calls the same getter-owned F-Droid catalog/autogen operations used by the CLI. `MethodChannelGetterAdapter` consumes the returned getter-style JSON envelopes. Flutter renders getter-owned preview/apply DTOs and scan/provider diagnostics, then passes the package ids from displayed accepted preview candidates back to getter on apply; it must not expose a product Dart `InstalledInventoryPlatform` scanner, convert Android/F-Droid package names into package ids, parse provider data, or generate package content. +```text +previewInstalledFdroidAutogen(scanOptions) +applyInstalledFdroidAutogen(preview, acceptedPackages) +``` + +`previewInstalledFdroidAutogen` scans Android installed inventory in the native/getter bridge, converts the platform inventory into getter's installed-inventory DTO, and asks getter-owned F-Droid autogen operations to build a preview from the cached default F-Droid catalog. Until live provider transport is accepted, a missing provider cache is reported as a getter-owned autogen error/diagnostic; Flutter must not provide `index_xml`, endpoint URL, cache mode, or raw provider payload fields to this product method. + +The Android product APK packages a slim `:getter_bridge` library under `app_flutter/android/getter_bridge`. It builds the Rust `api_proxy` cdylib and includes only the no-UI native bridge / installed-inventory provider classes needed by the Flutter product path, avoiding the legacy native `:app` UI and old `GetterPort` hub/RPC wrapper surface. `MainActivity` exposes a no-UI `net.xzos.upgradeall/getter_bridge` MethodChannel that derives the app-private getter data directory and forwards migration, installed-autogen, installed F-Droid autogen, F-Droid autogen, read-model, and runtime requests to JNI entrypoints returning getter-style JSON envelopes. + +Internally, Rust/native bridge code scans Android inventory through the platform adapter for installed-autogen, then asks getter-owned shared autogen operations to plan/apply output in the configured generated repository target (`generated_repository`, default `autogen`). The installed F-Droid autogen product bridge uses the same Rust-active inventory scan but routes matching package names through getter-owned F-Droid catalog/autogen operations backed by `cache.db`; it does not accept provider fixture bodies, endpoint URLs, or cache-mode controls from Flutter. F-Droid autogen bridge calls the same getter-owned F-Droid catalog/autogen operations used by the CLI for development/test payloads. `MethodChannelGetterAdapter` consumes the returned getter-style JSON envelopes. Flutter renders getter-owned preview/apply DTOs and scan/provider diagnostics, then passes the package ids from displayed accepted preview candidates back to getter on apply; it must not expose a product Dart `InstalledInventoryPlatform` scanner, convert Android/F-Droid package names into package ids, parse provider data, or generate package content. `loadSnapshot()` composes smaller getter-owned read-model operations into the UI shell's first snapshot DTO. In the Android/native product path, `MethodChannelGetterAdapter` calls `readOperation` for `repository_list`, `tracked_package_list`, and `package_eval`; Rust getter reads SQLite repository/tracked-package state and evaluates registered Lua packages. Flutter may parse and combine those returned DTOs for rendering, but must not perform repository resolution, Lua validation/evaluation, version comparison, migration mapping, or update selection in Dart. `readMigrationReports()` must go through a getter operation such as `legacy report-list`; Flutter must not inspect getter's data-directory layout directly. Runtime task rendering must use ADR-0011 runtime task snapshot APIs and opaque action/task controls; Flutter must not synthesize task states, retry policy, installer behavior, or update decisions.