Skip to content

fix: end DS activation race that broke system-realm auth at startup (#1882)#44

Merged
craigpnnl merged 3 commits into
developfrom
fix/gadp012-system-realm-activation-order
Jul 10, 2026
Merged

fix: end DS activation race that broke system-realm auth at startup (#1882)#44
craigpnnl merged 3 commits into
developfrom
fix/gadp012-system-realm-activation-order

Conversation

@craigpnnl

Copy link
Copy Markdown
Contributor

Problem

The GridAPPS-D develop tag failed to start nondeterministically (machine and startup-path dependent), reported in GRIDAPPSD/GOSS-GridAPPS-D#1882. On the failing path, goss-core-server died during OSGi Declarative Services activation:

ERROR pnnl.goss.core.server.impl.GridOpticsServer - The start method has thrown an exception
com.northconcepts.exception.SystemException: Realm [UnauthTokenBasedRealm] was unable to
find account data for the submitted AuthenticationToken [UsernamePasswordToken - system]
    at GridOpticsServer.start(GridOpticsServer.java:448)

Root cause

The DependencyManager to Declarative Services migration left the Shiro realm set populated asynchronously by two racing binders. GridOpticsServer.start() (@activate) opens the broker and immediately authenticates createConnection("system", "manager"). The AT_LEAST_ONE cardinality guard only ensured some realm was bound, not the system-authenticating one. When the broker authenticated system before SystemBasedRealm was in the set, only UnauthTokenBasedRealm answered, returned null (by design), and startup died.

Fix

  • Mark SystemBasedRealm with a realm.type=system service property.
  • Add a mandatory, target-filtered @Reference(target="(realm.type=system)") to both the SecurityManager component (Activator, gating service publication) and GridOpticsServer (gating @Activate/broker-connect). Declarative Services now holds activation until the system-authenticating realm is bound.
  • Collapse the realm-set write to a single owner: SecurityManagerRealmHandler no longer calls setRealms(), so a partial write can no longer transiently drop the system realm.
  • The filter lives as a shared compile-time constant on GossRealm.

UnauthTokenBasedRealm returning null for system is intentional and unchanged. The fix is ordering only: no client credential or permission contract changes.

Testing

SystemRealmActivationOrderTest (7 cases): behavioral (system auth fails with only a non-system realm wired, succeeds once the system realm binds, * permission survives a later binding) plus DS-descriptor contract assertions (property/target present, handler no longer references the SecurityManager). ./gradlew :pnnl.goss.core:build is green.

Coverage note: true multi-bundle container activation ordering is not unit-testable here (no Pax Exam configured). The tests assert the wiring/auth invariants the container ordering depends on; a live Felix boot on the affected machine confirms end-to-end sequencing.

Reviews

Reviewed for code quality and security (fail-closed confirmed, no fail-open window, no dropped realm).

Related

craigpnnl added 3 commits July 9, 2026 17:28
GridOpticsServer authenticated the system/manager principal against the
Shiro realm set before the system-authenticating realm was registered, so
platform startup died nondeterministically with "Realm UnauthTokenBasedRealm
was unable to find account data for system" (GridOPTICS/GOSS-GridAPPS-D#1882).
The DependencyManager to Declarative Services migration left the realm set
populated asynchronously by two racing binders, and the AT_LEAST_ONE guard
only ensured some realm was present, not the system realm.

Mark SystemBasedRealm with a realm.type=system service property, and add a
mandatory target-filtered @reference on it to both the SecurityManager
component (Activator, gating service publication) and GridOpticsServer
(gating @Activate/broker-connect). Collapse the realm-set write to a single
owner: SecurityManagerRealmHandler no longer calls setRealms(), so a partial
write can no longer transiently drop the system realm. The filter string is a
shared compile-time constant on GossRealm.

UnauthTokenBasedRealm returning null for system is intentional and unchanged;
the fix is ordering only. No client credential or permission contract changes.
Behavioral tests: system authentication fails when only a non-system realm is
wired (reproduces #1882), succeeds once the system realm is in the set, and
retains the "*" permission after a later realm binds. Contract tests parse the
bnd-emitted OSGI-INF descriptors to assert the realm.type=system property and
the (realm.type=system) reference targets are present and that the handler no
longer references the SecurityManager. Descriptor tests fail (not skip) when
run under Gradle with the bundle artifact absent.
Pre-existing spotlessCheck violation on develop, unrelated to the activation
fix but folded in so the branch CI is green. Closes #41.
@craigpnnl craigpnnl merged commit 214a5a0 into develop Jul 10, 2026
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant