Skip to content

Resolve OCL canonicals via $resolveReference instead of guessing#267

Closed
italomacedo wants to merge 9 commits into
mainfrom
ocl-resolve-reference
Closed

Resolve OCL canonicals via $resolveReference instead of guessing#267
italomacedo wants to merge 9 commits into
mainfrom
ocl-resolve-reference

Conversation

@italomacedo

Copy link
Copy Markdown
Collaborator

What

Teaches the OCL integration to ask OCL which repo holds a canonical URL, using OCL's
$resolveReference
operation, instead of guessing. This is the client half of OCL issue #2261, "replace repo/repo
version requests with $resolveReference"
.

Today cm-ocl answers "which repo holds this canonical?" heuristically: it derives a search
token from the URL, pages through /orgs/{org}/sources/?q=..., and matches canonical_url on
whatever the search surfaces. $resolveReference answers it authoritatively in one call.

Scope is deliberately narrow. $resolve is not touched: that is Terminology Ecosystem
infrastructure at the coordination layer, and nothing here may make OCL's resolution path depend
on a FHIRsmith facade being in front of it. registry/ is untouched and its tests still pass.

Nothing changes without a token

$resolveReference is authenticated on every OCL instance probed, while the /orgs/ enumeration
it replaces is public. So the resolver is constructed disabled when no token= is configured,
and every caller falls back to the path it used before. Existing ocl: source lines behave
exactly as they do today; adding a token is opt-in. token= is not new — tx/library.js has
always parsed it — it simply carries weight it did not before.

Configuration Canonical → repo Behaviour
ocl:https://ocl.example.org /orgs/{org}/sources/?q= search unchanged
ocl:https://ocl.example.org|token=... $resolveReference authoritative, falls back on failure

The resolver also disables itself permanently on 404 (not implemented) or 401/403
(credentials rejected), so an instance without the operation degrades to today's behaviour rather
than failing.

Commits

Each stands alone and is separately reviewable:

$resolveReference client new tx/ocl/resolve/reference-resolver.js
user-owned repos cm-ocl filtered candidates with startsWith('/orgs/') in three places, silently discarding every /users/{user}/... repo. Needs no token.
wire into ConceptMap resolver first, search as fallback
docs the token's role, and what the operation does not do
log which path ran resolver and search return the same thing, so success was unobservable
canonical casing searchConceptMaps lower-cased every param. #norm() lower-cases anyway so nothing noticed, and the text search tolerated it — but $resolveReference matches exactly, so every lookup silently fell back
source mappings see below
canonical_url the repo's own canonical was being discarded in favour of echoing the request back

The concept×mappings loop

searchConceptMaps has no concept code — it asks "what mappings does this source have?" — but
answered that with the per-concept endpoint: list the concepts, then one request per concept.

  • Correctness. The concept listing is capped at maxSearchPages (10 × 100 = 1000). LOINC has
    184,683 concepts, so it only ever saw 0.5% of them; any mapping past the first 1000 was
    silently invisible. No error — just fewer results.
  • Cost. Up to 1000 sequential requests per source. loinc, snomed, v3-ActCode and cmed
    all timed out and returned nothing at all.

{source}/mappings/ answers the actual question in one paginated call. Verified equivalent before
switching — for AlcoolSPA_uso_Mangara both paths return the identical mapping set, one in 1
request instead of 4.

canonical before after
http://loinc.org timeout (45s+) 0.44s, 1 ConceptMap
http://snomed.info/sct timeout (45s+) 0.98s, 5 ConceptMaps
v3-ActCode timeout (45s+) 0.18s
cmed timeout (45s+) 0.16s

loinc and snomed previously returned nothing, so this restores results rather than merely
speeding them up.

The per-concept endpoint stays where it belongs: findConceptMapForTranslation has a sourceCode
and asks about that one concept, so its single targeted request is already right. Untouched.

Testing

170 tests, positive and negative. tx/ocl/resolve/reference-resolver.js is at 100% statements /
lines / functions, 96.8% branches
:

npx jest --testPathPattern "tests/ocl" --coverage --collectCoverageFrom="tx/ocl/resolve/**/*.js"

jest.config.js is untouched. Note it globs collectCoverageFrom: ['**/*.js'] and not
**/*.cjs, so the .cjs modules in tx/ocl are invisible to coverage and the command documented
in tx/ocl/README.md reports on their one-line .js re-export stubs. That is why the new module
is plain .js — it makes the number real without changing the config. The cm-ocl.cjs edits are
covered behaviourally but cannot contribute to a coverage figure. README now says so.

Verified end to end against a live OCL instance, not just mocks:

[OCL] $resolveReference resolved
  https://mangara.hsl.org.br/fhir/CodeSystem/AlcoolSPA_uso_Mangara
  -> /orgs/HL7/sources/AlcoolSPA_uso_Mangara/ (namespace /, via namespace)

Worth stating plainly: the casing bug, the 0.5% ceiling and the echoed canonical were all
invisible to the mocked tests
— a mock returns what you told it to return. All three surfaced
only by running the server against real OCL. The response-shape tests are now built from a
captured payload rather than from the documented example, which omits most fields and reports
type: "Source Version" where OCL actually returns "Source".

Notes for reviewers

  • Namespace is derived from the existing org= (/orgs/{org}/), else / (global, OCL's own
    default). A FHIR request carries no namespace, so it is bound per source entry. Explicit
    namespace= would need parseOclConfig in tx/library.js and is left for later.
  • Namespace is a preference, not a boundary: OCL falls through to the Global URL Registry, so
    a namespaced resolve can still return another owner's repo. Sandboxing would have to be enforced
    client-side; not implemented, and documented as such.
  • Discovery is unchanged. $resolveReference resolves a known reference and cannot list, so
    the /orgs/ enumeration stays. This PR does not make the integration scale to all of OCL Online
    — that needs lazy provider registration in tx/library.js.
  • Batching groups references by namespace and uses the request-level query parameter; OCL
    discourages the per-reference namespace field, so it is never emitted.
  • Results are positional, so a count mismatch discards the whole group rather than risk
    attributing a resolution to the wrong canonical.

🤖 Generated with Claude Code

italomacedo and others added 9 commits July 15, 2026 14:38
Adds a client for OCL's $resolveReference operation, which resolves a canonical
URL (or relative OCL path) to the repo that holds it. This is the building block
for OCL #2261 ("replace repo/repo version requests with $resolveReference"):
today tx/ocl finds repos by hand-building /orgs/{owner}/... paths or by
heuristic text search.

Design notes:

- Namespace is derived from the existing `org=` source config (/orgs/{org}/),
  falling back to `/` (global). A FHIR request carries no namespace, so it is
  bound per source entry. A malformed namespace throws rather than silently
  degrading to global, which would look fine while resolving in the wrong
  context.
- References are grouped by namespace and sent one POST per group using the
  `namespace` query parameter. OCL discourages the per-reference `namespace`
  field, so it is never emitted in the body.
- Results are positional; if OCL returns a different count than we sent, the
  whole group is discarded rather than risk attributing a resolution to the
  wrong canonical.
- $resolveReference is auth-gated on every instance probed, while the /orgs/
  enumeration it replaces is public. Without a token the resolver stays disabled
  so callers keep their existing path, and 404/401/403 disable it permanently.

Plain .js rather than the .cjs+stub convention used elsewhere in tx/ocl: jest's
collectCoverageFrom globs **/*.js and not **/*.cjs, so a .cjs module here would
be invisible to coverage.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
cm-ocl filtered candidate repo paths with startsWith('/orgs/') in three places,
which silently excluded every user-owned repo. OCL repos are owned by an org OR
a user, so /users/{user}/sources/{id}/ is equally valid and was being discarded:
mappings on a user-owned source simply never resolved, with no error to explain
why.

Replaces the three checks with isOclRepoPath(), which accepts both owner types.
This needs no token and no $resolveReference support on the instance.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
#candidateSourceUrls answers "which OCL repo holds this canonical?" by heuristic:
it derives a search token from the URL, pages through /orgs/{org}/sources/?q=...,
and matches canonical_url on the results. That is guesswork, it costs several
requests, and it only finds what the search surfaces.

$resolveReference answers the same question authoritatively in one call, so ask
OCL first and keep the search as the fallback. This is the ConceptMap half of
OCL #2261.

Deliberately unchanged:

- Discovery (/orgs/ -> /orgs/{org}/sources/ enumeration) still stands.
  $resolveReference resolves a known reference; it cannot list what exists.
- The hand-built path builders in cs-ocl/vs-ocl are left alone. They are
  synchronous third-choice fallbacks behind OCL-supplied concepts_url /
  expansion_url, so routing them through an async resolver would restructure
  snapshot building for a path that rarely runs.

Behaviour is unchanged unless a token is configured: without one the resolver
stays disabled and #candidateSourceUrls keeps using the search exactly as before.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
`token=` is not a new parameter -- tx/library.js has always parsed it and the
README has always listed it -- but it now carries weight it did not before: it is
what enables $resolveReference. The docs said nothing about that, so a reader had
no way to know why the operation was or wasn't being used.

Adds a "Canonical resolution via $resolveReference" section covering:

- why a token is needed at all: $resolveReference is authenticated on every OCL
  instance probed, while the /orgs/ enumeration it replaces is public
- that the token stays optional and no existing config breaks: with no token the
  resolver is disabled and callers fall back to the previous path, shown as a
  before/after table
- when the resolver disables itself (404/401/403) versus logs and continues (400)
- how the namespace is derived from `org=`, and that OCL's fallthrough to the
  Global URL Registry means a namespace is a preference, not a boundary
- that it resolves a known reference and cannot list, so discovery is unchanged

Also warns against committing a real token (data/library.yml is tracked and there
is no env-var interpolation), and notes that the documented coverage command
reports on the one-line .js stubs rather than the .cjs implementations, since
collectCoverageFrom globs **/*.js only.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The previous wording told readers to keep the token uncommitted, full stop. That
is wrong for a private deployment repository that exists to carry environment
config, which is a normal way to run this server.

Describe the actual risk instead -- no env-var interpolation exists, so the token
lives in the tracked library YAML; committing it puts a live credential in
history, readable by anyone with repo access and revocable only by rotating it at
OCL -- and let the deployer judge. Keep the hard line where it belongs: not in a
public repo, and never travelling upstream with a contribution.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
$resolveReference and the source search return the same thing, so there was no
way -- from the HTTP response or from the logs -- to tell which one ran. The
resolver only logged on failure, which made the feature unobservable in exactly
the case you care about: success.

Logs one line per outcome:

- resolved: canonical -> repo, with namespace and whether an OCL url registry
  entry was involved
- tried but unresolved: says it is falling back to the search
- not enabled (typically no token): logged once per provider, not per lookup,
  since for a tokenless deployment that is the expected steady state rather than
  an error

The registry-entry detail also gives us the first real evidence of what
url_registry_entry looks like in practice, which the namespace sandbox design
currently rests on assumption for.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
searchConceptMaps lower-cased every search param value. Every consumer compared
through #norm(), which lower-cases anyway, so nothing noticed -- and the source
search is a text query, which tolerated it too.

$resolveReference matches the canonical exactly. OCL has no
.../fhir/codesystem/alcoolspa_uso_mangara, only .../fhir/CodeSystem/AlcoolSPA_uso_Mangara,
so every lookup came back unresolved and silently fell back to the search:

  [OCL] $resolveReference did not resolve
    https://mangara.hsl.org.br/fhir/codesystem/alcoolspa_uso_mangara;
    falling back to source search

Found by running the server against the live OCL instance -- the mocked tests
could not catch it, because they assert on whatever casing the test itself feeds
in. findConceptMapForTranslation was never affected: it takes the system from its
caller and never went through this map.

With the casing preserved, the same request now resolves:

  [OCL] $resolveReference resolved
    https://mangara.hsl.org.br/fhir/CodeSystem/AlcoolSPA_uso_Mangara
    -> /orgs/HL7/sources/AlcoolSPA_uso_Mangara/ (namespace /, via namespace)

and in 1.3s rather than 5.5s.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…cept

searchConceptMaps has no concept code -- it asks "what mappings does this source
have?" -- but it answered that with the per-concept endpoint: list the concepts,
then issue one request per concept and union the results.

Two problems, measured against the live OCL instance:

- Correctness. The concept listing is capped at maxSearchPages (10 x 100 = 1000).
  LOINC has 184683 concepts, so it only ever saw 0.5% of them, and any mapping on
  a concept past the first 1000 was silently invisible.
- Cost. Up to 1000 sequential requests per source. loinc, snomed, v3-ActCode and
  cmed all timed out (>45s) and returned nothing at all; in production the same
  thing shows up as 504s on BRCBO/BRCIAP2.

{source}/mappings/ answers the actual question in one paginated call. Verified
equivalent before switching -- for AlcoolSPA_uso_Mangara both paths return the
identical mapping set (2 mappings), one in 1 request instead of 4:

  /orgs/HL7/sources/AlcoolSPA_uso_Mangara/|NAOINFORMADO -[NARROWER-THAN]-> /orgs/SNOMED/sources/gps/|1157031005
  /orgs/HL7/sources/AlcoolSPA_uso_Mangara/|NAO          -[NARROWER-THAN]-> /orgs/SNOMED/sources/gps/|373067005

Measured end to end:

  http://loinc.org        timeout(45s+) -> 0.44s, 1 ConceptMap
  http://snomed.info/sct  timeout(45s+) -> 0.98s, 5 ConceptMaps
  v3-ActCode              timeout(45s+) -> 0.18s
  cmed                    timeout(45s+) -> 0.16s

loinc and snomed previously returned nothing, so this restores results rather
than merely speeding them up.

The per-concept endpoint stays where it belongs: findConceptMapForTranslation has
a sourceCode and asks about that one concept, so its single targeted request is
already the right call. Untouched.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…elling

I built the resolver's result handling from the documented example, which shows
result as {type, short_code, url}. A real response from oclapi2.ips.hsl.org.br
for ["/orgs/MS/sources/BRTabelaSUS/concepts/1948/"] carries far more:

  "result": {
    "short_code": "BRTabelaSUS", "name": "BRTabelaSUS",
    "url": "/orgs/MS/sources/BRTabelaSUS/",
    "owner": "MS", "owner_type": "Organization", "owner_url": "/orgs/MS/",
    "version": "HEAD", "source_type": "Dictionary", "type": "Source",
    "canonical_url": "https://terminologia.saude.gov.br/fhir/CodeSystem/BRTabelaSUS",
    "checksums": { "standard": "...", "smart": "..." }
  }

Note "type": "Source", not "Source Version" as the doc's example shows.

Three consequences:

- canonical_url is authoritative and was being thrown away. cm-ocl recorded the
  repo's canonical by echoing back whatever the caller asked with, so a caller
  using a different spelling (http vs https, say) poisoned _canonicalBySourceUrl
  with a canonical the repo does not actually claim. Now surfaced as .canonical
  and used for that bookkeeping.
- owner_type is surfaced too: OCL states the owner kind rather than leaving us to
  infer it from the path.
- The test fixture was modelled on the doc's example -- i.e. on a shape OCL does
  not return. Rebuilt from the captured response, plus a test asserting the
  verbatim payload so the doc drifting from reality cannot quietly mislead us
  again.

This also retires a claim I made earlier: that vs-ocl's owner/source -> canonical
lookup could not use $resolveReference because result had no canonical. It does.
Not worth switching (a direct GET of the source is one request either way, and
the URL registry is empty on this instance), but the stated reason was wrong.

The raw result object is still passed through untouched, so version and checksums
remain available to callers.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant