Resolve OCL canonicals via $resolveReference instead of guessing#267
Closed
italomacedo wants to merge 9 commits into
Closed
Resolve OCL canonicals via $resolveReference instead of guessing#267italomacedo wants to merge 9 commits into
italomacedo wants to merge 9 commits into
Conversation
Adds a client for OCL's $resolveReference operation, which resolves a canonical
URL (or relative OCL path) to the repo that holds it. This is the building block
for OCL #2261 ("replace repo/repo version requests with $resolveReference"):
today tx/ocl finds repos by hand-building /orgs/{owner}/... paths or by
heuristic text search.
Design notes:
- Namespace is derived from the existing `org=` source config (/orgs/{org}/),
falling back to `/` (global). A FHIR request carries no namespace, so it is
bound per source entry. A malformed namespace throws rather than silently
degrading to global, which would look fine while resolving in the wrong
context.
- References are grouped by namespace and sent one POST per group using the
`namespace` query parameter. OCL discourages the per-reference `namespace`
field, so it is never emitted in the body.
- Results are positional; if OCL returns a different count than we sent, the
whole group is discarded rather than risk attributing a resolution to the
wrong canonical.
- $resolveReference is auth-gated on every instance probed, while the /orgs/
enumeration it replaces is public. Without a token the resolver stays disabled
so callers keep their existing path, and 404/401/403 disable it permanently.
Plain .js rather than the .cjs+stub convention used elsewhere in tx/ocl: jest's
collectCoverageFrom globs **/*.js and not **/*.cjs, so a .cjs module here would
be invisible to coverage.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
cm-ocl filtered candidate repo paths with startsWith('/orgs/') in three places,
which silently excluded every user-owned repo. OCL repos are owned by an org OR
a user, so /users/{user}/sources/{id}/ is equally valid and was being discarded:
mappings on a user-owned source simply never resolved, with no error to explain
why.
Replaces the three checks with isOclRepoPath(), which accepts both owner types.
This needs no token and no $resolveReference support on the instance.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
#candidateSourceUrls answers "which OCL repo holds this canonical?" by heuristic:
it derives a search token from the URL, pages through /orgs/{org}/sources/?q=...,
and matches canonical_url on the results. That is guesswork, it costs several
requests, and it only finds what the search surfaces.
$resolveReference answers the same question authoritatively in one call, so ask
OCL first and keep the search as the fallback. This is the ConceptMap half of
OCL #2261.
Deliberately unchanged:
- Discovery (/orgs/ -> /orgs/{org}/sources/ enumeration) still stands.
$resolveReference resolves a known reference; it cannot list what exists.
- The hand-built path builders in cs-ocl/vs-ocl are left alone. They are
synchronous third-choice fallbacks behind OCL-supplied concepts_url /
expansion_url, so routing them through an async resolver would restructure
snapshot building for a path that rarely runs.
Behaviour is unchanged unless a token is configured: without one the resolver
stays disabled and #candidateSourceUrls keeps using the search exactly as before.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
`token=` is not a new parameter -- tx/library.js has always parsed it and the README has always listed it -- but it now carries weight it did not before: it is what enables $resolveReference. The docs said nothing about that, so a reader had no way to know why the operation was or wasn't being used. Adds a "Canonical resolution via $resolveReference" section covering: - why a token is needed at all: $resolveReference is authenticated on every OCL instance probed, while the /orgs/ enumeration it replaces is public - that the token stays optional and no existing config breaks: with no token the resolver is disabled and callers fall back to the previous path, shown as a before/after table - when the resolver disables itself (404/401/403) versus logs and continues (400) - how the namespace is derived from `org=`, and that OCL's fallthrough to the Global URL Registry means a namespace is a preference, not a boundary - that it resolves a known reference and cannot list, so discovery is unchanged Also warns against committing a real token (data/library.yml is tracked and there is no env-var interpolation), and notes that the documented coverage command reports on the one-line .js stubs rather than the .cjs implementations, since collectCoverageFrom globs **/*.js only. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The previous wording told readers to keep the token uncommitted, full stop. That is wrong for a private deployment repository that exists to carry environment config, which is a normal way to run this server. Describe the actual risk instead -- no env-var interpolation exists, so the token lives in the tracked library YAML; committing it puts a live credential in history, readable by anyone with repo access and revocable only by rotating it at OCL -- and let the deployer judge. Keep the hard line where it belongs: not in a public repo, and never travelling upstream with a contribution. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
$resolveReference and the source search return the same thing, so there was no way -- from the HTTP response or from the logs -- to tell which one ran. The resolver only logged on failure, which made the feature unobservable in exactly the case you care about: success. Logs one line per outcome: - resolved: canonical -> repo, with namespace and whether an OCL url registry entry was involved - tried but unresolved: says it is falling back to the search - not enabled (typically no token): logged once per provider, not per lookup, since for a tokenless deployment that is the expected steady state rather than an error The registry-entry detail also gives us the first real evidence of what url_registry_entry looks like in practice, which the namespace sandbox design currently rests on assumption for. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
searchConceptMaps lower-cased every search param value. Every consumer compared
through #norm(), which lower-cases anyway, so nothing noticed -- and the source
search is a text query, which tolerated it too.
$resolveReference matches the canonical exactly. OCL has no
.../fhir/codesystem/alcoolspa_uso_mangara, only .../fhir/CodeSystem/AlcoolSPA_uso_Mangara,
so every lookup came back unresolved and silently fell back to the search:
[OCL] $resolveReference did not resolve
https://mangara.hsl.org.br/fhir/codesystem/alcoolspa_uso_mangara;
falling back to source search
Found by running the server against the live OCL instance -- the mocked tests
could not catch it, because they assert on whatever casing the test itself feeds
in. findConceptMapForTranslation was never affected: it takes the system from its
caller and never went through this map.
With the casing preserved, the same request now resolves:
[OCL] $resolveReference resolved
https://mangara.hsl.org.br/fhir/CodeSystem/AlcoolSPA_uso_Mangara
-> /orgs/HL7/sources/AlcoolSPA_uso_Mangara/ (namespace /, via namespace)
and in 1.3s rather than 5.5s.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…cept
searchConceptMaps has no concept code -- it asks "what mappings does this source
have?" -- but it answered that with the per-concept endpoint: list the concepts,
then issue one request per concept and union the results.
Two problems, measured against the live OCL instance:
- Correctness. The concept listing is capped at maxSearchPages (10 x 100 = 1000).
LOINC has 184683 concepts, so it only ever saw 0.5% of them, and any mapping on
a concept past the first 1000 was silently invisible.
- Cost. Up to 1000 sequential requests per source. loinc, snomed, v3-ActCode and
cmed all timed out (>45s) and returned nothing at all; in production the same
thing shows up as 504s on BRCBO/BRCIAP2.
{source}/mappings/ answers the actual question in one paginated call. Verified
equivalent before switching -- for AlcoolSPA_uso_Mangara both paths return the
identical mapping set (2 mappings), one in 1 request instead of 4:
/orgs/HL7/sources/AlcoolSPA_uso_Mangara/|NAOINFORMADO -[NARROWER-THAN]-> /orgs/SNOMED/sources/gps/|1157031005
/orgs/HL7/sources/AlcoolSPA_uso_Mangara/|NAO -[NARROWER-THAN]-> /orgs/SNOMED/sources/gps/|373067005
Measured end to end:
http://loinc.org timeout(45s+) -> 0.44s, 1 ConceptMap
http://snomed.info/sct timeout(45s+) -> 0.98s, 5 ConceptMaps
v3-ActCode timeout(45s+) -> 0.18s
cmed timeout(45s+) -> 0.16s
loinc and snomed previously returned nothing, so this restores results rather
than merely speeding them up.
The per-concept endpoint stays where it belongs: findConceptMapForTranslation has
a sourceCode and asks about that one concept, so its single targeted request is
already the right call. Untouched.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…elling
I built the resolver's result handling from the documented example, which shows
result as {type, short_code, url}. A real response from oclapi2.ips.hsl.org.br
for ["/orgs/MS/sources/BRTabelaSUS/concepts/1948/"] carries far more:
"result": {
"short_code": "BRTabelaSUS", "name": "BRTabelaSUS",
"url": "/orgs/MS/sources/BRTabelaSUS/",
"owner": "MS", "owner_type": "Organization", "owner_url": "/orgs/MS/",
"version": "HEAD", "source_type": "Dictionary", "type": "Source",
"canonical_url": "https://terminologia.saude.gov.br/fhir/CodeSystem/BRTabelaSUS",
"checksums": { "standard": "...", "smart": "..." }
}
Note "type": "Source", not "Source Version" as the doc's example shows.
Three consequences:
- canonical_url is authoritative and was being thrown away. cm-ocl recorded the
repo's canonical by echoing back whatever the caller asked with, so a caller
using a different spelling (http vs https, say) poisoned _canonicalBySourceUrl
with a canonical the repo does not actually claim. Now surfaced as .canonical
and used for that bookkeeping.
- owner_type is surfaced too: OCL states the owner kind rather than leaving us to
infer it from the path.
- The test fixture was modelled on the doc's example -- i.e. on a shape OCL does
not return. Rebuilt from the captured response, plus a test asserting the
verbatim payload so the doc drifting from reality cannot quietly mislead us
again.
This also retires a claim I made earlier: that vs-ocl's owner/source -> canonical
lookup could not use $resolveReference because result had no canonical. It does.
Not worth switching (a direct GET of the source is one request either way, and
the URL registry is empty on this instance), but the stated reason was wrong.
The raw result object is still passed through untouched, so version and checksums
remain available to callers.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Closed
6 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Teaches the OCL integration to ask OCL which repo holds a canonical URL, using OCL's
$resolveReferenceoperation, instead of guessing. This is the client half of OCL issue #2261, "replace repo/repo
version requests with
$resolveReference".Today
cm-oclanswers "which repo holds this canonical?" heuristically: it derives a searchtoken from the URL, pages through
/orgs/{org}/sources/?q=..., and matchescanonical_urlonwhatever the search surfaces.
$resolveReferenceanswers it authoritatively in one call.Scope is deliberately narrow.
$resolveis not touched: that is Terminology Ecosysteminfrastructure at the coordination layer, and nothing here may make OCL's resolution path depend
on a FHIRsmith facade being in front of it.
registry/is untouched and its tests still pass.Nothing changes without a token
$resolveReferenceis authenticated on every OCL instance probed, while the/orgs/enumerationit replaces is public. So the resolver is constructed disabled when no
token=is configured,and every caller falls back to the path it used before. Existing
ocl:source lines behaveexactly as they do today; adding a token is opt-in.
token=is not new —tx/library.jshasalways parsed it — it simply carries weight it did not before.
ocl:https://ocl.example.org/orgs/{org}/sources/?q=searchocl:https://ocl.example.org|token=...$resolveReferenceThe resolver also disables itself permanently on
404(not implemented) or401/403(credentials rejected), so an instance without the operation degrades to today's behaviour rather
than failing.
Commits
Each stands alone and is separately reviewable:
$resolveReferenceclienttx/ocl/resolve/reference-resolver.jscm-oclfiltered candidates withstartsWith('/orgs/')in three places, silently discarding every/users/{user}/...repo. Needs no token.searchConceptMapslower-cased every param.#norm()lower-cases anyway so nothing noticed, and the text search tolerated it — but$resolveReferencematches exactly, so every lookup silently fell backcanonical_urlThe concept×mappings loop
searchConceptMapshas no concept code — it asks "what mappings does this source have?" — butanswered that with the per-concept endpoint: list the concepts, then one request per concept.
maxSearchPages(10 × 100 = 1000). LOINC has184,683 concepts, so it only ever saw 0.5% of them; any mapping past the first 1000 was
silently invisible. No error — just fewer results.
loinc,snomed,v3-ActCodeandcmedall timed out and returned nothing at all.
{source}/mappings/answers the actual question in one paginated call. Verified equivalent beforeswitching — for
AlcoolSPA_uso_Mangaraboth paths return the identical mapping set, one in 1request instead of 4.
http://loinc.orghttp://snomed.info/sctv3-ActCodecmedloincandsnomedpreviously returned nothing, so this restores results rather than merelyspeeding them up.
The per-concept endpoint stays where it belongs:
findConceptMapForTranslationhas asourceCodeand asks about that one concept, so its single targeted request is already right. Untouched.
Testing
170 tests, positive and negative.
tx/ocl/resolve/reference-resolver.jsis at 100% statements /lines / functions, 96.8% branches:
jest.config.jsis untouched. Note it globscollectCoverageFrom: ['**/*.js']and not**/*.cjs, so the.cjsmodules intx/oclare invisible to coverage and the command documentedin
tx/ocl/README.mdreports on their one-line.jsre-export stubs. That is why the new moduleis plain
.js— it makes the number real without changing the config. Thecm-ocl.cjsedits arecovered behaviourally but cannot contribute to a coverage figure. README now says so.
Verified end to end against a live OCL instance, not just mocks:
Worth stating plainly: the casing bug, the 0.5% ceiling and the echoed canonical were all
invisible to the mocked tests — a mock returns what you told it to return. All three surfaced
only by running the server against real OCL. The response-shape tests are now built from a
captured payload rather than from the documented example, which omits most fields and reports
type: "Source Version"where OCL actually returns"Source".Notes for reviewers
org=(/orgs/{org}/), else/(global, OCL's owndefault). A FHIR request carries no namespace, so it is bound per source entry. Explicit
namespace=would needparseOclConfigintx/library.jsand is left for later.a namespaced resolve can still return another owner's repo. Sandboxing would have to be enforced
client-side; not implemented, and documented as such.
$resolveReferenceresolves a known reference and cannot list, sothe
/orgs/enumeration stays. This PR does not make the integration scale to all of OCL Online— that needs lazy provider registration in
tx/library.js.discourages the per-reference
namespacefield, so it is never emitted.attributing a resolution to the wrong canonical.
🤖 Generated with Claude Code