Add OpenCRE as a map analysis resource for issue #469

[Bornunique911]

Related Issue

This branch addresses:

• #469 Add OpenCRE itself to mapanalysis
• Add OpenCRE itself to mapanalysis OWASP/OpenCRE#469

Problem

OpenCRE already acts as the central requirement graph, but it is not currently exposed as a selectable resource in map_analysis.

That means users can compare external standards to each other, but they cannot directly compare a standard against OpenCRE itself. This limits a useful workflow described in the issue, such as understanding how a standard maps into OpenCRE coverage.

Solution

This branch adds OpenCRE itself as a supported map-analysis resource.

1. Add OpenCRE to the standards list

The /rest/v1/standards response now includes OpenCRE so it can be selected like other map-analysis resources.

2. Add direct map-analysis support for OpenCRE

When one side of /rest/v1/map_analysis is OpenCRE, the backend now builds the result directly from CRE-to-standard links instead of sending the request through the normal queued standard-to-standard flow.

This works by:

• treating OpenCRE as the set of CRE documents in the local database
• matching the other selected standard through its linked CREs
• building a direct overlap result from those shared CRE relationships

3. Keep the implementation narrow

This branch only adds the minimal backend behavior needed for OpenCRE to behave as a map-analysis resource. It does not change the general gap-analysis queue model for other standard-to-standard requests.

Testing

Executed focused validation with:

./venv/bin/python -m pytest application/tests/web_main_test.py -k 'standards_from_db or supports_opencre_as_standard' -q

[ghost]

Rooviewer   Follow task

The merge-from-main issue is now fixed -- the OpenCRE dispatch block has been restored. Two previously flagged issues remain open.

• Merge dropped OpenCRE dispatch in map_analysis() -- fixed in ce3ce06; the if OPENCRE_STANDARD_NAME in standards: block is back
• Cache bypass for OpenCRE requests -- the OpenCRE code path in map_analysis() writes results to the gap-analysis cache but never reads from it, so every request recomputes from scratch
• N+1 queries in _get_opencre_documents -- one get_CREs call per CRE row in the database; should be batched into a single query

Previous reviews

• ce3ce06: Review #1
• 3ca88b8: Review #2

_{Mention @roomote in a comment to request specific changes to this pull request or fix all unresolved issues.}

[ghost]

The OpenCRE branch bypasses the cache lookup that the normal flow performs. _build_direct_cre_overlap_map_analysis writes to the cache via add_gap_analysis_result (line 403), but this code path never reads from it -- gap_analysis_exists / get_gap_analysis_result are only reached when OPENCRE_STANDARD_NAME is not in standards. Every OpenCRE map-analysis request will recompute the full result from scratch, which is especially costly given that _get_opencre_documents issues one query per CRE row. Adding a cache check before calling _build_direct_cre_overlap_map_analysis would fix this.

_{Fix it with Roo Code or mention @roomote and request a fix.}

[ghost]

This issues one get_CREs(internal_id=...) query per CRE row in the database. If there are N CREs, that is N+1 total queries (one to list all rows, then one per row to hydrate links). On a production-sized dataset this will be slow and put unnecessary load on the database for every uncached OpenCRE map-analysis request. Consider fetching all CREs with their links in a single batch query instead.

_{Fix it with Roo Code or mention @roomote and request a fix.}

[ghost]

The merge from main (3ca88b8) dropped the if OPENCRE_STANDARD_NAME in standards: dispatch block that was present in the pre-merge feature branch (445cbbd, line 417). Without it, _build_direct_cre_overlap_map_analysis and the other helper functions defined above are dead code -- requests with OpenCRE as a standard fall through to the normal RQ/Neo4j flow, which will fail because "OpenCRE" is not a real standard in the database. This effectively breaks the PR's core feature. The block needs to be re-added between the cache_key assignment and the HEROKU guard, e.g.:

Suggested change

	cache_key = gap_analysis.make_resources_key(standards)
	if database.gap_analysis_exists(cache_key):
	cached = database.get_gap_analysis_result(cache_key=cache_key)
	if cached:
	parsed = json.loads(cached)
	if "result" in parsed:
	return jsonify({"result": parsed.get("result")})
	if os.environ.get("HEROKU"):
	cache_key = gap_analysis.make_resources_key(standards)
	if database.gap_analysis_exists(cache_key):
	cached = database.get_gap_analysis_result(cache_key=cache_key)
	if cached:
	parsed = json.loads(cached)
	if "result" in parsed:
	return jsonify({"result": parsed.get("result")})
	if OPENCRE_STANDARD_NAME in standards:
	direct_gap_analysis = _build_direct_cre_overlap_map_analysis(
	standards, cache_key, database
	)
	if direct_gap_analysis:
	return jsonify(direct_gap_analysis)
	abort(404, "No direct overlap found for requested standards")
	if os.environ.get("HEROKU"):

_{Fix it with Roo Code or mention @roomote and request a fix.}