LITE-33583: Align bootstrapped extension deps with the runner#252
LITE-33583: Align bootstrapped extension deps with the runner#252pcaro wants to merge 2 commits into
Conversation
…-33583) Bootstrapping an extension produced an incoherent runner/connect-eaas-core pair, so the generated project could not run without manual fixes. The runner image was selected by the CLI's own major version (26, stale versus the real runner line 43) while the pyproject template hardcoded a connect-eaas-core ">=30" floor, letting `poetry update` resolve a version the runner image does not support. Select the latest published runner instead of filtering by the CLI major, and derive the connect-eaas-core specifier from that runner's PyPI requires_dist so the pair is always coherent and cannot drift again. The template now pins connect-eaas-core to the derived specifier, aligns python to the runner line (>=3.9,<3.13), and refreshes the stale dev-dependencies (pytest 8, flake8 >=6, pytest-asyncio >=0.23; drops the dead mock marker).
pcaro
changed the title
| [tool.poetry.dependencies] | ||
| python = ">=3.8,<4" | ||
| connect-eaas-core = ">=30" | ||
| python = ">=3.9,<3.13" |
There was a problem hiding this comment.
[Should-fix] python is hardcoded while connect-eaas-core is derived from the runner — the same drift this PR removes for eaas-core, left in place for python.
The runner's supported Python range is actively changing. When the runner shifts its floor/ceiling, the derived connect-eaas-core will follow it while this hardcoded python silently diverges, so Poetry may fail to resolve (or silently narrow) for a vendor whose interpreter is outside the derived eaas-core's real range.
Suggested fix: derive it from the runner too — the release JSON already fetched in get_pypi_runner_eaas_core_version carries info.requires_python (for runner 43.0 it is <3.13,>=3.9); template it the same way as connect_eaas_core_version.
There was a problem hiding this comment.
Good catch — fixed in 0513e0e. Instead of a second lookup I merged both into one function, get_pypi_runner_requirements(). It grabs the eaas-core and python ranges from the runner's metadata in a single fetch, and fails loudly if any of them is missing. The template now renders python = "{{ python_version }}".
| ) | ||
|
|
||
| assert get_pypi_runner_version() == '26.0' | ||
| assert get_pypi_runner_eaas_core_version('43.0') == '<38,>=37.4' |
There was a problem hiding this comment.
[Should-fix] The bootstrap tests mock a value the real code never produces, so the end-to-end render is never exercised.
test_extension_helpers.py mocks get_pypi_runner_eaas_core_version to return '>=37.4,<38', but the real function emits the sorted form '<38,>=37.4' (as asserted here). Both are valid Poetry, but nothing checks that the generated pyproject.toml is valid/loadable with the string the code actually emits.
Suggested fix: assert the generated pyproject.toml parses and pins connect-eaas-core using the real sorted specifier ('<38,>=37.4'), not the hand-written mock ordering.
There was a problem hiding this comment.
Fixed in 0513e0e. The mocks now use exactly what the code returns (('<38,>=37.4', '<3.13,>=3.9')), and the tests check both pins in the generated pyproject.toml.
|
[Should-fix] (follow-up, not in this diff) This PR correctly moves bootstrap/validate to the latest published runner (CLI and runner versions have diverged). But Suggested fix (likely a separate ticket): align |
…33583)
Address review feedback on the bootstrap dependency alignment: the
generated project's `python` constraint was still a hardcoded snapshot
of runner 43's range, recreating for python the same drift this branch
removes for connect-eaas-core.
Fold both pins into a single lookup: get_pypi_runner_requirements()
returns the (connect-eaas-core, python) specifiers from the runner's
PyPI release metadata (requires_dist + requires_python), failing loudly
when either is missing, and the template now renders both derived
values.
Also raise test fidelity: the bootstrap tests now mock the specifier
form the code actually emits ('<38,>=37.4') instead of a hand-reordered
variant, and assert both the connect-eaas-core and python pins in the
rendered pyproject.toml.
Agreed, but I'd rather keep it out of this PR. |
|
2 participants
Problem
Bootstrapping an extension produced an incoherent runner ↔ connect-eaas-core dependency pair, so the generated project could not run without manual fixes.
Root cause:
get_pypi_runner_version()selected the runner image by the CLI's own major version (26), which has diverged from the real runner line (currently 43). This baked a stale, incompatibleconnect-extension-runner:26.xinto the Dockerfile.pyproject.toml.j2template hardcodedconnect-eaas-core = ">=30", sopoetry updatein the Dockerfile resolved aconnect-eaas-coreversion the pinned runner image does not support.Fix
get_pypi_runner_version()now returns the latest published runner instead of filtering by the CLI major (removes the broken coupling).get_pypi_runner_eaas_core_version(runner)derives theconnect-eaas-corespecifier from the selected runner's PyPIrequires_dist, so the pair is always coherent and cannot drift again.connect-eaas-coreis pinned to the runner-derived specifier;pythonaligned to the runner line (>=3.9,<3.13); stale dev-deps refreshed (pytest 8, flake8 ≥6, pytest-asyncio ≥0.23) and the deadmockmarker dropped.Verification
43.0+connect-eaas-core <38,>=37.4, which poetry parses correctly (>=37.4,<38; allows 37.5, excludes 38.0).