Overhauled staging tables with schemas and centralised specifiers#22
Draft
nicoloesch wants to merge 2 commits into
Draft
Overhauled staging tables with schemas and centralised specifiers#22nicoloesch wants to merge 2 commits into
nicoloesch wants to merge 2 commits into
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes Make staging table location schema-aware instead of relying on the
_staging_prefix #15Backend now owns staging table naming.
staging_name_for_table(tablename)is a@staticmethod @abstractmethodonDatabaseBackendPostgresBackendandSQLiteBackendboth implementf"_staging_{tablename}"(Postgres relies on schema isolation; SQLite needs the prefix for collision avoidance).staging_nameremoved as a parameter from all backend abstract methodscreate_staging_table,drop_staging_table,merge_replace,merge_upsert,merge_insert,load_staging_fastBackends derive the name from
table_cls.__tablename__internally viastaging_name_for_table.qualify_identifier(name, schema)extracted tohelpers/sql.pyand exported fromhelpers/__init__.backends/base.pyandloaders/loading_helpers.py, replacing two independent inline f-strings.Circular import (
backends/postgres->loaders/loading_helpers->helpers->helpers/bulk->backends/resolve->backends/postgres) fixed by makingresolve_backendimports inhelpers/bulk.pylazy (inside function bodies).STAGING_SCHEMA: str = "staging"convention constant introduced inbackends/base.pyand exported frombackends/__init____init__defaultsstaging_schematoNone(schema-agnostic).PostgresBackend.__init__opts in by defaulting toSTAGING_SCHEMA.None.Dead
_staging_tablename: ClassVar[Optional[str]] = NoneonCSVLoadableTableInterfaceremovedstaging_tablename()classmethod removed from model and fromCSVTableProtocol.FakeBackendin tests updated to match current abstract interface (stalestaging_namepositional params removed;staging_name_for_tableimplemented).