Skip to content

Overhauled staging tables with schemas and centralised specifiers#22

Draft
nicoloesch wants to merge 2 commits into
mainfrom
15-staging
Draft

Overhauled staging tables with schemas and centralised specifiers#22
nicoloesch wants to merge 2 commits into
mainfrom
15-staging

Conversation

@nicoloesch

Copy link
Copy Markdown
Collaborator

Summary

  • Fixes Make staging table location schema-aware instead of relying on the _staging_ prefix #15

  • Backend now owns staging table naming. staging_name_for_table(tablename) is a @staticmethod @abstractmethod on DatabaseBackend

    • each concrete backend declares its own naming strategy
    • PostgresBackend and SQLiteBackend both implement f"_staging_{tablename}" (Postgres relies on schema isolation; SQLite needs the prefix for collision avoidance).
  • staging_name removed as a parameter from all backend abstract methods

    • create_staging_table,
    • drop_staging_table,
    • merge_replace,
    • merge_upsert,
    • merge_insert,
    • load_staging_fast
  • Backends derive the name from table_cls.__tablename__ internally via staging_name_for_table.

  • qualify_identifier(name, schema) extracted to helpers/sql.py and exported from helpers/__init__.

    • Used by both backends/base.py and loaders/loading_helpers.py, replacing two independent inline f-strings.
  • Circular import (backends/postgres -> loaders/loading_helpers -> helpers -> helpers/bulk -> backends/resolve -> backends/postgres) fixed by making resolve_backend imports in helpers/bulk.py lazy (inside function bodies).

  • STAGING_SCHEMA: str = "staging" convention constant introduced in backends/base.py and exported from backends/__init__

    • Named as a convention (not a constructor default)
    • The base class __init__ defaults staging_schema to None (schema-agnostic).
    • PostgresBackend.__init__ opts in by defaulting to STAGING_SCHEMA.
    • SQLite always passes None.
    • Future schema-capable backends (e.g. DuckDB) follow the Postgres pattern.
  • Dead _staging_tablename: ClassVar[Optional[str]] = None on CSVLoadableTableInterface removed

    • staging_tablename() classmethod removed from model and from CSVTableProtocol.
  • FakeBackend in tests updated to match current abstract interface (stale staging_name positional params removed; staging_name_for_table implemented).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Make staging table location schema-aware instead of relying on the _staging_ prefix

1 participant