Assumptions

[erivaninan]

add: modules and use cases notebooks for assumption checking
minor fix in DiD difference_in_differences/diagnostics.py: improve formula quoting using Q() instead of backticks

[rahulbshrestha]

1) Could you use class Variables in cais/models.py. Feel free to add more variables to it, else we can create a new shared object that holds all parameters in the functions here: e.g (I would still prefer if we could stick to the Variables class)

class AssumptionVariables(): # Feel free to rename this to something better
    df: Optional[pd.DataFrame]
    treatment: Optional[str]
    outcome: Optional[str]
    covariates: List[str] = []
    instruments: List[str] = []
    running_variable: Optional[str]
    time_var: Optional[str]
    dataset_description: Optional[str]
    variables_summary: Dict[str, Any]
    .......

This would mean we can make each assumption check cleaner:

def check_balance_after_matching(vars: AssumptionVariables) -> AssumptionResult:
       .......

2) Could you make def _result into a class in the form:

class AssumptionResult

This should also go inside cais/models.py

3) Could we also include a registry to associate each assumption check with a method? It would make it easier to apply and know the mapping from the assumption <> method.

ASSUMPTION_REGISTRY = {
    "iv": [
        check_iv_relevance,
        check_iv_exclusion,
        check_iv_exogeneity,
    ],
    "did": [
        check_parallel_trends,
        check_no_anticipation,
    ],
}

4) Could you also add test cases for each of the assumption check? That can go to a separate file in the tests directory.

[jacobemmerson]

Two comments:

1. I would like to see pass/failure rates for some of the assumption checks. On the real datasets we collected, we would expect a close to >90% pass rate since they were used in real causal experiments.

2. I like the Jupyter notebooks, but both, primarily the use cases notebooks, could use more markdown cells to explain the workflow and why adding the assumption checks is a valuable addition. Think of it as if you are creating an argument for why CAIS is better than something like TabPFN, i.e. we rigorously check different assumptions which is important for a real causal analysis.

Also, it doesn't pass the Unit Tests for the CAIS core functions---could you look into why? I can try to come up with a fix if it is on how the test cases are executed.

[erivaninan]

Thanks for the detailed feedback.
Here is a summary of the changes made in response to your comments:

@rahulbshrestha

1. AssumptionVariables class: Created as a Pydantic model in cais/models.py (with ConfigDict(arbitrary_types_allowed=True) to support DataFrames). All assumption check functions now take a single AssumptionVariables argument instead of individual parameters.

2. AssumptionResult class: Created as a Pydantic BaseModel in cais/models.py with three fields: passed: Optional[bool], reasoning: str, details: Dict[str, Any]. Replaces the old _result dict helper.

3. Assumption registry: added ASSUMPTION_REGISTRY and POST_ASSUMPTION_REGISTRY dicts in pre_model_assumption_utils.py and post_model_assumption_utils.py, which map each method name ("instrumental_variable", "difference_in_differences", "propensity_score_matching", etc) to its list of check functions.

4. Test cases: tests/cais/methods/test_assumption_utils.py covers all pre and post-modeling assumption checks with synthetic data.

@jacobemmerson

• The usecases notebook is significantly expanded in docs/examples/assumption_usecases.ipynb with markdown cells explaining the pre/post-modeling distinction, the rationale for each check, and why surfacing assumption violations early is valuable for rigorous causal analysis.