Skip to main content

falsify.FalsificationResult

Result of a permutation-based DAG falsification test.

Usage

falsify.FalsificationResult(
    given_lmc_violations,
    n_lmc_tests,
    given_lmc_violation_fraction,
    perm_lmc_violation_fractions,
    perm_tpa_violation_fractions,
    p_value_lmc,
    p_value_tpa,
    n_permutations,
    n_in_mec,
    significance_level,
    significance_ci,
    local_violations
)

Produced by falsify_graph(). The verdict follows Eulig et al. (2023): the DAG is falsifiable (informative) when few node permutations share its Markov equivalence class, and falsified (rejected) when, despite being informative, it does not violate fewer Local Markov Conditions than the permuted baseline.

Parameters

given_lmc_violations: int

Local Markov Condition violations of the independence that was written down. A rank-collapsed row is not counted.

n_lmc_tests: int

Parental tests that actually tested the written parent set. Rank-collapsed rows and tests that could not be run are excluded. Zero means the DAG cannot be evaluated on the requested independences.

given_lmc_violation_fraction: float

given_lmc_violations / n_lmc_tests (0 if no written tests ran).

perm_lmc_violation_fractions: np.ndarray

Written-test LMC violation fraction for each permuted DAG that produced a written parental test. A permutation whose tests all rank-collapsed is omitted, not stored as 0. The length can be shorter than n_permutations.

perm_tpa_violation_fractions: np.ndarray

Fraction of parental d-separation (tPA) violations for each permuted DAG relative to the given DAG, shape (n_permutations,).

p_value_lmc: float

Fraction of the scored permutations whose written-test violation fraction is at most the given DAG’s. Both sides exclude rank-collapsed rows, and a permutation with no written parental test is not in this fraction. Small values mean the DAG beats the random baseline on the independences that were actually tested. NaN when permutations ran but none produced a written parental test.

p_value_tpa: float

Fraction of permutations lying in the Markov equivalence class of the given DAG (zero tPA violations). Small values mean the DAG is informative / falsifiable.

n_permutations: int

Number of permuted DAGs evaluated.

n_in_mec: int

Number of permutations sharing the given DAG’s Markov equivalence class.

significance_level: float

Significance level for the permutation-based verdict.

significance_ci: float

Significance level used for each conditional independence test.

local_violations: pd.DataFrame
One row per LMC test on the given DAG with columns node, non_descendant, conditioning_set, requested_k, effective_k, rank_collapsed, df, p_value, and violation. requested_k is the number of conditioning columns that entered the test. effective_k is rank([1, Z]) - 1. rank_collapsed is true when that executed test used fewer columns than requested, or a named conditioner contributed no columns. violation is false on a collapsed row: the p-value is still the reduced test, and the column matches given_lmc_violations. A collapsed row stays in this frame for inspection, but it is excluded from given_lmc_violations, n_lmc_tests, violations, and the permutation fraction. A permuted DAG that produced no written parental test is omitted from that fraction, not scored as zero violations. If no written test was scored against a permutation, status is not a rejection decision and falsified is unset.

Attributes

Name Description
can_evaluate Whether status is a rejection decision.
falsifiable Whether the DAG is informative enough to be falsified.
falsified Whether the data falsify (reject) the DAG.
status The one verdict. Other properties and renderers read this.
violations Violated rows that tested the parent set that was written down.

can_evaluate

Whether status is a rejection decision.

can_evaluate: bool

False for a saturated DAG, an all-collapsed parent set, or a written test that was not compared with a scored permutation. falsifiable and falsified are then None.

falsifiable

Whether the DAG is informative enough to be falsified.

falsifiable: bool | None

None unless status is a rejection decision. False only for FalsificationStatus.NOT_FALSIFIABLE. A DAG sitting at p_value_tpa == significance_level is informative.

falsified

Whether the data falsify (reject) the DAG.

falsified: bool | None

True only for FalsificationStatus.REJECTED: strictly informative (p_value_tpa < significance_level) and not clearly better than the permuted baseline (p_value_lmc exceeds significance_level). None when status is not a rejection decision, including when permutations ran but none produced a written parental test.

The strict < on the informativeness side mirrors Eulig et al. (2023) / dowhy, where a DAG sitting exactly at the boundary (p_value_tpa == significance_level) is not rejected.

status

The one verdict. Other properties and renderers read this.

status: FalsificationStatus

violations

Violated rows that tested the parent set that was written down.

violations: pd.DataFrame

Methods

Name Description
plot() Plot histograms of permuted-baseline violation fractions.

plot()

Plot histograms of permuted-baseline violation fractions.

Usage

plot(ax=None, bins=None)

Shows the distribution of LMC violation fractions (blue) and tPA d-separation violation fractions (orange) across permuted DAGs, with dashed vertical lines marking the given DAG’s values. A given DAG far to the left of the LMC histogram beats the baseline.

Parameters
ax: matplotlib.axes.Axes | None = None

Axes to plot on. Creates a new figure if None.

bins: BinsLike | None = None
Passed through to matplotlib.axes.Axes.hist: a positive integer bin count, a binning strategy name such as "auto", or a sequence of bin edges. Defaults to an automatic choice.
Returns
matplotlib.figure.Figure
The figure containing the histogram.
Raises
RuntimeError

If every requested parent set rank-collapsed, or the DAG implies no parental test. A not-compared result still plots the tPA baseline.

ValueError
If bins is neither one of the accepted types nor, for an integer bin count, positive. Strategy names and bin edges are validated by matplotlib, which reports them more precisely.