Causal inference is a field at the intersection of statistics, computer science, and philosophy devoted to determining when and whether a correlation indicates a causal relationship. Instead of using randomization, controls, or other study designs to infer causation, causal inference uses probability distributions, independence tests, and other artifacts of the data. Causal inference methods can therefore be used on observational data as well as experimental data.
Directed acyclic graphs (DAGs) are a tool used in causal inference to rigorously but intuitively represent a causal model. A DAG is a visual depiction in which variables are represented by nodes, and their relationships are represented by edges. A node with a variable into another node causes that node. For instance, in this DAG, Stone Size causes Treatment and Outcome, and Treatment causes Outcome:
Parents, children, ancestors, and descendants are relationships that nodes in a graph can have with each other. A node X with an arrow into a node Y is the parent of Y, and Y is the child of X. In the above graph, Stone Size is a parent of Outcome and Treatment.
The backdoor criterion is a rule for determining which covariates to control for when determining the effect of a treatment on an outcome. Formally, the criterion states:
Given a treatment variable X and an outcome variable Y in a DAG G, a set of variables Z satisfies the backdoor criterion relative to (X, Y) if no node in Z is a descendant of X, and Z blocks every path between X and Y that contains an arrow into X.
Colliders occur when two variables have a common effect. In the above graph, Outcome is a collider between Stone Size and Treatment. Controlling for the child variable in a collider can make the two parent variables appear correlated, even if they aren't. This phenomenon is called collider bias.
Automated causal search is a type of tool used in causal inference in which causal models are discovered from data using computer algorithms.
Simpson's Paradox and the backdoor criterion are only a small introduction to the array of causal inference methods available. The readings below provide a broader, but still fairly accessible, introduction to causal inference.
This paper, presented at MLA 2024 in Portland, Oregon, tests the feasibility of using the backdoor criterion to shape PICO searches in practice. Three PubMed searches looked for studies comparing two kidney stone treatments: once without considering stone size or location; once using a proximity search and free text search to look for studies that considered stone size and location; and once using an RCT filter to limit the search to studies that randomized treatment, breaking the causal connections between stone size, stone location, and treatment.
A PICO search formed with causal inference methods did return different results than one formed without. However, the process of searching in this way presented significant obstacles that muddied the results and that might put this method outside the reach of an independent clinician or student.
Causal inference is highly relevant to question formation and study evaluation in literature search. The challenges in PICO searching with this method point towards other possible applications for causal inference, including critical appraisal tools and systematic reviews.
("Kidney Calculi"[Mesh]) AND "Nephrolithotomy, Percutaneous"[Mesh] AND "retrograde intrarenal surgery"
("Kidney Calculi"[Mesh]) AND "Nephrolithotomy, Percutaneous"[Mesh] AND "retrograde intrarenal surgery" AND ("stone size"[Title/Abstract:~10] OR "stone mm"[Title/Abstract:~10] OR "stone cm"[Title/Abstract:~10] OR "stone large"[Title/Abstract:~10] OR "stone small"[Title/Abstract:~10] OR "stone diameter"[Title/Abstract:~10] OR "stones size"[Title/Abstract:~10] OR "stones mm"[Title/Abstract:~10] OR "stones cm"[Title/Abstract:~10] OR "stones large"[Title/Abstract:~10] OR "stones small"[Title/Abstract:~10] OR "stones diameter"[Title/Abstract:~10] OR "calculus size"[Title/Abstract:~10] OR "calculus mm"[Title/Abstract:~10] OR "calculus cm"[Title/Abstract:~10] OR "calculus large"[Title/Abstract:~10] OR "calculus small"[Title/Abstract:~10] OR "calculus diameter"[Title/Abstract:~10] OR "calculus size"[Title/Abstract:~10] OR "calculi size"[Title/Abstract:~10] OR "calculi mm"[Title/Abstract:~10] OR "calculi cm"[Title/Abstract:~10] OR "calculi large"[Title/Abstract:~10] OR "calculi small"[Title/Abstract:~10] OR "calculi diameter"[Title/Abstract:~10] OR "calculi size"[Title/Abstract:~10]) AND ("pole"[Title/Abstract] OR "calyx"[Title/Abstract] OR "calyceal"[Title/Abstract] OR "pelvis"[Title/Abstract] OR "pelvic"[Title/Abstract])
("Kidney Calculi"[Mesh]) AND "Nephrolithotomy, Percutaneous"[Mesh] AND "retrograde intrarenal surgery" AND ("Randomized Controlled Trial"[pt] OR "Controlled Clinical Trial"[pt] OR "Pragmatic Clinical Trial"[pt] OR "Equivalence Trial"[pt] OR "Clinical Trial, Phase III"[pt] OR "Randomized Controlled Trials as Topic"[mh] OR "Controlled Clinical Trials as Topic"[mh] OR "Random Allocation"[mh] OR "Double-Blind Method"[mh] OR "Single-Blind Method"[mh] OR Placebos[Mesh:NoExp] OR "Control Groups"[mh] OR (random*[tiab] OR sham[tiab] OR placebo*[tiab]) OR ((singl*[tiab] OR doubl*[tiab]) AND (blind*[tiab] OR dumm*[tiab] OR mask*[tiab])) OR ((tripl*[tiab] OR trebl*[tiab]) AND (blind*[tiab] OR dumm*[tiab] OR mask*[tiab])) OR (control*[tiab] AND (study[tiab] OR studies[tiab] OR trial*[tiab] OR group*[tiab])) OR (Nonrandom*[tiab] OR "non random*"[tiab] OR "non-random*"[tiab] OR "quasi-random*"[tiab] OR quasirandom*[tiab]) OR allocated[tiab] OR (("open label"[tiab] OR "open-label"[tiab]) AND (study[tiab] OR studies[tiab] OR trial*[tiab])) OR ((equivalence[tiab] OR superiority[tiab] OR "non-inferiority"[tiab] OR noninferiority[tiab]) AND (study[tiab] OR studies[tiab] OR trial*[tiab])) OR ("pragmatic study"[tiab] OR "pragmatic studies"[tiab]) OR ((pragmatic[tiab] OR practical[tiab]) AND trial*[tiab]) OR ((quasiexperimental[tiab] OR "quasi-experimental"[tiab]) AND (study[tiab] OR studies[tiab] OR trial*[tiab])) OR (phase[ti] AND (III[ti] OR 3[ti]) AND (study[ti] OR studies[ti] OR trial*[ti])) OR (phase[ot] AND (III[ot] OR 3[ot]) AND (study[ot] OR studies[ot] OR trial*[ot])))
This poster, presented at the MAC/MLA 2023 Annual Conference, covers potential strategies for applying causal inference methods to improve the accuracy and relevance of PICO search results.
Defining the population in a PICO question too broadly or too narrowly may result in misleading outcomes, as demonstrated by Simpson’s Paradox, a statistical phenomenon in which an association that holds in a general population is reversed in all subpopulations. Simpson’s Paradox is, however, easily resolved using straightforward causal inference methods.
A directed acyclic graph (DAG) provides a rigorous visual depiction of the causal structure underlying data. In the case of the classical Simpson’s Paradox kidney stone data, the causal structure is already known. The backdoor principle is a causal inference method that identifies whether or not a variable should be conditioned on when determining the effect of a treatment on an outcome. Applying the backdoor principle to the kidney stone DAG determines whether or not stone size should be included in the population of the PICO question.
('nephrolithiasis'/exp OR 'calculosis, kidney' OR 'calculus, kidney' OR 'familial nephrolithiasis' OR 'kidney calculi' OR 'kidney calculosis' OR 'kidney calculus' OR 'kidney calix stone' OR 'kidney calyx stone' OR 'kidney lithiasis' OR 'kidney pelvis stone' OR 'kidney stone' OR 'kidney stone passage' OR 'kidney stone, pelvis' OR 'kidney stones' OR 'nephrolith' OR 'nephrolith passage' OR 'nephrolithiasis' OR 'renal calculogenesis' OR 'renal calculosis' OR 'renal calculus' OR 'renal lithiasis' OR 'renal pelvis stone' OR 'renal stone' OR 'renolithiasis' OR 'stone, kidney') AND ('open surgery'/exp OR 'open surgery' OR 'open surgical repair') AND ('percutaneous nephrolithotomy'/exp OR 'pcnl' OR 'nephrolithotomy via nephrostomy' OR 'nephrolithotomy, percutaneous' OR 'percutaneous kidney stone removal' OR 'percutaneous nephrolithotomy' OR 'percutaneous nephrostolithotomy') AND ('treatment outcome'/exp OR 'health care outcome and process assessment' OR 'healthcare outcome and process assessment' OR 'medical futility' OR 'outcome and process assessment (health care)' OR 'outcome and process assessment, health care' OR 'outcome management' OR 'patient outcome' OR 'therapeutic outcome' OR 'therapy outcome' OR 'treatment outcome')
('nephrolithiasis'/exp OR 'calculosis, kidney' OR 'calculus, kidney' OR 'familial nephrolithiasis' OR 'kidney calculi' OR 'kidney calculosis' OR 'kidney calculus' OR 'kidney calix stone' OR 'kidney calyx stone' OR 'kidney lithiasis' OR 'kidney pelvis stone' OR 'kidney stone' OR 'kidney stone passage' OR 'kidney stone, pelvis' OR 'kidney stones' OR 'nephrolith' OR 'nephrolith passage' OR 'nephrolithiasis' OR 'renal calculogenesis' OR 'renal calculosis' OR 'renal calculus' OR 'renal lithiasis' OR 'renal pelvis stone' OR 'renal stone' OR 'renolithiasis' OR 'stone, kidney') AND ('open surgery'/exp OR 'open surgery' OR 'open surgical repair') AND ('percutaneous nephrolithotomy'/exp OR 'pcnl' OR 'nephrolithotomy via nephrostomy' OR 'nephrolithotomy, percutaneous' OR 'percutaneous kidney stone removal' OR 'percutaneous nephrolithotomy' OR 'percutaneous nephrostolithotomy') AND ('treatment outcome'/exp OR 'health care outcome and process assessment' OR 'healthcare outcome and process assessment' OR 'medical futility' OR 'outcome and process assessment (health care)' OR 'outcome and process assessment, health care' OR 'outcome management' OR 'patient outcome' OR 'therapeutic outcome' OR 'therapy outcome' OR 'treatment outcome') AND ('randomized controlled trial'/exp OR 'controlled trial, randomized' OR 'randomised controlled study' OR 'randomised controlled trial' OR 'randomized controlled study' OR 'randomized controlled trial' OR 'trial, randomized controlled')
('nephrolithiasis'/exp OR 'calculosis, kidney' OR 'calculus, kidney' OR 'familial nephrolithiasis' OR 'kidney calculi' OR 'kidney calculosis' OR 'kidney calculus' OR 'kidney calix stone' OR 'kidney calyx stone' OR 'kidney lithiasis' OR 'kidney pelvis stone' OR 'kidney stone' OR 'kidney stone passage' OR 'kidney stone, pelvis' OR 'kidney stones' OR 'nephrolith' OR 'nephrolith passage' OR 'nephrolithiasis' OR 'renal calculogenesis' OR 'renal calculosis' OR 'renal calculus' OR 'renal lithiasis' OR 'renal pelvis stone' OR 'renal stone' OR 'renolithiasis' OR 'stone, kidney') AND '`stone size`' AND ('open surgery'/exp OR 'open surgery' OR 'open surgical repair') AND ('percutaneous nephrolithotomy'/exp OR 'pcnl' OR 'nephrolithotomy via nephrostomy' OR 'nephrolithotomy, percutaneous' OR 'percutaneous kidney stone removal' OR 'percutaneous nephrolithotomy' OR 'percutaneous nephrostolithotomy') AND ('treatment outcome'/exp OR 'health care outcome and process assessment' OR 'healthcare outcome and process assessment' OR 'medical futility' OR 'outcome and process assessment (health care)' OR 'outcome and process assessment, health care' OR 'outcome management' OR 'patient outcome' OR 'therapeutic outcome' OR 'therapy outcome' OR 'treatment outcome')
"Nephrolithiasis"[Mesh] AND "Nephrolithotomy, Percutaneous"[Mesh] AND "Treatment Outcome"[Mesh] AND "open surgery"
