Assessing Validity in Indirect Treatment Comparisons: A Comprehensive Guide to Common Comparator Methods and Best Practices

Abstract

This article provides researchers, scientists, and drug development professionals with a comprehensive framework for assessing the validity of Indirect Treatment Comparisons (ITCs) anchored by a common comparator. With head-to-head randomized controlled trials often unavailable, ITCs are indispensable for Health Technology Assessment (HTA) submissions and demonstrating comparative effectiveness. We explore the foundational assumptions, methodologies, and evolving guidelines from global HTA bodies. The content covers practical strategies for troubleshooting common pitfalls like heterogeneity and bias, and emphasizes validation techniques to ensure robust, defensible evidence for healthcare decision-making.

The Bedrock of Valid ITCs: Core Principles, Definitions, and the Central Role of the Common Comparator

Defining Indirect Treatment Comparisons and the Common Comparator Paradigm

In the realm of evidence-based medicine and health technology assessment, Indirect Treatment Comparisons (ITCs) have emerged as a crucial methodological approach when direct head-to-head evidence is unavailable. An ITC provides an estimate of the relative treatment effect between two interventions that have not been compared directly within randomized controlled trials (RCTs) [1]. This is typically achieved through a common comparator that acts as an anchor—often a standard drug, placebo, or control intervention—enabling the indirect comparison of treatments that lack direct comparative evidence [1].

The fundamental scenario for an ITC involves three interventions: if Treatment A has been compared to Treatment C in one trial, and Treatment B has been compared to Treatment C in another trial, then researchers can statistically derive an indirect comparison between Treatment A and Treatment B [1]. This paradigm has become increasingly important for healthcare decision-makers who need to compare all relevant interventions to inform reimbursement and treatment recommendations, particularly when direct comparisons are ethically challenging, economically unviable, or practically impossible to conduct [1] [2].

The Common Comparator Paradigm

Fundamental Principles and Workflow

The common comparator paradigm relies on a connected network of evidence where two or more interventions share a common reference point. The validity of this approach depends on several key assumptions that must be rigorously assessed to ensure the reliability of results.

The following diagram illustrates the logical relationship and workflow underlying the common comparator paradigm in indirect treatment comparisons:

Core Methodological Assumptions

The validity of the common comparator paradigm rests on three fundamental assumptions that must be critically evaluated:

• Similarity/Transitivity: This assumption requires that the trials being compared are sufficiently similar in their clinical and methodological characteristics to permit a fair comparison [1] [3]. This encompasses factors such as patient baseline characteristics, trial design, outcome definitions, and measurement methods. Violation of this assumption introduces significant uncertainty into ITC results [1] [4].

• Homogeneity: This refers to the similarity within the sets of trials comparing each intervention with the common comparator. There should be no substantial statistical heterogeneity within the A vs. C and B vs. C trial networks that would undermine the validity of pooling their results [5] [4].

• Consistency: When both direct and indirect evidence exists for the same comparison, the consistency assumption requires that these different sources of evidence produce similar treatment effect estimates [5] [4]. Significant discrepancies between direct and indirect evidence may indicate violation of underlying assumptions or methodological biases.

Methods for Indirect Treatment Comparisons

Numerous ITC techniques have been developed, each with distinct methodological approaches, data requirements, and applications. A recent systematic literature review identified seven primary ITC techniques used in contemporary research [2].

Table 1: Primary Indirect Treatment Comparison Methods

ITC Method	Key Features	Data Requirements	Common Applications	Key Assumptions
Network Meta-Analysis (NMA)	Simultaneously compares multiple interventions; most frequently described method (79.5% of articles) [2]	Aggregate data from multiple trials	Multiple intervention comparisons or treatment ranking [2] [3]	Consistency, homogeneity, similarity [3]
Bucher Method	Adjusted indirect comparison for pairwise comparisons through common comparator [3] [6]	Aggregate data from trials with common comparator	Pairwise indirect comparisons [2] [3]	Constancy of relative effects (homogeneity, similarity) [3]
Matching-Adjusted Indirect Comparison (MAIC)	Uses IPD from one treatment to match baseline characteristics of aggregate data from another [2] [7]	IPD from one trial plus aggregate data from another	Pairwise comparisons with population heterogeneity; single-arm studies [2] [3]	Constancy of relative or absolute effects [3]
Simulated Treatment Comparison (STC)	Predicts outcomes in aggregate data population using outcome regression model based on IPD [2] [3]	IPD from one trial plus aggregate data from another	Pairwise ITC with considerable population heterogeneity [3]	Constancy of relative or absolute effects [3]
Network Meta-Regression	Explores impact of study-level covariates on treatment effects [2] [3]	Aggregate data with covariate information	Multiple ITC with connected network to investigate effect modifiers [3]	Conditional constancy of relative effects with shared effect modifier [3]

Selection Framework for ITC Methods

Choosing the appropriate ITC method depends on several factors, including the available data, network structure, and specific research question. The following framework guides method selection:

Table 2: ITC Method Selection Framework

Scenario	Recommended Methods	Rationale	Key Considerations
Connected network with aggregate data only	Bucher method, NMA [2]	Preserves within-trial randomization without requiring IPD	Assess homogeneity and transitivity assumptions thoroughly [3] [4]
Single-arm studies or substantial population heterogeneity	MAIC, STC [2] [7]	Adjusts for cross-trial differences in patient populations	Requires IPD for at least one treatment; cannot adjust for unobserved differences [3] [7]
Effect modification by known covariates	Network meta-regression [2] [3]	Explores impact of study-level covariates on treatment effects	Requires multiple trials per comparison; limited with sparse networks [3]
Multiple interventions comparison	NMA [2] [3]	Simultaneously compares all interventions and provides ranking	Consistency assumption must be verified; complexity increases with network size [3]

Experimental Protocols and Methodological Workflows

Protocol for Conducting Adjusted Indirect Comparisons

The Bucher method, one of the foundational approaches for ITC, follows a specific statistical protocol [6] [5]:

• Identify direct evidence: Establish the available direct comparisons (A vs. C and B vs. C) from separate RCTs.
• Calculate effect estimates: Compute the relative treatment effects for each direct comparison (e.g., log odds ratios, risk ratios, or mean differences) with their standard errors.
• Compute indirect effect: The indirect estimate for A vs. B is derived using the formula: Effect_{A vs. B} = Effect_{A vs. C} - Effect_{B vs. C}
• Calculate variance: The variance of the indirect estimate is the sum of the variances of the two direct estimates: Var(Effect_{A vs. B}) = Var(Effect_{A vs. C}) + Var(Effect_{B vs. C})
• Construct confidence intervals: Using the calculated effect estimate and variance, construct 95% confidence intervals for the indirect comparison.

This method preserves the within-trial randomization and provides a statistically valid approach for indirect comparison, though it depends heavily on the similarity assumption [6] [5].

Protocol for Matching-Adjusted Indirect Comparison (MAIC)

MAIC has emerged as a valuable technique when IPD is available for at least one treatment [7]:

• Obtain IPD: Acquire individual patient data from trials of one treatment (e.g., Treatment A).
• Identify aggregate data: Collect published aggregate baseline characteristics and outcomes for the comparator treatment (Treatment B).
• Define target population: Typically, the population represented in the aggregate data serves as the target population.
• Calculate weights: Using propensity score methodology, calculate weights for each patient in the IPD such that the weighted distribution of baseline characteristics matches the aggregate distribution of the comparator trial.
• Analyze outcomes: Apply the calculated weights to the outcomes in the IPD and compare the weighted outcomes with the aggregate outcomes of the comparator.
• Assess uncertainty: Use appropriate methods (e.g., bootstrapping) to account for uncertainty in the weighting process.

MAIC effectively reduces observed cross-trial differences but cannot adjust for unobserved or unreported differences between trial populations [7].

Validity Assessment of Indirect Comparisons

Empirical Evidence on Validity

Empirical studies have investigated the validity of ITCs by comparing their results with direct evidence from head-to-head trials. A landmark study from 2003 examined 44 comparisons from 28 systematic reviews where both direct and indirect evidence were available [6].

Table 3: Validity Assessment of Indirect Comparisons

Validity Metric	Findings	Implications
Statistical agreement	Significant discrepancy (P<0.05) in 3 of 44 comparisons [6]	Indirect comparisons usually agree with direct evidence but not always
Direction of discrepancy	No consistent pattern of overestimation or underestimation [6]	Discrepancies are unpredictable and may go in either direction
Clinical importance	Most discrepancies were not clinically important, but some exceptions existed [6]	Clinical judgment is needed beyond statistical significance
Acceptance by HTA bodies	Varies by agency; some accept ITCs with caveats while others remain hesitant [1]	Uncertainty in similarity assessment affects acceptability

Assessment of Key Assumptions

Rigorous assessment of the underlying assumptions is critical for evaluating the validity of an ITC:

• Similarity Assessment: Compare patient characteristics (age, disease severity, comorbidities), trial methodologies (design, blinding, duration), and outcome definitions across the trials involved in the indirect comparison [3] [4]. Statistical methods like meta-regression can explore the impact of study-level covariates on treatment effects.

• Homogeneity Assessment: Evaluate statistical heterogeneity within each direct comparison using I² statistics, Cochran's Q test, or visual inspection of forest plots [5] [4]. Qualitative assessment of clinical and methodological diversity complements statistical measures.

• Consistency Assessment: When both direct and indirect evidence exist, use statistical tests for inconsistency (e.g., node-splitting) or compare direct and indirect estimates through sensitivity analyses [3] [4]. Significant inconsistency requires investigation of potential effect modifiers or methodological biases.

The Researcher's Toolkit for Indirect Comparisons

Table 4: Essential Methodological Tools for Indirect Treatment Comparisons

Tool Category	Specific Tools/Techniques	Function/Purpose	Key Considerations
Statistical Software	R (gemtc, pcnetmeta), WinBUGS/OpenBUGS, Stata	Implement various ITC methods including NMA, MAIC, and Bucher method	Bayesian frameworks preferred when source data are sparse [2]
Data Requirements	Individual Patient Data (IPD), Aggregate Data	IPD enables more sophisticated adjustment methods like MAIC	MAIC and STC require IPD for at least one treatment [2] [7]
Quality Assessment Tools	Cochrane Risk of Bias, PRISMA-NMA	Assess methodological quality and reporting of primary studies and ITCs	Critical for evaluating validity of underlying evidence [4]
Assumption Verification Methods	Meta-regression, Subgroup analysis, Sensitivity analysis	Investigate heterogeneity, consistency, and similarity assumptions	Should be pre-specified in analysis plan [3] [4]
Phenylephrone hydrochloride	Phenylephrone hydrochloride, CAS:94240-17-2, MF:C9H12ClNO2, MW:201.65 g/mol	Chemical Reagent	Bench Chemicals
Fraxiresinol 1-O-glucoside	Fraxiresinol 1-O-glucoside, MF:C27H34O13, MW:566.5 g/mol	Chemical Reagent	Bench Chemicals

Indirect Treatment Comparisons using the common comparator paradigm represent a sophisticated methodological approach that enables comparative effectiveness research when direct evidence is unavailable or insufficient. The validity of these comparisons hinges on carefully assessing the assumptions of similarity, homogeneity, and consistency. While various ITC methods are available—from the relatively simple Bucher method to more complex approaches like MAIC and NMA—method selection should be guided by the available data, research question, and need to adjust for cross-trial differences.

As therapeutic landscapes evolve rapidly, with new interventions emerging particularly in oncology and rare diseases, ITCs will continue to play a crucial role in informing healthcare decision-making. However, researchers must maintain rigorous standards in conducting and reporting ITCs, transparently communicating uncertainties and limitations to ensure appropriate interpretation by clinicians, policymakers, and patients.

In health technology assessment (HTA), decision-makers frequently need to compare the clinical efficacy and safety of treatments for which direct head-to-head randomized controlled trials (RCTs) are unavailable, unethical, or unfeasible [2]. Indirect treatment comparisons (ITCs) provide a methodological framework to address this evidence gap through quantitative techniques that enable comparative estimates between interventions that have not been studied directly against each other [3] [8]. The core distinction in ITC methodology lies between anchored and unanchored approaches, a classification dependent on the presence or absence of a common comparator that connects the evidence network [9] [10].

The validity and acceptance of these methods by HTA bodies worldwide hinge on their underlying assumptions and ability to minimize bias [8] [11]. As the European Union prepares to implement its Joint Clinical Assessment (JCA) procedure in 2025, understanding the critical distinctions between these approaches and their standing with HTA agencies becomes essential for researchers, scientists, and drug development professionals [10] [12]. This guide provides a comprehensive comparison of anchored versus unanchored ITCs, detailing their methodologies, applications, and relative positions in HTA decision-making.

Core Conceptual Distinctions

Foundational Principles and Network Structures

Anchored and unanchored ITCs differ fundamentally in their evidence network structures and the analytical assumptions they require. The following diagram illustrates the key differences in their evidence networks and analytical flow.

Anchored ITCs require a connected network of evidence where treatments are linked through a common comparator (e.g., placebo or a standard active treatment) [9] [10]. This common comparator serves as an "anchor" that preserves the randomization within each original trial, thereby minimizing bias in the resulting indirect treatment effect estimates [9]. The anchored approach encompasses methods such as network meta-analysis (NMA), the Bucher method, matching-adjusted indirect comparisons (MAIC), and simulated treatment comparisons (STC) when a common comparator is present [3] [10].

Unanchored ITCs, in contrast, are employed when the evidence network is disconnected and lacks a common comparator, typically involving single-arm studies or comparisons where the treatments share no mutual reference point [9] [10]. This approach relies on comparing absolute treatment effects across studies and requires much stronger assumptions, particularly that there are no unmeasured confounders or effect modifiers influencing the outcomes [9]. Unanchored comparisons are generally considered more susceptible to bias and receive greater scrutiny from HTA bodies [10] [13].

Comparative Characteristics and HTA Perspectives

The table below summarizes the key characteristics, methodological requirements, and HTA preferences for anchored versus unanchored ITC approaches.

Table 1: Core Characteristics and HTA Perspectives of Anchored vs. Unanchored ITCs

Characteristic	Anchored ITCs	Unanchored ITCs
Evidence Network	Connected network with common comparator	Disconnected network without common comparator
Foundation	Preserves within-trial randomization	Relies on comparison of absolute effects
Key Assumptions	Constancy of relative treatment effects	No unmeasured confounders or effect modifiers
Common Methods	NMA, Bucher, MAIC, STC (anchored)	MAIC (unanchored), STC (unanchored)
Data Requirements	Aggregate data or IPD from at least one trial with common comparator	Typically IPD for one treatment and aggregate for another
Strength of Evidence	Higher - respects randomization	Lower - vulnerable to confounding
HTA Acceptance	Generally preferred and accepted [10] [8]	Limited acceptance, require strong justification [9] [10]
Typical Applications	Connected networks of RCTs	Single-arm trials, disconnected evidence

The fundamental distinction in HTA acceptance stems from the preservation of randomization in anchored approaches versus the inherent risk of confounding in unanchored methods [9] [10]. HTA bodies consistently express preference for anchored methods when feasible, as they maintain the integrity of randomization and provide more reliable estimates of relative treatment effects [8]. Unanchored approaches are typically reserved for situations where anchored comparisons are impossible, such as when single-arm trials constitute the only available evidence, often in oncology or rare diseases [2] [10].

Methodological Approaches and Experimental Protocols

Various statistical methods have been developed to implement both anchored and unanchored ITCs, each with distinct requirements, strengths, and limitations. The following table provides a comparative overview of the primary ITC techniques used in practice.

Table 2: Methodological Approaches for Indirect Treatment Comparisons

ITC Method	Class	Data Requirements	Key Assumptions	Strengths	Limitations
Bucher Method [3] [2]	Anchored	Aggregate data	Constancy of relative effects (homogeneity, similarity)	Simple implementation for pairwise comparisons via common comparator	Limited to comparisons with common comparator; cannot incorporate multi-arm trials
Network Meta-Analysis (NMA) [3] [2]	Anchored	Aggregate data	Constancy of relative effects (homogeneity, similarity, consistency)	Simultaneous comparison of multiple interventions; can incorporate both direct and indirect evidence	Complex with challenging-to-verify assumptions; requires connected network
Matching-Adjusted Indirect Comparison (MAIC) [3] [9]	Anchored or Unanchored	IPD for index treatment, aggregate for comparator	Constancy of relative or absolute effects	Adjusts for population imbalances via propensity score weighting	Limited to pairwise comparisons; cannot adjust for unobserved confounding
Simulated Treatment Comparison (STC) [3] [9]	Anchored or Unanchored	IPD for index treatment, aggregate for comparator	Constancy of relative or absolute effects	Regression-based adjustment for population differences	Limited to pairwise comparisons; requires correct model specification
Network Meta-Regression (NMR) [3] [2]	Anchored	Aggregate data (IPD optional)	Conditional constancy of relative effects with shared effect modifiers	Explores impact of study-level covariates on treatment effects	Low power with aggregate data; ecological bias risk

According to a recent systematic literature review, NMA is the most frequently described ITC technique (79.5% of included articles), followed by MAIC (30.1%), network meta-regression (24.7%), the Bucher method (23.3%), and STC (21.9%) [2]. The appropriate selection among these methods depends on the evidence network structure, availability of individual patient data (IPD), magnitude of between-trial heterogeneity, and the specific research question [3] [2].

Detailed Experimental Protocols for Key Methods

Matching-Adjusted Indirect Comparison (MAIC) Protocol

MAIC is a population-adjusted method that requires IPD for at least one treatment and aggregate data for the comparator [9] [13]. The experimental protocol involves the following key steps:

• Effect Modifier Identification: Prior to analysis, identify and justify potential effect modifiers (patient characteristics that influence treatment effect) based on clinical knowledge and systematic literature review [9]. This pre-specification is critical for HTA acceptance [12].

• Propensity Score Estimation: Using the IPD, fit a logistic regression model to estimate the probability (propensity) that a patient belongs to the index trial versus the comparator trial, based on the identified effect modifiers [9].

• Weight Calculation: Calculate weights for each patient in the IPD as the inverse of their propensity score, effectively creating a "pseudo-population" where the distribution of effect matches the comparator trial [9].

• Outcome Analysis: Fit a weighted outcome model to the IPD and compare the adjusted outcomes with the aggregate outcomes from the comparator trial [9]. For anchored MAIC, this comparison is made relative to the common comparator; for unanchored MAIC, absolute outcomes are directly compared [9].

• Uncertainty Estimation: Use bootstrapping or robust variance estimation to account for the weighting uncertainty in confidence intervals [9]. HTA guidelines emphasize comprehensive sensitivity analyses to assess the impact of weighting and model assumptions [12].

A methodological review of 133 publications revealed inconsistent reporting of MAIC methodologies and potential publication bias, with 56% of analyses reporting statistically significant benefits for the treatment evaluated with IPD [13]. This highlights the importance of transparent reporting and rigorous methodology.

Network Meta-Analysis Protocol

NMA simultaneously compares multiple treatments by combining direct and indirect evidence across a connected network of trials [3] [2]. The experimental protocol involves:

• Systematic Literature Review: Conduct a comprehensive search to identify all relevant RCTs for the treatments and conditions of interest, following PRISMA guidelines [2].

• Evidence Network Mapping: Graphically represent the treatment network, noting all direct comparisons and identifying potential disconnected components [3].

• Assessment of Transitivity: Evaluate the clinical and methodological similarity of trials included in the network, ensuring that patient populations, interventions, comparators, outcomes, and study designs are sufficiently homogeneous [3].

• Statistical Analysis: Implement either frequentist or Bayesian statistical models to synthesize evidence [3] [12]. Bayesian approaches are particularly useful when data are sparse, as they allow incorporation of prior distributions [12].

• Consistency Evaluation: Assess the statistical agreement between direct and indirect evidence where available, using node-splitting or other diagnostic approaches [3].

• Uncertainty and Heterogeneity: Quantify heterogeneity and inconsistency in the network, and conduct sensitivity analyses to test the robustness of findings [12].

The 2024 EU HTA methodological guidelines emphasize pre-specification of statistical models, comprehensive sensitivity analyses, and transparent reporting of all methodological choices [12].

Global HTA Perspectives on ITC Methods

Health technology assessment bodies worldwide have established preferences regarding ITC methodologies, with clear distinctions in their acceptance of anchored versus unanchored approaches. The following diagram illustrates the key criteria that HTA bodies consider when evaluating ITC evidence.

A targeted review of worldwide ITC guidelines revealed that most jurisdictions favor population-adjusted or anchored ITC techniques over naïve comparisons or unanchored approaches [8]. The preference for anchored methods stems from their preservation of randomization and more testable assumptions compared to unanchored approaches, which rely on stronger, often untestable assumptions about the absence of unmeasured confounding [9] [8].

The European Union's upcoming JCA process emphasizes methodological flexibility without endorsing specific approaches, but clearly stresses the importance of pre-specification, comprehensive sensitivity analyses, and transparency in all ITC submissions [12]. Similarly, other HTA bodies acknowledge the utility of ITCs when direct evidence is lacking but maintain stringent criteria for their acceptance [8].

Acceptance Criteria for Different ITC Scenarios

Table 3: HTA Acceptance Criteria for Different ITC Scenarios

Scenario	Recommended Methods	HTA Acceptance Level	Key Requirements for Acceptance
Connected network of RCTs	NMA, Bucher method	High [2] [8]	Assessment of transitivity, homogeneity, consistency
Connected network with population heterogeneity	MAIC, STC, NMR	Moderate to High [3] [9]	IPD availability, justification of effect modifiers, adequate population overlap
Disconnected network with single-arm studies	Unanchored MAIC, Unanchored STC	Low to Moderate [9] [10]	Strong justification for effect modifiers, comprehensive sensitivity analyses, acknowledgment of limitations
Rare diseases with limited evidence	Population-adjusted methods	Case-by-case [2] [12]	Transparency about uncertainty, clinical rationale for assumptions

For unanchored comparisons, HTA acceptance remains limited due to the fundamental methodological challenges. The NICE Decision Support Unit emphasizes that unanchored comparisons "make much stronger assumptions, which are widely regarded as infeasible" [9]. Similarly, industry assessments note that unanchored approaches "are not recommended by most HTA agencies and should only be used when anchored methods are unfeasible" [10].

A review of MAIC applications found that studies frequently report statistically significant benefits for the treatment evaluated with IPD, with only one analysis significantly favoring the treatment evaluated with aggregate data [13]. This pattern suggests potential reporting bias and underscores the need for cautious interpretation of results from population-adjusted methods, particularly in unanchored scenarios [13] [11].

Research Reagent Solutions: Essential Methodological Components

The following table details key methodological components and their functions in conducting robust ITCs, representing the essential "research reagents" for this field.

Table 4: Essential Methodological Components for Indirect Treatment Comparisons

Component	Function	Application Notes
Individual Patient Data (IPD)	Enables patient-level adjustment for population imbalances	Required for MAIC, STC; allows examination of effect modifiers and prognostic factors [9] [13]
Aggregate Data	Provides comparison outcomes and population characteristics	Typically available from published literature; used in all ITC types [3]
Effect Modifier Identification Framework	Systematically identifies patient characteristics that influence treatment effects	Critical for population-adjusted methods; should be pre-specified and clinically justified [9] [12]
Propensity Score Models	Estimates probability of trial membership based on baseline characteristics	Foundation of MAIC; used to weight patients to achieve balance across studies [9]
Bayesian Statistical Models	Incorporates prior distributions for parameters	Particularly valuable when data are sparse; allows incorporation of external evidence [3] [12]
Frequentist Statistical Models	Provides traditional inference framework	Widely used in NMA; relies solely on current data without incorporating prior distributions [3]
Consistency Assessment Tools	Evaluates agreement between direct and indirect evidence	Includes node-splitting, design-by-treatment interaction tests; essential for NMA validation [3]
Sensitivity Analysis Framework	Tests robustness of results to methodological choices	Critical for HTA acceptance; should explore impact of model specifications, priors, and inclusion criteria [12]

The critical distinction between anchored and unanchored ITCs lies in the presence of a common comparator and the consequent strength of methodological assumptions. Anchored methods preserve the integrity of within-trial randomization and consequently receive higher acceptance from HTA bodies worldwide [10] [8]. Unanchored methods, while necessary in specific circumstances such as single-arm trials or disconnected evidence networks, require stronger assumptions and consequently face greater scrutiny and limited acceptance [9] [13].

For researchers and drug development professionals, selection between these approaches should be guided primarily by the available evidence network structure, with anchored methods preferred whenever possible [3] [2]. When population-adjusted methods like MAIC or STC are employed, comprehensive pre-specification of effect modifiers, transparent reporting, and rigorous sensitivity analyses are essential for HTA acceptance [12] [13]. As the European Union implements its new JCA process in 2025, adherence to methodological guidelines and early engagement with HTA requirements will be crucial for successful market access applications [10] [12].

In health technology assessment (HTA) and drug development, head-to-head randomized clinical trial data for all relevant treatments are often unavailable. Indirect Treatment Comparisons (ITCs) are methodologies used to compare the effects of different treatments through a common comparator, such as placebo or a standard care treatment. The validity of any ITC hinges on two fundamental assumptions: the constancy of relative effects and similarity [3].

The constancy of relative effects, also referred to as homogeneity or similarity, assumes that the relative effect of a treatment compared to a common comparator is constant across different study populations. When this assumption holds, simple indirect comparison methods like the Bucher method can provide valid results. Similarity extends beyond just the treatment effects and encompasses the idea that the studies being compared are sufficiently alike in their key characteristics, such as patient populations, interventions, outcomes, and study designs (the PICO framework). Violations of these assumptions can introduce significant bias into the comparison, leading to incorrect conclusions about the relative efficacy and safety of treatments [3].

This guide objectively compares the performance of different ITC methodologies, detailing their experimental protocols, inherent assumptions, and performance data to assist researchers in selecting and validating the most appropriate approach for their research.

Conceptual Framework and Key Terminology

Foundational Concepts in ITC

Indirect comparisons form a connected network of evidence, allowing for the estimation of relative treatment effects between interventions that have never been directly compared in a randomized trial. The simplest form is a pairwise indirect comparison via the Bucher method, which connects two treatments (e.g., B and C) through a common comparator A [3]. More complex Network Meta-Analyses (NMA) allow for the simultaneous comparison of multiple treatments [3]. Table 1 provides a glossary of essential terms used in the field.

Table 1: Key Terminology in Indirect Treatment Comparisons

Term	Definition	Key Considerations
Constancy of Relative Effects [3]	The assumption that relative treatment effects are constant across different study populations. Also referred to as homogeneity or similarity.	Fundamental for unadjusted ITCs. Violation introduces bias.
Conditional Constancy of Relative Effects [14]	A relaxed constancy assumption that holds true only after adjusting for all relevant effect modifiers.	Core assumption for anchored population-adjusted methods.
Similarity [3]	The degree to which studies in a comparison are alike in their PICO (Population, Intervention, Comparator, Outcome) elements.	Assessed both clinically and methodologically prior to analysis.
Effect Modifier [14]	A patient or study characteristic that influences the relative effect of a treatment.	Imbalance in these between studies violates the constancy assumption.
Anchored Comparison [14]	An indirect comparison that utilizes a common comparator shared between studies.	Relaxes the data requirements compared to unanchored comparisons.
Unanchored Comparison [14]	A comparison made in the absence of a common comparator, often involving single-arm studies.	Requires the much stronger assumption of conditional constancy of absolute effects.

The Role of Assumptions in Different ITC Frameworks

The following diagram illustrates the logical relationship between the core assumptions, data availability, and the appropriate selection of ITC methodologies.

Comparative Analysis of ITC Methodologies

The landscape of ITC methods has evolved to handle scenarios where the fundamental constancy assumption is violated. Population-Adjusted Indirect Comparisons (PAIC) leverage individual patient data (IPD) from at least one study to adjust for imbalances in effect modifiers [14]. The most common PAIC methods are Matching-Adjusted Indirect Comparison (MAIC), Simulated Treatment Comparison (STC), and Multilevel Network Meta-Regression (ML-NMR) [15]. The experimental workflow for implementing these methods is detailed below.

Performance Data from Simulation Studies

Simulation studies are critical for understanding the performance of different PAIC methods under controlled conditions, especially when assumptions are not fully met. A key simulation study assessed the performance of MAIC, STC, and ML-NMR across various scenarios [15]. The results are summarized in Table 2 below.

Table 2: Comparative Performance of Population Adjustment Methods in Simulation Studies [15]

Simulation Scenario	MAIC Performance	STC Performance	ML-NMR Performance	Key Implication
All Assumptions Met	Increased bias compared to standard ITC; poor performance.	Bias eliminated when assumptions were met.	Bias eliminated when assumptions were met; robust.	ML-NMR and STC are preferred when their specific assumptions are justified.
Missing Effect Modifier	Significant bias introduced.	Significant bias introduced.	Significant bias introduced.	Careful selection of all effect modifiers prior to analysis is essential for all methods.
Poor Population Overlap	Performance deteriorated severely; high variance due to low Effective Sample Size (ESS).	Performance impacted by extrapolation.	More robust to varying degrees of between-study overlap.	Check covariate distributions and ESS (for MAIC) before analysis.
Larger Treatment Networks	Not designed for larger networks; limited application.	Not designed for larger networks; limited application.	Effectively handles networks with multiple treatments and studies.	ML-NMR is the most flexible method for complex evidence networks.

The Scientist's Toolkit: Research Reagent Solutions

Successfully conducting a robust indirect comparison requires more than just statistical software. The following table details essential "research reagents" and their functions in the experimental process of an ITC.

Table 3: Essential Reagents for Indirect Comparison Research

Research Reagent	Function in ITC Analysis
Individual Patient Data (IPD)	The raw data from a clinical trial, allowing for patient-level analysis, validation of effect modifiers, and application of PAIC methods like MAIC and STC [14].
Aggregate Data (AgD)	Published summary data (e.g., mean outcomes, covariate summaries) from other studies used to build the evidence network. The quality and completeness of AgD reporting are critical [14].
Covariate Selection Framework	A pre-specified, principled approach (informed by clinical experts and prior evidence) for identifying effect modifiers and prognostic variables to adjust for, crucial for minimizing bias and avoiding "gaming" [14].
Effective Sample Size (ESS)	A metric calculated from the weights in a MAIC analysis. A large reduction in ESS indicates poor population overlap and may lead to an unstable and imprecise comparison [14].
Non-Inferiority Margin	A pre-defined threshold used in formal equivalence testing, which can be integrated with ITCs in a Bayesian framework to provide probabilistic evidence for clinical similarity in cost-comparison analyses [16].
13,14-Dihydro-15-keto-PGE2	13,14-Dihydro-15-keto-PGE2|High Purity
AZ14133346	AZ14133346, MF:C29H27N5O2, MW:477.6 g/mol

The Evolving Landscape of Global HTA Guidelines for ITC Acceptance

Health Technology Assessment (HTA) bodies worldwide face a persistent challenge: making informed recommendations about new health interventions often without direct head-to-head randomized clinical trial data against standard-of-care treatments [3]. In this evidence gap, Indirect Treatment Comparison (ITC) methodologies have become indispensable tools for generating comparative evidence. ITCs encompass statistical techniques that allow comparison of treatments that have not been directly studied in the same clinical trial, by using a common comparator or network of evidence [3] [17].

The global acceptance of ITC methods by HTA agencies remains varied, with overall acceptance rates generally low, creating a complex landscape for drug developers and researchers [17]. A comprehensive analysis of HTA evaluation reports between 2018 and 2021 found that only 22% presented an ITC, with an overall acceptance rate of just 30% [17]. This underscores the critical importance of understanding the methodological requirements and preferences of different HTA bodies. With the impending implementation of the EU HTA Regulation (EU 2021/2282) in 2025, which will standardize assessments across Europe, understanding this evolving landscape becomes even more crucial for successful HTA submissions [12].

This guide provides a comparative analysis of ITC acceptance across major HTA agencies, detailing methodological preferences, quantitative acceptance data, and strategic frameworks for selecting and implementing ITCs that meet rigorous HTA standards.

Comparative Analysis of HTA Agency Acceptance of ITC Methods

Quantitative Acceptance Rates Across Major HTA Markets

Comprehensive analysis of HTA evaluations reveals significant variation in ITC acceptance across different jurisdictions. The table below summarizes acceptance rates and methodological preferences for key HTA agencies based on recent publications (2018-2021) [17].

Table 1: ITC Acceptance Rates and Methodological Preferences by HTA Agency

HTA Agency/Country	Reports with ITC (%)	ITC Acceptance Rate (%)	Commonly Accepted Methods	Primary Criticisms
NICE (England)	51%	47%	NMA, Bucher, MAIC	Heterogeneity, statistical methods
G-BA/IQWiG (Germany)	24%	24%	NMA, Bucher	Data limitations, heterogeneity
AIFA (Italy)	17%	22%	NMA, MAIC	Lack of data, methodological concerns
AEMPS (Spain)	11%	19%	NMA, Bucher	Heterogeneity, statistical methods
HAS (France)	6%	0%	Limited acceptance	Data limitations, methodological concerns

The variation in acceptance rates reflects fundamental differences in methodological stringency, evidentiary standards, and regulatory philosophies across HTA agencies. England's NICE demonstrates the highest acceptance rate (47%), while France's HAS did not accept any ITCs in the studied period [17]. The most common criticisms cited by HTA agencies relate to data limitations (48%), heterogeneity between studies (43%), and concerns about statistical methods used (41%) [17].

The choice of ITC methodology significantly influences its likelihood of acceptance by HTA agencies. The table below illustrates the usage and acceptance rates of different ITC techniques based on a systematic literature review and analysis of HTA submissions [17] [2].

Table 2: ITC Method Usage and Acceptance Patterns

ITC Method	Description	Frequency of Use	Acceptance Rate	Key Considerations
Network Meta-Analysis (NMA)	Simultaneous comparison of multiple treatments using direct & indirect evidence	79.5% of ITC articles [2]	39% [17]	Preferred for connected networks; consistency assumptions critical
Bucher Method	Adjusted indirect comparison for simple networks via common comparator	23.3% of ITC articles [2]	43% [17]	Suitable for pairwise comparisons with common comparator
Matching-Adjusted Indirect Comparison (MAIC)	Reweighting IPD to match AgD baseline characteristics	30.1% of ITC articles [2]	33% [17]	Requires IPD from at least one trial; addresses population differences
Simulated Treatment Comparison (STC)	Predicts outcomes using regression models based on IPD	21.9% of ITC articles [2]	Limited data	Applied with single-arm studies; strong assumptions required
Network Meta-Regression (NMR)	Incorporates trial-level covariates to adjust for heterogeneity	24.7% of ITC articles [2]	Limited data	Addresses cross-trial heterogeneity; requires multiple studies

Recent trends indicate increased use of population-adjusted methods like MAIC, particularly in submissions involving single-arm trials, which are increasingly common in oncology and rare diseases [2]. Among recent articles (published from 2020 onwards), 69.2% describe population-adjusted methods, notably MAIC [2].

Methodological Framework for ITC Selection and Application

Strategic Selection of ITC Methods

The strategic selection of an appropriate ITC method depends on several factors, including the connectedness of the evidence network, availability of individual patient data (IPD), and the presence of heterogeneity between studies [3] [2]. The following decision framework illustrates the methodological selection process:

This structured approach to method selection emphasizes the importance of evidence network structure and data availability in determining the most appropriate ITC technique. HTA guidelines consistently emphasize that the choice between methods should be justified based on the specific scope and context of the analysis rather than defaulting to any single approach [12].

The EU HTA Regulation 2025: New Methodological Standards

The impending EU HTA Regulation (2021/2282), fully effective from January 2025, establishes new methodological standards for ITCs in Joint Clinical Assessments (JCAs) [12]. The regulation specifies several key methodological requirements:

• Pre-specification of analyses: Statistical analyses must be determined and documented before conducting any analysis to prevent selective reporting and ensure scientific rigor [12]
• Comprehensive uncertainty assessment: Sensitivity analyses must explore the impact of missing data and methodological assumptions [12]
• Transparency in methodology: Clear documentation of models and methods, avoiding "cherry-picking" of data [12]
• Accounting for effect modifiers: Identification and adjustment for all relevant baseline characteristics that could influence treatment effects [12]

The EU HTA guidance acknowledges both frequentist and Bayesian approaches without clear preference, noting that Bayesian methods are particularly useful in situations with sparse data due to their ability to incorporate prior information [12].

Experimental Protocols and Validation Frameworks for ITC

Methodological Protocols for Robust ITC

Implementing methodologically robust ITCs requires strict adherence to validated protocols. Based on HTA agency guidelines, the following experimental protocols are recommended:

Network Meta-Analysis Protocol:

• Systematic Literature Review: Comprehensive search across multiple databases following PRISMA guidelines to identify all relevant RCTs [2]
• Network Feasibility Assessment: Evaluation of transitivity and similarity assumptions across studies, including study designs, patient characteristics, treatments, and outcomes [3]
• Statistical Model Selection: Choice between fixed-effect and random-effects models based on heterogeneity assessment [3]
• Consistency Evaluation: Assessment of consistency between direct and indirect evidence using node-splitting or other appropriate methods [3]
• Uncertainty Quantification: Sensitivity analyses to assess robustness of findings to methodological assumptions [12]

Matching-Adjusted Indirect Comparison Protocol:

• IPD Preparation: Collection and validation of individual patient data from at least one trial [17]
• Effect Modifier Identification: Selection of baseline characteristics expected to modify treatment effect based on clinical knowledge [12]
• Propensity Score Estimation: Calculation of propensity scores based on the distribution of effect modifiers in IPD and aggregate data [17]
• Population Reweighting: Application of propensity score weights to match the IPD population to the aggregate data population [12]
• Outcome Comparison: Comparison of reweighted outcomes with the aggregate data using appropriate statistical methods [17]

Validation and Sensitivity Analysis Framework

HTA agencies emphasize the importance of comprehensive validation and sensitivity analyses to assess the robustness of ITC findings:

• Goodness-of-Fit Assessment: Evaluation of model fit using deviance information criteria (DIC) for Bayesian models or Akaike information criterion (AIC) for frequentist models [3]
• Heterogeneity Exploration: Investigation of between-study heterogeneity using I² statistics or predictive intervals [3]
• Influence Analysis: Assessment of the impact of individual studies on overall treatment effect estimates [3]
• Scenario Analyses: Evaluation of results under different assumptions, such as alternative prior distributions in Bayesian analyses or different handling of multi-arm trials [12]

Essential Research Reagent Solutions for ITC Analysis

Successful implementation of ITCs requires specific methodological tools and approaches. The following table details key "research reagent solutions" - core methodological components essential for robust ITC analysis.

Table 3: Essential Methodological Components for ITC Analysis

Methodological Component	Function	Application Context
Individual Patient Data (IPD)	Enables population-adjusted methods like MAIC and STC; allows exploration of treatment-effect modifiers	Essential when significant heterogeneity exists between study populations; required for unanchored comparisons [12] [17]
Aggregate Data (AgD)	Foundation for standard ITC methods like NMA and Bucher; required from comparator studies	Standard input for connected network meta-analyses; sufficient when population similarity exists [12]
Propensity Score Weighting	Balcomes baseline characteristics between IPD and AgD populations by assigning weights to patients	Core component of MAIC; adjusts for population differences when comparing across studies [12] [17]
Bayesian Hierarchical Models	Provides framework for evidence synthesis with incorporation of prior knowledge; handles sparse data effectively	Preferred for complex networks with multi-arm trials; useful when incorporating real-world evidence [12]
Frequentist Fixed/Random Effects Models	Traditional statistical approach for evidence synthesis; widely understood and implemented	Standard choice for conventional NMA; preferred when prior information is limited or controversial [3]
Network Meta-Regression	Explores impact of study-level covariates on treatment effects; adjusts for cross-trial heterogeneity	Applied when effect modifiers are identified at study level; requires multiple studies for sufficient power [3] [17]

The global landscape of ITC acceptance in HTA continues to evolve, with significant variations in methodological preferences and acceptance rates across different agencies. The forthcoming EU HTA Regulation (2025) represents a substantial shift toward standardization, while maintaining flexibility in methodological approach selection [12].

Successful navigation of this landscape requires:

• Strategic method selection based on evidence network structure and data availability
• Rigorous adherence to methodological protocols with comprehensive sensitivity analyses
• Early engagement with HTA bodies to align on methodological approach, particularly for novel ITC methods
• Transparent reporting of assumptions, limitations, and uncertainty in ITC findings

As HTA methodologies continue to advance, the development of more sophisticated ITC techniques and their increasing acceptance hold promise for more efficient and informative comparative effectiveness research, ultimately supporting better healthcare decision-making worldwide.

Why Common Comparators are Crucial for Minimizing Bias and Preserving Randomization

In clinical research, the choice of a common comparator is a fundamental aspect of trial design that directly impacts the validity, interpretability, and utility of study findings. Common comparators serve as a critical anchor, enabling fair and scientifically sound comparisons between interventions, especially when direct head-to-head evidence is absent. Their proper use preserves the integrity of randomization—the cornerstone of randomized controlled trials (RCTs)—by providing a baseline against which treatment effects can be measured without systematic bias. This guide explores the pivotal role of common comparators in minimizing bias, detailing the methodological frameworks for their application in both direct and indirect comparison analyses. Through explicit experimental protocols and data presentations, we provide researchers and drug development professionals with the tools to design more rigorous and unbiased clinical studies.

The Fundamental Role of Comparators in Clinical Research

A comparator (or control) is a benchmark or reference against which the effects of an investigational medical intervention are evaluated [18]. In clinical trials, this can be a placebo, an active drug representing the standard of care, a different dose of the study drug, or even no treatment [19] [18]. The use of a comparator is non-negotiable for establishing the relative efficacy and safety of a new treatment; without it, attributing observed effects solely to the intervention under investigation is impossible, as they could result from other factors such as the natural progression of the disease or patient expectations [18].

The selection of an appropriate comparator is deeply intertwined with the principle of randomization. Random allocation of participants to treatment or comparator groups is the most effective method for minimizing selection bias [20]. It works by eliminating systematic differences between comparison groups, ensuring that any differences in outcomes can be reliably attributed to the treatment effect rather than confounding variables, whether known or unknown [20]. The comparator group provides the essential reference point that allows this attributed effect to be quantified. Controversies in trial design often revolve around comparator choice, as this decision directly affects a trial's purpose, feasibility, fundability, and ultimate impact [19] [21].

Common Comparators as Anchors for Unbiased Indirect Comparisons

In an ideal world, all relevant treatment options would be compared directly in head-to-head randomized controlled trials. However, this is often impractical due to economic constraints, the dynamic nature of treatment landscapes, and the fact that drug registration in many markets historically required only demonstration of efficacy versus a placebo [22] [1] [8]. This evidence gap creates a critical need for methods to compare interventions that have never been directly studied against one another.

This is where the common comparator becomes indispensable. A common comparator enables Indirect Treatment Comparisons (ITCs), which are statistical techniques used to estimate the relative treatment effect of two interventions (e.g., Drug A and Drug B) by leveraging their direct comparisons against a shared anchor, or "common comparator" (e.g., Drug C or a placebo) [22] [1] [23].

• The Statistical Foundation: The most accepted method for ITC is the adjusted indirect comparison [22]. It preserves the original randomization of the constituent trials by comparing the relative treatment effects of each drug versus the common comparator. The difference between Drug A and Drug B is estimated by subtracting their respective effects versus the common comparator [22] [23].
• Contrast with Naïve Comparisons: It is crucial to distinguish adjusted indirect comparisons from "naïve direct comparisons," which simply compare the outcome of Drug A from one trial directly with the outcome of Drug B from another trial without any adjustment for the common comparator [22]. This naïve approach "breaks" the original randomization, is subject to significant confounding and bias from systematic differences between the trials (e.g., in patient populations or study design), and provides no more robust evidence than a comparison of observational studies [22].

The following diagram illustrates the logical structure of an adjusted indirect comparison using a common comparator.

Experimental Protocols for Indirect Comparison Analysis

Conducting a valid and credible indirect comparison requires a rigorous, multi-step methodology. The following protocol, consistent with guidelines from international health technology assessment (HTA) agencies like NICE (UK) and CADTH (Canada), outlines the core process [22] [8].

Protocol 1: Conducting an Adjusted Indirect Comparison

Objective: To estimate the relative efficacy and/or safety of Intervention A versus Intervention B using a common comparator C.

Step 1: Define the Research Question and Eligibility Criteria Clearly specify the interventions (A, B, C), the patient population, and the outcomes of interest. Develop detailed eligibility criteria for the studies to be included (e.g., study design, treatment duration, outcome measures) [8] [23].

Step 2: Systematic Literature Review Conduct a comprehensive and reproducible search of scientific literature databases (e.g., MEDLINE, Embase, Cochrane Central) to identify all relevant randomized controlled trials that compare A vs. C and B vs. C [23]. The search strategy, including keywords and filters, must be documented transparently.

Step 3: Study Selection and Data Extraction Screen search results against the eligibility criteria. From each included study, extract data on study characteristics, patient baseline characteristics, and outcome data for all treatment arms [23]. This is typically performed by at least two independent reviewers to minimize error and bias.

Step 4: Assess Similarity and Transitivity This is a critical qualitative step. Evaluate whether the trials for A vs. C and B vs. C are sufficiently similar in their key aspects (e.g., patient population, dosage of common comparator C, study definitions, and methods for measuring outcomes) to justify a fair comparison [22] [1] [8]. The validity of the ITC rests on this assumption of similarity (or transitivity).

Step 5: Perform Meta-Analysis (if required) If multiple trials exist for the same direct comparison (e.g., several A vs. C trials), a meta-analysis should be conducted to generate a single, precise estimate of the treatment effect for that comparison [23]. This can be done using software like Review Manager, applying either a fixed-effect or random-effects model depending on the presence of heterogeneity.

Step 6: Calculate the Adjusted Indirect Comparison Apply the Bucher method [22] [23] to compute the indirect estimate. For a continuous outcome (e.g., change in FEV1), the calculation is: D_IC = D_AC - D_BC, where D_AC is the mean difference between A and C, and D_BC is the mean difference between B and C. The standard error is: SE_IC = sqrt( SE_AC^2 + SE_BC^2 ). For a binary outcome (e.g., response rate), the comparison is done using relative risks (RR) or odds ratios (OR): RR_IC = RR_AC / RR_BC [22].

Step 7: Assess Inconsistency If a closed loop of evidence exists (i.e., there are direct comparisons for A vs. B, A vs. C, and B vs. C), statistically test for inconsistency between the direct and indirect estimates of the A vs. B effect. A significant difference may indicate a violation of the similarity assumption [23].

Case Study: Comparing Inhaled Corticosteroids in Asthma

A study by Kunitomi et al. (2015) provides a clear example of ITC in practice, comparing the efficacy of different inhaled corticosteroids (ICS) for asthma where direct head-to-head evidence was limited [23].

Objective: To indirectly compare the change in forced expiratory volume in 1 second (FEV1) for fluticasone propionate (FP) vs. budesonide (BUD), FP vs. beclomethasone dipropionate (BDP), and BUD vs. BDP.

Methodology:

• A systematic literature review identified 23 eligible RCTs.
• Two common comparators were used: Placebo (PLB) and an active drug, Mometasone (MOM).
• Meta-analyses were performed for each ICS against PLB and against MOM.
• Adjusted indirect comparisons were conducted using both PLB and MOM as the common anchor.

Results: The table below summarizes the key findings of the indirect comparisons for the change in FEV1.

Table 1: Indirect Comparison Results for Inhaled Corticosteroids (Change in FEV1) [23]

Comparison	Common Comparator	Mean Difference (L)	95% Confidence Interval
FP vs. BUD	Placebo	0.03	(-0.07, 0.13)
FP vs. BUD	Mometasone	0.04	(-0.08, 0.16)
FP vs. BDP	Placebo	0.08	(-0.03, 0.19)
FP vs. BDP	Mometasone	0.07	(-0.06, 0.20)
BUD vs. BDP	Placebo	0.05	(-0.06, 0.16)
BUD vs. BDP	Mometasone	0.03	(-0.10, 0.16)

Interpretation: The results demonstrated no statistically significant differences in efficacy between the various ICS, as all confidence intervals crossed zero. Crucially, the choice of common comparator (PLB or MOM) had no significant impact on the conclusions, as the point estimates and confidence intervals were very similar for both methods. This strengthens the credibility of the findings by showing robustness to the choice of a valid common comparator [23].

A Researcher's Toolkit for Comparator-Based Studies

Selecting and applying the right tools and methodologies is essential for conducting unbiased comparisons. The following table details key conceptual "reagents" and their functions in this process.

Table 2: Essential Toolkit for Comparator-Based Research

Tool / Concept	Primary Function	Key Considerations
Adjusted Indirect Comparison [22]	Provides a statistically valid estimate of the relative effect of two treatments via a common comparator.	Preserves randomization from source trials. Preferred over naïve comparisons by HTA bodies.
Common Comparator [22] [1]	Serves as the anchor or link that enables indirect comparisons.	Can be a placebo, standard of care, or an active drug. Must be identical or very similar in all trials used.
Assumption of Similarity (Transitivity) [1] [8]	The foundational assumption that the trials being linked are sufficiently similar to permit a fair comparison.	Requires assessment of patient populations, study designs, dosages, and outcome definitions. Violations can invalidate the analysis.
Network Meta-Analysis (NMA) [8]	A sophisticated extension of ITC that incorporates all available direct and indirect evidence into a single, coherent analysis for multiple treatments.	Reduces uncertainty but requires complex statistical models (e.g., Bayesian frameworks) and strong assumptions.
Pragmatic Model for Comparator Selection [19] [21]	A decision-making framework for selecting the optimal comparator for a randomized controlled trial.	Emphasizes that the primary purpose of the trial is the most important factor in comparator choice, balancing attributes like acceptability, feasibility, and relevance.
Nlrp3-IN-68	Nlrp3-IN-68, MF:C18H15FN2O3, MW:326.3 g/mol	Chemical Reagent
Dac590	Dac590, MF:C19H16ClFN2O4, MW:390.8 g/mol	Chemical Reagent

Visualization of Comparator Selection and Research Workflow

The process of selecting a comparator and designing a trial or evidence synthesis project is strategic. The following workflow, adapted from the NIH expert panel's Pragmatic Model, outlines the key decision points [19] [21].

The strategic use of common comparators is a pillar of unbiased clinical research. They are not merely passive control groups but active methodological tools that extend the power of randomization beyond single trials, enabling scientifically defensible comparisons in the absence of direct evidence. As the therapeutic landscape grows increasingly complex, mastery of indirect comparison methods and the principled selection of comparators—guided by frameworks such as the Pragmatic Model—will be indispensable for researchers, clinicians, and health policy makers. By rigorously applying these principles, the scientific community can ensure that decisions about the relative value of medical interventions are based on the most valid and least biased evidence possible.

A Practical Guide to ITC Methodologies: From Bucher to Bayesian and Population Adjustment

In the field of health technology assessment (HTA) and drug development, Indirect Treatment Comparisons (ITCs) have become indispensable statistical tools for evaluating the relative efficacy and safety of interventions when head-to-head randomized clinical trial (RCT) data are unavailable or infeasible [3]. The fundamental challenge facing researchers and drug development professionals lies in selecting the most appropriate ITC method from a growing arsenal of techniques, each with specific assumptions, data requirements, and limitations. This guide provides a structured decision framework based on evidence structure to navigate this complex methodological landscape, emphasizing the critical assessment of validity through the lens of common comparators research.

The necessity for ITCs arises from practical realities in global drug development: comparing a new treatment against all relevant market alternatives in head-to-head trials is often statistically impractical, economically unviable, or ethically constrained, particularly in oncology and rare diseases [24]. Furthermore, standard comparators vary significantly across jurisdictions, making single-trial comparisons insufficient for global market access [1]. ITCs address this evidence gap by enabling comparative effectiveness research through statistical linking of different studies, with the common comparator serving as the anchor that facilitates this indirect evidence synthesis [1].

Fundamental ITC Methodology and Classifications

Core Principles and Terminology

ITCs encompass a broad range of methods with inconsistent terminologies across the literature [3]. At their core, all ITCs aim to provide estimates of relative treatment effects between interventions that have not been directly compared in RCTs, using a common comparator as the statistical bridge. This common comparator (often a standard of care, placebo, or active control) enables the transitive linking of evidence across separate studies [1].

The validity of any ITC depends on satisfying fundamental assumptions that vary by method class. The constancy of relative effects assumption requires that treatment effects remain stable across the studies being compared, encompassing homogeneity (similar trial characteristics), similarity (comparable patient populations and trial designs), and consistency (coherence between direct and indirect evidence where available) [3]. When these assumptions are violated, methods based on conditional constancy may be employed, which incorporate effect modifiers through statistical adjustment [3].

Classification of ITC Methods

ITC methods can be categorized into four primary classes based on their underlying assumptions and the number of comparisons involved [3]:

• Bucher Method: Also known as adjusted or standard ITC, this approach facilitates pairwise comparisons through a common comparator using frequentist framework.
• Network Meta-Analysis (NMA): Extends the Bucher method to multiple interventions simultaneously, available in both frequentist and Bayesian frameworks.
• Population-Adjusted Indirect Comparisons (PAIC): Encompasses techniques that adjust for population imbalances across studies when individual patient data (IPD) are available.
• Naïve ITC: Refers to unadjusted comparisons that do not account for differences in study populations or characteristics.

The following table summarizes the key ITC methods, their applications, and fundamental requirements:

Table 1: Classification of Indirect Treatment Comparison Methods

Method Category	Specific Methods	Evidence Structure Required	Data Requirements	Key Assumptions
Unadjusted Methods	Bucher ITC [3]	Two interventions connected via common comparator	Aggregate data (AD)	Constancy of relative effects
	Naïve ITC [3]	Interventions with no common comparator	AD	None (highly prone to bias)
Multiple Treatment Comparisons	Network Meta-Analysis (NMA) [3]	Connected network of multiple interventions	Primarily AD	Homogeneity, similarity, consistency
	Indirect NMA [3]	Multiple interventions with only indirect connections	AD	Homogeneity, similarity
	Mixed Treatment Comparison (MTC) [3]	Network with both direct and indirect evidence	AD	Homogeneity, similarity, consistency
Population-Adjusted Methods	Matching-Adjusted Indirect Comparison (MAIC) [3]	Pairwise comparisons with population imbalance	IPD for one trial, AD for another	Constancy of relative or absolute effects
	Simulated Treatment Comparison (STC) [3]	Pairwise comparisons with population imbalance	IPD for one trial, AD for another	Constancy of relative or absolute effects
Effect Modifier Adjustment	Network Meta-Regression (NMR) [3]	Connected network with effect modifiers	AD with study-level covariates	Conditional constancy with shared effect modifier
	Multi-Level NMA (ML-NMR) [3]	Connected network with effect modifiers	IPD for some trials, AD for others	Conditional constancy with shared effect modifier

Decision Framework for ITC Method Selection

Selecting the appropriate ITC method requires systematic evaluation of the available evidence structure, data resources, and clinical context. The following decision pathway provides a visual representation of the key considerations in method selection:

Decision Pathway for Selecting ITC Methods

This decision framework emphasizes that method selection depends primarily on three factors: the connectedness of the evidence network, the comparability of patient populations across studies, and the availability of data for adjustment. The pathway systematically guides researchers through these considerations to arrive at methodologically appropriate options.

Evidence Structure Assessment

The initial evidence assessment involves mapping all available comparative evidence to identify potential connecting pathways between the target interventions. This process includes:

• Systematic Literature Review: Comprehensive identification of all relevant RCTs for the interventions of interest and potential common comparators [3].
• Evidence Network Mapping: Visual representation of treatment comparisons as a network where nodes represent interventions and edges represent direct comparative evidence [3].
• Feasibility Evaluation: Assessment of whether the available evidence network supports connected indirect comparisons or requires more advanced adjustment methods.

When the evidence structure reveals a connected network with multiple interventions, NMA approaches are typically preferred as they enable simultaneous comparison of all interventions while borrowing strength from the entire network [3]. For simple pairwise comparisons through a common comparator, the Bucher method provides a straightforward approach, though its validity depends heavily on population similarity [3].

Data Requirements and Method Capabilities

Different ITC methods have varying data requirements and capabilities for addressing methodological challenges. The choice between them often depends on the availability of individual patient data (IPD) and the presence of effect modifiers:

Table 2: Data Requirements and Applications of Advanced ITC Methods

Method	Data Requirements	Analytical Framework	Key Applications	Limitations
Matching-Adjusted Indirect Comparison (MAIC) [3]	IPD for index treatment, AD for comparator	Frequentist, often with propensity score weighting	Adjusting for population imbalances in pairwise comparisons; single-arm studies in rare diseases	Limited to pairwise comparisons; requires adequate IPD quality and sample overlap
Simulated Treatment Comparison (STC) [3]	IPD for index treatment, AD for comparator	Bayesian, often with outcome regression models	Addressing cross-study heterogeneity; unanchored comparisons	Limited to pairwise comparisons; model specification challenges
Network Meta-Regression (NMR) [3]	AD with study-level covariates	Frequentist or Bayesian	Exploring impact of study-level covariates on treatment effects; connected networks with effect modifiers	Cannot adjust for patient-level effect modifiers; not suitable for multi-arm trials
Multi-Level NMA (ML-NMR) [3]	IPD for some trials, AD for others	Bayesian with hierarchical models	Complex networks with both IPD and AD; patient-level effect modifier adjustment	Computational complexity; requires substantial statistical expertise

Experimental Protocols and Validation Procedures

Core Analytical Workflow for ITC Implementation

Implementing a robust ITC requires meticulous attention to methodological details and validation procedures. The following diagram outlines the standard workflow for conducting and validating ITC analyses:

Standard Workflow for ITC Implementation

Protocol Details for Key ITC Methods

Network Meta-Analysis Protocol

NMA implementation requires specific methodological steps to ensure validity:

• Data Preparation: Extract relative treatment effects (log odds ratios, log hazard ratios) and their variances from each study. For time-to-event outcomes, extract number of events and log-rank statistics or hazard ratios [3].
• Consistency Assessment: Use node-splitting techniques to evaluate disagreement between direct and indirect evidence where both exist [3].
• Statistical Modeling: Implement either frequentist (using multivariate meta-analysis) or Bayesian approaches (using Markov Chain Monte Carlo methods with non-informative priors) [3].
• Uncertainty Quantification: Present results with confidence/credible intervals and ranking probabilities, supplemented by sensitivity analyses exploring the impact of inclusion criteria and model choices [3].

Population-Adjusted ITC Protocol

MAIC implementation follows a distinct protocol when IPD is available for at least one study:

• Covariate Selection: Identify effect modifiers based on clinical knowledge and preliminary analyses [3].
• Propensity Score Estimation: Fit a logistic regression model comparing the index trial IPD to the aggregate comparator trial population [3].
• Weight Calculation: Assign weights to IPD patients using the method of moments to achieve balance on selected effect modifiers [3].
• Outcome Analysis: Fit weighted regression models to the IPD and combine with aggregate results from the comparator trial [3].
• Assess Effective Sample Size: Evaluate the loss of precision due to weighting and conduct bootstrap resampling for uncertainty estimation [3].

The acceptability of different ITC methods varies across HTA bodies worldwide, with clear preferences for certain methodologies based on their ability to minimize bias and adjust for confounding factors.

Current Acceptance Patterns

Recent analyses of HTA submissions reveal distinct patterns in the acceptance of various ITC methods:

Table 3: HTA Body Preferences and Acceptance of ITC Methods

HTA Body	Preferred Methods	Less Favored Methods	Key Considerations
European Medicines Agency (EMA) [24]	NMA, Population-adjusted methods	Naïve comparisons	Justification of similarity assumption; adequacy of statistical methods
Canada's Drug Agency (CDA-AMC) [24]	Anchored ITCs, MAIC	Unadjusted comparisons	Transparency; adjustment for cross-trial differences
Australian PBAC [24]	NMA, Adjusted comparisons	Unanchored comparisons	Clinical homogeneity; appropriate connectivity
French HAS [24]	PAIC, NMA	Naïve ITCs	Methodological rigor; relevance to decision context
German G-BA [24]	Advanced adjusted methods	Unadjusted ITCs (84% rejection rate)	Comprehensive adjustment for confounding

Impact on Reimbursement Decisions

The strategic selection of ITC methods has demonstrated tangible impacts on HTA outcomes. Recent evidence indicates that orphan drug submissions incorporating ITCs were associated with a higher likelihood of positive recommendations compared to non-orphan submissions [24]. Furthermore, submissions employing population-adjusted or anchored ITC techniques were more favorably received by HTA bodies compared to those using naïve or unadjusted comparisons, reflecting agency preferences for methods with robust bias mitigation capabilities [24].

Analysis of recent oncology submissions reveals that among 188 unique HTA recommendations supported by 306 ITCs, authorities demonstrated greater acceptance of methods that explicitly addressed cross-study heterogeneity through statistical adjustment [24]. This underscores the importance of aligning method selection with both the evidence structure and HTA body expectations.

Essential Research Reagent Solutions

Implementing robust ITCs requires both methodological expertise and appropriate analytical tools. The following table details key resources in the ITC researcher's toolkit:

Table 4: Essential Research Reagent Solutions for ITC Implementation

Tool Category	Specific Solutions	Primary Function	Application Context
Statistical Software	R (gemtc, pcnetmeta) [3]	Bayesian NMA implementation	Complex evidence networks with sparse data
	Stata (mvmeta, network) [3]	Frequentist NMA	Standard NMA with aggregate data
	SAS (PROC NLMIXED) [3]	Custom ITC implementation	Advanced simulation studies
Specialized Packages	R (MAIC, SIC) [3]	Population-adjusted comparisons	Individual patient data scenarios
	OpenBUGS/JAGS [3]	Bayesian hierarchical modeling	Complex evidence structures
Quality Assessment	Cochrane Risk of Bias [3]	Study quality evaluation	Evidence assessment phase
	GRADE for NMA [3]	Evidence quality rating	Results interpretation
Data Visualization	Network graphs [3]	Evidence structure mapping	Study planning and reporting
	Contribution plots [3]	Source of evidence visualization	Transparency in NMA

Successful application of these tools requires interdisciplinary collaboration between health economics and outcomes research (HEOR) scientists and clinical experts. HEOR scientists contribute methodological expertise in identifying available evidence and designing statistically sound comparisons, while clinicians provide essential context for evaluating the clinical plausibility of assumptions and the relevance of compared populations and outcomes [3]. This collaboration ensures that selected ITC methods are both methodologically robust and clinically credible for HTA submissions.

In the evaluation of new health technologies, head-to-head randomized controlled trials (RCTs) are considered the gold standard for evidence. However, it is frequently unethical, unfeasible, or impractical to conduct direct comparison trials for all relevant treatment options, particularly in rapidly evolving therapeutic areas or for rare diseases [2]. In such situations, indirect treatment comparisons (ITCs) become indispensable analytical tools for health technology assessment (HTA) bodies and drug developers needing to make evidence-based decisions [3] [2].

Among the various ITC techniques, the Bucher method represents a foundational approach for simple pairwise comparisons through a common comparator. First described by Bucher et al. in 1997, this method addresses the common scenario where two treatments (B and C) have been compared with the same control treatment (A) in separate studies but have not been directly compared with each other [25]. Statistical methods for indirect comparisons have seen increasing use in HTA reviews, with the Bucher method serving as a fundamental technique for evidence synthesis when direct evidence is lacking [26].

The accessibility and implementation simplicity of the Bucher method have contributed to its enduring relevance. Recent publications continue to highlight its utility, with researchers providing simple, easy-to-use tools such as Excel spreadsheets to facilitate practical application of these techniques by researchers and HTA bodies [25]. This guide examines the foundational techniques of the Bucher method, its statistical properties, implementation protocols, and performance relative to other comparison methods.

Methodological Foundations and Key Assumptions

Conceptual Framework and Basic Principles

The Bucher method, also termed adjusted indirect treatment comparison or standard ITC, operates on a simple network structure where two interventions (B and C) are connected through a common comparator (A) [3]. The core statistical principle involves deriving the indirect comparison between B and C by combining the results from direct comparisons of A versus B and A versus C [25].

For ratio effect estimates such as odds ratios (OR), risk ratios (RR), or hazard ratios (HR), calculations are performed on a logarithmic scale. The indirect effect estimate for B versus C is calculated as the difference between the log-effect estimates of A versus B and A versus C [25] [26]. The variance of the indirect estimate equals the sum of the variances of the two direct comparisons, which directly impacts the confidence interval width of the indirect comparison [25].

Table 1: Core Statistical Components of the Bucher Method

Component	Formula	Explanation
Effect Estimate	d_BC = d_AB - d_AC	Indirect comparison of B vs. C derived from direct comparisons of A vs. B and A vs. C
Variance	Var(d_BC) = Var(d_AB) + Var(d_AC)	Variance of indirect estimate is sum of variances of direct estimates
95% Confidence Interval	d_BC ± 1.96 × √Var(d_BC)	Confidence interval for the indirect comparison

Figure 1: Simple Network for Bucher Indirect Comparison. The Bucher method enables comparison between treatments B and C through common comparator A when direct evidence is unavailable [25].

Critical Assumptions for Valid Application

The validity of the Bucher method rests on several fundamental assumptions that must be carefully assessed before application. The transitivity assumption requires that the A versus B and A versus C trials should not differ with respect to potential effect modifiers, such as participant characteristics, eligibility criteria, or treatment regimens in the shared arm [25]. When this assumption is met, the similarity assumption is satisfied, meaning the studies are sufficiently comparable to allow meaningful indirect comparison [3].

The homogeneity assumption requires that relative treatment effects are consistent across trials comparing the same interventions. For the Bucher method to provide valid results, there should be no important clinical or methodological heterogeneity between the studies being compared [25] [3]. Violations of these assumptions can introduce bias and compromise the validity of the indirect comparison.

Table 2: Key Assumptions of the Bucher Method

Assumption	Definition	Assessment Method
Transitivity	The A vs. B and A vs. C trials do not differ in potential effect modifiers	Comparison of study characteristics, participant eligibility, treatment regimens
Similarity	Studies are comparable with respect to all important effect modifiers	Evaluation of study designs, populations, interventions, outcomes, and methodologies
Homogeneity	Consistent relative treatment effects across trials comparing same interventions	Statistical tests for heterogeneity (I², Q-statistic), comparison of point estimates

Implementation Protocols and Experimental Validation

Step-by-Step Application Workflow

Implementing the Bucher method requires a systematic approach to ensure methodological rigor. The first critical step involves a comprehensive systematic review to identify all relevant studies comparing A versus B and A versus C. This should follow established guidelines to minimize selection bias and ensure all available evidence is considered [25] [2].

Next, researchers must conduct a thorough assessment of transitivity by comparing study characteristics, participant demographics, intervention details, and outcome definitions across the identified trials. This qualitative assessment helps verify whether the fundamental assumption of similarity is plausible [25]. If the direct comparisons come from multiple trials, pairwise meta-analyses should be performed to generate summary effect estimates for A versus B and A versus C, using either fixed-effect or random-effects models depending on the presence of heterogeneity [25].

The statistical combination follows, applying the Bucher formulas to derive the indirect estimate and its variance. Finally, certainty assessment using established frameworks like GRADE is essential, as the certainty of evidence for the indirect comparison cannot be higher than the certainty for either of the two direct comparisons used in the analysis [25].

Figure 2: Bucher Method Implementation Workflow. The process begins with evidence identification and proceeds through transitivity assessment, statistical analysis, and certainty evaluation [25].

Experimental Validation and Case Study Application

The statistical properties of the Bucher method have been rigorously evaluated through simulation studies. When there are no biases in primary studies, the Bucher method is on average unbiased. However, depending on the extent and direction of biases in different sets of studies, it may be more or less biased than direct treatment comparisons [26]. The method has been shown to have larger mean squared error (MSE) compared to direct comparisons and more complex mixed treatment comparison methods, reflecting the additional uncertainty introduced through the indirect comparison process [26].

A practical application of the Bucher method is demonstrated in a Cochrane review on techniques to preserve donated livers for transplantation [25]. Both cold and warm machine perfusion had been compared with standard ice-box storage in several randomized trials, but no trials directly compared cold versus warm machine perfusion. After confirming no important differences in potential effect modifiers and no statistical heterogeneity in the pairwise meta-analyses, researchers applied the Bucher method, yielding an indirect HR of 0.38 (95% CI 0.11 to 1.25, p=0.11) for cold versus warm machine perfusion [25].

This case illustrates several key points: despite high-certainty evidence that cold machine perfusion is superior to standard storage, while warm machine perfusion appears no better, the indirect comparison provided low certainty evidence with a wide confidence interval crossing 1.0, leaving uncertainty about which perfusion technique is superior [25]. This highlights how the precision of indirect comparisons is inherently less than for direct comparisons due to the additive variance component in the Bucher method.

Comparative Performance and Methodological Context

Performance Relative to Other Comparison Methods

The relative performance of the Bucher method has been systematically evaluated against other comparison approaches. Simulation studies comprehensively investigating statistical properties have revealed that the Bucher method has the largest mean squared error among commonly used ITC and mixed treatment comparison methods [26]. Direct treatment comparisons consistently demonstrate superiority to indirect comparisons in terms of both statistical power and mean squared error [26].

When comparing the Bucher method to more complex network meta-analysis (NMA) approaches, for the simple network of three treatments with a common comparator, frequentist NMA generates identical results to the Bucher method [25]. This equivalence has been demonstrated in practical applications, where both approaches yield the same point estimates and confidence intervals for the indirect comparison [25].

Table 3: Performance Comparison of Treatment Comparison Methods

Method	Strength	Limitation	Best Application Context
Bucher Method	Simple implementation, accessible to non-statisticians, requires only aggregate data	Limited to comparisons with common comparator, largest MSE, lower precision	Simple networks with three treatments connected through common comparator
Network Meta-Analysis	Simultaneous multiple treatment comparisons, more precise estimates	Complex implementation, challenging assumption verification	Complex networks with multiple interconnected treatments
Population-Adjusted ITC	Adjusts for population imbalances, can address some transitivity violations	Requires individual patient data, strong assumptions about effect modifiers	Studies with considerable heterogeneity in population characteristics
Direct Treatment Comparison	Highest validity, greatest precision, minimizes confounding	Often unavailable, resource-intensive to obtain	Gold standard when feasible and available

Appropriate Application Contexts and Limitations

The Bucher method is particularly well-suited for specific clinical scenarios that commonly arise in therapeutic development. It is ideally applied when two new treatments or procedures are developed and assessed against placebo or standard of care rather than each other, or when clinicians and patients consider two different treatments as suitable options but need to weigh potential benefits and harms carefully [25]. The Cochrane Handbook specifically recommends the Bucher method for simple networks where two interventions are connected through a common comparator [25].

The method does have recognized limitations. It is restricted to pairwise indirect comparisons and cannot be used for complex networks with multiple interconnected treatments [3]. The requirement for a common comparator means it cannot be applied in disconnected networks where no such comparator exists [27]. Additionally, the method is particularly sensitive to violations of the transitivity assumption, which can introduce significant bias if effect modifiers are imbalanced across comparisons [25] [3].

Recent analyses of HTA guidelines indicate that while the Bucher method remains accepted for appropriate simple networks, there is increasing methodological expectation for more sophisticated approaches when transitivity concerns exist or when more complex networks need to be analyzed [27]. Health technology assessment bodies generally express a clear preference for direct comparisons, with indirect comparisons like the Bucher method accepted on a case-by-case basis when direct evidence is unavailable [2].

Successful implementation of the Bucher method requires access to specific methodological resources and analytical tools. Researchers should be familiar with key materials that facilitate rigorous application of this indirect comparison approach.

Table 4: Essential Research Reagent Solutions for Bucher Method Implementation

Tool/Resource	Function	Implementation Notes
Systematic Review Protocols	Identify all relevant studies for direct comparisons	PRISMA guidelines, predefined search strategy, inclusion/exclusion criteria
Risk of Bias Assessment Tools	Evaluate methodological quality of included studies	Cochrane RoB tool for randomized trials, assess impact of biases on indirect comparison
Statistical Software	Perform pairwise meta-analyses and Bucher calculations	Excel spreadsheet tools, R, Stata, or specialized meta-analysis software
Transitivity Assessment Framework	Systematically evaluate similarity assumption	Structured comparison of study characteristics, participants, interventions, outcomes
GRADE Framework	Assess certainty of evidence from indirect comparison	Rate down for imprecision, intransitivity, and other limitations

Recent developments have focused on increasing the accessibility and implementation of the Bucher method for applied researchers. User-friendly tools such as the Excel spreadsheet referenced in recent literature provide practical resources for performing these calculations without requiring advanced statistical programming skills [25]. These tools often include additional utilities for calculating confidence intervals from p values and handling situations where treatment effects are reported in different directions [25].

For the statistical implementation, both frequentist and Bayesian approaches are available, though the frequentist approach is more commonly used for the Bucher method specifically. The Bayesian approach may offer advantages when dealing with sparse data or when incorporating prior knowledge, but for the simple network shown in Figure 1, both approaches yield similar results when uninformative priors are used [25].

The Bucher method remains a fundamental technique in the evidence synthesis toolkit, particularly valuable for straightforward indirect comparisons in situations commonly encountered in therapeutic development and health technology assessment. Its accessibility and methodological transparency ensure its continued relevance despite the development of more complex network meta-analysis approaches for handling more elaborate evidence structures.

Network Meta-Analysis (NMA), also known as Mixed Treatment Comparison (MTC), is an advanced statistical methodology that enables the simultaneous comparison of multiple interventions by synthesizing both direct evidence (from head-to-head randomized trials) and indirect evidence (estimated through a common comparator) within a single, coherent analysis [28] [29] [30]. This guide provides a comparative framework for understanding NMA, its key assumptions, statistical approaches, and application in clinical research and drug development.

Core Concepts and Definitions

Direct Evidence: Comes from studies that directly compare two interventions of interest (e.g., Intervention A vs. Intervention B) within a randomized trial [30].

Indirect Evidence: An estimate of the relative effect of two interventions that have not been directly compared in a trial, obtained by leveraging their common comparisons to a third intervention [31] [30]. For example, if A has been compared to C, and B has been compared to C, the indirect estimate for A vs. B can be derived mathematically.

Network Meta-Analysis: A comprehensive analysis that integrates all direct and indirect evidence across a network of three or more interventions, producing pooled effect estimates for all possible pairwise comparisons [28] [31] [30].

The following diagram illustrates how direct and indirect evidence form a connected network, allowing for the estimation of treatment effects that may never have been directly studied.

Figure 1: A Network Diagram Illustrating Direct and Indirect Evidence. Nodes (circles) represent different interventions. Solid lines represent direct comparisons from head-to-head trials. Dashed lines represent indirect comparisons that can be statistically estimated.

Fundamental Assumptions and Validation

The validity of any NMA hinges on three fundamental assumptions, which must be critically assessed.

Transitivity

This conceptual assumption requires that the different sets of studies included for the various direct comparisons are sufficiently similar in all important factors that could influence the relative treatment effects (effect modifiers), such as patient population, study design, or outcome definitions [31] [3]. For an indirect comparison A vs. B via C to be valid, the A vs. C and B vs. C trials must be "jointly randomizable"—meaning that, in principle, the patients in one set of trials could have been enrolled in the other [31].

Consistency

This is the statistical manifestation of transitivity [31]. It means that the direct evidence and the indirect evidence for a specific treatment comparison are in agreement [28] [3]. For example, within a closed loop (e.g., A-B-C), the direct estimate of A vs. C should be consistent with the indirect estimate of A vs. C obtained via B. Significant inconsistency, or incoherence, suggests a violation of the transitivity assumption and undermines the network's validity [28].

Similarity

Also referred to as homogeneity, this assumption requires that the studies contributing to each direct comparison are sufficiently similar to each other. This is analogous to the assumption in a standard pairwise meta-analysis [28] [3].

Statistical Frameworks for NMA

NMA can be implemented within two primary statistical frameworks, each with distinct advantages. The table below summarizes the key features of the Bayesian and Frequentist approaches.

Table 1: Comparison of Bayesian and Frequentist Frameworks for Network Meta-Analysis

Feature	Bayesian Framework	Frequentist Framework
Core Philosophy	Updates prior beliefs with observed data to produce a posterior probability distribution [28] [32].	Relies on the frequency properties of estimators; does not incorporate prior knowledge [3].
Result Presentation	Credible Intervals (CrI), which can be interpreted as the range in which the true effect lies with a certain probability [28].	Confidence Intervals (CI), which represent the range in which the true effect would lie in repeated sampling [28].
Treatment Ranking	Directly outputs probabilities for each treatment being the best, second best, etc., using metrics like SUCRA (Surface Under the Cumulative Ranking curve) [29] [32].	Provides P-scores, which are frequentist analogues to ranking probabilities [3].
Handling Complexity	Highly flexible for complex models, incorporation of different sources of evidence, and prediction [28] [32].	Standardized packages are available, often with a gentler learning curve for simpler networks [32].
Common Software	WinBUGS/OpenBUGS [28], JAGS [32], R packages (gemtc, BUGSnet) [32].	R packages (netmeta) [32], STATA [28].

Workflow for a Bayesian NMA

The Bayesian framework is currently the most common approach for NMA, particularly for complex networks [32]. The following diagram outlines a typical workflow for conducting a Bayesian NMA.

Figure 2: Workflow for a Bayesian Network Meta-Analysis. The process begins with a systematic review and progresses through model specification, computation, and validation.

Experimental Protocols and Analytical Steps

Protocol 1: Performing an Adjusted Indirect Comparison (Bucher Method)

This is the foundational method for a simple indirect comparison involving three interventions (A, B, C) where B is the common comparator [31] [30] [3].

• Conduct Pairwise Meta-Analyses: Perform standard meta-analyses for the direct comparisons A vs. B and A vs. C.
• Calculate the Indirect Estimate: The indirect log odds ratio (OR) for B vs. C is calculated as: ( \log(OR{BC}^{indirect}) = \log(OR{AC}) - \log(OR{AB}) ) where ( OR{AC} ) and ( OR_{AB} ) are the summary ORs from the direct meta-analyses [31].
• Calculate the Variance: The variance of the indirect log OR is: ( Var(\log(OR{BC}^{indirect})) = Var(\log(OR{AC})) + Var(\log(OR_{AB})) ) [31]
• Construct Confidence Interval: The 95% CI is derived using the calculated variance and the normal approximation [31].

Protocol 2: Assessing Inconsistency in a Network

Checking for disagreement between direct and indirect evidence is a critical step.

• Node-Splitting Method: This is a common technique. It involves separating ("splitting") the direct evidence for a particular comparison from the indirect evidence [28] [31].
• Compare Estimates: The model estimates the treatment effect using only the direct evidence and again using only the indirect evidence for the same comparison.
• Statistical Test: A statistical test is performed to evaluate if the difference between the direct and indirect estimates is significant. A non-significant p-value supports the consistency assumption [28].

Table 2: Key "Research Reagent Solutions" for Conducting a Network Meta-Analysis

Item / Resource	Function / Explanation
Systematic Review Protocol (PRISMA-NMA)	A pre-defined, registered protocol ensures the review is comprehensive, transparent, and minimizes bias, forming the foundation of a valid NMA [33].
Effect Modifier Table	A structured table listing patient or study characteristics (e.g., age, disease severity) that may influence treatment effects. Used to assess the transitivity assumption [31] [3].
Risk of Bias Tool (e.g., Cochrane RoB 2)	A critical appraisal tool to assess the methodological quality of individual randomized trials, as biased studies can distort network estimates [31].
GRADE for NMA	A framework for rating the overall confidence (quality) in the evidence generated by the NMA, beginning with confidence in each direct comparison [31].
Statistical Software (R with gemtc/BUGSnet)	Software packages that provide the computational engine for fitting Bayesian NMA models, running MCMC simulations, and generating outputs like rankings and forest plots [32].
JAGS / WinBUGS	Platform-independent programs for Bayesian analysis that use Markov Chain Monte Carlo (MCMC) methods. They are often called from within R to perform the actual model fitting [28] [32].

Indirect treatment comparisons (ITCs) are essential methodologies for assessing the relative efficacy and safety of medical interventions when direct head-to-head randomized controlled trials are unavailable, unethical, or impractical to conduct [2]. Standard network meta-analysis (NMA) combines aggregate data from multiple studies but relies on the assumption that study populations are sufficiently similar with respect to effect modifiers—variables that influence the relative treatment effect [34] [15]. When this assumption is violated due to heterogeneity across trial populations, population adjustment methods are necessary to produce valid comparative estimates.

Matching-Adjusted Indirect Comparison (MAIC) and Multilevel Network Meta-Regression (ML-NMR) have emerged as two prominent population adjustment techniques that utilize individual patient data (IPD) from one or more studies to adjust for cross-trial differences in effect modifiers [34] [15]. MAIC is a reweighting approach that balances the distribution of effect modifiers between IPD and aggregate data (AgD) studies, while ML-NMR is an extension of the NMA framework that integrates an individual-level regression model with aggregate data by accounting for the entire covariate distribution in AgD studies [15]. This guide provides a comprehensive objective comparison of these methodologies, focusing on their theoretical foundations, performance characteristics, and appropriate applications within evidence synthesis for healthcare decision-making.

Theoretical Foundations and Methodological Frameworks

Matching-Adjusted Indirect Comparison (MAIC)

MAIC was developed specifically for scenarios where researchers have access to IPD from one study (typically their own trial) but only published aggregate data from a comparator study [15]. The method employs a weighting approach to create a "pseudo-population" from the IPD that matches the aggregate covariate distribution of the comparator study. The core mechanism involves estimating weights for each individual in the IPD study such that the weighted moments of their baseline characteristics align with those reported for the AgD study population [15]. These weights are typically derived using method of moments or entropy balancing techniques.

The MAIC approach produces population-adjusted treatment effects specific to the AgD study population. A significant limitation is that MAIC does not naturally generalize to larger networks involving multiple treatments and studies [34]. Furthermore, the method is primarily designed for anchored comparisons where studies share a common comparator, though unanchored applications exist with additional strong assumptions [35].

Multilevel Network Meta-Regression (ML-NMR)

ML-NMR represents a more recent advancement that extends the standard NMA framework to coherently synthesize evidence from networks of any size containing mixtures of IPD and AgD [34] [15]. Unlike MAIC, ML-NMR defines an individual-level regression model that is fitted directly to the IPD and incorporates AgD by integrating this model over the covariate distribution in each aggregate study. This approach avoids aggregation bias and noncollapsibility issues that can affect other methods [34].

A key advantage of ML-NMR is its ability to produce estimates for any target population of interest, not just the populations of the included studies [34]. The method reduces to standard AgD NMA when no covariates are adjusted for and to full-IPD network meta-regression when IPD are available from all studies, making it a flexible generalization of existing approaches [34].

Table 1: Key Methodological Characteristics of MAIC and ML-NMR

Characteristic	MAIC	ML-NMR
Data Requirements	IPD from ≥1 study, AgD from others	IPD from ≥1 study, AgD from others
Core Mechanism	Reweighting IPD to match AgD covariate moments	Integrating individual-level model over AgD covariate distribution
Network Size	Designed for 2-study comparisons; extensions problematic	Networks of any size and complexity
Target Population	AgD study population only	Any specified target population
Assumption Checking	Limited capability for assessing key assumptions	Enables assessment of conditional constancy and shared effect modifier assumptions
Statistical Properties	Prone to aggregation bias in nonlinear models	Avoids aggregation and noncollapsibility biases

Performance Comparison: Experimental Data and Simulation Results

Bias Performance Under Ideal and Non-Ideal Conditions

Simulation studies provide critical insights into the performance of population adjustment methods under various scenarios. According to extensive simulation assessments, ML-NMR and Simulated Treatment Comparison (STC, a regression-based approach similar in some aspects to ML-NMR) generally eliminate bias when all effect modifiers are included in the model and the requisite assumptions are met [15]. In contrast, MAIC has demonstrated poor performance in nearly all simulation scenarios, sometimes even increasing bias compared with standard unadjusted indirect comparisons [15].

All population adjustment methods incur bias when important effect modifiers are omitted from the analysis, highlighting the critical importance of carefully selecting potential effect modifiers based on expert clinical opinion, systematic review, or quantitative analyses of external evidence prior to analysis [15]. When trial populations exhibit substantial differences in effect modifier distributions, methods that adequately adjust for these differences (ML-NMR and STC) outperform both standard indirect comparisons and MAIC.

Table 2: Performance Comparison Based on Simulation Studies

Performance Metric	MAIC	ML-NMR
Bias with All Effect Modifiers	Poor in nearly all scenarios	Minimal when assumptions met
Bias with Missing Effect Modifiers	Substantial	Substantial
Efficiency	Variable; can be low with extreme weights	Generally good with uncertainty reduced by explaining variation
Handling of Non-Linear Models	Problematic due to aggregation bias	Appropriate, avoids aggregation bias
Robustness to Violations of Shared Effect Modifier Assumption	Poor	Good when assumptions can be assessed and relaxed

Application to Real-World Evidence: Plaque Psoriasis Case Study

A practical application of these methods involved a network of treatments for moderate-to-severe plaque psoriasis, comprising 9 studies comparing 6 active treatments and placebo with a mix of IPD and AgD [34]. The analysis adjusted for potential effect modifiers including duration of psoriasis, previous systemic treatment, body surface area covered, weight, and psoriatic arthritis.

In this real-world application, ML-NMR demonstrated better model fit than standard NMA and reduced uncertainty by explaining within- and between-study variation [34]. The estimated population-average treatment effects were similar across study populations because differences in the distributions of effect modifiers were relatively small. Researchers found little evidence that the key assumptions of conditional constancy of relative effects or shared effect modifiers were invalid in this case [34].

Methodological Protocols and Implementation

MAIC Implementation Workflow

The standard protocol for implementing MAIC involves several sequential steps. First, researchers must identify potential effect modifiers based on clinical knowledge and prior evidence. Second, aggregate baseline characteristics are extracted from published reports of the comparator study. Third, weights are estimated for each individual in the IPD study such that the weighted covariate distribution matches the aggregate characteristics of the comparator study. Fourth, the outcomes from the reweighted IPD are compared with the published aggregate outcomes to estimate the population-adjusted treatment effect [15]. This workflow is visualized in the following diagram:

ML-NMR Implementation Workflow

The implementation of ML-NMR follows a more integrated statistical modeling approach. The process begins with specifying an individual-level regression model that includes treatment-covariate interactions for effect modifiers. This model is then fitted simultaneously to the IPD and AgD, with the integration over the covariate distribution in AgD studies performed numerically. Model fit can be assessed using residual heterogeneity and inconsistency checks, and key assumptions can be tested by relaxing the shared effect modifier assumption for each covariate in turn [34]. Finally, population-adjusted treatment effects can be produced for any target population with known covariate distribution. This comprehensive workflow is illustrated below:

Research Reagent Solutions: Methodological Components

Successful implementation of population adjustment methods requires specific methodological components that function as essential "research reagents" in the analytical process.

Table 3: Essential Methodological Components for Population Adjustment Analyses

Component	Function	Implementation Considerations
Individual Patient Data	Provides information on individual covariate-outcome relationships	IPD from at least one study is mandatory for both MAIC and ML-NMR
Aggregate Data	Supplies comparative evidence from other studies	Must include baseline covariate summaries and outcomes for population adjustment
Effect Modifier Selection	Identifies variables that interact with treatment effects	Should be prespecified based on external evidence to avoid selective reporting bias
Statistical Software	Implements complex estimation procedures	Specialized code required, particularly for ML-NMR integration procedures
Target Population Data	Defines the population for final estimates	Covariate distribution from registries, cohort studies, or specific populations of interest

Discussion and Comparative Guidance

The comparative analysis of MAIC and ML-NMR reveals distinct advantages and limitations that should guide method selection in practice. MAIC's primary limitations include restriction to specific network structures, production of estimates applicable only to the AgD study population, and performance issues identified across simulation scenarios [15]. ML-NMR addresses many of these limitations by accommodating networks of any size, producing estimates for any target population, and providing frameworks for assessing key assumptions [34].

For researchers with access to IPD from at least one study, ML-NMR generally represents a more robust and flexible approach for population adjustment, particularly when dealing with complex treatment networks or when estimates are required for specific decision-making populations beyond the study populations [34] [15]. MAIC may still have a role in simple two-study comparisons where its computational simplicity is advantageous and its limitations are carefully considered.

Both methods share the critical requirement for identifying all important effect modifiers prior to analysis. Omission of relevant effect modifiers results in biased estimates regardless of the methodological sophistication [15]. This underscores the importance of thorough systematic review of potential treatment-effect modifiers and clinical expert input during the planning stages of population-adjusted indirect comparisons.

As health technology assessment agencies increasingly encounter these methods in submissions, understanding their relative performance, assumptions, and appropriate application contexts becomes essential for researchers, drug development professionals, and decision-makers involved in comparative effectiveness research.

Matching-Adjusted Indirect Comparison (MAIC) is a advanced statistical methodology used in health technology assessment (HTA) and comparative effectiveness research. It enables comparisons between treatments when head-to-head randomized controlled trials are unavailable, a common challenge in drug development, particularly in oncology and rare diseases [2]. MAIC operates by reweighting individual patient-level data (IPD) from one study to match the aggregate baseline characteristics of a comparator study for which only aggregate data (AgD) is available [36]. This process creates a balanced comparison platform, adjusting for cross-trial differences in patient populations that could otherwise bias treatment effect estimates [3].

The methodology has gained significant importance under evolving HTA frameworks like the EU HTA Regulation 2021/2282, which mandates joint clinical assessments and recognizes quantitative evidence synthesis methods including MAIC [12]. MAIC is particularly valuable in two scenarios: "anchored" comparisons where studies share a common comparator treatment, and the more methodologically challenging "unanchored" comparisons involving single-arm studies without a common control [37]. The latter relies on stronger assumptions about conditional constancy of absolute effects and requires careful handling to ensure validity [3].

Methodological Framework and Key Concepts

Core Principles and Assumptions

MAIC implementation rests on several foundational assumptions that must be carefully considered during study design. The positivity assumption requires adequate overlap in patient characteristics between the IPD and AgD populations to enable meaningful matching [37]. The exchangeability assumption (no unmeasured confounding) stipulates that all important effect modifiers and prognostic factors have been identified and included in the weighting model [37]. The consistency assumption maintains that the treatment effect is consistent across studies after proper adjustment [3].

The methodological framework operates on the principle of propensity score weighting, where weights are applied to the IPD cohort to create a pseudo-population that matches the AgD cohort's baseline characteristics [36]. This approach effectively simulates a conditional randomization scenario, balancing the distribution of covariates between the treatment groups being compared indirectly [3]. The method is particularly useful when there are limited treatment options and disconnected evidence networks, common scenarios in precision medicine and rare diseases [38].

MAIC in the Context of Other Indirect Comparison Methods

MAIC represents one of several population-adjusted indirect comparison (PAIC) methods available to researchers [3]. The broader landscape of indirect treatment comparisons includes both unadjusted methods like the Bucher method and network meta-analysis (NMA), and other adjusted approaches like simulated treatment comparison (STC) and network meta-regression [2]. The table below compares MAIC with other common indirect comparison methods:

Table 1: Comparison of Indirect Treatment Comparison Methods

Method	Data Requirements	Key Assumptions	Strengths	Limitations
MAIC	IPD for index treatment; AgD for comparator	No unmeasured confounding; adequate population overlap	Adjusts for cross-study differences; uses IPD more efficiently	Limited to pairwise comparisons; reduces effective sample size
Network Meta-Analysis	AgD from multiple studies	Consistency, homogeneity, similarity	Simultaneously compares multiple treatments; well-established methodology	Requires connected evidence network; challenging assumption verification
Bucher Method	AgD from two studies with common comparator	Constancy of relative effects	Simple implementation for connected networks	Limited to simple indirect comparisons; no population adjustment
Simulated Treatment Comparison	IPD for index treatment; AgD for comparator	Correct specification of outcome model	Models treatment effect directly; can incorporate effect modifiers	Relies on correct model specification; potentially high statistical uncertainty

Experimental Protocol for MAIC Implementation

Pre-Analysis Planning and Covariate Selection

A rigorous MAIC implementation begins with comprehensive pre-specification of the statistical analysis plan to minimize data dredging and ensure transparency [12]. The target trial framework provides a structured approach for defining the protocol, specifying inclusion/exclusion criteria, treatment regimens, outcomes, and covariate selection based on clinical knowledge and literature review [37]. Covariate selection should prioritize prognostic factors and effect modifiers known to influence the outcome of interest, rather than including all available baseline variables [3].

The protocol should explicitly document the variable selection process, including the clinical rationale for each included covariate [37]. For the case study in metastatic ROS1-positive NSCLC, researchers pre-specified covariates including age, gender, ECOG Performance Status, tumor histology, smoking status, and brain metastases based on clinical expert input and literature review [37]. This transparent pre-specification is crucial for HTA submission acceptance, as it demonstrates methodological rigor and reduces concerns about selective reporting [12].

Weight Estimation and Balance Assessment

The core technical implementation of MAIC involves estimating weights using a method of moments approach [36]. This process involves solving a logistic regression model to find weights that balance the means of selected covariates between the IPD and AgD populations [38]. The optimization can be represented as finding weights ( w_i ) that satisfy the condition:

[ \sum wi \cdot Xi = \bar{X}_{AgD} ]

where ( Xi ) represents covariates from the IPD and ( \bar{X}{AgD} ) represents the aggregate means from the comparator study [36].

Following weight estimation, researchers must assess covariate balance between the weighted IPD population and the AgD comparator. Standardized mean differences should be calculated for each covariate, with values below 0.1 indicating adequate balance [37]. The effective sample size (ESS) of the weighted population should also be calculated, as substantial reduction indicates that the weights are highly variable, which increases variance and reduces statistical precision [38].

Outcome Analysis and Uncertainty Estimation

After achieving satisfactory balance, the outcome analysis proceeds by applying the estimated weights to the IPD and comparing the adjusted outcomes with the AgD [36]. For time-to-event outcomes like overall survival or progression-free survival, weighted Kaplan-Meier curves or weighted Cox proportional hazards models are typically used [37]. Treatment effect estimates (hazard ratios, risk ratios, or mean differences) should be reported with appropriate measures of uncertainty.

A critical challenge in MAIC is accounting for the additional uncertainty introduced by the weight estimation process. Bootstrapping or robust variance estimators should be employed to generate valid confidence intervals that reflect both the sampling uncertainty and the weighting uncertainty [37]. Some implementations use Bayesian approaches with regularization to address small sample sizes and improve stability of estimates [38].

Case Study: MAIC in ROS1-Positive NSCLC

The practical application of MAIC is illustrated through a case study comparing entrectinib with standard of care in metastatic ROS1-positive non-small cell lung cancer (NSCLC) [37]. This context represents typical MAIC application scenarios: a rare molecular subset where randomized trials are infeasible, with single-arm trials supporting accelerated approval. The intervention data came from an integrated analysis of three single-arm entrectinib trials (ALKA-372-001, STARTRK-1, and STARTRK-2) with reconstructed IPD for 60-64 patients [37]. The comparator data derived from the ESME Lung Cancer Data Platform, a real-world database containing 30 patients receiving standard therapies [37].

The primary outcome was progression-free survival (PFS), with a hierarchical testing strategy controlling Type I error at 5% (two-sided) [37]. The target trial framework defined precise inclusion criteria aligning both data sources, including adults with ROS1-positive metastatic NSCLC receiving first-line or second-line treatment, with careful consideration of outcome definition differences between clinical trials and real-world data [37].

Implementation Challenges and Solutions

The case study highlighted several practical MAIC challenges, particularly small sample sizes and missing data (approximately 50% missingness in ECOG Performance Status) [37]. Researchers addressed these through:

• Multiple imputation for missing baseline covariates using chained equations
• Predefined variable selection workflow with balance assessment to ensure convergence
• Transparent reporting of all modeling steps to avoid data dredging accusations [37]

Notably, the small sample size increased risks of model non-convergence and high variance, requiring careful implementation. The ESS after weighting was substantially reduced, reflecting the considerable adjustment needed to balance populations [37]. This precision loss is a fundamental MAIC limitation, particularly problematic when demonstrating superiority of new treatments [38].

Sensitivity Analyses and Bias Assessment

Comprehensive sensitivity analyses assessed result robustness to key assumptions. Quantitative bias analysis (QBA) for unmeasured confounding included E-value calculations and bias plots [37]. The E-value quantifies the minimum strength of association an unmeasured confounder would need to explain away the observed treatment effect [37]. Tipping-point analysis assessed the potential impact of violations of the missing-at-random assumption for ECOG Performance Status [37].

Table 2: Key Outcomes from ROS1-Positive NSCLC Case Study

Analysis Component	Implementation Details	Key Findings
Primary MAIC	Pre-specified covariates; method of moments weighting	Significant PFS improvement with entrectinib vs. standard of care
Effective Sample Size	Calculated post-weighting	Substantial reduction, reflecting considerable population differences
E-value Analysis	Assessed unmeasured confounding	Large E-values suggested robustness to potential unmeasured confounders
Tipping-point Analysis	Violation of missing data assumptions	Conclusions remained robust under plausible missing data mechanisms
Convergence Success	Predefined variable selection workflow	Achieved balance without convergence issues across all subpopulations

These extensive sensitivity analyses provided supporting evidence for the primary findings, demonstrating that results were robust to plausible violations of key assumptions [37]. This comprehensive approach addresses common HTA reviewer concerns about the validity of indirect comparisons, particularly for unanchored MAIC where assumptions are strongest [3].

Statistical Software and Packages

Several R packages provide specialized functionality for MAIC implementation, largely building on the foundational code from the National Institute for Health and Care Excellence (NICE) Technical Support Document 18 [36]. The table below summarizes key available resources:

Table 3: Research Reagent Solutions for MAIC Implementation

Tool/Resource	Type	Function	Implementation Considerations
R Package 'maic'	Software	Generalized workflow for MAIC weight generation	Native support for aggregate-level medians; CRAN availability ensures quality control
R Package 'maicChecks'	Software	Alternative weight calculation methods	Maximizes effective sample size; provides additional diagnostic capabilities
NICE TSD-18 Code	Methodology	Reference implementation for MAIC	Foundational code used by most packages; includes comprehensive theoretical background
Regularized MAIC	Methodological Extension	Addresses small sample size challenges	Uses L1/L2 penalties to improve effective sample size; particularly valuable in precision medicine

Recent methodological advancements include regularized MAIC implementations that apply L1 (lasso), L2 (ridge), or elastic net penalties during weight estimation to address small sample size challenges [38]. Simulation studies demonstrate these approaches can achieve better bias-variance tradeoffs, with markedly better ESS compared to default methods [38].

Methodological Frameworks and Guidelines

Successful MAIC implementation requires adherence to established methodological frameworks and emerging HTA guidelines. The target trial framework provides a structured approach for defining the protocol using observational data [37]. The EUnetHTA methodological guidelines for quantitative evidence synthesis provide specific recommendations for indirect comparisons, emphasizing pre-specification, transparency, and comprehensive sensitivity analyses [12].

Key documentation elements for regulatory and HTA submissions include:

• Pre-specified statistical analysis plan with covariate selection rationale [12]
• Comprehensive balance assessment before and after weighting [37]
• Detailed reporting of effective sample size and weight distributions [38]
• Uncertainty quantification that accounts for the weighting process [36]
• Sensitivity analyses for unmeasured confounding and missing data [37]

Implementing MAIC with transparency and rigor requires careful attention to methodological details throughout the analysis lifecycle. Based on current methodological research and case study experiences, several best practices emerge:

Pre-specification and transparency are fundamental to HTA acceptance [12]. Document all analytical decisions before conducting analyses, including covariate selection rationale, model specifications, and success criteria for balance assessment. Comprehensive sensitivity analyses should address potential biases from unmeasured confounding, missing data, and model specifications [37]. Appropriate uncertainty quantification must account for the weight estimation process, not just sampling variability [36].

Emerging methodologies like regularized MAIC offer promising approaches for addressing small sample size challenges, particularly in precision medicine contexts [38]. Quantitative bias analysis frameworks, including E-values and tipping-point analyses, provide structured approaches for communicating robustness to HTA bodies [37].

As HTA requirements evolve globally, particularly with implementation of the EU HTA Regulation, MAIC methods will continue to play important roles in evidence generation [12]. Maintaining methodological rigor while advancing statistical techniques will ensure these approaches provide reliable evidence for healthcare decision-making, ultimately supporting patient access to innovative treatments.

Navigating Real-World Challenges: Strategies for Heterogeneity, Bias, and Small Samples

In clinical research, direct evidence from head-to-head randomized controlled trials (RCTs) is traditionally considered the gold standard for comparing interventions. However, the rapid proliferation of treatment options makes it impractical to conduct direct trials for every possible comparison [39]. Indirect treatment comparisons have emerged as a crucial methodological approach that allows researchers to compare interventions that have never been directly evaluated in RCTs by leveraging evidence from a network of trials connected through common comparators [6]. This approach is formally extended in network meta-analysis (NMA), which simultaneously synthesizes and compares multiple interventions using both direct and indirect evidence [39].

The validity of conclusions derived from indirect comparisons rests on two fundamental methodological assumptions: transitivity and consistency. Transitivity concerns the legitimacy of combining different sources of evidence, while consistency addresses the agreement between different types of evidence within a network [40] [41]. Understanding, evaluating, and safeguarding these assumptions is paramount for researchers, drug development professionals, and decision-makers who rely on indirect comparisons to inform clinical guidelines and health policy. This guide provides a comprehensive framework for identifying and mitigating threats to these core assumptions, supported by experimental data and analytical protocols.

Conceptual Foundations: Transitivity and Consistency

The Transitivity Assumption

Transitivity is the conceptual foundation that justifies the validity of making indirect comparisons. It posits that the relative effect of two interventions (e.g., A vs. B) can be validly estimated through a common comparator (C) if the trials contributing to the A vs. C and B vs. C comparisons are sufficiently similar in all characteristics that could modify the treatment effect (effect modifiers) [40] [41]. In essence, the assumption is that the participants in the A vs. C trials could have been randomized to B, and those in the B vs. C trials could have been randomized to A.

The transitivity assumption can be understood through several interchangeable interpretations, which are systematically outlined in Table 1 below.

Table 1: Interchangeable Interpretations of the Transitivity Assumption

Interpretation	Description	Methodological Implication
Distribution of Effect Modifiers	Effect modifiers are similarly distributed across the comparisons in the network [40].	Requires careful examination of clinical and methodological trial characteristics.
Similarity of Interventions	The interventions are comparable across the different trials in the network [40].	Ensures that "Drug A" is conceptually the same in all trials where it appears.
Missing-at-Random Interventions	The set of interventions investigated in each trial is independent of the underlying treatment effects [40].	Suggests that the reason a trial did not include a particular intervention is not related to that intervention's efficacy.
Exchangeability of Effects	Observed and unobserved underlying treatment effects are exchangeable [40].	Supports the mathematical combination of different comparisons.
Joint Randomizability	Participants in the network could, in principle, have been randomized to any of the interventions [40].	This is the most stringent conceptual test of transitivity.

The Consistency Assumption

Consistency is the statistical manifestation of transitivity. It refers to the agreement between direct evidence (from head-to-head trials of A vs. B) and indirect evidence (the estimate of A vs. B obtained via the common comparator C) [39] [41]. While transitivity is a conceptual assumption about the design and patients, consistency is an empirically testable statistical property of the data. If the network is consistent, the direct and indirect estimates for the same comparison are in agreement. The presence of inconsistency (or disagreement) indicates a potential violation of the transitivity assumption or other methodological biases [39].

The logical relationship between a network of trials, transitivity, and the derivation of direct, indirect, and mixed (network meta-analysis) estimates is illustrated below.

Methodological Protocols for Evaluation

Protocol for Evaluating Transitivity

Evaluating transitivity is a qualitative, conceptual process that must be planned a priori in the systematic review protocol [40]. The following workflow outlines a structured approach for this assessment.

Step 1: Pre-specify Potential Effect Modifiers Before data extraction, researchers must pre-specify a set of patient and trial characteristics that are known or suspected to modify the treatment response. These often include:

• Patient characteristics: Disease severity, age, gender, comorbidities, prior treatment history.
• Trial design characteristics: Setting (primary vs. tertiary care), intervention dose/duration, follow-up period, outcome definition and measurement, risk of bias (e.g., blinding, allocation concealment) [4] [41].

Step 2: Systematic Data Collection Extract data on all pre-specified characteristics for every trial included in the evidence network. This data collection should be as comprehensive as possible.

Step 3: Assess Comparability Across Comparisons This is the core of the evaluation. The distribution of effect modifiers should be compared across the different pairwise comparisons in the network (e.g., are patients in the A vs. C trials similar to those in the B vs. C trials?). This can be done using summary tables or statistical tests for baseline characteristics. For example, one would check if the mean disease severity in A vs. C trials is comparable to that in B vs. C trials.

Step 4: Judge the Plausibility of Transitivity Based on the comparability assessment, researchers must make a judgment on whether the transitivity assumption is plausible. If critical effect modifiers are imbalanced across comparisons, the validity of the indirect comparison or NMA is threatened [40].

Protocol for Evaluating Consistency

Consistency is evaluated statistically after data synthesis. The following table summarizes the key methods, their application, and interpretation.

Table 2: Experimental Protocols for Evaluating Consistency in a Network Meta-Analysis

Method Name	Description & Workflow	Data Requirements	Interpretation of Results
Design-by-Treatment Interaction Model	A global model that assesses inconsistency across the entire network simultaneously [39].	Network with at least one closed loop.	A significant p-value (e.g., < 0.05) indicates overall inconsistency in the network.
Node-Splitting	A local method that separates direct and indirect evidence for a specific comparison and tests for a statistically significant difference between them [41].	A closed loop with both direct and indirect evidence for at least one comparison.	A significant p-value for a specific node-split indicates local inconsistency for that particular comparison.
Side-by-Side Comparison	Visually or statistically comparing direct and indirect estimates for the same comparison without formally synthesizing them [4].	Direct and indirect evidence for the same comparison.	Overlapping confidence intervals suggest agreement; non-overlapping intervals suggest potential inconsistency.

Empirical Data on Current Reporting and Evaluation Practices

The reporting and evaluation of transitivity and consistency in published systematic reviews has been empirically investigated. A large-scale systematic survey of 721 network meta-analyses published between 2011 and 2021 revealed critical insights and trends, summarized in Table 3 below [40].

Table 3: Reporting and Evaluation of Transitivity in 721 Network Meta-Analyses (2011-2021)

Evaluation Aspect	Systematic Reviews Before PRISMA-NMA (2011-2015) (n=361)	Systematic Reviews After PRISMA-NMA (2016-2021) (n=360)	Change (Odds Ratio, 95% CI)
Provided a Protocol	Not Reported	Not Reported	OR: 3.94 (2.79-5.64)
Pre-planned Transitivity Evaluation	Not Reported	Not Reported	OR: 3.01 (1.54-6.23)
Defined Transitivity	Not Reported	Not Reported	OR: 0.57 (0.42-0.79)
Evaluated Transitivity Conceptually	12%	11%	Not Significant
Evaluated Transitivity Statistically	40%	54%	Not Reported
Used Consistency Evaluation	34%	47%	Not Reported
Inferred Plausibility of Transitivity	22%	18%	Not Significant

Key Findings from Empirical Data:

• Improved Planning: There has been a significant increase in the provision of protocols and pre-planning for transitivity evaluation since the publication of the PRISMA-NMA reporting guidelines, with odds ratios of 3.94 and 3.01, respectively [40].
• Persistent Conceptual Gaps: Despite improved planning, the conceptual evaluation of transitivity remains severely neglected. Only about 1 in 10 reviews attempted to assess the distribution of effect modifiers across trials, which is the cornerstone of the transitivity assumption [40].
• Over-reliance on Statistical Tests: Reviews increasingly used statistical methods to evaluate consistency (a rise from 34% to 47%), but these methods are often applied without a proper conceptual foundation. Statistical tests for consistency may be underpowered and cannot identify the cause of the problem [40].
• Justification of Conclusions: When justifying their conclusions about transitivity, reviews most frequently considered the comparability of trials (24% before vs. 30% after PRISMA-NMA) and results from consistency evaluations (23% before vs. 16% after) [40].

The Researcher's Toolkit: Essential Reagents and Methodological Solutions

Successfully navigating the assumptions of transitivity and consistency requires a toolkit of both conceptual approaches and statistical methods. The following table details key solutions and their functions.

Table 4: Essential Toolkit for Mitigating Threats to Validity in Indirect Comparisons

Tool / Solution	Function / Purpose	Application Context
PRISMA-NMA Checklist	A reporting guideline that ensures transparent and complete reporting of systematic reviews incorporating NMA [40].	Should be followed for any published NMA to improve reproducibility and critical appraisal.
Cochrane Risk of Bias Tool	Assesses the internal validity of individual RCTs, which is a key component of the similarity assessment [41].	Apply to every included study. Differences in risk of bias across comparisons can threaten transitivity.
Network Meta-Regression	A statistical technique to adjust for aggregate-level study characteristics (effect modifiers) that may be causing heterogeneity or inconsistency [40].	Used when an effect modifier is imbalanced across comparisons and sufficient trials are available.
Node-Splitting Analysis	A specific statistical method for detecting local inconsistency between direct and indirect evidence for a particular comparison [41].	Apply to every closed loop in the network where both direct and indirect evidence exists.
Subgroup & Sensitivity Analysis	To explore the impact of a specific trial characteristic (e.g., high vs. low risk of bias) or set of trials on the overall results [4].	Used to test the robustness of the NMA results and identify sources of heterogeneity.
Repunapanor	Repunapanor, CAS:1870822-78-8, MF:C54H66Cl4N12O6, MW:1121.0 g/mol	Chemical Reagent
Acth (1-17) tfa	Acth (1-17) tfa, MF:C97H146F3N29O25S, MW:2207.4 g/mol	Chemical Reagent

The validity of indirect comparisons and network meta-analysis hinges on the often-overlooked conceptual assumption of transitivity and its statistical counterpart, consistency. Empirical data shows that while the research community has made strides in planning and statistically testing these assumptions, there remains a critical gap in the foundational, conceptual work of evaluating the distribution of effect modifiers across a network of trials [40].

Researchers and consumers of this evidence must prioritize the qualitative assessment of transitivity, which involves the meticulous pre-specification of effect modifiers and systematic evaluation of clinical and methodological similarity across trials. Statistical tests for consistency should be viewed as a complementary safety check, not a substitute for conceptual reasoning. By adhering to rigorous protocols, such as those outlined in this guide, and leveraging the provided toolkit, drug development professionals and scientists can generate more trustworthy evidence from indirect comparisons, ultimately leading to better-informed healthcare decisions.

Addressing Cross-Trial Heterogeneity in Patient Populations and Study Designs

In health technology assessment (HTA), randomized controlled trials (RCTs) represent the gold standard for evaluating the comparative efficacy of medical interventions [2]. However, ethical considerations, practical constraints, and the rapid development of new treatments often make direct head-to-head comparisons unfeasible or impossible [2]. This evidence gap has led to the development of indirect treatment comparison (ITC) methodologies, which enable the estimation of relative treatment effects when no direct trial evidence exists [2].

A fundamental challenge in conducting valid ITCs is addressing cross-trial heterogeneity, which encompasses systematic differences in patient characteristics, study designs, outcome definitions, and clinical practice across separate trials [2] [7]. Failure to adequately account for these differences can introduce comparator bias and lead to misleading conclusions about relative treatment efficacy [42]. This guide objectively compares established ITC methodologies, their approaches to addressing heterogeneity, and their applicability in different evidence scenarios, with a specific focus on validating comparisons through common comparator research.

Methodologies for Indirect Comparison

Multiple statistical methodologies have been developed to address cross-trial heterogeneity in indirect comparisons. The appropriate technique selection depends on several factors, including the connectedness of the evidence network, availability of individual patient data (IPD), the degree of observed heterogeneity, and the number of relevant studies [2].

Table 1: Comparison of Indirect Treatment Comparison Methodologies

Methodology	Data Requirements	Key Approach to Address Heterogeneity	Primary Applications	Reported Use in Literature
Network Meta-Analysis (NMA)	Aggregate Data (AD) from multiple trials	Statistical modeling of a connected treatment network; evaluates inconsistency [2]	Multiple treatment comparisons in a connected network [2]	79.5% of included articles [2]
Matching-Adjusted Indirect Comparison (MAIC)	IPD for one treatment; AD for comparator	Weighting IPD to match aggregate baseline characteristics of comparator trial [2] [7]	Single-arm trials or when IPD is available for only one treatment [2]	30.1% of included articles [2]
Simulated Treatment Comparison (STC)	IPD for one treatment; AD for comparator	Regression-based adjustment using effect modifiers identified from IPD [2]	Similar to MAIC; incorporates outcome modeling [2]	21.9% of included articles [2]
Bucher Method	AD from two trials with a common comparator	Simple adjusted indirect comparison via a common comparator [2]	Basic connected network with minimal heterogeneity [2]	23.3% of included articles [2]
Network Meta-Regression	AD from multiple trials	Incorporates trial-level covariates into NMA model to explain heterogeneity [2]	When heterogeneity is expected to modify treatment effects [2]	24.7% of included articles [2]

Experimental Protocols and Methodological Workflows

Protocol for Matching-Adjusted Indirect Comparison (MAIC)

MAIC is a population-adjusted indirect comparison method that requires IPD for at least one treatment in the comparison. The experimental protocol involves a structured workflow to balance patient populations across trials.

MAIC Experimental Procedure:

• Data Acquisition and Preparation: Obtain IPD for the index treatment (Treatment A) and published aggregate data for the comparator treatment (Treatment B) from their respective clinical trials. Ensure outcome variables are harmonized across datasets [7].
• Covariate Selection: Identify effect modifiers (variables that influence treatment effect) and prognostic factors (variables influencing outcome regardless of treatment) through clinical expert input and systematic literature review. Common covariates include age, disease severity, prior treatments, and comorbidities [7].
• Weight Calculation: Using the method of moments or maximum likelihood estimation, calculate weights for each patient in the IPD cohort such that the weighted baseline characteristics match the aggregate means of the comparator trial. The propensity score model is: ( logit(ei) = \alpha0 + \alpha'Xi ), where ( ei ) is the propensity to be in the IPD trial, and ( Xi ) are the baseline characteristics. Weights are defined as ( wi = 1 / (1 - e_i) ) [7].
• Balance Assessment: Evaluate the success of the weighting procedure by comparing the weighted mean values of covariates in the IPD dataset to the published aggregate values from the comparator trial. Standardized mean differences <0.1 indicate adequate balance.
• Outcome Comparison: Analyze the weighted IPD population and compare outcomes to the aggregate comparator using appropriate statistical models (e.g., weighted regression for continuous outcomes, weighted logistic regression for binary outcomes).
• Assumption Validation: A critical assumption of MAIC is that there are no unobserved cross-trial differences that could confound the comparison. This cannot be tested statistically and requires clinical justification [7].

Protocol for Network Meta-Analysis

NMA extends conventional meta-analysis to simultaneously compare multiple treatments through a connected network of trials. The protocol focuses on evaluating and accounting for heterogeneity and inconsistency.

NMA Experimental Procedure:

• Systematic Literature Review: Conduct a comprehensive search of multiple databases (e.g., PubMed, Embase) to identify all relevant RCTs for the treatments of interest, following PRISMA guidelines [2] [43].
• Network Geometry Definition: Map the evidence structure to ensure treatments are connected through direct and indirect evidence. Disconnected networks cannot yield valid indirect comparisons.
• Model Selection: Choose between fixed-effects models (assume no heterogeneity) and random-effects models (account for between-study heterogeneity). The choice depends on clinical and methodological similarity between trials [2].
• Heterogeneity Assessment: Quantify statistical heterogeneity using I² statistics and between-study variance (τ²). I² values of 25%, 50%, and 75% indicate low, moderate, and high heterogeneity, respectively.
• Inconsistency Evaluation: Check for disagreement between direct and indirect evidence using node-splitting or design-by-treatment interaction models. Significant inconsistency suggests violations of the underlying NMA assumptions [2].
• Treatment Effect Estimation: Compute relative treatment effects with 95% confidence or credible intervals. Bayesian approaches typically use Markov Chain Monte Carlo (MCMC) methods with non-informative priors.
• Uncertainty Presentation: Present results using rank probabilities, surface under the cumulative ranking curve (SUCRA) values, and forest plots to communicate the precision and uncertainty of estimates.

The Researcher's Toolkit: Essential Materials and Reagents

Table 2: Key Research Reagent Solutions for Indirect Comparisons

Tool/Resource	Function	Application Example
Individual Patient Data (IPD)	Enables patient-level adjustment for cross-trial differences via MAIC or STC [2] [7]	Re-weighting patients in MAIC to match comparator trial baseline characteristics [7]
Systematic Review Protocols	Provides structured framework for evidence identification and synthesis (e.g., PRISMA, Cochrane) [43]	Minimizing selection bias in trial identification for NMA [2]
Statistical Software Packages	Implements complex statistical models for population adjustment and evidence synthesis	R (gemtc, pcnetmeta), SAS, WinBUGS/OpenBUGS for Bayesian NMA
Risk of Bias Assessment Tools	Evaluates methodological quality of included studies (e.g., Cochrane RoB, ROBINS-I) [43]	Informing sensitivity analyses and interpreting NMA results [43]
Contrast Checker Tools	Ensures data visualizations meet WCAG 2.1 accessibility standards (≥4.5:1 ratio) [44]	Creating accessible figures for publications and HTA submissions
BRD9 Degrader-4	BRD9 Degrader-4, MF:C30H40N4O4, MW:520.7 g/mol	Chemical Reagent
Bulnesol	Bulnesol, CAS:73003-40-4, MF:C15H26O, MW:222.37 g/mol	Chemical Reagent

Decision Framework for Methodology Selection

The choice of an appropriate ITC methodology depends on several evidence and data-related factors. The following decision pathway provides guidance for researchers in selecting the most suitable approach.

Key Decision Considerations:

• MAIC/STC Application: Most appropriate when IPD is available for only one treatment, particularly for single-arm trials in oncology or rare diseases [2]. MAIC has been reported in 30.1% of methodological articles, with increasing adoption in recent years [2].
• NMA Application: The most frequently described technique (79.5% of articles) when multiple studies form a connected treatment network [2]. Preferred when aiming to compare multiple treatments simultaneously and evaluate the consistency of direct and indirect evidence.
• Bucher Method Application: Suitable for simple comparisons involving two trials with a common comparator, representing a simplified form of NMA [2]. Reported in 23.3% of methodological articles [2].
• Network Meta-Regression: Valuable when observed heterogeneity is present that may modify treatment effects, allowing incorporation of trial-level covariates into the model [2].

Validation and Avoiding Comparator Bias

Valid indirect comparisons require careful attention to comparator bias, which occurs when inappropriate comparators are selected, leading to unfair tests of treatments [42]. This bias can manifest through two primary mechanisms:

• Use of Inactive Comparators: When treatments known to be beneficial are withheld from patients in control groups, as occurred in rheumatoid arthritis trials where placebos were used despite effective treatments being available [42].
• Use of Inappropriate Active Comparators: When comparators are selected based on known inferiority rather than clinical relevance, such as using atenolol in antihypertensive trials despite evidence of its inferiority to thiazide diuretics [42].

To minimize comparator bias and enhance the validity of ITCs, researchers should:

• Conduct systematic reviews of existing evidence to inform comparator selection [42]
• Ensure the choice of comparators reflects genuine uncertainty (equipoise) in the clinical community [42]
• Validate that outcome measures are harmonized across compared trials
• Acknowledge that unobserved cross-trial differences remain a fundamental limitation of any ITC, particularly with population-adjusted methods like MAIC [7]

Addressing cross-trial heterogeneity is fundamental to producing valid indirect treatment comparisons. The expanding methodology toolkit—including NMA, MAIC, STC, and meta-regression—provides researchers with sophisticated approaches to adjust for observed differences in patient populations and study designs. Methodology selection should be guided by the available data, network structure, and specific heterogeneity concerns. While these methods continue to evolve, all ITCs share the fundamental limitation of potentially being confounded by unobserved cross-trial differences. Transparent reporting, rigorous methodology, and validation through sensitivity analyses remain essential for generating reliable evidence to inform healthcare decision-making.

Quantitative Bias Analysis (QBA) for Unmeasured Confounding and Missing Data

In the evolving landscape of clinical research, indirect treatment comparisons (ITCs) and studies using real-world data (RWD) have become indispensable when randomized controlled trials are infeasible or unethical, particularly in rare diseases and oncology [45] [46]. These approaches, however, are inherently vulnerable to systematic errors, with unmeasured confounding and missing data representing two of the most significant threats to validity [47] [48]. Quantitative Bias Analysis (QBA) comprises a collection of statistical methods that quantitatively assess the potential impact of these systematic errors on study results, moving beyond qualitative acknowledgment to formal quantification of uncertainty [49] [50].

Regulatory and health technology assessment (HTA) agencies now recognize QBA as a valuable tool for strengthening evidence derived from non-randomized studies. The National Institute for Health and Care Excellence (NICE) recommends QBA when concerns about residual bias impact the ability to make recommendations, while the U.S. Food and Drug Administration (FDA) encourages sponsors to develop a priori plans for assessing confounding and biases [50]. Similarly, Canada's Drug and Health Technology Agency (CADTH) highlights that QBA reduces undue confidence in results by providing ranges of potential bias impacts [50].

This guide provides a comparative examination of QBA methodologies for addressing unmeasured confounding and missing data, focusing on their application in indirect comparisons and analyses incorporating external control arms. We present structured comparisons of methods, detailed experimental protocols, and practical implementation resources to support researchers in assessing the robustness of their findings.

QBA for Unmeasured Confounding

Methodological Approaches and Comparisons

Unmeasured confounding occurs when variables influencing both treatment assignment and outcomes are not accounted for in the analysis, potentially distorting the observed treatment effect [48]. QBA methods for unmeasured confounding enable researchers to quantify how robust their conclusions are to potential unmeasured confounders through various analytical approaches.

Table 1: QBA Methods for Unmeasured Confounding

Method Category	Key Examples	Required Bias Parameters	Output Type	Applicable Study Designs
Bias-Formula Methods	E-value [45] [49]	Minimum strength of association for confounder to explain away effect	Threshold of robustness	Cohort, case-control, cross-sectional
	Indirect Adjustment [51]	Effect of unmeasured confounder on outcome and exposure	Bias-adjusted effect estimate	Cohort studies with time-to-event outcomes
Simulation-Based Methods	Bayesian Data Augmentation [46]	Prior distributions for confounder prevalence and associations	Distribution of adjusted effect estimates	Individual-level indirect treatment comparisons
	Monte Carlo Bias Analysis [49] [52]	Probability distributions for bias parameters	Frequency distribution of bias-adjusted estimates	Various observational designs

The E-value approach, one of the most accessible QBA methods, quantifies the minimum strength of association that an unmeasured confounder would need to have with both the exposure and outcome to explain away an observed effect [45] [50]. While computationally straightforward, its interpretation requires careful contextualization, as the same E-value may indicate different levels of robustness depending on the strength of the observed association and plausible confounder relationships in the specific research domain [50].

Simulation-based methods offer greater flexibility by allowing researchers to construct specific confounding scenarios of interest. These approaches treat unmeasured confounding as a missing data problem, using multiple imputation with user-specified confounder characteristics to generate bias-adjusted effect estimates [46]. Recent advancements have extended these methods to handle non-proportional hazards in time-to-event analyses, a common challenge in oncology research [46].

Experimental Protocol: Simulation-Based QBA for Proportional Hazards Violation

For researchers implementing simulation-based QBA in the presence of non-proportional hazards, the following protocol adapted from recent methodological research provides a robust framework [46]:

Step 1: Model Specification

• Define the outcome model relating the time-to-event outcome to treatment, measured covariates, and the unmeasured confounder
• Specify the treatment propensity model relating treatment assignment to measured and unmeasured covariates
• Select appropriate distributions for time-to-event outcomes that reflect expected hazard patterns

Step 2: Multiple Imputation of Unmeasured Confounders

• Implement Bayesian data augmentation to impute values for the unmeasured confounder
• Specify prior distributions for the confounder's prevalence and its associations with treatment and outcome
• Generate multiple complete datasets (typically 20-50) incorporating the imputed confounder values

Step 3: Bias-Adjusted Analysis

• For each completed dataset, perform a weighted analysis adjusting for both measured and imputed unmeasured confounders
• Estimate the difference in restricted mean survival time (dRMST) between treatment groups
• Pool estimates across multiply imputed datasets using Rubin's rules

Step 4: Tipping Point Analysis

• Iterate steps 2-3 across a range of plausible confounder characteristics
• Identify the confounder-outcome and confounder-exposure associations that would nullify the study conclusions
• Plot the results to visualize the relationship between confounder strength and adjusted treatment effect

This protocol enables researchers to quantify the sensitivity of dRMST estimates to unmeasured confounding while accommodating violations of the proportional hazards assumption, a common limitation in immunotherapy studies and other scenarios where treatment mechanisms differ substantially [46].

Diagram 1: Simulation-based QBA workflow for unmeasured confounding. This approach uses multiple imputation and tipping point analysis to quantify robustness of findings [46].

QBA for Missing Data

Comparative Performance of Missing Data Methods

Missing data presents a ubiquitous challenge in real-world evidence studies, particularly when using electronic health records (EHR) where data collection is incidental to clinical care rather than designed for research purposes [47]. The mechanism of missingness—classified as Missing Completely at Random (MCAR), Missing at Random (MAR), or Missing Not at Random (MNAR)—determines the appropriate analytical approach and potential for bias [50].

Table 2: Performance Comparison of Missing Data Methods in CER

Method	Mechanism Addressed	Bias Reduction	Power Preservation	Key Limitations
Complete Case Analysis	MCAR	Low (if not MCAR)	Low (reduces sample size)	Highly biased if missing not MCAR
Multiple Imputation	MAR	Moderate	Moderate	Limited when missingness depends on unobserved factors
Spline Smoothing	MAR/MNAR with temporal patterns	High	High	Requires longitudinal data with temporal patterns
Tipping Point Analysis	All mechanisms	N/A (assesses robustness)	N/A (assesses robustness)	Does not adjust estimates, assesses sensitivity

Empirical evaluations have demonstrated that when missing data depends on the stochastic progression of disease and medical practice patterns—common in EHR data—spline smoothing methods that leverage temporal information generally outperform multiple imputation approaches, producing smaller estimation bias and less power loss [47]. Spline smoothing utilizes observed values of the same variable at multiple time points to interpolate missing values, effectively capturing disease trajectory information that is often ignored by cross-sectional imputation methods [47].

In scenarios where missingness does not depend on disease progression, multiple imputation remains a valuable approach, reducing bias and power loss by leveraging correlations among observed variables [47]. However, even after imputation, missing data can still lead to biased treatment effect estimates and false negative findings in comparative effectiveness research, highlighting the importance of QBA to assess potential residual bias [47].

Experimental Protocol: Tipping Point Analysis for Missing Data

Tipping point analysis provides a structured framework for assessing how different assumptions about missing data mechanisms could influence study conclusions. The following protocol can be implemented when missing data concerns exist:

Step 1: Characterize Missing Patterns

• Document the proportion of missing values for each variable included in the analysis
• Explore patterns of missingness across patient subgroups and time points
• Formulate plausible scenarios for the missing data mechanism based on clinical knowledge

Step 2: Implement Primary Analysis

• Conduct the primary analysis using the preferred imputation method (e.g., spline smoothing for longitudinal data)
• Record the point estimate and confidence intervals for the treatment effect

Step 3: Specify Bias Parameters

• For missing outcome data, specify the difference in outcomes between missing and observed cases
• For missing covariate data, specify the difference in treatment effect between complete and incomplete cases
• Define a plausible range for each bias parameter based on clinical expertise or literature review

Step 4: Conduct Tipping Point Analysis

• Systematically vary the bias parameters across their plausible ranges
• At each parameter combination, recalculate the treatment effect estimate
• Identify the threshold values at which the conclusion would change (e.g., from significant to non-significant)

Step 5: Interpret Results

• If implausibly extreme assumptions are required to alter conclusions, results are considered robust
• If plausible assumptions nullify findings, acknowledge the fragility of conclusions
• Report the range of treatment effects under different missing data assumptions [45] [50]

This approach does not eliminate bias from missing data but provides a systematic framework for assessing how susceptible study conclusions are to different assumptions about missingness.

Diagram 2: QBA workflow for missing data using tipping point analysis. This approach systematically tests how different missingness assumptions affect study conclusions [45] [50].

Successful implementation of QBA requires both methodological understanding and practical tools. Recent reviews have identified numerous software options for implementing QBA, though accessibility and documentation vary substantially [52].

Table 3: Essential Resources for QBA Implementation

Resource Category	Specific Tools/Approaches	Key Functionality	Implementation Considerations
Statistical Software	R packages (multiple) [52]	Regression adjustment, misclassification analysis, probabilistic bias analysis	Requires programming expertise; flexible for complex scenarios
	Stata commands [52]	Misclassification adjustment, selection bias analysis	More accessible for Stata users; some menu-driven options
Online Web Tools	Interactive web applications [52]	E-value calculation, simple sensitivity analyses	Most accessible for non-programmers; limited to simpler analyses
Educational Resources	FDA/CERSI Decision Tree [53]	Method selection guidance based on study characteristics	Helps researchers identify appropriate QBA methods for their context
Validation Approaches	Negative control outcomes [49]	Empirical evaluation of confounding structure	Provides empirical data to inform bias parameter selection

When selecting software for QBA, researchers should consider their specific analysis needs, programming expertise, and the complexity of their bias scenarios. For straightforward sensitivity analyses, such as E-value calculations, online web tools may suffice. For more complex analyses involving multiple unmeasured confounders or probabilistic bias analysis, dedicated statistical packages in R or Stata offer greater flexibility [52].

A critical challenge in QBA implementation is specifying plausible values for bias parameters. Researchers should prioritize using external validation data, published literature, or expert elicitation to inform these parameters rather than relying solely on arbitrary assumptions [49]. The increasing involvement of regulatory agencies in QBA methodology development, exemplified by the FDA collaboration on a QBA decision tree, highlights the growing importance of these methods in regulatory science [53].

Quantitative Bias Analysis represents a paradigm shift in how researchers address systematic errors in indirect comparisons and real-world evidence studies. By moving from qualitative acknowledgments of limitation to quantitative assessment of potential bias, QBA provides a more transparent and rigorous framework for interpreting study findings.

For unmeasured confounding, methods range from straightforward E-value calculations to complex simulation-based approaches that can accommodate violations of common statistical assumptions. For missing data, tipping point analyses and specialized imputation methods that leverage temporal information offer robust approaches for assessing and addressing potential biases.

As regulatory and HTA agencies increasingly recognize the value of these methodologies, researchers should incorporate QBA as a routine component of study design and analysis when working with non-randomized data. The continuing development of software tools and implementation resources will likely make these methods more accessible to a broader range of researchers, ultimately strengthening the evidence derived from indirect comparisons and real-world data.

Overcoming Convergence and Transparency Issues in Complex ITC Models

Indirect Treatment Comparisons (ITCs) are statistical methodologies used to compare the efficacy and safety of multiple interventions when direct, head-to-head randomized controlled trial (RCT) data are unavailable or non-existent. Health Technology Assessment (HTA) bodies worldwide rely on ITCs to inform reimbursement decisions for new health technologies, especially in therapeutic areas like oncology and rare diseases where direct comparisons are often logistically or ethically challenging [3] [2]. The validity of any ITC hinges critically on the fundamental assumption of constancy of relative treatment effects across studies, often termed homogeneity, similarity, or consistency, depending on the specific ITC method employed [3]. When this assumption is violated, ITC models face significant convergence challenges and produce results with substantial uncertainty and potential bias, undermining their transparency and reliability for decision-making.

The core premise of assessing validity through common comparators research rests on forming a connected network of evidence where all treatments are linked, directly or indirectly, through one or more common comparator interventions (e.g., placebo or a standard of care) [3] [2]. This network allows for the estimation of relative treatment effects between interventions that have never been compared in the same trial. However, as therapeutic landscapes evolve and treatment pathways become more complex, ITC models must incorporate increasingly sophisticated methods to adjust for cross-trial differences in patient populations, study designs, and outcome definitions. This article compares the performance of established and emerging ITC methodologies, providing researchers and drug development professionals with a guide to navigating convergence and transparency issues in this critical field.

Comparative Analysis of ITC Methodologies

Researchers have developed numerous ITC methods with various and inconsistent terminologies, which can be categorized based on their underlying assumptions and the number of comparisons involved [3]. The four primary classes are the Bucher method, Network Meta-Analysis (NMA), Population-Adjusted Indirect Comparisons (PAIC), and Naïve ITC (which is generally avoided due to high bias potential) [3]. The appropriate selection of an ITC technique is a critical strategic decision that should be based on the connectedness of the evidence network, the degree of heterogeneity between studies, the total number of relevant studies, and the availability of Individual Patient Data (IPD) [2].

Table 1: Key Indirect Treatment Comparison Methods and Characteristics

ITC Method	Fundamental Assumptions	Data Requirements	Key Applications	Reported Use in Literature
Network Meta-Analysis (NMA)	Constancy of relative effects (homogeneity, similarity, consistency) [3]	Aggregate Data (AD) from multiple studies [2]	Simultaneous comparison of multiple interventions; treatment ranking [3]	79.5% of included articles [2]
Bucher Method	Constancy of relative effects (homogeneity, similarity) [3]	AD from at least two studies with a common comparator [3]	Pairwise indirect comparisons via a common comparator [3]	23.3% of included articles [2]
Matching-Adjusted Indirect Comparison (MAIC)	Constancy of relative or absolute effects [3]	IPD for one trial and AD for the comparator trial(s) [3] [2]	Pairwise comparisons with considerable population heterogeneity; single-arm studies [3]	30.1% of included articles [2]
Simulated Treatment Comparison (STC)	Constancy of relative or absolute effects [3]	IPD for one trial and AD for the comparator trial(s) [2]	Pairwise ITC; applications similar to MAIC [2]	21.9% of included articles [2]
Network Meta-Regression (NMR)	Conditional constancy of relative effects with shared effect modifier [3]	AD with study-level covariates [3]	Exploring impact of study-level covariates on treatment effects in connected networks [3]	24.7% of included articles [2]

Quantitative Performance and Convergence Challenges

Convergence in ITC models refers to the stability and reliability of statistical estimates, which can be compromised by sparse networks, heterogeneity, and inconsistency. The performance of different ITC methods varies significantly based on the underlying evidence structure.

Table 2: Comparative Performance and Convergence Properties of ITC Methods

ITC Method	Strengths	Limitations & Convergence Challenges	Transparency & Validity Assessment
Network Meta-Analysis (NMA)	Allows simultaneous comparison of multiple interventions; can incorporate both direct and indirect evidence [3].	Complex with assumptions that are challenging to verify; inconsistency in closed loops can cause model convergence failure [3].	Consistency can be evaluated statistically (e.g., node-splitting); transparent presentation of the network geometry is essential [3].
Bucher Method	Simple pairwise comparisons through a common comparator; computationally straightforward [3].	Limited to comparisons with a common comparator; cannot handle multi-arm trials or complex networks; susceptible to effect measure modification [3] [2].	Transparency is high due to simplicity, but validity is entirely dependent on the homogeneity and similarity assumptions [3].
Matching-Adjusted Indirect Comparison (MAIC)	Adjusts for population imbalance across studies using propensity score weighting [3].	Limited to pairwise ITC; effective sample size can drop drastically after weighting, leading to imprecise estimates (convergence to wide confidence intervals) [3] [2].	Weighted population characteristics should be presented to validate the success of the balancing. Relies on the availability of IPD and the conditional exchangeability assumption [3].
Network Meta-Regression (NMR)	Uses regression to explore the impact of study-level covariates on treatment effects, potentially reducing heterogeneity [3].	Requires a large number of studies to be informative; does not work for multi-arm trials; ecological bias is a key concern [3].	Transparency in covariate selection and modeling is critical. Helps validate the similarity assumption by investigating effect modifiers [3].

A systematic literature review found that among recent articles (published from 2020 onwards), the majority describe population-adjusted methods like MAIC (9/13; 69.2%), indicating a growing focus on addressing cross-study heterogeneity, a major source of convergence problems [2]. Furthermore, the acceptance rate of ITC findings by HTA bodies appears relatively low due to various criticisms of source data, applied methods, and clinical uncertainties [3]. This underscores the critical need for robust methodologies that directly address convergence and transparency to enhance the validity and acceptability of ITC results.

Experimental Protocols for Validating ITC Models

Protocol for Evaluating Consistency in Network Meta-Analysis

The validity of a complex NMA depends on the statistical agreement between direct and indirect evidence for any treatment contrast within the network, a property known as consistency.

Objective: To evaluate the presence of inconsistency in a connected network of interventions and ensure the model converges to a reliable estimate.

Methodology:

• Network Geometry Mapping: Visually map all available direct comparisons to illustrate the connectivity and identify potential weak links or sparse areas in the network. This is a prerequisite for any NMA [3].
• Design-by-Treatment Interaction Model: Employ a global test for inconsistency across the entire network. This method is preferred when the network includes multi-arm trials, as it accounts for the correlations within them [3].
• Node-Splitting Analysis: Conduct a local test for inconsistency by separating direct evidence for a specific comparison from the indirect evidence and statistically testing for a difference. This helps pinpoint the source of inconsistency [3].
• Comparison of Fit: Compare the model fit (e.g., using Deviance Information Criterion (DIC) for Bayesian models) between a consistency model and an inconsistency model that allows for disagreement between direct and indirect evidence.

Data Requirements: Aggregate data (odds ratios, hazard ratios, etc.) and standard errors from all included trials, with clear documentation of multi-arm trials to ensure correct modeling.

Interpretation: A non-significant p-value in inconsistency tests and a lower DIC for the consistency model support the assumption of consistency. Significant inconsistency requires investigation of clinical or methodological heterogeneity driving the disagreement.

Protocol for Population Adjustment via Matching-Adjusted Indirect Comparison

MAIC is a common technique used when comparing treatments from single-arm studies or from RCTs with importantly different patient populations, a frequent scenario in oncology and rare diseases [2].

Objective: To weight the patients from an IPD study such that its baseline characteristics match those of an aggregate comparator study, effectively creating a simulated common comparator population.

Methodology:

• Effect Modifier Identification: Prior to analysis, clinical experts must identify a set of prognostic variables and potential effect modifiers that are believed to influence the treatment outcome. This is a critical step for ensuring the clinical validity of the adjustment [3].
• Propensity Score Estimation: Using the IPD, fit a logistic regression model where the outcome is the study indicator (e.g., 0 for IPD trial, 1 for aggregate trial—though the aggregate data is not used in the model). The covariates are the identified effect modifiers.
• Weight Calculation: Calculate weights for each patient in the IPD trial as the odds of being in the aggregate study based on their propensity score. These weights are used to create a balanced pseudo-population.
• Effective Sample Size (ESS) Calculation: Compute the ESS after weighting. A large drop in ESS indicates that the weighting scheme is extreme and may lead to convergence issues and unstable, imprecise estimates [3].
• Outcome Analysis: Fit a weighted regression model (e.g., using a Generalized Linear Model) to the outcomes in the IPD, using the calculated weights. The estimated treatment effect from this model is the adjusted comparison.

Data Requirements: Individual Patient Data (IPD) for the index intervention trial and published Aggregate Data (AD), including baseline summary statistics, for the comparator trial [3] [2].

Interpretation: The success of balancing is assessed by comparing the weighted baseline characteristics of the IPD study to the aggregate characteristics of the comparator study. The adjusted treatment effect should be interpreted with caution if the ESS is very low, as this indicates a lack of common support and a high risk of model failure.

Visualization of ITC Workflows and Conceptual Frameworks

Analytical Workflow for Complex Indirect Treatment Comparisons

The following diagram outlines a systematic workflow for developing and validating a complex ITC, highlighting key decision points to ensure convergence and transparency.

ITC Analytical Workflow

Conceptual Framework for Validity Assessment via Common Comparators

This diagram illustrates the logical relationships and core concepts underpinning the assessment of ITC validity through the use of common comparators, highlighting potential sources of bias.

Validity Assessment Framework

The Scientist's Toolkit: Essential Reagents for ITC Research

Successful execution and critical appraisal of ITC studies require a suite of methodological tools and conceptual frameworks. The following table details key components of the researcher's toolkit for overcoming convergence and transparency issues.

Table 3: Research Reagent Solutions for Advanced ITC Analysis

Toolkit Item	Function & Application	Key Considerations
PRISMA-NMA Guidelines	Provides a checklist for transparent and complete reporting of systematic reviews incorporating NMA, enhancing reproducibility and clarity [2].	Essential for the initial evidence synthesis phase to minimize selection and reporting biases.
Specialized Software (R, Python, WinBUGS/OpenBUGS)	Statistical platforms capable of performing complex ITCs, including Bayesian NMA and advanced population-adjusted models.	R packages like gemtc and multinma are widely used. Software choice affects modeling flexibility and accessibility.
Individual Patient Data (IPD)	Enables application of population-adjusted methods (MAIC, STC) to balance baseline characteristics and explore effect modifiers at the patient level [3] [2].	Often difficult to obtain. Its use is critical for validating the similarity assumption in anchored comparisons.
NICE-DSU Technical Support Documents	A series of guidance documents providing detailed methodological recommendations on conducting and critiquing ITCs, widely referenced by HTA bodies [2].	Serves as a de facto standard for best practices, particularly for submissions to health technology assessment agencies.
Effect Modifier Inventory	A pre-specified list of clinical and methodological variables hypothesized to modify treatment effects, informed by clinical expertise [3].	Fundamental for designing a valid PAIC or network meta-regression. Failure to include key modifiers leads to residual bias.
Consistency & Inconsistency Models	Statistical models used to test the fundamental assumption of consistency between direct and indirect evidence within a network [3].	Key for model validation. Inconsistency often indicates underlying heterogeneity or biases not accounted for in the model.

Pre-Specification and Sensitivity Analyses to Bolster Result Credibility

Indirect treatment comparisons (ITCs) are statistical methodologies used to compare the efficacy or safety of multiple interventions when head-to-head randomized controlled trials (RCTs) are unavailable, unethical, or impractical to conduct [2] [54]. In evidence-based medicine and health technology assessment (HTA), these methods provide crucial comparative evidence for decision-making regarding new health interventions [2] [3]. The credibility of ITC findings hinges on rigorous pre-specification of the analytical plan and comprehensive sensitivity analyses to explore the impact of methodological assumptions and potential biases [54] [55].

Pre-specification involves detailing the ITC methodology, including inclusion/exclusion criteria, choice of comparators, outcomes, and statistical models, before conducting the analysis [54]. This practice minimizes data-driven decisions and selective reporting, thereby increasing the transparency and reliability of the results. Sensitivity analyses then test the robustness of these pre-specified findings under different assumptions, models, or data selections [55]. Together, these practices help validate the credibility of indirect comparisons, particularly when relying on common comparators to connect treatments across different studies [54].

Foundational Methods and Their Applications

Several statistical techniques enable indirect comparisons, each with specific applications, data requirements, and underlying assumptions. Understanding these methods is fundamental to selecting the appropriate approach and implementing credible pre-specification and sensitivity analyses.

Table 1: Key Indirect Treatment Comparison Methods and Characteristics

Method	Key Assumption	Data Requirements	Primary Application	Key Considerations
Bucher Method [2] [54]	Constancy of relative effects (Homogeneity, Similarity)	Aggregate data (AD) from at least two trials sharing a common comparator	Pairwise indirect comparisons via a common comparator	Limited to simple networks with a single common comparator; unsuitable for complex networks or multi-arm trials [3] [54].
Network Meta-Analysis (NMA) [2] [3]	Constancy of relative effects (Homogeneity, Similarity, Consistency)	AD from a connected network of trials (can include direct and indirect evidence)	Simultaneous comparison of multiple interventions; ranking treatments	The Bayesian framework is often preferred when data are sparse. Consistency between direct and indirect evidence must be assessed [3] [55].
Matching-Adjusted Indirect Comparison (MAIC) [2] [3]	Conditional constancy of effects	Individual Patient Data (IPD) for one trial and AD for the other	Pairwise comparisons when population heterogeneity exists; often used with single-arm trials	Adjusts for imbalances in effect modifiers but is limited to the population of the aggregate data trial.
Simulated Treatment Comparison (STC) [2] [3]	Conditional constancy of effects	IPD for one trial and AD for the other	Pairwise comparisons with population heterogeneity, like MAIC	Uses outcome regression models to predict counterfactuals in the aggregate study population.
Network Meta-Regression (NMR) [2] [3]	Conditional constancy of relative effects with shared effect modifiers	AD (IPD optional) from a connected network	Exploring impact of study-level covariates (effect modifiers) on treatment effects	Cannot adjust for differences in treatment administration or co-treatments.

The workflow for establishing and validating an indirect comparison involves a sequence of critical steps, from defining the research question to interpreting the final validated results.

The Critical Role of Pre-Specification

Pre-specification in an ITC study protocol establishes a defensible analytical framework before data analysis begins, safeguarding against subjective data dredging and post-hoc manipulations that can inflate false-positive findings [54]. The ISPOR Task Force on Indirect Treatment Comparisons emphasizes that a clear, pre-defined plan is a cornerstone of good research practice [55]. Key components of a robust pre-specification include a clearly articulated PICO (Population, Intervention, Comparator, Outcome) framework, a systematic literature review strategy, detailed inclusion/exclusion criteria for studies, and a comprehensive statistical analysis plan [54].

A particularly critical aspect of pre-specification is the justification for the choice of common comparators [54]. The validity of an indirect comparison hinges on the common comparator acting as a reliable "bridge" between treatments. The rationale for selecting one common comparator over others must be clearly documented, as this choice can significantly influence the results [54]. Furthermore, the pre-specified plan should outline how the fundamental ITC assumptions—similarity, homogeneity, and consistency—will be evaluated [54] [55]. Similarity requires that studies are comparable in terms of potential effect modifiers (e.g., patient characteristics, trial design), homogeneity implies that study results within a pairwise comparison are similar, and consistency means that direct and indirect evidence for a treatment effect are in agreement [54].

Designing Sensitivity and Scenario Analyses

Sensitivity analyses are indispensable for probing the robustness of ITC findings and are a mandatory component of HTA submissions [16] [55]. These analyses test whether the primary conclusions change under different plausible scenarios, methodological choices, or data handling assumptions.

Table 2: Taxonomy of Sensitivity Analyses for Indirect Comparisons

Analysis Type	Objective	Typical Approach	Interpretation Focus
Model Selection	Test robustness to choice of statistical model.	Compare Fixed-Effect vs. Random-Effects models; Bayesian vs. Frequentist frameworks [55].	Direction and magnitude of effect estimates; changes in statistical significance.
Inconsistency Exploration	Evaluate the impact of disagreement between direct and indirect evidence.	Use node-splitting or design-by-treatment interaction models to assess inconsistency [55].	Identify loops in the network where inconsistency is present and its impact on treatment rankings.
Influence Analysis	Determine if results are driven by a single study or data point.	Iteratively remove each study from the network and re-run the analysis.	Stability of the overall effect estimate and ranking after excluding influential studies.
Population Heterogeneity	Assess impact of variable patient characteristics across studies.	Perform network meta-regression or subgroup analysis if data permit [3].	Whether treatment effects are modified by specific patient or study-level covariates.
Prior Distributions (Bayesian)	Examine influence of prior choices in Bayesian NMA.	Vary non-informative or informative prior distributions for heterogeneity parameters.	Sensitivity of posterior estimates, particularly in sparse networks.

For cost-comparison analyses where clinical similarity must be established indirectly, a powerful sensitivity approach is the non-inferiority ITC within a Bayesian framework [16]. This involves probabilistically comparing the indirectly estimated treatment effect against a pre-specified non-inferiority margin. The result provides a direct assessment of whether the evidence supports an assumption of clinical equivalence, moving beyond a simple lack of statistical significance.

Experimental Protocols for Validated ITCs

Protocol for a Bucher Indirect Comparison

The Bucher method is a foundational technique for a simple pairwise indirect comparison via a common comparator [54]. The following protocol ensures a standardized and credible analysis.

Objective: To estimate the relative effect of Intervention B vs. Intervention A using a common comparator C. Materials: Aggregate data (e.g., effect estimates and variances) from RCTs comparing A vs. C and B vs. C. Procedure:

• Extract Effect Estimates: For each pairwise comparison (A vs. C and B vs. C), obtain the logged effect measure (e.g., log-odds ratio, log-risk ratio) and its variance. If multiple studies exist for a comparison, perform a meta-analysis to get a pooled estimate and variance [54].
• Calculate Indirect Effect: Compute the indirect effect of B vs. A using the formula: Effect_AB = Effect_AC - Effect_BC [54].
• Calculate Variance: The variance of the indirect estimate is the sum of the variances of the two direct estimates: Variance_AB = Variance_AC + Variance_BC [54].
• Derive Confidence Interval: Construct the 95% confidence interval for the indirect effect on the log scale: Effect_AB ± Z_{0.975} * √(Variance_AB), where Z_{0.975} ≈ 1.96. Convert the point estimate and confidence limits back to the natural scale (e.g., odds ratio) [54]. Validation Steps:

• Similarity Assessment: Prior to analysis, compare the study and patient characteristics of the A vs. C and B vs. C trials to qualitatively assess the similarity assumption [54].
• Sensitivity Analysis: If multiple studies are available for either direct comparison, conduct a sensitivity analysis using different meta-analysis models (fixed vs. random effects) for pooling the direct evidence.

Protocol for a Bayesian Network Meta-Analysis

Bayesian NMA allows for the simultaneous comparison of multiple treatments and is particularly useful when data are sparse [3] [55].

Objective: To synthesize all available direct and indirect evidence to estimate relative effects and rank all interventions in a network. Materials: Aggregate data from all RCTs in a connected network of interventions. Software: Requires specialized software (e.g., JAGS, Stan, or dedicated R packages like gemtc or BUGSnet). Procedure:

• Define Model: Specify a Bayesian hierarchical model. A common starting point is a random-effects model that accounts for heterogeneity between studies.
• Choose Priors: Select prior distributions for model parameters. For heterogeneity, a vague prior such as Uniform(0, 5) or Half-Normal(0, 1) for the log-scale standard deviation is often used for a minimally informative analysis [55].
• Run MCMC Simulation: Use Markov Chain Monte Carlo (MCMC) methods to generate posterior distributions for all treatment effects. Run multiple chains (e.g., 3) to assess convergence.
• Check Convergence: Assess MCMC convergence using diagnostics like the Gelman-Rubin statistic (potential scale reduction factor ˆR ≤ 1.05) and trace plots [55].
• Output Results: Extract posterior medians as point estimates and 95% credible intervals for all pairwise comparisons. Calculate surface under the cumulative ranking curve (SUCRA) values or rank probabilities for each treatment. Validation Steps:

• Convergence Diagnosis: Confirm all parameters have achieved convergence via diagnostics.
• Inconsistency Check: Fit both consistency and inconsistency models (e.g., using node-splitting) and compare them to check for significant disagreement between direct and indirect evidence [55].
• Model Fit: Compare models using measures like the Deviance Information Criterion (DIC). A model with a lower DIC is generally preferred.

The Scientist's Toolkit: Essential Reagents for ITC Research

Table 3: Key Research Reagent Solutions for Indirect Comparisons

Tool / Resource	Category	Primary Function	Application Notes
R (with gemtc, netmeta packages)	Statistical Software	Provides comprehensive, peer-reviewed functions for conducting frequentist and Bayesian NMA.	The gemtc package provides a frontend for JAGS for Bayesian analysis. netmeta is a widely used package for frequentist NMA [55].
JAGS / WinBUGS / OpenBUGS	Statistical Software	Specialized platforms for Bayesian analysis using MCMC sampling.	Often called from within R. Requires careful specification of the model and priors, and thorough convergence checking [55].
PRISMA-NMA Checklist	Reporting Guideline	Ensures transparent and complete reporting of systematic reviews incorporating NMA.	Adherence is considered a hallmark of quality and is recommended by major journals and HTA bodies [54].
CINeMA (Confidence in NMA) Framework	Assessment Tool	Provides a systematic approach for rating the confidence in the results from an NMA.	Assesses domains such as within-study bias, reporting bias, indirectness, imprecision, heterogeneity, and incoherence [3].
IPD (Individual Patient Data)	Data	Gold standard data for population-adjusted ITCs like MAIC and STC, allowing for adjustment of patient-level effect modifiers.	Often difficult to obtain but significantly strengthens the validity of an ITC when patient populations differ [2] [3].

Pre-specification and sensitivity analysis are not merely supplementary components but are foundational to producing credible and defensible evidence from indirect treatment comparisons. As therapeutic landscapes evolve and head-to-head evidence remains scarce, the reliance on ITCs by researchers, clinicians, and HTA bodies will only grow [2] [16]. A disciplined approach, involving a pre-specified protocol transparently justified on clinical and methodological grounds, coupled with rigorous sensitivity analyses that probe the robustness of conclusions, is paramount. This practice directly addresses the inherent uncertainties of indirect evidence, builds confidence in the findings, and ultimately supports better healthcare decision-making.

Ensuring Robust Evidence: Formal Validation Techniques and HTA Appraisal Insights

Formal Methods for Assessing Similarity and Equivalence in ITCs

Indirect Treatment Comparisons (ITCs) have become indispensable methodological tools in health technology assessment (HTA) and drug development, providing critical comparative evidence when head-to-head randomized controlled trials are unavailable, unethical, or impractical [2]. The fundamental challenge in conducting valid ITCs lies in establishing similarity and equivalence between treatments that have never been directly compared in clinical trials. This methodological guide examines the formal approaches for assessing similarity and equivalence within ITC frameworks, with particular focus on their application through common comparators.

Health technology assessment bodies worldwide increasingly rely on cost-comparison (cost-minimization) analyses to manage growing demands for healthcare resource allocation, yet such approaches necessitate robust demonstration of clinical similarity between interventions [16] [56]. While head-to-head comparisons from equivalence or noninferiority studies are typically accepted as evidence of similarity, significant guidance gaps exist regarding when equivalence may be assumed from ITCs, whether quantitative or qualitative in nature [56]. This guide systematically compares the available formal methods, their underlying assumptions, implementation protocols, and applications within modern drug development contexts.

The Methodological Spectrum of Indirect Treatment Comparisons

Fundamental ITC Approaches and Classifications

The landscape of ITC methodologies encompasses multiple techniques with varying and sometimes inconsistent terminologies across the literature [3]. Based on underlying assumptions regarding the constancy of treatment effects and the number of comparisons involved, ITC methods can be categorized into four primary classes:

• Bucher Method (adjusted or standard ITC): Enables pairwise comparisons through a common comparator using frequentist framework with constancy of relative effects assumption [3]
• Network Meta-Analysis (NMA): Allows simultaneous comparison of multiple interventions within either frequentist or Bayesian frameworks, requiring constancy of relative effects including homogeneity, similarity, and/or consistency assumptions [3] [2]
• Population-Adjusted Indirect Comparisons (PAIC): Adjust for population imbalance across studies using individual patient data (IPD) through methods like Matching-Adjusted Indirect Comparison (MAIC) and Simulated Treatment Comparison (STC) [3]
• Naïve ITC (unadjusted ITC): Generally avoided in formal HTA submissions due to susceptibility to bias from confounding factors [8]

Table 1: Fundamental ITC Method Classes and Characteristics

Method Class	Key Assumptions	Analytical Framework	Primary Applications
Bucher Method	Constancy of relative effects (homogeneity, similarity)	Frequentist	Pairwise indirect comparisons through common comparator
Network Meta-Analysis	Constancy of relative effects (homogeneity, similarity, consistency)	Frequentist or Bayesian	Multiple intervention comparisons or ranking
Population-Adjusted Methods (MAIC, STC)	Constancy of relative or absolute effects	Frequentist (often MAIC), Bayesian (often STC)	Studies with population heterogeneity, single-arm studies in rare diseases
Network Meta-Regression	Conditional constancy of relative effects with shared effect modifier	Frequentist or Bayesian	Investigate how covariates affect relative treatment effects

Current Application Gaps in Formal Similarity Assessment

Despite the availability of various formal methods, a significant implementation gap exists between methodological development and real-world application. A comprehensive review of National Institute for Health and Care Excellence (NICE) technology appraisals revealed that none of the 33 appraisals using cost-comparison based on ITC incorporated formal methods to determine similarity [16] [56]. Instead, companies predominantly used narrative summaries reliant on traditional ITC approaches without noninferiority testing to assert similarity, leading to committee uncertainty that was typically resolved through clinical expert input alone [16].

This practice-policy gap highlights the critical need for greater methodological rigor in similarity assessment. The most promising methods identified in the literature review include estimation of noninferiority ITCs in a Bayesian framework followed by straightforward, probabilistic comparison of the indirectly estimated treatment effect against a prespecified noninferiority margin [56]. The continued reliance on significance testing rather than equivalence testing represents a fundamental methodological shortcoming in current practice, as absence of evidence (statistical significance) is not evidence of absence (clinical equivalence).

Formal Methodologies for Similarity Assessment

Noninferiority Indirect Treatment Comparisons

The most methodologically robust approach for establishing similarity in ITCs involves formal noninferiority testing within either frequentist or Bayesian frameworks. This approach requires researchers to prespecify a noninferiority margin (Δ) that represents the maximum clinically acceptable difference between treatments, then evaluate whether the confidence or credible interval for the indirect treatment effect lies entirely within the range -Δ to Δ.

Experimental Protocol Implementation:

• Define the Noninferiority Margin: Establish Δ based on clinical justification, often derived from historical placebo-controlled trials or clinical consensus regarding minimally important differences
• Conduct Standard Indirect Treatment Comparison: Perform ITC using appropriate methodology (Bucher, NMA, or population-adjusted methods) depending on available evidence and network structure
• Estimate Confidence/Credible Intervals: Calculate the range of plausible values for the indirect treatment effect, typically at 95% confidence/credibility
• Compare to Noninferiority Margin: Determine if the entire interval falls within the prespecified equivalence bounds
• Interpret Results: If the interval excludes values beyond the equivalence bounds, conclude similarity; otherwise, reject the similarity hypothesis

The Bayesian framework offers particular advantages for noninferiority ITCs through natural probabilistic interpretation, allowing direct statements about the probability that the treatment effect lies within the equivalence bounds [16].

Population-Adjusted Methods for Similarity Assessment

When population differences exist across studies, population-adjusted methods provide approaches for adjusting for cross-trial imbalances that might otherwise invalidate similarity assumptions. These methods require individual patient data (IPD) for at least one study in the comparison.

Matching-Adjusted Indirect Comparison (MAIC) Protocol:

• Obtain IPD for the Index Treatment: Secure individual patient data for the study involving the intervention of interest
• Identify Effect Modifiers: Determine baseline characteristics that may modify treatment effects, based on clinical knowledge and previous research
• Create Weighted Population: Use propensity score weighting to balance the IPD population to match the aggregate baseline characteristics of the comparator study
• Re-estimate Outcomes: Analyze the weighted population to obtain adjusted outcome estimates for the index treatment
• Compare to Comparator: Conduct indirect comparison using the adjusted estimates against the aggregate comparator outcomes

Simulated Treatment Comparison (STC) Protocol:

• Develop Outcome Model: Using IPD, create a regression model predicting outcomes based on treatment and effect modifiers
• Apply to Comparator Population: Use the model to predict outcomes for the index treatment in the population characteristics of the comparator study
• Compare Outcomes: Conduct indirect comparison between the predicted outcomes and the observed comparator outcomes

Table 2: Comparison of Population-Adjusted ITC Methods for Similarity Assessment

Method Characteristic	Matching-Adjusted Indirect Comparison	Simulated Treatment Comparison
Data Requirements	IPD for index treatment, aggregate for comparator	IPD for index treatment, aggregate for comparator
Analytical Approach	Propensity score weighting	Outcome regression modeling
Key Assumptions	All effect modifiers measured and included	Correct outcome model specification
Strengths	Intuitive balancing of populations	More efficient use of data when model correct
Limitations	May reduce effective sample size, limited to pairwise	Model misspecification risk, limited to pairwise
Similarity Assessment	Comparison after balancing populations	Comparison after outcome model adjustment

Network Meta-Analysis for Equivalence Evaluation

Network meta-analysis extends similarity assessment to multiple treatment comparisons simultaneously, allowing for both direct and indirect evidence to be incorporated in a coherent analytical framework.

Implementation Protocol for NMA-Based Similarity:

• Systematic Literature Review: Identify all relevant randomized trials comparing treatments of interest within the connected network
• Evaluate Transitivity Assumption: Assess whether studies are sufficiently similar in key characteristics (population, interventions, outcomes, methods) to permit valid indirect comparisons
• Check Consistency: Evaluate whether direct and indirect evidence are in agreement using node-splitting or other statistical approaches
• Estimate Relative Effects: Calculate relative treatment effects between all pairs of treatments in the network
• Conduct Equivalence Testing: For treatments hypothesized to be similar, evaluate whether confidence/intervals for relative effects fall within prespecified equivalence bounds

The Bayesian framework for NMA offers particular advantages for equivalence assessment through natural incorporation of uncertainty and direct probability statements regarding similarity [3].

Analytical Framework for Similarity Assessment

Diagram Title: Similarity Assessment Workflow

The Researcher's Toolkit: Essential Methodological Reagents

Table 3: Essential Methodological Reagents for ITC Similarity Assessment

Research Reagent	Function in Similarity Assessment	Key Considerations
Individual Patient Data	Enables population-adjusted methods (MAIC, STC)	Data quality, variable consistency, sample size
Noninferiority Margin (Δ)	Defines clinical equivalence boundary	Clinical justification, historical data, regulatory input
Statistical Software (R, Python, WinBUGS)	Implements complex ITC analyses	Bayesian vs frequentist capabilities, model flexibility
Systematic Review Protocol	Identifies all relevant evidence for network	Comprehensive search, inclusion/exclusion criteria
Effect Modifier List	Identifies variables for population adjustment	Clinical knowledge, previous research, data availability
Consistency Assessment Tools	Evaluates agreement between direct/indirect evidence	Node-splitting, inconsistency models, diagnostic plots

Comparative Performance of ITC Methods

The selection of appropriate ITC method depends on multiple factors including available data, network structure, and specific research question. Systematic reviews of methodological literature indicate that network meta-analysis is the most frequently described technique (79.5% of included articles), followed by matching-adjusted indirect comparison (30.1%), and network meta-regression (24.7%) [2].

Recent analyses of HTA submissions reveal that while formal methods for assessing equivalence in ITC-based cost comparison are emerging, they have not yet been widely applied in practice [16]. Instead, qualitative methods such as evaluation of the plausibility of class effects and clinical expert input remain primary approaches for addressing uncertainties in assuming equivalence [56]. This practice-policy gap represents a significant opportunity for methodological improvement in drug development and health technology assessment.

Population-adjusted methods have gained substantial traction in recent applications, particularly for oncology and rare disease assessments where single-arm trials are common. Among recent methodological articles (published from 2020 onwards), the majority describe population-adjusted methods, with MAIC appearing in 69.2% of these recent publications [2]. This trend reflects growing recognition of the importance of addressing cross-trial heterogeneity when assessing treatment similarity.

Formal methods for assessing similarity and equivalence in indirect treatment comparisons represent a methodologically sophisticated approach to addressing one of the most challenging aspects of comparative effectiveness research. The current methodological arsenal includes several robust approaches, with noninferiority ITCs in Bayesian frameworks and population-adjusted methods showing particular promise for rigorous similarity assessment.

The field continues to evolve rapidly, with ongoing methodological development in areas such as multilevel network meta-regression, more flexible approaches for population adjustment, and standardized frameworks for equivalence margin specification. As health technology assessment bodies worldwide increasingly rely on indirect comparisons for healthcare decision-making, the implementation of these formal similarity assessment methods will be crucial for ensuring valid conclusions regarding treatment equivalence and enabling appropriate resource allocation decisions.

Future directions include development of standardized reporting guidelines for similarity assessment in ITCs, improved statistical methods for evaluating the transitivity assumption, and better integration of quantitative and qualitative approaches for equivalence determination. Through continued methodological refinement and improved implementation in practice, formal similarity assessment in ITCs will play an increasingly important role in the evidence ecosystem for drug development and healthcare policy.

The European Network for Health Technology Assessment (EUnetHTA) actively prepared for the new European Union Joint Clinical Assessment (JCA) process through pilot joint Relative Effectiveness Assessments (REAs) conducted between 2006 and 2021 [57]. These assessments served as crucial testing ground for methodologies that would later form the basis of the EU's centralized HTA process, which began implementation in January 2025 [57] [58]. The EUnetHTA initiative connected over 80 organizations across thirty European countries with the goal of producing sustainable HTA and enabling information exchange to support policy decisions [58].

Within these REAs, Indirect Treatment Comparisons (ITCs) became a fundamental component of the evidentiary base to inform submissions, particularly given the broad scoping of relevant Population, Intervention, Comparators, and Outcomes (PICO) criteria that included multiple comparators reflecting variations in standards of care across EU member states [57]. This article systematically analyzes the common critiques and success factors emerging from the EUnetHTA REA experience, providing crucial insights for researchers, scientists, and drug development professionals preparing evidence for European HTA submissions.

Quantitative Analysis of ITCs in EUnetHTA REAs

A systematic review of EUnetHTA REAs conducted between 2010 and 2021 provides compelling quantitative data on the role and reception of ITCs in the pilot program. The analysis encompassed 23 REAs of pharmaceutical products, offering valuable insights into assessment patterns [57].

Table 1: Prevalence of ITCs in EUnetHTA REAs (2010-2021)

Assessment Category	Number	Percentage	Key Findings
Total REAs Identified	23	100%	Spanning three Joint Action phases
REAs Including ITCs	12	52%	More than half incorporated indirect evidence
Oncology REAs with ITCs	6	50% of ITC REAs	Half of ITC submissions were in oncology
Non-Oncology REAs with ITCs	6	50% of ITC REAs	Equal distribution across therapeutic areas
Comparisons Requiring Indirect Evidence	25 out of 64	39%	Median of 4 comparators per REA (range: 1-18)

The data reveals that ITCs were not merely supplementary analyses but central components in more than half of all REAs, addressing nearly 40% of all required comparisons. The broad PICO scoping necessitated this reliance on indirect evidence, with a median of four comparators per assessment [57].

Table 2: EUnetHTA Assessment of ITC Suitability

Suitability Category	Number of Comparisons	Percentage	EUnetHTA Interpretation
Appropriate	1	4%	Clearly stated ITC was appropriate/adequate
Unsuitable	0	0%	No ITCs clearly deemed inappropriate
Unclear	24	96%	"Interpret with caution," "no firm conclusions," or "results not reliable"

A striking finding emerges from the suitability assessment: despite the frequent submission of ITCs, assessors considered the ITC data and/or methods appropriate in only one instance [57]. The overwhelming majority (96%) of ITC submissions fell into an "unclear" category, where assessors recommended interpretation with caution, offered no firm conclusions, or questioned the reliability of results [57].

Common Methodological Critiques of ITCs in REAs

Categorization of ITC Limitations

The EUnetHTA assessors identified specific limitations in the submitted ITCs, which can be categorized into four primary domains:

Table 3: Categorization of ITC Limitations in EUnetHTA REAs

Limitation Category	Specific Critiques	Impact on Assessment
Data-Related	Heterogeneity between studies, sample size limitations, data scarcity	Challenges in determining if treatment effects are comparable across studies
Methodological	Feasibility of the ITC approach, inappropriate statistical techniques	Raises questions about validity of the entire comparison
Uncertainty	Missing sensitivity analyses, discordance with real-world evidence	Undermines confidence in point estimates for decision-making
Other	Unspecified limitations, incomplete submissions	Suggests inadequate justification or documentation

EUnetHTA Guidance on Statistical Methods

Beyond these categorical limitations, EUnetHTA has established specific methodological expectations for ITCs, particularly regarding statistical approaches to control for confounding. The network's guidance states that naïve comparisons (unadjusted indirect comparisons) should not be performed and that formal statistical comparisons based on individual patient-level data are required [58].

For propensity score modeling, the guidance outlines three critical assumptions that must be met to have confidence in the results:

• Positivity: Patients in both groups theoretically must be eligible for both treatments of interest.
• Overlap: The distribution of propensity scores must be similar between groups.
• Balance: After adjusting for confounding, balance on confounders must be achieved between treatment groups, with acceptable absolute standard difference thresholds between groups ranging from 0.1 to 0.25 depending on context [58].

The guidance further specifies appropriate statistical techniques for confounding control, including multiple regression, instrumental variables, g-computation, and propensity scores [58]. This level of methodological detail, while common in epidemiological and statistical literature, is rare in HTA guidance documents and establishes a high standard for evidence generation [58].

Success Factors for International HTA Projects

Evaluation of the EUnetHTA Joint Action revealed several critical success factors that extended beyond methodological considerations to encompass project management and stakeholder engagement dimensions.

Quantitative Evaluation Metrics

The EUnetHTA JA employed systematic evaluation through annual questionnaires achieving notably high response rates: 86% to 88% from project participants and 65% to 88% from external stakeholders [59] [60]. This high engagement level provided robust data for identifying success factors, which included both quantitative and qualitative dimensions [59].

Table 4: Success Factors for International HTA Collaboration

Success Factor Category	Specific Elements	Implementation in EUnetHTA
Project Delivery	Production of deliverables according to workplan, achievement of objectives	Timely completion of REAs according to established timelines
Value Generation	Added value generated, progress from preceding projects	Building on EUnetHTA 2006-2008 project foundations
Stakeholder Engagement	Effective communication, involvement of external stakeholders	High questionnaire response rates (65-88%) from stakeholders
Management Structure	Workstream management, clear governance	Coordinated efforts across 80+ organizations in 30 countries

Strategic Implications for Evidence Generation

The EUnetHTA experience demonstrates that future assessments of international HTA projects should strive to measure outcomes and impact, not just outputs and process [59]. This principle extends to the evaluation of ITCs, where the focus should be on the reliability and decision-making relevance of the evidence rather than merely the completion of required analyses.

The success of the collaborative approach is evidenced by its influence on the evolving EU HTA landscape, with the EUnetHTA framework forming the basis for the official joint clinical assessments that commenced in 2025 [58]. The practical implementation of these assessments will likely require health technology developers to use various ITC approaches to address the multiple PICOs requested, while acknowledging the inherent limitations of these methodologies [57].

Methodological Protocols for Robust Indirect Comparisons

ITC Method Selection Framework

Based on the EUnetHTA experience and methodological guidance, researchers can follow a structured approach to selecting and implementing ITC methods.

Table 5: ITC Method Selection Based on Evidence Structure

Evidence Scenario	Recommended ITC Method	Key Assumptions	EUnetHTA Considerations
Pairwise comparison with common comparator	Bucher method (adjusted ITC)	Constancy of relative effects (homogeneity, similarity)	Limited to comparisons with common comparator; not for closed loops
Multiple interventions comparison simultaneously	Network Meta-Analysis (NMA)	Constancy of relative effects (homogeneity, similarity, consistency)	Preferred when source data are sparse; multiarm trials manageable
Population imbalance across studies with IPD available	Matching-Adjusted Indirect Comparison (MAIC)	Constancy of relative or absolute effects	Adjusts for population imbalance but limited to pairwise ITC
Considerable heterogeneity in study population	Simulated Treatment Comparison (STC)	Constancy of relative or absolute effects	Uses outcome regression model based on IPD for prediction
Connected network with effect modifiers	Network Meta-Regression (NMR)	Conditional constancy of relative effects with shared effect modifier	Investigates how distinct factors affect relative treatment effects

Experimental Protocol for ITC Implementation

For researchers planning to incorporate ITCs in HTA submissions, the following protocol synthesizes requirements from EUnetHTA guidance:

Phase 1: Feasibility Assessment

• Map all available evidence using the PICO framework
• Identify common comparators across studies
• Assess clinical and methodological heterogeneity between trials
• Determine whether individual patient data (IPD) is available for population-adjusted methods

Phase 2: Method Selection

• Select the ITC method appropriate for the evidence base (refer to Table 5)
• For population-adjusted methods (MAIC, STC), ensure IPD quality and completeness
• For NMA, verify network connectivity and assess consistency assumptions

Phase 3: Analysis Execution

• Apply appropriate statistical techniques for confounding control
• For propensity score methods, verify positivity, overlap, and balance assumptions
• Conduct extensive sensitivity analyses to assess robustness of findings
• Quantify uncertainty using appropriate statistical measures

Phase 4: Documentation and Reporting

• Justify method selection based on evidence structure and PICO requirements
• Document all assumptions and their clinical rationale
• Report limitations transparently with discussion of potential impact on results
• Align with EUnetHTA guidance for assessors and sponsors [58]

The Scientist's Toolkit: Essential Materials for ITC Research

Table 6: Research Reagent Solutions for ITC Analysis

Tool Category	Specific Methods	Function	Application Context
Foundational ITC Methods	Bucher method, Naïve ITC	Basic indirect comparison framework	Preliminary assessments; pairwise comparisons with common comparator
Multiple Treatment Comparisons	Network Meta-Analysis (NMA), Indirect NMA, Mixed Treatment Comparisons (MTC)	Simultaneous comparison of multiple interventions	Complex evidence networks with multiple relevant comparators
Population Adjustment Methods	Matching-Adjusted Indirect Comparison (MAIC), Simulated Treatment Comparison (STC)	Adjust for cross-study population imbalances	When heterogeneity in patient populations exists between studies
Effect Modification Analysis	Network Meta-Regression (NMR), Multilevel Network Meta-Regression (ML-NMR)	Explore impact of covariates on treatment effects	When effect modifiers are present and need investigation
Advanced Statistical Packages	Bayesian frameworks, Frequentist approaches, Time-varying outcome models	Handle complex statistical challenges and sparse data	When proportional hazard assumptions are violated or data are limited

The EUnetHTA REA experience provides critical insights for researchers and drug development professionals preparing evidence for European HTA submissions. The analysis reveals that while ITCs are frequently necessary to address multiple PICO criteria, their acceptance remains challenging, with only 4% of submitted ITCs deemed appropriate by assessors without major reservations.

Success in this evolving landscape requires adherence to several key principles: selection of statistically robust ITC methods appropriate for the evidence structure, comprehensive assessment and transparent reporting of limitations, meticulous attention to EUnetHTA's methodological guidance on confounding control, and understanding that ITCs must ultimately support decision-making despite inherent limitations.

As the EU moves toward full implementation of joint clinical assessments by 2030, the lessons from EUnetHTA's pilot REAs become increasingly vital. Researchers should prioritize early engagement with HTA methodologies, invest in high-quality IPD collection to enable population-adjusted analyses, and proactively address the methodological critiques that have diminished the persuasiveness of previous ITC submissions. Through application of these evidence generation strategies, the drug development community can contribute to more reliable and informative relative effectiveness assessments, ultimately supporting better healthcare decisions for patients across Europe.

The Role of Clinical Expert Input in Validating ITC Assumptions and Findings

Indirect Treatment Comparisons (ITCs) have become indispensable tools in health technology assessment (HTA) and drug development, enabling the comparison of interventions when head-to-head randomized controlled trials are unavailable or impractical. These methodologies, which include network meta-analyses (NMAs) and population-adjusted indirect comparisons (PAICs), rely on fundamental statistical assumptions—similarity, homogeneity, and consistency—to generate valid evidence. However, these assumptions are fundamentally clinical in nature and cannot be verified through statistical methods alone. This creates a critical dependency on clinical expert input to assess their plausibility and interpret findings within appropriate clinical contexts.

The growing complexity of therapeutic landscapes and the implementation of new HTA frameworks like the European Union's Joint Clinical Assessment (JCA) have intensified the importance of robust ITC validation. Under these frameworks, assessment bodies must evaluate technologies across diverse national healthcare contexts with varied Populations, Interventions, Comparators, and Outcomes (PICOs). This multiplicity amplifies the methodological challenges for comparative effectiveness research, making clinical expert involvement not merely beneficial but essential for ensuring that ITC outputs meaningfully inform healthcare decision-making. This guide examines how clinical expertise validates ITC assumptions and findings, comparing this human-driven validation against purely methodological approaches.

The Foundation: ITC Methods and Their Fundamental Assumptions

Health technology assessment relies on various ITC techniques when direct comparative evidence is lacking. These methods form a hierarchy of sophistication, from simple indirect comparisons to complex population-adjusted analyses. The table below summarizes the primary ITC methods, their applications, and their core requirements.

Table 1: Key Indirect Treatment Comparison Methods and Characteristics

ITC Method	Core Assumptions	Data Requirements	Primary Applications	Key Limitations
Bucher Method [3] [2]	Constancy of relative effects (similarity, homogeneity)	Aggregate data from trials with common comparator	Simple indirect comparisons in connected networks	Limited to pairwise comparisons through common comparator; cannot handle multi-arm trials
Network Meta-Analysis (NMA) [3] [2]	Similarity, homogeneity, consistency	Aggregate data from multiple trials forming connected network	Simultaneous comparison of multiple interventions; treatment ranking	Complexity increases with network size; assumptions challenging to verify statistically
Matching-Adjusted Indirect Comparison (MAIC) [3] [2]	Conditional constancy of relative effects	Individual patient-level data (IPD) for at least one trial; aggregate data for comparator	Adjusting for population imbalances when similarity is violated	Limited to pairwise comparisons; requires IPD; cannot adjust for unobserved effect modifiers
Simulated Treatment Comparison (STC) [3] [27]	Conditional constancy of relative effects	IPD for at least one trial; aggregate data for comparator	Addressing cross-trial heterogeneity through outcome regression	Complex modeling; sensitive to model specification; limited to pairwise comparisons
Network Meta-Regression [3] [2]	Conditional constancy with shared effect modifiers	Aggregate or IPD from multiple trials	Exploring impact of study-level covariates on treatment effects	Cannot adjust for patient-level effect modifiers without IPD

The Critical Role of Clinical Expertise in Assumption Validation

The validity of any ITC depends on three fundamental assumptions that require clinical expertise for proper evaluation:

• Similarity: Trials being compared must be sufficiently similar with respect to effect modifiers—patient characteristics, concomitant treatments, or trial design features—that influence treatment effects. Statistical methods can identify observed differences, but clinical experts determine whether these differences are clinically meaningful and whether unobserved effect modifiers might bias comparisons [3] [27].

• Homogeneity: Relative treatment effects should be consistent across trials comparing the same interventions. Clinical context informs whether between-trial differences represent random variation or clinically significant heterogeneity [3].

• Consistency: Direct and indirect evidence should agree within connected networks. Clinical experts help interpret inconsistency patterns and identify plausible clinical explanations [3] [27].

The European Union HTA Coordination Group guidelines emphasize that violation of these assumptions undermines ITC validity, yet provides limited practical guidance on verification, creating a reliance on clinical judgment [27].

Current Practices: How Clinical Experts Shape ITC Validation

Documented Applications in Health Technology Assessment

Real-world evidence from health technology assessment bodies demonstrates how clinical expert input currently validates ITCs in practice. A review of National Institute for Health and Care Excellence (NICE) technology appraisals revealed that formal statistical methods to determine similarity for cost-comparison analyses were rarely employed. Instead, companies frequently used narrative summaries relying on traditional ITC approaches without formal testing to assert similarity [16]. This approach created uncertainties in several appraisals, which were "usually resolved through clinical expert input alone" [16].

This practice highlights a crucial gap in ITC validation: when statistical methods are insufficient or unavailable, clinical expertise becomes the primary mechanism for resolving uncertainty. Clinical experts provide critical contextual understanding that purely quantitative approaches cannot capture, particularly when:

• Evaluating the clinical relevance of differences in patient populations between trials
• Assessing whether variations in concomitant treatments or medical practice patterns might materially influence relative treatment effects
• Interpreting whether observed outcome differences are clinically plausible given the mechanism of action and disease pathophysiology

Expert Integration in Methodological Selection and Design

Beyond assumption validation, clinical experts play essential roles in selecting appropriate ITC methods and designing analyses. The collaboration between health economics and outcomes research (HEOR) scientists and clinicians is "pivotal in selecting ITC methods in evidence generation" [3]. This partnership operates through complementary responsibilities:

• HEOR Scientists contribute methodological expertise—identifying available evidence, understanding ITC applications, and designing statistical approaches [3].

• Clinicians enhance strategic ITC selection by deciding inclusion/exclusion of source data, rationalizing method adoption, contributing to ITC design, and communicating clinical perspectives to HTA bodies [3].

This collaboration ensures that ITC methodologies align with clinical understanding of the disease and treatment pathways, creating analyses that are both statistically sound and clinically relevant.

Methodological Framework: Systematizing Expert Input

A Roadmap for Early and Continuous Expert Engagement

To maximize the validity of ITCs, clinical expert input should be integrated throughout the evidence generation process rather than merely at the endpoint validation stage. A proposed roadmap for manufacturers suggests initiating preliminary ITC assessments before Phase 3 trials to enable "JCA-ready ITCs across PICOs" [61]. This roadmap includes five key steps that benefit from clinical expertise:

• Targeted searches and PICO simulations to characterize treatment pathways and identify ongoing/planned comparator trials [61]

• Identification of potential treatment effect modifiers and prognostic factors via literature searches and clinical expert input [61]

• Formal comparisons of trial designs, patient populations, and reported outcomes to inform validity of similarity and homogeneity assumptions [61]

• Recommendations for pivotal trial design elements to facilitate consistent comparisons across PICOs [61]

• Recommendations for supplementary evidence generation including real-world evidence-based external comparators [61]

This structured approach ensures evidence generation captures the unique value of an intervention while facilitating fit-for-purpose comparative evidence for HTA assessment [61].

Visualizing the ITC Validation Workflow

The following diagram illustrates how clinical expert input integrates with methodological rigor throughout the ITC development and validation process:

ITC Validation Workflow: Integrating Methodological and Clinical Expertise

Comparative Analysis: Expert Input vs. Statistical Methods

Relative Strengths and Limitations

The validation of ITC assumptions requires both statistical testing and clinical judgment, with each approach offering distinct advantages and limitations. The table below compares these complementary validation approaches:

Table 2: Comparative Analysis of Statistical vs. Clinical Validation Approaches for ITCs

Validation Aspect	Statistical Methods	Clinical Expert Input
Similarity Assessment	Quantifies differences in observed patient characteristics and trial design elements	Evaluates clinical relevance of differences; identifies potential unobserved effect modifiers
Homogeneity Evaluation	Tests for statistical heterogeneity (I², Q-statistic)	Distinguishes clinically meaningful heterogeneity from random variation
Consistency Verification	Node-splitting, inconsistency models, back-calculation methods	Provides clinical explanations for inconsistency; assesses biological plausibility
Handling Unobserved Variables	Limited capability; cannot assess what is not measured	Can hypothesize potential unmeasured confounders based on disease mechanism knowledge
Contextual Interpretation	Limited to quantitative results without clinical context	Places findings within real-world clinical practice and patient care considerations
Transparency and Objectivity	Highly transparent and reproducible	Potentially subjective; requires documentation and justification of reasoning
Regulatory Acceptance	Well-established in guidelines	Often essential for resolving uncertainty but may be viewed as supplementary

Synthesis: Complementary Roles in ITC Validation

The most robust ITC validation emerges from the complementary application of statistical methods and clinical expertise rather than relying exclusively on either approach. Statistical methods provide essential quantitative assessment of observed differences and patterns, while clinical experts interpret these findings within the broader context of disease biology, treatment mechanisms, and clinical practice.

This complementary relationship is particularly crucial when ITC findings challenge clinical expectations or when statistical power is limited. Clinical experts can identify plausible explanations for counterintuitive findings and help determine whether methodological artifacts or genuine clinical phenomena underlie unexpected results. Furthermore, as noted in critical assessments of HTA guidelines, "the exclusion of non-randomized comparisons in rare or rapidly evolving indications may inadvertently hinder access to effective treatments" [27], highlighting situations where clinical expertise becomes particularly vital for interpreting limited evidence.

Experimental Protocols and Research Reagents

Methodological Protocols for Integrating Clinical Expertise

To systematically incorporate clinical expert input into ITC validation, researchers should implement structured protocols:

Protocol 1: Clinical Expert Panel for Assumption Validation

• Objective: Formally assess the plausibility of ITC assumptions (similarity, homogeneity, consistency) through structured expert consultation
• Participants: 3-5 clinical specialists with expertise in the disease area, trial methodology, and treatment pathways
• Materials: Dossier containing trial protocols, patient characteristics, treatment regimens, and outcome definitions for all studies included in ITC
• Procedure:
  • Independent review of materials by each panelist
  • Structured assessment of potential effect modifiers using standardized forms
  • facilitated discussion to identify clinical concerns regarding similarity
  • Consensus development on assumption plausibility with documentation of divergent opinions
  • Formal reporting of clinical rationale for assumption acceptance or rejection

Protocol 2: Clinical Contextualization of ITC Findings

• Objective: Interpret statistically significant ITC findings within clinical context and assess practical significance
• Participants: Multidisciplinary panel including clinicians, methodologies, and patient representatives
• Materials: ITC results including effect estimates, confidence intervals, ranking probabilities, and sensitivity analyses
• Procedure:
  • Presentation of ITC findings with emphasis on magnitude and precision of effects
  • Evaluation of clinical meaningfulness of observed differences
  • Assessment of consistency with biological mechanisms and prior evidence
  • Identification of potential biases or contextual factors affecting interpretation
  • Development of clinical conclusions with transparency about limitations

Table 3: Essential Methodological Tools for Robust ITC Validation

Tool Category	Specific Solutions	Function in ITC Validation	Implementation Considerations
Statistical Software	R (gemtc, pcnetmeta), SAS, Python	Conduct NMAs, PAICs, and statistical assumption testing	Open-source solutions provide flexibility; commercial packages may offer standardized implementations
Data Standardization	CDISC standards, OMOP common data model	Harmonize patient-level data from different trials to facilitate comparison	Critical for reducing methodological heterogeneity when combining data sources
Bias Assessment	ROB-MEN, Cochrane risk of bias, ROBINS-I	Evaluate potential systematic errors in source trials that might affect ITC validity	Should be conducted independently by multiple reviewers with clinical expertise
Effect Modifier Identification	Systematic literature review, clinical guidelines	Identify potential treatment effect modifiers for similarity assessment	Requires comprehensive clinical knowledge of disease and treatment mechanisms
Visualization Tools	Network diagrams, forest plots, inconsistency plots	Communicate complex evidence networks and findings to clinical experts	Essential for facilitating understanding among non-methodologist stakeholders

The validation of ITC assumptions and findings requires a sophisticated integration of methodological rigor and clinical expertise. As health technology assessment evolves toward more complex and cross-national frameworks like the EU Joint Clinical Assessment, the role of clinical experts in validating the similarity, homogeneity, and consistency assumptions underlying ITCs becomes increasingly critical. Statistical methods provide essential quantitative assessments, but clinical expertise supplies the necessary context to interpret these findings and assess their plausibility.

The most robust approach to ITC validation systematically integrates clinical input throughout the evidence generation process—from initial planning through final interpretation—rather than treating it as an endpoint check. This integrated validation framework enhances the credibility and utility of ITCs for healthcare decision-making, ensuring that conclusions reflect both statistical precision and clinical relevance. As ITC methodologies continue to evolve, maintaining this balance between methodological innovation and clinical grounding will be essential for generating evidence that truly informs patient care.

Applying Non-Inferiority Margins and Bayesian Frameworks for Equivalence Testing

Health technology assessment bodies and clinical researchers are increasingly tasked with evaluating whether new medical treatments are clinically similar to existing standards of care. Unlike superiority trials, which test whether a new treatment is better, equivalence testing aims to determine if a new intervention is "not unacceptably worse" than an established comparator [62]. This approach is particularly valuable when new treatments offer secondary advantages such as reduced cost, fewer side effects, or easier administration, making them attractive alternatives even if they are not more efficacious [63] [64].

Within this context, indirect treatment comparisons (ITCs) have emerged as a crucial methodology when head-to-head trial data are unavailable. However, a significant gap exists between methodological potential and practical application. A recent systematic review found that while formal methods for determining equivalence through ITCs exist, they have not yet been widely adopted in practice. Instead, assertions of similarity often rely on narrative summaries without formal testing, leading to uncertainty in decision-making [16].

This guide compares traditional frequentist approaches with emerging Bayesian frameworks for equivalence testing, providing researchers with practical methodologies for implementing these advanced statistical techniques in comparative effectiveness research.

Fundamental Concepts: Equivalence, Non-Inferiority, and Statistical Frameworks

Defining Key Trial Types and Hypotheses

Clinical trials designed to demonstrate similarity between treatments can be categorized into three primary types, each with distinct hypothesis structures:

• Non-Inferiority Trials: Test whether a new treatment is not unacceptably worse than an active control. The goal is to reject the null hypothesis that the new treatment is inferior by more than a pre-specified margin [62] [63].
• Equivalence Trials: Test whether a new treatment is neither superior nor inferior to a comparator, with differences falling within a pre-specified equivalence margin [62].
• Superiority Trials: Test whether a new treatment is statistically significantly better than a control treatment [64].

Table 1: Hypothesis Formulations for Different Trial Types

Trial Type	Null Hypothesis (H₀)	Alternative Hypothesis (H₁)
Superiority	Treatment = Control	Treatment ≠ Control
Non-Inferiority	Treatment ≤ Control - Δ	Treatment > Control - Δ
Equivalence		Treatment - Control	≥ Δ		Treatment - Control	< Δ

In non-inferiority testing, the inferiority margin (Δ) represents the maximum clinically acceptable difference that would still allow the new treatment to be considered non-inferior [63]. This margin must be pre-specified in the study protocol and justified based on clinical considerations, historical data, and statistical reasoning [63].

The Bayesian Framework for Statistical Inference

Bayesian statistics offers an alternative approach to conventional frequentist methods by formally incorporating prior knowledge or beliefs into analytical models [65]. The Bayesian framework is built on several key components:

• Prior Distribution: Represents pre-existing knowledge or beliefs about parameters before observing current trial data [66].
• Likelihood Function: Reflects information contained in the observed trial data [67].
• Posterior Distribution: Combines prior knowledge and observed data to provide updated probability statements about treatment effects [65] [67].

This approach aligns naturally with clinical reasoning, as it allows for probabilistic interpretations such as "there is a 95% probability that the treatment effect lies within a specific range" [67]. For equivalence testing, Bayesian methods enable direct probability statements about whether a treatment effect falls within the equivalence margin [68].

Comparative Analysis: Frequentist vs. Bayesian Approaches

Traditional Frequentist Methods

The Two One-Sided Tests (TOST) procedure serves as the standard frequentist approach for equivalence testing [68]. In this method:

• Two separate one-sided tests are conducted at significance level α
• The first test examines if the treatment difference is greater than -Δ
• The second test examines if the treatment difference is less than +Δ
• Equivalence is declared if both tests are statistically significant

For comparing multiple groups, the frequentist approach extends to examining all possible pairwise differences, requiring that the maximum absolute difference between any two group means remains below the equivalence margin [68].

Table 2: Comparison of Frequentist and Bayesian Approaches to Equivalence Testing

Feature	Frequentist Approach	Bayesian Approach
Interpretation of Results	Confidence intervals and p-values	Direct probability statements
Incorporation of Prior Evidence	Limited to design phase	Formal integration via prior distributions
Handling Multiple Groups	Complex multiplicity adjustments	Natural extension through hierarchical models
Decision Framework	Reject/accept hypotheses based on statistical significance	Probability-based decision rules
Sample Size Requirements	Often larger to achieve adequate power	Can be reduced with informative priors

Bayesian Methods for Equivalence Assessment

Bayesian equivalence testing provides a more nuanced understanding of similarity among treatments compared to traditional hypothesis testing [68]. The Bayesian framework allows researchers to calculate the exact probability that the difference between treatments falls within the equivalence margin, moving beyond dichotomous reject/accept decisions [68].

For multi-group equivalence assessments, Bayesian methods offer particular advantages. Rather than relying on complex multiple testing corrections, Bayesian models naturally accommodate multiple groups through hierarchical structures, providing simultaneous equivalence assessments across all comparisons [68].

A significant advantage of Bayesian methods in regulatory settings is their ability to leverage external data, such as historical trials or real-world evidence, through carefully specified prior distributions [66]. This approach is particularly valuable in situations with limited sample sizes, such as pediatric drug development or rare diseases [66].

Experimental Protocols and Implementation

Establishing the Equivalence Margin

The process for determining an appropriate equivalence margin should include:

• Clinical Justification: The margin should represent the maximum difference considered clinically irrelevant [63]. This requires input from clinical experts and consideration of patient perspectives.
• Historical Context: Analysis of previous placebo-controlled trials to determine the effect size of the active control [63]. This ensures the margin preserves a fraction of the control's effect.
• Statistical Considerations: The margin must be sufficiently small to ensure that a non-inferior treatment would remain superior to placebo [63].
• Regulatory Alignment: Consultation with regulatory guidelines from agencies such as the FDA and EMA regarding margin specifications for specific therapeutic areas [62].

Bayesian Equivalence Testing Protocol

Implementing Bayesian equivalence testing involves the following methodological steps:

• Prior Specification:

  • Select appropriate prior distributions for model parameters
  • Justify prior choices based on historical data or clinical expertise
  • Conduct sensitivity analyses to assess prior influence [66]

• Model Development:

  • Define the statistical model relating outcomes to treatment groups
  • Specify likelihood function based on data distribution
  • For multiple groups, consider hierarchical structures to share information [68]

• Posterior Computation:

  • Use Markov Chain Monte Carlo (MCMC) methods for posterior sampling
  • Validate convergence of computational algorithms
  • Check model fit and adequacy [67]

• Equivalence Assessment:

  • Calculate posterior probability that |θ| < Δ, where θ is the treatment difference
  • Define decision rule (e.g., declare equivalence if probability > 0.95)
  • Report posterior probabilities and credible intervals [68]

• Sensitivity Analysis:

  • Evaluate impact of prior specifications using different prior distributions
  • Assess robustness of conclusions to modeling assumptions [66]

Sample Size Determination

For frequentist non-inferiority trials with continuous outcomes, the sample size per arm can be calculated as:

[ n = \frac{2(Z{1-\beta} + Z{1-\alpha})^2 \sigma^2}{((\mu{new} - \mu{control}) - \Delta)^2} ]

Where σ is the standard deviation, Δ is the non-inferiority margin, and Z represents critical values from the standard normal distribution [63].

Bayesian sample size determination typically involves simulation studies to characterize the operating characteristics of the design, ensuring adequate probability of correctly declaring equivalence when treatments are truly similar [66].

Visualization of Analytical Workflows

Bayesian Equivalence Testing Workflow

Bayesian Analytical Process: This workflow illustrates the sequential learning process in Bayesian equivalence testing, from prior specification to final decision.

Multi-Group Equivalence Assessment

Multi-Group Assessment Logic: This diagram outlines the process for evaluating equivalence across multiple groups or sites, highlighting the Bayesian approach to simultaneous inference.

Research Reagent Solutions: Essential Methodological Components

Table 3: Essential Methodological Components for Equivalence Testing

Component	Function	Implementation Considerations
Statistical Software (R/Stan)	Bayesian posterior computation	Enables MCMC sampling for complex models; Stan provides No-U-Turn Sampler for efficient exploration of parameter space
Prior Distribution Elicitation Tools	Formalize historical evidence	Structured expert judgment protocols; meta-analytic predictive priors; power priors for historical data incorporation
Equivalence Margin Justification Framework	Establish clinically relevant Δ	Includes analysis of historical effect sizes; clinical stakeholder input; regulatory guidance consultation
Model Validation Procedures	Verify analytical robustness	Posterior predictive checks; cross-validation; sensitivity analysis to prior specifications
Decision Rule Framework	Pre-specify equivalence criteria	Bayesian decision-theoretic approaches; posterior probability thresholds (e.g., P(	θ	<Δ) > 0.95)

The integration of non-inferiority margins with Bayesian statistical frameworks represents a methodological advancement in equivalence testing, particularly for indirect treatment comparisons. While traditional frequentist methods like TOST provide established approaches for simple two-group comparisons, Bayesian methods offer enhanced flexibility for complex scenarios involving multiple groups or incorporation of external evidence [68].

Current evidence suggests that formal Bayesian methods for equivalence testing, while methodologically promising, have not yet seen widespread adoption in practical applications such as health technology assessment [16]. As regulatory acceptance of Bayesian approaches continues to grow, particularly in settings where conventional trials are impractical or unethical, these methods are likely to play an increasingly important role in demonstrating treatment similarity [66] [69].

For researchers implementing these methods, careful attention to prior specification, computational robustness, and transparent reporting remains essential for generating credible evidence of equivalence. The Bayesian framework ultimately provides a more intuitive and probabilistic approach to answering the fundamental question in equivalence testing: "How similar is similar enough?" [16].

Comparative Analysis of ITC Performance Across Different Therapeutic Areas

Indirect Treatment Comparisons (ITCs) are statistical methodologies used to estimate the relative treatment effects of two or more interventions when direct, head-to-head evidence from randomized controlled trials (RCTs) is unavailable or insufficient [70] [2]. In the evolving landscape of clinical research and Health Technology Assessment (HTA), ITCs provide valuable comparative evidence for healthcare decision-making, particularly where direct comparisons are ethically challenging, practically unfeasible, or economically non-viable [2] [1].

The fundamental principle underlying ITC involves comparing interventions through a common comparator, which serves as an analytical anchor [1]. For instance, if Treatment A has been compared to Treatment C in one trial, and Treatment B has been compared to the same Treatment C in another trial, an indirect comparison can estimate the effect of A versus B [1]. The validity of this approach hinges on key assumptions, including homogeneity (similarity in trial designs and patient characteristics) and similarity (consistent effect modifiers across trials) [70] [1]. While adjusted ITC methods aim to account for differences between trials, their acceptance by HTA bodies varies significantly based on methodological rigor and the clinical context in which they are applied [70] [8].

ITC Methodologies and Applications

The selection of an appropriate ITC technique is a critical decision that depends on the available evidence base, the nature of the connected treatment network, and the presence of effect modifiers across patient populations.

Core ITC Techniques

• Network Meta-Analysis (NMA): NMA extends traditional meta-analysis to simultaneously compare multiple treatments within a connected network of trials. It is the most frequently described ITC technique, suitable when multiple trials share common comparators and form a connected network [2]. NMA allows for the ranking of treatments and provides estimates for all pairwise comparisons, even those never directly compared in trials [2].

• Bucher Method: This approach facilitates an adjusted indirect comparison between two treatments via a common comparator. It is particularly useful for simple comparisons involving three treatments and requires no individual patient data (IPD) [70] [2]. The Bucher method is valid when the relative treatment effects are consistent across the included trials [6].

• Matching-Adjusted Indirect Comparison (MAIC): MAIC is a population-adjusted method used when patient-level data (IPD) is available for at least one trial. It re-weights the IPD to match the aggregate baseline characteristics of the comparator trial, effectively aligning the patient populations to reduce bias from cross-trial differences [70] [2]. MAIC is especially valuable in single-arm trials or when comparing across disconnected evidence networks.

• Simulated Treatment Comparison (STC): Similar to MAIC, STC is another population-adjusted method that utilizes IPD from one trial to model outcomes based on the aggregate data from another trial. It adjusts for imbalances in effect modifiers through regression techniques [70] [2].

• Network Meta-Regression (NMR): NMR incorporates trial-level covariates into an NMA model to account for variability between studies and adjust for heterogeneity between trials. It helps explore whether treatment effects vary with specific trial characteristics [70] [2].

Table 1: Key Indirect Treatment Comparison Techniques and Their Applications

ITC Technique	Data Requirements	Key Applications	Major Considerations
Network Meta-Analysis (NMA) [2]	Aggregate data from multiple trials	Comparing multiple treatments; ranking interventions	Requires connected evidence network; assumes consistency and transitivity
Bucher Method [70] [2]	Aggregate data from ≥2 trials	Simple comparisons via common comparator	Limited to three treatments; vulnerable to cross-trial heterogeneity
Matching-Adjusted Indirect Comparison (MAIC) [70] [2]	IPD from one trial + aggregate from another	Disconnected networks; differing patient populations	Dependent on IPD availability; limited to effect modifiers measured in both trials
Simulated Treatment Comparison (STC) [70] [2]	IPD from one trial + aggregate from another	Aligning populations across trials; adjusting for effect modifiers	Requires correct identification of all effect modifiers
Network Meta-Regression (NMR) [70] [2]	Aggregate data + trial-level covariates	Explaining heterogeneity; adjusting for trial-level differences	Limited by ecological bias; requires sufficient number of trials

Logical Framework for ITC Methodology Selection

The following diagram illustrates the decision-making process for selecting an appropriate ITC methodology based on trial network connectivity and data availability.

Comparative Performance of ITCs Across Therapeutic Areas

The acceptance and performance of ITCs vary considerably across therapeutic areas, influenced by disease heterogeneity, available endpoints, and clinical trial characteristics.

ITC Performance in Oncology

Oncology represents a particularly challenging yet common application for ITCs due to rapid drug development, parallel innovation, and ethical constraints on placebo controls in life-threatening cancers [70]. A comprehensive analysis of HTA evaluations in solid tumors revealed several key findings:

• Overall Acceptance: Among HTA evaluation reports for oncology treatments, only 22% presented an ITC, with an overall acceptance rate of 30% by HTA agencies [70].
• Cross-Country Variation: Acceptance rates varied significantly across European HTA bodies, with the highest rate observed in England (47%) and the lowest in France (0%) [70].
• Methodological Preferences: Network meta-analysis was the most commonly used ITC technique in oncology (23% of ITCs), followed by the Bucher method (19%) and MAIC (13%) [70].
• Common Criticisms: HTA agencies most frequently cited data limitations, including heterogeneity between trials (48%) and lack of data (43%), as well as concerns about statistical methods (41%) when evaluating ITCs in oncology [70].

Table 2: ITC Acceptance in Oncology HTA Assessments Across European Countries (2018-2021) [70]

Country	HTA Agency	Reports Presenting ITC	ITC Acceptance Rate	Most Common ITC Technique
England	National Institute for Health and Care Excellence (NICE)	51%	47%	Network Meta-Analysis
Germany	Institut für Qualität und Wirtschaftlichkeit im Gesundheitswesen (IQWiG)	Not specified	Not specified	Not specified
France	Haute Autorité de Santé (HAS)	6%	0%	Not specified
Italy	Agenzia Italiana del Farmaco (AIFA)	Not specified	Not specified	Not specified
Spain	Red de Evaluación de Medicamentos del Sistema Nacional de Salud (REvalMed–SNS)	Not specified	Not specified	Not specified
Overall	Across five European countries	22%	30%	Network Meta-Analysis (23%)

Methodological Performance and Validation

The empirical validation of ITC methodologies provides crucial insights into their comparative performance across therapeutic areas:

• Agreement with Direct Evidence: A foundational study comparing 44 direct and adjusted indirect comparisons found that results usually agree, with significant discrepancies in only 3 of 44 comparisons (7%) [6] [71]. The direction of discrepancy was inconsistent, indicating no systematic bias toward overestimation or underestimation of treatment effects [6].

• Technique-Specific Performance: More complex population-adjusted methods (MAIC, STC) have gained prominence in recent years, particularly for single-arm trials commonly encountered in oncology and rare diseases [2]. Among recent publications (2020 onwards), 69.2% described population-adjusted methods, notably MAIC [2].

• HTA Guidance Alignment: A targeted review of 68 ITC guidelines worldwide revealed that most jurisdictions favor population-adjusted or anchored ITC techniques over naive comparisons, with methodology suitability depending on data sources, available evidence, and the magnitude of benefit or uncertainty [8].

Experimental Protocols for ITC Implementation

Implementing a robust ITC requires meticulous methodology and adherence to established statistical principles.

Protocol for Network Meta-Analysis

A standard NMA implementation follows these key methodological steps [2]:

• Systematic Literature Review: Conduct a comprehensive search of relevant databases to identify all RCTs comparing interventions of interest, using predefined inclusion criteria.

• Data Extraction: Extract trial characteristics, patient demographics, and outcome data using standardized forms. Assess study quality using appropriate tools (e.g., Cochrane Risk of Bias tool).

• Network Geometry Evaluation: Map all available comparisons to establish network connectivity and identify potential evidence gaps.

• Statistical Analysis:

  • Model selection between fixed-effect and random-effects models based on heterogeneity assessment.
  • Estimate relative treatment effects using frequentist or Bayesian approaches.
  • Assess consistency between direct and indirect evidence where possible.
  • Rank treatments using surface under the cumulative ranking curve (SUCRA) values.

• Assess Heterogeneity and Inconsistency: Evaluate statistical heterogeneity using I² statistics and assess local and global inconsistency using node-splitting and design-by-treatment interaction models.

Protocol for Matching-Adjusted Indirect Comparison

MAIC implementation requires specific steps to align patient populations [70] [2]:

• IPD Acquisition: Obtain individual patient data for the index treatment (e.g., the new intervention).

• Effect Modifier Identification: Identify and select baseline characteristics that are prognostic for outcomes or modify treatment effects, based on clinical knowledge and previous research.

• Weight Calculation: Assign weights to each patient in the IPD cohort using the method of moments or maximum likelihood estimation so that the weighted baseline characteristics match the aggregate values reported in the comparator trial.

• Outcome Analysis: Fit a weighted outcome model to the IPD (e.g., weighted survival model for time-to-event outcomes) to estimate the treatment effect in the aligned population.

• Comparison and Uncertainty: Compare the adjusted outcome from the weighted IPD to the aggregate outcome from the comparator trial, appropriately propagating uncertainty in the weighting process through bootstrapping or robust standard errors.

Essential Research Toolkit for ITC Implementation

Successfully implementing ITCs requires specific methodological tools and statistical resources.

Table 3: Essential Research Reagent Solutions for Indirect Treatment Comparisons

Research Tool Category	Specific Examples	Function in ITC Implementation
Statistical Software Packages [2]	R (gemtc, pcnetmeta), SAS, WinBUGS/OpenBUGS, Stata	Perform statistical analyses for various ITC methods including NMA and MAIC
Systematic Review Tools [2]	Covidence, Rayyan, DistillerSR	Facilitate study screening, selection, and data extraction during evidence synthesis
Quality Assessment Tools [2]	Cochrane Risk of Bias tool, ISPOR questionnaire	Assess methodological quality and risk of bias in included studies
Data Sources [70] [2]	ClinicalTrials.gov, IPD from sponsors, published aggregates	Provide input data for comparisons, with IPD enabling population-adjusted methods
Methodological Guidelines [8]	NICE TSD Series, ISPOR Task Force reports, EUnetHTA Guidance	Inform appropriate methodology selection and implementation standards

The performance of Indirect Treatment Comparisons varies significantly across therapeutic areas, with oncology demonstrating both the highest application and considerable scrutiny from HTA agencies. The generally low acceptance rate of ITC methods in oncology (30%) underscores the critical importance of methodological rigor, appropriate technique selection, and comprehensive sensitivity analyses [70].

The empirical evidence suggests that adjusted indirect comparisons usually agree with direct evidence, supporting their use when head-to-head trials are unavailable [6]. However, the validity of any ITC depends fundamentally on the internal validity and similarity of the included trials, with population-adjusted methods offering promising approaches to address cross-trial heterogeneity [70] [2].

As therapeutic landscapes continue to evolve rapidly, particularly in oncology and rare diseases, ITC methodologies will play an increasingly vital role in healthcare decision-making. Future developments should focus on standardizing methodology, improving transparency, and establishing clearer international consensus on acceptability criteria to enhance the credibility and utility of ITCs across all therapeutic areas [2] [8].

Conclusion

The validity of an Indirect Treatment Comparison hinges on a meticulous, multi-faceted approach that spans from rigorous methodology to transparent reporting. A successful ITC is not defined by a single statistical technique but by the strategic selection of methods appropriate for the available evidence, coupled with proactive efforts to assess and address limitations. As the landscape evolves with new EU Joint Clinical Assessments, the demand for robust ITCs will only intensify. Future success requires closer collaboration between HEOR scientists and clinicians, adherence to dynamic HTA guidelines, and the adoption of formal validation and bias analysis techniques to generate reliable evidence that truly informs healthcare decisions and improves patient outcomes.

References

• 1. Indirect Treatment Comparisons Guide: Everything you ... [https://remapconsulting.com/training/indirect-treatment-comparisons-guide-everything-you-need-to-know/]
• 2. Methods for Indirect Treatment Comparison: Results from a ... [https://pmc.ncbi.nlm.nih.gov/articles/PMC11036291/]
• 3. Selection of indirect treatment comparisons for health ... [https://pmc.ncbi.nlm.nih.gov/articles/PMC11938225/]
• 4. : A Review of Reporting and... | PLOS One Indirect Comparisons [https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0011054]
• 5. Primer on Adjusted Indirect Meta-Analyses Comparison [https://www.slideserve.com/loren/primer-on-adjusted-indirect-comparison-meta-analyses]
• 6. Validity of indirect comparison for estimating efficacy ... [https://pmc.ncbi.nlm.nih.gov/articles/PMC150178/]
• 7. Matching-Adjusted Indirect Comparisons: A New Tool for ... [https://www.sciencedirect.com/science/article/pii/S1098301512016105]
• 8. A Targeted Review of Worldwide Indirect Treatment ... [https://www.sciencedirect.com/science/article/pii/S1098301524024021]
• 9. Methods for Population-Adjusted Indirect Comparisons in ... [https://pmc.ncbi.nlm.nih.gov/articles/PMC5774635/]
• 10. Leveraging indirect treatment comparisons in joint clinical ... [https://www.putassoc.com/insights/leveraging-indirect-treatment-comparisons-in-joint-clinical-assessments-navigating-tight-timelines-with-precision/]
• 11. A Comparison of Relative-Efficacy Estimate(S) Derived ... [https://www.sciencedirect.com/science/article/pii/S1098301523030619]
• 12. EU HTA 2025 readiness: Key methodological updates and ... [https://www.cencora.com/resources/pharma/eu-hta-2025-key-methodological-updates-and-practical-tips-on-statistical-guidelines]
• 13. A methodological review of population-adjusted indirect ... [https://www.sciencedirect.com/science/article/pii/S0895435623002378]
• 14. Population Adjustment Methods for Indirect Comparisons [https://pmc.ncbi.nlm.nih.gov/articles/PMC6650293/]
• 15. Assessing the performance of population adjustment ... [https://pmc.ncbi.nlm.nih.gov/articles/PMC8690023/]
• 16. How Similar is Similar Enough? Assessment of Indirect ... [https://www.sciencedirect.com/science/article/pii/S1098301525023629]
• 17. The Acceptance of Indirect Treatment Comparison Methods in... [https://link.springer.com/article/10.1007/s41669-023-00455-6]
• 18. Comparator - Clinical Research Explained [https://viares.com/blog/clinical-research-explained/comparator/]
• 19. The Selection of Comparators for Randomized Controlled ... [https://pmc.ncbi.nlm.nih.gov/articles/PMC6543841/]
• 20. Randomisation to protect against selection bias in healthcare ... [https://pmc.ncbi.nlm.nih.gov/articles/PMC7150228/]
• 21. The selection of comparators for randomized controlled ... [https://www.sciencedirect.com/science/article/abs/pii/S0895435618305390]
• 22. Overview of methods for comparing the efficacies of drugs ... [https://pmc.ncbi.nlm.nih.gov/articles/PMC3895352/]
• 23. Effect of Common Comparators in Indirect Comparison ... [https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0120836]
• 24. Indirect Treatment Comparisons in Healthcare Decision ... [https://pmc.ncbi.nlm.nih.gov/articles/PMC11782458/]
• 25. Simple tool implementing Bucher's method for indirect ... [https://pmc.ncbi.nlm.nih.gov/articles/PMC12013566/]
• 26. Simulation evaluation of statistical properties of methods for ... [https://bmcmedresmethodol.biomedcentral.com/articles/10.1186/1471-2288-12-138]
• 27. Between Rigor and Relevance: Why the EU Hta Guidelines ... [https://www.preprints.org/manuscript/202506.0161/v1]
• 28. Multiple treatment and indirect treatment comparisons [https://pmc.ncbi.nlm.nih.gov/articles/PMC4170532/]
• 29. Network meta-analysis: a powerful tool for clinicians ... [https://www.sciencedirect.com/science/article/pii/S0895435624002932]
• 30. Network meta-analysis: a technique to gather evidence ... [https://pmc.ncbi.nlm.nih.gov/articles/PMC5386629/]
• 31. Chapter 11: Undertaking network meta-analyses [https://www.cochrane.org/authors/handbooks-and-manuals/handbook/current/chapter-11]
• 32. How to Conduct a Bayesian Network Meta-Analysis [https://www.frontiersin.org/journals/veterinary-science/articles/10.3389/fvets.2020.00271/full]
• 33. Understanding Network Meta‐Analysis: A Practical ... [https://pmc.ncbi.nlm.nih.gov/articles/PMC12271649/]
• 34. Validating the Assumptions of Population Adjustment [https://pmc.ncbi.nlm.nih.gov/articles/PMC9742635/]
• 35. Performance of unanchored matching-adjusted indirect ... [https://bmcmedresmethodol.biomedcentral.com/articles/10.1186/s12874-020-01124-6]
• 36. R Packages for Matching-Adjusted Indirect Comparison [https://www.sciencedirect.com/science/article/abs/pii/S1098301525023095]
• 37. Addressing challenges with Matching-Adjusted Indirect ... [https://bmcmedresmethodol.biomedcentral.com/articles/10.1186/s12874-025-02500-w]
• 38. Regularized MAIC As a Modern Solution for Indirect ... [https://www.ispor.org/heor-resources/presentations-database/presentation-cti/ispor-2025/poster-session-4/regularized-maic-as-a-modern-solution-for-indirect-treatment-comparison-challenges]
• 39. Concepts and emerging issues of network meta-analysis [https://pmc.ncbi.nlm.nih.gov/articles/PMC8497908/]
• 40. Low awareness of the transitivity assumption in complex ... [https://bmcmedicine.biomedcentral.com/articles/10.1186/s12916-024-03322-1]
• 41. The key assumptions of network meta-analysis [https://www.quantics.co.uk/blog/the-key-assumptions-of-network-meta-analysis/]
• 42. Comparator bias: why comparisons must address genuine ... [https://www.jameslindlibrary.org/articles/comparator-bias-why-comparisons-must-address-genuine-uncertainties/]
• 43. Conducting a High-Quality Systematic Review [https://www.jrheum.org/content/52/7/636]
• 44. Contrast Checker [https://webaim.org/resources/contrastchecker/]
• 45. Quantitative bias analysis for external control arms using ... [https://pmc.ncbi.nlm.nih.gov/articles/PMC10945419/]
• 46. Flexible quantitative bias analysis for unmeasured ... [https://bmcmedresmethodol.biomedcentral.com/articles/10.1186/s12874-025-02551-z]
• 47. Missing data matter: an empirical evaluation of the impacts ... [https://pmc.ncbi.nlm.nih.gov/articles/PMC10280351/]
• 48. Quantitative bias analysis for unmeasured confounding in ... [https://pmc.ncbi.nlm.nih.gov/articles/PMC12527536/]
• 49. Quantitative bias analysis methods for summary-level ... [https://pmc.ncbi.nlm.nih.gov/articles/PMC12135099/]
• 50. Quantifying Uncertainty in RWE Studies with Quantitative ... [https://cytel.com/perspectives/quantifying-uncertainty-in-rwe-studies-with-quantitative-bias-analysis/]
• 51. Validity evaluation of indirect adjustment method for ... [https://www.sciencedirect.com/science/article/abs/pii/S0013935121012871]
• 52. Quantitative bias analysis for mismeasured variables in health ... [https://bmcmedresmethodol.biomedcentral.com/articles/10.1186/s12874-025-02635-w]
• 53. Identifying, Selecting, and Utilizing Quantitative Bias ... [https://www.fda.gov/science-research/advancing-regulatory-science/identifying-selecting-and-utilizing-quantitative-bias-analysis-methods]
• 54. Indirect Comparisons and Network Meta-Analyses [https://pmc.ncbi.nlm.nih.gov/articles/PMC4678383/]
• 55. Report of the ISPOR Task Force on Indirect Treatment ... [https://www.sciencedirect.com/science/article/pii/S1098301511013283]
• 56. How Similar is Similar Enough? Assessment of Indirect ... [https://pubmed.ncbi.nlm.nih.gov/40414457/]
• 57. Indirect Treatment Comparisons in EUnetHTA Relative ... [https://link.springer.com/article/10.1007/s41669-025-00575-1]
• 58. EUnetHTA's Direct and Indirect Comparisons guidance [https://evidence-hub.aetion.com/rwe-guidance-watch-eunethta-direct-and-indirect-comparisons-guidance]
• 59. Success factors for international HTA projects: evaluating ... [https://pubmed.ncbi.nlm.nih.gov/25747565/]
• 60. EVALUATING EUnetHTA JOINT ACTION AS AN EXEMPLAR [https://www.cambridge.org/core/journals/international-journal-of-technology-assessment-in-health-care/article/success-factors-for-international-hta-projects-evaluating-eunethta-joint-action-as-an-exemplar/B40AB39DDFF5CDA6921240F71F6F7448]
• 61. An Early Start: Preliminary Indirect Treatment Comparison ... [https://www.ispor.org/heor-resources/presentations-database/presentation-cti/ispor-europe-2025/methodological-developments-in-indirect-treatment-comparisons-and-network-meta-analyses/an-early-start-preliminary-indirect-treatment-comparison-assessments-informing-evidence-generation-planning-for-joint-clinical-assessment]
• 62. Through the looking glass: understanding non-inferiority - Trials [https://trialsjournal.biomedcentral.com/articles/10.1186/1745-6215-12-106]
• 63. - Non statistics and inferiority studies - PMC equivalence [https://pmc.ncbi.nlm.nih.gov/articles/PMC7808096/]
• 64. of Superiority, Non-superiority, Comparisons - Non , and... inferiority [https://chenghanyang728.medium.com/comparisons-of-superiority-non-superiority-non-inferiority-and-equivalence-trials-87c3d148e82c]
• 65. A Tutorial on Modern Bayesian Methods in Clinical Trials [https://pmc.ncbi.nlm.nih.gov/articles/PMC10117244/]
• 66. Using Bayesian statistics in confirmatory clinical trials in the ... [https://bmcmedresmethodol.biomedcentral.com/articles/10.1186/s12874-024-02235-0]
• 67. Bayesian Analytical Methods in Cardiovascular Clinical Trials [https://www.sciencedirect.com/science/article/pii/S0828282X24011309]
• 68. A Bayesian approach for evaluating equivalence over ... [https://pmc.ncbi.nlm.nih.gov/articles/PMC11389643/]
• 69. A Bayesian framework for virtual comparative trials and ... [https://pmc.ncbi.nlm.nih.gov/articles/PMC11319711/]
• 70. The Acceptance of Indirect Treatment Comparison ... [https://pmc.ncbi.nlm.nih.gov/articles/PMC10781913/]
• 71. Validity of indirect comparison for estimating efficacy ... [https://pubmed.ncbi.nlm.nih.gov/12609941/]