Analyzing the Age-Related Macular Degeneration Dataset

The bivariate mixed-effects (hierarchical) approach

After the Surrogate package and the ARMD dataset are loaded in memory (see Section 13.1), the following commands can be used to evaluate whether the change in visual acuity after 24 weeks (coded as Diff24 in the ARMD dataset) is an appropriate surrogate for the change in visual acuity after 52 weeks (coded as Diff52) using the full and the reduced bivariate mixed-effects (hierarchical) approaches (for details see Chapter 4):

# Fit full bivariate mixed-effects model

> BimixedContCont(Dataset = ARMD, Surr = Diff24, True = Diff52, Treat = Treat, Trial.ID = Center, Pat.ID = Id, Model = "Full")

# Fit reduced bivariate mixed-effects model

> BimixedContCont(Dataset = ARMD, Surr = Diff24, True = Diff52, Treat = Treat, Trial.ID = Center, Pat.ID = Id,

Model = "Reduced")

When these commands are run, error messages are given that indicate that the (full and reduced) bivariate mixed-effects models failed to converge. For example, R gives the following error messages when the reduced hierarchical model is fitted:

Error in lme.formula(outcome ~ -1 + as.factor(endpoint):Treat + as.factor(endpoint), :

nlminb problem, convergence error code = 1 message = singular convergence (7)

As was noted in Section 4.3, such model convergence issues often occur when the hierarchical modeling approach is used in real-life datasets. When such problems occur, the simplified modeling strategies (detailed in 4.3) can be used. This is illustrated below.

The bivariate fixed-effects approach

The function BifixedContCont() uses a two-stage approach to evaluate surrogacy. In Stage 1, the bivariate model (4.10) is fitted, and in Stage 2, the trial-specific treatment effects on T are regressed on the trial-specific treatment effects on S (when a reduced model is fitted) or on the trial-specific intercepts and treatment effects on S (when a full model is fitted). In Stage 2, a weighted or unweighted model can be fitted. Thus, four different bivariate fixed-effect models can be fitted, i.e., a full weighted, full unweighted, reduced weighted, and reduced unweighted bivariate fixed-effect model. By means of illustration, a reduced weighted bivariate fixed-effects model is used here. Such a model can be specified by using the arguments Weighted = TRUE and Model = "Reduced" in the function call:

> ARMD_Bifixed_Fit <- BifixedContCont(Dataset = ARMD,

Surr = Diff24, True = Diff52, Treat = Treat,

Trial.ID = Center, Pat.ID = Id, Model = "Reduced",

Weighted = TRUE)

The fitted object ARMD_Bifixed_Fit (of class Bif ixedContCont) is placed in the R workspace and can subsequently be examined. For example, the summary () function can be applied to the fitted object ARMD_Bifixed_Fit to explore the results:

> summary(ARMD_Bifixed_Fit)

# Generated output:

Function call:

BifixedContCont(Dataset = ARMD, Surr = Diff24, True = Diff52, Treat = Treat, Trial.ID = Center, Pat.ID = Id,

Model = "Reduced", Weighted = TRUE)

# Data summary and descriptives

Total number of trials: 36 Total number of patients: 181

M(SD) patients per trial: 5.0278 (2.9129) [min: 2; max: 18] Total number of patients in experimental treatment group: 84 Total number of patients in control treatment group: 97

Mean surrogate and true endpoint values in each treatment group:

Control.Treatment Experimental.treatment Surrogate -6.0309 -7.8095

True endpoint -11.7423 -14.6548

Var surrogate and true endpoint values in each treatment group:

Control.Treatment Experimental.treatment Surrogate 188.9261 132.6380

True endpoint 264.7975 231.7709

Correlations between the true and surrogate endpoints in the control (r_T0S0) and the experimental treatment groups (r_T1S1):

Estimate Standard Error CI lower limit CI upper limit r_T0S0 0.7693 0.0478 0.7022 0.8228

r_T1S1 0.7118 0.0525 0.6315 0.7770

# Meta-analytic results summary

R2 Trial Standard Error CI lower limit CI upper limit 0.6585 0.0965 0.4695 0.8476

R2 Indiv Standard Error CI lower limit CI upper limit 0.5318 0.0512 0.4315 0.6321

R Trial Standard Error CI lower limit CI upper limit

0.8115 0.1002 0.6585 0.9001

R Indiv Standard Error CI lower limit CI upper limit 0.7293 0.0511 0.5270 0.8533

The first part of the output shows descriptives like the number of trials in the data, the number of patients per treatment condition, and the correlations between S and T in both treatment conditions. The second part of the output is of main interest, as it provides the estimates of Rrial and R2_ndiv, their standard errors, and their (1 — a)% confidence intervals. As can be seen, R²rial = 0.6585 with 95% confidence interval [0.4695; 0.8476], and R2_ndiv = 0.5318 with 95% confidence interval [0.4315; 0.6321] (see also Table 4.2). To depict the results graphically, the plot() function can be applied to the fitted object ARMD_Bifixed_Fit.

# Plot of trial-level surrogacy

> plot(ARMD_Bifixed_Fit, Trial.Level = TRUE,

Indiv.Level = FALSE)

# Generated output:

The R Package Surrogate

223

# Plot of individual-level surrogacy

> plot(ARMD_Bifixed_Fit, Trial.Level = FALSE, Indiv.Level = TRUE)

# Generated output:

The trial-level plot (first plot in the output) shows the estimated trial-specific (here: center-specific) treatment effects on T = visual acuity after 52 weeks (i.e., Pi) against the estimated trial-specific (center-specific) treatment effects on S = visual acuity after 24 weeks (i.e., a_i). By default, the sizes of the circles that depict вг, а_г are proportional to number of patients in the trials (centers). When circles of the same size are desired for all trials (centers) irrespective of their sample size, the option Weighted=FALSE can be used in the plot() function call. The individual-level plot (second plot in the output) shows the trial- and treatment-corrected residuals for T (i.e., e_Tij) against the trial- and treatment-corrected residuals for S (i.e., e_Sij).

Overall, the results (surrogacy metrics and plots) lead to the conclusion that, at the level of the trial (center), the treatment effect on T = visual acuity after 52 weeks can be predicted with moderate accuracy based on the treatment effect on S = visual acuity after 24 weeks (moderate Rt_rial estimates). At the level of the individual patients, the accuracy by which T = visual acuity after 52 weeks can be predicted based on S = visual acuity after 24 weeks is moderate as well (moderate .R²ndiv estimates).

Examining the components of a fitted object of class BifixedContCont()

The summary() function can be applied to a fitted object of class BifixedContCont to obtain a summary of the most relevant results (see previous paragraph). The different components of a fitted object can also be examined in more detail. When the function names() is applied to the fitted object, an overview of the components that can be accessed is provided:

> names(ARMD_Bifixed_Fit) # Generated output:

[1] "Data.Analyze" "Obs.Per.Trial" "Results.Stage.1"

[4] "Residuals.Stage.1" "Results.Stage.2" "Trial.R2"

[7] "Indiv.R2" "Trial.R" "Indiv.R"

[10] "Cor.Endpoints" "D.Equiv" "Sigma"

[13] "Call"

These components contain the following information:

• • Data.Analyze: A data.frame that contains the data that were actually included in the analysis. This is not always exactly the same dataset as the one that was specified by the user in the Dataset= argument of the BifixedContCont() function call. Indeed, prior to conducting the analysis, data of patients who have a missing value for the surrogate and/or the true endpoint are excluded. In addition, the data of trials (i) in which only one type of the treatment was administered (i.e., all patients within a trial received the control treatment or all patients within the trial received the experimental treatment), and (ii) in which either the surrogate or the true endpoint was a constant (i.e., all patients within a trial had exactly the same surrogate and/or true endpoint value) are excluded. The user can also specify the minimum number of patients that a trial should contain in order to be included in the analysis (using the Min.Trial.Size= argument in the function call). If the number of patients in a particular trial is smaller than the value specified by Min.Trial.Size, the data of the trial are excluded as well.
• • Obs.Per.Trial: A data.frame that contains the total number of patients per trial and the number of patients who were administered the control treatment and the experimental treatment in each of the trials.
• • Results. Stage. 1: The results of Stage 1 of the two-stage model-fitting approach, i.e., a data.frame that contains the trial-specific intercepts and treatment effects for S and T.
• • Residuals.Stage.1: A data.frame that contains the residuals for S and T that are obtained in Stage 1 of the analysis, i.e., e_Sij and e_Tij.
• • Results.Stage.2: An object of class lm (linear model) that contains the parameter estimates of the regression model that is fitted in Stage 2 of the analysis.
• • Trial. R2: A data.frame that contains the trial-level coefficient of determination (R_riai), its standard error, and confidence interval.
• • Indiv.R2: A data.frame that contains the individual-level coefficient of determination (R2_ndiv), its standard error, and confidence interval.
• • Trial.R: A data.frame that contains the trial-level correlation coefficient (R_trial), its standard error, and confidence interval.
• • Indiv.R: A data.frame that contains the individual-level correlation coefficient (R_indiv), its standard error, and confidence interval.
• • Cor.Endpoints: A data.frame that contains the correlations between S and T in the control treatment group and in the experimental treatment group, their standard errors, and their confidence intervals.
• • D. Equiv: The variance-covariance matrix of the trial-specific intercept and treatment effects for S and T. The name D.Equiv refers to the fact that the matrix is similar to the D matrix that is obtained when the hierarchical (bivariate mixed-effects) approach is used.
• • Sigma: The 2-by-2 variance-covariance matrix of the residuals (esij and
• ?Tij ⁾.
• • Call: The function call that was specified by the user.

These components can be accessed using the

[Name_Fitted_Object] $ [Component JJame] command. For example, the trial-

specific intercepts and treatment effects on S and T can be obtained using

the following command:

# Request trial-specific results for first 5 trials (centers)

> ARMD_Bifixed_Fit$Results.Stage.1[1:5,]

# Generated output:

Trial Obs.per.trial Treatment.S Treatment.T

• 1 13395 2 1.500000 -5.500000
• 2 13396 4 -4.250000 -4.250000
• 3 13745 3 -4.417648 2.626629
• 4 13746 7 2.106722 1.839984
• 5 13748 5 0.949411 -4.024023

The output shows that, for example, in the first trial (center) (coded as Trial ID = 13395 in the ARMD dataset) there were 2 patients and the estimated treatment effects on S and T in this center equaled a₁ = 1.5 and = -5.5, respectively. As another example, the results of the Stage 2 analysis can be obtained by the command:

# Request Stage 2 model results

> ARMD_Bifixed_Fit$Results.Stage . 2

# Generated output:

Call:

lm(formula = Treatment.T ~ Treatment.S, data = Results.Stage.1, weights = Results.Stage.1$Obs.per.trial)

Coefficients:

(Intercept) Treatment.S -0.4467 1.1252

Thus, when the Stage 2 model вг = A₀ + ец + е_г is fitted based on the Stage 1 results, it is obtained that A₀ = -0.4467 and Ai = 1.1252. As noted above, ARMD_Bif ixed_Fit$Results. Stage. 2 is a fitted object of class lm and thus it can be explored in a straightforward way using the available functions for this class. For example, diagnostic plots of model fit (e.g., a normal Q-Q plot) can be obtained by applying the plot() function, or more details regarding the fitted model (e.g., the standard errors of the parameter estimates) can be obtained by using the summary() function. By means of illustration, the summary() function is applied here:

> summary(ARMD_Bifixed_Fit$Results.Stage.2)

# Generated output:

Call:

lm(formula = Treatment.T ~ Treatment.S, data = Results.Stage.1, weights = Results.Stage.1$Obs.per.trial)

Weighted Residuals:

Min 1Q Median 3Q Max

-29.485 -8.913 1.746 6.538 18.591

Coefficients:

Estimate Std. Error t value Pr(>|t|)

(Intercept) -0.4467 0.7666 -0.583 0.564

Treatment.S 1.1252 0.1390 8.098 1.92e-09 ***

Signif. codes: 0 '***' 0.001 '**' 0.01 0.05 '.' 0.1 ' ' 1

Residual standard error: 10.18 on 34 degrees of freedom Multiple R-squared: 0.6585,Adjusted R-squared: 0.6485 F-statistic: 65.57 on 1 and 34 DF, p-value: 1.925e-09

Notice that the Multiple R-squared: 0.6585 component toward the end of the output corresponds to the estimated R_rial.

The expected treatment effect on T in a new trial

The main motivation to evaluate a candidate surrogate endpoint is to be able to predict the treatment effect on T based on the estimated treatment effect on S in a new trial i = 0. For example, suppose that a new clinical trial is conducted using the same treatments as the ones that were used in the ARMD trial in a similar patient population. In this new trial, visual acuity is only measured after 24 weeks but not after 52 weeks. Interest is in the prediction of the expected treatment effect on visual acuity after 52 weeks, based on the treatment effect on visual acuity after 24 weeks. Suppose that So = — 1, i.e., the estimated treatment effect on visual acuity after 24 weeks in the new trial, equals —1. Based on this estimate and using the results of the bivariate weighted reduced fixed-effects model that was fitted above, the expected treatment effect on T, i.e., E (в + b₀ | a₀ = —1), can be obtained using the following commands:

# Compute expected treatment effect on T in new trial

# i = 0 where alpha_0 = -1

> ExpTreatT <- Pred.TrialT.ContCont(ARMD_Bifixed_Fit, alpha_0 = -1)

The results can be explored by applying the summary() and plot() functions to the fitted ExpTreatT object:

# Request summary of the results:

> summary(ExpTreatT)

# Generated output:

Function call:

Pred.TrialT.ContCont(Object = Reduced_Weighted, alpha_0 = -1) Results:

Expected treatment effect (variance) on T (beta_0) = -1.9324 (25.1379) 95% CI: [-11.7592; 7.8944]

# Plot the results

> plot(ExpTreatT)

# Generated output:

The plot shows the trial-specific treatment effects on T and S that were obtained in the ARMD dataset using the weighted reduced bivariate fixed-effects model (gray circles), the expected treatment effect on T in the hypothetical new trial i = 0 (black circle), and the 95% confidence interval around the expected treatment effect on T in the new trial (black dashed line).

The output of the summary () function shows that the expected treatment effect on T given So = —1 equals E (p + b₀ | a₀ = —1) = -1.9324, with 95% confidence interval [—11.7592; 7.8944]. The confidence interval around the expected treatment effect on T is thus wide, as could be expected given the fact that the trial-level surrogacy coefficient was moderate (i.e., i?²rial = 0.6585).

The univariate mixed- and fixed-effects approaches

Similar to what was the case with the BifixedContCont() function, the functions UnimixedContCont() and UnifixedContCont() use a two-stage approach to assess surrogacy. In Stage 1, two univariate mixed- or fixed-effects models are fitted. In Stage 2, the trial-specific treatment effects on T are regressed on the trial-specific treatment effects on S (when a reduced model is fitted) or on both the trial-specific intercepts and treatment effects on S (when a full model is fitted). In Stage 2, one can opt again for a weighted or an unweighted analysis.

For example, suppose that we want to use a reduced weighted univariate mixed-effects model to evaluate surrogacy in the ARMD dataset. Such a model can be specified by the following command:

# Fit the model

> ARMD_Unimixed_Fit <- UnimixedContCont(Dataset = ARMD,

Surr = Diff24, True = Diff52, Treat = Treat,

Trial.ID = Center, Pat.ID = Id, Model = "Reduced",

Weighted = TRUE)

Similar to what was done above, the output can be examined by applying the summary() and plot() functions or by directly accessing components in the fitted object ARMD_Unimixed_Fit using the ARMD_Unimixed_Fit$ [Name.Component] command.