Critical appraisal of journal club article

N Engl J Med. 2013 Jan 31;368(5):416-24. Priming after a fractional dose of inactivated poliovirus vaccine. Resik S1, Tejeda A, Sutter RW, Diaz M, Sarmiento L, Alemañi N, Garcia G, Fonseca M, Hung LH, Kahn AL, Burton A, Landaverde JM, Aylward RB.

Abstract

BACKGROUND: To reduce the costs of maintaining a poliovirus immunization base in low-income areas, we assessed the extent of priming immune responses after the administration of inactivated poliovirus vaccine (IPV).

METHODS: We compared the immunogenicity and reactogenicity of a fractional dose of IPV (one fifth of a full dose) administered intradermally with a full dose administered intramuscularly in Cuban infants at the ages of 4 and 8 months. Blood was collected from infants at the ages of 4 months, 8 months, 8 months 7 days, and 8 months 30 days to assess single-dose seroconversion, single-dose priming of immune responses, and two-dose seroconversion. Specimens were tested with a neutralization assay.

RESULTS: A total of 320 infants underwent randomization, and 310 infants (96.9%) fulfilled the study requirements. In the group receiving the first fractional dose of IPV, seroconversion to poliovirus types 1, 2, and 3 occurred in 16.6%, 47.1%, and 14.7% of participants, respectively, as compared with 46.6%, 62.8%, and 32.0% in the group receiving the first full dose of IPV (p < 0.008 for all comparisons). A priming immune response to poliovirus types 1, 2, and 3 occurred in 90.8%, 94.0%, and 89.6% of participants, respectively, in the group receiving the fractional dose as compared with 97.6%, 98.3%, and 98.1% in the group receiving the full dose (P=0.01 for the comparison with type 3). After the administration of the second dose of IPV in the group receiving fractional doses, cumulative two-dose seroconversion to poliovirus types 1, 2, and 3 occurred in 93.6%, 98.1%, and 93.0% of participants, respectively, as compared with 100.0%, 100.0%, and 99.4% in the group receiving the full dose (P < 0.006 for the comparisons of types 1 and 3). The group receiving intradermal injections had the greatest number of adverse events, most of which were minor in intensity and none of which had serious consequences.

CONCLUSIONS: This evaluation shows that vaccinating infants with a single fractional dose of IPV can induce priming and seroconversion in more than 90% of immunized infants. (Funded by the World Health Organization and the Pan American Health Organization; Australian New Zealand Clinical Trials Registry number, ACTRN12610001046099.).

journal club review comments for above published research paper

Antibody titers have been measured repeatedly at 4 different time points (4 mo, 8mo, 8 mo 17 days & 9 month as highlighted in figure 1) making the data as serial measurement data. Ideally, such serially measured data should have been analyzed between two study groups using ‘Repeated measure ANOVA’ analysis with appropriate adjustment for ‘time× study groups’ interaction. On the contrary, authors have misleadingly carried out a separate cross sectional comparison using KS nonparametric test (a test nearly similar to Mann Whitney U test) for each time point which can easily inflate the possibility of type 1 error in their study.

Serological data like antibody titers are best described statistically as geometric means which again has been ignored by authors who have used medians and their 95% CIs as their descriptive statistical indices.

Authors have conspicuously ignored the reporting of any effect size measures (except between group differences) for their outcome data.

For example, what was the ‘number needed to treat to benefit’ (NNTB) (for cumulative seroconversion) for each type of polio virus for FD vs. Full dose.

Poliovirus type Control Experimental Absolute difference in
cumulative seroconversion
NNTB (95% CI)
Type 1 153/153 (100) 147/157 (93.6) 6.4 (2.0 to 11.7) 15 (8-50)
Type 2 153/153 (100) 154/157 (98.1) 1.9 (−1.5 to 5.9) 50
Type 3 152/153 (99.3) 146/157 (93.0) 6.4 (1.6 to 11.9) 15 (8-62)

Also, what was the ‘number needed to treat to harm’ (NNTH) for adverse effects.

Adverse effect Experimental Control Absolute risk difference (95% CI) NNTH (95%CI)
Redness
First dose 47/157 3/153 28%(20%-35%) 4(3-5)
Second dose 37/157 2/153 22%(15%-30%) 5(3-7)
Induration
First dose 11/157 2/153 5.7% (1.2%-10.9%) 18(9-85)
Second dose 14/157 1/153 8.2%(3.7%-13.8%) 12(7-27)

In their sample size calculation and study design planning, authors have failed to clarify whether they planned a superiority, equivalence or non inferiority trial of their fractional dose with full dose IPV.

Non-inferiority analysis (one-sided): For example, if authors planned this RCT as non inferiority trial, then sample size calculation will be as follow:

A new fractional dose of IPV is tested against standard full dose IPV for the primary outcome of seroconversion. For the sample size calculations, the investigators assumed the proportion converted in both arms would be 40% (pi=0.40). They also considered that a difference in seroconversion rate as large as 20% in favor of full dose IPV would still allow the new fractional dose IPV to be non-inferior (delta=0.20). Then calculate the sample size based on 90% power (i.e. beta of 10% or 0.10) and a one-sided confidence level of 97.5% (i.e. one-sided alpha of 0.025 which when two tailed would have been 5%).

Table: Sample size for non inferiority trial using “Stata” software with ssi command

Seroconversion in fractional dose IVP (p1) Seroconversion in full dose IVP (p2) Delta (p2-p1) Power (Beta) Alpha (one-sided) Sample size in each study arm Total sample size
40% 60% 20% 90% (10%) 0.025 127 254
45% 60% 15% 90% (10%) 0.025 232 464
50% 60% 10% 90% (10%) 0.025 526 1052

Equivalence analysis (two-sided): On the other hand, if investigators had thought that the fractional dose could be even better than full dose, but they were not certain, therefore they wanted an equivalent study design which would allow for a two sided test relaxing their significance assumption to a more conventional alpha of 0.05. Then new sample size would be 156 subjects in each group.

Table: Sample size for equivalence trial using “Stata” software with ssi command

Seroconversion in fractional dose IVP (p1) Seroconversion in full dose IVP (p2) Delta (p2-p1) Power (Beta) Alpha (two-sided) Sample size in each study arm Total sample size
40% 60% 20% 90% (10%) 0.05 156 312
45% 60% 15% 90% (10%) 0.05 286 572
50% 60% 10% 90% (10%) 0.05 650 1300

Using equivalence sample size calculation (which is two sided), sample size in each arm of study comes to 156 which closely mirrors the finally estimated sample size in current study. Hence, authors have used equivalence trial design.

But what was the basis of selection of such a wide “minimally detectable difference” of 20% (60%-40%) for sample size calculation in this study? Ideally authors should have kept this difference not more than 10% to 15%.