Regression models for inter-study heterogeneity

9. Technical notes A. Meta-analysis According to Glass (28), a primary analysis is the original analysis of data in a research study. Secon- dary analysis is the re-analysis of data for the pur- pose of answering the original research questions with better statistical techniques, or answering new questions with old data. Meta-analysis refers to the analysis of analyses or "the statistical analysis of a large collection of analysis results from individual studies for the purpose of integrating the findings. It connotes a rigorous altemative to the casual, nar- rative discussions of research studies which typify our attempts to make sense of a large volume of re- search literature". In the context of the present project, original data sets were obtained and all analyses were conducted on these. Statistics (i.e., ORs) from individual studies were then combined formally by means of the Confidence Profile (CP) method. Meta-analysis by the Confidence Profile method Basically, as applied here, the Confidence Profile method (7) takes information on the parameter of interest from the available studies and, by means of a Bayesian model, derives a joint posterior probability distribution for this parameter. The process is illus- trated in the following. The observed result from the analysis of a single study is denoted as 0 (the parameter of interest, e.g., the odds ratio). The evidence supporting this parame- ter is supplied by the data. This may be the propor- tion of cases y - low-birth-weight infants, for exam- ple - in the study. The likelihood function for the evidence, given the parameter, is: L(y 0). If our belief in the parameter is expressed as a prior distribution for 0: 7(O), then we can calculate the posterior distribution as the product of the like- lihood function and the prior distribution: T(1|y) = k L(yI|);ir() If we have no strong belief in the value of the prior ditribution, then a non-informative prior can be used. In the CP method this is based on a beta distri- bution with parameters x = 1/2 and f = 1/2. The posterior distribution obtained from the evidence derived from our first study becomes the prior distri- bution for evaluating the second study: ,r( IY1 Y2) = k L(y21 9)Z(9 I Y1) And this gives the new posterior distribution based on both studies. The process can be extended in this manner to incorporate evidence from all available studies. The result is a posterior probability distribution which may be graphed and studied. B. Regression models for inter- study heterogeneity A number of references have been made in Chapters 2 and 4 on the need to test the odds ratios for homo- geneity before meta-analysis. This test is of the null hypothesis that all study ORs are estimating the same underlying value, versus the alternative that at least one of the ORs differs from the remainder (the 'Q' test for homogeneity, see Hedges & Olkin (10)). Should the test be significant, a weighted regression analysis may supply an explanation of the inter-study variation in the ORs. The regression model will include some of the study characteristics described in the study quality table at the end of Chapter 2, includ- ing study design (prospective versus retrospective), prevalence level of the outcome, country groupings, regional groups (Africa, southern Asia, south-east Asia, Latin America, USA and Europe), location of the study (urban, rural, mixed), decade of the study, WHO Collaborative study versus other, as well as maternal characteristics such as mean maternal age, mean gestational age at delivery, etc. As was noted in Chapter 4, many of the tests were confirmed as statistically significant, and these cases were exam- ined by regression analysis. Some of the results will be given in brief to provide an indication of the fac- tors having some explanatory power. Analysis for IUGR Predictor: attained weight at month 5. For this pre- dictor and outcome, the regression analysis accoun- ted for around 34% of the observed variation in the study ORs. The one variable found to be significant was the WHO study versus other study factor. This results from a significant difference in the means of the odds ratios for both categories. The reason is unlikely to be a different biological relationship be- tween predictor and outcome in the geographical areas covered by either study. Some bias to external validity is more reasonable. None of the remaining factors was statistically significant 48 WHO Bulletin OMS: Supplement Vol. 73 1995 Technical notes Predictor: attained weight at month 7. The test for homogeneity is rejected; however, the regression analysis turned up no variable significant at the tradi- tional 5% level. There is no evidence for country group differences. The studies contributing most to the significant test statistic were China and Vietnam. An examination of the components of the test statis- tic identifies the two main contributions to the heterogeneity statistic (see Fig. 27). The individual contributions by China and Viet Nam to the overall Q test statistics (designated qi and highlighted on the right side of Fig. 27) are dominant. No obvious study effect can be called upon to account for this. Predictor: Attained weight at month 9. The test for homogeneity was significant and the regression analysis identified both the WHO collaborative fac- tor and the study mean duration of pregnancy. The country grouping variable was not significant. The coefficient of determination (R2) indicates that 51% of the variation has been explained between these factors. Analysis for low birth weight Predictor: attained weight at month 9. This model identified both the country group effects and the WHO study factor. The R2 was 75%. Additional fac- tors were not significant. In summary, when significant inter-study differ- ences between the ORs - suggesting that all ORs were not estimating the same underlying effect were found, the subsequent regression analysis iden- tified one or more study-related factors that account- ed for a proportion of the observed total variation. Most commonly, the analysis identified the country grouping effect. Other factors were found in some instances, for example, the mean gestational age at delivery or the WHO study versus other studies. In only a very few instances did the model fail to find some significant factor to help explain the result of the homogeneity test. In these cases the studies con- tributing to the significant test result could be identi- fied. Normally this would be one or two apparently anomalous results. The proportion of variation accounted for in any one model varied from around 30% to as high as 80%. In part this difference reflects the structure of the model, especially the number of actual parameters present. For example, a model with the regional factor has 5 dummy vari- ables coding for 6 regional areas. This is very likely to result in a higher R2 than a model with only one or two levels. Accounting for inter-study differences in ORs is of course not quite the same as explaining these dif- ferences. To state that the country grouping factor 'explained' 50% of the total variation in the model only indicates that one or more group means were statistically different. Whether this is really reflective of a genuine difference in the basic biological rela- tionships is questionable. Obvious differences in study design and quality of the data, and in the char- acteristics of the study populations (e.g., the preva- lence of anaemia or malaria, parity differences, etc.), which were not captured by the available study fac- tors, are likely to lead to measurable differences in the ORs. For this reason, and in spite of some sig- nificant tests for homogeneity, the combined OR estimates provided by the meta-analysis are felt to be adequate for purposes of ranking the indicators. It might also be recalled that detectable differences in, for example, group means does not imply that such differences would necessarily be regarded as of practical significance. C. Predicting pre-pregnancy weight based on arm circumference In many different country settings mothers appear rather late in pregnancy, thereby limiting the utility of weight gain charts. If information on pre-pregnan- cy weight were available, then 2 points could be entered onto the chart and some indication of the mother's pattem of weight gain would be available. Mid-upper-arm circumference has been suggested in the past as being well correlated with pre-pregnancy weight (29-31) and might therefore be employed to back-predict for this purpose. However, a high corre- lation does not necessarily signify that MUAC will be an accurate and reasonably precise predictor of pre-pregnancy weight. A regression analysis for each data set with both variables was undertaken to Fig. 27. Contribution of individual studies to the 0 test for homogeneity. The values for Vietnam (vie) and China (chi) dominate this test statistic. Country qi vie 8 uk 7 tha nepu 6 nepr myn mal 4 les ire inp ino gainm chi1 bot WHO Bulletin OMS: Supplement Vol. 73 1995 49