Topic Review - Newest First (only newest 5 are displayed)

rqayyum

Meta-analysis - Effect Size Selection

Before one starts planning to combine study-results, one needs to consider whether it is appropriate to combine these studies. This is important as the studies may be so different in methodology that combining them may provide misleading or unreliable results. If all studies can’t be combined, one can further evaluate whether some of the studies, with similar methodology, can be combined. For example, it may not be appropriate to combine randomized controlled trials with trials that have no control group and compare results before and after a treatment. However, it may be appropriate to combine randomized trials only or to analyze these two different types of trials separately. If studies can’t be combined meaningfully, then one should not perform a meta-analysis and instead, should stop at systematic review of the literature.

Meta-analysis is performed in two steps or levels. First step involves calculation of an effect size for each individual study. Second step is to pool the results from individual studies to calculate an overall effect size. It is important to note from this two-step or two-level approach that in meta-analysis, data is not combined from all the trials as if they are from a single trial. In other words, one can consider meta-analysis as an example of multilevel modeling.

Selection of a summary statistic to express effect size is probably one of the most important steps in performing a meta-analysis. This selection depends on the study question and the type of data at hand. There are different summary statistics for trials with events data (binary outcome) as compared to trials that report results on other scales.

In case of binary outcomes, where there are only two possibilities (for example dead or alive, sick or healthy, etc.), multiple summary statistics are available. Most commonly used summary statistics are odds ratio, relative risk ratio, relative risk reduction, absolute risk reduction, and number needed to treat. Sometimes risk ratios are expressed as percentage; however, statistical analyses are performed on original values and not on percentage values. A summary statistic should be easy to interpret and should have a reliable variance estimate which is important in performing a meta-analysis. As number needed to treat does not have such an estimate for its variance, it is not a good choice for a summary effect. Another important point is that odds ratio and relative risk ratios are combined on a natural log scale. For a typical 2x2 table following are formulas for calculating these statistics

Odds Ratio = ad/bc
Natural log of odds ratio = ln (ad/bc)
Natural log of Variance of odds ratio = (1/a) + (1/b) + (1/c) + (1/d)
Relative Risk = (a / (a+b) / (c / (c+d))
Natural log of Relative Risk = ln (a / (a+b) / (c / (c+d))
Natural log of variance of Relative Risk = (1/a) - (1/ (a+b)) + (1/c) - (1/(c+d)).

If outcomes are on a continuous scale, choice of a summary statistic is either mean difference (if all studies used same scale for outcome measurement) or standardized mean difference (if studies used different scales for outcome measurement). For example, change in BP in response to a certain treatment is measured on the same scale and thus the summary statistic will be mean difference. On the other hand, there are multiple scales for evaluation of depression and different studies may use different scales. In such a scenario, a standardized mean-difference will be used to summarize trial results. However, pooled summary statistics obtained from meta-analysis of trials summarized with standardized mean-difference may be difficult to interpret.

rqayyum

Meta-analysis - Searching Relevant Studies

Once we have a research question and we have established inclusion and exclusion criteria for relevant studies, we need to determine a search strategy to identify relevant clinical trials. In developing our search strategy, we should keep in mind that our research should find as many studies as possible, while at the same time it should be efficient.

One can try to find all relevant trials ever done on a particular topic, but this is practically impossible and quite inefficient. Generally, the harder one tries to find studies, more relevant studies one will find, but after a certain number of studies are identified, every incremental effort result in a decrease in the number of identified studies. When should one stop searching for additional studies is controversial.

In general, relevant studies are identified by searching medical databases. PakMediNet is one such database of Pakistani medical journals. PubMed is another database which is very much bigger, containing more than 10 million references, and more than 400,000 references are added annually. It covers more than 3900 medical journals in 40 languages (88% in English). One can put “Limits” to one’s search, which helps to decrease the number of returned references. Its major deficiencies are that it covers only about 33% of medical journals and that it only goes back to 1966. A second database is EMBASE, it is somewhat larger than PubMed, but is commercial and no free version is available. Another important source of randomized controlled trials is “The Cochrane Controlled Trials Register”. This Register includes all randomized controlled trials published in 1700 medical journals. It does not contain non-randomized clinical studies. OVID Online is another database that can be searched. Although OVID is a commercial database, one can access it through Merck Medicus website. One should remember that there is overlap between these databases and most of the articles retrieved will be the same. Other databases that can be searched are AMED, BIOSIS, CINHAL, PsycINFO, and Science Citation Index.

An important aspect of search for relevant clinical trials is to perform a hand-search of references of the retrieved articles as well as relevant review articles. This search usually retrieves a significant number of relevant studies that have not been properly indexed by databases.

Whether one decides to include clinical trials that are not (yet) published in medical journals determines the next step in search. If one decides to search unpublished clinical trials, there are multiple resources that can be searched. For example, abstracts from relevant conference proceedings, ClinicalTrials.gov, CRISP database of NIH, or FDA database of clinical trials. Investigators can be contacted individually to learn about ongoing or recently completely but unpublished trials.

If properly done, a comprehensive search of the relevant clinical trials can tremendously improve the quality of the meta-analysis. On the other hand, an incomplete search will result in publication bias which can severely compromise the results and conclusion of the meta-analysis.