Variability of outcome reporting in Hirschsprung’s Disease and gastroschisis: a systematic review

Heterogeneity in outcome reporting limits identification of gold-standard treatments for Hirschsprung’s Disease(HD) and gastroschisis. This review aimed to identify which outcomes are currently investigated in HD and gastroschisis research so as to counter this heterogeneity through informing development of a core outcome set(COS). Two systematic reviews were conducted. Studies were eligible for inclusion if they compared surgical interventions for primary treatment of HD in review one, and gastroschisis in review two. Studies available only as abstracts were excluded from analysis of reporting transparency. Thirty-five HD studies were eligible for inclusion in the review, and 74 unique outcomes were investigated. The most commonly investigated was faecal incontinence (32 studies, 91%). Seven of the 28 assessed studies (25%) met all criteria for transparent outcome reporting. Thirty gastroschisis studies were eligible for inclusion in the review, and 62 unique outcomes were investigated. The most commonly investigated was length of stay (24 studies, 80%). None of the assessed studies met all criteria for transparent outcome reporting. This review demonstrates that heterogeneity in outcome reporting and a significant risk of reporting bias exist in HD and gastroschisis research. Development of a COS could counter these problems, and the outcome lists developed from this review could be used in that process.

Systematic reviews comparing key treatments for both Hirschsprung's Disease and gastroschisis have failed to identify gold standard treatments for either condition [1][2][3][4] . This is likely to be due to a combination of reasons including the small size and retrospective nature of many of the primary studies, but also inadequate data reporting, heterogeneity in outcome definition, and use of surrogate markers of success instead of clinically relevant outcomes 1 . In a recent study reviewing 283 randomly selected Cochrane systematic reviews covering a range of conditions and interventions, Kirkham et al. demonstrated that over 30% of the randomised controlled trials included in these systematic reviews either did not publish at all, or did not fully publish the results for their identified primary outcomes 5 . This suggests that the potential for reporting bias is widespread within the medical literature.
Potential reporting bias and potential lack of patient relevance, combined with the fact that chance and confounding are likely to be impacting the results of many Hirschsprung's Disease and gastroschisis studies 6 limits the ability of the existing evidence base to help guide clinical management. It has been proposed that the development of core outcome sets for key conditions, combined with increased collaboration can help address these problems 1,7,8 .
Core outcome sets are groups of outcomes that have been identified through a systematic review and Delphi process, and ratified by key stakeholders as the outcomes that should at a minimum be reported in every study of that condition 8 . They are a tool that has been championed by the COMET (Core Outcome Measures in Effectiveness Trials) initiative as a method for reducing reporting bias, increasing patient relevance of research, and improving meta-analysis. Their use in other conditions has been shown to significantly improve the quality of research being conducted 9 , thereby enabling more evidence-based clinical practice.
Prior to developing a core outcome set it is essential to know which outcomes are currently reported in the published literature, and the quality of their reporting. The aims of this work were therefore to conduct two systematic reviews, in order to identify which outcomes are currently reported in studies comparing surgical Scientific RepoRts | 6:38969 | DOI: 10.1038/srep38969 Data Synthesis. The number of outcomes reported in each eligible study and the number of times each outcome was reported were counted, and the medians and interquartile ranges calculated. The number and percentage of studies answering yes to all five core questions proposed by Harman et al., and the number and percentage with full reporting according to ORBIT criteria for all investigated outcomes were also calculated.

Results
Hirschsprung's Disease included studies. The search retrieved 696 unique titles related to Hirschsprung's Disease, 640 of which were excluded following title and abstract review. Following full paper review, a further 21 papers were excluded with reasons (Supplementary File 3), leaving 35 studies as eligible for inclusion in the review (Fig. 1). Eighteen manuscripts (51%) were retrospective case series, nine (26%) were prospective cohort studies, three (9%) were case control studies, three (9%) were systematic reviews, one (3%) was a registry based study, and one (3%) was a randomised controlled trial. The median number of study participants was 62 (IQR44-156). Hirschsprung's Disease reported outcomes. In the 35 included studies, 95 outcomes were investigated a total of 337 times. Thirty-five outcomes were considered to be too similar to at least one other outcome to be meaningfully differentiated, and these outcomes were therefore mapped to one common term (e.g. continence/ incontinence, or frequency of stool/bowel movement frequency). Following this exercise, 74 unique outcomes were identified as having been reported (Table 1).
Overall, faecal incontinence was the most commonly reported outcome and, appearing in 32 studies (91%), the only one to be investigated in more than 75% of studies. Outcomes investigated in more than 50% of studies were enterocolitis (23 studies, 66%), constipation (20 studies, 57%) and length of stay (18 studies, 51%). Thirty-three (45%) of the 74 unique outcomes were only investigated in one study. The median number of outcomes investigated per study was 11 (IQR6-13). Due to the retrospective nature of many of the included studies, and the lack of clearly defined end-points within them, it was not possible to make any meaningful assessment of the ages at which key outcomes were measured. achieving complete, high quality reporting of a study's results. When assessed against the ORBIT criteria, of the 28 full papers, only seven (25%) fully reported each of the outcomes they set out to investigate ( Table 2).

Gastroschisis included Studies.
The search strategy returned 234 titles related to gastroschisis, reduced to 211 after exclusion of duplicates. Following review of titles and abstracts, 167 records were excluded. A further fourteen records were then excluded with reasons following full paper analysis (Supplementary File 4), resulting in 30 studies that were deemed eligible for inclusion in the review (Fig. 2). Twenty-two papers (73%) were retrospective case series, four were prospective cohort studies (13%), two were systematic reviews (7%), one was a case-control study (3%), and one was a registry study (3%). There were no eligible randomised controlled trials. The median number of study participants was 122 (IQR 53-285).

Gastroschisis outcomes.
Within the included studies, 102 outcomes were investigated a total of 247 times.
Within these 102 outcomes there were 63 that were felt to be too similar to at least one other outcome to be meaningfully differentiated, and these were therefore mapped to one common term. Following this mapping process, there remained 62 unique outcomes (Table 3).
Outcomes were mapped to the OMERACT filter 2.0. Overall, 22 'adverse event' outcomes (35%), 18 'pathophysiological manifestation' outcomes (29%), 12 'life impact' outcomes (20%), nine 'resource use' outcomes (15%), and one 'death' outcome (1%) were reported. ' Adverse event' outcomes were reported 97 times (39% of all reported outcomes), whilst 'life impact' outcomes were reported 58 times (23% of all reported outcomes).  Eight of the 62 identified outcomes related to feeding (13%), and five related to mechanical ventilation (8%). These two areas therefore appear to be of interest to researchers. However, the greatest number of studies in which any individual outcome from either of these areas was investigated was 13 (43%), suggesting there is little agreement on what the most important outcome in each area is. Overall, the most commonly investigated outcome was total length of stay, appearing in 24 studies (80%). Mortality (19 studies, 63%), and development of necrotising enterocolitis (16 studies, 53%) were the only other outcomes to be reported in more than 50% of studies. The median number of times an outcome was reported was once (IQR1-2). Thirty-one outcomes (50%) were only reported in one study. The median number of outcomes reported per study was 9 (IQR5-11).

Number of infants
Gastroschisis completeness of reporting. Of the 30 included studies, three were only available as abstracts, and therefore excluded from analysis of completeness and quality of reporting. Of the remaining 27 studies, none met all five of the core criteria defined by Harman et al. for complete, high quality outcome reporting. When assessed against the ORBIT criteria, 12 studies (44%) fully reported each of the outcomes they set out to investigate (Table 4).

Discussion
This review shows substantial heterogeneity of outcome reporting in Hirschsprung's Disease and gastroschisis studies. For each condition, over 60 unique outcomes were investigated, only 5% of which were investigated in more than half of the included studies, and over 40% of which were only investigated in one study. Within Hirschsprung's Disease research, 32% of investigated outcomes were in the 'life impact' core area of the OMERACT filter 2.0, suggesting that there is potential for current research to be investigating outcomes of relevance to patients. The lack of meaningful information on when outcomes are measured however, meant that it was not possible to determine whether these outcomes were investigated at time points where their results would be considered valid (e.g. whether faecal continence was measured in infants who were old enough to be expected to be continent). With 64% of gastroschisis outcomes fitting into either the 'pathophysiological manifestations' or 'adverse events' core areas, it would suggest that gastroschisis research tends to have a shorter-term focus. With no gastroschisis studies, and only 25% of Hirschsprung's Disease studies meeting all of the criteria for high quality, complete outcome reporting, and only 25% of Hirschsprung's Disease and 44% of gastroschisis studies fully reporting all outcome data, there also appears to be a high risk of reporting bias within both research fields.
The search strategy and inclusion criteria for this study were designed to be sensitive whilst maintaining contemporaneity of the identified outcomes. This should provide the most robust basis for development of a core outcome set. Excluding studies conducted prior to 2010 does however introduce the potential for these reviews to miss important outcomes that were fully investigated in high quality studies prior to this point. One example of this is the only randomised controlled trial carried out to date which compares operative primary fascial closure with silo placement for treatment of gastroschisis 12 . Such exclusions have the potential to alter the data on completeness of reporting. However, we were unable to identify any plausible reasons why the study designs used, or quality of reporting should be significantly different pre and post 2010. We therefore do not believe that these exclusions will have significantly altered the conclusions of this review. By including in the current analysis outcomes investigated by systematic reviews, outcomes reported by significant primary studies prior to 2010 (including that of Pastor et al.) should also have been captured, and will therefore be included in the list of outcomes used for the development of the core outcome sets.  In order to assess the quality and completeness of outcome reporting, we used tools that were initially designed for assessing randomised controlled trials and systematic reviews. However, it is not unreasonable to expect that any study making active comparison of two interventions meets all five of Harman et al.'s core criteria for high quality, transparent reporting, and that they fully report data for all stated a priori outcomes.
It has previously been suggested that variability in outcome definition and reporting is limiting the development of a high quality evidence in Hirschsprung's Disease and gastroschisis 1,2,4 . Our review reinforces that message, and adds to it by suggesting that there is also a high risk of reporting bias in research into both conditions, thereby further reducing the reliability of conclusions drawn from the current literature. This high risk of reporting bias echoes what was shown in a large selection of randomised controlled trials included in Cochrane systematic reviews 5 .
The focus of the included studies on pathophysiological manifestations and adverse events, and lack of clarity of time-points for measurement of life impact outcomes raises the possibility that patients and their parents were not involved in determining which outcomes should be investigated by researchers and clinicians. This apparent lack of knowledge as to which outcomes are important to patients and parents in determining the success of gastroschisis and Hirschsprung's Disease treatment may account in part for the heterogeneity of outcome reporting that our reviews have demonstrated. The difficulties this heterogeneity creates for supporting clinical practice are compounded by the high risk of reporting bias that this review has also demonstrated. Both of these factors suggest that at present, caution should be exercised when using the existing literature to argue in favour of a particular treatment for either condition. There are two concrete steps that can be taken to remedy this situation, firstly, development of core outcome sets for use in gastroschisis and Hirschsprung's Disease, and secondly, enhanced national and international collaborative research studies and trials. These measures would reduce reporting bias, ease meta-analysis, increase statistical power to answer clinically relevant questions, and improve the patient relevance of research [7][8][9] .
These systematic reviews provide the evidence that development and implementation of core outcome sets are required for Hirschsprung's Disease and gastroschisis. The outcomes identified by these reviews could be used as the starting point for a robust Delphi processes involving patients, parents and multi-disciplinary clinical groups  to develop such core outcome sets. These will be the first core outcome sets to be developed for any paediatric surgical condition, and we do not wish them to be developed simply as an academic exercise. Hirschsprung's Disease and gastroschisis were specifically chosen as the first conditions in which to develop paediatric surgical core outcome sets in order to limit the possibility of this occurring. We believe they limit this risk for two reasons. Firstly, both conditions are actively studied at present, and therefore, have a community of researchers who are anticipating the use of the developed core outcome sets. Secondly, there have been contemporaneous, UK-wide cohorts of infants established for both conditions, in whom long-term follow-up studies could be conducted utilising the developed core outcome sets. We have elected to present the results of both systematic reviews as one manuscript, as there are several common themes which lend themselves to being demonstrated best through reporting in a single article. Doing this illustrates to the paediatric surgical community the need for development and implementation of core outcome sets in both of these conditions, and potentially, in others as well.