Abstract#

Note due to the Journal of Simulation house style and word limit for an abstract we were unable to include all information. For completeness we include a PRISMA style abstract in the online material and the full checklist below.

2. PRISMA 2020 abstract checklist#

2.1 Identify the report as a systematic review#

COMPLETE

See title

2.2 Provide an explicit statement of the main objective(s) or question(s) the review addresses.#

COMPLETE

This systematic scoping review investigates to what extent authors share computer models, and audits if sharing adheres to best practice.

2.3 Specify the inclusion and exclusion criteria for the review.#

COMPLETE

Cost-effectiveness, Health service research and methodology studies in a health context were included.

2.4 Specify the information sources (e.g. databases, registers) used to identify studies and the date when each was last searched.#

COMPLETE

The Web of Science, Scopus, PubMed, and ACM Digital Library databases were searched between 1st January 2019 till 31st December 2022.

2.5 Specify the methods used to assess risk of bias in the included studies.#

NOT APPLICABLE

This study does not estimate an effect.

2.6 Specify the methods used to present and synthesise results.#

COMPLETE

We measured the proportion of literature that shared models; we report analyses by publication type, year of publication, Covid-19 application; and free and open source versus commercial software.

2.7 Give the total number of included studies and participants and summarise relevant characteristics of studies.#

COMPLETE

47 (8.3%) of the 564 studies included cited a published DES computer model; rising to 9.0% in 2022.

2.8 Present results for main outcomes#

COMPLETE

Studies were more likely to share models if they had been developed using free and open source tools. Studies rarely followed best practice when sharing computer models.

2.9 Provide a brief summary of the limitations of the evidence included in the review#

COMPLETE

Our study cannot make any definitive statements about the reproducibility of the studies that shared their code versus those that used a reporting guideline versus those that did neither. Our aim instead was to focus on the practice of sharing and the current deficiencies as defined by gold standard guides such as the Turing Way and Open Modelling Initiative.

Our findings are based on information we found in the publication. We recognize that the model code may have been published online but not mentioned in the article. We feel our approach is the most appropriate as articles are the primary means by which researchers find research studies and their artifacts.

2.10 Provide a general interpretation of the results and important implications.#

COMPLETE

Although still in the minority, there is evidence that healthcare DES authors are increasingly sharing their computer model artifacts. Although commercial software dominates the DES literature, free and open source software plays a crucial role in sharing. The DES community can adopt many simple best practices to improve the quality of sharing.

2.11 Specify the primary source of funding for the review.#

COMPLETE

Unfunded.

2.12 Provide the register name and registration number.#

NOT APPLICABLE

We did not pre-register this study as it was not estimating an effect.