NICE updates real-world evidence framework to address new approaches in external validity and transportability of real-world data

NICE has updated its real-world evidence framework to incorporate new methodologies for enhancing the external validity and transportability of real-world data. This update aims to ensure the framework reflects the latest advancements in real-world research.

Published in June 2022, the National Institute for Health and Care Excellence (NICE) real-world evidence (RWE) framework is a comprehensive guideline designed to improve the quality, transparency, and applicability of real-world data (RWD) in healthcare decision-making carried out by the institute. The framework was developed to define best practices for designing and reporting RWE studies, encouraging rigorous standards and consistent appraisal, and providing detailed guidance on identifying suitable data and conducting studies without being overly prescriptive. A dynamic framework, NICE is ‘taking a living approach to the framework to ensure that it remains useful, usable, and up to date.’

The new updates address transportability of data and offer guidance on assessing and adjusting for external validity bias when utilizing international data. As Stephen Duffield from NICE notes on LinkedIn, NICE already considers international RWD in certain situation but, “can we make better assessments of its suitability and to what extent can methods help us to adjust for population differences?”

To address these questions, the framework contains several updates including:

Page 28 – new section ‘External validity bias’, which explains how NICE defines external validity in terms of the generalizability or transportability of study findings. This involves ensuring that the analytical sample accurately represents the target population, considering factors such as patient characteristics, healthcare settings, and potential biases due to exclusions, dropouts, and missing data.

Page 36 – information on sampling methods, discussing the various methods available to mitigate for selection bias for sampling in new studies requiring primary data collection.

Page 75 – new section ‘Addressing external validity’, which discusses methodologies aimed at evaluating and addressing external validity bias that may arise from differences in patient attributes (such as age and disease risk scores) between the analytical sample and the intended population.