Objective All electronic health (e-health) interventions require validation as health information technologies, ideally in randomized controlled trial settings. However, as with other types of complex interventions involving various active components and multiple targets, health informatics trials often experience problems of design, methodology, or analysis that can influence the results and acceptance of the research. Our objective was to review selected key methodologic issues in conducting and reporting randomized controlled trials in health informatics, provide examples from a recent study, and present practical recommendations.

Design For illustration, we use the COMPETE III study, a large randomized controlled clinical trial investigating the impact of a shared decision-support system on the quality of vascular disease management in Ontario, Canada.

Results We describe a set of methodologic, logistic, and statistical issues that should be considered when planning and implementing trials of complex e-health interventions, and provide practical recommendations for health informatics trialists.

Conclusions Our recommendations emphasize validity and pragmatic considerations and would be useful for health informaticians conducting or evaluating e-health studies.