Time to evaluate COVID-19 contact-tracing apps

To the Editor—Digital contact tracing is a public-health intervention. Real-time monitoring and evaluation of the effectiveness of app-based contact tracing is key for improvement and public trust.

SARS-CoV-2 is likely to become endemic in many parts of the world, and there is still no certainty about how quickly vaccination will become available or how long its protection will last. For the foreseeable future, most countries will rely on a combination of various measures, including vaccination, social distancing, mask wearing and contact tracing.

Digital contact tracing via smartphone apps was established as a new public-health intervention in many countries in 2020. Most of these apps are now at a stage at which they need to be evaluated as public-health tools. We present here five key epidemiological and public-health requirements for COVID-19 contact-tracing apps and their evaluation.

1. Integration with local health policy. App notifications should be consistent with local health policies. The app should be integrated into access to testing, medical care and advice on isolation, and should work in conjunction with conventional contact tracing where available1. Apps should be interoperable across countries, as envisaged by the European Commission’s eHealth Network.

2. High user uptake and adherence. Contact-tracing apps can reduce transmission at low levels of uptake, including for those without smartphones2. However, large numbers of users increase effectiveness3,4. An effective communication strategy that explains the apps’ role and addresses privacy concerns is essential for increasing adoption5. Design, implementation and deployment should make the apps accessible to harder-to-reach communities. Adherence to quarantine should be encouraged and supported.

3. Quarantine infectious people as accurately as possible. The purpose of contact tracing is to quarantine as many potentially infectious people as possible, but to minimize the time spent in quarantine by uninfected people. To achieve optimal performance, apps’ algorithms must be ‘tunable’, to adjust to the epidemic as it evolves6.

4. Rapid notification. The time between the onset of symptoms in an index case and the quarantine of their contacts is of key importance in COVID-19 contact tracing7,8. Where a design feature introduces a delay, it needs to be outweighed by gains in, for example, specificity, uptake or adherence. If the delays exceed the period during which most contacts transmit the disease, the app will fail to reduce transmission.

5. Ability to evaluate effectiveness transparently. The public must be provided with evidence that notifications are based on the best available data. The tracing algorithm should therefore be transparent, auditable, under oversight and subject to review. Aggregated data (not linked to individual people) are essential for evaluation of and improvement in the performance of the app. Data on local uptake at a sufficiently coarse-grained spatial resolution are equally key. As apps in Europe do not ‘geolocate’ people, this additional information can be provided by the user or through surveys. Real-time monitoring should be performed whenever possible.

A proof-of-principle evaluation is available for the Swiss app9. More detailed analysis on the epidemiological effectiveness of contact-tracing apps is needed. For example, index cases seeking healthcare could be asked if they routinely use the app; if so, interviews with contacts identified by traditional tracing would allow assessment of the secondary attack rate among people who were notified. Surveys, epidemiological analyses10 and experimental studies such as the Radar COVID pilot in Spain can offer further evaluation.

Digital contact tracing is a sustainable measure that can reduce levels of COVID-19 transmission. A rigorous assessment of its effectiveness allows public-health benefits to be weighed against unwanted effects for individual people and society. Stringent evaluation is needed to develop contact-tracing apps into an accepted and ethical tool for future outbreaks of other infectious diseases.


  1. 1.

    Kucharski, A. J. et al. Lancet Infect. Dis. 20, 1151–1160 (2020).

    CAS  Article  Google Scholar 

  2. 2.

    Abueg, M et al. Preprint at https://doi.org/10.1101/2020.08.29.20184135 (2020).

  3. 3.

    Cencetti, G. et al. Preprint at https://www.medrxiv.org/content/10.1101/2020.05.29.20115915v2 (2020).

  4. 4.

    Moreno López, J.A. et al. Preprint at https://www.medrxiv.org/content/10.1101/2020.07.22.20158352v1 (2020).

  5. 5.

    Montagni, i., Roussel, N., Thiébaut, R. & Tzourio, C. Preprint at https://www.medrxiv.org/content/10.1101/2020.10.23.20218214v2 (2020).

  6. 6.

    Wilson, A.M. et al. Preprint at https://doi.org/10.1101/2020.07.17.20156539 (2020).

  7. 7.

    Ferretti, L. et al. Science 368, eabb6936 (2020).

    CAS  Article  Google Scholar 

  8. 8.

    Kretzschmar, M. E. et al. Lancet Public Health 5, e452–e459 (2020).

    Article  Google Scholar 

  9. 9.

    Salathé, M. et al. Swiss Med. Wkly. 150, w20457 (2020).

    PubMed  Google Scholar 

  10. 10.

    Kendall, M. et al. Lancet Digit. Health 2, e658–e666 (2020).

    Article  Google Scholar 

Download references

Author information



Corresponding authors

Correspondence to Luca Ferretti or Christophe Fraser.

Ethics declarations

Competing interests

The authors declare no competing interests.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Colizza, V., Grill, E., Mikolajczyk, R. et al. Time to evaluate COVID-19 contact-tracing apps. Nat Med (2021). https://doi.org/10.1038/s41591-021-01236-6

Download citation


Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing