Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • Article
  • Published:

Mechano-acoustic sensing of physiological processes and body motions via a soft wireless device placed at the suprasternal notch

Abstract

Skin-mounted soft electronics that incorporate high-bandwidth triaxial accelerometers can capture broad classes of physiologically relevant information, including mechano-acoustic signatures of underlying body processes (such as those measured by a stethoscope) and precision kinematics of core-body motions. Here, we describe a wireless device designed to be conformally placed on the suprasternal notch for the continuous measurement of mechano-acoustic signals, from subtle vibrations of the skin at accelerations of around 10−3 m s−2 to large motions of the entire body at about 10 m s−2, and at frequencies up to around 800 Hz. Because the measurements are a complex superposition of signals that arise from locomotion, body orientation, swallowing, respiration, cardiac activity, vocal-fold vibrations and other sources, we exploited frequency-domain analysis and machine learning to obtain—from human subjects during natural daily activities and exercise—real-time recordings of heart rate, respiration rate, energy intensity and other essential vital signs, as well as talking time and cadence, swallow counts and patterns, and other unconventional biomarkers. We also used the device in sleep laboratories and validated the measurements using polysomnography.

This is a preview of subscription content, access via your institution

Access options

Buy this article

Prices may be subject to local taxes which are calculated during checkout

Fig. 1: A wireless, skin-interfaced MA measurement technology designed for mounting on the SN.
Fig. 2: Representative MA data in the form of accelerations measured along three orthogonal axes from the MA device mounted on the SN of a healthy subject.
Fig. 3: Flow diagram of signal processing and corresponding results from representative MA data acquired from healthy subjects.
Fig. 4: Results from MA data recorded at the SN in field studies with comparisons to reference measurements.
Fig. 5: Application of MA sensing from the SN in clinical sleep studies.
Fig. 6: Insights into sleep patterns determined by MA sensing from the SN.

Similar content being viewed by others

Data availability

The main data supporting the results in this study are available within the paper and its Supplementary Information. The raw and analysed datasets generated for the studies in Figs. 26 are available for research purposes from the corresponding authors upon reasonable request.

Code availability

The codes used for the embedded system and data collection are available on GitHub at https://github.com/johnrogersgroup/Wireless_MA. The analysis codes used in this study are available from the authors upon request.

References

  1. Liu, Y. et al. Epidermal mechano-acoustic sensing electronics for cardiovascular diagnostics and human-machine interfaces. Sci. Adv. 2, e1601185 (2016).

    PubMed  PubMed Central  Google Scholar 

  2. Kaniusas, E. in Biomedical Signals and Sensors II: Linking Acoustic and Optic Biosignals and Biomedical Sensors (Springer, 2015).

  3. Hu, Y., Kim, E. G., Cao, G., Liu, S. & Xu, Y. Physiological acoustic sensing based on accelerometers: a survey for mobile healthcare. Ann. Biomed. Eng. 42, 2264–2277 (2014).

    PubMed  Google Scholar 

  4. Vavrinský, E. et al. Application of acceleration sensors in physiological experiments. J. Electr. Eng. 65, 304–308 (2014).

    Google Scholar 

  5. Makarenkova, A., Poreva, A. & Slozko, M. Efficiency evaluation of electroacoustic sensors for auscultation devices of human body life-activity sounds. In Proc. IEEE 1st Ukraine Conference on Electrical and Computer Engineering (IEEE, 2017).

  6. Dudik, J. M., Coyle, J. L. & Sejdic, E. Dysphagia screening: contributions of cervical auscultation signals and modern signal-processing. Tech. IEEE Trans. Hum. Mach. Syst. 45, 465–477 (2015).

    Google Scholar 

  7. Kok, M., Hol, J. D. & Schön, T. B. Using inertial sensors for position and orientation estimation. Found. Trends Signal Process. 11, 1–153 (2017).

    Google Scholar 

  8. Makaryus, A. N., Swarup, S. & Makaryus, A. Digital stethoscope: technology update. Med. Devices 11, 29–36 (2018).

    Google Scholar 

  9. Brond, J. C. & Arvidsson, D. Sampling frequency affects the processing of Actigraph raw acceleration data to activity counts. J. Appl. Physiol. 120, 362–369 (2016).

    PubMed  Google Scholar 

  10. Di Rienzo, M. et al. A wearable system for the seismocardiogram assessment in daily life conditions. In Proc. Annual International Conference of the IEEE Engineering in Medicine and Biology Society (IEEE, 2011).

  11. Jafari Tadi, M. et al. Gyrocardiography: A new non-invasive monitoring method for the assessment of cardiac mechanics and the estimation of hemodynamic variables. Sci. Rep. 7, 1–11 (2017).

    CAS  Google Scholar 

  12. Inan, O. T. et al. Novel wearable seismocardiography and machine learning algorithms can assess clinical status of heart failure patients. Circ. Heart Fail. 11, e004313 (2018).

    PubMed  PubMed Central  Google Scholar 

  13. Shandhi, M. M. H.et al. Performance analysis of gyroscope and accelerometer sensors for seismocardiography-based wearable pre-ejection period estimation. IEEE J. Biomed. Health 23, 2365–2374 (2019).

    PubMed  Google Scholar 

  14. Hernandez, J., McDuff, D., Quigley, K. S., Maes, P. & Picard, R. W. Wearable motion-based heart-rate at rest: a workplace evaluation. IEEE J. Biomed. Health 23, 1920–1927 (2019).

  15. Hung, P. D., Bonnet, S., Guillemaud, R., Castelli, E. & Yen, P. T. N. Estimation of respiratory waveform using an accelerometer. In Proc. 5th IEEE International Symposium on Biomedical Imaging: From Nano to Macro (IEEE, 2008).

  16. Bates, A., Ling, M. J., Mann, J. & Arvind, D. K. Respiratory rate and flow waveform estimation from tri-axial accelerometer data. In Proc. 2010 International Conference on Body Sensor Networks (IEEE, 2010).

  17. Liu, G. Z., Guo, Y. W., Zhu, Q. S., Huang, B. Y. & Wang, L. Estimation of respiration rate from three-dimensional acceleration data based on body sensor network. Telemed. J. E. Health 17, 705–711 (2011).

    PubMed  PubMed Central  Google Scholar 

  18. Lapi, S. et al. Respiratory rate assessments using a dual-accelerometer device. Respir. Physiol. Neurobiol. 191, 60–66 (2014).

    PubMed  Google Scholar 

  19. Tadi, M. J. et al. A miniaturized MEMS motion processing system for nuclear medicine imaging applications. Comput. Cardiol. 43, 133–136 (2016).

    Google Scholar 

  20. Preejith, S. P., Jeelani, A., Maniyar, P., Joseph, J. & Sivaprakasam, M. Accelerometer based system for continuous respiratory rate monitoring. In Proc. IEEE International Symposium on Medical Measurements and Applications (IEEE, 2017).

  21. Pompilio, P. P., Sgura, A., Pedotti, A. & Dellaca, R. A MEMS accelerometers based system for the measurement of lung sound delays. In Proc. 5th Cairo International Biomedical Engineering Conference (IEEE, 2010).

  22. Lee, J., Steele, C. M. & Chau, T. Time and time-frequency characterization of dual-axis swallowing accelerometry signals. Physiol. Meas. 29, 1105–1120 (2008).

    CAS  PubMed  Google Scholar 

  23. Damouras, S., Sejdić, E., Steele, C. M. & Chau, T. An online swallow detection algorithm based on the quadratic variation of dual-axis accelerometry. IEEE Trans. Signal Process. 58, 3352–3359 (2010).

    Google Scholar 

  24. Dudik, J. M., Jestrović, I., Luan, B., Coyle, J. L. & Sejdić, E. A comparative analysis of swallowing accelerometry and sounds during saliva swallows. Biomed. Eng. Online 14, 3 (2015).

    PubMed  PubMed Central  Google Scholar 

  25. Kumari, S. K. & Mathana, J. M. Blood sugar level indication through chewing and swallowing from acoustic MEMS sensor and deep learning algorithm for diabetic management. J. Med. Syst. 43, 1 (2018).

    PubMed  Google Scholar 

  26. Mehta, D. D., Zañartu, M., Feng, S. W., Cheyne, H. A. I. & Hillman, R. E. Mobile voice health monitoring using a wearable accelerometer sensor and a smartphone platform. IEEE Trans. Biomed. Eng. 59, 3090–3096 (2012).

    PubMed  PubMed Central  Google Scholar 

  27. Michalevsky, Y., Boneh, D. & Nakibly, G. Gyrophone: recognizing speech from gyroscope signals. In Proc. 23rd USENIX Security Symposium (USENIX Association, 2014).

  28. Nyan, M. N., Tay, F. E. H., Manimaran, M. & Seah, K. H. W. Garment-based detection of falls and activities of daily living using 3-axis MEMS accelerometer. J. Phys. Conf. Ser. 34, 1059–1067 (2006).

    Google Scholar 

  29. Curone, D., Bertolotti, G. M., Cristiani, A., Secco, E. L. & Magenes, G. A real-time and self-calibrating algorithm based on triaxial accelerometer signals for the detection of human posture and activity. IEEE Trans. Inf. Technol. Biomed. 14, 1098–1105 (2010).

    PubMed  Google Scholar 

  30. Yang, C. C. & Hsu, Y. L. A review of accelerometry-based wearable motion detectors for physical activity monitoring. Sensors 10, 7772–7788 (2010).

    PubMed  Google Scholar 

  31. Posatskiy, A. O. & Chau, T. The effects of motion artifact on mechanomyography: A comparative study of microphones and accelerometers. J. Electromyogr. Kinesiol. 22, 320–324 (2012).

  32. Maki, H., Ogawa, H., Matsuoka, S., Yonezawa, Y. & Caldwell, W. M. A daily living activity remote monitoring system for solitary elderly people. Proc. Annu. Int. Conf. IEEE Eng. Med. Biol. Soc. EMBS 2011, 5608–5611 (2011).

    Google Scholar 

  33. Zheng, Y. L. et al. Unobtrusive sensing and wearable devices for health informatics. IEEE Trans. Biomed. Eng. 61, 1538–1554 (2014).

    PubMed  Google Scholar 

  34. Phan, D. H., Bonnet, S., Guillemaud, R., Castelli, E. & Thi, N. Y. P. Estimation of respiratory waveform and heart rate using an accelerometer. In Proc. 30th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (IEEE, 2008).

  35. Vertens, J. et al. Measuring respiration and heart rate using two acceleration sensors on a fully embedded platform. In Proc. 3rd International Congress on Sport Sciences Research and Technology Support (Scitepress, 2015).

  36. Sánchez Morillo, D., Ojeda, J. L. R., Foix, L. F. C. & Jiménez, A. L. An accelerometer-based device for sleep apnea screening. IEEE Trans. Inf. Technol. Biomed. 14, 491–499 (2010).

    Google Scholar 

  37. He, D. Da, Winokur, E. S. & Sodini, C. G. An ear-worn continuous ballistocardiogram (BCG) sensor for cardiovascular monitoring. Conf. Proc. IEEE Eng. Med. Biol. Soc. 2012, 5030–5033 (2012).

    PubMed  PubMed Central  Google Scholar 

  38. Rahman, T. et al. BodyBeat: a mobile system for sensing non-speech body sounds. In Proc. 12th Annual International Conference on Mobile Systems, Applications, and Services (ACM, 2014).

  39. Kim, D. H. et al. Epidermal electronics. Science 333, 838–843 (2011).

    CAS  Google Scholar 

  40. Jang, K. I. et al. Soft network composite materials with deterministic and bio-inspired designs. Nat. Commun. 6, 6566 (2015).

    CAS  PubMed  PubMed Central  Google Scholar 

  41. Fan, J. A. et al. Fractal design concepts for stretchable electronics. Nat. Commun. 5, 3266 (2014).

    PubMed  Google Scholar 

  42. Kim, D. H. et al. Electronic sensor and actuator webs for large-area complex geometry cardiac mapping and therapy. Proc. Natl Acad. Sci. USA 109, 19910–19915 (2012).

    CAS  PubMed  Google Scholar 

  43. Kim, D. H. et al. Materials and noncoplanar mesh designs for integrated circuits with linear elastic responses to extreme mechanical deformations. Proc. Natl Acad. Sci. USA 105, 18675–18680 (2008).

    CAS  PubMed  Google Scholar 

  44. Muroga, T., Ito, Y., Aoyagi, K., Yamamoto, Y. & Yokomizo, K. Rolled copper foil. US patent 20090017325A1 (2009).

  45. Titze, I. R. Principles of Voice Production (Prentice Hall, 1994).

  46. Baken, R. J. & Orlikoff, R. F. Clinical measurement of speech and voice (Cengage Learning, 1999).

  47. Wu, K. Gender recognition from speech. Part II: fine analysis. J. Acoust. Soc. Am. 90, 1841–1856 (1991).

    PubMed  Google Scholar 

  48. Lin, S. J. et al. A pilot study on BSN-based ubiquitous energy expenditure monitoring. In Proc. 6th International Workshop on Wearable and Implantable Body Sensor Networks (IEEE, 2009).

  49. Jin, A., Yin, B., Morren, G., Duric, H. & Aarts, R. M. Performance evaluation of a tri-axial accelerometry-based respiration monitoring for ambient assisted living. In Proc. 31st Annual International Conference of the IEEE Engineering in Medicine and Biology Society: Engineering the Future of Biomedicine (IEEE, 2009).

  50. Dash, S., Shelley, K. H., Silverman, D. G. & Chon, K. H. Estimation of respiratory rate from ECG, photoplethysmogram, and piezoelectric pulse transducer signals: a comparative study of time-frequency methods. IEEE Trans. Biomed. Eng. 57, 1099–1107 (2010).

    PubMed  Google Scholar 

  51. Chon, K. H., Dash, S. & Ju, K. Estimation of respiratory rate from photoplethysmogram data using time-frequency spectral estimation. IEEE Trans. Biomed. Eng. 56, 2054–2063 (2009).

    PubMed  Google Scholar 

  52. Berry, R. B. et al. AASM scoring manual updates for 2017 (version 2.4). J. Clin. Sleep Med. 13, 665–666 (2017).

    PubMed  PubMed Central  Google Scholar 

  53. Watanabe, N., Reece, J. & Polus, B. I. Effects of body position on autonomic regulation of cardiovascular function in young, healthy adults. Chiropr. Osteopat. 15, 19 (2007).

    PubMed  PubMed Central  Google Scholar 

  54. Toyota, S. & Amaki, Y. Hemodynamic evaluation of the prone position by transesophageal echocardiography. J. Clin. Anesth. 10, 32–35 (1998).

    CAS  PubMed  Google Scholar 

  55. Pump, B., Talleruphuus, U., Christensen, N. J., Warberg, J. & Norsk, P. Effects of supine, prone, and lateral positions on cardiovascular and renal variables in humans. Am. J. Physiol. 283, R174–R180 (2002).

    CAS  Google Scholar 

  56. Issa, F. G. & Sullivan, C. E. Upper airway closing pressures in snorers. J. Appl. Physiol. 57, 528–535 (1984).

    CAS  PubMed  Google Scholar 

  57. Aurégan, Y. & Depollier, C. Snoring: linear stability analysis and in-vitro experiments. J. Sound Vib. 188, 39–53 (1995).

    Google Scholar 

  58. Fajdiga, I. Snoring imaging: could Bernoulli explain it all? Chest 128, 896–901 (2005).

    PubMed  Google Scholar 

  59. Javaid, A. Q. et al. Quantifying and reducing motion artifacts in wearable seismocardiogram measurements during walking to assess left ventricular health. IEEE Trans. Biomed. Eng. 64, 1277–1286 (2017).

    PubMed  Google Scholar 

  60. Schwindt, D. A., Wilhelm, K. P., Miller, D. L. & Maibach, H. I. Cumulative irritation in older and younger skin: a comparison. Acta Derm. Venereol. 78, 279–283 (1998).

    CAS  PubMed  Google Scholar 

Download references

Acknowledgements

The materials and device-engineering aspects of the research were supported by the Material Science and Engineering Department and Center for Bio-Integrated Electronics at Northwestern University. K.L. acknowledges support from the Samsung Scholarship. K.L. acknowledges the help from A. Sahakian on radio frequency communication and antenna tuning. Z.X. acknowledges support from the National Natural Science Foundation of China (grant no.11402134). S.X. and J.A.R. recognizes support from the National Institute on Aging of the National Institutes of Health under R41AG062023 and R43AG060812. R.A. acknowledges support from National Science Foundation (NSF) Graduate Research Fellowship under grant no. 1842165 and the Ford Foundation Predoctoral Fellowship. Y.H. acknowledges support from the NSF (CMMI1635443). S.M. is grateful to Indo-U.S. Science and Technology Forum (SERB-IUSSTF) for her SERB Indo-U.S. Postdoctoral Award. X.N. thanks D. Lu for helpful discussions. K.L. thanks the Dion family for volunteering for the data collection.

Author information

Authors and Affiliations

Authors

Contributions

K.L., Z.X. and J.A.R. performed the structural design of the system. Z.X., R.A., Y.D. and Y.H. performed mechanical and electromagnetic modelling and theoretical studies. K.L., J.Y.L., J.H.L., J.B.P. and J.K. developed the embedded system and the user interface. K.L., X.N. and J.A.R. designed and performed the experimental studies on the technology. X.N., K.L. and J.A.R. designed and performed the human subject studies. X.N., K.L., M.I., R.L.E., D.J.P. and D.H.K developed the signal-processing algorithms and performed the data analysis. K.L., H.A., D.J.P., H.U.C., O.O.O., S.G., E.C., M.H., J.B., H.J., C.L., S.B.K., S.M., J.T.R and I.H. manufactured the devices. S.X., A.T. and C.R.D. provided clinical advice. X.N. and J.A.R. wrote the signal processing algorithm part of the manuscript. K.L., X.N., Z.X., Y.H. and J.A.R. contributed to the other sections.

Corresponding authors

Correspondence to Zhaoqian Xie, Charles R. Davies, Yonggang Huang or John A. Rogers.

Ethics declarations

Competing interests

The authors declare no competing interests.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Supplementary Information

Supplementary Methods, Supplementary References, Supplementary Figures and Supplementary Tables.

Reporting Summary

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Lee, K., Ni, X., Lee, J.Y. et al. Mechano-acoustic sensing of physiological processes and body motions via a soft wireless device placed at the suprasternal notch. Nat Biomed Eng 4, 148–158 (2020). https://doi.org/10.1038/s41551-019-0480-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/s41551-019-0480-6

This article is cited by

Search

Quick links

Nature Briefing: Translational Research

Sign up for the Nature Briefing: Translational Research newsletter — top stories in biotechnology, drug discovery and pharma.

Get what matters in translational research, free to your inbox weekly. Sign up for Nature Briefing: Translational Research