Mechano-acoustic sensing of physiological processes and body motions via a soft wireless device placed at the suprasternal notch

Abstract

Skin-mounted soft electronics that incorporate high-bandwidth triaxial accelerometers can capture broad classes of physiologically relevant information, including mechano-acoustic signatures of underlying body processes (such as those measured by a stethoscope) and precision kinematics of core-body motions. Here, we describe a wireless device designed to be conformally placed on the suprasternal notch for the continuous measurement of mechano-acoustic signals, from subtle vibrations of the skin at accelerations of around 10−3 m s−2 to large motions of the entire body at about 10 m s−2, and at frequencies up to around 800 Hz. Because the measurements are a complex superposition of signals that arise from locomotion, body orientation, swallowing, respiration, cardiac activity, vocal-fold vibrations and other sources, we exploited frequency-domain analysis and machine learning to obtain—from human subjects during natural daily activities and exercise—real-time recordings of heart rate, respiration rate, energy intensity and other essential vital signs, as well as talking time and cadence, swallow counts and patterns, and other unconventional biomarkers. We also used the device in sleep laboratories and validated the measurements using polysomnography.

Access options

Rent or Buy article

Get time limited or full article access on ReadCube.

from$8.99

All prices are NET prices.

Fig. 1: A wireless, skin-interfaced MA measurement technology designed for mounting on the SN.
Fig. 2: Representative MA data in the form of accelerations measured along three orthogonal axes from the MA device mounted on the SN of a healthy subject.
Fig. 3: Flow diagram of signal processing and corresponding results from representative MA data acquired from healthy subjects.
Fig. 4: Results from MA data recorded at the SN in field studies with comparisons to reference measurements.
Fig. 5: Application of MA sensing from the SN in clinical sleep studies.
Fig. 6: Insights into sleep patterns determined by MA sensing from the SN.

Data availability

The main data supporting the results in this study are available within the paper and its Supplementary Information. The raw and analysed datasets generated for the studies in Figs. 26 are available for research purposes from the corresponding authors upon reasonable request.

Code availability

The codes used for the embedded system and data collection are available on GitHub at https://github.com/johnrogersgroup/Wireless_MA. The analysis codes used in this study are available from the authors upon request.

References

  1. 1.

    Liu, Y. et al. Epidermal mechano-acoustic sensing electronics for cardiovascular diagnostics and human-machine interfaces. Sci. Adv. 2, e1601185 (2016).

  2. 2.

    Kaniusas, E. in Biomedical Signals and Sensors II: Linking Acoustic and Optic Biosignals and Biomedical Sensors (Springer, 2015).

  3. 3.

    Hu, Y., Kim, E. G., Cao, G., Liu, S. & Xu, Y. Physiological acoustic sensing based on accelerometers: a survey for mobile healthcare. Ann. Biomed. Eng. 42, 2264–2277 (2014).

  4. 4.

    Vavrinský, E. et al. Application of acceleration sensors in physiological experiments. J. Electr. Eng. 65, 304–308 (2014).

  5. 5.

    Makarenkova, A., Poreva, A. & Slozko, M. Efficiency evaluation of electroacoustic sensors for auscultation devices of human body life-activity sounds. In Proc. IEEE 1st Ukraine Conference on Electrical and Computer Engineering (IEEE, 2017).

  6. 6.

    Dudik, J. M., Coyle, J. L. & Sejdic, E. Dysphagia screening: contributions of cervical auscultation signals and modern signal-processing. Tech. IEEE Trans. Hum. Mach. Syst. 45, 465–477 (2015).

  7. 7.

    Kok, M., Hol, J. D. & Schön, T. B. Using inertial sensors for position and orientation estimation. Found. Trends Signal Process. 11, 1–153 (2017).

  8. 8.

    Makaryus, A. N., Swarup, S. & Makaryus, A. Digital stethoscope: technology update. Med. Devices 11, 29–36 (2018).

  9. 9.

    Brond, J. C. & Arvidsson, D. Sampling frequency affects the processing of Actigraph raw acceleration data to activity counts. J. Appl. Physiol. 120, 362–369 (2016).

  10. 10.

    Di Rienzo, M. et al. A wearable system for the seismocardiogram assessment in daily life conditions. In Proc. Annual International Conference of the IEEE Engineering in Medicine and Biology Society (IEEE, 2011).

  11. 11.

    Jafari Tadi, M. et al. Gyrocardiography: A new non-invasive monitoring method for the assessment of cardiac mechanics and the estimation of hemodynamic variables. Sci. Rep. 7, 1–11 (2017).

  12. 12.

    Inan, O. T. et al. Novel wearable seismocardiography and machine learning algorithms can assess clinical status of heart failure patients. Circ. Heart Fail. 11, e004313 (2018).

  13. 13.

    Shandhi, M. M. H.et al. Performance analysis of gyroscope and accelerometer sensors for seismocardiography-based wearable pre-ejection period estimation. IEEE J. Biomed. Health 23, 2365–2374 (2019).

  14. 14.

    Hernandez, J., McDuff, D., Quigley, K. S., Maes, P. & Picard, R. W. Wearable motion-based heart-rate at rest: a workplace evaluation. IEEE J. Biomed. Health 23, 1920–1927 (2019).

  15. 15.

    Hung, P. D., Bonnet, S., Guillemaud, R., Castelli, E. & Yen, P. T. N. Estimation of respiratory waveform using an accelerometer. In Proc. 5th IEEE International Symposium on Biomedical Imaging: From Nano to Macro (IEEE, 2008).

  16. 16.

    Bates, A., Ling, M. J., Mann, J. & Arvind, D. K. Respiratory rate and flow waveform estimation from tri-axial accelerometer data. In Proc. 2010 International Conference on Body Sensor Networks (IEEE, 2010).

  17. 17.

    Liu, G. Z., Guo, Y. W., Zhu, Q. S., Huang, B. Y. & Wang, L. Estimation of respiration rate from three-dimensional acceleration data based on body sensor network. Telemed. J. E. Health 17, 705–711 (2011).

  18. 18.

    Lapi, S. et al. Respiratory rate assessments using a dual-accelerometer device. Respir. Physiol. Neurobiol. 191, 60–66 (2014).

  19. 19.

    Tadi, M. J. et al. A miniaturized MEMS motion processing system for nuclear medicine imaging applications. Comput. Cardiol. 43, 133–136 (2016).

  20. 20.

    Preejith, S. P., Jeelani, A., Maniyar, P., Joseph, J. & Sivaprakasam, M. Accelerometer based system for continuous respiratory rate monitoring. In Proc. IEEE International Symposium on Medical Measurements and Applications (IEEE, 2017).

  21. 21.

    Pompilio, P. P., Sgura, A., Pedotti, A. & Dellaca, R. A MEMS accelerometers based system for the measurement of lung sound delays. In Proc. 5th Cairo International Biomedical Engineering Conference (IEEE, 2010).

  22. 22.

    Lee, J., Steele, C. M. & Chau, T. Time and time-frequency characterization of dual-axis swallowing accelerometry signals. Physiol. Meas. 29, 1105–1120 (2008).

  23. 23.

    Damouras, S., Sejdić, E., Steele, C. M. & Chau, T. An online swallow detection algorithm based on the quadratic variation of dual-axis accelerometry. IEEE Trans. Signal Process. 58, 3352–3359 (2010).

  24. 24.

    Dudik, J. M., Jestrović, I., Luan, B., Coyle, J. L. & Sejdić, E. A comparative analysis of swallowing accelerometry and sounds during saliva swallows. Biomed. Eng. Online 14, 3 (2015).

  25. 25.

    Kumari, S. K. & Mathana, J. M. Blood sugar level indication through chewing and swallowing from acoustic MEMS sensor and deep learning algorithm for diabetic management. J. Med. Syst. 43, 1 (2018).

  26. 26.

    Mehta, D. D., Zañartu, M., Feng, S. W., Cheyne, H. A. I. & Hillman, R. E. Mobile voice health monitoring using a wearable accelerometer sensor and a smartphone platform. IEEE Trans. Biomed. Eng. 59, 3090–3096 (2012).

  27. 27.

    Michalevsky, Y., Boneh, D. & Nakibly, G. Gyrophone: recognizing speech from gyroscope signals. In Proc. 23rd USENIX Security Symposium (USENIX Association, 2014).

  28. 28.

    Nyan, M. N., Tay, F. E. H., Manimaran, M. & Seah, K. H. W. Garment-based detection of falls and activities of daily living using 3-axis MEMS accelerometer. J. Phys. Conf. Ser. 34, 1059–1067 (2006).

  29. 29.

    Curone, D., Bertolotti, G. M., Cristiani, A., Secco, E. L. & Magenes, G. A real-time and self-calibrating algorithm based on triaxial accelerometer signals for the detection of human posture and activity. IEEE Trans. Inf. Technol. Biomed. 14, 1098–1105 (2010).

  30. 30.

    Yang, C. C. & Hsu, Y. L. A review of accelerometry-based wearable motion detectors for physical activity monitoring. Sensors 10, 7772–7788 (2010).

  31. 31.

    Posatskiy, A. O. & Chau, T. The effects of motion artifact on mechanomyography: A comparative study of microphones and accelerometers. J. Electromyogr. Kinesiol. 22, 320–324 (2012).

  32. 32.

    Maki, H., Ogawa, H., Matsuoka, S., Yonezawa, Y. & Caldwell, W. M. A daily living activity remote monitoring system for solitary elderly people. Proc. Annu. Int. Conf. IEEE Eng. Med. Biol. Soc. EMBS 2011, 5608–5611 (2011).

  33. 33.

    Zheng, Y. L. et al. Unobtrusive sensing and wearable devices for health informatics. IEEE Trans. Biomed. Eng. 61, 1538–1554 (2014).

  34. 34.

    Phan, D. H., Bonnet, S., Guillemaud, R., Castelli, E. & Thi, N. Y. P. Estimation of respiratory waveform and heart rate using an accelerometer. In Proc. 30th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (IEEE, 2008).

  35. 35.

    Vertens, J. et al. Measuring respiration and heart rate using two acceleration sensors on a fully embedded platform. In Proc. 3rd International Congress on Sport Sciences Research and Technology Support (Scitepress, 2015).

  36. 36.

    Sánchez Morillo, D., Ojeda, J. L. R., Foix, L. F. C. & Jiménez, A. L. An accelerometer-based device for sleep apnea screening. IEEE Trans. Inf. Technol. Biomed. 14, 491–499 (2010).

  37. 37.

    He, D. Da, Winokur, E. S. & Sodini, C. G. An ear-worn continuous ballistocardiogram (BCG) sensor for cardiovascular monitoring. Conf. Proc. IEEE Eng. Med. Biol. Soc. 2012, 5030–5033 (2012).

  38. 38.

    Rahman, T. et al. BodyBeat: a mobile system for sensing non-speech body sounds. In Proc. 12th Annual International Conference on Mobile Systems, Applications, and Services (ACM, 2014).

  39. 39.

    Kim, D. H. et al. Epidermal electronics. Science 333, 838–843 (2011).

  40. 40.

    Jang, K. I. et al. Soft network composite materials with deterministic and bio-inspired designs. Nat. Commun. 6, 6566 (2015).

  41. 41.

    Fan, J. A. et al. Fractal design concepts for stretchable electronics. Nat. Commun. 5, 3266 (2014).

  42. 42.

    Kim, D. H. et al. Electronic sensor and actuator webs for large-area complex geometry cardiac mapping and therapy. Proc. Natl Acad. Sci. USA 109, 19910–19915 (2012).

  43. 43.

    Kim, D. H. et al. Materials and noncoplanar mesh designs for integrated circuits with linear elastic responses to extreme mechanical deformations. Proc. Natl Acad. Sci. USA 105, 18675–18680 (2008).

  44. 44.

    Muroga, T., Ito, Y., Aoyagi, K., Yamamoto, Y. & Yokomizo, K. Rolled copper foil. US patent 20090017325A1 (2009).

  45. 45.

    Titze, I. R. Principles of Voice Production (Prentice Hall, 1994).

  46. 46.

    Baken, R. J. & Orlikoff, R. F. Clinical measurement of speech and voice (Cengage Learning, 1999).

  47. 47.

    Wu, K. Gender recognition from speech. Part II: fine analysis. J. Acoust. Soc. Am. 90, 1841–1856 (1991).

  48. 48.

    Lin, S. J. et al. A pilot study on BSN-based ubiquitous energy expenditure monitoring. In Proc. 6th International Workshop on Wearable and Implantable Body Sensor Networks (IEEE, 2009).

  49. 49.

    Jin, A., Yin, B., Morren, G., Duric, H. & Aarts, R. M. Performance evaluation of a tri-axial accelerometry-based respiration monitoring for ambient assisted living. In Proc. 31st Annual International Conference of the IEEE Engineering in Medicine and Biology Society: Engineering the Future of Biomedicine (IEEE, 2009).

  50. 50.

    Dash, S., Shelley, K. H., Silverman, D. G. & Chon, K. H. Estimation of respiratory rate from ECG, photoplethysmogram, and piezoelectric pulse transducer signals: a comparative study of time-frequency methods. IEEE Trans. Biomed. Eng. 57, 1099–1107 (2010).

  51. 51.

    Chon, K. H., Dash, S. & Ju, K. Estimation of respiratory rate from photoplethysmogram data using time-frequency spectral estimation. IEEE Trans. Biomed. Eng. 56, 2054–2063 (2009).

  52. 52.

    Berry, R. B. et al. AASM scoring manual updates for 2017 (version 2.4). J. Clin. Sleep Med. 13, 665–666 (2017).

  53. 53.

    Watanabe, N., Reece, J. & Polus, B. I. Effects of body position on autonomic regulation of cardiovascular function in young, healthy adults. Chiropr. Osteopat. 15, 19 (2007).

  54. 54.

    Toyota, S. & Amaki, Y. Hemodynamic evaluation of the prone position by transesophageal echocardiography. J. Clin. Anesth. 10, 32–35 (1998).

  55. 55.

    Pump, B., Talleruphuus, U., Christensen, N. J., Warberg, J. & Norsk, P. Effects of supine, prone, and lateral positions on cardiovascular and renal variables in humans. Am. J. Physiol. 283, R174–R180 (2002).

  56. 56.

    Issa, F. G. & Sullivan, C. E. Upper airway closing pressures in snorers. J. Appl. Physiol. 57, 528–535 (1984).

  57. 57.

    Aurégan, Y. & Depollier, C. Snoring: linear stability analysis and in-vitro experiments. J. Sound Vib. 188, 39–53 (1995).

  58. 58.

    Fajdiga, I. Snoring imaging: could Bernoulli explain it all? Chest 128, 896–901 (2005).

  59. 59.

    Javaid, A. Q. et al. Quantifying and reducing motion artifacts in wearable seismocardiogram measurements during walking to assess left ventricular health. IEEE Trans. Biomed. Eng. 64, 1277–1286 (2017).

  60. 60.

    Schwindt, D. A., Wilhelm, K. P., Miller, D. L. & Maibach, H. I. Cumulative irritation in older and younger skin: a comparison. Acta Derm. Venereol. 78, 279–283 (1998).

Download references

Acknowledgements

The materials and device-engineering aspects of the research were supported by the Material Science and Engineering Department and Center for Bio-Integrated Electronics at Northwestern University. K.L. acknowledges support from the Samsung Scholarship. K.L. acknowledges the help from A. Sahakian on radio frequency communication and antenna tuning. Z.X. acknowledges support from the National Natural Science Foundation of China (grant no.11402134). S.X. and J.A.R. recognizes support from the National Institute on Aging of the National Institutes of Health under R41AG062023 and R43AG060812. R.A. acknowledges support from National Science Foundation (NSF) Graduate Research Fellowship under grant no. 1842165 and the Ford Foundation Predoctoral Fellowship. Y.H. acknowledges support from the NSF (CMMI1635443). S.M. is grateful to Indo-U.S. Science and Technology Forum (SERB-IUSSTF) for her SERB Indo-U.S. Postdoctoral Award. X.N. thanks D. Lu for helpful discussions. K.L. thanks the Dion family for volunteering for the data collection.

Author information

K.L., Z.X. and J.A.R. performed the structural design of the system. Z.X., R.A., Y.D. and Y.H. performed mechanical and electromagnetic modelling and theoretical studies. K.L., J.Y.L., J.H.L., J.B.P. and J.K. developed the embedded system and the user interface. K.L., X.N. and J.A.R. designed and performed the experimental studies on the technology. X.N., K.L. and J.A.R. designed and performed the human subject studies. X.N., K.L., M.I., R.L.E., D.J.P. and D.H.K developed the signal-processing algorithms and performed the data analysis. K.L., H.A., D.J.P., H.U.C., O.O.O., S.G., E.C., M.H., J.B., H.J., C.L., S.B.K., S.M., J.T.R and I.H. manufactured the devices. S.X., A.T. and C.R.D. provided clinical advice. X.N. and J.A.R. wrote the signal processing algorithm part of the manuscript. K.L., X.N., Z.X., Y.H. and J.A.R. contributed to the other sections.

Correspondence to Zhaoqian Xie or Charles R. Davies or Yonggang Huang or John A. Rogers.

Ethics declarations

Competing interests

The authors declare no competing interests.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Supplementary Information

Supplementary Methods, Supplementary References, Supplementary Figures and Supplementary Tables.

Reporting Summary

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Lee, K., Ni, X., Lee, J.Y. et al. Mechano-acoustic sensing of physiological processes and body motions via a soft wireless device placed at the suprasternal notch. Nat Biomed Eng (2019). https://doi.org/10.1038/s41551-019-0480-6

Download citation