Tutorial 4: Tracking Human Activities in an Interactive Space: Unveiling the Potential of mmWave Sensing

Speakers: Argha Sen, Sandip Chakraborty and Swadhin Pradhan

Duration: 1.5 hr

Abstract: In this tutorial, we explore the transformative capabilities of millimeter-wave (mmWave) sensing in continuously tracking human activities in smart spaces. Leveraging Commercial off-the-shelf (COTS) mmWave radars, we demonstrate how to overcome key challenges such as multi-user activity tracking, handling macro and micro activity scales, and ensuring real-time, robust sensing. Participants will gain an in-depth understanding of radar signal processing, including range-doppler heatmaps, clustering techniques, and dynamic radar reconfiguration for enhanced sensing. We will discuss how mmWave sensing outperforms traditional modalities by offering higher resolution and adaptability. The session also highlights the Multi-user Activity Tracking via mmWave Sensing system, novel methodologies for multi-user localization and robust activity classification, which ensure seamless operation even in challenging scenarios like occlusion or multi-user interference. This tutorial aims attendees to harness the potential of mmWave sensing for developing intelligent environments by bridging the gap between theoretical insights and practical applications. The knowledge shared is applicable across various domains, from smart homes, driver monitoring, and healthcare to industrial automation and beyond.

Argha Sen earned his B.Tech from NIT Durgapur in 2020 and is currently pursuing a Ph.D. in Computer Science and Engineering at IIT Kharagpur under the supervision of Dr. Sandip Chakraborty. His research interests include Millimeter Wave Sensing, FMCW Radars, Human-Computer Interaction (HCI), next-generation cellular networks, and UAVs. His works have been published in prestigious venues such as IPSN, IEEE PerCom, ACM IMWUT, Elsevier Pervasive and Mobile Computing Journal, IEEE TNSM, EWSN, NeurIPS Datasets and Benchmarks Track, COMSNETS, etc., among others. He has also filed an Indian patent application in mmWave Sensing Technology. His work has been supported by multiple travel grants, including the ACM MobiSys Student Travel Grant, ACM IARCS Travel Grants, COMSNETS LRN Travel Grant, IEEE INFOCOM Travel Grant, etc. He also visited Singapore Management University (SMU) as a Visiting Postgraduate Research Student for six months. Learn more about him through his website: https://arghasen10.github.io/.

Dr. Sandip Chakraborty is working as an Associate Professor in the Department of Computer Science and Engineering at IIT Kharagpur. He obtained his B.E from Jadavpur University in 2009 and M.Tech and Ph.D. degrees both from IIT Guwahati, India, in 2011 and 2014, respectively. His primary research interest is in the intersection of Computer Systems, Pervasive Computing, and Human-Computer Interaction. He is one of the founding members of ACM IMOBILE, the ACM SIGMOBILE chapter in India. He is currently working as Area Editor of IEEE Transactions on Services Computing, Elsevier Ad Hoc Networks, and Elsevier Pervasive and Mobile Computing Journal. He is a recipient of the Google Academic Research Award 2023 and 2024, the Indian National Academy of Engineering (INAE) Young Engineer Award 2019, multiple best paper awards at various conferences, and the University Gold Medal at Jadavpur University, Kolkata. His works have been published in conferences like IEEE PerCom, IEEE INFOCOM, ACM CHI, ACM MobileHCI, ACM CSCW, TheWebConference, etc., among others. More details about him can be found at http://cse.iitkgp.ac.in/~sandipc/.

Swadhin Pradhan is a senior research engineer at Cisco Meraki, where he builds machine learning systems to improve wireless network performance and management. He holds a Ph.D. degree from the Department of Computer Science at UT Austin. His research is focused on mobile computing and internet-of-things (IoT) systems. He has designed, built, and deployed systems that deliver ubiquitous sensing, enhanced communication capabilities, and new human-computer interfaces. His works have been published in top-tier conferences like MobiCom, CHI, UbiComp, PerCom, SenSys, etc., and featured in the popular press like BBC, MIT Tech Review, Boston Globe, The Guardian, etc. He is a recipient of the UT Austin Graduate Fellowship and the University Gold Medal at Jadavpur University, Kolkata.

Tutorial Outline

Part 1: Introduction to mmWave Sensing (20 minutes)

  • Overview of FMCW radar principles.
  • Advantages of mmWave sensing over other modalities.

Part 2: Sensing Framework (20 minutes)

  • Architecture: Hardware setup and data collection pipeline.
  • Localization and tracking:
    • Removing static clutter and isolating subjects.
    • Global coordinate transformation.
  • Activity classification:
    • Differentiated radar configurations.
    • Using range-doppler data for temporal activity recognition.

Part 3: Challenges and Solutions (15 minutes)

  • Handling multi-user interference.
  • Robust tracking under occlusion and non-line-of-sight (NLoS) conditions.
  • Dynamic reconfiguration for macro and micro activities.

Part 4: Evaluation and Comparisons (20 minutes)

  • Performance analysis of the classification across diverse environments.
  • Key metrics: Accuracy, response time, and scalability.
  • Comparison with existing benchmarks like RadHAR, Pantomime, Vid2Doppler, and VGG-16.
  • Open-source resources and guidelines for reproducibility.

Part 5: Future Directions and Q&A (15 minutes)

  • Emerging trends in mmWave sensing.
  • Expanding to outdoor and multi-room settings.
  • Open discussion and participant questions.