Surya Sharma

In this work, we describe a new method to detect periods of eating by tracking wrist motion during everyday life. Eating uses hand-to-mouth gestures for ingestion, each of which lasts a few seconds. Previous works have detected these gestures individually and then aggregated them to identify meals. The novelty of our approach is that we analyze a much longer window (0.5–15 min) using a convolutional neural network. Longer windows can contain other gestures related to eating, such as cutting or… Show more

In this work, we describe a new method to detect periods of eating by tracking wrist motion during everyday life. Eating uses hand-to-mouth gestures for ingestion, each of which lasts a few seconds. Previous works have detected these gestures individually and then aggregated them to identify meals. The novelty of our approach is that we analyze a much longer window (0.5–15 min) using a convolutional neural network. Longer windows can contain other gestures related to eating, such as cutting or manipulating food, preparing foods for consumption, and resting between ingestion events. The context of these other gestures can improve the detection of periods of eating. We test our methods on the public Clemson all-day dataset, which consists of 354 recordings containing 1063 eating episodes. We found that accuracy at detecting eating increased by 15% in ≥4 min windows compared to ≤15 s windows. Using a 6 min window, we detected 89% of eating episodes, with 1.7 false positives for every true positive (FP/TP). These are the best results achieved to date on this dataset. Show less

Many works in the field of automated dietary monitoring (ADM) have analyzed small data sets consisting of < 10 subjects and < 20 meals. This is often the first step in researching new sensors or body positions for detecting consumption. Metrics tend to focus on within-meal accuracy by quantifying physiological event detection (bites, chews, swallows). As analysis shifts to larger datasets containing many days of data from everyday life and researchers build methods that can be used in… Show more

Many works in the field of automated dietary monitoring (ADM) have analyzed small data sets consisting of < 10 subjects and < 20 meals. This is often the first step in researching new sensors or body positions for detecting consumption. Metrics tend to focus on within-meal accuracy by quantifying physiological event detection (bites, chews, swallows). As analysis shifts to larger datasets containing many days of data from everyday life and researchers build methods that can be used in everyday life, it becomes equally important to quantify the accuracy of how many meals are detected. In small data sets most meals can be detected at least partially. In larger datasets, some meals are missed and false positives occur. In this work we discuss the pros and cons of time-based metrics and episode level metrics. We demonstrate how class imbalance affects some of the commonly used time metrics, and discuss why episode level metrics need to be reported as the field transitions from small data sets to big data sets. Show less

This paper considers detecting eating in free-living humans by tracking wrist motion.
We are specifically interested in the effect of secondary activities that people conduct while simultaneously eating, such as walking, watching television, or working. These secondary activities cause wrist motions that obfuscate those associated with eating, increasing the difficulty of detecting periods of eating. We collected a large dataset of 4,680 hours of wrist motion from 351 participants during… Show more

This paper considers detecting eating in free-living humans by tracking wrist motion.
We are specifically interested in the effect of secondary activities that people conduct while simultaneously eating, such as walking, watching television, or working. These secondary activities cause wrist motions that obfuscate those associated with eating, increasing the difficulty of detecting periods of eating. We collected a large dataset of 4,680 hours of wrist motion from 351 participants during free-living. Participants reported secondary activities in 72% of meals. Analysis of wrist motion data revealed that the wrist was resting 12.8% of the time during self-reported meals, compared to only 6.8% of the time in a cafeteria dataset, while walking motion was found during 5.5% of the time during meals in free-living, compared to 0% in a cafeteria. Augmenting an eating detection classifier to include walking and resting detection improved accuracy from 74% to 77% on our free-living dataset (t[353]=7.86, p<0.001). While eating detection could be improved using more sophisticated machine learning methods or sensor modalities, all approaches would be affected by secondary activities as they affect the labeling of data itself. Our work suggests that future work should collect detailed ground truth on secondary activities being conducted during eating {as these activities could hold insights on when an eating activity starts or stops in the absence of video-based ground truth.

(Accepted to ACM Health) Show less

Accelerometers have gained popularity in biomedical and m-health applications such as actigraphy or automated dietary monitoring due to their ease of use and their ability to characterize motion. These sensors report raw acceleration from which gravity and linear acceleration must be separated, with commercial packages reporting raw acceleration, linear acceleration or both. New researchers to the field may often be confused when to use raw acceleration or linear acceleration, especially given… Show more

Accelerometers have gained popularity in biomedical and m-health applications such as actigraphy or automated dietary monitoring due to their ease of use and their ability to characterize motion. These sensors report raw acceleration from which gravity and linear acceleration must be separated, with commercial packages reporting raw acceleration, linear acceleration or both. New researchers to the field may often be confused when to use raw acceleration or linear acceleration, especially given the susceptibility of linear acceleration to noise, and the lack of published distributions of these signals. This paper provides a short tutorial on obtaining linear acceleration estimates. Using these methods we analyze a large dataset containing 4,680 hours of wrist tracking data, the largest such dataset known to us. We learn the range of wrist motion accelerations, and quantify the expected noise in the linear acceleration signal. We explain the sources of this noise, and a filtering technique to mitigate it. For the first time, we report the range of wrist acceleration values observed during free-living, and quantify the expected range of noise in this wrist acceleration. We show that while previous work has reported average accelerations at the feet and body ranging from 0-15g during spots-like activities like walking, running or jumping, wrist acceleration in free-living subjects during daily activities is often much lower, and ranges from 0-0.2g. We show that noise in linear acceleration can range from 0-0.06g, an overlap of 70%. This suggests that in applications where the wrist acceleration is in this range of noise, linear acceleration may not provide useful features, and researchers should only rely on raw acceleration instead. Show less

More than one third of adults in the United States are now classified as obese. Self monitoring of energy in take has been shown to have a positive impact on weight loss, but existing tools for logging eating activities are tedious to use and prone to bias which leads to noncompliance and underestimation. In this paper we describe preliminary results from our on going data collection from 500 free living participants, in an effort to improve our previous algorithm that detects periods of… Show more

More than one third of adults in the United States are now classified as obese. Self monitoring of energy in take has been shown to have a positive impact on weight loss, but existing tools for logging eating activities are tedious to use and prone to bias which leads to noncompliance and underestimation. In this paper we describe preliminary results from our on going data collection from 500 free living participants, in an effort to improve our previous algorithm that detects periods of eating. We see a 75% accuracy in our data collected to date. Show less

Surya Prakash Sharma, a PhD student in electrical engineering at Clemson University, describes his research as part of the 3-Minute Thesis competition at Clemson. The title of his presentation is "Bite Counting: A Simpler Approach to Dealing with Obesity."

CyberPLAYce is our novel, interactive-computational construction kit for elementary school children and their teachers. CyberPLAYce bridges the physical and digital worlds, allowing young students to bring their ideas, stories and class subjects to life through the construction of cyber-physical environments. The CyberPLAYce construction kit is comprised of hand-sized, magnetic modules integrating a variety of electronic components, and rectangular panels, nearly two-feet measured diagonally… Show more

CyberPLAYce is our novel, interactive-computational construction kit for elementary school children and their teachers. CyberPLAYce bridges the physical and digital worlds, allowing young students to bring their ideas, stories and class subjects to life through the construction of cyber-physical environments. The CyberPLAYce construction kit is comprised of hand-sized, magnetic modules integrating a variety of electronic components, and rectangular panels, nearly two-feet measured diagonally, that receive the modules and serve as physical building blocks for constructing cyber-physical environments imagined by children. Through play, children become comfortable with the working modules and panels; subsequently, they are provided matching non-electronic module cards allowing them to quickly compose pattern sequences to map ideas, stories and class content. Additionally, students are provided action and story cards to spark their imagination. CyberPLAYce merges play and learning in the physical world while transitioning students from consumers of virtual and digital-centric technologies into technological innovators and cyber-playful storytellers. Show less

As the consumption of electricity increases, so does its wastage. This paper describes our attempt to develop a low cost device to monitor electricity usage, and control outlets with various methods from simple Infrared Remotes to complicated SMS based communication. The device can easily be embedded into household power points used across India, and communicates various power parameters to a computer using readily available methods such as RS-232, Ethernet or Wifi. (Abstract)