Module 3 Activity Research

Weekly Activity Template

Sanren Zhou

Project 3

Module 3

In Project 3, we will continue to explore and refine our prototype. The emphasis of our final prototype is on compelling interaction and an improved user experience.

Workshop 1

To better support bodystorming, we first created several props using cardboard to help users understand the next steps. At the same time, we made some signs that guide user actions and inform observers of the current state. In our concept, a smart kitchen should first include a smart refrigerator that can remind users which ingredients are about to expire and generate recipes based on those ingredients, helping users consume near-expiration food in time. The refrigerator is also connected to the oven. Once a recipe is selected, the oven is activated and begins preheating. The refrigerator and oven are arranged in a parallel layout, making it convenient for user movement. Of course, during the cooking process, users may inevitably make mistakes. For example, in this bodystorming scenario, the user accidentally spills food, but a smart vacuum robot promptly cleans the kitchen. The robot is placed in a corner area so that it does not interfere with the user’s movement path. I believe bodystorming is a very engaging and effective way to help us understand user behavior. Therefore, after class I read several related articles, such as Alison Hu’s *Bodystorming: Research Methods Case Study*. These readings deepened my understanding of the benefits of bodystorming and how it can be integrated with other UX research methods.

Activity 1: My Research

At the beginning of Project 3, we realized that it might be better to replace the light sensor with a pressure sensor and a heart rate sensor. The force sensor can detect when a user leans on or sits on the cushion, thereby triggering visuals in TouchDesigner. The heart rate sensor, on the other hand, can change the type of visuals played in TouchDesigner based on the user’s heart rate. We purchased a heart rate sensor online, so we needed to test whether it worked properly. One advantage of this heart rate sensor is that it has a built-in LED, allowing us to verify that it is functioning without relying on additional methods. The force sensor, however, does not have an indicator light. Therefore, we connected it to the Arduino and used the same testing method as in our previous project. We wrote a piece of code to run the sensor and observed the serial monitor: when the sensor was squeezed, the values fluctuated on the screen. From this, we confirmed that both sensors were functioning properly. During testing of the heart rate sensor, we noticed that the readings were inaccurate, with values far below a normal adult heart rate. We investigated possible causes and tried different versions of the code, but this did not lead to significant improvement. Eventually, we rebuilt the circuit and found that the issue was caused by nearby movement and the device not being placed stably enough. After changing the environment, we observed that the readings returned to a normal range. From this, we learned that this sensor is not highly precise and is strongly affected by its surroundings. This is an important factor we will need to consider in the final setup. Based on several tutorials, we connected both sensors to the circuit at the same time. To test whether they were functioning correctly, we opened the serial plotter and observed two separate lines representing different data streams. This confirmed that we could proceed to the next step: adding both sensors to the serial input.

Activity 2: My Research

In Activity 2, our main goal was to import sensor data into TouchDesigner and use it to trigger visuals within the software. Before importing the data into TouchDesigner, we conducted a final test of both sensors in the Arduino IDE. Two columns of data appeared in the display, representing heart rate and pressure respectively. After referring to several tutorials, we learned how to connect Arduino with TouchDesigner. The main components we used were Serial and DAT to. The remaining challenge was how to separate the two data streams and assign each dataset to a different function. When importing the data, TouchDesigner produced an error as shown in the image. After consulting tutorials and searching online, we learned that we needed to output only pure numeric values separated by a delimiter (such as a comma) in Arduino, and set the Serial DAT to Table, so that the Select DAT could correctly separate and use the data. This image shows our circuit connections and some of the TouchDesigner components we are using. Starting from here, we will build the data pipeline in TouchDesigner step by step. We tested both sensors in TouchDesigner. After several hours of learning and debugging, the data was finally successfully imported into TouchDesigner. As shown in the serial data, the left values represent heart rate and the right values represent force, with no abnormal readings. This means we can now use the data ranges to trigger and modify the visuals.

Additional Research or Workshops

In my personal time, my main focus was researching how to design and create appropriate visuals in TouchDesigner that could work effectively with our data. My initial idea was to transform calming, concrete objects—such as butterflies, deer, and forests—into particles and animate them. I first downloaded FBX files and imported the models into TouchDesigner. The image shows an error that occurred during import due to missing textures. However, since my project did not require textures and only needed the shape of the objects, I directly removed the texture component. To make the animation transitions more natural and smoother, we adjusted the start and end times of the animation in the **Play → Trim** settings. By using the **Sprinkle** feature, we were able to convert the FBX models into particle clusters that retain the same overall shape. After that, we could manipulate the particles in the same way as we did in Project 2. The image shows a custom material for **ParticlesGPU**. By adjusting it, we can modify the particles’ state, size, density, and other properties. Within the **Noise** settings, we can change the speed of the noise animation by entering `absTime.seconds/10`. When we tried to use the serial data to control brightness, TouchDesigner produced an error like the one shown. After consulting tutorials and searching online, as well as asking GPT, we were not able to resolve the issue on our own. Since Python is not a programming language we are familiar with, we eventually asked a programmer friend for help, and she helped us correct the incorrect exporting content. Although we did not solve this problem independently, we at least learned more about how TouchDesigner handles Python-based scripting and data exporting, which expanded our technical understanding. About the visual for an exciting mood, we want to create a visual with intense effects that run quickly. The main purpose of this visual is to warn the users that their heart rate is too high (over 100bpm), they need to calm down and relax. Moreover, we also discovered the visuals that are needed for a calm mood, which might work better with steady effects and slow movement. The main purpose of this visual is to reflect the user’s mood, to show that they are being calm and steady at that moment, which can also help them to fall asleep or to do meditation. We made the heart rate wristband using non-woven fabric because it is easy to obtain, soft, and relatively stable. To ensure comfortable use for most users, we designed it to be adjustable so that it fits closely around the wrist, allowing us to obtain the most accurate heart rate readings possible. We placed the force sensor inside the cushion, while the Arduino and heart rate sensor were positioned on a stable, level tabletop. Compared to our Project 2 setup, where the Arduino was placed outside the cushion, this design is more comfortable—since no one wants to lean against a circuit board—and better aligns with users’ natural resting habits.

Project 3

Project 3 Final Prototype

Our prototype video demonstrates how the entire device is intended to function under ideal conditions. For more details, please watch the video

The prototype for Project 3 uses a force sensor and a pulse sensor connected to an Arduino to detect changes in pressure and the user's heart rate. When the user leans on or sits against the cushion, the sensorvalues change. These changes trigger TouchDesigner to display predefined visuals through a projector. Depending on the user’s heart rate, the system presents either fast and intense geometric animations or calm and soothing geometric motions. It can serve as a reminder device or assist users in calming down, meditating, or falling asleep. The prototype consists of a cushion with a built-in force-sensor circuit and a wristband with an integrated pulse sensor, requiring only that the user lean on the cushion and wear the band to activate the system.<a href='https://youtube.com/shorts/zRFuHhrke08'>Prototype Demo Video</a>
×

Powered by w3.css