Introduction
This work aims to explore force and experience the intrinsic properties of matter. At the same time, it serves as an experimental confrontation between balance and imbalance.
Participants are required to support a heavy pumpkin with their palm, experiencing its gravitational pull while sensors detect the hand's position. The goal is to align the palm with a hollow circle displayed on the screen. Throughout this process, the interaction showcases the struggle between human endurance and willpower, arm muscle strength, and the downward force of the pumpkin's weight. The pumpkin continuously exerts pressure downward, while the hand persistently pushes upward in resistance. However, as time progresses, the hand’s ability to support the weight gradually weakens, causing the pumpkin to slowly descend, ultimately leading to hand tremors and inevitable imbalance.
This transition from force maintenance to the confrontation of imbalance is narrated in real-time through music. As the participant’s endurance diminishes, the rhythm, timbre, and intensity of the music change accordingly, creating an immersive auditory experience that dynamically evolves with the body’s state. This allows the audience to perceive the gradual disappearance of balance and the decline of strength in an experiential and tangible manner.
Technical Implementation
Visual aspect: The primary function of the visual component is to provide visual references for interaction. Using Touchdesigner to capture data from the Leap Motion sensor, the spatial coordinates of the palm are linked to the X, Y, and Z axes of a solid circle, making it move with the palm. Additionally, a hollow circle of the same size as the solid circle, but stationary, is placed in the center of the screen, surrounded by anchor points. Thus, when the solid circle overlaps with the hollow circle, it indicates the center of the screen.
Communication part: Utilize Touchdesigner and Max/Msp for serial communication, transmitting the movement data of the hand to Max/Msp. In Max/Msp, extract the received hand's X, Y, Z-axis movement data and connect them with various audio oscillators, filters, and other components, enabling the control of different waveform generators through the movement of the hand in different axes.
Sound part: The movement of the palm influences the frequency and amplitude of the sound waveform, dynamically adjusting based on the center point (zero point):
When the palm is positioned at the center point, both the sound frequency and amplitude drop to zero, creating a state of silence.
As the palm moves away from the center point (or descends along the Z-axis), the frequency and amplitude gradually increase, generating an auditory experience of escalating tension.
The sound design primarily utilizes short, intermittent "point-like" sounds (such as the crisp sound of tapping on glass) to emphasize the immediacy of feedback and heighten a sense of tension. As the palm deviates from the center, the rhythm and intensity of the sound progressively intensify, allowing the audience to perceive the struggle of force, the dissolution of balance, and the evolving relationship between sound and bodily movement.
