Presentation

<
Audio Matters Too! Enhancing Markerless Motion Capture With Audio Signals for String Performance Capture
SessionMotion Capture
DescriptionMotion Capture of musical instrument performance is challenging even with markers. By extracting playing cues inherent in the audio for markerless video motion capture, our method recovers subtle finger-string contacts and intricate playing movements. We further contribute the first large-scale String Performance Dataset (SPD) with high-quality motion and contact annotations.
Authors
Yitong Jin
Central Conservatory of Music
Tsinghua University
Zhiping Qiu
Central Conservatory of Music
Tsinghua University
Yi Shi
Central Conservatory of Music
Tsinghua University
Shuangpeng Sun
Tsinghua University
Chongwu Wang
Central Conservatory of Music
Donghao Pan
Central Conservatory of Music
Jiachen Zhao
Tsinghua University
Zhenghao Liang
Weilan Tech
Yuan Wang
Central Conservatory of Music
Xiaobing Li
Central Conservatory of Music
Feng Yu
Central Conservatory of Music
Tao Yu
Tsinghua University
Qionghai Dai
Tsinghua University
Event Type
Technical Paper
TimeWednesday, 31 July 20249:30am - 9:40am MDT
LocationMile High 4
ACM Digital Library Journal Papers' PDFs
Conference Papers' PDFs
Session Time & Location
Sunday, 28 July 20246:00pm - 8:45pm MDTBluebird Ballroom
Wednesday, 31 July 20249:00am - 10:30am MDTMile High 4
Interest Areas
Research & Education
Recordings
Livestreamed
Recorded
Keywords
Animation
Modeling
Registration Categories
Full Conference
Full Conference Supporter
Virtual Access
Exhibitor Full Conference
Wednesday
Next PresentationNext PresentationHand-Object Interaction Controller (HOIC): Deep Reinforcement Learning for Reconstructing Interactions With Physics