Publication:
Harmonizing Emotion and Sound: A Novel Framework for Procedural Sound Generation Based on Emotional Dynamics

Simple item page
dc.citedby0
dc.contributor.authorHariyady H.en_US
dc.contributor.authorIbrahim A.A.A.en_US
dc.contributor.authorTeo J.en_US
dc.contributor.authorAjis A.F.M.en_US
dc.contributor.authorAhmad A.en_US
dc.contributor.authorYassin F.M.en_US
dc.contributor.authorSalimun C.en_US
dc.contributor.authorWeng N.G.en_US
dc.contributor.authorid57824533800en_US
dc.contributor.authorid23392655000en_US
dc.contributor.authorid57201882145en_US
dc.contributor.authorid57209978636en_US
dc.contributor.authorid55390963300en_US
dc.contributor.authorid55056163600en_US
dc.contributor.authorid36675707100en_US
dc.contributor.authorid59550696100en_US
dc.date.accessioned2025-03-03T07:45:18Z
dc.date.available2025-03-03T07:45:18Z
dc.date.issued2024
dc.description.abstractThe present work proposes a novel framework for emotion-driven procedural sound generation, termed SONEEG. The framework merges emotional recognition with dynamic sound synthesis to enhance user schooling in interactive digital environments. The framework uses physiological and emotional data to generate emotion-adaptive sound, leveraging datasets like DREAMER and EMOPIA. The primary innovation of this framework is the ability to capture emotions dynamically since we can map them onto a circumplex model of valence and arousal for precise classification. The framework adopts a Transformer-based architecture to synthesize associated sound sequences conditioned on the emotional information. In addition, the framework incorporates a procedural audio generation module employing machine learning approaches: granular and wavetable synthesis and physical modeling to generate adaptive and personalized soundscapes. A user study with 64 subjects evaluated the framework through subjective ratings of sound quality and emotional fidelity. Analysis revealed differences among samples in sound quality, with some samples getting consistently high scores and some getting mixed reviews. While the emotion recognition model reached 70.3% overall accuracy, it performed better at distinguishing between high-arousal emotions but struggled to distinguish between emotions of similar arousal. This framework can be utilized in different fields such as healthcare, education, entertainment, and marketing; real-time emotion recognition can be applied to deliver personalized adaptive experiences. This step includes acquiring multimodal emotion recognition in the future and utilizing physiological data to understand people's emotions better. ? 2024, Politeknik Negeri Padang. All rights reserved.en_US
dc.description.natureFinalen_US
dc.identifier.doi10.62527/joiv.8.4.3101
dc.identifier.epage2488
dc.identifier.issue4
dc.identifier.scopus2-s2.0-85217489069
dc.identifier.spage2479
dc.identifier.urihttps://www.scopus.com/inward/record.uri?eid=2-s2.0-85217489069&doi=10.62527%2fjoiv.8.4.3101&partnerID=40&md5=b3e3164607280418a00baae22670d7a3
dc.identifier.urihttps://irepository.uniten.edu.my/handle/123456789/36863
dc.identifier.volume8
dc.pagecount9
dc.publisherPoliteknik Negeri Padangen_US
dc.relation.ispartofAll Open Access; Gold Open Access
dc.sourceScopus
dc.sourcetitleInternational Journal on Informatics Visualization
dc.titleHarmonizing Emotion and Sound: A Novel Framework for Procedural Sound Generation Based on Emotional Dynamicsen_US
dc.typeArticleen_US
dspace.entity.typePublication
Files
Collections
SCOPUS