![](/fileadmin/_processed_/e/d/csm_avatar_ce1601c958.jpg)
![](../../fileadmin/_processed_/e/d/csm_avatar_ce1601c958.jpg)
Lakomkin Egor
Prosodic feature recognition for threat detection
Principal Supervisor:
Prof. Dr. Stefan Wermter
Universität Hamburg
Collaboration partners:
- Fondazione Istituto Italiano Di Tecnologia
- Honda Research Institute Europe GmbH
Competence Area: Situation
Research summary
In an interaction, cues for detecting threatening situations could be changes in the intonation or loudness indicating dangerous situations. Robots should be able to perceive and react to such signals and change their behaviour to avoid the threatening situation or, in case the threat cannot yet be identified, be alert and act more carefully. In order to achieve this goal, this project aims at identifying robust representation of an acoustic signal using neural networks.
I identify three main directions in my research: 1) features and signal representations learning for speech emotion recognition 2) investigation of neural architectures which allow robust to an internal robot's and an environmental noise emotion recognition 3) research on the methods and approaches to incorporate information contained in modalities other than auditory to improve speech emotion recognition. For example, linguistic analysis of a spoken text can help in difficult situations when analyzing only the acoustic signal is not enough to infer an affective state of the speaker.
Publications
Lakomkin, E., Zamani M., Weber C., Magg S., Wermter, S. (2019, May)
Incorporating End-to-End Speech Recognition Models for Sentiment Analysis
Proceedings of the 2019 International Conference on Robotics and Automation (ICRA), Montreal, QC, Canada, 2019, pp. 7976-7982.
Lakomkin, E., Weber C., Magg S., Wermter, S. (2018, November)
KT-Speech-Crawler: Automatic Dataset Construction for Speech Recognition from YouTube Videos
Proceedings of the Conference on Empirical Methods in Natural Language Processing, pages 90--95 - 2018
Lakomkin, E., Zamani M., Weber C., Magg S., Wermter, S. (2018, October)
On the Robustness of Speech Emotion Recognition for Human-Robot Interaction with Deep Neural Networks
Proceedings of the International Conference on Intelligent Robots, pages 854--860 - 2018
Qu L., Weber C., Lakomkin E., Twiefel J., Wermter S.
Combining Articulatory Features with End-to-end Learning in Speech Recognition
Proceedings of the International Conference on Artificial Neural Networks (ICANN) - Oct 2018.
Barros P., Churamani N., Lakomkin E., Siqueira H., Sutherland A., Wermter S. (2018, July)
The omg-emotion behavior dataset
Proceedings of the 2018 International Joint Conference on Neural Networks (IJCNN)
Springenberg S., Lakomkin E., Weber C., Wermter S.
Image-to-Text Transduction with Spatial Self-Attention https://www2.informatik.uni-hamburg.de/wtm/publications/2018/SLWW18/ESANN_spatial_self_attention_final.pdf
Proceedings of the 26th European Symposium on Artificial Neural Networks, Computational Intelligence and Machine Learning (ESANN), pages 43--48 - Apr 2018.
Lakomkin, E., Zamani M., Weber C., Magg S., Wermter, S. (2018, January)
EmoRL: Real-time Acoustic Emotion Classification using Deep Reinforcement Learning
Proceedings of the International Conference on Robotics and Automation (ICRA), pages 4445--4450 - May 2018
Lakomkin, E., Weber C., Magg S., Wermter, S. (2017, November)
Reusing neural speech representations for auditory emotion recognition.
Proceedings of the Eighth International Joint Conference on Natural Language Processing, Volume 1, pages 423--430 - Nov 2017
Lakomkin, E., Bothe, C., and Wermter, S. (2017, September).
GradAscent at EmoInt-2017: Character- and Word-Level Recurrent Neural Network Models for Tweet Emotion Intensity Detection.
Proceedings of the 8th Workshop on Computational Approaches to Subjectivity, Sentiment and Social Media Analysis at EMNLP-2017, pages 169--174 - Sep 2017
Lakomkin, E., Weber, C., Wermter, S. (2017, April).
Automatically augmenting an emotion dataset improves classification using audio
Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics (EACL), Valencia, Spain.
Short Curriculum Vitae
- Since May 2019: Applied Scientist II at Amazon Alexa
- Summer 2018: Research Intern at Amazon Alexa Cambridge
- 2016-2019: Research Associate (PhD Student: SECURE Project) at Knowledge Technology Research Group, Department of Computer Science, University of Hamburg, Germany
- M.Sc. in Computer Science, Moscow State Technical University n.a. Bauman, Moscow, Russia. Faculty – Informatics and control systems, department - Automatic Information Processing and Control Systems, diploma in Computer Engineering, class of 2011, GPA 4,4 (of 5)
- Nanyang Technological University, researcher and developer, Summer Research Internship, School of Computer Engineering
- Nanyang Technological University, Research associate, Computer Linguistics and Bioinformatics
Contact
firstname.lastname at gmail.com