Research summary

In an interaction, cues for detecting threatening situations could be changes in the intonation or loudness indicating dangerous situations. Robots should be able to perceive and react to such signals and change their behaviour to avoid the threatening situation or, in case the threat cannot yet be identified, be alert and act more carefully. In order to achieve this goal, this project aims at identifying robust representation of an acoustic signal using neural networks.

I identify three main directions in my research: 1) features and signal representations learning for speech emotion recognition 2) investigation of neural architectures which allow robust to an internal robot's and an environmental noise emotion recognition 3) research on the methods and approaches to incorporate information contained in modalities other than auditory to improve speech emotion recognition. For example, linguistic analysis of a spoken text can help in difficult situations when analyzing only the acoustic signal is not enough to infer an affective state of the speaker.

Publications

Lakomkin, E., Zamani M., Weber C., Magg S., Wermter, S. (2019, May)

Incorporating End-to-End Speech Recognition Models for Sentiment Analysis

Proceedings of the 2019 International Conference on Robotics and Automation (ICRA), Montreal, QC, Canada, 2019, pp. 7976-7982.

Lakomkin, E., Weber C., Magg S., Wermter, S. (2018, November)

KT-Speech-Crawler: Automatic Dataset Construction for Speech Recognition from YouTube Videos

Proceedings of the Conference on Empirical Methods in Natural Language Processing, pages 90--95 - 2018

Lakomkin, E., Zamani M., Weber C., Magg S., Wermter, S. (2018, October)

On the Robustness of Speech Emotion Recognition for Human-Robot Interaction with Deep Neural Networks 

Proceedings of the International Conference on Intelligent Robots, pages 854--860 - 2018

Qu L., Weber C., Lakomkin E., Twiefel J., Wermter S.

Combining Articulatory Features with End-to-end Learning in Speech Recognition  

Proceedings of the International Conference on Artificial Neural Networks (ICANN) - Oct 2018.

Barros P., Churamani N., Lakomkin E., Siqueira H., Sutherland A., Wermter S. (2018, July)

The omg-emotion behavior dataset

Proceedings of the 2018 International Joint Conference on Neural Networks (IJCNN)

Springenberg S., Lakomkin E., Weber C., Wermter S.

Image-to-Text Transduction with Spatial Self-Attention   https://www2.informatik.uni-hamburg.de/wtm/publications/2018/SLWW18/ESANN_spatial_self_attention_final.pdf

Proceedings of the 26th European Symposium on Artificial Neural Networks, Computational Intelligence and Machine Learning (ESANN), pages 43--48 - Apr 2018.

Lakomkin, E., Zamani M., Weber C., Magg S., Wermter, S. (2018, January)

EmoRL: Real-time Acoustic Emotion Classification using Deep Reinforcement Learning 

Proceedings of the International Conference on Robotics and Automation (ICRA), pages 4445--4450 - May 2018

Lakomkin, E., Weber C., Magg S., Wermter, S. (2017, November)

Reusing neural speech representations for auditory emotion recognition.
Proceedings of the Eighth International Joint Conference on Natural Language Processing, Volume 1, pages 423--430 - Nov 2017

Lakomkin, E., Bothe, C., and Wermter, S. (2017, September). 
GradAscent at EmoInt-2017: Character- and Word-Level Recurrent Neural Network Models for Tweet Emotion Intensity Detection. 
Proceedings of the 8th Workshop on Computational Approaches to Subjectivity, Sentiment and Social Media Analysis at EMNLP-2017, pages 169--174 - Sep 2017

Lakomkin, E., Weber, C., Wermter, S. (2017, April). 
Automatically augmenting an emotion dataset improves classification using audio 
Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics (EACL), Valencia, Spain.

Short Curriculum Vitae

• Since May 2019: Applied Scientist II at Amazon Alexa
• Summer 2018: Research Intern at Amazon Alexa Cambridge
• 2016-2019: Research Associate (PhD Student: SECURE Project) at Knowledge Technology Research Group, Department of Computer Science, University of Hamburg, Germany

• M.Sc. in Computer Science, Moscow State Technical University n.a. Bauman, Moscow, Russia. Faculty – Informatics and control systems, department - Automatic Information Processing and Control Systems, diploma in Computer Engineering, class of 2011, GPA 4,4 (of 5)
• Nanyang Technological University, researcher and developer, Summer Research Internship, School of Computer Engineering
• Nanyang Technological University, Research associate, Computer Linguistics and Bioinformatics

Contact

firstname.lastname at gmail.com

egorlakomkin.github.io

www.linkedin.com/in/egor-lakomkin-1032b337