Researcher:
Kesim, Ege

Profile Picture
ORCID

Job Title

Master Student

First Name

Ege

Last Name

Kesim

Name

Name Variants

Kesim, Ege

Email Address

Birth Date

Filters

2019 - 201912020 - 20235

Advanced Search

Filter by

Settings

Researcher Of Publications: search.filters.isAuthorOfPublication.7bb5ea66-1836-4699-9a41-f281a920ba63

Search Results

Now showing 1 - 6 of 6
  • Placeholder
    PublicationMetadata only
    Realtime engagement measurement in human-computer interaction
    (Ieee, 2020) N/A; N/A; N/A; N/A; Department of Computer Engineering; Department of Computer Engineering; Department of Computer Engineering; Kesim, Ege; Numanoğlu, Tuğçe; Türker, Bekir Berker; Erzin, Engin; Yemez, Yücel; Sezgin, Tevfik Metin; Master Student; Master Student; PhD Student; Faculty Member; Faculty Member; Faculty Member; Department of Computer Engineering; Graduate School of Sciences and Engineering; Graduate School of Sciences and Engineering; Graduate School of Sciences and Engineering; College of Engineering; College of Engineering; College of Engineering; N/A; N/A; N/A; 34503; 107907; 18632
    Social robots are expected to understand their interlocutors and behave accordingly like humans do. Endowing robots with the capability of monitoring user engagement during their interactions with humans is one of the crucial steps towards achieving this goal. In this work, an interactive game is designed and implemented, which is played with a robot. During the interaction, the user engagement is monitored in realtime via detection of user gaze, turn-taking, laughters/smiles and head nods from audio-visual data. In the experiments conducted, the real-time monitored engagement is found to be consistent with the human-annotated engagement levels.
  • Placeholder
    PublicationMetadata only
    The eHRI database: a multimodal database of engagement in human-robot interactions
    (Springer, 2023) N/A; N/A; N/A; N/A; N/A; Department of Computer Engineering; Department of Computer Engineering; Department of Computer Engineering; Kesim, Ege; Numanoğlu, Tuğçe; Bayramoğlu, Öykü Zeynep; Türker, Bekir Berker; Hussain, Nusrah; Sezgin, Tevfik Metin; Yemez, Yücel; Erzin, Engin; Master Student; Master Student; Master Student; Researcher; PhD Student; Faculty Member; Faculty Member; Faculty Member; Department of Computer Engineering; Koç Üniversitesi İş Bankası Yapay Zeka Uygulama ve Araştırma Merkezi (KUIS AI)/ Koç University İş Bank Artificial Intelligence Center (KUIS AI); Graduate School of Sciences and Engineering; Graduate School of Sciences and Engineering; Graduate School of Sciences and Engineering; N/A; Graduate School of Sciences and Engineering; College of Engineering; College of Engineering; College of Engineering; N/A; N/A; N/A; N/A; N/A; 18632; 107907; 34503
    We present the engagement in human-robot interaction (eHRI) database containing natural interactions between two human participants and a robot under a story-shaping game scenario. The audio-visual recordings provided with the database are fully annotated at a 5-intensity scale for head nods and smiles, as well as with speech transcription and continuous engagement values. In addition, we present baseline results for the smile and head nod detection along with a real-time multimodal engagement monitoring system. We believe that the eHRI database will serve as a novel asset for research in affective human-robot interaction by providing raw data, annotations, and baseline results.
  • Placeholder
    PublicationMetadata only
    Realtime engagement measurement in human-computer interaction
    (Institute of Electrical and Electronics Engineers Inc., 2020) Department of Computer Engineering; Department of Computer Engineering; Department of Computer Engineering; N/A; N/A; N/A; Sezgin, Tevfik Metin; Yemez, Yücel; Erzin, Engin; Türker, Bekir Berker; Numanoğlu, Tuğçe; Kesim, Ege; Faculty Member; Faculty Member; Faculty Member; PhD Student; Master Student; Master Student; Department of Computer Engineering; College of Engineering; College of Engineering; College of Engineering; Graduate School of Sciences and Engineering; Graduate School of Sciences and Engineering; Graduate School of Sciences and Engineering; 18632; 107907; 34503; N/A; N/A; N/A
    Social robots are expected to understand their interlocutors and behave accordingly like humans do. Endowing robots with the capability of monitoring user engagement during their interactions with humans is one of the crucial steps towards achieving this goal. In this work, an interactive game is designed and implemented, which is played with a robot. During the interaction, the user engagement is monitored in realtime via detection of user gaze, turn-taking, laughters/smiles and head nods from audio-visual data. In the experiments conducted, the realtime monitored engagement is found to be consistent with the humanannotated engagement levels.
  • Placeholder
    PublicationMetadata only
    Investigating contributions of speech and facial landmarks for talking head generation
    (Isca-int Speech Communication assoc, 2021) N/A; N/A; Department of Computer Engineering; Kesim, Ege; Erzin, Engin; Master Student; Faculty Member; Department of Computer Engineering; Koç Üniversitesi İş Bankası Yapay Zeka Uygulama ve Araştırma Merkezi (KUIS AI)/ Koç University İş Bank Artificial Intelligence Center (KUIS AI); Graduate School of Sciences and Engineering; College of Engineering; N/A; 34503
    Talking head generation is an active research problem. It has been widely studied as a direct speech-to-video or two stage speech-to-landmarks-to-video mapping problem. in this study, our main motivation is to assess individual and joint contributions of the speech and facial landmarks to the talking head generation quality through a state-of-the-art generative adversarial network (Gan) architecture. incorporating frame and sequence discriminators and a feature matching loss, we investigate performances of speech only, landmark only and joint speech and landmark driven talking head generation on the CREMa-D dataset. Objective evaluations using the peak signal-to-noise ratio (PSNR), structural similarity index (SSIM) and landmark distance (LMD) indicate that while landmarks bring PSNR and SSIM improvements to the speech driven system, speech brings LMD improvement to the landmark driven system. Furthermore, feature matching is observed to improve the speech driven talking head generation models significantly.
  • Thumbnail Image
    PublicationOpen Access
    Investigating contributions of speech and facial landmarks for talking head generation
    (International Speech Communication Association (ISCA), 2021) N/A; Department of Computer Engineering; Kesim, Ege; Erzin, Engin; Faculty Member; Department of Computer Engineering; Graduate School of Sciences and Engineering; College of Engineering; N/A; 34503
    Talking head generation is an active research problem. It has been widely studied as a direct speech-to-video or two stage speech-to-landmarks-to-video mapping problem. In this study, our main motivation is to assess individual and joint contributions of the speech and facial landmarks to the talking head generation quality through a state-of-the-art generative adversarial network (GAN) architecture. Incorporating frame and sequence discriminators and a feature matching loss, we investigate performances of speech only, landmark only and joint speech and landmark driven talking head generation on the CREMA-D dataset. Objective evaluations using the peak signal-to-noise ratio (PSNR), structural similarity index (SSIM) and landmark distance (LMD) indicate that while landmarks bring PSNR and SSIM improvements to the speech driven system, speech brings LMD improvement to the landmark driven system. Furthermore, feature matching is observed to improve the speech driven talking head generation models significantly.
  • Thumbnail Image
    PublicationOpen Access
    X-Ray chest image classification by a small-sized convolutional neural network
    (Institute of Electrical and Electronics Engineers (IEEE), 2019) Dokur, Zümray; Ölmez, Tamer; N/A; Kesim, Ege; Graduate School of Sciences and Engineering
    Convolutional Neural Networks are widely used in image classification problems due to their high performances. Deep learning methods are also used recently in the classification of medical signals or images. It is observed that well-known pre-trained large networks are used in the classification of X-ray chest images. The performances of these networks on the training set are satisfactory, but their practical use includes some difficulties. The usage of the different imaging modalities in the training process decreases the generalization ability of these networks. And also, due to their large sizes, they are not suitable for real-time applications. In this study, new network structures and the size of the input image are investigated for the classification of X-ray chest images. It is observed that chest images are assigned to twelve classes with approximately 86% success rate by using the proposed network, and the training is carried out in a short time due to the small network structure. The proposed network is run as a real time application on an embedded system with a camera and it is observed that the classification result is produced in less than one second.