Research Outputs

Permanent URI for this communityhttps://hdl.handle.net/20.500.14288/2

Browse

Filters

2020 - 20212

Advanced Search

Filter by

Settings

Full Text: YesSubject: search.filters.subject.Computer architecture

Search Results

Now showing 1 - 2 of 2
  • Thumbnail Image
    PublicationRestricted
    Foveated near-eye display using computational holography
    (Koç University, 2020) Cem, Ali; Ürey, Hakan; 0000-0002-2031-7967; Koç University Graduate School of Sciences and Engineering; Electrical and Electronics Engineering; 8579
  • Thumbnail Image
    PublicationOpen Access
    Investigating contributions of speech and facial landmarks for talking head generation
    (International Speech Communication Association (ISCA), 2021) N/A; Department of Computer Engineering; Kesim, Ege; Erzin, Engin; Faculty Member; Department of Computer Engineering; Graduate School of Sciences and Engineering; College of Engineering; N/A; 34503
    Talking head generation is an active research problem. It has been widely studied as a direct speech-to-video or two stage speech-to-landmarks-to-video mapping problem. In this study, our main motivation is to assess individual and joint contributions of the speech and facial landmarks to the talking head generation quality through a state-of-the-art generative adversarial network (GAN) architecture. Incorporating frame and sequence discriminators and a feature matching loss, we investigate performances of speech only, landmark only and joint speech and landmark driven talking head generation on the CREMA-D dataset. Objective evaluations using the peak signal-to-noise ratio (PSNR), structural similarity index (SSIM) and landmark distance (LMD) indicate that while landmarks bring PSNR and SSIM improvements to the speech driven system, speech brings LMD improvement to the landmark driven system. Furthermore, feature matching is observed to improve the speech driven talking head generation models significantly.