Researcher:
Runyun, Şerife Leman

Profile Picture
ORCID

Job Title

PhD Student

First Name

Şerife Leman

Last Name

Runyun

Name

Name Variants

Runyun, Şerife Leman

Email Address

Birth Date

Filters

20222

Advanced Search

Filter by

Settings

Researcher Of Publications: search.filters.isAuthorOfPublication.63f63a70-c9c2-4e8e-8b9b-027c280c5c1e

Search Results

Now showing 1 - 2 of 2
  • Placeholder
    PublicationMetadata only
    On measuring social biases in prompt-based multi-task learning
    (Association for Computational Linguistics (ACL), 2022) Akyürek, Afra Feyza; Paik, Sejin; Koçyiğit, Muhammed Yusuf; Akbıyık, Seda; Wijaya, Derry; N/A; Runyun, Şerife Leman; PhD Student; Koç University Research Center for Translational Medicine (KUTTAM) / Koç Üniversitesi Translasyonel Tıp Araştırma Merkezi (KUTTAM); Graduate School of Social Sciences and Humanities; N/A
    Large language models trained on a mixture of NLP tasks that are converted into a textto- text format using prompts, can generalize into novel forms of language and handle novel tasks. A large body of work within prompt engineering attempts to understand the effects of input forms and prompts in achieving superior performance. We consider an alternative measure and inquire whether the way in which an input is encoded affects social biases promoted in outputs. In this paper, we study T0, a large-scale multi-task text-to-text language model trained using prompt-based learning. We consider two different forms of semantically equivalent inputs: question-answer format and premise-hypothesis format. We use an existing bias benchmark for the former BBQ (Parrish et al., 2021) and create the first bias benchmark in natural language inference BBNLI with hand-written hypotheses while also converting each benchmark into the other form. The results on two benchmarks suggest that given two different formulations of essentially the same input, T0 conspicuously acts more biased in question answering form, which is seen during training, compared to premisehypothesis form which is unlike its training examples.
  • Thumbnail Image
    PublicationOpen Access
    On measuring social biases in prompt-based multi-task learning
    (Association for Computational Linguistics (ACL), 2022) Akyürek; A.F.; Paik, S.; Koçyiğit, M.Y.; Akbıyık, S.; Wijaya, D.; Department of Psychology; Runyun, Şerife Leman; PhD Student; Department of Psychology; Graduate School of Social Sciences and Humanities
    Large language models trained on a mixture of NLP tasks that are converted into a textto- text format using prompts, can generalize into novel forms of language and handle novel tasks. A large body of work within prompt engineering attempts to understand the effects of input forms and prompts in achieving superior performance. We consider an alternative measure and inquire whether the way in which an input is encoded affects social biases promoted in outputs. In this paper, we study T0, a large-scale multi-task text-to-text language model trained using prompt-based learning. We consider two different forms of semantically equivalent inputs: question-answer format and premise-hypothesis format. We use an existing bias benchmark for the former BBQ (Parrish et al., 2021) and create the first bias benchmark in natural language inference BBNLI with hand-written hypotheses while also converting each benchmark into the other form. The results on two benchmarks suggest that given two different formulations of essentially the same input, T0 conspicuously acts more biased in question answering form, which is seen during training, compared to premisehypothesis form which is unlike its training examples.