Publication:
KU_ai at MEDIQA 2019: domain-specific pre-training and transfer learning for medical NLI

dc.contributor.departmentN/A
dc.contributor.departmentDepartment of Computer Engineering
dc.contributor.kuauthorCengiz, Cemil
dc.contributor.kuauthorSert, Ulaş
dc.contributor.kuauthorYüret, Deniz
dc.contributor.kuprofileFaculty Member
dc.contributor.otherDepartment of Computer Engineering
dc.contributor.schoolcollegeinstituteGraduate School of Sciences and Engineering
dc.contributor.schoolcollegeinstituteCollege of Engineering
dc.contributor.yokidN/A
dc.contributor.yokidN/A
dc.contributor.yokid179996
dc.date.accessioned2024-11-09T11:50:26Z
dc.date.issued2019
dc.description.abstractIn this paper, we describe our system and results submitted for the Natural Language Inference (NLI) track of the MEDIQA 2019 Shared Task (Ben Abacha et al., 2019). As KU ai team, we used BERT (Devlin et al., 2018) as our baseline model and pre-processed the MedNLI dataset to mitigate the negative impact of de-identification artifacts. Moreover, we investigated different pre-training and transfer learning approaches to improve the performance. We show that pre-training the language model on rich biomedical corpora has a significant effect in teaching the model domain-specific language. In addition, training the model on large NLI datasets such as MultiNLI and SNLI helps in learning task-specific reasoning. Finally, we ensembled our highest-performing models, and achieved 84.7% accuracy on the unseen test dataset and ranked 10th out of 17 teams in the official results.
dc.description.fulltextYES
dc.description.indexedbyWoS
dc.description.openaccessYES
dc.description.publisherscopeInternational
dc.description.sponsoredbyTubitakEuN/A
dc.description.sponsorshipHuawei Turkey R&D Center, Huawei Graduate Research Support Scholarship
dc.description.versionPublisher version
dc.formatpdf
dc.identifier.doi10.18653/v1/W19-5045
dc.identifier.embargoNO
dc.identifier.filenameinventorynoIR02193
dc.identifier.isbn9781950737284
dc.identifier.linkhttps://doi.org/10.18653/v1/W19-5045
dc.identifier.quartileN/A
dc.identifier.urihttps://hdl.handle.net/20.500.14288/678
dc.identifier.wos521946800045
dc.languageEnglish
dc.publisherAssociation for Computational Linguistics (ACL)
dc.relation.grantnoNA
dc.relation.urihttp://cdm21054.contentdm.oclc.org/cdm/ref/collection/IR/id/8833
dc.sourceProceedings of the BioNLP 2019 Workshop
dc.subjectComputer science, artificial intelligence
dc.subjectMedical informatics
dc.titleKU_ai at MEDIQA 2019: domain-specific pre-training and transfer learning for medical NLI
dc.typeConference proceeding
dspace.entity.typePublication
local.contributor.authoridN/A
local.contributor.authoridN/A
local.contributor.authorid0000-0002-7039-0046
local.contributor.kuauthorCengiz, Cemil
local.contributor.kuauthorSert, Ulaş
local.contributor.kuauthorYüret, Deniz
relation.isOrgUnitOfPublication89352e43-bf09-4ef4-82f6-6f9d0174ebae
relation.isOrgUnitOfPublication.latestForDiscovery89352e43-bf09-4ef4-82f6-6f9d0174ebae

Files

Original bundle

Now showing 1 - 1 of 1
Thumbnail Image
Name:
8833.pdf
Size:
365.12 KB
Format:
Adobe Portable Document Format