Publications without Fulltext
Permanent URI for this collectionhttps://hdl.handle.net/20.500.14288/3
Browse
202 results
Search Results
Publication Metadata only Self-supervised object-centric learning for videos(Neural information processing systems foundation, 2023) Xie, Weidi; Department of Computer Engineering; Aydemir, Görkay; Güney, Fatma; Department of Computer Engineering; Graduate School of Sciences and Engineering; College of EngineeringUnsupervised multi-object segmentation has shown impressive results on images by utilizing powerful semantics learned from self-supervised pretraining. An additional modality such as depth or motion is often used to facilitate the segmentation in video sequences. However, the performance improvements observed in synthetic sequences, which rely on the robustness of an additional cue, do not translate to more challenging real-world scenarios. In this paper, we propose the first fully unsupervised method for segmenting multiple objects in real-world sequences. Our object-centric learning framework spatially binds objects to slots on each frame and then relates these slots across frames. From these temporally-aware slots, the training objective is to reconstruct the middle frame in a high-level semantic feature space. We propose a masking strategy by dropping a significant portion of tokens in the feature space for efficiency and regularization. Additionally, we address over-clustering by merging slots based on similarity. Our method can successfully segment multiple instances of complex and high-variety classes in YouTube videos.Publication Metadata only Developing a multimodal classroom engagement analysis dashboard for higher-education(Association for Computing Machinery, 2023) Sabuncuoglu, Alpay; Department of Computer Engineering; Sezgin, Tevfik Metin; Department of Computer Engineering; College of EngineeringDeveloping learning analytics dashboards (LADs) is a growing research interest as online learning tools have become more accessible in K-12 and higher education settings. This paper reports our multimodal classroom engagement data analysis and dashboard design process and the resulting engagement dashboard. Our work stems from the importance of monitoring classroom engagement, which refers to students' active physical and cognitive involvement in learning that influences their motivation and success in a given course. To monitor this vital facade of learning, we developed an engagement dashboard using an iterative and user-centered process. We first created a multimodal machine learning model that utilizes face and pose features obtained from recent deep learning models. Then, we created a dashboard where users can view their engagement over time and discover their learning/teaching patterns. Finally, we conducted user studies with undergraduate and graduate-level participants to obtain feedback on our dashboard design. Our paper makes three contributions by (1) presenting a student-centric, open-source dashboard, (2) demonstrating a baseline architecture for engagement analysis using our open-Access data, and (3) presenting user insights and design takeaways to inspire future LADs. We expect our research to guide the development of tools for novice teacher education, student self-evaluation, and engagement evaluation in crowded classrooms.Publication Metadata only Snoopie: a multi-GPU communication profiler and visualizer(Assoc Computing Machinery, 2024) Department of Computer Engineering; Issa, Mohammad Kefah Taha; Sasongko, Muhammad Aditya; Turimbetov, İlyas; Baydamirli, Javid; Sağbili, Doğan; Erten, Didem Unat; Department of Computer Engineering; Graduate School of Sciences and Engineering; College of EngineeringWith data movement becoming one of the most expensive bottlenecks in computing, the need for profiling tools to analyze communication becomes crucial for effectively scaling multi-GPU applications. While existing profiling tools including first-party software by GPU vendors are robust and excel at capturing compute operations within a single GPU, support for monitoring GPU-GPU data transfers and calls issued by communication libraries is currently inadequate. To fill these gaps, we introduce Snoopie, an instrumentation-based multi-GPU communication profiling tool built on NVBit, capable of tracking peer-to-peer transfers and GPU-centric communication library calls. To increase programmer productivity, Snoopie can attribute data movement to the source code line and the data objects involved. It comes with multiple visualization modes at varying granularities, from a coarse view of the data movement in the system as a whole to specific instructions and addresses. Our case studies demonstrate Snoopie's effectiveness in monitoring data movement, locating performance bugs in applications, and understanding concrete data transfers abstracted beneath communication libraries. The tool is publicly available at https://github.com/ParCoreLab/snoopie.Publication Metadata only Bringing order to sparsity: a sparse matrix reordering study on multicore CPUs(Association for Computing Machinery, Inc, 2023) Trotter, James D.; Ekmekçibaşı, Sinan; Langguth, Johannes; Ilic, Aleksandar; Department of Computer Engineering; Torun, Tuğba; Düzakın, Emre; Erten, Didem Unat; Department of Computer Engineering; College of Engineering; Graduate School of Sciences and EngineeringMany real-world computations involve sparse data structures in the form of sparse matrices. A common strategy for optimizing sparse matrix operations is to reorder a matrix to improve data locality. However, it's not always clear whether reordering will provide benefits over the unordered matrix, as its effectiveness depends on several factors, such as structural features of the matrix, the reordering algorithm and the hardware that is used. This paper aims to establish the relationship between matrix reordering algorithms and the performance of sparse matrix operations. We thoroughly evaluate six different matrix reordering algorithms on 490 matrices across eight multicore architectures, focusing on the commonly used sparse matrix-vector multiplication (SpMV) kernel. We find that reordering based on graph partitioning provides better SpMV performance than the alternatives for a large majority of matrices, and that the resulting performance is explained through a combination of data locality and load balancing concerns. © 2023 Owner/Author(s).Publication Metadata only FLAGS simulation framework for federated learning algorithms(Institute of Electrical and Electronics Engineers Inc., 2023) Department of Computer Engineering; Lodhi, Ahnaf Hannan; Shamsizade, Toghrul; Al Asaad, Omar Mohammad; Akgün, Barış; Özkasap, Öznur; Department of Computer Engineering; Graduate School of Sciences and Engineering; College of EngineeringFederated Learning (FL) provides an effective mechanism for distributed learning. However, it is expected to operate in a highly diverse setting with distinct behaviors from the participating nodes as well as dynamic network conditions. The FL performance, therefore, is subject to change due to the highly transitory nature of the overall system. An efficient simulation framework must be flexible to allow a range of participant behaviors, interactions, and environment characteristics. In this demo paper, we present the Federated Learning Algorithm Simulation (FLAGS) framework that we propose as a lightweight FL implementation and testing platform. FLAGS framework allows for a wide range of device behaviors and cooperative mechanisms, enabling rapid testing of multiple FL algorithms. © 2023 IEEE.Publication Metadata only Neurocache: Efficient Vector Retrieval for Long-range Language Modeling(Association for Computational Linguistics (ACL), 2024) Department of Computer Engineering; Safaya, Ali; Yüret, Deniz; Department of Computer Engineering; Koç Üniversitesi İş Bankası Yapay Zeka Uygulama ve Araştırma Merkezi (KUIS AI)/ Koç University İş Bank Artificial Intelligence Center (KUIS AI); Graduate School of Sciences and Engineering; College of EngineeringThis paper introduces Neurocache, an approach to extend the effective context size of large language models (LLMs) using an external vector cache to store its past states. Like recent vector retrieval approaches, Neurocache uses an efficient k-nearest-neighbor (kNN) algorithm to retrieve relevant past states and incorporate them into the attention process. Neurocache improves upon previous methods by (1) storing compressed states, which reduces cache size;(2) performing a single retrieval operation per token which increases inference speed;and (3) extending the retrieval window to neighboring states, which improves both language modeling and downstream task accuracy. Our experiments show the effectiveness of Neurocache both for models trained from scratch and for pre-trained models such as Llama2-7B and Mistral-7B when enhanced with the cache mechanism. We also compare Neurocache with text retrieval methods and show improvements in single-document question-answering and few-shot learning tasks. We made the source code available under: https://github.com/alisafaya/neurocache © 2024 Association for Computational Linguistics.Publication Metadata only Gamu blue: a practical tool for game theory security equilibria(Institute of Electrical and Electronics Engineers Inc., 2024) Yıldız, Burcu; Department of Computer Engineering; Taweel, Ameer; Küpçü, Alptekin; Department of Computer Engineering; College of EngineeringThe application of game theory in cybersecurity enables strategic analysis, adversarial modeling, and optimal decision-making to address security threats' complex and dynamic nature. Previous studies by Abraham et al. and Biçer et al. presented various definitions of equilibria to examine the security aspects of games involving multiple parties. Nonetheless, these definitions lack practical and easy-to-use implementations. Our primary contribution is addressing this gap by developing Gamu Blue, an easy-to-use tool with implementations for computing the equilibria definitions including k-resiliency, l-repellence, t-immunity, (l, t)-resistance, and m-stability. © 2024 IEEE.Publication Metadata only GPU-initiated resource allocation for irregular workloads(Assoc Computing Machinery, 2024) Department of Computer Engineering; Turimbetov, İlyas; Sasongko, Muhammad Aditya; Erten, Didem Unat; Department of Computer Engineering; College of EngineeringGPU kernels may suffer from resource underutilization in multi-GPU systems due to insufficient workload to saturate devices when incorporated within an irregular application. To better utilize the resources in multi-GPU systems, we propose a GPU-sided resource allocation method that can increase or decrease the number of GPUs in use as the workload changes over time. Our method employs GPU-to-CPU callbacks to allowGPU device(s) to request additional devices while the kernel execution is in flight. We implemented and tested multiple callback methods required for GPU-initiated workload offloading to other devices and measured their overheads on Nvidia and AMD platforms. To showcase the usage of callbacks in irregular applications, we implemented Breadth-First Search (BFS) that uses device-initiated workload offloading. Apart from allowing dynamic device allocation in persistently running kernels, it reduces time to solution on average by 15.7% at the cost of callback overheads with a minimum of 6.50 microseconds on AMD and 4.83 microseconds on Nvidia, depending on the chosen callback mechanism. Moreover, the proposed model can reduce the total device usage by up to 35%, which is associated with higher energy efficiency.Publication Metadata only Learning markerless robot-depth camera calibration and end-effector pose estimation(Ml Research Press, 2023) Department of Computer Engineering; Sefercik, Buğra Can; Akgün, Barış; Department of Computer Engineering; Koç Üniversitesi İş Bankası Yapay Zeka Uygulama ve Araştırma Merkezi (KUIS AI)/ Koç University İş Bank Artificial Intelligence Center (KUIS AI); College of Engineering; Graduate School of Sciences and EngineeringTraditional approaches to extrinsic calibration use fiducial markers and learning-based approaches rely heavily on simulation data. In this work, we present a learning-based markerless extrinsic calibration system that uses a depth camera and does not rely on simulation data. We learn models for end-effector (EE) segmentation, single-frame rotation prediction and keypoint detection, from automatically generated real-world data. We use a transformation trick to get EE pose estimates from rotation predictions and a matching algorithm to get EE pose estimates from keypoint predictions. We further utilize the iterative closest point algorithm, multiple-frames, filtering and outlier detection to increase calibration robustness. Our evaluations with training data from multiple camera poses and test data from previously unseen poses give sub-centimeter and sub-deciradian average calibration and pose estimation errors. We also show that a carefully selected single training pose gives comparable results. © 2023 Proceedings of Machine Learning Research. All rights reserved.Publication Metadata only Have we ever encountered this before? Retrieving out-of-distribution road obstacles from driving scenes(Institute of Electrical and Electronics Engineers Inc., 2024) Shoeb, Y.; Chan, R.; Schwalbe, G.; Nowzad, A.; Gottschalk, H.; Department of Computer Engineering; Güney, Fatma; Department of Computer Engineering; College of EngineeringIn the life cycle of highly automated systems operating in an open and dynamic environment, the ability to adjust to emerging challenges is crucial. For systems integrating data-driven AI-based components, rapid responses to deployment issues require fast access to related data for testing and reconfiguration. In the context of automated driving, this especially applies to road obstacles not included in the training data, commonly referred to as out-of-distribution (OoD) road obstacles. Given the availability of large uncurated driving scene recordings, a pragmatic approach is to query a database to retrieve similar scenarios featuring the same safety concerns due to OoD road obstacles. In this work, we extend beyond identifying OoD road obstacles in video streams and offer a comprehensive approach to extract sequences of OoD road obstacles using text queries, thereby proposing a way of curating a collection of OoD data for subsequent analysis. Our proposed method leverages the recent advances in OoD segmentation and multi-modal foundation models to identify and efficiently extract safety-relevant scenes from unlabeled videos. We present a first approach for the novel task of text-based OoD object retrieval, which addresses the question "Have we ever encountered this before?". © 2024 IEEE.