Research Project: Interval Timing, Decision Making, and Reward Maximization
Loading...
Contributors
Funders
ID
EC.00025
Authors
Balcı, Fuat
Faculty Member
Publications
Time-based reward maximization
(Royal Society of London, 2014) Balcı, Fuat; Çavdaroğlu, Bilgehan; Zeki, Mustafa; Department of Psychology; Yes; College of Social Sciences and Humanities
Humans and animals time intervals from seconds to minutes with high accuracy but limited precision. Consequently, time-based decisions are inevitably subjected to our endogenous timing uncertainty, and thus require temporal risk assessment. In this study, we tested temporal risk assessment ability of humans when participants had to withhold each subsequent response for a minimum duration to earn reward and each response reset the trial time. Premature responses were not penalized in Experiment 1 but were penalized in Experiment 2. Participants tried to maximize reward within a fixed session time (over eight sessions) by pressing a key. No instructions were provided regarding the task rules/parameters. We evaluated empirical performance within the framework of optimality that was based on the level of endogenous timing uncertainty and the payoff structure. Participants nearly tracked the optimal target inter-response times (IRTs) that changed as a function of the level of timing uncertainty and maximized the reward rate in both experiments. Acquisition of optimal target IRT was rapid and abrupt without any further improvement or worsening. These results constitute an example of optimal temporal risk assessment performance in a task that required finding the optimal trade-off between the 'speed' (timing) and 'accuracy' (reward probability) of timed responses for reward maximization.
Optimal response rates in humans and rats
(American Psychological Association Inc., 2015) Balcı, Fuat; Freestone, David M.; Simen, Patrick; Church, Russell M.; Department of Psychology; KUTTAM (Koç University Research Center for Translational Medicine); Yes; College of Social Sciences and Humanities; Research Center
The analysis of response rates has been highly influential in psychology, giving rise to many prominent theories of learning. There is, however, growing interest in explaining response rates, not as a global response to associations or value, but as a decision about how to space responses in time. Recently, researchers have shown that humans and mice can time a single response optimally; that is, in a way that maximizes reward. Here, we use the well-established differential reinforcement of low rates (DRL) timing task to show that humans and rats come close to optimizing reinforcement rate, but respond systematically faster than they should.
Timescale invariance in the pacemaker-accumulator family of timing models
(Brill Academic Publishers, 2013) Balcı, Fuat; Simen, Patrick; Rivest, Francois; Ludvig, Elliot A.; Killeen, Peter; Department of Psychology; Yes; College of Social Sciences and Humanities
Pacemaker-accumulator (PA) systems have been the most popular kind of timing model in the half-century since their introduction by Treisman (1963). Many alternative timing models have been designed predicated on different abumptions, though the dominant PA model during this period-Gibbon and Church's Scalar Expectancy Theory (SET)-invokes most of them. As in Treisman, SET's implementation abumes a fixed-rate clock-pulse generator and encodes durations by storing average pulse counts; unlike Treisman's model, SET's decision proceb invokes Weber's law of magnitude-comparison to account for timescale-invariant temporal precision in animal behavior. This is one way to deal with the 'Poibon timing' ibue, in which relative temporal precision increases for longer durations, contrafactually, in a simplified version of Treisman's model. First, we review the fact that this problem does not afflict Treisman's model itself due to a key abumption not shared by SET. Second, we develop a contrasting PA model, an extension of Killeen and Fetterman's Behavioral Theory of Timing that accumulates Poibon pulses up to a fixed criterion level, with pulse rates adapting to time different intervals. Like Treisman's model, this time-adaptive, opponent Poibon, drift-diffusion model accounts for timescale invariance without first abuming Weber's law. It also makes new predictions about response times and learning speed and connects interval timing to the popular drift-diffusion model of perceptual decision making. With at least three different routes to timescale invariance, the PA model family can provide a more compelling account of timed behavior than may be generally appreciated.
Interval timing by long-range temporal integration
(Frontiers, 2011) Balcı, Fuat; Simen, Patrick; deSouza, Laura; Cohen, Jonathan D.; Holmes, Philip; Department of Psychology; Yes; College of Social Sciences and Humanities
Optimal temporal risk assessment
(Frontiers, 2011) Balcı, Fuat; Freestone, David; Simen, Patrick; deSouza, Laura; Cohen, Jonathan D.; Holmes, Philip; Department of Psychology; Yes; College of Social Sciences and Humanities
Time is an essential feature of most decisions, because the reward earned from decisions frequently depends on the temporal statistics of the environment (e.g., on whether deci- sions must be made under deadlines). Accordingly, evolution appears to have favored a mechanism that predicts intervals in the seconds to minutes range with high accuracy on average, but significant variability from trial to trial. Importantly, the subjective sense of time that results is sufficiently imprecise that maximizing rewards in decision-making can require substantial behavioral adjustments (e.g., accumulating less evidence for a decision in order to beat a deadline). Reward maximization in many daily decisions therefore requires optimal temporal risk assessment. Here, we review the temporal decision-making litera- ture, conduct secondary analyses of relevant published datasets, and analyze the results of a new experiment. The paper is organized in three parts. In the first part, we review literature and analyze existing data suggesting that animals take account of their inherent behavioral variability (their “endogenous timing uncertainty”) in temporal decision-making. In the second part, we review literature that quantitatively demonstrates nearly optimal temporal risk assessment with sub-second and supra-second intervals using perceptual tasks (with humans and mice) and motor timing tasks (with humans). We supplement this section with original research that tested human and rat performance on a task that requires finding the optimal balance between two time-dependent quantities for reward maximization. This optimal balance in turn depends on the level of timing uncertainty. Cor- roborating the reviewed literature, humans and rats exhibited nearly optimal temporal risk assessment in this task. In the third section, we discuss the role of timing uncertainty in reward maximization in two-choice perceptual decision-making tasks and review literature that implicates timing uncertainty as an important factor in performance quality. Together, these studies strongly support the hypothesis that animals take normative account of their endogenous timing uncertainty. By incorporating the psychophysics of interval timing into the study of reward maximization, our approach bridges empirical and theoretical gaps between the interval timing and decision-making literatures.
