Development and validation of the visual reasoning test for the NIH toolbox

Abstract

The Cognition Battery of the NIH Toolbox (NIHTB) is an iPad-based set of brief assessments with documented test score reliability and validity, covering a range of cognitive functions for ages 3-85+. The construct of visual reasoning (VR) was identified as a measurement gap in previous versions of the NIHTB by a contingent of NIH-sponsored researchers. The VR Test was developed de novo to address this gap across the entire age spectrum. Given the intended brevity of NIHTB measures, VR was designed for computer-adaptive test administration. VR items were calibrated on 3,768 participants from the NIHTB Version 3 norming study. To evaluate convergent validity of the derived test score, a subsample of 283 participants was administered the age-appropriate Wechsler Intelligence Scale, including the commonly used Matrix Reasoning subtest to assess VR. A separate subsample of 190 individuals was readministered VR 1-14 days later to evaluate test-retest reliability of VR scores. Results yielded a robust bank of 180 items measuring a wide range on the ability spectrum, with correlations of .45-.55 with Wechsler Matrix Reasoning tests and .48-.65 with Wechsler Full-Scale IQ scores. Test-retest intraclass correlation coefficient for VR scores was .77. The new NIHTB VR Test is a computer-adaptive test that can be used to assess VR from preschool to older adulthood, showing evidence of convergent validity of test scores with those of similar constructs, and test-retest reliability of scores, as well as an overall strong relationship to general cognitive ability. This measure broadens the scope of the NIHTB.