Does administration format influence the construct validity of SAT comprehension passages?

The Scholastic Achievement Test (i.e. SAT) is arguably one of the most important assessment measures that a student takes. This study examined whether changing the administration format of the reading comprehension passages of the VSAT increased the construct validity of the passages. Using an individual differences approach for 51 undergraduates (18 males), we tested whether prohibiting students from referring to passages while answering the accompanying multiple-choice questions tapped the higher-level cognitive processes of working memory and epistemic belief more so than the standard administration format that allowed students to refer to passages while answering the accompanying questions. The overall results of the study suggest that changing the administration format of the VSAT passages changed the underlying factors of reading comprehension being assessed. However, we were unable to reliably identify the higher-level cognitive processes of reading comprehension tapped by each administration format.