The Two-Questions Method of Word Meaning Assessment: A Group Procedure for Use with Preliterate Children (Kearns & Biemiller)

This post is courtesy of Gail Kearns and Andrew Biemiller. Gail Kearns is a reading specialist and former teacher who founded a diagnostic and remedial reading clinic in Concord, Massachusetts. She worked for Jeanne Chall at Harvard University as a research assistant on readability projects and on the first edition of the Diagnostic Assessments of Reading (DAR); more recently she co-authored the Second Edition of the DAR. Gail also co-authored Trial Teaching Strategies (TTS), a useful tool for planning tutorials and interventions.  Andrew Biemiller, Professor Emeritus, University of Toronto, is a vocabulary researcher and the author of Words Worth Teaching: Closing the Vocabulary Gap. Andrew is advisor to the WGBH television program for young children, Martha Speaks. Please see  Andrew's prior post describing this educational and yet entertaining PBS program. Andrew is a national expert in reading and language comprehension, and has also investigated self-direction in schools. Gail and Andrew have collaborated together on vocabulary research, described below.

For too long, classroom teachers and researchers have been unable to assess vocabulary reliably and efficiently with classes of preliterate children—young children who either can’t read or can’t read reliably. These difficulties have made individual assessment of word meaning with such students a necessity. We believe the amount of time that has been needed for assessment is partly why educators have tended to neglect vocabulary with children in the primary grades and also why so few word meaning studies have been able to be conducted in these grades.

Young children’s general vocabulary level has been assessed to date with tests such as the well-known Peabody Picture Vocabulary Test (PPVT). However, this test, like other such vocabulary tests for students in these grades, requires individual assessment, taking 15 to 25 minutes per student. This is more time than classroom teachers can spend with each child, particularly for regular assessment at intervals following instruction. Furthermore, the PPVT is largely limited to assessment of concrete meanings, i.e., word meanings that can be readily pictured.

We have recently published a study of our method for group testing vocabulary word meanings that was conducted with 257 students in grades K, 1, and 2 (Kearns & Biemiller, 2010). The results of our validation study demonstrated that results similar to individual PPVT assessments could also be obtained with our group method. (Correlations between overall children’s PPVT raw scores and scores from our group test were r = .76 in Kindergarten, r = .71 in Grade One, and r = .70 in Grade Two. Correlations between word mean scores from the two methods were .76 (Kindergarten), .79 (Grade 1), and .94 (Grade 2). In short, these results indicate that teachers could use this method both to determine overall vocabulary needs in classrooms, and to monitor the acquisition of word meanings and success of vocabulary instruction. Researchers could use this new group method to shorten the amount of time needed to conduct vocabulary acquisition studies in these grades.

In this entry, we want to briefly describe how the Two-Questions Vocabulary method is used for assessment of word meanings in the primary grades. Additional details for conducting such assessments are available from the authors and in our journal article about the study.

When constructing a Two-Questions assessment, each word should be assessed with two short questions, each of which can be answered with either “yes” or “no,” and with one “yes” question and one “no” question used for each word. For a child to be credited with knowledge of a word meaning, both questions have to be answered correctly. (When this is done, the odds of guessing a correct answer is the same as on the PPVT, where the student must choose 1 of 4 possible alternative pictures.)

With the exception of the assessed word, all other words in the questions are of high text frequency, and the questions are kept simple (no complex clauses, etc.). The two questions for each word are placed in random order on the test; they never follow each other and are kept as far apart as possible. In the study, we used one of the questions for each word in each test session. We chose equal numbers of “yes” and “no” questions (and arranged them in random order) for each test session.

Here are some examples of Two-Questions items taken from a test administrator’s form. (Words in these sample items have not been taken from the copyrighted PPVT test.)

Items from a sample administrator’s test form. (© Kearns & Biemiller, 2010). Graphics
used are from collection of 6,000 “Literacy Support Pictures,” courtesy Slater Software.

The authors strongly recommend use of a Student Response form, particularly when assessing kindergartners. Children are asked to circle a “yes” or “no” alternative for each item. We found that use of 5”x 7” cards for tracking items were also needed with kindergartners and for many first graders. Details of how best to use the cards and Response Sheets are available in the article or from the authors.

Sample items from a Student Response Sheet. (© Kearns & Biemiller, 2010). Graphics
used are from collection of 6,000 “Literacy Support Pictures,” courtesy Slater Software.

How long does administering the Two-Questions tests take? In our experience, the first time, we recommend two sessions on different days be used for assessing approximately 20 words. The first session will take about 45 minutes, with at least half of the session used to introduce the Response Sheet and to practice answering sample test items. The second session, conducted on a different day, should take 15 to 20 minutes. Second graders may be able to be assessed in a single session of about 50 minutes.

Later in the year, after the group assessment method has been used earlier with a classroom, and after the students have become familiar with how these tests are conducted, additional word meaning assessments for instructional purposes with the Two-Questions method may be able to be completed with single sessions of about 30 minutes for 20 words.

During the course of the study, we came to believe that an additional separate initial group session for training purposes would have been desirable for the kindergarten children and should be built into other studies involving group testing of word meanings in kindergarten.

In conclusion, we believe that this method can be used to create items for both assessing general vocabulary knowledge and for regularly testing samples of word meanings taught throughout the school year in the classroom. (We recommend teaching word meanings in the context of stories read aloud by the teacher and through other on-going classroom activities. Methods of teaching are discussed in Beck, McKeown, & Kucan (2002 and 2008), Graves (2006 and 2009), Stahl & Nagy (2006), Blachowicz & Fisher (2006), and Biemiller (2010). Ways of choosing words for instruction are discussed in the Beck et al and Biemiller books.

References
Beck, I. L., McKeown, M. G., & Kucan, L. (2002). Bringing Words to Life: Robust Vocabulary Instruction. New York: NY, Guilford Press Inc.

Beck, I. L., McKeown, M. G., & Kucan, L. (2008). Creating Robust Vocabulary: Frequently Asked Questions and Expanded Examples. New York: NY, Guilford Press Inc.

Biemiller, A. (2010). Words Worth Teaching. Columbus, OH: SRA/McGraw-Hill.

Blachowicz & Fisher (2009). Teaching Vocabulary in All Classrooms. Boston, MA: Allyn & Bacon.

Graves, M. F. (2006). The Vocabulary Book: Learning and Instruction. New York, NY: Teachers College Press.

Graves, M. F. (2009). Teaching Individual Words: One Size Does Not Fit All. New York, NY: Teachers College Press.

Kearns, G., & Biemiller, A. (2010). Two-Questions vocabulary assessment: Developing a new method for group testing in kindergarten through second grade. Journal of Education, 190 (1/2), 31-41.

Stahl, S. A. & Nagy, W. E. (2006). Teaching Word Meanings. Mahwah, NJ: Lawrence Erlbaum Associates.