Link to TOC:  Current module is Planning, Preparing, and Administering Classroom Tests
   
 
   
 

B1. Planning the Test

 

One essential step in planning a test is to decide why you are giving the test. (The word "test" is used although we are using it in a broad sense that includes performance assessments as well as traditional paper and pencil tests.)

        Are you trying to sort the students (so you can compare them, giving higher scores to better students and lower scores to poor students)? If so you will want to include some difficult questions that you expect only a few of the better students will be able to answer correctly. Or do you want to know how many of the students have mastered the content? If your purpose is the latter, you have no need to distribute the scores, so very difficult questions are unnecessary. You will, however, have to decide how many correct answers are needed to demonstrate mastery. Another way to address the "why" question is to identify if this is to be a formative assessment to help you diagnose students' problems and guide future instruction, or a summative measure to determine grades that will be reported to parents.

        Airasian (1994) lists six decisions usually made by the classroom teacher in the test development process: 1. what to test, 2. how much emphasis to give to various objectives, 3. what type of assessment (or type of questions) to use, 4. how much time to allocate for the assessment, 5. how to prepare the students, and 6. whether to use the test from the textbook publisher or to create your own. Other decisions, such as whether to use a separate answer sheet, arise later.

        You, as the teacher, decide what to assess. The term "assess" is used here  Not a link: Current module is Planning, Preparing, and Administering Classroom because the term "assess" is frequently associated only with traditional paper and pencil assessments, to the exclusion of alternative assessments such as performance tasks and portfolios. Classroom assessments are generally focused on content that has been covered in the class, either in the immediate past or (as is the case with unit, semester, and end-of-course tests) over a longer period of time. For example, if we were constructing a test for preservice teachers on writing test questions, we might have the following objectives:

   
 
  The student will:
     
  1. Know the advantages and disadvantages of the major selection-types of questions.
  2. Be able to differentiate between well and poorly written selection-type questions.
  3. Be able to construct appropriate selection-type questions using the guidelines and rules that were presented in class.
   
  We could have listed only the topics we have covered (e.g., true-false questions, short-answer questions, multiple-choice questions, and test format) instead of the objectives.
   
          Now that we have made the what decision, we can move to the next step: deciding how much emphasis to place on each objective. We can look at the amount of time in class we have devoted to each objective. We can also review the number and types of assignments the students have been given. For this example, let's assume that 20% of the assessment will be based on knowing the advantages and disadvantages, 40% will be on differentiating between well written and poorly written questions, and the other 40% will be on writing good questions. Now our planning can be illustrated with the use of a table of specifications (also called a test plan or a test blueprint) as shown in example 1a below.
   
   
 
1a    
Table of Specifications
 
Objectives/Content area/Topics Knowledge Comprehension Application # items/
% of test
1. Know the advantages & disadvantages of the major selection-types of questions.
  
           
20%
2. Be able to differentiate between well and poorly written selection-type questions
  
  
    
40%
3. Be able to construct appropriate selection-type questions using the guidelines and rules that were presented in class.         
  
40%
   
 
Back to Top of Page
          A table of specifications is a two-way table that matches the objectives or content you have taught with the level at which you expect students to perform. It contains an estimate of the percentage of the test to be allocated to each topic at each level at which it is to be measured. In effect we have established how much emphasis to give to each objective or topic.
   
          We also have to take into account the type of thinking skills we wish to assess. Whether you use Bloom's taxonomy or another structure, the levels of learning can help Not a link: Current module is Planning, Preparing, and Administering Classroom you identify the types of questions (or other type of assessment) that are appropriate. For ease of use we have used only three levels: knowledge (recall or recognition), comprehension (or understanding) and application (or skill), and labeled the columns accordingly. The important thing is to use levels of thinking that are relevant for your students and have been incorporated in your instruction. At this stage it can be helpful to mark an "x" or make a check mark in the cells to show the levels at which each objective will be measured, as shown in example 1b.
   
 
1b    
Table of Specifications
 
Objectives/Content area/Topics Knowledge Comprehension Application # items/
% of test
1. Know the advantages & disadvantages of the major selection-types of questions.
x
20%
     
20%
2. Be able to differentiate between well and poorly written selection-type questions
x
x
  
40%
3. Be able to construct appropriate selection-type questions using the guidelines and rules that were presented in class.      

x
40%
40%
   
          We can see that objective 1 (20% of the test) is to be measured totally at the Knowledge level, and objective 3 at the Application level, allowing us to go ahead and enter that information in the Knowledge column for objective 1 and the Application column for objective three in example 1b. Objective 2 is to be measured at two different levels. So we must decide how to divide the 40% of the test for objective two between Knowledge and Comprehension.
   
          We may feel a need to include some questions in which students have to recognize guidelines for writing questions (knowledge) along with examples of good and poor questions from which the students are to select the better constructed questions (comprehension). If we have spent similar amounts of time and emphasis on the two, we may allocate 20% to the Knowledge component and 20% to Comprehension for objective 2 (example 1c).
   
 
1c    
Table of Specifications
 
Objectives/Content area/Topics Knowledge Comprehension Application # items/
% of test
1. Know the advantages & disadvantages of the major selection-types of questions.
x
20%
     
20%
2. Be able to differentiate between well and poorly written selection-type questions
x
20%
x
20%
  
40%
3. Be able to construct appropriate selection-type questions using the guidelines and rules that were presented in class.      

x
40%
40%
   
 
Back to Top of Page
          At this point we recognize that 40% of our test is to be on knowledge, 20% on comprehension, and 40% on application. This does not mean that we must have 40 knowledge questions; it does mean that the score on the test will reflect knowledge and application in equal amounts, and comprehension to a lesser degree than knowledge or application.
   
          It may be that at this point you want to compare the test(s) provided by the textbook publisher with your completed table of specifications. If they match and you think the questions are well written, you may decide to use the test (or parts of the test) provided with the text. On the other hand, you may find that it will be necessary for you to create a test to provide an accurate assessment of what the students in your class have learned.
   
          One question frequently asked is how many questions are needed to adequately sample the content representing an objective or topic. Increasing the number of questions increases the probability that we will have a good estimate of what the child knows and can do.
   
          The number of questions and the type(s) of questions used both affect the amount Not a link: Current module is Planning, Preparing, and Administering Classroom of time needed for completion of the test. Nitko (2001, p. 117), provides some estimates of time to complete various types of questions for junior and senior high school students. Oosterhof (2001, p. 161), gives similar estimates but indicates that elementary students and poor readers might need more time.
   
 
  True-False questions 15-30 seconds per question
  Multiple choice (recall questions that are brief) 30-60 seconds
  More complex multiple choice questions 60-90 seconds
  Multiple choice problems with calculations 2-5 minutes
  Short answer (one word) 30-60 seconds
  Short answer (longer than one word) 1-4 minutes
  Matching (5 premises, 6 responses) 2-4 minutes
  Short essays 15-20 minutes
  Data analyses/graphing 15-25 minutes
  Drawing models/labeling 20-30 minutes
  Extended essays 35-50 minutes
   
          These estimates provide information needed to decide what type(s) of questions and how many of them to use. More true-false questions can be answered during a given period of time than multiple choice or short answer questions. However, our choice of question types must be based on the level of learning at which we are assessing our students. We can decide to use true-false and short-answer questions for the knowledge component, and multiple choice for the comprehension. We will award the lowest number of points per question (1 per question) for the easiest questions (in this case the true-false and short answer). This helps us determine the number of true-false or short answer questions to include, as shown in example 1d.
   
          Measuring objective 3, in our example, may be fairly time-consuming if we provide students with a written selection and ask them to create a number of questions based on that selection. While this is a paper-and-pencil task, it is basically a performance task. The table of specifications can include a mixture of assessments so long as they are appropriate to your objectives and content. We are asking students to demonstrate their ability to do something that our instruction has, hopefully, prepared them to do. Because this is going to take more time than answering questions by selecting answers, we may decide to weight each question constructed by the student at 5 times the number of points given for other questions.
   
 
1d    
Table of Specifications
 
Objectives/Content area/Topics Knowledge Comprehension Application
# items/
% of test
1. Know the advantages & disadvantages of the major selection-types of questions.
x
20%
10 Q x 1
     
20%
10 Q 10 pts
2. Be able to differentiate between well and poorly written selection-type questions
x
20%
10 Q x 1
x
20%
5 Q x 2
  
40%
15 Q 20 pts
3. Be able to construct appropriate selection-type questions using the guidelines and rules that were presented in class.      
x
40%
4 Q x 5
40%
4 Q 20 pts
         
TOTAL
20 Q
5 Q
4 Q
29 Q 50 pts
   
 
Back to Top of Page
          Some authors recommend putting the number of items in each cell, either along with or in place of the percentage. If all questions are weighted equally in the scoring, this is appropriate. If, however, some more difficult item types are given more points than other questions that are more susceptible to guessing (such as true-false), the number of questions can be misleading and should be replaced by the number of points that can be earned. A true-false question might be worth one point while a multiple-choice question or a short-answer question might be worth two points.
   
          In looking at the total number of questions at the bottom of each column, we see that our test needs 20 knowledge questions (that we have decided will be true-false) worth 1 point each, five comprehension questions (multiple choice, 2 points each), and four application tasks (developing questions task, 5 points each). In looking at the numbers of points in the right-hand column, we have a total of 50 points, with 20% for objective 1 and 40% each for objectives 2 and 3. If we have carefully considered what we have been trying to achieve with our students and that is accurately reflected in this table, our test will be a valid measure of student learning and the effectiveness of our instruction.
   
          In estimating the time needed for this test, students would probably need from 5 to 10 minutes for the 20 True-False questions (15-30 seconds each), 5-7 1/2 minutes for the five comprehension questions (60-90 seconds each), and 20-30 minutes (rough estimate) to read the material and write the four questions measuring application. The total time needed would be from 30 to 48 minutes. If you are a middle or high school teacher, estimated response time is an important consideration. You will need to allow enough time for the slowest students to complete your test, and it will need to fit within a single class period.
   
          Another consideration in planning a classroom test may be alignment with standardized tests used in your state to measure similar areas of student learning. How are those tests constructed? What objectives are measured on those tests? How are they measured; i.e., what kinds of items are used and what levels of learning (knowledge, comprehension, application, etc.) are emphasized? On your classroom test you need to measure what you have taught in the ways you have taught it, but in both the teaching and the testing, consider that your work is part of a broader educational system.
   
          The final step in planning the test will be to write the test questions. If more information is needed on item writing, please consult the other modules that correspond to the types of questions of interest to you.
   
   
  Accommodations
   
          Accommodations may be needed for some of your students. It is helpful to keep those students in mind as you plan your assessments. Some examples of accommodations include:
   
 
  Providing written instructions for students with hearing problems
   
  Using large print, reading or recording the questions on audiotape (The student could record the answers on tape.)
   
  Having an aide or assistant write/mark the answers for the student who has coordination problems, or having the student record the answers on audiotape or type the answers
   
  Using written assessments for students with speech problems
   
  Administering the test in sections if the entire test is too long for the attention of a student
   
  Asking the students to repeat the directions to make sure they understand what they are to do
   
  Starting each sentence on a new line helps students identify it as a new sentence
   
  Including an example with each type of question, showing how to mark answers
   
 
Back to Top of Page
   
 

B2. Constructing and Assembling The Test

   
 
        Before beginning to construct your own test, you may want to compare your table of specifications with test items provided by the publisher or other sources to see what, if anything, from those sources can be incorporated into your assessment.
   
* Begin with simpler item types, then proceed to more complex, from easy to difficult, from concrete to abstract. Usually this means going from selection to supply-type items. Selection-type items would usually begin with the most limited selection type (true-false) and progress to multiple choice or matching in which options can be used more than once. The objective is to determine what the student knows. If more difficult items appear early in the test, the student may spend too much time on them and not get to the simpler ones that s/he can answer. For the test we were planning in example 1d of this module, we would begin with true-false, followed in order by short answer, multiple choice, and the performance tasks.
   
* Group items of the same type (true-false, multiple choice, etc.) together so that you only write directions for that item type once. Once you have a good set of directions for a particular type of item, save them so you can use them again the next time you use that same type of item.
   
* Check to see that directions for marking/scoring (point values, etc.) are included with each type of item.
   
* Provide directions for recording responses, and have students circle or underline correct responses when possible rather than writing them to avoid problems arising from poor handwriting.
   
* If a group of items of the same type (multiple choice, etc.) carry over from one page to another, repeat the directions at the top of the second page.
   
* All parts of an item should be on the same page.
   
* If graphs, tables, charts, or illustrations are used, put them near the questions based on them (on the same page, if at all possible).
   
* Check to see that items are independent (one item does not supply the answer or a clue to the answer of another question).
   
* Make sure the reading level is appropriate for your students. (This may be a problem with tests supplied by textbook publishers).
   
* Space the items for easy reading.
   
* Leave appropriate space for writing answers if completion/short answer, listing, or essay questions are used. (Younger children need larger spaces than older students because their print/handwriting is larger.)
   
* When possible, have answers recorded in a column down either the left or right side of the paper to facilitate scoring.
   
* Decide if students are to mark answers on the test, use a separate answer sheet, or use a blank sheet of paper. Usually separate answer sheets are not recommended for students in primary or early elementary grades.
   
* Include on the answer sheet (or on the test if students put answers on the test itself) a place for the student's name and the date.
   
* Make an answer key. (This is easy to do as you write the questions.)
   
* Check the answer key for a response pattern. If necessary, rearrange the order of questions within a question type so the correct answers appear to be in a random order.
   
* Set the test aside for awhile.
   
* Re-read the questions; proofread the test one last time before duplication. If possible, have someone else read the test as well.
   
* Prepare a copy of the test for each student (plus 2 or 3 extra copies). Questions written on the board may cause difficulties for students with visual problems. Reading the test questions to the students (except in the case of spelling tests) can be problematic for students with deficiencies in attention, hearing, comprehension, or short-term memory.
   
* Plan accommodations for individual students when appropriate.
   
 
Back to Top of Page
   
 

B3. Test Administration

   
          A teacher's test administration procedures can have great impact on student test performance. As you will see in the guidelines below, test administration involves more than simply handling out and collecting the test.
   
 
Before the test:
   
* Avoid instilling anxiety
   
* Give as many of the necessary oral directions as possible before distributing the tests, but keep them to a minimum.
   
* Tell students purpose of the test.
   
* Give test-taking hints about guessing, skipping and coming back, etc.
   
* Tell students the amount of time allowed for the test. You may want to put the length of time remaining for the test on the board. This can be changed periodically to help students monitor their progress. If a clock is prominently available, an alternative would be to write the time at which they must be finished.
   
* Tell the students how to signal you if they have a question.
   
* Tell the students what to do with their papers when they are finished (how papers are to be collected).
   
* Tell the students what they are to do when they are finished, particularly if they are to go on to another activity (also write these directions on the chalkboard so they can refer back to them).
   
* Rotate the method of distributing papers so you don't always start from the left or the front row.
   
* Make sure the room is well lighted and has a comfortable temperature.
   
* If a student is absent, write his/her name on a blank copy of the test as a reminder that it needs to be made up.
   
   
After Distributing Test Papers
   
* Remind students to put their names on their papers (and where to do so).
   
* If the test has more than one page, have each student check to see that all pages are there.
   
   
During the Test
   
* Minimize interruptions and distractions.
   
* Avoid giving hints.
   
* Monitor to check student progress and discourage cheating.
   
* Give time warnings if students are not pacing their work appropriately.
   
* Make a note of any questions students ask during the test so that items can be revised for future use.
   
   
After the Test
   
* Grade the papers (and add comments if you can); do test analysis (see the module on test analysis) after scoring and before returning papers to students if at all possible. If it is impossible to do your test analysis before returning the papers, be sure to do it at another time. It is important to both evaluation of your students and improvement of your tests.
   
* If you are recording grades, record them in pencil in your gradebook before returning papers. If there are errors/adjustments in grading, they (grades) are easier to change when recorded in pencil.
   
* Return papers in a timely manner.
   
* Discuss test items with the students. If students have questions, agree to look over their papers again, as well as the papers of others who have the same question. It is usually better not to agree to make changes in grades on the spur of the moment while discussing the tests with the students but to give yourself time to consider what action you want to take. The test analysis may have already alerted you to a problem with a particular question that is common to several students, and you may already have made a decision regarding that question (to disregard the question and reduce the highest possible score accordingly, to give all students credit for that question, etc.).
   
   
  THIS CONCLUDES THE INFORMATION ON PLANNING, PREPARING, AND ADMINISTERING YOUR TESTS, GO TO THE NEXT SECTION TO CHECK YOUR KNOWLEDGE.
 
Back to Top of Page
 
Previous Section: A
Next Section: C
 
Copyright © 2001-2002 Alabama Department of Education
All Rights Reserved.