SlideShare a Scribd company logo

Test item writing and analysis

•Download as PPT, PDF•
•
Melanio Florino
Melanio Florino

The document discusses test construction and item analysis. It provides illustrations of sampling populations and characteristics. It presents a three-stage model of classroom measurement that emphasizes content validity from objectives to test items. It also discusses research designs, thinking skills, and sample tables of test specification. The document provides examples of multiple choice item types and guidelines for writing multiple choice items.

Related slideshows

Test construction
Test constructionTest construction
Test construction
 
Validity (Educational Assessment)
Validity (Educational Assessment)Validity (Educational Assessment)
Validity (Educational Assessment)
 
Classical Test Theory (CTT)- By Dr. Jai Singh
Classical Test Theory (CTT)- By Dr. Jai SinghClassical Test Theory (CTT)- By Dr. Jai Singh
Classical Test Theory (CTT)- By Dr. Jai Singh
 
1 of 65
Test item writing and analysis
September 10, 2011 UIC TEST ITEM WRITING and ITEM ANALYSIS
The Systematic Planning Process: Identify Instructiona l Goals Analyze Learners Identify Objectives Plan instructional activities Choose Instructional Media Design Assessment Tools Implement Instruction Revise Instruction
Steps in Test Construction Identify Objectives Develop TOS Write Items
Illustrations on Sampling Population (Universe of Characteristics) Sample ( Best estimates of the Population) Test
Illustrations on Sampling Universe of Characteristics A Better Model Best Model A Poor Model
Illustrations on Sampling Invalid and Unreliable Test •Too many errors Valid but Unreliable Test • Errors causes test scores to be unstable and inconsistent Valid and Reliable Test • Observed errors are not significant (or negligible)
THREE-STAGE MODEL OF CLASSROOM MEASUREMENT CONTENT VALIDITY Stage 1 Stage 2 Stage 3 OBJECTIVES ACTIVITIES TEST Test items must validly measure instructional objectives
A. Research Designs 1. Differentiate Quantitative and Qualitative reseach. 2. Classify the various research designs 3. Identify common research designs 4. Contruct research problems in each research designs. total Topics / Objectives K C A AN S E total THINKING SKILLS 1,2 3,4,5,6 7,8,9,10 11,12 Item Placement 2 2 2 2 4 1 2 1 4 2 2 60 Hint: # of items = # of days per topic / total # of days X total # of items 1 6 2 12 1 6 1 6 10 DAYS IDEAL SAMPLE TABLE OF SPECIFICATION
WORKSHOP 1 You need to have: a. Sample syllabus b. Test Questionnaire c. Paper and Pen
Instructions: Choose a syllabus and a questionnaire from your compilation Match the items in the questionnaire with the competencies measured in the syllabus Make an evaluation about the content validity of the test – whether it measures what it is suppose to measure. Volunteers will share their summary of evaluation
Matching Requirements: Item versus objective Thinking Skill versus objective Proportion of items per objective versus the total number of items, total number of objectives, and the time element.
Template Objectives List of Items Skills Time Frame Total A. Solve the …. 5, 3, 5 ok 3
Test item writing and analysis
What do you see in the picture?
How many legs does this elephant have?
Test item writing and analysis
Multiple Choice Type of Test
Advantages A large number of ideas can be addressed in a short period of response time. These questions are easily and quickly scored. Questions can elicit responses from all cognitive levels, from knowledge to evaluation. Questions can be improved over time by analyzing them in light of student performance.
Disadvantages It is time-consuming to write good items, especially those at higher cognitive levels. Test-wise and English fluent students tend to be favored.
varieties of multiple-choice items A. The correct answer variety Who invented the sewing machine? a. Fulton *b. Howe c. Singer d. White e. Whitney
B. The best answer variety What was the basic purpose of a test? a.Measure teacher’s ability b.determine students’ achievement c. Assess learning outcomes d. Identify students’ weaknesses
The multiple response variety What factors are principally responsible for the clotting of blood? a.contact of blood with a foreign substance *b. contact of blood with injured tissue c. oxidation of hemoglobin *d. presence of unchanged pro thrombin
Use of Analogies Man is to woman as boy is to a.Father b.Mother c.Girl d.Lady
The incomplete statement variety In 2008, millions of pesos worth of corn, rice, and wheat are destroyed in the Philippines by a. rats b. grasshoppers c. birds *d. pests
The negative variety Which of these is NOT true of viruses? a. Viruses live only in plants and animals. b. Viruses reproduce themselves. *c. Viruses are composed of very large living cells. d. Viruses can cause diseases.
The substitution variety Passages to be read Surely the forces of education should be fully utilized to acquaint youth with the real nature of the dangers to democracy, (1) for no other place offers (2) as good or better opportunities than the school for; a (3) rational consideration of the problems involved. Items to be answered 1. *a. , for b. . For c. —for d. no punctuation needed 2. a. As good or better opportunities than b. as good opportunities or better than c. as good opportunities as or better than *d. better opportunities than 3. *a. rational b. radical c. reasonable d. realistic
The combined response variety In what order should these sentences be written in order to make a coherent paragraph? A. A sharp distinction must be drawn between table manners and sporting manners. B. This kind of handling of a spoon at the table, however, is likely to produce nothing more than an angry protest against squirting grapefruit juice about. C. Thus, for example, a fly ball caught by an outfielder in baseball or a completed pass in football is a subject for applause. D. Similarly, the dexterous handling of a spoon in golf to release a ball from a sand trap may win a championship match. E. But a biscuit or a muffin tossed and caught at the table produces scorn and reproach. a. A. B, C. D. E *b. A, C. E, D. B c. A, E. C. D, B d. B, E, D. C. A
Basic Guidelines
Adolescents obtain information about marriage A. in order to reduce the possibility of pregnancy B. from parents C. from same-sex friends* D. in a haphazard fashion
PRESENT A CLEAR PROBLEM IN THE STEM Most adolescents obtain information about marriage from A. same sex friends* B. parents C. health service personnel D. teachers
Before the Civil War, the South's _____was one of the major reasons manufacturing developed more slowly than it did in the North. A. emphasis on staple-crop production* B. lack of a suitable supply of raw materials C. short supply of personnel capable of operating the necessary machinery
PUT ALTERNATIVES AT THE END OF THE QUESTION Before the Civil War, manufacturing developed more slowly in the South than in the North. One of the major causes of this was the South's___ A. emphasis on staple-crop production* B. lack of a suitable supply of raw materials C. short supply of personnel capable of operating the necessary machinery
In objective testing, the term objective A. refers to the method of identifying the learning outcomes B. refers to the method of selecting the test content C. refers to the method of presenting the problem D. refers to the method of scoring the answers*
PUT MOST OF THE WORDING IN THE STEM In objective testing, the term objective refers to the method of A. identifying the learning outcomes B. selecting the test content C. presenting the problem D. scoring the answers*
For almost a century, the Rhine river has been used by Europeans for a variety of purposes. However, in recent years, the increased river traffic has resulted in increased levels of diesel pollution in the waterway. Which of the following would be the most dramatic result if, because of the pollution, the Council of Ministers of the European Community decided to close the Rhine to all shipping? A. closure of the busy river Rhine ports of rotterdam, Marseilles and Genoa B. increased prices for Ruhr products* C. reduced competitiveness of the French Aerospace Industry D. shortage of water for Italian industries
AVOID UNNECESSARY WORDINESS Which of the following would be the most dramatic result if, because of diesel pollution from ships, the river Rhine were closed to all shipping? A.closure of the busy river Rhine ports of Rotterdam, Marseilles and Genoa B. increased prices for Ruhr products* C. reduced competitiveness of the French Aerospace Industry D. shortage of water for Italian industries
Sometimes a teacher finds it necessary to use a mild form of punishment. When this occurs, which of the following should not happen? A.Children should not believe all of their behavior is bad. B. Children should understand the reason(s) why they are being punished. C. Children should understand that the teacher, not the children, controls when the punishment will end.*
AVOID NEGATIVELY WORDED STEMS Sometimes a teacher finds it necessary to use a mild form of punishment. When this occurs, it is important that the children understand: A.that it may be a long time before happy times return to the classroom B.the reason(s) why they are being punished* C. that the teacher, not the children, controls when the punishment will end
Good Thurstone's 7 primary mental abilities include all of the following EXCEPT: A. word fluency B. reasoning C. social interaction* D. perceptual speed
Which of the following men contributed most towards he defeat of Hitler‘s Germany in World War II? A. Winston Churchill B. Josef Stalin C. Franklin D. Roosevelt D. George Patton
AVOID REQUIRING PERSONAL OPINION
What is the area of the rectangle described in Question 1? A. 1050 sq. cm. B. 7396 sq. cm. C. 7654 sq. cm.* D. 8188 sq. cm.
AVOID LINKED OR CLUED ITEMS
What is the official state bird of Pennsylvania? A.mountain laurel B.B. Philadelphia C. ruffed grouse* D. ibex
ALL OPTIONS SHOULD BE HOMOGENEOUS What is the official state bird of Pennsylvania? A. goldfinch B. robin C. ruffed grouse* D. wild turkey
Who succeeded Giscard d'Estaing as President of France in l981? A. Georges Pompidou B. Charles De Gaulle C. Francois Mitterand* D. Mick Jagger
ALL OPTIONS SHOULD BE PLAUSIBLE
Which of the following is the best definition of a seismograph ? A. An apparatus for measuring sound waves. B. An apparatus for measuring heat waves. C. An apparatus for measuring earthquake waves.* D. An apparatus for measuring ocean waves.
PUT REPEATED WORDS IN THE STEM, NOT THE OPTIONS A seismograph is an apparatus for measuring: A. earthquake waves* B. heat waves C. ocean waves D. sound waves What type of waves does a seismograph measure? A. earthquake* B. heat C. ocean D. sound
An angle of 90o is called a A.acute angle B.obtuse angles C.right angle*
MAKE ALL OPTIONS GRAMMATICALLY CONSISTENT WITH STEM Angles of 90o are called A.acute angles B.obtuse angles C.right angles*
The average difference between males and females in the attainment of puberty is: A. 1 year B. 2 years* C. 3 years D. no difference
ORDER OPTIONS LOGICALLY
RULES FOR WRITING MULTIPLE- CHOICE ITEMS  Design each item to measure an important learning outcome.  Present a single clearly formulated problem in the stem of the item.  Put the alternatives at the end of the question, not in the middle nor at the beginning.  Put as much of the wording as possible in the stem.  Eliminate unnecessary wordiness
 Avoid negatively worded stems. "Which of the following is not..........“  Avoid requiring personal opinion. Other item types are more suitable for this.  Avoid textbook wording.  Do not have linked or clued items.  All options should be homogeneous.
 All options should be plausible.  Put repeated words in the stem, not in the options  Punctuation should be consistent.  Make all options grammatically consistent with the stem of the item.  List options vertically.
 Order options logically.  Use the option "all of the above" sparingly.  Use the option "none of the above" sparingly. Make All distracters the same length
Exercise: Joseph Estrada was an Actor Former President Driver Scientist
In what year did humans first set foot on the moon? 1975 1957 1969 1963
Some test items Are too difficult Are objective Are poorly constructed Have multiple defensible answers
Ferdinand Magellan came to the Philippines in a Car Boat Airplane balloon
WORKSHOP 2 You need to have: a. Sample syllabus b. Test Questionnaire c. Paper and Pen
Instructions: Evaluate your items using the guidelines in item writing as your criteria. Revise poor items. Identify the objective/competence in every revised item. Build an item bank
Test item writing and analysis

More Related Content

Test item writing and analysis

  • 2. September 10, 2011 UIC TEST ITEM WRITING and ITEM ANALYSIS
  • 3. The Systematic Planning Process: Identify Instructiona l Goals Analyze Learners Identify Objectives Plan instructional activities Choose Instructional Media Design Assessment Tools Implement Instruction Revise Instruction
  • 4. Steps in Test Construction Identify Objectives Develop TOS Write Items
  • 5. Illustrations on Sampling Population (Universe of Characteristics) Sample ( Best estimates of the Population) Test
  • 6. Illustrations on Sampling Universe of Characteristics A Better Model Best Model A Poor Model
  • 7. Illustrations on Sampling Invalid and Unreliable Test •Too many errors Valid but Unreliable Test • Errors causes test scores to be unstable and inconsistent Valid and Reliable Test • Observed errors are not significant (or negligible)
  • 8. THREE-STAGE MODEL OF CLASSROOM MEASUREMENT CONTENT VALIDITY Stage 1 Stage 2 Stage 3 OBJECTIVES ACTIVITIES TEST Test items must validly measure instructional objectives
  • 9. A. Research Designs 1. Differentiate Quantitative and Qualitative reseach. 2. Classify the various research designs 3. Identify common research designs 4. Contruct research problems in each research designs. total Topics / Objectives K C A AN S E total THINKING SKILLS 1,2 3,4,5,6 7,8,9,10 11,12 Item Placement 2 2 2 2 4 1 2 1 4 2 2 60 Hint: # of items = # of days per topic / total # of days X total # of items 1 6 2 12 1 6 1 6 10 DAYS IDEAL SAMPLE TABLE OF SPECIFICATION
  • 10. WORKSHOP 1 You need to have: a. Sample syllabus b. Test Questionnaire c. Paper and Pen
  • 11. Instructions: Choose a syllabus and a questionnaire from your compilation Match the items in the questionnaire with the competencies measured in the syllabus Make an evaluation about the content validity of the test – whether it measures what it is suppose to measure. Volunteers will share their summary of evaluation
  • 12. Matching Requirements: Item versus objective Thinking Skill versus objective Proportion of items per objective versus the total number of items, total number of objectives, and the time element.
  • 13. Template Objectives List of Items Skills Time Frame Total A. Solve the …. 5, 3, 5 ok 3
  • 15. What do you see in the picture?
  • 16. How many legs does this elephant have?
  • 19. Advantages A large number of ideas can be addressed in a short period of response time. These questions are easily and quickly scored. Questions can elicit responses from all cognitive levels, from knowledge to evaluation. Questions can be improved over time by analyzing them in light of student performance.
  • 20. Disadvantages It is time-consuming to write good items, especially those at higher cognitive levels. Test-wise and English fluent students tend to be favored.
  • 21. varieties of multiple-choice items A. The correct answer variety Who invented the sewing machine? a. Fulton *b. Howe c. Singer d. White e. Whitney
  • 22. B. The best answer variety What was the basic purpose of a test? a.Measure teacher’s ability b.determine students’ achievement c. Assess learning outcomes d. Identify students’ weaknesses
  • 23. The multiple response variety What factors are principally responsible for the clotting of blood? a.contact of blood with a foreign substance *b. contact of blood with injured tissue c. oxidation of hemoglobin *d. presence of unchanged pro thrombin
  • 24. Use of Analogies Man is to woman as boy is to a.Father b.Mother c.Girl d.Lady
  • 25. The incomplete statement variety In 2008, millions of pesos worth of corn, rice, and wheat are destroyed in the Philippines by a. rats b. grasshoppers c. birds *d. pests
  • 26. The negative variety Which of these is NOT true of viruses? a. Viruses live only in plants and animals. b. Viruses reproduce themselves. *c. Viruses are composed of very large living cells. d. Viruses can cause diseases.
  • 27. The substitution variety Passages to be read Surely the forces of education should be fully utilized to acquaint youth with the real nature of the dangers to democracy, (1) for no other place offers (2) as good or better opportunities than the school for; a (3) rational consideration of the problems involved. Items to be answered 1. *a. , for b. . For c. —for d. no punctuation needed 2. a. As good or better opportunities than b. as good opportunities or better than c. as good opportunities as or better than *d. better opportunities than 3. *a. rational b. radical c. reasonable d. realistic
  • 28. The combined response variety In what order should these sentences be written in order to make a coherent paragraph? A. A sharp distinction must be drawn between table manners and sporting manners. B. This kind of handling of a spoon at the table, however, is likely to produce nothing more than an angry protest against squirting grapefruit juice about. C. Thus, for example, a fly ball caught by an outfielder in baseball or a completed pass in football is a subject for applause. D. Similarly, the dexterous handling of a spoon in golf to release a ball from a sand trap may win a championship match. E. But a biscuit or a muffin tossed and caught at the table produces scorn and reproach. a. A. B, C. D. E *b. A, C. E, D. B c. A, E. C. D, B d. B, E, D. C. A
  • 30. Adolescents obtain information about marriage A. in order to reduce the possibility of pregnancy B. from parents C. from same-sex friends* D. in a haphazard fashion
  • 31. PRESENT A CLEAR PROBLEM IN THE STEM Most adolescents obtain information about marriage from A. same sex friends* B. parents C. health service personnel D. teachers
  • 32. Before the Civil War, the South's _____was one of the major reasons manufacturing developed more slowly than it did in the North. A. emphasis on staple-crop production* B. lack of a suitable supply of raw materials C. short supply of personnel capable of operating the necessary machinery
  • 33. PUT ALTERNATIVES AT THE END OF THE QUESTION Before the Civil War, manufacturing developed more slowly in the South than in the North. One of the major causes of this was the South's___ A. emphasis on staple-crop production* B. lack of a suitable supply of raw materials C. short supply of personnel capable of operating the necessary machinery
  • 34. In objective testing, the term objective A. refers to the method of identifying the learning outcomes B. refers to the method of selecting the test content C. refers to the method of presenting the problem D. refers to the method of scoring the answers*
  • 35. PUT MOST OF THE WORDING IN THE STEM In objective testing, the term objective refers to the method of A. identifying the learning outcomes B. selecting the test content C. presenting the problem D. scoring the answers*
  • 36. For almost a century, the Rhine river has been used by Europeans for a variety of purposes. However, in recent years, the increased river traffic has resulted in increased levels of diesel pollution in the waterway. Which of the following would be the most dramatic result if, because of the pollution, the Council of Ministers of the European Community decided to close the Rhine to all shipping? A. closure of the busy river Rhine ports of rotterdam, Marseilles and Genoa B. increased prices for Ruhr products* C. reduced competitiveness of the French Aerospace Industry D. shortage of water for Italian industries
  • 37. AVOID UNNECESSARY WORDINESS Which of the following would be the most dramatic result if, because of diesel pollution from ships, the river Rhine were closed to all shipping? A.closure of the busy river Rhine ports of Rotterdam, Marseilles and Genoa B. increased prices for Ruhr products* C. reduced competitiveness of the French Aerospace Industry D. shortage of water for Italian industries
  • 38. Sometimes a teacher finds it necessary to use a mild form of punishment. When this occurs, which of the following should not happen? A.Children should not believe all of their behavior is bad. B. Children should understand the reason(s) why they are being punished. C. Children should understand that the teacher, not the children, controls when the punishment will end.*
  • 39. AVOID NEGATIVELY WORDED STEMS Sometimes a teacher finds it necessary to use a mild form of punishment. When this occurs, it is important that the children understand: A.that it may be a long time before happy times return to the classroom B.the reason(s) why they are being punished* C. that the teacher, not the children, controls when the punishment will end
  • 40. Good Thurstone's 7 primary mental abilities include all of the following EXCEPT: A. word fluency B. reasoning C. social interaction* D. perceptual speed
  • 41. Which of the following men contributed most towards he defeat of Hitler‘s Germany in World War II? A. Winston Churchill B. Josef Stalin C. Franklin D. Roosevelt D. George Patton
  • 43. What is the area of the rectangle described in Question 1? A. 1050 sq. cm. B. 7396 sq. cm. C. 7654 sq. cm.* D. 8188 sq. cm.
  • 44. AVOID LINKED OR CLUED ITEMS
  • 45. What is the official state bird of Pennsylvania? A.mountain laurel B.B. Philadelphia C. ruffed grouse* D. ibex
  • 46. ALL OPTIONS SHOULD BE HOMOGENEOUS What is the official state bird of Pennsylvania? A. goldfinch B. robin C. ruffed grouse* D. wild turkey
  • 47. Who succeeded Giscard d'Estaing as President of France in l981? A. Georges Pompidou B. Charles De Gaulle C. Francois Mitterand* D. Mick Jagger
  • 48. ALL OPTIONS SHOULD BE PLAUSIBLE
  • 49. Which of the following is the best definition of a seismograph ? A. An apparatus for measuring sound waves. B. An apparatus for measuring heat waves. C. An apparatus for measuring earthquake waves.* D. An apparatus for measuring ocean waves.
  • 50. PUT REPEATED WORDS IN THE STEM, NOT THE OPTIONS A seismograph is an apparatus for measuring: A. earthquake waves* B. heat waves C. ocean waves D. sound waves What type of waves does a seismograph measure? A. earthquake* B. heat C. ocean D. sound
  • 51. An angle of 90o is called a A.acute angle B.obtuse angles C.right angle*
  • 52. MAKE ALL OPTIONS GRAMMATICALLY CONSISTENT WITH STEM Angles of 90o are called A.acute angles B.obtuse angles C.right angles*
  • 53. The average difference between males and females in the attainment of puberty is: A. 1 year B. 2 years* C. 3 years D. no difference
  • 55. RULES FOR WRITING MULTIPLE- CHOICE ITEMS  Design each item to measure an important learning outcome.  Present a single clearly formulated problem in the stem of the item.  Put the alternatives at the end of the question, not in the middle nor at the beginning.  Put as much of the wording as possible in the stem.  Eliminate unnecessary wordiness
  • 56.  Avoid negatively worded stems. "Which of the following is not..........“  Avoid requiring personal opinion. Other item types are more suitable for this.  Avoid textbook wording.  Do not have linked or clued items.  All options should be homogeneous.
  • 57.  All options should be plausible.  Put repeated words in the stem, not in the options  Punctuation should be consistent.  Make all options grammatically consistent with the stem of the item.  List options vertically.
  • 58.  Order options logically.  Use the option "all of the above" sparingly.  Use the option "none of the above" sparingly. Make All distracters the same length
  • 59. Exercise: Joseph Estrada was an Actor Former President Driver Scientist
  • 60. In what year did humans first set foot on the moon? 1975 1957 1969 1963
  • 61. Some test items Are too difficult Are objective Are poorly constructed Have multiple defensible answers
  • 62. Ferdinand Magellan came to the Philippines in a Car Boat Airplane balloon
  • 63. WORKSHOP 2 You need to have: a. Sample syllabus b. Test Questionnaire c. Paper and Pen
  • 64. Instructions: Evaluate your items using the guidelines in item writing as your criteria. Revise poor items. Identify the objective/competence in every revised item. Build an item bank