Embedded Files
Reducing Bias in Automatic Scoring of Standardized Tests
Reducing Bias in Automatic Scoring of Standardized Tests
Dr. Maggie Beiting-Parrish
Using the NAEP (National Assessment for Educational Progress) data for Reading, Math and Civics tests for 4th and 8th graders, Dr. Maggie Beiting-Parrish is seeking to identify the best algorithms to accurately and efficiently score students’ written texts from the NAEP assessments. Representativeness is often missing from the sample data that is used to train the scoring algorithms, which introduces bias.
Using the NAEP (National Assessment for Educational Progress) data for Reading, Math and Civics tests for 4th and 8th graders, Dr. Maggie Beiting-Parrish is seeking to identify the best algorithms to accurately and efficiently score students’ written texts from the NAEP assessments. Representativeness is often missing from the sample data that is used to train the scoring algorithms, which introduces bias.
Potential solutions to this issue that Dr. Beiting-Parrish discussed in this interview include involving educators and computer scientists in creating scoring algorithms; writing questions that are culturally relevant and appropriate for all test takers; including more questions relevant to students’ lived experiences or interests; avoiding the use of polysemic words (words spelled the same that have different contextual meaning); improving the sample of test responses so that it is more representative (i.e., it includes writing from more diverse populations) so that algorithms can appropriately score texts; and formatting tests’ designs so they are appropriate for all populations. Dr. Beiting-Parrish firmly believes that algorithm-based assessment, particularly when applied to student learning, should be designed in thoughtful, representative and communal ways to be of ultimate utility.
Potential solutions to this issue that Dr. Beiting-Parrish discussed in this interview include involving educators and computer scientists in creating scoring algorithms; writing questions that are culturally relevant and appropriate for all test takers; including more questions relevant to students’ lived experiences or interests; avoiding the use of polysemic words (words spelled the same that have different contextual meaning); improving the sample of test responses so that it is more representative (i.e., it includes writing from more diverse populations) so that algorithms can appropriately score texts; and formatting tests’ designs so they are appropriate for all populations. Dr. Beiting-Parrish firmly believes that algorithm-based assessment, particularly when applied to student learning, should be designed in thoughtful, representative and communal ways to be of ultimate utility.
Link:
Link:
Page updated
Google Sites
Report abuse