Robo Assessment
The creators of the new robo-readers claim that
they can grade thousands of essays in the blink of an eye. These automated
readers can scan documents and use algorithms to count sentence length, check
verb agreement, and measure sentence complexity. But can they really assess?
Grading is about giving numerical ratings and rankings to student work. Assessment is about using the data to monitor
progress, identify gaps in learning, and guide instructional planning.
Creative work, original thoughts, synthesis of
ideas are difficult enough for teachers to assess even when they know the
content and the learning targets. Beautifully written narratives and exchanges
of idea are important outcomes that robots may not be able to assess for nuance
and inference. And, if teachers are going to have robots read and grade student
work, then the teachers will not be monitoring growth, providing feedback, and
responding with appropriate interventions.
With the new common core writing requirements,
teachers will need to help students express ideas, support their ideas with
facts, and write for a target audience. While a robot may be able to measure
writing techniques, it takes a teacher to analyze ideas and assess growth.
posted by Assessment Network @ 4:06 AM
5 Comments
5 Comments:
I see many benefits in the use of computer grading systems. Where would science teachers like myself be without scantrons? (still sitting at a desk correcting multiple choice I would guess!). They can be an effective way to save time and energy in some types of assessment. That being said, I agree with Greenstien that there is no way that a machine can accurate grade written work!
Although not always the most important part of a relationship in the classroom, grading does have a part in the way in which we as teachers build our relationships with our students. It can show students that we are there to help them and can also build a trust and understanding between two people. No student will ever have a relationship with a computer! The relationship aspect of grading is critical. Students should be able to go to the person who graded their assignments and defend their choices or ask questions. I don’t think they will be talking at their computer screens!
Written assignments are graded on so many facets. Although I believe that a computer has some ability to assess grammar, spelling, sentence structure, etc. (just look at spell check), I do not believe that a computer is able to accurately assess a students’ understanding of neither complex thoughts nor are they able to assess the content that a student would put into a well thought out essay. I do not believe a computer is capable of grading the content of an essay at in a way that even compares to a teacher.
As teachers we are taught how to accurately grade writing. Aside from those skills, we also learn about students’ cognitive ability. We learn where in their development the student should stand and therefore, at what level their writing abilities should lie. A computer cannot know a student like a teacher can. They cannot understand the complexities and individuality of different students. Computers also cannot differentiate grading practices like a teacher can in order to have grades reflect students’ individuality.
I believe that as teachers moving through the 21st century, with students expressing so many new, technologically advanced skills, will need to begin using more authentic assessment that utilizes technology. That being said, I do not believe that teachers can be removed from the assessment equation and replaced with computers. Grading is an important part of building critical relationships in the classroom, and there is also no way for a computer to understand the complexity of content and the individuality that our students possess.
As a language arts teacher in training, robo-readers sound dreamy! I know a large portion of my teacher career will be spent assessing essays. But realistically how can I make an individual connection with a student if I utilize robo-readers in my classroom?
I am always open minded to new ideas and am always looking for great collaboration. Can I find that with a robo-reader? How can I use this tool to my full advantage as a teacher and for my students?
I’d like to think I could use the robo-reader as a pre-assessment tool to my assessment. I would like to try robo-reader by using it to give me a generic overview of my students’ essays. It can assess sentence length, grammar and punctuation, and sentence complexity for me. I can then take the much-needed extra time to assess each essay for its creativity, original thoughts, and synthesis of ideas. As the instructor I can add the individuality aspect to the assessment. I can agree to disagree with robo-reader if necessary. Hopefully this tool can allow me to focus more on the students’ individual work while it works to check for the formalities of essay writing.
I’d prefer not look at robo-readers as the end all of essay assessment, rather I would like to try it out as a tool I can utilize to help me assess. It could give me a differing perspective on the assessment or as a new teacher validate my assessments. Quite possibly this could be a tool I can partner with. It could allow me quick access to students’ previous work to check for growth or to monitor ongoing mistakes.
I am always one to embrace new technology and I hope that this type of software will help me be a better assessor. It is up to me to use it as a tool in conjunction with my individual assessments rather than depend on it to do my job.
This comment has been removed by the author.
This instantly reminded me of a Q&A session I read concerning an essay reviewer for some large agency. The essay reviewers are given a standard rubric to follow and generally have to get through so many per hour (2-3 minutes per essay or else they get flagged in a system). They are not to grade on the correctness but rather the flow and structure of the essay. With averaging 2-3 minutes per essay one would have to pick up on signs or similarities between great essays and worse essays.
At first I was a bit taken back by the thought of robotic test reading but if it can be shown to be accurate and efficient then I have no problem with it. However technology is far from perfect and it may very well grade a perfect paper incorrectly. Although as it currently stands, grading SAT papers at 2-3 minutes per essay you are bound to make a mistake here and there.
With a computer-based system you are adhering to a stricter set of rules and system. Another concern I have is what if a student figures out, one way or another, how the computer grades. They could, very well, trick the system if given the right information.
I feel though, at the end, tech-assisted or tech-dependent grading may work out. We can't simply dismiss it because of its strict technological nature. I think it may work
The main problem I have when I read something about these robot graders and the implication that they could be used with the common core, is that it conflicts with the main goal of the common core that I agreed with.
Assessing student growth, rather than assessing students on a flat numerical level is a change that I feel was in the right direction. Especially with the increased "teacher accountability" that is masquerading as student results on standardized tests.
Robot assessors may be able assess structure, syntax, and search for key power words, but is the machine able to view main ideas? At the elementary level, will the machine be able to properly recognize a text to self connection if the student writes it in a way that is different from, "this reminds me of".
However, I also feel as though it is naive to think that technology has no place in assessment. If we truly plan on grading these writing sections on grammar, syntax, etc. than by all means allow the system to have a field day on those concrete operations. However, when it comes to the fluid dynamic of writing, leave it to the experts.
Post a Comment
Subscribe to Post Comments [Atom]
<< Home