I recently came across this article from Education Week. It made my blood boil. There is so much wrong with it, that I felt that just posting it on my Facebook wall and commenting wasn’t sufficient. So I decided to do what’s becoming my habit when something about the privatization of our public education system makes me really, really, mad: write about it here.
The piece begins by talking about the vast number of students (12 million according to the article) who are taking either the SBAC or PARCC assessments this year and the need for many items on these tests to be hand scored. According to the article 42,000 people have been hired to do that scoring. The article is about testing behemoth Pearson, which is training scorers for PARCC states and “permitted a rare visit” to one of it’s scoring centers where the author was able to glimpse the process that is “typically shrouded in secrecy”.
The writer spoke to “Launica Jones…(a) 23-year-old (who) taught elementary school for a while, but now she's going to law school at night, and scoring the PARCC exam by day for extra cash.” I found myself at this point wondering how long “a while” was, considering that Ms. Jones is only 23 and the typical elementary education program around the country requires 4 or 5 years of college in order to become a certified teacher. I’m no math wizard, but most students graduate high school around 18, so “a while” cannot possibly be more than one year. I found myself wondering if it were even that long.
The article then goes on to describe the training process for scorers, which involves multiple trainees scoring the same anchor items in order to assure that all are scoring in about the same way. This is something that I, as a teacher, have participated in many times. Oregon used to provide training to all teachers, during our in-service days (which the public has already paid for with their tax dollars) to do the exact same sort of scoring calibration. The major difference here is that many of those doing the scoring are not teachers. According to the article, they are also “accountants, technology geeks, lawyers, unemployed corporate executives”. And almost jokingly, “oh, yes, teachers”, some of whom may have taught for “a while”.
The article states that Pearson requires that all scorers hold a bachelor’s degree. That could be in Business, or Art History or Underwater Basket Weaving, I guess, as there is no requirement for scorers to be in any way trained as educators. The requirement of a four year degree is something that Pearson and PARCC states have agreed on. States using the Smarter Balanced test choose their own “vendor”, not necessarily Pearson, to do their test scoring. But Pearson is driving the train in this brave, new world of outsourcing K-12 assessment.
As blogger Hyung Nam discovered, in Oregon the hiring of test scorers, just as in PARCC states, is often done off Craigslist or through temporary services like Kelly. Those scorers work in various locations around the country. But they don’t work in Oregon. Oregon's students' tests will be scored, according to ODE official Derek Brown, “somewhere in the midwest”. The Oregon Department of Education has contracted with a company called DRC (Data Recognition Corporation). They are advertising here for scorers. This is their recruiting flyer for jobs.
The EdWeek article says this about California, another SBAC state:
“Through its contractor, the Educational Testing Service, the state is hiring only scorers who have bachelor's degrees, though they can be ‘in any field,’ according to an ETS flyer. Teaching experience is ‘strongly preferred,’ but not required. Certified teachers, however, must be paid $20 per hour to score, while non-teachers earn $13 per hour. As of late April, only 10 percent of the scorers hired in California were teachers, according to the state department of education.”
And, “Pearson reports that among the scorers hired as of April 30, 72 percent had one or more years of teaching experience, but only half are still teaching. Any teaching experience—not just that obtained in mainstream public or private K-12 classrooms—is acceptable…”
Teaching first aid classes? Providing fire extinguisher training? Lean production training? Does that count as being a teacher? Does it count same as spending years working in an actual classroom with actual children?
As members of Oregon Save Our Schools were told in an email from ODE Assistant Superintendent Doug Kosty, “DRC follows industry standards and best practices for scoring large-scale assessments.” Calling this “best practices” is laughable.
The article also talks about not only those temp workers hired off Craigslist, Monster, et. al. that work out of central locations, but also those who “train and work from home—three-quarters of those scoring the PARCC tests for Pearson”. I wonder how secure students’ private data is when just any one who has any 4 year degree and is hired off Craigslist uses their home computer to rate student work?
But of course not just the “quality” of the raters’ answers is monitored by Pearson or DRC or ETS or whichever corporation is checking your child’s work, but also their “productivity” (a.k.a. speed) because, of course, time is money. Again from the article:
“Pearson staffers monitor raters' productivity. Acceptable scoring speeds are based on Pearson's past experience scoring tests. Those reviewing 3rd grade math, for instance, are expected to score 50 to 80 answers per hour, while raters for 3rd and 4th grade English/language arts responses are expected to complete 20 to 40 per hour, and those scoring high school English/language arts responses are expected to complete 18 to 19 per hour, Pearson officials said.”
Wow. That’s pretty darn fast! Makes me feel like I’ve been wasting a lot of evenings and weekends when I could have been watching "Dancing with the Stars" fretting too much over just what score to give something a child has done his or her very best work on.
But don’t be concerned about someone who doesn’t know your child and may or may not be an actual educator scoring his or her answers. You can rest easy in this assurance from the Center for Assessment, a testing industry center in New Hampshire:
“Joseph Martineau, a senior associate at the center, said his worry isn't focused on the subjectivity of 42,000 human scorers.” (God forbid we should have subjective human scorers evaluating the efforts of human children.)
“Established practice in the field ensures careful training and monitoring, making hand-scoring ‘much more of a science than people think it is,’ he said. But he questioned the assessment industry's capacity to supply a key component: the supervisors who train and oversee those scorers.” (We used to call "supervisors who... oversee scorers" principals.)
“ ‘[Hand-scoring the Smarter Balanced and PARCC tests] requires a major increase in scoring capacity of the vendors,’ said Mr. Martineau, who used to oversee testing as a deputy state superintendent in Michigan. ‘You have to have table leaders, team leaders, who are really good. Those may be few and far between.’ ”
“If the vendors who are training raters adhere to best practices… (Chris) Domaleski, (senior associate at the Center for Assessment) is optimistic that the scores will be reliable and valid. Some of those practices include requiring appropriate qualifications of scorers, creating a rigorous process to qualify them as scorers, and monitoring whether they're scoring within expected ranges. ‘At the same time, though, I don't want to minimize the risk of how difficult it's going to be, given the extraordinary scope of these operations,’ (Domaleski) added.”
So there’s “risk”? I thought that high in the upper echelons, where this was all created, they were so sure this was all going to be the perfect system! And that’s IF they adhere to best practices. But then, why wouldn’t they? After all, the only thing involved is their company’s profits. And heaven knows, we couldn’t continue to trust teachers who know the children to rate the work, for how on Earth would we be able to keep them honest and hold them accountable? (It doesn’t mention in the article how those test centers will be held accountable for anything at all).
No one asks these questions: Why are we spending even more money that we don't have paying to train temporary workers to do a job that teachers are already trained to do? In Oregon during the 1990s and into the early 2000s, Oregon teachers hand scored student work at the state level as part of state testing. And the training that they received scoring that work, usually over spring break, was brought back with them to their classrooms to help students understand how to formulate better methods and explanations for solving math problems and write better essays. And that training was often propagated in their schools and districts by having those teachers train and work with other teachers on those skills. Why did we stop doing that? We were told it was “too expensive”. Why was that “too expensive” but THIS isn’t? What could we have done with the amount of money we are spending on that? Would it pay to restore Music and PE to all Oregon elementary students? How much is it? And exactly how will the evaluation centers be evaluated?
And now comes my favorite line from the article, right at the end: “Coverage of the implementation of college- and career-ready standards is supported in part by a grant from the Bill & Melinda Gates Foundation.”
As Iago, the parrot sidekick of evil Jafar in Disney’s Aladdin says, “Oh, now THERE’S a big surprise!”
Does Gates funding of media taint objectivity? That’s a great question. What do you think? Bill Gates is a major driver of Common Core standards and big proponent of new tests. Certainly Gates wouldn't be doing this just for profit! Certainly Gates and Pearson couldn’t have any common financial interests . I’m sure it’s all about the children.
And if you click on the original article, don’t forget to read the comments at the end. They are the best. This was my favorite:
“For an innocent child to worry, fret, and do their best on these tests, only to have their carefully crafted answer shipped off to a call center employee, is unimaginably undignified. I cannot conceive of submitting my child to this, and I can't believe that any well-intentioned parent would.”
Had enough of Gates and Pearson control over education? Join Oregon and Washington BATs in Portland to protest Pearson influence!
The piece begins by talking about the vast number of students (12 million according to the article) who are taking either the SBAC or PARCC assessments this year and the need for many items on these tests to be hand scored. According to the article 42,000 people have been hired to do that scoring. The article is about testing behemoth Pearson, which is training scorers for PARCC states and “permitted a rare visit” to one of it’s scoring centers where the author was able to glimpse the process that is “typically shrouded in secrecy”.
The writer spoke to “Launica Jones…(a) 23-year-old (who) taught elementary school for a while, but now she's going to law school at night, and scoring the PARCC exam by day for extra cash.” I found myself at this point wondering how long “a while” was, considering that Ms. Jones is only 23 and the typical elementary education program around the country requires 4 or 5 years of college in order to become a certified teacher. I’m no math wizard, but most students graduate high school around 18, so “a while” cannot possibly be more than one year. I found myself wondering if it were even that long.
The article then goes on to describe the training process for scorers, which involves multiple trainees scoring the same anchor items in order to assure that all are scoring in about the same way. This is something that I, as a teacher, have participated in many times. Oregon used to provide training to all teachers, during our in-service days (which the public has already paid for with their tax dollars) to do the exact same sort of scoring calibration. The major difference here is that many of those doing the scoring are not teachers. According to the article, they are also “accountants, technology geeks, lawyers, unemployed corporate executives”. And almost jokingly, “oh, yes, teachers”, some of whom may have taught for “a while”.
The article states that Pearson requires that all scorers hold a bachelor’s degree. That could be in Business, or Art History or Underwater Basket Weaving, I guess, as there is no requirement for scorers to be in any way trained as educators. The requirement of a four year degree is something that Pearson and PARCC states have agreed on. States using the Smarter Balanced test choose their own “vendor”, not necessarily Pearson, to do their test scoring. But Pearson is driving the train in this brave, new world of outsourcing K-12 assessment.
As blogger Hyung Nam discovered, in Oregon the hiring of test scorers, just as in PARCC states, is often done off Craigslist or through temporary services like Kelly. Those scorers work in various locations around the country. But they don’t work in Oregon. Oregon's students' tests will be scored, according to ODE official Derek Brown, “somewhere in the midwest”. The Oregon Department of Education has contracted with a company called DRC (Data Recognition Corporation). They are advertising here for scorers. This is their recruiting flyer for jobs.
The EdWeek article says this about California, another SBAC state:
“Through its contractor, the Educational Testing Service, the state is hiring only scorers who have bachelor's degrees, though they can be ‘in any field,’ according to an ETS flyer. Teaching experience is ‘strongly preferred,’ but not required. Certified teachers, however, must be paid $20 per hour to score, while non-teachers earn $13 per hour. As of late April, only 10 percent of the scorers hired in California were teachers, according to the state department of education.”
And, “Pearson reports that among the scorers hired as of April 30, 72 percent had one or more years of teaching experience, but only half are still teaching. Any teaching experience—not just that obtained in mainstream public or private K-12 classrooms—is acceptable…”
Teaching first aid classes? Providing fire extinguisher training? Lean production training? Does that count as being a teacher? Does it count same as spending years working in an actual classroom with actual children?
As members of Oregon Save Our Schools were told in an email from ODE Assistant Superintendent Doug Kosty, “DRC follows industry standards and best practices for scoring large-scale assessments.” Calling this “best practices” is laughable.
The article also talks about not only those temp workers hired off Craigslist, Monster, et. al. that work out of central locations, but also those who “train and work from home—three-quarters of those scoring the PARCC tests for Pearson”. I wonder how secure students’ private data is when just any one who has any 4 year degree and is hired off Craigslist uses their home computer to rate student work?
But of course not just the “quality” of the raters’ answers is monitored by Pearson or DRC or ETS or whichever corporation is checking your child’s work, but also their “productivity” (a.k.a. speed) because, of course, time is money. Again from the article:
“Pearson staffers monitor raters' productivity. Acceptable scoring speeds are based on Pearson's past experience scoring tests. Those reviewing 3rd grade math, for instance, are expected to score 50 to 80 answers per hour, while raters for 3rd and 4th grade English/language arts responses are expected to complete 20 to 40 per hour, and those scoring high school English/language arts responses are expected to complete 18 to 19 per hour, Pearson officials said.”
Wow. That’s pretty darn fast! Makes me feel like I’ve been wasting a lot of evenings and weekends when I could have been watching "Dancing with the Stars" fretting too much over just what score to give something a child has done his or her very best work on.
But don’t be concerned about someone who doesn’t know your child and may or may not be an actual educator scoring his or her answers. You can rest easy in this assurance from the Center for Assessment, a testing industry center in New Hampshire:
“Joseph Martineau, a senior associate at the center, said his worry isn't focused on the subjectivity of 42,000 human scorers.” (God forbid we should have subjective human scorers evaluating the efforts of human children.)
“Established practice in the field ensures careful training and monitoring, making hand-scoring ‘much more of a science than people think it is,’ he said. But he questioned the assessment industry's capacity to supply a key component: the supervisors who train and oversee those scorers.” (We used to call "supervisors who... oversee scorers" principals.)
“ ‘[Hand-scoring the Smarter Balanced and PARCC tests] requires a major increase in scoring capacity of the vendors,’ said Mr. Martineau, who used to oversee testing as a deputy state superintendent in Michigan. ‘You have to have table leaders, team leaders, who are really good. Those may be few and far between.’ ”
“If the vendors who are training raters adhere to best practices… (Chris) Domaleski, (senior associate at the Center for Assessment) is optimistic that the scores will be reliable and valid. Some of those practices include requiring appropriate qualifications of scorers, creating a rigorous process to qualify them as scorers, and monitoring whether they're scoring within expected ranges. ‘At the same time, though, I don't want to minimize the risk of how difficult it's going to be, given the extraordinary scope of these operations,’ (Domaleski) added.”
So there’s “risk”? I thought that high in the upper echelons, where this was all created, they were so sure this was all going to be the perfect system! And that’s IF they adhere to best practices. But then, why wouldn’t they? After all, the only thing involved is their company’s profits. And heaven knows, we couldn’t continue to trust teachers who know the children to rate the work, for how on Earth would we be able to keep them honest and hold them accountable? (It doesn’t mention in the article how those test centers will be held accountable for anything at all).
No one asks these questions: Why are we spending even more money that we don't have paying to train temporary workers to do a job that teachers are already trained to do? In Oregon during the 1990s and into the early 2000s, Oregon teachers hand scored student work at the state level as part of state testing. And the training that they received scoring that work, usually over spring break, was brought back with them to their classrooms to help students understand how to formulate better methods and explanations for solving math problems and write better essays. And that training was often propagated in their schools and districts by having those teachers train and work with other teachers on those skills. Why did we stop doing that? We were told it was “too expensive”. Why was that “too expensive” but THIS isn’t? What could we have done with the amount of money we are spending on that? Would it pay to restore Music and PE to all Oregon elementary students? How much is it? And exactly how will the evaluation centers be evaluated?
And now comes my favorite line from the article, right at the end: “Coverage of the implementation of college- and career-ready standards is supported in part by a grant from the Bill & Melinda Gates Foundation.”
As Iago, the parrot sidekick of evil Jafar in Disney’s Aladdin says, “Oh, now THERE’S a big surprise!”
Does Gates funding of media taint objectivity? That’s a great question. What do you think? Bill Gates is a major driver of Common Core standards and big proponent of new tests. Certainly Gates wouldn't be doing this just for profit! Certainly Gates and Pearson couldn’t have any common financial interests . I’m sure it’s all about the children.
And if you click on the original article, don’t forget to read the comments at the end. They are the best. This was my favorite:
“For an innocent child to worry, fret, and do their best on these tests, only to have their carefully crafted answer shipped off to a call center employee, is unimaginably undignified. I cannot conceive of submitting my child to this, and I can't believe that any well-intentioned parent would.”
Had enough of Gates and Pearson control over education? Join Oregon and Washington BATs in Portland to protest Pearson influence!
RSS Feed
