For my first question I asked “What mechanism do teachers have for flagging test items that are flawed (e.g no correct answer, more than one possible correct answer, interface malfunction, poor or incorrect translation, etc.)? The confidentiality agreement seems to be very prohibitive to that process.”
The answer I received was that there is no mechanism for a teacher to flag items and I was reminded that teachers may not review or analyze items as that is considered a test impropriety.
I then typed in, “So we are expecting 8 year olds to make a determination as to whether a test that their teacher, an adult authority figure has given them, to provide meaningful feedback?” (Clearly my brain jumped ahead of my hand when I asked that, but you get the idea.) Then I added, “That a third grader should be able to determine that an item is flawed and comment on that? Does that seem problematic to anyone at ODE?” Those questions were not answered during the webinar session. The ODE representatives did state that when a student flags an item, “we go through and look at that and may deactivate the item and generally re-score the test”.
I was unable to see questions that others who were participating typed in, but there were apparently a lot of questions about how to handle the Performance Task activity when students are absent for the initial “lesson” presentation by the teacher. There was no real answer from ODE. They asked that districts let them know what was working and “inform (ODE) about promising practices”. Someone stated that their district is recording PT presentations for absent students. Someone asked if videos could be imbedded in the PTs to make them more standardized. ODE stated that they were not expecting any changes in SBAC for 2015-2016 school year.
Someone stated that elementary students had trouble knowing where to answer questions. They were told that the practice test would be revamped. (so that students could practice the testing format.) Someone also stated that elementary students are having trouble understanding “high school language” in the test directions. I’m not sure if they were asking about directions for specific tasks or directions for beginning the test. (All students, grades 3 through 11, receive the same verbatim directions for beginning the test from the test administrators manual)
I asked, “How is ODE and SBAC responding to concerns over the disruption to school schedules and the amount of instructional time lost due to the length of the tests?” I was apparently not the only one asking about the length of the tests. ODE answered that it is unlikely we will see changes to the “blueprint” in testing time for next year, but that doesn’t mean it is being discounted or might not be changed in the future.
Test scores will be coming out at the end of August (though some schools are receiving them already). Scores will be publicly released on September 17.
There was apparently a lot of conversation about opt out. ODE stated they would not be answering questions about that as they hadn’t planned to discuss it.
Someone else again asked about the “overall volume of questions” especially for 3rd and 4th graders and the unclear instructions. Again, we were told no changes are anticipated for next year but that ODE would be working with SBAC and AIR on these issues.
I asked, “Is there any concern that when ELL students are tested on ELPA and thus demonstrate that they are not yet proficient in English that the ELA score would be invalid, in that it is only a second measure of their not yet English proficient status? This is especially a problem in dual immersion programs where English is not introduced until later.” Of course the answer was that after one year in the US, federal “accountability” requires those students to take the tests.
Someone asked a question about the text to speech feature and why the text was not prerecorded. The answer is that that would put “too much strain on the system”.
Someone asked why there is no score on the practice test. The answer was that the practice test is only intended to provide students familiarity with functionality and that a score based on that small sampling would be very misleading.
Someone asked if SBAC interim (formative) assessments will be available. The answer was that this depends on the budget process, but that there are dollars earmarked for those tests. Hooray.
Will the calculator be improved? Yes.
There was an AIR planning meeting about a week ago and preliminary decisions were to be shared today.
Someone asked if once results were in, there might be a possibility of lowering cut scores. The answer was no and contained the words “rigorous” and “deeper” and which point I wrote as the rest of my notes “blah, blah, blah, fresher scent, more flavor, etc.”
The last question I asked was, “Is there any intent to respond to concerns about the developmental appropriateness of some of the standards and testing for young children? That question was not answered, but it may be because I got it in too late, near the end of the webinar. I did receive a follow up email from Derek Brown apologizing for not answering as the connection ended, and asking me to send him an email with my questions and he would be happy to answer. That email will be my next blog post.