South Dakota mounts revolutionary online exam
Computer-adaptive test targets students’ skills

Elizabeth B. Guerard Associate Editor

In a move that may mark the first statewide implementation of a purely internet-delivered assessment test, South Dakota legislators recently approved the use of an online exam linked directly to that state’s standards of learning.

Bill 234, signed into law by Gov. Bill Janklow March 5, specifies that "every school district shall administer the same criterion-referenced academic achievement test, once in the fall semester and once again in the spring semester, to all students in grades three, six, and 10."

But what is revolutionary about the Dakota Assessment of Curriculum Standards (DACS) test, created by EdVISION Corp., is that it marks the first time a state-mandated test will be delivered only through online means, state and company officials say.

According to Ray Christensen, state secretary of education and cultural affairs, the online test has several features that will streamline testing procedures and help educators make data-driven decisions about curriculum.

"A major cost of testing traditionally has been the scoring," he said. Administering the test online will "limit that cost and provide instant feedback to students, teachers, and parents."

What’s more, the online curriculum standards test—unlike its pencil-and-paper counterparts—is computer-adaptive.

"That means the test adjusts ‘on the fly’ as students answer the questions," Christensen said. "For instance, if a child is highly proficient in math and keeps getting all the math questions right, the test will keep making the questions more difficult."

Computer-adaptive testing helps accurately identify the skill level of children who are outside the norm, either far advanced for their grade or below standard grade-level proficiency. In doing so, it give educators more information about a student’s specific strengths and weaknesses than they receive from traditional paper-and-pencil tests, its advocates say.

"We knew we needed to develop a test that any teacher could use to ... determine what grade level [students] were [working at]," said Bill Tudor, president and chief executive of EdVISION (formerly Tudor Publishing), based in San Diego. "We call this ‘standards-based adaptive testing.’"

EdVISION developed a general set of learning objectives that encompass all state tests by looking at standards for all areas of learning, from first through 12th grade.

"Basically, we developed a test that will work for all 50 states, but we have also developed a propriety ‘sleeve’ that allows us to do reporting that is relevant to South Dakota’s specific state standards," Tudor said.

The system employs an "expert artificial intelligence system" that determines which grade level a student is testing at in each individual unit, such as fractions, decimals, and algebraic equations.

In one class period, said Tudor, the EdVISION test can find exactly where a student is performing in a whole 12-grade range. The company then reports these results using the standards of a specific state or district.

Each learning objective is assigned a difficulty level, and students who are proficient at combining these objectives would raise the bar until they encountered a question they could not answer.

"This test is like a high-bar," said Christensen. "It pushes each kid to go until [he or she] can’t go any more."

And, he added, "It’s also great for kids who might be less proficient in a subject because, as the questions adjust to their level, [the test] becomes less intimidating."

Christensen said the state has purchased the test on a per-student subscription basis, enabling educators to test as many times per year as they see fit.

The law currently mandates twice-yearly DACS testing in the three grades, but Christensen added, "I think we’ll be doing these tests three times per year, eventually—in the fall, winter, and spring."

The test is intended to look at student growth, not to decide whether students or teachers should be promoted.

Christensen said the test will enable data-driven decision-making at both the school and district level. Teachers will get reports on each child, so they can know where to direct instruction, and parents will receive copies of the report as well.

Infrastructure in place

Despite his excitement about the test, Christensen said one disadvantage to online testing is that it requires a substantial technology infrastructure at each school to support the program.

But that’s no problem for South Dakota.

"We have a T1 [line] to every classroom in the state, and our Digital Dakota Network links every school and every K-12 classroom with the state government, tech schools, and higher education institutions," Christensen said. "We also have two very large pipelines—both DS3s—going out to the public internet, and we have five or six hard-wired drops in each classroom."

Elaine Roberts, president of the South Dakota Education Association, said South Dakota has "a real opportunity to do something like this" because of the level of access its schools have to the internet.

"Most teachers find [the test] takes about 30 minutes," Roberts added. "They can take their students to the lab and administer the test all at once, or they can use their four or five classroom computers and rotate the students through them over a couple of days."

South Dakota has been running a volunteer-only trial program for the past year. About 13 percent of the state’s students have participated in the trial program.

Though state officials are encouraged by the responses they’ve gotten so far, not everyone says the trial has been a success.

Carl Lund, computer technology coordinator for the Eagle Butte School District, said he does not think testing itself is a bad idea, but his district had "horrible problems" with the EdVISION exam.

"We saw a lot of frustration, especially when the server would time out," Lund said. "We had whole classes come to the computer lab, start the testing, get through five to 10 questions, and have the EdVISION servers stop responding. Students then had to be taken back to class and their [testing] rescheduled."

It took some classes up to three sessions to complete the test, said Lund, who questioned the validity of these scores.

Lund also cited some bugs in the system, including a glitch that reported the results of a student’s math test as if she had taken a reading test. "When I eMailed EdVISION technical support, I was told that the student would have to retake both the reading and math tests," he said.

"In general, I felt that we were unpaid beta-testers for this product," he concluded. "Perhaps the tests eventually will fulfill the promise that EdVISION and Gov. Janklow envision. The key concept, though, is eventually."

Roberts acknowledged that the trials revealed some early glitches.

"It is our understanding that EdVISION was not prepared for the large number of students [who] signed on at the same time," she said. "We’ve had numerous conversations ... about improving this, and I think it has been improved. We have not had complaints since that time."

Now that the system’s initial problems have been addressed, teachers "are pleased with the results and the kind of data they’re getting," Roberts said.

And kids are happy to test online as well, she added: "What I’ve heard is that most children like [being tested] online, and they don’t find it at all difficult. They are just so eager to use technology."

Once the DACS is up and running, South Dakota will have three mandatory state tests in place.

The Stanford Achievement Test 9 (SAT 9) still will be administered to students in grades two, four, eight, and 11. "We are not using this to replace the SAT 9. That is still a proven and valuable test for students," Christensen said.

The state also requires yearly writing tests for students in grades five and nine.

State officials say pricing will be reasonable for DACS. Though an exact rate has not yet been determined, published reports indicate the state has set aside $500,000 for the program in its proposed budget for fiscal 2002.

"We’re working on the pricing right now, but we know the cost will be comparable to what it would be for a paper-and-pencil test," said Christensen.

The legislation calls for the test to be implemented as of next spring.

(Close Window)