Web-based Instruction - Annotations

Web-based Assessment../Library/link%20to%20presentation%20pages

The first interactive use of the Web was in "assessment" or quizzing. Of course, assessment is much more general than quizzes and there is a lot which we won't touch on. For example, the Web has been used for "cognitive mentorship", and in this context communications between mentors and students have been analyzed as a form of assessment. In our own work we view the essential assessment pieces as the student portfolio, and some instructors use software to keep track of time spent on a page as a formative assessment tool. In standards-based education, work samples play a key role and the Web is an ideal medium for collecting, storing, and delivering work samples. This has been integrated into a prototype electronic transcripting system we are developing for the State of Oregon. (See Electronic Transcripts in the Age of Standards).

The assessment we discuss here is the more traditional "test" - but we want to include possibilities beyond straight multiple choice. These include adaptive testing, where the question delivered depends on previous performance, full integration of tests into a course management system, item analysis, and so on. Item analysis can be a valuable tool, even on multiple choice tests. Often more information about the cognitive state of a group of students is available from examining the detractors chosen than from looking at how many got the right answer. There is at least one system which wires a lecture hall with palmtops and a projected console with the intention of delivering on-the-spot quizzes and displaying item analyses for discussion. The same could be done via the Web. Asymetrix and undoubtedly other products have this feature.

Multiple Choice Etc.

Once you try to accept non-multiple choice answers, you can be in big trouble. In our work we accept "fill-in-the-blank" answers and match them to Perl regular expressions, so that if the answer is 11 we can accept "any number of spaces followed by 11 or by elev(e or i)n with case ignored followed by any number of spaces" as an example. Asymetrix has the capability of doing "sounds like" matching and has its own code for pattern matching.

Constructing robust pattern matching answers is time consuming and probably not that accurate, but it covers most cases and if the students understand in vague terms that the computer is matching their answers, they will be forgiving when the computer makes mistakes.

A different kind of "pattern matching" is visual matching (or matching pictures to sounds) which in substance is the same as multiple choice but is quite different in form and delivery.

The Bleeding Edge

Some of the most interesting ideas are in mathematics. Just as the Web was coming out some of us had the idea of shipping answers to Maple for evaluation. We dreamed of being able to write open-ended calculus problems and have them delivered and automatically evaluated over the Web. This dream has come true to some extent, as is exemplified by the Calculus on the Web project at Temple. I have been in discussions where the problem of checking graphs for accuracy was discussed, and although I have never pursued it, I have been told that Informix has sophisticated ways of searching for pictures on the basis of graphical similarities. This could be used for computer evaluation of multimedia answers!

The media lab at MIT has been working hard on "intelligent agents" which seek other agents and negotiate transactions. For example, if you want to sell a bicycle, you program a sales agent with a starting price and negotiating strategy. It can interact with buying agents who also pursue individual strategies. A more mundane example is something like Travelocity which searches for the best airfares for you. Surely this technology can be applied to analyzing student writing - a sophisticated item analysis. Could we not send out an agent to look at student essays and bring back some course information to the instructor on what sort of common mistakes are being made? What about using expert systems? Last Fall I saw a talk by an Australian Computer Science professor who was using a public domain Java based expert system to recommend tutoring options on the basis of test performance.

The use of all this technology is emphatically NOT to replace the live teacher and create "teachbots". Quite to the contrary, it must be used in support of intentional pedagogic approaches and as a means for freeing the teacher from tasks less related to student learning.

The examples ../Library/link%20to%20presentation%20pagesfrom the presentation are pretty much self-explanatory and worth examining.


Home | Presentation Topics | Previous Annotation Page | Next Annotation Page


© 1998, Robby Robson, All Rights Reserved. Contact: robby@orst.edu for information and permission requests.

Last Modified: 11/27/2024 23:32:21