January 25, 2010

Three experiments were conducted on CBT.

　OECD-PISA plans to undertake a computer-based testing trial in 2012. It is a matter of time before a wave of CBT (Computer-Based Testing) reaches school education.
　The group of Advancement of Testing Technology of CRET implemented the following three experiments in August, in order to verify the merits and demerits of CBT:

　1. The assessment and development of CBT which includes math questions using animation.
　2. The effects of note-taking (making annotations or not) on tests.
　3. The impact on tests of the appearance of images of characters, uttering encouraging words.

　The content and outcome of these experiments were presented respectively at the World Conference on E-Leaning and at JSET (Japan Society for Educational Technology) in October, 2009. Visit the page on "Research presentations/papers(Japanese only)" to view the presentations and the details of the experiments.
　In March 2010, follow-up experiments are scheduled and planning is underway. The results will also be introduced on this page.

September 25, 2009

Report by the Research Group on Advancement of Testing Technology led by Dr. Kanji Akahori.

　I would like to introduce a research on annotation, which is one of the three topics we work on in 2009.

　In conventional paper testing, some test takers scribble on test papers in addition to filling out the answer sheets. In reading questions, some people may underline, and in diagram questions, additional lines may be drawn to the original diagrams. We call these marks annotations.

　In most Computer Based Testing (CBT), however, it is impossible to engage in annotations during test taking. One can assume that a lack of annotations may be causing excessive stress to the test takers, resulting in lower performances than their actual abilities. We can develop an insight into the influence of annotations on test results and competence measurement in two different testing environments.

　Preceding tests in this area indicate that annotations do not necessarily result in enhancing comprehension of the written materials. Moreover, depending on the difficulty and types of test problems, the effect of annotations seems to differ. It is also fair to assume that there are individual variations. Some people prefer underlining while reading, and some do not. Whether to engage in annotations or not and the types of annotations used depend largely on each individual.

　The outcome of our research is expected to have practical applications such as examining the annotation functions which can be implemented for CBT, and tracking thought processes while answering. The results will be officially reported at the academic society, and similar conferences.