21 July 2001
TestGraf98 is a stand alone program for use on personal computers using the Windows 98/2000/NT operating systems.
TestGraf98 is designed to aid the development, evaluation, and use of multiple choice examinations, psychological scales, questionnaires, and other types of data characterized by:
This potentially covers a rather wide range of data collection situations, but TestGraf98 will be most useful when the number of examinees or respondents is of the order of 100 or more and the number of questions or choices exceeds 20 or so.
TestGraf98 was conceived for use with tests or questionnaires where the responses of any individual are primarily determined by the amount or level of some single ability, characteristic, or trait. Or, alternatively, where the user of the program is interested in the extent to which a single ability or trait determines performance.
In the context of multiple choice examinations, TestGraf98 can assist both instructors and students in analyzing and evaluating particular questions, as well as the examination as a whole. For scales and questionnaires, where there is seldom a particular option for a question designated as ``correct'', TestGraf98 can use numerical weights assigned to each option to determine how effectively the question contributes to the assessment of the trait in question.
TestGraf98 was designed for use without any formal statistical knowledge. The essential aspects of each display were designed to be self- \newline explanatory, although more statistically sophisticated users will also find information that they may find helpful. Most of the output from TestGraf98 is in graphical form, and the program was designed to be used interactively. Since the analyses that TestGraf98 performs take very little time on even personal computers of modest power, there will be only a limited need to produce hard or permanent copy of the displays that TestGraf98 produces, but TestGraf98 does have hard copy capability.
TestGraf98 makes use of modern statistical methods to produce accurate estimates of examinee or respondent characteristics. For example, for examination data, TestGraf98 enables better estimates of examinee proficiency or ability by making use of the information provided by which wrong options were chosen for incorrectly answered items. Such estimates will tend to be much precise than the conventional estimates based only on number correct, and especially for examinees of low to medium proficiency. These more efficient estimates, which are still expressed in familiar terms as percentages, can be used to either replace or modify the classical percentage correct scores reported for examinees.
TestGraf98 also displays graphically what range of proficiency or trait values are reasonably consistent with the set of choices made by an individual. This means that TestGraf98 conveys the relative precision or level of confidence attached to the best estimate, so that one can assess how much information is provided by a respondent's data about her/his position on the proficiency or trait continuum in question.
Instructors or questionnaire developers will find TestGraf98 helpful for diagnosing problems with items, and for deciding whether to rewrite items in order to clear up ambiguous wording or to offer wrong options that are more plausible.
Although by default TestGraf98 is used to study the internal structure of a test or scale, TestGraf98 can also be used to study how individual items relate to scores on some entirely separate set of scores of measures on the examinees or respondents. For example, TestGraf98 might be used by an instructor to see how well test items relate to the final grade of examinees, which might be a composite of other tests as well as this one.
A manual containing a detailed description of the program and examples of how it can be used is included with the program.
TestGraf98 and its manual can be obtained by anonymous ftp from the my ftp site or on a 3 1/2 inch floppy diskette by writing to:J. O. Ramsay
A contribution of \$35.00 is asked to cover the cost of reproducing the manual and the diskette. Preferably this should be sent as a cheque payable to McGill University. If for some reason payment is difficult (e.g. persons from currency blocks where obtaining dollars is difficult, or students), TestGraf98 is happily supplied free of charge.