The ARTIST website consists of different components which provide resources to aid in assessment in algebra based introductory statistics courses. The contexts for the items used would be most appropriate for courses aimed toward the social sciences: education and psychology in particular. The main components of the website are: 1. Item Database [Assessment Builder]: a collection of about 1100 items and a tool to access the items in the database. 2. General Resources: information, guidelines, copies of articles, and links. 3. Research Instruments: that may be useful for research and evaluation projects. 4. Implementation issues: questions and answers on practical issues related to assessments. 5. Presentations: copies of conference papers and presentations on the ARTIST project. 6. Events: information on past and upcoming ARTIST events. 7. Participation: ways to participate as a class tester for ARTIST materials. The most important component of the website is the item bank and assessment builder. This allows instructors to select the items according to topic and level. Beginning instructors might find the section on implementation issues helpful as well. For that reason I will review only these two components as well as the overall feel of the website.
Type of Material:
Assessment creation website.
This site could be used to create quizzes, exams or homework.
Identify Major Learning Goals:
This site addresses the full range of topics in an introductory statistics course.
Target Student Population:
This website would be useful for any instructor of an algebra based introductory statistics course from the high school to graduate level.
Prerequisite Knowledge or Skills:
This is intended for use by the instructor. No other special knowledge is necessary.
Evaluation and Observation
This site has a very extensive warehouse of questions for introductory statistics. The questions included in this warehouse are appropriate for introductory courses in that they use standard vocabulary and topic coverage. Many include graphics, although some of these are limited in size for portability. This warehouse can be searched based on the topic such as Analysis of variance or comparing groups. It can also be searched based on one of three levels of thought required to complete the question, which include statistical literacy, statistical thinking and statistical reasoning. After the assessment is constructed, it can then be downloaded as a RTF file for easy editing.
The site also includes expert advice on strategies for creating, grading and using assessments in the classrooms. These experts come from a wide variety of backgrounds and address a variety of issues.
Instructors that teach a more traditional course may find questions that only address computational tasks are missing. The ARTIST assessment builder intentionally does not include these types of items. This also means that some topics that have limited non-computational questions such as the binomial distribution are rare in the assessment builder (this is mentioned by the authors in the FAQ section of the site).
One of the strengths of the assessment builder are the high level questions that elicit higher level thinking by students. Some of the depth of these higher end questions may not be apparent to a novice instructor. These questions are *designed* with a very particular misconception in mind and this may be lost on an instructor that does not realize the point of the question.
Notably lacking among those providing advice in the implementation issues section,
is an instructor of AP statistics. Although the ideas presented by the upper level instructors can be applied in the K-12 arena they could also benefit from someone who currently works in the K-12 setting.
Also in the implementation area there is discussion of using real data in constructing of exams. There is little discussion of using data that has particular pedagogical value. For example, using data that has outliers or a non-straight line pattern.
Potential Effectiveness as a Teaching Tool
The assessment builder has an extensive collection of questions that address several levels of learning. This site can greatly improve the quality of assessments for introductory courses. A focus on the assessment can improve the focus of the course and spur the instructor to improve the overall instruction.
The discussion sections of the site that examine implementation issues can be very useful for new instructors, or instructors considering changing their courses.
By design there are no straight computational problems and some topics that are primarily computational are not included the assessment builder. Hence, a teacher who wants to include these in their assessment could not use this item bank to build his/her entire test.
Although there is extensive advice offered on constructing exams, it is still easy to construct and exam that only gets at the most basic ideas without challenging the students. The materials presented here have excellent potential, but it is still up to the instructor to dictate how they use these materials.
Ease of Use for Both Students and Faculty
The question bank is very easy to navigate and requires no special software to use. The novice user can master most functions without special instructions. Exams created by the assessment builder are stored on the ARTIST website and can be retrieved for future use.
In the implementation issues section of the site the matrix format allows the user to quickly find the information required.
In the Assessment Builder the order of questions cannot be changed. To change the order, the user must make the changes manually in the RTF file after it is downloaded. Also, there is no mention of compatability of the ARTIST Assessment Builder with learning managment systems (LMS) such as Blackboard or Webct. Although most of the popular LMSs' can directly import the RTF file of questions, there may be formating issues that make the process cumbersome.
The overall site does not include a process for updating personal information. An instructor may wish to change information such as email address without loosing previously created exams. Aparently the only method for doing this is to manually email the site authors. (Editors note: the ARTIST team has recently addressed this concern by making a change in the user interface.)