Post a composite review
Unpost a composite review
Search all MERLOT
Click here to go to your profile
Select to go to your workspace
Click here to go to your Dashboard Report
Click here to go to your Content Builder
Click here to log out
Search Terms
Enter username
Enter password
Please give at least one keyword of at least three characters for the search to work with. The more keywords you give, the better the search will work for you.
select OK to launch help window
cancel help

MERLOT II


    

Peer Review


ARTIST Assessment Builder

 

Ratings

Overall Rating:

4.6 stars
Content Quality: 4.6 stars
Effectiveness: 4.4 stars
Ease of Use: 4.5 stars
Reviewed: Jun 26, 2006 by Statistics Editorial Board
Overview: The ARTIST website consists of different components which provide resources to
aid in assessment in algebra based introductory statistics courses. The
contexts for the items used would be most appropriate for courses aimed toward
the social sciences: education and psychology in particular. The main
components of the website are:
1. Item Database [Assessment Builder]: a collection of about 1100 items and a
tool to access the items in the database.
2. General Resources: information, guidelines, copies of articles, and links.
3. Research Instruments: that may be useful for research and evaluation
projects.
4. Implementation issues: questions and answers on practical issues related to
assessments.
5. Presentations: copies of conference papers and presentations on the ARTIST
project.
6. Events: information on past and upcoming ARTIST events.
7. Participation: ways to participate as a class tester for ARTIST materials.
The most important component of the website is the item bank and assessment
builder. This allows instructors to select the items according to topic and
level. Beginning instructors might find the section on implementation issues
helpful as well. For that reason I will review only these two components as
well as the overall feel of the website.
Learning Goals: This site addresses the full range of topics in an introductory statistics
course.
Target Student Population: This website would be useful for any instructor of an algebra based introductory
statistics course from the high school to graduate level.
Prerequisite Knowledge or Skills: This is intended for use by the instructor. No other special knowledge is
necessary.
Type of Material: Assessment creation website.
Recommended Uses: This site could be used to create quizzes, exams or homework.
Technical Requirements: None.

Evaluation and Observation

Content Quality

Rating: 4.6 stars
Strengths: This site has a very extensive warehouse of questions for introductory
statistics. The questions included in this warehouse are appropriate for
introductory courses in that they use standard vocabulary and topic coverage.
Many include graphics, although some of these are limited in size for
portability. This warehouse can be searched based on the topic such as Analysis
of variance or comparing groups. It can also be searched based on one of three
levels of thought required to complete the question, which include statistical
literacy, statistical thinking and statistical reasoning. After the assessment
is constructed, it can then be downloaded as a RTF file for easy editing.

The site also includes expert advice on strategies for creating, grading and
using assessments in the classrooms. These experts come from a wide variety
of backgrounds and address a variety of issues.
Concerns: Instructors that teach a more traditional course may find questions that only
address computational tasks are missing. The ARTIST assessment builder
intentionally does not include these types of items. This also means that some
topics that have limited non-computational questions such as the binomial
distribution are rare in the assessment builder (this is mentioned by the
authors in the FAQ section of the site).

One of the strengths of the assessment builder are the high level questions that
elicit higher level thinking by students. Some of the depth of these higher
end questions may not be apparent to a novice instructor. These questions are
*designed* with a very particular misconception in mind and this may be lost on
an instructor that does not realize the point of the question.

Notably lacking among those providing advice in the implementation issues
section, is an instructor of AP statistics. Although the ideas presented by the
upper level instructors can be applied in the K-12 arena they could also
benefit from someone who currently works in the K-12 setting.

Also in the implementation area there is discussion of using real data in
constructing of exams. There is little discussion of using data that has
particular pedagogical value. For example, using data that has outliers or a
non-straight line pattern.

Potential Effectiveness as a Teaching Tool

Rating: 4.4 stars
Strengths: The assessment builder has an extensive collection of questions that address
several levels of learning. This site can greatly improve the quality of
assessments for introductory courses. A focus on the assessment can improve the
focus of the course and spur the instructor to improve the overall instruction.

The discussion sections of the site that examine implementation issues can be
very useful for new instructors, or instructors considering changing their
courses.
Concerns: By design there are no straight computational problems and some topics that are
primarily computational are not included the assessment builder. Hence, a
teacher who wants to include these in their assessment could not use this item
bank to build his/her entire test.

Although there is extensive advice offered on constructing exams, it is still
easy to construct and exam that only gets at the most basic ideas without
challenging the students. The materials presented here have excellent
potential, but it is still up to the instructor to dictate how they use these
materials.

Ease of Use for Both Students and Faculty

Rating: 4.5 stars
Strengths: The question bank is very easy to navigate and requires no special software to
use. The novice user can master most functions without special instructions.
Exams created by the assessment builder are stored on the ARTIST website and can
be retrieved for future use.

In the implementation issues section of the site the matrix format allows the
user to quickly find the information required.
Concerns: In the Assessment Builder the order of questions cannot be changed. To change
the order, the user must make the changes manually in the RTF file after it is
downloaded. Also, there is no mention of compatability of the ARTIST Assessment
Builder with learning managment systems (LMS) such as Blackboard or Webct.
Although most of the popular LMSs' can directly import the RTF file of
questions, there may be formating issues that make the process cumbersome.

The overall site does not include a process for updating personal information.
An instructor may wish to change information such as email address without
loosing previously created exams. Aparently the only method for doing this is
to manually email the site authors. (Editors note: the ARTIST team has recently
addressed this concern by making a change in the user interface.)

Other Issues and Comments: