- Peer Review: “ARTIST Assessment Builder”
ARTIST Assessment Builder
- Sep 16, 2008 by Statistics
- Assessment Resource Tools for Improving Statistical Thinking (ARTIST) provides a collection of assessment resources for teaching first courses in statistics.
The ARTIST website consists of different components which provide resources to aid in assessment in algebra based introductory statistics courses. The contexts for the items used would be most appropriate for courses aimed toward the social sciences: education and psychology in particular. The main components of the website are:
1. Item Database [Assessment Builder]: a collection of about 1100 items and a tool to access the items in the database.
2. General Resources: information, guidelines, copies of articles, and links.
3. Research Instruments: that may be useful for research and evaluation projects.
4. Implementation issues: questions and answers on practical issues related to assessments.
5. Presentations: copies of conference papers and presentations on the ARTIST project.
6. Events: information on past and upcoming ARTIST events.
7. Participation: ways to participate as a class tester for ARTIST materials. The most important component of the website is the item bank and assessment builder. This allows instructors to select the items according to topic and level. Beginning instructors might find the section on implementation issues helpful as well. For that reason I will review only these two components as well as the overall feel of the website.
This project was funded by a grant from the National Science Foundation (NSF CCLI -ASA- 0206571) and supported by the National Science Foundation, University of Minnesota, and California Polytechnic State University.
- Type of Material:
- Database and collection and Assessment creation website.
- Recommended Uses:
- The major focus of this website revolves around the Assessment Builder. This site could be used to create quizzes, exams or homework. The Assessment Builder is an online tool that gives the instructor access to a collection of over 1100 high quality assessment items in a variety of item formats. Each item is categorized according to statistical topic and type of learning outcome assessed. Assessment Builder can be used to generate files that can be edited with a word processing program by statistics instructors.
- Technical Requirements:
- Browser and Microsoft Powerpoint reader to access that component.
- Identify Major Learning Goals:
- Per their website: "Our goal is to help teachers assess statistical literacy, statistical reasoning, and statistical thinking in first courses of statistics."
- Target Student Population:
- This website would be useful for any instructor of an algebra based introductory statistics course from the high school to graduate level.
- Prerequisite Knowledge or Skills:
- Basic knowledge and understanding of statistical literacy, statistical reasoning, and statistical thinking will benefit users of this site. Basic definitions and rationales are provided.
- This site has a very extensive warehouse of questions for introductory statistics and incorporates best practices for statistics instruction. The questions included in this warehouse are appropriate for introductory courses in that they use standard vocabulary and topic coverage. Basic definitions are provided in a clear and concise format. Many include graphics, although some of these are limited in size for portability. This warehouse can be searched based on the topic such as Analysis of variance or comparing groups. It can also be searched based on one of three levels of thought required to complete the question, which include statistical literacy, statistical thinking and statistical reasoning. After the assessment is constructed, it can then be downloaded as a RTF file for easy editing.
The site also includes expert advice on strategies for creating, grading and using assessments in the classrooms. These experts come from a wide variety of backgrounds and address a variety of issues. Resources available for investigation include an item database (assessment builder), scales and tests, research instruments dealing with achievement measures and attitudes and anxiety measures, publications, and presentations about the project. It is free to register and make use of the collected resources.
- Instructors that teach a more traditional course may find questions that only address computational tasks are missing. The ARTIST assessment builder intentionally does not include these types of items. This also means that some topics that have limited non-computational questions such as the binomial distribution are rare in the assessment builder (this is mentioned by the authors in the FAQ section of the site).
One of the strengths of the assessment builder are the high level questions that elicit higher level thinking by students. Some of the depth of these higher end questions may not be apparent to a novice instructor. These questions are *designed* with a very particular misconception in mind and this may be lost on an instructor that does not realize the point of the question.
Notably lacking among those providing advice in the implementation issues section, is an instructor of AP statistics. Although the ideas presented by the upper level instructors can be applied in the K-12 arena they could also benefit from someone who currently works in the K-12 setting.
Also in the implementation area there is discussion of using real data in constructing of exams. There is little discussion of using data that has particular pedagogical value. For example, using data that has outliers or a non-straight line pattern.
Potential Effectiveness as a Teaching Tool
- The assessment builder has an extensive collection of questions that address several levels of learning. This site can greatly improve the quality of assessments for introductory courses. A focus on the assessment can improve the focus of the course and spur the instructor to improve the overall instruction.
The discussion sections of the site that examine implementation issues can be very useful for new instructors, or instructors considering changing their courses.
This teaching tool would be valuable for new and veteran instructors of statistics courses. It provides research instruments as well, related to scales, achievement and attitudes toward statistics. The attitude instruments could be very effective as instructors seek to address students' fears and/or dislike for statistics. The Assessment Builder provides instructors with a large database from which to develop their own assessments. Additionally, there is a PowerPoint presentation that could be used related to a first course in statistics. The database, research instruments, and online tests offer solid material that has the potential to be used in a variety of creativity and highly effective ways depending on the user.
- You not only need to register to use the database, but also to acquire the ARTIST online tests. This could be an additional step that might dissuade some instructors from using this resource. By design there are no straight computational problems and some topics that are primarily computational are not included the assessment builder. Hence, a teacher who wants to include these in their assessment could not use this item bank to build his/her entire test.
Ease of Use for Both Students and Faculty
- This web site has easy-to-read, intuitive links, all highlighted in bold, blue. Easy navigation is accessible on the left hand navigation bar and throughout the text. It is a very user friendly site. The question bank is very easy to navigate and requires no special software to use. The novice user can master most functions without special instructions. Exams created by the assessment builder are stored on the ARTIST website and can be retrieved for future use. In the implementation issues section of the site the matrix format allows the user to quickly find the information required.
- In the Assessment Builder the order of questions cannot be changed. To change the order, the user must make the changes manually in the RTF file after it is downloaded. Also, there is no mention of compatibility of the ARTIST Assessment Builder with learning management systems (LMS) such as Blackboard or WebCTt. Although most of the popular LMSs' can directly import the RTF file of questions, there may be formating issues that make the process cumbersome.
The overall site does not include a process for updating personal information. An instructor may wish to change information such as email address without loosing previously created exams. Aparently the only method for doing this is to manually email the site authors. (Editors note: the ARTIST team has recently addressed this concern by making a change in the user interface.)
- Other Issues and Comments:
- The last publication update was July 2006. This is a website worth keeping up-to-date. The fact that the answers are not provided might cause some new instructors to avoid using this resource. Many adjunct professors do not have extensive statistical backgrounds and would benefit from having access to answers and proofs for their reference.
- Creative Commons: