Post a composite review
Unpost a composite review
Search all MERLOT
Select to go to your profile
Select to go to your workspace
Select to go to your Dashboard Report
Select to go to your Content Builder
Select to log out
Search Terms
Enter username
Enter password
Please give at least one keyword of at least three characters for the search to work with. The more keywords you give, the better the search will work for you.
select OK to launch help window
cancel help

MERLOT II


    

Peer Review


Performance Assessment Links in Science

 

Ratings

Overall Rating:

5 stars
Content Quality: 4 stars
Effectiveness: 5 stars
Ease of Use: 5 stars
Reviewed: Aug 10, 2002 by Teacher Education
Overview: Performance Assessment Links in Science (PALS) is a resource library of science
performance assessment tasks that are linked to the National Science Education
Standards (NSES). The tasks include instructions for teachers, directions for
students, assessment rubrics, examples of students' work, and most tasks display
score distributions from actual test results. The Center for Technology in
Learning at SRI International developed PALS as a Web-based approach for sharing
exemplary assessment resources, collaborating on the development of new ones,
and understanding how the use of standards-based performance assessment can
advance science education reform at all levels of the educational system.
Learning Goals: PALS provides quality science tasks for K-12 teachers. This site can assist
teachers in identifying and evaluating lessons that help students achieve the
science concepts and processes listed in the National Science Education
Standards. The PALS resource library includes performance assessment tasks
from multiple sources, such as state assessment programs and consortia and
national reference exams (NAEP, TIMSS, State and National Standards-based
items), and is intended for use by teachers and professional development
organizations. A second password-protected tier is intended to be a secure
Accountability Pool of science performance assessments for use by state
assessment programs and systemic reform programs (e.g., Systemic Initiatives).
All PALS Tasks contain scientific inquiry.
Target Student Population: The PALS website provides science assessment tasks for students from
kindergarten through 12th grade.
Prerequisite Knowledge or Skills: None.
Type of Material: Reference/Resource Materials. Quiz/Test
Recommended Uses: The use of standards-based performance assessment is important for the
advancement of science education reform at all levels of the educational system.
This website could provide quality lessons for classroom and science teachers.
Preservice teachers could use the tasks as models for developing performance
assessment lessons.
Technical Requirements: No special technical requirements.

Evaluation and Observation

Content Quality

Rating: 4 stars
Strengths: The PALS resource was developed using a systematic process for collecting,
formatting, and posting the assessment task components (administration
procedures, student booklet, scoring rubrics, scored student work, and technical
quality information), using tasks contributed and posted by organizations such
as the Council of Chief State School Officers (CCSSO/SCASS), the New York
Department of Education (NYDOE), RAND, and Third International Math and Science
Study (TIMSS). Science tasks were included that had been developed according to
a systematic test development process, including content and sensitivity
reviews; field tested with at least 100 students; reported score distributions;
and had established acceptable levels of interrater reliability. The assessments
are indexed to the National Science Education Standards (NSES). Each task has
a common format for assessment of a National Science Education Standard. In
addition, to insure quality each task includes information of their testing
population.
Concerns: This site plans to continuously build the database of assessment tasks.
Currently, there are National Science Education Standard areas with no matching
tasks. The validity of some of the assessments as a measure of the assigned
Science Standard is doubtful, and no data supports the content validity of the
items. We agree with the report by the SRI assessment team finding that an
over-abundance of performance assessment tasks address concepts from the
physical sciences, with a poorer representation of tasks in the areas of Life
and Earth/Space science. We encourage the PALS project team to continue to seek
out field-tested investigations for under-represented NSES standards. PALS can
provide a wake-up call if quality science performance tasks are not being
developed for a number of the NSES.

Potential Effectiveness as a Teaching Tool

Rating: 5 stars
Strengths: This site could be used in a variety of ways. The tasks could be used by
classroom teachers as a source for ideas about teaching and assessing
standards-based science. Higher education faculty could use the tasks as
examples and models for a standards-based lesson, lesson format, rubrics, or to
survey testing results for population samples. References in the form of
publications and papers presented at the annual meeting of the American
Education Research Association are posted on the PALS Web. Users provide PALS
task ratings, with a comment feature that is outstanding. In response to the
question, "How likely are you to use this task?" the average rating of forty-six
users on a scale of 1(low) to 4(high) was 3.2. Their average rating on the
question "how likely are you to adapt this task?" was 3.5. Users submit
comments about the specific task, the usefulness of the rubrics, examples of
student work, and how easy it was to incorporate or adapt the tasks for their
existing curriculum. PALS serves as an outstanding teacher education tool that
can also serve as a well organized source for the National Science Education
Standards (NSES).
Concerns: The qualifications of those who respond to the task ratings providing the
average rating of 3.2 for "How likely are you to use this task?" and an average
rating of 3.5 for "how likely are you to adapt this task?" have not been
established. It is not clear whether the star ratings for each task represent
the user?s impression of the validity of the task for assessing the specific
NSES or whether the stars simply show that more teachers would use the task.

Ease of Use for Both Students and Faculty

Rating: 5 stars
Strengths: Flexibility and ease of PALS resource navigation is outstanding. Performance
tasks, categorized by grade level, can be selected by NSES, or by discipline.
Administration guidelines, task, rubric, sample student work, and technical
information are easy to access for each task. Each task is clearly categorized
according to multiple National Science Education Standards categories using tabs
on the task home page. Navigation to any other screen from within a task is
possible. A search tool at the bottom of each page works well. Users can
search for PALS tasks relevant to curriculum modules if the program framework
has been posted on the Web site. The architecture allows dynamic generation of
assessment charts to support assessment planning by displaying the PALS tasks
indexed to science standards selected by the user and also to Full Option
Science System (FOSS) and Science and Technology for Children (STC). Ease of
use was supported by the explicit tour that was accessed from the main page of
the website, although one reviewer who skipped the tour had no trouble
navigating the resources. This site would be easy for in-service and preservice
teachers to navigate. All the linked areas for the tasks were labeled clearly
and easy to follow.
Concerns: Links to the Curriculum frameworks (FOSS and STC) are somewhat hidden at the
bottom of the Standards home page.

Other Issues and Comments: PALS displays exemplary technical architecture for a relational database. The
results show what happens when designers conduct studies with individual users
who inform the design and redesign of the interface. In addition to the core
functions for supporting task selection and site navigation, features including
threaded discussion boards, personalized assessment charts (My Chart), user
ratings and discussion of individual tasks, and searching by some state science
standards and curriculum frameworks make PALS a Web site that offers far more
functions and features than other assessment Web sites.The feedback form
provided on each task's main page could be helpful to the evaluation process for
the PALS Steering Committee, providing another exemplary use of the Web. To
provide a model to other Web developers, however, it would be useful to know
what procedures are used to confirm the validity of the comments submitted.