Post a composite review
Unpost a composite review
Search all MERLOT
Click here to go to your profile
Select to go to your workspace
Click here to go to your Dashboard Report
Click here to go to your Content Builder
Click here to log out
Search Terms
Enter username
Enter password
Please give at least one keyword of at least three characters for the search to work with. The more keywords you give, the better the search will work for you.
select OK to launch help window
cancel help

MERLOT II


    

Peer Review


Science Questions: NAEP the Nation's Report Card

 

Ratings

Overall Rating:

5 stars
Content Quality: 4 stars
Effectiveness: 5 stars
Ease of Use: 4 stars
Reviewed: Aug 10, 2002 by Teacher Education
Overview: This website provides easy access to some of the questions NAEP asks students at
4th, 8th, and 12th grades to probe students? knowledge of specific science
knowledge/concepts. Scoring guides and national data make it possible to compare
one?s students? performance on a specific question to that of the students
across the nation. The questions are presented for the use of teachers, parents,
students, and others, but only a small portion of each NAEP assessment is
released. Consequently, the released questions in this tool do not represent
complete coverage of the content, cognitive skills, and range of difficulty in
the NAEP assessment for a particular subject area. The National Assessment of
Educational Progress (NAEP), also known as "the Nation's Report Card," is the
only nationally representative and continuing assessment of what America's
students know and can do in various subject areas. Since 1969, assessments have
been conducted periodically in reading, mathematics, science, writing, U.S.
history, civics, geography, and the arts. This particular site focuses on
science education.
Learning Goals: To become familiar with the type of content assessed by questions on the NAEP
tests (reflecting current educational and assessment practices) and to use the
questions to compare achievement to national results.
Target Student Population: The site is useful for researchers, educators, teachers, and student teachers.
It is especially valuable for prospective science teachers of grades 3 ? 12.
NAEP assesses representative samples of students in certain grades or at certain
ages in public and nonpublic schools in the United States. For the main NAEP
assessment, students are selected from grades 4, 8, and 12 in public and
nonpublic schools. For the state assessments, the number of subjects and grades
selected varies, depending on available funding. (State-level samples have
included public and nonpublic school students since 1994. In 2000, however,
nonpublic schools will not be included in the state-level samples.) For the
long-term trend assessments, students at grades 4, 8, and 11 are sampled for
writing, and students at ages 9, 13, and 17 are sampled for science,
mathematics, and reading. Since 1984, NAEP has been administered every two
years.
Prerequisite Knowledge or Skills: Navigation using a browser.
Type of Material: Database of test questions, scoring rubrics, and national data.
Recommended Uses: In teacher education this would be a good site for a science methods class, an
educational assessment and evaluation course, and/or graduate education research
in education. This database has the potential to provide valid and reliable
measures that could be used as standardized measures to compare results in
classroom action research.
Technical Requirements: Requires version 4.0 or higher of Netscape or Microsoft Internet Explorer
browsers and has been optimized for displays at a resolution of 800 x 600 or
higher. The small font of the question selection charts are not readable in
some browser configurations (Netscape on a Mac).

Evaluation and Observation

Content Quality

Rating: 4 stars
Strengths: NAEP has two major goals in relation to science education in K-12 schools: (1)
to reflect current educational and assessment practices, and (2)to measure
change reliably over time. To meet these dual goals, NAEP selects nationally
representative samples of students who participate in either the main NAEP
assessments or the long-term trend NAEP assessments. Making standardized test
questions public is an important first step toward putting these standardized
tests into perspective. Teachers wonder how they are doing when they teach, but
they rarely have measures that empower them to compare their performance with
that of other teachers. These questions place the assessment of teaching and
learning where it belongs ? in the hands of professional teachers. The site
provides 114 different science education questions asked of 4th, 8th, and 12th
graders. Along with each question, the educator can view performance date,
content classification, scoring guide/key, a range of student response examples
from strong and appropriate to inappropriate, and more data on the collected
information of science education at the time these questions were asked. The
layout of the site allows you to move progressively through the sequenced
content questions from grade 4 to grade 12. This is a very informative site,
especially since other content areas addressed in the NAEP report card can also
be accessed from this site. The questions provide the opportunity for scrutiny
by the stakeholders. Scientists, teachers, and other community members should
all have a voice in what is assessed by the standardized measures that are paid
for by tax dollars. For the main NAEP assessment, students are selected from
grades 4, 8, and 12 in public and nonpublic schools. For the state assessments,
the number of subjects and grades selected varies, depending on available
funding. (State-level samples have included public and nonpublic school students
since 1994. In 2000, however, nonpublic schools will not be included in the
state-level samples.) For the long-term trend assessments, students at grades 4,
8, and 11 are sampled for writing, and students at ages 9, 13, and 17 are
sampled for science, mathematics, and reading. Since 1984, NAEP has been
administered every two years.
Concerns: Some of the questions are deemed invalid by faculty with content expertise. For
example, the evolution question with cladogram answers has no correct answer.
Also, poor content coverage of some topics limits the utility of this database.
In the future, we hope that questions could be posted and screened before they
are actually used on standardized tests.

Potential Effectiveness as a Teaching Tool

Rating: 5 stars
Strengths: Selection and discussion of assessment questions is a very important part of the
teaching and learning process. This site can be used to show how to assess
science education competency but can also be used to help university students
learn about the range of venues within science and how students may respond to
these venues through questioning. A variety of questioning techniques are
presented with supporting data on how students respond. Analysis of data and
information covered in the NAEP report card on science can offer educators a
pertinent and important perspective in teaching. Furthermore, the NAEP testing
results reports combined with review of these questions make discussion of
equity issues in terms of learning outcomes by race and sex more meaningful.
Concerns: None.

Ease of Use for Both Students and Faculty

Rating: 4 stars
Strengths: Navigation, selection of questions, and printing of questions with scoring
rubrics and national data is very easy. Using Internet Explorer, the site is
visually well designed and easy to navigate.
Concerns: Unfortunately, saving the questions for further editing or to combine them with
other questions on a unit or final exam does not work. When viewing certain
pages the download time is long because of the charts and images, so viewers
should be prepared to wait. The test question selection charts used at this web
site were constructed on a PC using Internet Explorer and are unreadable using
Netscape on a Mac. The fonts cannot be changed. A magnifying glass is needed
to read the links within the charts when viewing the site with Netscape.
Furthermore, the links for reviewing the Scorer's Commentary within the Student
Responses section do not work on a Mac using Netscape

Other Issues and Comments: Please continue and increase access to NAEP test questions. This work is
appreciated by teacher educators!