- Peer Review: “Promoting Effective Program Evaluation”
Promoting Effective Program Evaluation
- Feb 14, 2005 by Teacher Education
This interactive online short course includes six lessons that lead the user
through readings on evaluation basics, questions raised by program directors,
and resources available both on and off line. Each lesson is presented in a step
by step format and includes an optional interactive component where the user
develops an evaluation planning document for his/her program. A read only
version is also available. At the end of the short course (if the user has used
the interactive version), the evaluation planning document is sent directly to
the user via email as a Rich Text File (.rtf) that can be opened in a word
processing program for editing.
- Type of Material:
There are six lessons presented; Why Evaluate, Setting Goals and Objectives,
Writing Questions and Setting Priorities, Collecting Data, Considering External
vs. Internal Evaluators, and From Planning to Plan. Each lesson is made up of
five sections, the Introduction, Readings, FAQ's, Resources, and the Lesson
Summary. The user can move between lessons and the lesson's sections as they
- Recommended Uses:
This site is highly versatile; it can be used, 1)to develop a program
evaluation, 2) as a reference or support in evaluating programs and grants,
3) as a resource in writing grants, or 3) as a teaching tool to demonstrate the
process and components of evaluation.
- Technical Requirements:
Adobe Reader for documents in pdf format.
Macromedia Shockwaver Plug-in 8.0 or higher required for the short course.
- Identify Major Learning Goals:
The user will, 1)improve their understanding of the benefits of program
evaluation, 2) improve their skills in developing both formative and summative
evaluation plans, and 3) improve their understanding of the pros and cons of
internal and external evaluations.
- Target Student Population:
Graduate students or faculty PIs of grant applications that require a program
evaluation component. Administrators and project directors in most fields could
benefit from this structured evaluation format. Certainly beneficial for
education administrators with the current emphasis in documenting
progress using 'scientifcally-based' methods for NCLB.
This is also an excellent learning object for graduate students in education.
Upper level education majors could benefit in seeing a step by step process of
evaluation. This could be used by graduate students (and upper level students)
as a project based assignment.
Professors applying for grant monies could also use to assist in writing grant
- Prerequisite Knowledge or Skills:
Some understanding or background in developing goals and objectives would be
helpful, but is not critical.
The goal of this project was to promote the development and continuation of
effective programs by assisting program staff to develop and implement effective
formative and summative evaluation plans.
The tutorial can help the user understand the benefits of program evaluation,
improve skills in developing both formative and summative evaluation plans,
understand the pro?s and con?s of internal and external evaluations; develop an
initial evaluation plan, and begin implementation of their evaluation plan. The
tutorial may contribute to creation of competitive grant proposals.
Promoting Effective Program Evaluation is an extremely versatile and powerful
resource for learning, using, or teaching the process and components of
evaluation. The option of the interactive or 'read only' versions offer the user
variant levels of support and interaction as determined by the user.
Each lesson is presented with clarity, conciseness, and provides specific
examples. The user can navigate within the site quickly and move between lessons
easily. Audio is available during the Introduction section of each lesson which
allows for greater use with a group or audience.
Potential Effectiveness as a Teaching Tool
Audio is available during the Introduction section of each lesson and in the
general overview of the six lessons which allows for greater use with a group or
audience. Because of the clarity in describing the components and process of
evaluation this could be used as a resource to teach the basics of evaluation.
Additionally, due to the interactive version and additional Resources this could
also be used to teach upper level and graduate students about evaluation at
greater depth and range.
The interactive course holds the user's information for sixty days allowing
faculty and students to utilize this as an in-depth teaching tool during the
traditional semester scheduling.
After each of the six lessons specific review questions are posed in which the
user can respond and then receives a message indicating if their response is
correct or not.
Additional resources provided in lesson 4 are a highlight of the course due to
the explicit nature of the data collection instruments to guide observations,
interviews, focus groups, and survey guidelines in pdf format - clear and easy
Would benefit from more diverse actual examples (only a few are provided) ? Perhaps a community of volunteers willing to provide peer review or feedback
would be of interest to some users to convert this course into a more
distributed source for knowledge and leadership development.
Ease of Use for Both Students and Faculty
Eight issues of The PEN: Program Evaluation are among the great features of this
web site. The newsletter is easy to print and it provides information on both
this specific project and on program evaluation in general. Copies of the eight
issue are available as PDF files via links.
Once the user figures out that each oral introduction can be skipped, the
navigation structure to find the content and resources provided in six
lessons for the short course are outstanding. At the time of this review, all
of the external links to additional resources and the pdf files with guidelines
were clear, relevant, and up-to-date.
The site is easy to use;the user can skip, move ahead or go back easily. The
siteprovides links for downloading Adobe and Macromedia; programs that are used
to complete the optional interactive lessons. The interactive version holds the
user's input information for 60 days and then allows the user to save this to a
Rich Text File (.rtf)that can be downloded into a word program.
Some users may not view the short course due to the five minute oral
introduction explaining the course design and objectives. The user may not at
first be aware that the oral components of the course can be skipped.
There is no way to print the outstanding readings and references in the short
course for later reference and review, although the short lesson summaries do
show up when one prints the browser window.
The resolution of the eight issues of The PEN: Program Evaluation is poor ?
particularly for the 1998 issues.
For novice computer users the terms (operating system, .rtf file) and program
names (Macromedia, Adobe) may be a bit intimidating. Can only edit at the end
of the course.