Program Evaluation

2018 Revised Annotated Bibliography of Postsecondary Peer Cooperative Learning Programs

I am happy to announce that the 2018 Revised Annoated Bibliography of Postsecondary Peer Cooperative Learning Programs is now available to download. Click on this link for a PDF or Word version of it.

There are now nearly 1,500 entries spanning 488 pages. I noticed recent listserv conversations about locating research studies to support Supplemental Instruction programs or similar approaches operating at the college level. The directory grew significantly in the past two years. The directory includes the Emerging Scholars Program (Dr. Uri Treisman model), Peer-Led Team Learning, Supplemental Instruction, and Video-based Supplemental Instruction, Structured Learning Assistance, Accelerated Learning Groups, and Peer Assisted Learning.

You can download the directory as a PDF or Word document. I also included some sub-topics of the directory such as facilitator development, vocational influence, identity development and more. I also provide the EndNote library file to allow you to more easily search the database for the topic you want.  Be sure also to download the keyword search guide to discover all the ways to search the contents for the information you want. Other bibliographic database systems may be able to open the EndNote file but I am not an expert with that process.

No doubt I missed some citations related to these seven major peer learning programs. Please send me the citation and perhaps a copy of the publication and I will be happy to include in an update. Thanks for consideration.

Reaping what you sow: How the University of Bedfordshire uses experienced Peer Assisted Learning (PAL) students to inspire and nurture future generations of PAL leaders.

Rapley, E. (2015). Reaping what you sow: How the University of Bedfordshire uses experienced Peer Assisted Learning (PAL) students to inspire and nurture future generations of PAL leaders. Journal of Pedagogic Development, 5(2). Retrieved from https://journals.beds.ac.uk/ojs/index.php/jpd/article/view/172/266

As staff awareness and understanding of Peer Assisted Learning (PAL) has continued to develop, a conscious decision has been made to hand over greater responsibility and ownership of PAL to the PAL Leader student team. PAL is based on the Supplemental Instruction (SI) model with a broader interest in holistic development of the students beyond just subject course competence.  The success of any PAL initiative rests upon the quality of the PAL Leaders who facilitate the sessions. Motivated, committed and enthusiastic PAL Leaders are key to ensuring that engaging and meaningful sessions are provided for first year students. With our mission to ensure PAL Leaders truly benefit and develop themselves during their tenure, it was felt that this transformation could only take place if PAL Leaders really had opportunities to step up and take ownership of PAL.

To download the complete annotated bibliography of more than 1,100 citations on postsecondary peer cooperative learning programs, click on the following link, http://z.umn.edu/peerbib

2016 Annotated Bibliograpy of Postsecondary Peer Cooperative Learning Programs Updated

For many years I have maintained an annotated bibliography of publications about peer learning programs at the postsecondary level. I wanted to share it more widely with others so it is provided in several forms:  PDF, Word, and EndNote database.  Please observe the license under which it is made available for your personal and scholarly use.  The unabridged version of the bibliography is now 399 pages.  [Click this link to reach the annonated bibliography page, http://z.umn.edu/peerbib ]  If you download the EndNote database, be sure to also download the keyword guide which I created to code each entry to make searching easier.

This annotated bibliography does not attempt to be inclusive of this broad field of literature concerning peer collaborative learning.  Instead, it is focused intentionally on a subset of the educational practice that shares a common focus with increasing student academic achievement and persistence towards graduation.

The seven student peer learning programs included in this bibliography meet the following characteristics: (a) the program must have been implemented at the postsecondary or tertiary level; (b) the program has a clear set of systematic procedures for its implementation that could be replicated by another institution; (c) program evaluation studies have been conducted and are available for review; (d) the program intentionally embeds learning strategy practice along with review of the academic content material; (e) the program outcomes include increased content knowledge, higher final course grades, higher pass rates, and higher college persistence rates; and (f) the program has been replicated at another institution with similar positive student outcomes. From a review of the professional literature, six programs emerged: (a) Accelerated Learning Groups (ALGs), (b) Emerging Scholars Program (ESP), (c) Peer Assisted Learning (PAL), (d) Peer-Led Team Learning (PLTL), (e) Structured Learning Assistance (SLA), (f) Supplemental Instruction (SI), and (g) Video-based Supplemental Instruction (VSI).  As will be described in the following narrative, some of the programs share common history and seek to improve upon previous practices.Other programs were developed independently.

Regrets for scholarship I have overlooked.  Please send me items you think should be included in the next edition.  Happy reading.

Updated Postsecondary Peer Cooperative Learning Groups Annotated Bibliography(Updated 1/1/2015)

Postsecondary Peer Cooperative Learning Programs: Annotated Bibliography by David R. Arendale is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.  Based on a work at http://z.umn.edu/peerbib

Background on the Bibliography

For many years I have maintained an annotated bibliography of publications about peer learning programs at the postsecondary level. I wanted to share it more widely with others so it is provided in several forms:  PDF, Word, and EndNote database.  Please observe the license under which it is made available for your use.  Links to versions of the bibliography are at the bottom of this page.

This annotated bibliography does not attempt to be inclusive of this broad field of literature concerning peer collaborative learning.  Instead, it is focused intentionally on a subset of the educational practice that shares a common focus with increasing student persistence towards graduation.  At the end of this overview, several suggestions are made for differentiating the models from each other and the level of institutional resources and resolve with implementing them.

The six student peer learning programs included in this bibliography meet the following characteristics: (a) the program must have been implemented at the postsecondary or tertiary level; (b) the program has a clear set of systematic procedures for its implementation that could be replicated by another institution; (c) program evaluation studies have been conducted and are available for review; (d) the program intentionally embeds learning strategy practice along with review of the academic content material; (e) the program outcomes include increased content knowledge, higher final course grades, higher pass rates, and higher college persistence rates; and (f) the program has been replicated at another institution with similar positive student outcomes. From a review of the professional literature, six programs emerged: (a) Accelerated Learning Groups (ALGs), (b) Emerging Scholars Program (ESP), (c) Peer-Led Team Learning (PLTL), (d) Structured Learning Assistance (SLA), (e) Supplemental Instruction (SI), and (f) Video-based Supplemental Instruction (VSI).  As will be described in the following narrative, some of the programs share common history and seek to improve upon previous practices.Other programs were developed independently.

Versions of the Bibliography for Downloading

Click on this link to download the bibliography as a PDF format document (Updated 1/1/2015). This version will always be months behind the current database.  If you want the most recent database, download the EndNote database file below and you can create your own custom print version of the latest citations.

Click on this link to download the bibliography as a Word document which you can easily edit (Updated 1/1/2015).  Using Microsoft Word software makes it easy to edit the bibliography as you like and use the search engine to find key words of your own choice.

Click on this link to download the actual database file of the bibliography in the EndNote format (Updated 1/1/2015). This file has been "compressed" and will require "unzipping" to open and use it with EndNote.  Click on the above web link and "save" the file to your computer (I recommend saving it to the desktop to make it easy to find.)  If you are unsure how to import into your own copy of EndNote, talk with someone who knows or search for the answer through Google and YouTube.  Use of this database requires purchase of the EndNote software or importing into another citation reference manager.  There are other free citation management systems such as Zotario.  It is possible to import this database into these other software systems.  However, I can not provide technical information how to do so.

Click on one of the two links below to download the Directory of Keywords I created to code the database entries and make it easier to search through EndNote: [Word document version]  [PDF format version]  (Updated 5/8/14) While you can search the bibliography by keywords within the titles or abstract, many of my additional keywords added to the database entry will not appear within the text.  Using EndNote's search function along with this list of the keywords I used to index it will allow more productive searchers.  It would be easy to create custom bibliographies as needed.  For example, "SI" plus "science" plus "academic achievement" would create a custom bibliography of every SI research document that included data for improved academic achievement of participating students.

Impact of Linked Learning Communities with Higher Student Outcomes

Effectiveness

Overall, the effects of linked learning communities on academic achievement, degree attainment, postsecondary enrollment, credit accumulation, and progress in developmental education for postsecondary students were neither statistically significant nor large enough to be considered to be substantively important. Therefore, the WWC considers linked learning communities to have no discernible effects on these outcomes for community college students in developmental education.

Program Description

Linked learning communities in postsecondary education are programs defined by having social and curricular linkages that provide undergraduate students with intentional integration of the themes and concepts that they are learning. Linked learning communities are based on the theory that active learning in a community-based setting can improve academic outcomes by increasing social as well as academic integration. To that end, linked learning communities tend to incorporate two characteristics: a shared intellectual theme with a linked or integrated curriculum and a community or common cohort of learners.

Research

The What Works Clearinghouse (WWC) identified six studies of linked learning communities in postsecondary education that both fall within the scope of the Developmental Students in Postsecondary Education topic area and meet WWC group design standards. All six studies meet WWC standards without reservations. Together, these studies included about 7,400 undergraduate students across six community colleges.

The WWC considers the extent of evidence for linked learning communities to be medium to large for four outcome domains—academic achievement, postsecondary enrollment, credit accumulation, and progress in developmental education. These outcomes were assessed in all six of the studies that met WWC group design standards. The WWC considers the extent of evidence for linked learning communities to be small for one outcome domain—degree attainment.

This intervention report was prepared for the WWC by Development Services Group, Inc. under contract ED–IES–12–C–0084.  To download the complete report, click on this link.

Validated SSS Practice Added: Integrated Learning Course for Entering SSS College Students

Integrated Learning Course for Entering SSS College Students.  University of Minnesota (approved Validated Practice 8/10/14)  In 1972, the TRIO program leaders at the University of Minnesota developed the Integrated Learning (IL) course to meet academic and transition needs of their Upward Bound (UB) students.  These courses were offered during the UB summer bridge program for its students who were concurrently enrolled in academically-challenging college courses following graduation from high school.  Later, use of IL courses shifted from the UB program to the college-level TRIO Student Support Services program.  Long before the widespread use of learning communities within higher education, the IL course is an example of a linked-course learning community.  A historically-challenging course like an introductory psychology is linked with an IL course.  The IL course is customized to use content of its companion class as context for mastering learning strategies and orienting students to the rigor of the college learning environment.  For the past four decades, the IL course approach has assisted TRIO students improve their academic success in the rigorous academic environment as well as acclimate to the social climate of the University of Minnesota (UMN), one of the largest universities in the United States.  UMN is a Research I Intensive public university with highly selective admissions and high expectations for students by the course professors.  Two quasi-experimental studies examined the possible benefits of the IL course.  One was in connection with a General Psychology course. The IL course students earned statistically significantly higher final course grades than nonparticipants.  Another study with a General Biology course replicated the results of higher final course grades for the IL course students.  The IL courses fostered not only higher final course grades, but also expanded positive study behaviors and their metacognitive skills necessary for academic success.  [Click on this link to download this best education practice.]

Tools to Help with Program Evaluation

Regional Educational Laboratory Pacific at McREL has recently released a set of tools for educators to use in designing and evaluating programs. These tools help educators frame discussions around using data, create logic models, and measure outcomes of programs.

Education Logic Model Tool: Logic models are an invaluable tool for education program planning, collaboration, and monitoring. They visually represent a program’s structure, process, and goals and help practitioners and evaluators better understand a program’s mechanics and structure and chart a course toward improved policy and practice. The Education Logic Model Tool guides you through a series of questions and providing opportunities to enter your program resources, activities, outputs, and outcomes. The end result is a printable logic model that functions as a map for you and your team, visually connecting your intended activities with your intended outcomes.  For more information, please see the IES guides Logic Models: A Tool for Designing and Monitoring Program Evaluations and stay tuned for more information about a forthcoming guide, Logic Models: A Tool for Effective Program Planning.

Program Outcomes, Measures, and Targets Tool: Track your progress toward meeting program goals with this tool. It guides users through the process of naming indicators, measures, and targets for your program’s short-, mid-, and long-term outcomes. This information is then displayed in a color-coded interactive dashboard to illustrate progress that’s on-track, behind, or exceeding the specified target.
Interested in finding out more about program monitoring and its connection to effective leadership? Download the IES guide Understanding Program Monitoring: The Relationships Among Outcomes, Indicators, Measures, and Targets and stay tuned for more information about a forthcoming guide, Program Monitoring: The Role of Leadership in Planning, Assessment, and Communications

Guide and Webinar: Join REL Pacific on Wednesday, March 12 for a webinar on“Facilitating Data-Informed Conversations: 5 Steps Forward." This live, interactive webinar, designed for educators, administrators, and researchers, will feature an overview of REL Pacific's Five steps for structuring data-informed conversations and action in education guide. Learn about the five key steps in using data for informed decisionmaking and strategic action: setting the stage, examining the data, understanding the findings, developing an action plan, and monitoring progress and measuring success. Using guiding questions, suggested activities, and activity forms, the guide provides education data teams with a framework and the tools and vocabulary needed to support informed conversations around data.