Using Technology to Promote Postsecondary Success and Savings

An Evaluation of the University of Texas at Tyler's PATSS Initiative

by Trey Miller, Jesse M. Cunha, Julie A. Delello, Rafiq Dossani, Thomas S. Marzilli

Download eBook for Free

FormatFile SizeNotes
PDF file 1.5 MB

Use Adobe Acrobat Reader version 10 or higher for the best experience.

Research Questions

  1. Does the PATSS program provide high-quality, flexible courses for students?
  2. Do participants — both students and faculty — achieve cost savings from the PATSS program?
  3. What are the academic outcomes in PATSS courses relative to traditional courses?

In 2013, the University of Texas at Tyler (UTT) began the implementation of the PATSS (Patriots Accessing Technology for Savings and Success) program, an initiative that seeks to increase the use of online technologies for instruction. A portion of a course's classes — a third, half, or two-thirds, depending on the course — is held face-to-face, while the remaining portion is provided online. UTT faculty members who teach PATSS courses are required to attend an extensive training program on how to create effective online material and have access to professional instructional designers to help develop hybrid courses.

This research evaluates the implementation of the PATSS program, looking at student academic performance, course evaluations, and cost savings to students, educators, and the university. The authors found that while PATSS had a slightly negative impact on ratings given in student course evaluations, it had no effect on academic performance and provided a monetizable benefit of $80 per student per year. The research itself has already driven improvements to the PATSS program, resulting in clearer and better communication to students, as well as more-refined training for educators. The authors recommend that the PATSS system continue under a more centralized authority and note that faculty would benefit from continued professional development.

Key Findings

Hybrid instruction courses provide near-identical student outcomes, cost savings

  • There was little discernible impact of hybrid instruction PATSS courses on student academic outcomes relative to non-PATSS courses.
  • PATSS had a slightly negative impact on student course evaluations.
  • PATSS provided a net monetizable benefit to all stakeholders of approximately $225,000 annually, or $80 per student per year.

Research results in data-driven improvements to hybrid instruction format

  • UTT developed a process to ensure that accurate information on course format is included in the course schedule at the time of registration.
  • Courses shifted toward professionally produced video lectures broken into digestible pieces.
  • Administrators adapted PATSS training and professional development opportunities to provide more technical training.

Overall findings show that hybrid courses can produce savings, but coordination is key

  • Shifting toward hybrid courses can produce savings without harming student performance.
  • Reducing commuting needs can produce significant savings for both students and faculty.
  • Coordinating course schedules to allow students and faculty to benefit from reduced commuting needs is challenging.
  • Coordinating facilities and classroom space to capitalize on reduced needs can be challenging.
  • Achieving faculty buy-in is difficult but crucial.
  • The extent of implementation of PATSS varied considerably across academic departments and colleges at UTT, and the administration had little control over whether and to what extent faculty engaged with the program.

Recommendations

  • Develop a process to ensure that accurate information on course format is included in the course schedule at the time of registration.
  • For hybrid or online-only courses, use professionally produced video lectures broken into digestible pieces.
  • Adapt training and professional development opportunities to provide more technical training on how to use particular technology and develop pedagogy.
  • More-centralized authority could be useful for future institution-wide reform efforts on the implementation of online-learning initiatives.
  • Faculty could benefit from continued professional development opportunities.

Table of Contents

  • Chapter One

    Introduction

  • Chapter Two

    Quasi-Experimental Impact Evaluation of PATSS

  • Chapter Three

    Qualitative Data for Continuous Improvement

  • Chapter Four

    Estimation of the Costs and Benefits of PATSS to Students, Faculty, and Institution

  • Chapter Five

    Conclusion

  • Appendix A

    Data and Methods

  • Appendix B

    Qualitative Data Instruments

  • Appendix C

    Estimating the Impact of Hybrid Courses on Commuting Behavior

Research conducted by

The research described in this report was funded by the University of Texas at Tyler (UTT) and conducted by RAND Education and Labor.

This report is part of the RAND Corporation research report series. RAND reports present research findings and objective analysis that address the challenges facing the public and private sectors. All RAND reports undergo rigorous peer review to ensure high standards for research quality and objectivity.

Permission is given to duplicate this electronic document for personal use only, as long as it is unaltered and complete. Copies may not be duplicated for commercial purposes. Unauthorized posting of RAND PDFs to a non-RAND Web site is prohibited. RAND PDFs are protected under copyright law. For information on reprint and linking permissions, please visit the RAND Permissions page.

The RAND Corporation is a nonprofit institution that helps improve policy and decisionmaking through research and analysis. RAND's publications do not necessarily reflect the opinions of its research clients and sponsors.