Cover: Steps Toward a Formative Evaluation of NSDL
Read Online

Steps Toward a Formative Evaluation of NSDL

by Tora K. Bikson, Nidhi Kalra, Lionel A. Galway, Grace Agnew

Download eBook for Free

Full Document

FormatFile SizeNotes
PDF file 6.2 MB

Use Adobe Acrobat Reader version 10 or higher for the best experience.

Summary Only

FormatFile SizeNotes
PDF file 0.5 MB

Use Adobe Acrobat Reader version 10 or higher for the best experience.

The National Science Foundation's (NSF) National Science Digital Library/Distributed Learning (NSDL) program turned 10 years old in 2010. This report presents results of a preliminary program evaluation carried out by RAND and is organized around three principal goals:

  • Provide an initial evaluation of NSDL based on existing information resources, including results of a prior phase of evaluation;
  • Conduct limited primary data collection studies to pilot test instruments and approaches and complement findings from extant information; and
  • Design a robust formative evaluation for a final phase of effort, refining research questions and methods.

An interdisciplinary team directed project research efforts to: examine the state of health of the digital resource collections; assess the evidence that currently used web metrics provide about activities users undertake to find and access those resources; and identify early indicators of positive effects from the use of NSDL resources among science teachers or learners in NSDL-supported research projects.

Initial answers to these questions are supplemented by primary data collection aimed at exploring the appropriateness of metadata intended to assist users in finding, identifying, selecting and obtaining sought digital contents and at investigating the usability of available resources for accomplishing selected routine tasks in science teaching. A minicase study explored one full instantiation of the NSDL program model — a demonstration of curriculum customization to meet the needs of earth science teachers using resources from the Digital Library for Earth System Education (DLESE).

The report synthesizes the findings from these exercises to yield results and recommendations, both for what NSF and NSDL should do now as well as for the design of a full formative evaluation in the future.

Table of Contents

  • Chapter One

    Introduction

  • Chapter Two

    Analysis of NSDL Collection Health

  • Chapter Three

    Web Metrics in the Evaluation of the NSDL

  • Chapter Four

    Teaching and Learning Effects

  • Chapter Five

    Metadata Assessment

  • Chapter Six

    Usability Pilot Trials

  • Chapter Seven

    Minicase Study of a Successful Implementation of the NSDL Model

  • Chapter Eight

    Formative Evaluation of the NSDL: Results and Next Steps

  • Appendix A

    Funded Collections Summary of Assessment Data

  • Appendix B

    Pathways Summary of Assessment Data

  • Appendix C

    Web Metrics

  • Appendix D

    NSDL Teaching and Learning Project Synopsis

  • Appendix E

    NSDL Teaching and Learning Study Synopsis

  • Appendix F

    Usability Studies

  • Appendix G

    Sample Find, Identify, Select, and Obtain Instrument

The research in this report was sponsored by the National Science Foundation and was conducted under the auspices of the Transportation, Space, and Technology (TST) Program within RAND Infrastructure, Safety, and Environment.

This report is part of the RAND Corporation Technical report series. RAND technical reports may include research findings on a specific topic that is limited in scope or intended for a narrow audience; present discussions of the methodology employed in research; provide literature reviews, survey instruments, modeling exercises, guidelines for practitioners and research professionals, and supporting documentation; or deliver preliminary findings. All RAND reports undergo rigorous peer review to ensure that they meet high standards for research quality and objectivity.

This document and trademark(s) contained herein are protected by law. This representation of RAND intellectual property is provided for noncommercial use only. Unauthorized posting of this publication online is prohibited; linking directly to this product page is encouraged. Permission is required from RAND to reproduce, or reuse in another form, any of its research documents for commercial purposes. For information on reprint and reuse permissions, please visit www.rand.org/pubs/permissions.

The RAND Corporation is a nonprofit institution that helps improve policy and decisionmaking through research and analysis. RAND's publications do not necessarily reflect the opinions of its research clients and sponsors.