Provider Resiliency Evaluation Project: More Information for Awardees
Overview
The Health Resources and Services Administration (HRSA) has contracted NORC at the University of Chicago to provide a comprehensive evaluation of Programs Supporting the Mental Health of the Health Professions Workforce including three HRSA funded programs:
- Health and Public Safety Workforce Resiliency Training Program (HPSWRTP)
- Promoting Resilience and Mental Health among Health Professional Workforce (PRMHW)
- Health and Public Safety Workforce Resiliency Technical Assistance Center (HPWRTAC)
This evaluation will help inform HRSA about the progress, cost and benefits, and impact of program efforts to support healthcare delivery in the United States.
Results from the evaluation will be used to improve future programs and services to support the well-being of the health workforce. Evaluation activities include the Healthcare Workforce Survey, Fielding Tracker, Awardee Training and Services Report, Awardee Cost Workbook, Awardee Organizational Assessment Interviews, and Awardee Survey about the Technical Assistance Center.
The goals of the Provider Resiliency Evaluation include:
- To assess the effectiveness of the programs in promoting resiliency and addressing burnout in the health workforce.
- To understand the progress of program implementation and identify potential improvements in processes and cost-effectiveness.
- To develop recommendations and actionable strategies that HRSA can use to inform and enhance future programming and investment strategies.
- To measure the impact of program activities by comparing their outcomes with those of a relevant comparison group.
Join Us for Office Hours
NORC will be hosting office hours for the Health and Public Safety Workforce Resiliency Training Program and the Promoting Resilience and Mental Health among Health Professional Workforce program staff. Download the schedule below and email norceval@norc.org for the link to join the office hours.
Resources & Tools
Awardee Survey About the TAC
NORC at the University of Chicago is conducting an evaluation on behalf of the Health Resources and Services Administration (HRSA) to better understand the impact and implementation of program efforts to promote resiliency and mental health in the health workforce. As part of the evaluation, HRSA would like to understand your experience with the training and technical assistance (TA) provided by the Technical Assistance Center (TAC), also known as the Workplace Change Collaborative (WCC), at George Washington University. We greatly value your organization’s feedback on the WCC and it is critical to guide TA for future HRSA programs that aim to reduce burnout and improve resiliency. NORC has developed a 15-minute online survey to assess the trainings, TA activities, and other resources the WCC provided to your organization and others implementing grant-funded initiatives. NORC will invite you to take the survey about the TAC via an email invitation. We understand that your time is valuable; you will be able to save your responses and come back to the online survey if needed. If you wish to collect information from your team to answer the survey questions, we encourage you to do so.
To review the survey questions in advance or print the questions out for reference go to Awardee Survey About the TAC (PRMHW program) or Awardee Survey About the TAC (HPSWRT program) or email NORC at NORCeval@norc.org.
If you or your team have any questions about the survey, please visit our frequently asked questions (FAQs) at tacsurveyfaqs.norc.org.
Awardee Cost Workbook
The Cost-Benefit Assessment being conducted by NORC at the University of Chicago aims to understand the costs and potential benefits of the Health and Public Safety Workforce Resiliency Training Program and the Promoting Resilience and Mental Health among Health Professional Workforce program funded by the Health Resources and Services Administration (HRSA).
This assessment will ultimately help HRSA better understand the costs and potential benefits of these programs and will inform recommendations for future program incentives and investment strategies.
The assessment will estimate the overall cost of Awardee interventions by dividing related costs into categories. The information for the different categories is collected each year through a data collection tool called the Cost Workbook. The Cost Workbook contains the following sections:
Project Year: Identifies the project year corresponding to the activities reported in the workbook. Each cost workbook will encompass the costs associated with one year of grantee project activities.
Personnel: Captures name, role, and labor expense for paid, contract, and in-kind employees supporting the grantee project.
Labor Allocation: Divides the time each person spent supporting the project by activities related to intervention development, intervention delivery, recruitment, evaluation and research, and/or management and other HPSWRTP/PRMHW related activities.
Contracted Services: For grantee project related services delivered by contracted organizations. Divided into broad categories related to repair and maintenance, security, advertising/marketing, or other services.
Participant Information: Captures the number of participants by profession type and particular training activity, as defined by each grantee.
Facility Costs: Costs the grantee project incurred to use, rent, or otherwise pay for space and facility related costs for the project reporting period.
Supplies and Materials: Collects costs of materials including software, office supplies, local travel related costs, etc.
Other Costs: Includes costs not captured in other categories including continuing education fees or costs, out-of-town related travel expenses, participant incentives, and/or training fees, etc.
Overhead and Administrative Charges: Indirect costs and fringe benefits.
Attrition and Turnover: The attrition rate represents the proportion of employees leaving an organization during a set period, based on average employment numbers. The turnover rate shows how well the organization fills its vacancies by comparing new hires to recruited positions.
This information will be collected annually and compiled at the end of the three-year grant period.
Resources & Tools
Cost-Benefit Assessment Videos
Introduction
Cost Benefit Assessment. Provides an overview of the Cost Benefit Assessment, an introduction to the Cost Workbook tool, and a review of the types of data collected.
Section A – Project Year. Identifies the project year corresponding to the activities reported in the workbook. Each cost workbook will encompass the costs associated with one year of grantee project activities.
Completing the Annual Performance Report Data Sections
Section E – Participant Information. Captures the number of participants by profession type and particular training activity, as defined by each grantee.
Section J – Attrition and Turnover.
Completing the Expense Sections
Section D – Contracted Services. For grantee project related services delivered by contracted organizations. Divided into broad categories related to repair and maintenance, security, advertising/marketing, or other services.
Section F – Facility Costs. Costs the grantee project incurred to use, rent, or otherwise pay for space and facility related costs for the project reporting period.
Section G – Supplies and Materials. Collects costs of materials including software, office supplies, local travel related costs, etc.
Section H – Other Costs. Includes costs not captured in other categories including continuing education fees or costs, out-of-town related travel expenses, participant incentives, and/or training fees, etc.
Section I – Overhead and Administrative Charges. Indirect costs and fringe benefits.
Completing the Staffing Sections
Section B – Personnel. Captures name, role, and labor expense for paid, contract, and in-kind employees supporting the grantee project.
Section C – Labor Allocation. Divides the time each person spent supporting the project by activities related to intervention development, intervention delivery, recruitment, evaluation and research, and/or management and other HPSWRTP/PRMHW related activities.
Healthcare Workforce Survey
The Healthcare Workforce Survey is a web-based survey that will ask respondents about their experiences participating in HRSA-funded organizational activities to improve resiliency and address burnout among healthcare workers and students. NORC will use these survey responses to assess the impact of various organizational activities funded by HRSA, as well as to make recommendations to improve trainings and resources to better support the well-being of the healthcare workforce. Questions will ask respondents about their work environment, resiliency, burnout in the workplace, and their experiences with organizational trainings, classes, services, or other activities. The survey should take no more than 15 minutes.
This survey has been drafted by NORC and will be administered by the awardee organizations. Additional resources for the administration of this survey are available below.
Fielding Tracker
The Fielding Tracker is an Excel file used to capture basic information about the data collection progress of the Healthcare Workforce Survey (see above). Inputs include the overall number of emails sent, the dates the emails were sent, the time of day the emails were sent, and number of undeliverable emails sent. The fielding tracker also captures demographic information on the target population, including age, gender, race, ethnicity, and type of profession to assess the target population’s representativeness.
Awardee Training & Services Report (ATSR)
The Awardee Training and Services Report serves as a vital tool for the overall evaluation. Accurately completing the Awardee Training and Services Report will help the evaluation team collect comprehensive information from awardees regarding the training and services provided to participants across the Health and Public Safety Workforce Resiliency Training Program and the Promoting Resilience and Mental Health among Health Professional Workforce programs.
This report's purpose is to confirm and gather information about the activities funded within each grant:
- Training/ Service/ Other Activity Name: Name of the training, service, or other activity listed in the grant
- Participant Facing Name: Name of the offering as known by the participants
- Training/ Service/ Other Activity: Categorization of the training, service, or other activity, distinguishing whether it is a Training, Service, or falls under the category of Other Activity
- Type of Training/Service/Other Activity: A brief description of the training, service, or other activity
- Number of Times the Training/Service/Other Activity Offered: The number of times the training, service, or other activity was conducted during the current program year
- Average Time Individual Participants Spend on Training/Service/Other Activity per Offering: The average time participants typically spend on the Training, Service, or Other Activity during each training, service, or other activity
- Goals of the Training/Service/Other Activities: The objectives and goals associated with Training, Service, or Other Activity
This information will be essential for helping survey participants recognize awardee grant activities, and for us to better understand key information about awardee programs. Our ability to measure the impact of Awardee programs depends on the clarity of the information provided in this report.
Resources & Tools
Awardee & Organizational Assessment Interviews
Interviews with Health and Public Safety Workforce Resiliency Training Program and the Promoting Resilience and Mental Health among Health Professional Workforce program staff will help us better understand lessons learned and barriers and facilitators to implementing awardee programs.
As part of this evaluation, NORC and Change Matrix are interviewing staff from all awardee organizations to learn more about their program. This is the first time that HRSA’s Bureau of Health Workforce has invested in resiliency programs of this kind, so program experience and feedback is incredibly valuable.
