Workshopping Questions to Evaluate Your Humanities Program

At the Virtual National Humanities Conference, which took place earlier this month, Emily McDonald and I conducted a workshop on evaluating the impact of humanities programming. The workshop aimed to introduce participants to our Impact Survey Toolkit, as well as to offer tips about how to write stronger questions for surveys they might be pulling together. In addition, participants had the chance to discuss programs they were considering evaluating and practice writing some questions for those programs.

The Impact Survey Toolkit, which the National Humanities Alliance published in March of this year and introduced at our annual meeting, offers humanities professionals a quick introduction to surveying humanities programs, with advice on identifying goals, constructing quantitative and qualitative questions that align with those goals, and then analyzing and using survey results. To facilitate survey design, the toolkit includes questions grouped by type of impact area and downloadable surveys for a range of programs. It is a growing resource—most recently, we added a survey for public programs that address the COVID-19 pandemic.

Emily offered quick tips for writing strong questions. These included making sure to measure one thing at a time (avoid the word “and” in questions!) and pairing quantitative questions with qualitative questions to enhance the usefulness of both measures. She also went over the kinds of Likert scales you might use in quantitative questions: scales of agreement, frequency, quality, and importance.

In conversation, participants identified a few areas or programs that they were interested in assessing, such as grant programs (for state humanities councils) and reading and discussion programs tailored for different populations. Given the realities of the COVID-19 pandemic, there was a special focus on the difficulties inherent in evaluating online programs. It can be difficult to get participants to respond to virtual surveys. And virtual programs might have their own set of impacts. How do we identify those? We addressed many of these issues as a group. And because NHA has been working with partners to survey online programs, we were able to offer a few questions that have worked well over the past few months. One question documents the a benefit of participating in online programs:

I appreciated the opportunity to connect with others during this time of physical distancing.

Others document changes in motivation as a result of the program.  These may showcase some of the benefits cultural institutions accrue by offering virtual programs as well as educational benefits to participants:

As a result of this program, I feel motivated to visit [name of institution].
As a result of this program, I feel motivated to visit other cultural institutions.
As a result of this program, I feel motivated to participate in other continuing education opportunities.
As a result of this program, I feel motivated to learn more about my own cultural heritage.

If you participated in the Virtual National Humanities Conference, you can view a video of the workshop through December 6. And anyone can access the Impact Survey Toolkit by visiting the National Humanities Alliance website.


Read more: neh for all