In Part One of this blog series we discussed six key challenges that can arise when conducting evaluations in the research world. Evaluation work is tough; mistakes can be made, big complications can arise and the challenges can feel overwhelming. Here we discuss solutions to the evaluation challenges that we highlighted in Part One, which we hope will guide you toward more effective evaluations in the future.
Challenge 1: Poor Planning
Solution: Plan Ahead
If you don’t properly plan for an evaluation it won’t go well. It’s that simple. So what is the solution? It’s also simple – do your homework and plan ahead. Be sure to understand the program or project to be evaluated. Conduct a thorough literature and document review for the project, create a timeline detailing the data collection efforts, make a logic model for the evaluation and stick to it.
Challenge 2: Lack of Readiness
Solution: Reach Out
Get input from key internal and external stakeholders to the project before starting the evaluation to gauge their commitment and understanding. If those are lacking, additional outreach and education can help participants understand the value of the evaluation. Determining the desired outcomes for the evaluation and outlining how to achieve them ahead of time is also key to having all aspects of the project run smoothly.
Challenge 3: Ineffective Approaches
Solution: Do Your Research
Before starting an evaluation, research which evaluation approach is the right one for the project. Evaluations are not a one-type-fits all; different approaches work with different programs. Once you’ve picked an approach, create an evaluation plan and a logic model and communicate with key stakeholders about the evaluation effort so that everyone understands and is supportive of the approach. Also, make sure to seek outside help from a research evaluator if you need it.
Challenge 4: Bad Questions
Solutions: Ask Better Questions
Asking the right questions is key to getting you the results you’re looking for in an evaluation. Examples of useful questions include:
- What is the need for the program?
- Was the structure of the program appropriate?
- Was the program well-implemented?
- Was the program responsible for the outcomes that actually occurred?
- Was the program adequate, efficient, relevant, cost beneficial?
- Did the program achieve its intended objectives?
Challenge 5: Bad Data
Solution: Collect Quality Data
To collect high quality data you need high standards for data collection. Research and identify the existing reliable and valid measures available for the evaluation. Choose the right data collection method for the project, have a clear protocol for data entry, and ensure that staff are properly trained in and understand the importance of the protocol.
Challenge 6: Too Much Data
Solution: Be Efficient
As we’ve highlighted, developing a logic model and an evaluation plan are key ways of solving many of the challenges involved in an evaluation, and the same is true for collecting the right amount of data. In addition to those tools, identifying clear research and evaluation questions and identifying the desired outcomes for the project are crucial elements to successfully collecting exactly the data you need, and that nothing extraneous is included.
Thank you for reading our two-part series on evaluation challenges and solutions. We hope you’ve gained valuable insight on how to successfully implement an evaluation. If you have any specific follow-up questions to this blog post, or any other research and evaluation needs, please free to contact Dr. Annette Shtivelband.