When organizations collect poor data, they get poor results.
Too often, I’ve seen examples of bad surveys, forms, and questionnaires that at best contain some useful information and at worst generate meaningless information. What’s the point of collecting data when it is incomplete, inaccurate, and riddled with error? How can organizations make good decisions when they have bad data? The purpose of this blog post is to provide some tips to collect quality data. Don’t let a weak link in your data break your organization.
1) Identify what you want and need to measure
When you know what you want and need to measure you are one step closer to quality data. It’s very likely that someone has already developed and validated questions and scales that can measure what you want to measure. Try to identify these existing measures. Get permission to use them. Modify them for your organization. Do not think that you need to generate all your questions from scratch. In fact, scratch that idea. For example, if you want to measure self-efficacy or confidence to perform a specific behavior do some research and identify whether other organizations or researchers have developed the tools you need. When I searched “self-efficacy measure” in Google I generated scholarly articles and links to published measures of this construct. By knowing what you want and need to measure you can be confident that you are one your way towards collecting quality data.
2) Select the appropriate data collection method/s
There are various methods in which you can collect your data. The first question you need to ask is: “Are you planning to collect qualitative or quantitative data”. Qualitative data represent open-ended responses, typically generated from focus groups and individual interviews. Quantitative data is a number, numbers you collect from surveys, forms, and questionnaires. Often times, these methods are blended. However, select a data collection method that produces the information you need. For instance, if you want to know a participant’s age, you can ask them to directly specify their age or select their age range from a list of options. It is unlikely that you would ask open-ended questions about their age and what it means to them. When in doubt, talk to your colleagues about how they have collected data on similar topics.
3) Create a system for collecting your data
Once you have identified what you want and need to measure how you plan to collect this data – it is time to create a system. In other words, how will you ensure that you collect data from your participants in a consistent and neutral manner? Typically this involves developing a set of procedures that outline how the data will be collected from participants. In the research world, this likely would involve developing a consent form, identifying your pool of participants, contacting your participants and inviting them to participate in your study, sending a follow-up email to solicit participation, debriefing participants after they participated, etc. After the data is collected, how will you store this information? How will you protect participants’ information? When you start to reflect on these questions, you can develop a system that will work for your specific project. The important thing is that you treat your participants ethically and do everything you can to implement your system.
4) Train your staff
Your staff plays a major role in determining the quality of your data. From designing the survey to conducting focus groups and interviews to entering, managing, cleaning, and analyzing data, it is critical that your staff know what they are doing. It has been my experience that anyone who interacts with the data needs training. Training can take the form of professional development opportunities, conferences, webinars, reading books on the topic, taking university classes, and working with researchers and evaluators. Make sure everyone is on the same page and leave little to no room for interpretation when it comes to working with the data.
5) Ensure data integrity
Data integrity involves the protection of the data you collect. By ensuring data integrity, your organization produces data that is more accurate and consistent. Participant confidentiality is part of ensuring data integrity. For instance, I highly recommend that you protect your participant’s identity by creating a unique identifier for each participant. Your participant’s name shouldn’t be associated with their responses, especially if this is sensitive information. It’s also important to make the programs that you are using to house the data (e.g., Excel, Microsoft Word, SurveyMonkey) password-protected so that others can’t access this information. For further information on this topic, I highly recommend the free training that is offered through the CITI Program. This certification program covers the APA Code of Ethics and encouraging your organization to take this training will help you collect quality data.
6) Collaborate with researchers and evaluators
Work with the experts! If you are not a researcher or evaluator by training, you can learn a lot from these folks! Good researchers and evaluators know how to collect quality data. They do this every day, they learn about best practices in school, they have learned through experience and they are there to guide you! Think of it this way, if you need to fix your car would you go to a mechanic or would you do it yourself? Sure you could teach yourself, ask friends, look up information online, but the easiest option is to trust your car with the person who has this kind of training. Researchers and evaluators can save you time, money, and the headaches that come from poorly designed research designs and data collection strategies. Trust me, I know! I’ve seen too many instances in which organizations could have avoided mistakes. Don’t be that organization that tries to save some money in the short-run only to pay the consequences in the long-run.
Feel free to contact me with any research and evaluation questions or needs at annette@researchevaluationconsulting.com
very informative piece of literature
A very well summarized quality and informative piece.