Like Holmes, we need to be strategic about how we answer these questions so that the data we acquire is relevant, sound, and efficiently procured. We need a data collection strategy.
This blog will continue the ongoing Actionable Impact Management series with this theme in mind. Previous blogs laid the groundwork for this step in their exploration of Ground-work (e.g. establishing a Theory of Change model) and Impact Metrics Discovery (choosing impact metrics).
Moving forward now with impact metrics identified, it’s time to map out a high level strategy for collecting data.
As with the metrics discovery phase, Aravind Eye Care System will again be used as an applied example. These are the sample impact metrics previously chosen for our Aravind program:
So, with a metrics plan established for Aravind Eye, we now need a strategy for acquiring those data. Let’s break it down.
Devising a data collection strategy
This is about assessing existing capacity, identifying data sources, and defining a means of managing data flow. Ultimately, this process will help plot a feasible path to implementation so that the data acquired is relevant and effectively managed.
It is helpful at the outset of this process to break down each of the established metrics separately, identifying the data collection needs for each. Then, you can examine them as a whole to see where there might be overlap in offline and online data collection processes for a more streamlined approach.
In the case of Aravind Eye we identified the following social impact metrics to assess the program area: cataract surgery for low-income patients.
We’ll use the following questions to guide our capacity assessment process:
- Do we already collect this data?
- If yes, how is the data managed and is it efficient?
- If no, how might we begin to collect and manage such data?
- How might we aggregate these data with other data?
Beginning with Metric 1 (# of successful surgeries), we can assume that Aravind Eye already identifies what a ‘successful’ surgery is and how many they are performing over a period of time. This is a basic output that most hospitals would track.
Therefore, they can identify how the data is currently being collected, stored, and managed to assess whether it is an efficient system.
Let’s say the doctors report these numbers to branch administrators who then use a simple Excel sheet for data input and storage. In talking with these administrators, Aravind may find that the Excel model, especially for sharing across the organization and aggregating other data, has presented difficulties. There is an opportunity here for a better tool (e.g. Impact Cloud) — we’ll come to that in the next blog.
Moving into the next two metrics, let’s say Aravind has no existing tracking system for these data. Here, they would need to devise a strategy by first asking, does this data already exist? Can it be drawn from, for example, existing operational data? In both metrics cases, the answer is no.
Thus, Aravind needs to assess:
- What type of data they will need to collect
- Where they will get data from
- How they will procure the data.
In Aravind’s case, and using patient’s perception of healthcare delivery as our metric, the data format would be qualitative. If you remember, we also marked this in the Metrics Discovery phase.
Moving to data sources, we know we need to have a point of contact with the patient because this would be patient-provided data. If regular follow-ups are required, this could be an excellent time to have the doctor deliver the perception assessment.
Even better would be to have a trained individual conduct the collection to ensure data quality (especially if you use an interview format).
This leads to the next planning item, how you will acquire the data. You could utilize a structured interview approach, or a more simple validated survey. Deciding on the approach involves a number of factors, including available time and resources, as well as feasibility for staff and beneficiaries.
These details, and the tools for data management, are crucial to implementation and will be discussed in further detail in Part 2.
But by the end of your initial planning for data collection you should have a foundational strategic springboard that identifies:
- Existing data and data systems, particularly if and how they can be leveraged for your impact measurement.
- Potential gaps in your impact data collection capacities (e.g. tools, training)
- An outline of the implementation needs for each impact metric (data format, data sources, acquisition).
This strategic insight will position you to effectively implement the next phases of your data collection process.
Conclusion and next steps for better online and offline data collection
Up to this point in the data collection strategy phase, we have assessed the existing data systems within Aravind, whether they are already optimized, and to what extent they can be leveraged within our impact metrics framework.
We then turned to those metrics without an existing data system in order to identify the various needs relevant to collecting those data — the format, who is collecting/giving the data, what instruments are used to do so, etc.
Now, as we move conceptually into the next phase of lean impact measurement, Aravind will need to utilize certain tools for online or offline data collection and have somewhere to put it.
Part 2 of these data collection strategy blogs will simplify those often daunting details of implementation. Stay tuned!
Learn More: Impact Measurement