For Difficult to Reach Stakeholders
Young populations, rural populations or day laborers in factories are not easy to reach, but they can often be reached through SMS- or a Voice-based approach. There are many modern technologies such as Twilio which deliver an API-based approach, combining effective communication tools.
For Online Users
Tools like Survey Monkey, Qualtrics, Google Forms, and online database spreadsheets like Airtable are widely-used tools for collecting online survey results from stakeholders and beneficiaries.
Each delivers on a unique value proposition while also fundamentally serving the purpose of recording data. Once the data arrives, however, it is up to the data collector to analyze, build useful reports and tell compelling stories.
The main value of these tools is often to understand product satisfaction, net-promoter score (NPS) or even understand outcomes. However, survey tools or survey-based measurement (often packaged by grant management systems) should not be used in place of a flexible measurement system. which leverages other tools to greater effect.
5 Things to Consider For Impact Assessment Survey
1. Surveys are just one piece of a puzzle
Often, surveys unveil just a slice of data, just a piece of the full insight you can be garnering through your impact measurement efforts. True social impact demands a hybrid approach and depends on the impact thesis of an organization or program. Ask yourself:
- Is our ecosystem investment based on outcome alignment with investors?
- Is program investment based on program effectiveness (efficiency in implementation)?
- Is program investment based on demonstrating social change?
An impact thesis helps an organization define a strategy which covers operational capacity, scorecards, stakeholder assessment, and general alignment of required criteria that demonstrate risk and effectiveness.
Often organizations have to develop custom analysis processes to bring all the data together to understand program effectiveness, social change, and outcome alignment.
Without an integrated analysis, superficial survey data can pose a danger to data integrity.
2. Survey vs. Results Collection
Surveys are useful in gathering one-dimensional data from stakeholders, such as baseline data, and pre and post feedback data. This process should not be confused with collecting results. At a high level, there are two types of results-gathering from organizations or partners:
1) Output and outcome-oriented results
2) Process-based results with or without a scorecard
While the first allows for understanding of overall outputs & outcomes of an initiative, the second is relevant for understanding program effectiveness and risk.
Often organizations use well-defined standards or custom metrics/indicators to collect results. A simple survey cannot collect RESULTS.
3. Better data analysis requires better relationships
Social impact results-gathering and analyses tend to be unique to each organization. High precision results can be achieved when metrics are assigned to programs based on logical relationships.
However, it is up to you to design your data structure, often done using tools like Excel which unfortunately do not represent well the relationship with Program, Partner, Product, Project, Applicant, etc.
A well-designed impact management platform (such as the Impact Cloud shown above), on the other hand, allows you to collect data from the online or offline surveys and index them in the database for a future reference. This will ultimately help streamline your reporting without creating islands of data stories.
4. Pitfalls of using irrelevant systems for impact management
Many of the grant management software solutions available include impact measurement applications. Unfortunately, most of them enable only the survey-like data gathered from grantees/investees.
Survey respondents are likely to respond based on a size of an investment or based on local compliance requirements before providing true results to the funder. A funder-designed the survey is likely to to be limited to assessing solely for program objectives.
Usually, grantee impact is unique and survey questions are not always relevant to them and their impact context. Other times they simply don't have the necessary data capacity either (a system which tracks activity, outputs or outcomes).
Grantees may also feel that they have already provided answers in a similar survey (frequent survey syndrome) and believe that another survey with similar questions does not serve a useful purpose.
5. Generic Surveys fail to understand deep impact
To understand actual impact risk you must aggregate results based on an integrated stakeholder approach.
For example, if a foundation is funding selected non-profits and wants to understand if recipient organizations are improving volunteer capacity or not, survey tools can measure volunteer management but then have to be compared against an indicator like volunteer opportunity.
Once results have been acquired, all the data manipulation usually happens in a tool like Excel, with customized pivot tables and data analysis. As the years and number of organizations increases, an Excel-based approach becomes increasingly messy and unmanageable.
In other words, it is difficult to leverage just survey data to tell a compelling impact story. What do you need to tell an effective story? We recommend:
- Collecting data from partners and beneficiaries/clients that are relevant to them.
- Aggregate data in such a way that offer unique impact insights to programs, partners, or projects with logically similar entities.
- Provide consistent and easy to use impact insights for different types of data aggregation in a single location.