Unmesh Sheth 11/23/21 12:55 PM 8 min read

Design Effective Impact Experiments

 

Are you a social purpose organization?

Are you looking to maximize the social impact of your program or investment?

In an earlier article, we discussed why continuous learning and improvement are foundations for scaling social impact. We also introduced how impact experiments can benefit social purpose organizations.

Based on this, How do you design practical impact experiments to maximize the impact of your investments or program? However, before we do that, let's look at the three most important questions.

  • How can you build impact evidence of your program on investment?
  • What is impact experiments, and how can it help you scale the impact of your program or investment and the target?
  • What is the best way to collect the impact evidence?

As climate crisis, racial and social inequality are growing, everyone from corporate, impact investors to philanthropy must rethink their approach to social impact. Some examples from the corporate world, most often the CSR, are also called corporate social responsibility. Often, this effort amounts to impact washing or Green Washing social impact by highlighting limited positive stories and hiding negative impact. So how can they improve the impact?

  • How can businesses improve their social investment? How can impact evidence help make better decisions for programs?

It is interesting when community programs, employee engagement programs can demonstrate the impact evidence for the corporates or impact investments?  The chances are that you are using top-down metrics measurement to a portfolio company, but is that giving you the right kind of understanding of the impact and creating those results?  Often those data are unreliable because they are not outcome-oriented, so it isn't easy to define how they can scale impact-wise. So focusing on the additionality of your portfolio is a better approach, and building their capacities might be the right thing. The most significant barrier is the funders-driven metrics approach for large philanthropy or a social purpose organization. This approach forces social purpose organizations to focus on impact justification. Unfortunately, impact justification is the most significant barrier to scaling the program's social impact and creating a culture of impact evidence and learning. However, you can improve the social impact returns through a multidimensional and learning process.

And that's the best way forward, too; infact, you need to do the product-market fit of your product by listening to your stakeholders continuously and reducing the barriers to help you advance your journey faster.

Pitfalls of Today's Impact Measurement Framework

Today's Impact Measurement framework is, needless to say, is quite complex.

Theory of Change with the 15 plus outcomes?

And often, these outcomes are not even achievable because those outcomes are beyond your control. Many of them require a policy change, and sometimes they are pretty heavy resource-based approaches. And that's why it can create endless analysis, and ultimately it can stop you from achieving any of the outcomes. We often see people spending months designing Theory of Change with many, many impact metrics, and that's a single red flag that says you are on the wrong path.

So how do you start the impact journey - the right way?

Creating the proper process with internal data culture that focuses on a continuous learning approach is the right approach.  This is an impact experiment that focuses on short and frequent impact experiments. You need to define it by experience or by concentrating on reaching product services market fit in a short time. For example, listening to your stakeholders, if you don't have enough data, having interviews with them, having a one-on-one discussion, and having a more significant number of stakeholders ready to use the product. Then you have a market that's is also prepared to use a product.  Start gathering the data from these stakeholders. So impact experiment is a short process but effectively depicts the impact evidence of your impact, immediate outcome, and not the long-term outcome.

And you can also do this in many different ways; you can get to these stakeholders through external research. For example, in some cases, if you're working on an alcohol prevention program and within your region, let's say Midwest. There's already research that proves that 15% are abuse prevention. Suppose you can't find the study and need more evidence; define the stakeholder survey. It's a very effective method that helps you understand the multidimensional impact. And this can be described as the impact or external output and testimony. Make sure these experiments do not take more than two to four months.

How to design impact experiments?

The steps to the impact experiment are not to create a very long Theory of Change. Instead, each experiment has to be time-bound for a specific impact object. Typically,  it is crucial to develop a stakeholder survey aligned with an impact management project for 5 Dimensions of impact so that you can get dimensional results and analyze them for results and evidence.

Here are the steps.

  • First and foremost, Align your stakeholder survey or stakeholder questions with the five dimensions. The questions in the survey must provide answers to the following:
  • What is the outcome that you'd like to create?
  • Who is your stakeholder?
  • Who is likely to benefit from programs, services, or a product? 
  • What is the scale, depth, or duration  (HOW MUCH), contribution, and impact risk?
  • The five impact dimensions defined by the Impact Management Project can be used by anybody who understands and communicates social impact. This can also help them understand, learn and communicate the impact in the right way.

    IDP1

Tips for designing an effective stakeholder survey

Make sure surveys do not have more than 15 to 20 questions. A lengthy survey can create more friction than learning. So creating an effective survey is much more important than a long survey.

Ensure that you embed the right open-ended questions because surveys are often designed from your point of view, your organization's point of view but fail to capture stakeholders' perspectives. Therefore, it must be prepared to capture what they are thinking and make sure there is an open room to listen to them and understand their sentiments and emotions. 

If you don't get significant responses from your stakeholders, defining the evidence-based on your data collection isn't easy.  A technological approach like Impact Cloud can completely automate your data collection, aggregation, analytics, and reporting in a singular platform. The intervention area can be related to birth equity issues to housing affordability, job creation, mental health, sustainable fishing, clean energy, waste management.

Read More:

avatar

Unmesh Sheth

32 years of track record In technology companies, innovation, leadership. Deep understanding of bottom-up and top-down data trust challenges in high impact philanthropy and impact investments.