On November 5th, 2020, as part of ANDE’s 2020 Metrics Conference, Alice Waweru from Technoserve and Brianna Losoya-Evora from SEAF led a session on collecting data from entrepreneurs, especially in times of crisis. Alice presented Teschnoserve, a capacity development organization that supports Microentrepreneurs in Sub-Saharan Africa and Latin America. Brianna introduced SEAF’s technical assistance arm, the Center for Entrepreneurship and Executive Development (CEED), a global network of growth-stage entrepreneurs active in ten countries.
With their collective years of experience, the organizations faced similar challenges and developed innovative solutions for collecting data from entrepreneurs. However, the COVID-19 pandemic brought new challenges for entrepreneurs and the capacity development organizations that support them. The session unearthed data collection difficulties and solutions from all 40 participants and raised important topics for this community of practice.
| Challenge | Solution | Why it works |
| Low Response Rates | Non-financial incentives | Technoserve uses mentorship and other linkages as a non-financial incentive for their entrepreneurs. While CEED uses entrepreneur reports that allow CEED members to benchmark their company’s performance against their CEED peers and others in their region or sector. |
| Financial incentives | CEED experimented with a raffle for financial rewards among entrepreneurs in 2019 and found anecdotally that the raffle amount was not significant enough to increase response rates meaningfully. The entrepreneurs who won the raffle donated the funds or used it to sponsor an informal CEED activity for their peers. | |
| Providing advanced notice and consistency in the collection period | Technoserve and several participants indicated the importance of the timing of a survey or interview with an entrepreneur. Providing advanced notice for digital surveys and asking for timing preferences before collecting data can play an essential role in boosting response rates. | |
| Staff training and incentives | Technoserve found that training staff and enumerators were effective ways of increasing response rates. CEED offers prizes for their centers with the highest response rates and emphasizes the importance of persistence with growth-stage entrepreneurs. | |
| Survey Fatigue | Shorten the survey | Shortening a survey is easier said than done. CEED’s strategy is to look at the data from the previous year and map how each data point translated into adapted management. Questions that were not used often or at all were removed from the survey. |
| Lean data collection approaches | Technoserve uses lean data approaches or collected a few data points more frequently as a substitute for lengthy annual or semi-annual surveys. | |
| Make the Survey Engaging | Ivy Syovata, CEO of Incubator Nest Hub, posed the idea of creating a more engaging survey, make sure that questions are relevant to entrepreneurs and not redundant. | |
| Entrepreneurs in Survival Mode | Embed data collection into services and tools entrepreneurs need to survive | A universal challenge of collecting data from entrepreneurs is that they are inherently busy people, with little time for not mission-critical activities. The pandemic only exacerbated this challenge. CEED Slovenia created a COVID-19 SOS shared inbox where their entrepreneurs could ask an expert about specific business challenges. The form to submit a question includes questions about the impact of COVID on their business. CEED intern used this data to advocate for policy that supported Slovenian entrepreneurs. |
| Survey Design | Look to experts and open source tools | Survey design is both a challenge and a solution because it’s incredibly important to collect data in a rigorous way that does not overwhelm entrepreneurs. Leveraging open-source tools, like GALI from established entrepreneur research organizations, is one potential solution. |
| Beneficiaries’ inability to track business performance data | Support Tools | Technoserve highlighted that microentrepreneurs face inherent difficulty collecting business performance data and even more challenges using that data to inform decisions. Alice explained that many of the tools Technoserve offers their entrepreneurs can also help TechnoServe collect and validate data. |
| Fears around sharing specific data points (like revenues, number of employees) | Trust Building | Building trust is a potential solution to every challenge highlighted, and it can be easier said than done. CEED makes a point of including data use terms prominently in every survey and having the local CEED staff, who entrepreneurs know and trust, send out surveys. |
| Validating data/Ensuring data quality | Use open-sources, leverage reationships | One of the toughest challenges is validating data once it does come in. CEED tries to do this with publically available data sources, comparing data to the previous year, and highlighting outliers. These anomalies are then confirmed via phone call. Technoserve ensures data quality by embedding data collection into the tools that their beneficiaries use. This incentivizes accurate data entry by entrepreneurs so that the tool is ultimately useful. |
| Technical constraints (inability to administer online, cost of tools, etc.) | Enumerator Training | Technoserve ensures that data collection tools are mobile and agile and that enumerators are adequately trained on data collection techniques. |
| Building the business case for data collection | For centers that require a membership fee for service, CEED trains staff on how the data and CRM can be used to track potential clients, serve as a member management tool, and allow for automized communications and marketing. | |
| Considering segmentation by age | Technoserve highlighted the extreme differences in the effectiveness of online data collection between younger and older entrepreneurs. Alice recommended segmenting by age and creating a data collection strategy for each group. | |
| The capacity of staff to collect/analyze data | Training and staff buy-in | Both organizations highlighted the importance of training staff on effective data collection and getting their buy-in and understanding of how the data will be used and why it matters. |
| Cost of Data Collection (striking a balance between cost and utility | Qualitative Data Analysis & Mock Analysis | Ivy Syovata, CEO of Incubator Nest Hub raised the vital point that data collection costs time and resources, so striking a balance between rigor and utility is critical. One way to do this is to no neglect the value of qualitative analysis. Matt Guttentage, ANDE’s Director of Research and Impact, recommends using mock analysis to test the effectiveness of data collected before launching an extensive data collection initiative. |
| Positive Data Bias | Qualitative data and external data sources | Human nature is to share positive data points and gloss over more negative data, so how can you correct for positive data bias? Underscoring data confidentiality is a possible solution. Validating data with public data sources might be another way to correct for this. |
| Analyzing small sample sizes | Many ANDE members work with smaller cohorts of entrepreneurs, and gaining insight from data with a small sample size can be challenging. This further emphasizes the importance of qualitative data and using publically available data sources (like GALI) to benchmark where possible. Other statistics basics, like using medians and not averages, become necessary with smaller sample sizes. | |
| All of the Above | Consider additional data sources | While the discussion focused on large data collection initiatives, like an annual survey, we are continually collecting data from entrepreneurs in reality. We track their attendance, responsiveness to communications, and even a friendly conversation can provide invaluable qualitative data insights. Being creative about how to capture and analyze these data points facilitate data collection efforts. Technoserve uses a WhatsApp group to monitor how engaged their entrepreneurs are and keep a pulse on their needs. CEED uses its Salesforce CRM to create warning systems when an entrepreneur does not fully engage with the network. |
| Use date for real-time decisionmaking and be transparent about data usage | As Ivy highlighted, every dollar spent on data collection is a dollar not spent on directly supporting entrepreneurs, so ANDE members must support their entrepreneurs by being transparent about data collection initiatives and actually using that data to improve programming. Technology can facilitate real-time analysis, but there are low-tech ways to ensure tight feedback loops, like regularly checking in with entrepreneurs. |
Participants cited low response rates as their most considerable challenge, and there were plenty of other associate challenges, like small sample sizes or inability to ensure data integrity. While the group produced several recommendations for each challenge, participants unearthed other data collection challenges that may make an excellent follow-up conversation in the 2021 Metrics Conference. Here are a few examples:
A balance between cost and utility
Ivy Syovata, CEO of Incubator Nest Hub raised the vital point that data collection costs time and resources, so striking a balance between rigor and utility is critical. This can be easier said than done. How do we distinguish a nice-to-have data point from one that is mission-critical?
How can we promote an information exchange instead of a one-way information flow?
Asgar Bhikoo, Monitoring and Evaluation Specialist at the Allan Gray Orbis Foundation, aptly pointed out that, “Power, position and clout-affects response rates. But the magic ingredient in getting responses: relational clout and utility.”
In addition to leveraging relationships and building trust, showing entrepreneurs how their data will be used can be much more effective than telling them. Entrepreneur reports can be one way to do this, are there others we have not yet thought of?
How can we leverage public data sources like GALI to create entrepreneur reports?
Hedda Ngan, Monitoring, Learning and Evaluation Officer at Ygap agrees that entrepreneur reports could be very effective in promoting information exchange, but what goes into this report? What are some publicly available data points?
CEED used enterprise-level data from the Global Accelerator Learning Initiative (GALI) to benchmark entrepreneurs against other companies in their region, sector, or country. You can see an example here.
How can we make these entrepreneur reports useful?
CEED is still testing whether this report contains vital information for its members or if it’s just interesting, but ultimately will not lead to any definitive action. What if this report could help entrepreneurs gain access to financing? Could providing feedback loops reinforce ANDE member efforts to scale SGB companies?
Conclusion
Collecting data from entrepreneurs is inherently challenging, but maybe reframing the problem as facilitating information exchange between ANDE members and the entrepreneurs we aim to serve. While the session did not resolve all of the challenges that surfaced, we made progress in crowdsourcing several potentials solutions. We also have potential leads on topics for ANDE’s 2021 Metrics Conference.
To learn more about Technoserve and SEAF use their data read CEED’s 2019 Impact Report and TechnoServe’s 2019 Impact Report.
The presentation from the session can be found here.
