Download our white paper here.
Roger Perez, Fund Director
Tamsin Chen, Director of Impact
Funders routinely ask grantees to define their outcomes, learn from evidence and adjust when reality inevitably changes a plan. This summer, GitLab Foundation applied that expectation to our own grantmaking record, borrowing a practice that’s standard in venture capital. VC firms regularly step back to review their portfolios, looking across the full range of investments to understand what drove results and sharpen how they pick and support the next company. We believe philanthropy deserves the same discipline. We sought to understand what our portfolio could teach us to make stronger decisions, deliver more useful grantee support and evidence that arrives at the right time.
After almost four years and 200 grants made and the 70+ completed, what does our own track record teach us? We wanted to dig deeper into what worked, what didn’t, and, most importantly, why. A grant that missed its target wasn’t necessarily a bad idea, and one that beats it isn’t always proof we were right. We wanted to identify and understand the mechanisms behind our results, because that’s something we can actually learn from.
Unlike a venture firm, though, our returns don’t show up on a balance sheet — they show up in people’s paychecks and pockets. In philanthropy, accountability usually flows one way: grantees report to funders. We believe that we should hold ourselves accountable, too, and transparently share our insights in service of field learnings. Our mission is measured in one simple way: whether we’re maximizing the impact of every dollar we grant. Our North Star is to generate at least $100 in increased lifetime earnings for every dollar we invest, so the question isn’t just whether a grant made a difference, but whether it made the most of the dollars behind it.
How We Did It
We started with the 65 concluded grants that had the most complete records: concept notes and applications, annual impact surveys, check-in and closeout notes and everything in between.
Our method moved through three stages. We began with some initial hypotheses. A large language model then read deeply across the documents and surfaced patterns. Our team pressure-tested those patterns, sharpening them into four drivers, and applied those back across every grant. While AI did the reading, people did the thinking: setting the questions, judging which patterns held up, and deciding what the evidence could and could not support.
Rather than asking only whether a grant met its stated target, we asked what mechanism helped or hindered progress, what else might explain the result and what a funder could learn from it. Through our review, we uncovered four primary drivers.
The Four Drivers
Grantee execution: the choices within a grantee’s control, including how they delivered, managed partners and adapted when things shifted.
External change: shifts in policy, markets or context after the grant was made, outside of the grantee’s control.
Measurement window: impact that showed up outside the period we could observe.
Theory of change: whether the intervention’s design and logic actually fit the problem and the people it aimed to serve.
A disappointing result can mean very different things. A partner-led scaling model can stall when a partner doesn’t follow through. A program can prepare hundreds of people for good careers, but if no employer is ready to hire them, their effort and hope hit a wall. A new project can be working exactly as planned and still not show results before the grant ends. Policy changes, AI-driven hiring slowdowns and market swings can also upend a grant’s economics even when the team executes well.
Separating these explanations, rather than lumping them together, is what makes a review like this worth trusting (and doing it in the first place). It’s also how we learn where and when a funder can act. We now have a much clearer view of which operating choices to look for during diligence, which dependencies to surface before a grant launches and how to set evidence expectations that match the maturity and timing of the work.
Putting Retrospective Analysis to Work
We’re already putting our findings to work. Starting with our next round of grants, we’re making four changes, one for each driver:
Screening for strong execution: looking more closely during diligence at the behaviors behind strong results, like measurement discipline, continuous improvement and a willingness to pivot when the data calls for it.
Mapping dependencies at launch: identifying each grantee’s key partners, policy and market conditions upfront, instead of discovering them at the closeout meeting.
Matching evidence windows to the work: aligning when we measure with when impact can realistically appear, so we don’t mistake “not yet” for “no.”
Stress-testing the theory of change: making sure an intervention accounts for every side of the problem it’s solving, like whether employers are ready to hire the people a program prepares.
This was a learning exercise built on the records we had, not a formal external evaluation. It doesn’t close every data gap or make every grant perfectly comparable, and some impact almost certainly sits beyond our measurement window. But that isn’t a flaw to hide. Being clear about what a review can’t tell us is what makes the rest of it credible. Some bets may not pay off right away, and some may succeed far beyond expectations. But the question worth asking isn’t simply “did it work?” but “why or why not, and what will we do differently?”
In next month’s blog post, we’ll dig deeper into our findings, sharing what the strongest grants had in common and why some underperformed. Until then, follow Gitlab Foundation on LinkedIn or subscribe to our quarterly newsletter to stay up to date on our work.