The Three-Cups-of-Tea guy’s charity is getting sued, for allegedly misleading donors (posted on Chris Blattman and AidSpeak.)  I agree it’s not a bad thing for NGOs to be held accountable for doing what they say they are going to do. However, I think this raises a very interesting point about whom NGOs are accountable to.

NGOs, like businesses, are accountable to their clients. Those of us who work where the rubber hits the road (assuming there is a road) probably like to think of the people who are the beneficiaries of our projects are our clients, and that the product we provide is improvement in their welfare.  Time for a wake-up call.  NGOs, like all economic entities, are, first and foremost, beholden to the people who give them money, and that means not beneficiaries, but donors. NGOs exist because a donor somewhere gets utility from a beneficiary getting a new school, or deworming, or ugly shoes, not because those people actually want them.  The beneficiaries of our projects, and their welfare, are the product that we sell to donors.

This isn’t a very flattering way to look at NGO work, but I think it’s a necessary one. There is potential for dissonance between what donors want people to have, what NGOs have an incentive to say they do, and what beneficiaries actually want and need, and this potential dissonance should be addressed, not ignored. 

This brings up the issue of research NGOs, like IPA. When viewed through this paradigm, research NGOs perhaps look the worst: the product that is being sold is not even beneficiary welfare itself, but data about beneficiary welfare.  But this product is the key to addressing the problem of donor-NGO-beneficiary dissonance: donors who have good information about what actually helps people, and what NGOs are actually doing, what they demand from NGOs is more likely to align with the needs of beneficiaries.  

So maybe I can still feel pretty good about what I did today.

Chris Blattman recently blogged about the moral absurdity of running regressions where the dependent variable is “war deaths”.  While looking at death, illness, hunger, and poverty through the lens of statistics may seem rather reptilian, I think many researchers have emotional reactions to the data they work with.  For me, these connections hit hard and unexpectedly, often when I am tired and working late, and they come despite efforts to be dispassionate about the data I am looking at.

Survey editing is prime territory for emotional connections to data. When editing surveys, you see the story of an individual respondent in a way that you don’t when you are looking at columns of aggregated data .  Once, I was reading a survey where a respondent reported that a household member had experienced a headache.  I turned the page to the question on outcomes of health events.  The headache had resulted in death for that household member—despite the family spending an amount equal to roughly one-fourth of Ghana’s annual GDP per capita on health care for that individual.  The shock of the outcome hit me almost physically. Another respondent reported testing positive for HIV. Sitting alone in the Tamale office at night, I struggled to pull myself together, shoo the bugs out of my computer keyboard, and make my way home.

The “death” outcome became a dependent variable in regressions I later ran looking at determinants of health outcomes.  Luckily, there were very few events of death in my sample.  We also looked at a number of food insecurity events: individuals going to bed hungry, or not eating for an entire day, for example.  These were, unfortunately, common among our respondents.  I don’t deal well with feeling hungry myself, and for me, food insecurity statistics evoke desperately sad, human images: a man’s disappointment at foregoing his favorite fish; a young student trying to sleep before an exam while feeling the distracting ache of hunger; an elderly woman going without food for a day so her grandchildren can eat; a mother having to tell her thin children there is no food today.  

These emotional connections often seem like a distraction, something that prevents us from approaching our analysis logically and dispassionately.  In all honesty, part of my attraction to quantitative research tools might be to protect myself from these types of emotional connections to problems.  But it our ability to have these connections, even through layers of statistics, is tied to a very deep belief in the importance of what we are doing, and that counts for something.  Hey, at least I’m not working in finance.

A major study was recently found to contain an error that led the study to overestimate the cost-effectiveness of deworming by a factor of 100. (Note that this was NOT the IPA RCT study of deworming in Kenya, which found deworming to be highly cost-effective in part due to large positive spillover effects.)

This is the type of thing that keeps project associates up at night-- when we aren't staying up trying to track the surveys that came in that day, writing .do files to analyze our data, or drafting reports containing our results. The fact that we work long hours, on tight schedules, sometimes while delirious with malaria doesn't help. My most recent report was 120 pages long, and based on what must be tens of thousands of lines of stata code.  It's hard to believe there are zero errors in that code.

So how I am going to sleep tonight? I know two sets of eyes have looked over the code used in the analysis. I've looked critically at the findings to see if they make sense, and if they are consistent with the rest of the data. We might not be able to catch everything (were there some observations I should have recoded for that question?) But hopefully we can catch the "factor of 100" errors.

Also, I am really tired.

Military spending by the top military spenders, from the Economist.

What if we replace military spending with  official development assistance for these same countries?

I'm starting to get data-- even if it's from training and not real-- which is exciting and daunting at the same time.

I haven't had time to do much with the numbers yet, but every survey has a place for the surveyor to write in comments about how the survey went, and I scanned through some of those. My favorite comment so far? "NO."

Can't wait to look that one up!

I am currently working with our data entry staff to start data entry for my baseline survey. Throughout the day, I keep finding new reasons to be appreciative of the staff, who do a difficult, tedious job while holding themselves to a high standard of accuracy. Here are just a few reasons data entry staff deserve the respect of the research teams they work with:

·         They have to learn the logic of a survey in one day that took a week for surveyors to learn.

·         They have to work with surveys that have been battered by surveyors, up to three stages of editing, three stages of transportation, and storage in Tamale during dusty harmittan.

·         They have to try to decipher surveyor’s sometimes-awful handwriting.

·         They have to remember how to consistently enter responses that surveyors record inconsistently.

·         They don’t eat lunch till 1:30.

I find this last particularly impressive, as I consider it to be a drastic sacrifice to wait to eat longer than 1.  So, thank you to the data entry team!!