Useful Knowledge is what you gain by applying and fitting what you were taught in school to solving real-life problems. Let me take you back to a formative lesson for me.
In early 2010, I had just completed my second graduate school program and was fresh from heavy lifting in courses on development economics, project evaluation, applied statistics, decision theory, and public policy analysis and decided to quit my job with a bank to undergo a three months internship with a Ministry of Economic Planning in one of the Northern states of Nigeria. The then government wanted to evaluate the foreign aid it received. Over 15 aid projects were underway. They ranged from rural roads to immunization to environmental protection to the development of the health sector. One project enumerated the out-of-school girl children and initiated a conditional cash transfer program to tackle the same.
The then Commissioner for Economic Planning explained to me what he sought. “We want to tell the donors what we want and need, not just take what they are giving. Last year, some people from DFID and the World Bank were here and said, ‘Would you like this project? We have funding for it. But if you don’t, we’ll take it to another state.’ We need evidence to be able to say, ‘this kind of project works well, this kind doesn’t – and given our State Economic Empowerment Development Strategy (SEEDS), here’s what we’d like you donors and Development Bank to do.”
Sure, I said in effect to the Commissioner of Economic Planning. Can do.
Brimming with the confidence of a newly minted graduate student of Development Studies, I imagined answering the Commissioner’s question with a study. We should assemble the social cost-benefit analysis of all the development projects. Then we would create a multivariate model with the cost-benefit ratios on the left-hand side and, on the right-hand side, variables about the sector, donor, project design, budget, degree of popular participation, and who knows what – something like this equation:
Social c/b =f (project characteristics, sector characteristics, budget, ………)
We’d estimate the model statistically, and the results would tell us what kinds of projects worked best, other things equal.
The department I was assigned to at the Ministry of Economic Planning had only three professionals and few funds, but we got to work. The staff and I pulled the files on each project. There were “evaluations,” but only in terms of this happened then or this money went there. We found almost no data on social benefits and costs.
Distraught, we sought in the new State Plan (SEEDS as it was referred to) the state’s social welfare function. To my surprise, nothing there correspond to the mathematical and statistical constructs of my economics and public finance courses. I met again with the Commissioner of Economic Planning and asked him about the state’s social welfare function. He responded, ” social welfare what?”
After a couple of weeks, I realized my dream equation was more like a nightmare. And so, I send a rather long email to Professor Halidu I. Abubakar an excellent economist and public finance expert (Presently the Emir of Ilesa Baruba in Baruten LGA of Kwara State) who taught me project evaluation three years earlier while studying for a master’s degree in Public Administration at the Ahmadu Bello University Zaria. I described the evaluation task and its impossibility, and I implicitly laid the impossibility on the models of policy and project analysis and evaluation he taught his graduate students.
Professor Halidu I. Abubakar replied to my email. He said in effect: People can never agree on what benefits and costs are in the abstract. But they can and do agree on specific examples of outrageous success and outrageous failure. Find these among your projects. Study them. Compare them. Share your results and learn some more.”
We followed his advice. My colleagues and I went from ministry to ministry, asking for outstanding examples of aid projects. (We decided to forgo the failures.) Eventually, we studied four of the successes. At the same time, we put together data on all the projects – how much money was being spent each year in which ministries and from which donors. And we reviewed academic hypotheses about why some sort of foreign aid projects work better than others.
At the end of my internship, the Commissioner of Economic Planning called a meeting. He invited his colleagues (other Commissioners) and the donors. He asked me to facilitate. Me! We brought a stack of printouts with all the data on all the projects, and we presented a summary. We had shared in advance brief descriptions of all the six successful projects, which we now quickly reviewed. Then I asked participants, “Why do you think these projects succeeded?”
One of the Commissioner said local participation was key. But another commissioner cited a project that has failed despite participation. One of the donors cited his experience in other countries with the participation of various kinds.
Soon we had a full-blown discussion of what works where and why. Local wisdom and experience were evoked, and so was international knowledge. Every once in a while, I would ask a question based on development literature. By end of the session, ideas had been shared, and there was a movement towards priorities for the projects the state needed next.
Professor Halidu I. Abubakar’s advice worked, in several senses. The donor was impressed and cut the state government more slack in deciding what projects would be funded and how the projects would be structured. Commissioners worked better together and with donors. The head of my office got promoted and then was later plucked away by the UNDP, which two years later asked him to replicate this same “evaluation” of foreign aid in two other northwest states.
But my dream study never happened. I did not provide anything like an equation to say what worked wherein the state. Instead, a combination of data about projects plus examples of success plus a room full of expertise led to creative thinking about what works where. The participants ended up with better knowledge, better relationships, and the beginning of collaborative problem-solving.
My idea of policy analysis and evaluation was altered forever. Yes, I still admire and appreciate welfare economics, but I have come to realize that no country, not to talk of sub-nationals actually possessed a social welfare function (and perhaps none ever would). I enjoy and use statistics, quantitative techniques, and econometrics, but as I have learned since that perfect equation is an illusion. The stylized models I love are almost impossible to evaluate in a world where local differences are huge and contexts are changing.
A convening can help. That’s what happened when commissioners considered together with data about foreign aid in the state and four case studies of projects that seemed to be working. The hoped-for results of a convening are better inferences about what will work here and the kindling of creative problem solving that goes beyond what an outsider could provide.
