Panel Discussion 1
| Richard Hummelbrunner: | Senior associate of OEAR Regionalberatung; practitioner, trainer and author. |
| Bob Williams: | Freelance consultant, specializing in empirical field research and evaluation technique, systems theory and practice, organizational development and action research |
| Daniela Körppen: | Researcher at the Berghof Foundation for Peace Support in Berlin |
| Reinhard Stockmann: | Founder and director of the Center for Evaluation (CEval) at the Saarland University |
| Esteban Tapella: | Lecturer and researcher at Programa de Estudios del Trabajo, el Ambiente y la Sociedad, National University of San Juan |
On the panel, different perspectives were represented.
There is no such thing as the systemic evaluation
Reinhard Stockmann stressed that it needed to be clarified what is understood by systemic approaches in evaluation. There was an agreement among the panelists that there is no such thing as the systemic evaluation. Instead, one should talk about applying systems ideas in evaluations. Bob Williams compared a good evaluator to a thief who steals appropriate tools and methods from different fields including the systems field.
Systemic approaches and "traditional" evaluation approaches: Antithetic or complementary?
The five panelists discussed common aspects of and differences between "traditional" and rigorous evaluation approaches on the one hand and systemic approaches on the other. All panelists advocated multi-method approaches. Richard Hummelbrunner und Bob Williams stressed that systemic approaches are not always needed. They can be particularly useful in highly complex situations - but not every setting is complicated or complex. Daniela Körppen stressed that it is important to integrate systems ideas into more "traditional" evaluation approaches instead of creating the image of a completely new evaluation approach that replaces the others.
Reinhard Stockmann pointed out, however, that there are some fundamental epistemological differences between the constructivist systems thinking and evaluation approaches that are based on neo-positivist/critical rationalist assumptions. In his opinion, development programs and their environment are not as chaotic and unpredictable as complexity science suggests. He also noted that "traditional" evaluation approaches are not as naïve as some proponents of systemic thinking argue. Context and different perspectives are usually taken into consideration also in "traditional" evaluations.
Process vs. impact orientation; learning vs. accountability
Daniela Körppen and Esteban Tapella put great emphasis on the process orientation of evaluations that are based on systems thinking. Explaining the South American evaluation approach "sistematización", Esteban Tapella stressed its focus on learning and understanding processes from different perspectives rather than on providing accountability.
On the other hand, Reinhard Stockmann warned against focusing too much on processes while losing sight of the impact of development interventions. When it comes to providing accountability, he argued, rigorous methods are needed.
According to Richard Hummelbrunner, there is no contradiction between a focus on processes and a focus on outcomes and impacts. Bob Williams proposed that we rethink our notion of accountability. According to him, in complex settings accountability is an illusion and only a sense of responsibility can be created.
Cultural appropriateness of systemic thinking in partner countries
Daniela Körppen argued that in the Western discussion, one might get the impression that systems ideas are very theoretical. According to her experience, systems thinking is part of everyday life in many cultures and often easier to understand for partners than result chains. From a Latin American perspective, Esteban Tapella agreed to this.
Systemic approaches in evaluation as the latest fashion
Systemic approaches in evaluation are "fashionable". Bob Williams considered this fact to be the movement's greatest help and biggest hinderer at the same time. He pointed out that many evaluations of poor quality are labelled "systemic" in order to get attention. He requested that systems thinking be respected as a field of knowledge and urged not just to use it to tag a fashionable name on evaluations.
