Current developments and needs in the German evaluation sector
Initial policy recommendations from the "Zukunftswerkstätten" and pilot studies were presented at the PrEval Symposium 2023
This year's PrEval Symposium took place in Berlin on 30.11. and 01.12.2023. The collaborative project "PrEval - Zukunftswerkstätten" would like to thank everyone who came together on site and via livestream to discuss current developments and challenges in the evaluation of extremism prevention, political education and democracy promotion. The interim results of the different "Zukunftswerkstätten" and pilot studies were discussed with representatives from federal institutions, state and local administrations and civic education as well as experts from the PrEval network.
Day 1: Evaluation research in Germany and in international comparison
After introductory speeches by project head Julian Junk from the Peace Research Institute Frankfurt (PRIF) and Stefan Uecker from the Federal Ministry of the Interior and Community (BMI), the symposium started with an extensive program. On the first day, the participants were given broad insights into current challenges and potentials of German and international evaluation practice and were introduced to central debates in evaluation research. Additionally, speakers and the audience discussed development processes of evaluation designs for specific applications.
Andreas Uhl from the Institute for Interdisciplinary Research on Conflict and Violence (IKG) presented the updated PrEval platform with the latest findings from the online survey, which continuously visualizes the evaluation capacities in the field of extremism prevention, democracy promotion and political education in Germany. Subsequently, Sophie Ebbecke (GPPi) and Lotta Rahlf (PRIF) shared their research findings on evaluation practice in international comparison and emphasized the potential of continuous international exchange and mutual learning. The first part of the conference program was concluded by the authors of the PrEval expert reports, who shared different discourses of evaluation research with the audience. While Jana Klemm and Rainer Strobl (proVal) argued for the use of impact models in the context of evaluation, Hermann J. Abs (University of Duisburg) examined monitoring approaches for civic education. Finally, Simon Müller (DeZIM) discussed the potential of causal analyses for evaluation.
A couple of recommendations for dealing with evaluation emerged from the diverse presentations by our researchers: Methodological diversity should be recognized, for example, and the use of methods appropriate to the subject matter should be ensured. In addition, questions of impact should always be included in project and program planning at an early stage in order to be able to evaluate in a systematic manner. The importance of systematic exchange of knowledge was also emphasized multiple times. In an open round, participants and experts discussed the findings from the various presentations and debated how, for example, the idea of the impact model could be applied to quality assurance in the prevention of extremism, civic education and the promotion of democracy in Germany.
In the second panel, the interim results from two of our pilot studies were presented, which focus on the increasing digitalization in civic education. Susanne Johansson (PRIF) and Andrea Prytula (DeZIM) spoke about methodological challenges in a study on digitally assisted evaluation in civic education. Marcus Kindlinger (University of Duisburg) then introduced the participants to the concept of digital citizenship education, which can be used to further differentiate, operationalize and thus evaluate the educational goals of digital civic education. Building on this, a student questionnaire on young people's critical approach to online content was presented, which was developed as part of a PrEval pilot study at the University of Duisburg.
Svetla Koynova (VPN) and Juliane Kanitz (i-unito) concluded the first day with a creative workshop discussion on evaluative challenges in secondary and tertiary prevention that they encountered as part of the PrEval pilot studies. The focus of the dialogue was on different objectives and role perceptions of security authorities and civil society actors, trust issues and dealing with causality relationships in cooperation contexts.
Day 2: Need for support and customized support services
The second part of the symposium was all about the PrEval "Zukunftswerkstätten". Our scientists presented concrete support concepts tailored to the needs of practitioners. Experts from politics, administration and practitioners discussed and commented on potential approaches to implementation on the podium.
Lina Hartmann and Svetla Koynova (VPN) presented five prototypes with the aim of exchange and networking, which build on existing knowledge networks and bring various stakeholders from political education, democracy promotion and extremism prevention together. On the one hand, short interventions such as evaluation consultations with experts, peer-to-peer exchange sessions with experience sharing on external evaluations or workshops on self-evaluation were discussed and, on the other hand, long-term concepts such as measure-specific in-house consultation for practitioners and a personalized DeGEval scholarship.
Janusz Biene-Clément (i-unito) and Maximilian Ruf (VPN) then reported on the needs for support in professional practice that emerged in the surveys of the "Zukunftswerkstatt Helpdesk". Based on these findings, they presented three alternative approaches that could be used to implement low-threshold contact points for evaluators, administrative staff or project staff: Either a centralized or decentralized structure or the tendering of a nationwide project network would be an option. Expert comments from Sarah Bressan (GPPi), Rüdiger José Hamm (BAG RelEx), Samir Čolić (Arbeit & Leben) and Carolin Scholz (Landes-Demokratiezentrum Niedersachsen) resulted in constructive dialogue on the possibilities and limits of feasibility, specific contexts of use as well as important and necessary framework conditions for such support structures.
The final panel focused on the question of the implementation requirements of an evaluation database that collects and increases the accessibility of useful empirical knowledge from the evaluation sector, facilitates interdisciplinary learning and identifies research gaps. After an introduction by Lotta Rahlf (PRIF), Julian Junk (PRIF) moderated a lively panel discussion in which Ulrike von Streit (Landes-Demokratiezentrum Baden-Württemberg), Janoš Klocke (DEXT-Fachstelle Kreis Offenbach) and Dirk Posenau (Transfer für Bildung e.V.) emphasized the benefits of an evaluation database from different perspectives. Ethical concerns, for example with regard to security issues, the risk of political instrumentalization of evaluation results and worries about potential resource cuts also found their place in the discussion. In the spirit of praising failure, there was a repeated plea to publish even seemingly negative evaluation results in order to generate sustainable learning effects.
Outlook for the project year 2024
Thanks to the broad range of expertise of all participants, interesting discussions were created that will make a significant contribution to the future work of the PrEval project. The publication of the revised versions of our PrEval expert reports and the accompanying policy briefs, parts of which were already presented at the symposium, is planned for the beginning of 2024, for example. Detailed research reports from our "Zukunftswerkstätten" and other PrEval reports will also be available on our website over the course of the year.
Everyone is still most welcome to participate in PrEval. As always, we look forward to your input!