エピソード

  • 48: The Garden of Evaluation Approaches

    48: The Garden of Evaluation Approaches
    2025/04/22
    In this episode, I chat with the authors of The Garden of Evaluation Approaches about their innovative and helpful framework for thinking about evaluation theory and approaches. They each share their personal journeys into evaluation as well as into this work they have been collaborating on for years. We also discuss evaluation theory more broadly, including how to make it more accessible to new and emerging evaluators. About guests: Bianca Montrosse-Moorhead (https://education.uconn.edu/person/bianca-montrosse-moorhead/) is a Professor of Research Methods, Measurement, and Evaluation at the University of Connecticut, where she also directs the Partnership for Evaluation and Educational Research (PEER). As Co-Editor-in-Chief of New Directions for Evaluation and a fervent advocate for evaluation, Bianca has dedicated her career to bridging the space between evaluation theory and practice. Her work encompasses a broad spectrum of contributions, from evaluating various educational and social programs using diverse methodologies to enhancing the professional training of evaluators worldwide. Daniela Schroeter (https://wmich.edu/spaa/directory/schroeter-0) serves as a Presidential Innovation Professor and Associate Professor at Western Michigan University. With a Ph.D. in Interdisciplinary Evaluation and over twenty years of research, Daniela has contributed substantially to developing evaluation theories and methodologies. Her global work has involved diverse sectors, focusing on enhancing the capacity and effectiveness of evaluations. Daniela also co-edits the Teaching & Learning of Evaluation section of the American Journal of Evaluation. Lyssa Wilson Becho (https://wmich.edu/evaluation/directory/becho) serves as a Principal Research Associate at The Evaluation Center of Western Michigan University. Garnering the 2024 Marcia Guttentag Promising New Evaluator Award, Lyssa has made a significant impact in advancing evaluation methodologies, focusing on culturally responsive practices and promoting equity within evaluation processes. Lyssa is a co-Executive Editor for the Journal of MultiDisciplinary Evaluation. Collectively, their innovative work in evaluation spans continents, cementing their reputation as leaders in advancing the practical, theoretical, and methodological facets of the discipline. Contact information: Bianca Montrosse-Moorhead Email: bianca@uconn.edu (mailto:bianca@uconn.edu) LinkedIn: https://www.linkedin.com/in/BMMoorhead/ (https://www.linkedin.com/in/BMMoorhead/) ResearchGate: https://www.researchgate.net/profile/Bianca-Montrosse-Moorhead (https://www.researchgate.net/profile/Bianca-Montrosse-Moorhead) Daniela Schröter Email: daniela.schroeter@wmich.edu (mailto:daniela.schroeter@wmich.edu) LinkedIn: https://www.linkedin.com/in/danielaschroeter/ (https://www.linkedin.com/in/danielaschroeter/) ResearchGate: https://www.researchgate.net/profile/Daniela-Schroeter (https://www.researchgate.net/profile/Daniela-Schroeter) Lyssa Wilson Becho Email: lyssa.becho@wmich.edu (mailto:lyssa.becho@wmich.edu) LinkedIn: https://www.linkedin.com/in/lyssa-wilson-becho/ (https://www.linkedin.com/in/lyssa-wilson-becho/) ResearchGate: https://www.researchgate.net/profile/Lyssa-Becho (https://www.researchgate.net/profile/Lyssa-Becho) Resources mentioned: Project vita (biography or résumé) with links to all free and publicly available resources: https://tinyurl.com/EvalGardenVita (https://tinyurl.com/EvalGardenVita) Montrosse-Moorhead, B., Schröter, D., & Becho, L. W. (2024). The garden of evaluation approaches visualization. Journal of MultiDisciplinary Evaluation, 20(48), 49–58. https://journals.sfu.ca/jmde/index.php/jmde_1/article/view/1029 (https://journals.sfu.ca/jmde/index.php/jmde_1/article/view/1029) Montrosse-Moorhead, B., Schröter, D., & Becho, L. W. (2024). The garden of evaluation approaches. American Journal of Evaluation, 45(2), 166–185. https://doi.org/10.1177/10982140231216667 (https://doi.org/10.1177/10982140231216667) Bledsoe, K. L., & Graham, J. A. (2005). The use of multiple evaluation approaches in program evaluation. American Journal of Evaluation, 26(3), 302-319. https://doi.org/10.1177/1098214005278749 (https://doi.org/10.1177/1098214005278749) King, J. (2024) Garden of evaluation approaches with SROI and CBA https://linkedin.com/posts/julian-king-87a015a_cba-sroi-vfi-activity-7201338499776618497-J3-z/ (https://linkedin.com/posts/julian-king-87a015a_cba-sroi-vfi-activity-7201338499776618497-J3-z/) Miller, R. L. (2010). Developing standards for empirical examinations of evaluation theory. American Journal of Evaluation, 31(3), 390–399. https://doi.org/10/fmzjxp
    続きを読む 一部表示
    1 時間 21 分
  • 47: Jennifer Villalobos

    47: Jennifer Villalobos
    2025/03/25
    In this week’s episode, I talk with Dr. Jennifer Villalobos about evaluator education, the scholar-practitioner model, evaluation careers, and how the current administration is affecting evaluation. Contact information: Jennifer P. Villalobos - jennifer.villalobos@cgu.edu About Dr. Jennifer Villalobos: Jennifer P. Villalobos, PhD, is an Assistant Professor of Practice and Director of the Doctor of Evaluation Practice (D.Eval) program at Claremont Graduate University. As an evaluator, scholar-practitioner, and organizational psychologist, her work focuses on advancing socially responsive evaluation, helping organizations increase their DEI profile, positive psychological interventions, and the intersection of evaluation education and practice. Outside of work, Jennifer enjoys spending time with her husband and three kids, cheering them on in their many sports and theatrical events. She’s also a firm believer in the power of community—whether that means mentoring students, collaborating with peers, or just gathering people around good food and conversation. Music by Matt Ingelson, http://www.mattingelsonmusic.com/
    続きを読む 一部表示
    56 分
  • 46: Reflections on Foundations of Evaluation

    46: Reflections on Foundations of Evaluation
    2025/02/25
    I share my reflections on teaching Foundations of Evaluation after a major course revision, including how it's been going for the first four weeks. Music by Matt Ingelson, http://www.mattingelsonmusic.com/
    続きを読む 一部表示
    50 分
  • 45: Sheila Robinson and Kim Leonard

    45: Sheila Robinson and Kim Leonard
    2024/05/28
    This episode featured Kim Leonard and Sheila Robinson discussing their careers in evaluation and survey design. They shared how they met on Twitter in 2012 and began collaborating, writing blog posts that eventually became their book "Designing Quality Survey Questions (https://us.sagepub.com/en-us/nam/designing-quality-survey-questions/book249048)" published by Sage Publications. Sheila and Kim discussed their process for writing the book, starting with blog content and building it out over several years into a full manuscript. They also covered common issues they see in surveys and tips for developing effective survey questions. Sheila Robinson: https://www.sheilabrobinson.com/ & https://www.linkedin.com/in/sheilabrobinson/ Kim Leonard: https://leonardrande.com/ & https://www.linkedin.com/in/kim-firth-leonard-1ba9447/ Music by Matt Ingelson, http://www.mattingelsonmusic.com/
    続きを読む 一部表示
    1 時間 2 分
  • 44: Brittany Dernberger

    44: Brittany Dernberger
    2024/04/23
    This month I chat with Brittany Dernberger about her evaluation background and practice. Brittany Dernberger is a sociologist and gender inequality expert who has led research, evaluation, and organizational learning across academia, philanthropy, government, large international NGOs, and small nonprofits. Brittany currently leads global initiatives to measure systems-level change at CARE and co-chairs the American Evaluation Association Systems in Evaluation Topical Interest Group. Learn more about Brittany’s work at http://brittanydernberger.com/. Access CARE’s completed Systems-Level Impact evaluations at https://careevaluations.org/evaluation/keywords/systems-level-impact/. The AEA Systems in Evaluation TIG paper on Principles for Systems Thinking in is available at https://www.betterevaluation.org/sites/default/files/SETIG-Principles-FINAL-DRAFT-2018-9-9.pdf. Music by Matt Ingelson, http://www.mattingelsonmusic.com/
    続きを読む 一部表示
    51 分
  • 43: Allison Prieur

    43: Allison Prieur
    2024/03/26
    This month I talk with Allison Prieur about her experiences as a graduate student in evaluation and running a business. Some resources mentioned or suggested for listeners: * Logic of evaluation (https://www.psychologytoday.com/ca/blog/evaluation-social-betterment/201810/the-application-the-logic-evaluation-the-real-world) * Evaluation of logic in practice (https://doi.org/10.1016/j.evalprogplan.2019.101681) * Destination Dissertation (https://rowman.com/ISBN/9781442246140/Destination-Dissertation-A-Traveler's-Guide-to-a-Done-Dissertation-Second-Edition) * Allison Prieur's LinkedIn profile (https://www.linkedin.com/in/allisonprieur) Music by Matt Ingelson, http://www.mattingelsonmusic.com/
    続きを読む 一部表示
    50 分
  • 42: Making ECB sticky with Corey Newhouse and Jessica Manta Meyer

    42: Making ECB sticky with Corey Newhouse and Jessica Manta Meyer
    2023/06/20
    In this episode, I chat with Corey Newhouse and Jessica Manta Meyer, both from Public Profit, about how to make evaluation capacity building sticky for the organizations we work with. Some resources mentioned: TIERS Framework (https://aea365.org/blog/ie-tig-week-laura-beals-and-rachel-albert-on-tiers-a-tool-for-allocating-evaluation-resources-at-nonprofit-agencies/) Tipping Point Community (https://tippingpoint.org/) Irvine Foundation (https://www.irvine.org/) Change Cadet (https://www.changecadet.com/) NonProfit AF (https://nonprofitaf.com/) article on capacity building does not work (https://nonprofitaf.com/2020/10/capacity-buildings-necessary-existential-crisis/) Info on the Public Profit’s evaluation cohort (https://www.publicprofit.net/Driving-Toward-Impact) Contact information: info@publicprofit.net About Corey: Corey got her start as a teacher at Summerbridge Cincinnati in the early 1990s and has been involved in educational equity and social justice movements ever since. She founded Public Profit to build a team that would seamlessly blend social science research methods, organizational change strategy, and a deep commitment to supporting changemakers. As the Founder and Principal of Public Profit, Corey leads the team’s strategic direction, external relationships, and business development. In addition, Corey serves as an internal thought partner to project teams, assisting with the design of Public Profit’s engagements in evaluation, capacity building, and strategic program design. She is co-author of Public Profit’s Creative Ways to Solicit Stakeholder Feedback and Dabbling in the Data, and a contributor to Evaluation Failures: 22 Tales of Mistakes Made and Lessons Learned. She is a co-editor of the volume, Measure, Use, Improve! Data Use in Out-of-School Time. About Jes: Jessica began her career as a peer volunteer on a national youth talkline providing referrals and support to youth in crisis. Challenged by what she heard on the talkline, and by some of her own experiences with the health care system, she developed an interest in health education, equity and advocacy, which launched a career in the social services sector. Eventually, Jessica combined that with a lifetime love of numbers and inquiry, which led her to program evaluation. Prior to joining Public Profit, Jessica had several years of experience creating, directing and evaluating a range of youth development, LGBT, and health programs. Her work has run the gamut of nonprofit and social service roles including nonprofit finance, human resources, development and agency-led evaluations. Jessica directs many of the projects at Public Profit. She designs evaluation studies, develops project strategy, and manages implementation including all aspects of data collection, analysis, and reporting. An expert facilitator, Jessica also facilitates large stakeholder meetings and evaluation capacity building trainings, and provides evaluation coaching to clients and staff alike.
    続きを読む 一部表示
    1 時間 13 分
  • 41: Theory-based approaches for navigating complexity with Michael Moses

    41: Theory-based approaches for navigating complexity with Michael Moses
    2023/05/23
    In this episode, I chat with Michael Moses about theory-based approaches for navigating complexity, adaptive management, participatory strategy, and actions to shift power. Things and resources mentioned: Systems thinking in evaluation (https://www.bobwilliams.co.nz/ewExternalFiles/Wicked.pdf) (including a discussion of complexity) - Bob Williams and Sjon Van't Hof Outcome mapping (https://www.betterevaluation.org/methods-approaches/approaches/outcome-mapping) and outcome harvesting (https://www.betterevaluation.org/methods-approaches/approaches/outcome-harvesting) and emergent learning (https://emergentlearning.org/) Emergent Learning (https://emergentlearning.org/) (including before action reviews, after action reviews, emergent learning tables, and more approaches for supporting collective learning and action) Strategy Testing (https://asiafoundation.org/wp-content/uploads/2015/10/Strategy-Testing-An-Innovative-Approach-to-Monitoring-Highly-Flexible-Aid-Programs.pdf) - Deborah Ladner and the Asia Foundation Kat Haugh (https://www.katherinehaugh.com/) - visual facilitation, sensemaking, and notetaking Learning to Make All Voices Count Initiative - Summary (https://www.makingallvoicescount.org/supporting-local-learning-adaptation-understanding-effectiveness-adaptive-processes) and full paper (https://opendocs.ids.ac.uk/opendocs/bitstream/handle/20.500.12413/13351/MAVC_RR_Moses%20FINAL.pdf) Cynara training on Decolonizing M&E and Research (https://cynara.co/trainingstore/decolonize), led by Michelle Lokot (https://uk.linkedin.com/in/michellelokot) Contact information: Michael Moses mmoses@encompassworld.com (mailto:mmoses@encompassworld.com) LinkedIn (https://www.linkedin.com/in/michael-moses-a9bb9616/) About Michael Moses: Michael Moses is a strategist, facilitator, and evaluator with over 12 years of experience working with public and private sector partners to achieve social impact. He advises foundations, companies, governments, and nonprofits in their efforts to design and implement strategies for advancing change, including by capturing and using data to navigate the complex systems in which they work. In doing so, he helps changemakers learn how to improve programs and organizations, adapt, and over time, strengthen their impact.
    続きを読む 一部表示
    55 分