SYSTEM EVALUATION
WHAT IS A SYSTEM?
A system is “an integrated whole whose essential properties arise from the relationship [interdependence] between its parts (p. 142)."
Ison, R. L. (2008). Systems thinking and practice for action research.
“There's always a flaw in the system” - Beetee (The Hunger Games Catching Fire)
OUR CONTRIBUTION TO SYSTEM EVALUATION
We are proud to be one of the world leaders in system evaluation. Dr. Renger published system evaluation theory (SET) which he developed based on his work in the emergency response sector. Our evaluation family has since published 7 articles on the application of SET. SET combines systems thinking and system theory to walk the client through three steps to evaluate their systems. SET is especially helpful in evaluating agencies working together around a common goal.
WHAT SYSTEM EVALUATION SERVICES DO YOU OFFER?
When evaluating a system we explore your needs in 2 main areas.
1. DEFINING YOUR SYSTEM
Working with leadership we will define who shares your common goal and how are you connected. For more information, please refer to our article on defining systems linked below.
2. EVALUATING SYSTEM EFFICIENCY
We will examine your standard operating procedures for:
1. System waste: we look for unnecessary duplication of effort (waste). We will then evaluate the source for this waste. Is it a staff training issue? Is your information technology (IT) causing delays? If your leadership supportive and providing the resources needed? Is your staff culture supportive of your goal? For more information, please refer to our article on system waste linked below.
2. Cascading failures: we don’t just look at where we see the problem, but evaluate to determine whether there is an upstream cause. For more information, please refer to our article on system cascading failures linked below.
3. Functional feedback loops: system health depends on functional feedback loops. We examine whether information is getting back to who needs it and whether the feedback is viewed as credible, relevant, specific, timely and sufficiently frequent. For more information, please refer to our article on system
feedback loops linked below.
4. Reflex arcs: is there a way to streamline your operations without sacrificing quality?
PRIOR SYSTEM EVALUATION WORK
Becker, K., Renger, R., & McPherson, M. (2015). Indicators of buy-in to gauge evaluation success. Evaluation Journal of Australasia, 15 (2) 12-21.
Renger, R., Atkinson, L., Renger, J., Renger, J., & Hart, G. (2019). The connection between logic models and systems thinking concepts. Evaluation Journal of Australasia, 19(2), 79-87.
Renger, R., Basson, M. D., Hart, G., Van Eck, R., Souvannasacd, E., Renger, J., & Foltysova, J. (2020). Lessons learned in evaluating the infrastructure of a Centre for Translational Research. Evaluation Journal of Australasia, 20(1), 6-22.
Renger, R., Foltysova, J., Renger, J., & Booze, W. (2017). Defining systems to evaluate system efficiency and effectiveness. Evaluation Journal of Australasia, 17 (3), 4-13.
Renger, R., Keogh, B., Hawkins, A., Foltysova, K., & Souvannasacd, E. (2018) Reworks: A robust system efficiency measure. Evaluation Journal of Australasia, 18 (3), 183-191.
Renger, R. (2015). System Evaluation Theory (SET). Evaluation Journal of Australasia, 15 (4), 16-28.
Renger, R., Foltysova, J., Ienuso, S., Renger, J., & Booze, W. (2017). Evaluating system cascading failures. Evaluation Journal of Australasia, 17 (2), 29-36.
Renger, R., Wood, S., Williamson, S., & Krapp, S. (2011). Systemic evaluation, impact evaluation, and logic models. Evaluation Journal of Australasia, 11 (2), 24-30.
Renger, R. (2016). Illustrating the Evaluation of System Feedback Mechanisms using System Evaluation Theory (SET). Evaluation Journal of Australasia, 16 (4), 15-21.
Renger, R., Foltysova, J., Renger, J., Donaldson, S.I., Hart, G., & Hawkins, A. Comparing and contrasting a program versus system approach to evaluation: the example of a cardiac care system. Canadian Journal of Program Evaluation, 35(2), 240-257.
Renger, R., Hart, G., Van Eck, R., Souvannasacd, E., Renger., J., & Basson, M.D. Lessons Learned in Evaluating a NIH Centre for Translational Research (CTR). Evaluation Journal of Australasia, 20 (1), 1-17