System Evaluation

WHAT IS A SYSTEM?

“An integrated composite of components that provide function and capability to satisfy a stated need or objective. A system is a holistic unit that is greater than the sum of its parts. It has structure, function, behavior, characteristics, and interconnectivity. Modern day systems are typically composed of people, products, and environments that together generate complexity and capability (p. 402).”

Ericson, C. A. (2011). Concise encyclopedia of system safety: Definition of terms and concepts. John Wiley & Sons: Hoboken, N.J.

“There's always a flaw in the system” - Beetee (The Hunger Games Catching Fire)

OUR CONTRIBUTION TO SYSTEM EVALUATION

We are proud to be one of the world leaders in system evaluation. Dr. Renger published system evaluation theory (SET) which he developed based on his work in the emergency response sector. Our evaluation family has since published 7 articles on the application of SET. SET combines systems thinking and system theory to walk the client through three steps to evaluate their systems. SET is especially helpful in evaluating agencies working together around a common goal.

WHAT SYSTEM EVALUATION SERVICES DO YOU OFFER?

When evaluating a system we explore your needs in 2 main areas.

1. DEFINING YOUR SYSTEM

Working with leadership we will define who shares your common goal and how are you connected. For more information, please refer to our article on defining systems linked below.

Defining Systems

2. EVALUATING SYSTEM EFFICIENCY

We will examine your standard operating procedures for:

1. System waste: we look for unnecessary duplication of effort (waste). We will then evaluate the source for this waste. Is it a staff training issue? Is your information technology (IT) causing delays? If your leadership supportive and providing the resources needed? Is your staff culture supportive of your goal? For more information, please refer to our article on system waste linked below.

2. Cascading failures: we don’t just look at where we see the problem, but evaluate to determine whether there is an upstream cause. For more information, please refer to our article on system cascading failures linked below.

3. Functional feedback loops: system health depends on functional feedback loops. We examine whether information is getting back to who needs it and whether the feedback is viewed as credible, relevant, specific, timely and sufficiently frequent. For more information, please refer to our article on system

feedback loops linked below.

4. Reflex arcs: is there a way to streamline your operations without sacrificing quality?

Evaluating System Cascading Failures

Reworks: A robust system efficiency measure

Evaluation of System Feedback Loops

PRIOR SYSTEM EVALUATION WORK

Becker, Renger, Mcpherson Inidcators of

Renger Foltysova Renger Booze Defining s

Renger Keogh Hawkins EJA article Robust

Renger SET article in EJA December 2015.

Renger, Foltysova, Ienuso, Renger & Booz

Renger, Krapp, Wood, Williamson 2011 Sys

Renger, R. Evaluation of system feedback

Becker, K., Renger, R., & McPherson, M. (2015). Indicators of buy-in to gauge evaluation success. Evaluation Journal of Australasia, 15 (2) 12-21.

Renger, R., Foltysova, J., Renger, J., & Booze, W. (2017). Defining systems to evaluate system efficiency and effectiveness. Evaluation Journal of Australasia, 17 (3), 4-13.

Renger, R., Keogh, B., Hawkins, A., Foltysova, K., & Souvannasacd, E. (2018) Reworks: A robust system efficiency measure. Evaluation Journal of Australasia, 18 (3), 183-191.

Renger, R. (2015). System Evaluation Theory (SET). Evaluation Journal of Australasia, 15 (4), 16-28.

Renger, R., Foltysova, J., Ienuso, S., Renger, J., & Booze, W. (2017). Evaluating system cascading failures. Evaluation Journal of Australasia, 17 (2), 29-36.

Renger, R., Wood, S., Williamson, S., & Krapp, S. (2011). Systemic evaluation, impact evaluation, and logic models. Evaluation Journal of Australasia, 11 (2), 24-30.

Renger, R. (2016). Illustrating the Evaluation of System Feedback Mechanisms using System Evaluation Theory (SET). Evaluation Journal of Australasia, 16 (4), 15-21.