When Failure Happens Beyond Control of Designer

The classic design engineering approach is based on the outdated idea that failure is not an option. For decades, if not centuries, engineers have designed systems to prevent their collapse. But today, we know that systems will go down, so the real focus should be, “OK, if this fails — how do we make it easy to fix?”

Computer systems, never designed for security in cyber space, are undergoing critical tests and often are targeted by malicious actors to fall. Infrastructure was designed with limits from bygone eras. Supply chains were never designed for today’s volume and international commerce shocks; they resulted from business decisions, not based on system failure modes.

For government employees and policymakers, this is a brave new world. Failures, while unacceptable, will happen, and everyone needs to understand how to shift the paradigm from classic engineering design to dynamic system views.

For example, the General Accountability Office (GAO) recently reported how some airports implemented electrical resiliency. Airports were able to identify a particular challenge and interruptions in power, and then develop solutions, such as microgrids, to not only complement the existing power supply but as extra control when the power grid was being challenged. Improvement means not just replacing but considering resiliency as part of the future. Other solutions can create innovative thinking across other airport management areas.

Airport infrastructure is only one type of physical asset government policymakers and employees need to be concerned about. In 2013 the Rockefeller Foundation began work focused on cities’ resiliency more broadly. Considering resilience projects gets everyone thinking about risks they have already seen and ones they may not until it’s too late.

Government stakeholders should first identify potential failures or interrupts. In the GAO report, it was defined as an unplanned power loss of five minutes or longer. Once the unplanned event was identified, the next step was determining the impact to which systems, and which users of the system would be most affected.

During the COVID-19 pandemic, various systems were impacted in many ways. For some states, it was their unemployment system. The experience now provides policymakers with boundary points that indicate how a large, unforeseen event might impact their societal systems.

While many in emergency planning still talk about “black swan” unpredictable events, everyone needs to consider how resiliency can be placed into systems, beyond engineers’ touchstone of duplicate systems. Resiliency can be designed into a system or considered afterwards. Effective change management will be needed in many sectors to shift the paradigm, and to reimagine alternative solutions, such as public-private partnerships, and to draw in more innovators and creative thinkers.

Christopher Rabzak is the founder of CRXJEM Consulting LLC, a management consulting firm that focuses on technology companies.  Chris received his Aerospace Engineering degree from Penn State University, and worked in private industry for Teledyne Ryan and Boeing, as well as government entities including NASA.  Chris earned an MBA and a JD from Widener University.  He founded CRXJEM in 2008, and works with tech firms, develops continuing legal education courses to help attorneys understand how technology works, and is a certified continuing legal education provider in Pennsylvania. He is currently forming an international advisory board for a growing tech company.

Photo courtesy of the National Archives