Normalization of Deviance

Normalization of Deviance

Speakers: Marco Ayala

Conference: S4x24

YouTube: https://www.youtube.com/watch?v=u1xmyJmGsS0

Overview

Marco Ayala's S4x24 talk, "Normalization of Deviance," delves into a critical, often overlooked phenomenon impacting industrial control systems (ICS) and operational technology (OT) environments. The talk explores how organizations, individuals, and groups gradually accept a lower standard of performance or safety, leading to what becomes a "new normal" over time. This insidious process, rooted in human behavior and organizational culture, poses a significant threat to the integrity and security of critical infrastructure as it undergoes digital transformation.

Ayala, drawing on decades of experience in the petrochemical industry and ICS security, emphasizes that this normalization isn't a sudden collapse but a slow erosion of standards. He highlights how the increasing digitization of previously electromechanical and pneumatic systems, while offering efficiency, concurrently introduces new vectors for failure and compromise that are often dismissed or ignored due to a lack of immediate catastrophic consequences. The talk serves as a stark warning and a call to action for both seasoned OT professionals and newcomers to actively question established practices before seemingly minor deviations accumulate into a major disaster.

Background

▶ Watch: Defining normalization of deviance and its impact (0:45)

The concept of normalization of deviance was meticulously studied and popularized by sociologist Dr. Diane Vaughn in her seminal work on the Space Shuttle Challenger disaster. On January 28, 1986, seven astronauts tragically lost their lives due to a catastrophic failure attributed to faulty O-rings, a known issue that had been repeatedly dismissed by NASA engineers and management over time because previous flights with similar issues had not resulted in disaster. Dr. Vaughn defines this phenomenon as "a process where a clearly unsafe practice comes to be considered normal if it does not immediately cause a catastrophe." This definition is central to understanding the insidious nature of risk accumulation in complex systems.

Ayala draws parallels from his extensive career, including nearly two decades in petrochemical facilities dealing with highly hazardous materials, where safety systems, policies, engineering controls, and Management of Change (MOC) procedures were paramount. He notes that, much like the Challenger incident, and other industrial accidents such as the BP Texas City refinery explosion, a common thread is the long dwell time or incubation period. During this period, early warning signs are either misinterpreted, missed, neglected, or outright ignored, leading to a gradual acceptance of suboptimal or unsafe conditions.

The digital transformation sweeping through OT environments mirrors this historical pattern. Ayala challenges the common narrative that vendors are solely responsible for introducing these new complexities, asserting that asset owners themselves "asked for it" as they sought greater efficiency, connectivity, and data insights. He illustrates this with a relatable analogy: the evolution of vehicle ignition systems. Decades ago, starting a car involved a physical key turning an ignition switch, triggering an electromechanical relay to engage the starter. Today, a key fob sends a digital request, fundamentally changing the interaction from a direct physical action to a software-mediated command. This shift, while convenient, introduces layers of digital abstraction and potential points of failure or compromise that were absent in purely mechanical systems.

Historically, industrial control systems were far simpler and more physically isolated. Ayala recounts working on plants built in the 1940s, where pneumatic control systems were the norm. He shared images of Foxboro pneumatic computers and pneumatic chart recorders, emphasizing that basic process control was handled pneumatically, while emergency shutdowns relied on robust electromechanical shutdowns. These systems, while less flexible, operated on physical principles that were often more transparent and less susceptible to the complex, systemic failures associated with software bugs or network vulnerabilities. The transition from these tangible, direct control mechanisms to interconnected, software-driven digital systems represents a profound paradigm shift, one whose full safety and security implications are still being navigated, often under the shadow of normalized deviance.

Key Findings

▶ Watch: Understanding 'long dwell time' and neglected early warnings (3:00)

The central insight from Marco Ayala's talk is the critical and often unacknowledged prevalence of normalization of deviance within operational technology and industrial control systems. This phenomenon is not merely an academic concept but a tangible, ongoing process directly impacting the safety, reliability, and security posture of critical infrastructure worldwide.

Ayala's key findings highlight several crucial aspects:

1. Pervasive Nature in OT: The talk unequivocally establishes that normalization of deviance is deeply embedded in OT practices. As industrial systems become increasingly digitized and interconnected, practices that would have been considered unacceptable or highly risky in purely analog or electromechanical environments are gradually accepted because they do not immediately result in catastrophic failure. This leads to a creeping erosion of safety margins and security baselines.
2. The Danger of "No Immediate Catastrophe": A core tenet of normalized deviance is that unsafe practices become normalized precisely because they do not instantly cause harm. This creates a false sense of security, leading organizations to defer necessary fixes, bypass established protocols, or continue using outdated/vulnerable systems. The absence of an immediate incident reinforces the belief that the deviation is acceptable, or even the "new normal."
3. The Critical Role of Long Dwell Times: Ayala emphasizes the concept of long incubation periods or long dwell times where early warning signs of accumulating risk are consistently misinterpreted, overlooked, or actively ignored. These subtle indicators, such as minor system glitches, unexplained anomalies, or repeated policy circumventions, are dismissed as isolated incidents rather than symptoms of systemic issues. Over time, these unaddressed deviations compound, creating a highly vulnerable environment ripe for a major incident.
4. Asset Owner Responsibility: A significant finding is the assertion that the drive towards digital transformation in OT was largely initiated by asset owners themselves, rather than solely being pushed by vendors. This places a direct onus on asset owners to recognize and address the inherent risks introduced by this shift. Their requests for greater connectivity, remote access, and data integration have fundamentally altered the risk landscape, making it imperative for them to lead the charge in establishing robust, cyber-resilient safety and security practices.
5. Overlooked Early Warning Indicators: Ayala points out that the industry has been subjected to "clear early warning indicators" that have been "neglected, omitted or just said, ah, nothing bad's really happened." These indicators could range from minor cyber incidents in IT affecting OT, to near-misses in process safety, or even audit findings that are deprioritized. The failure to critically assess and act upon these signals is a direct manifestation of normalized deviance, paving the way for more severe future events.

In essence, the talk argues that the current state of OT security is, in many respects, a product of this long-term normalization process. The transition from robust, physically isolated, and electromechanical systems to complex, interconnected, and software-dependent digital infrastructures has introduced new, abstract risks that are harder to perceive and quantify, making them particularly susceptible to being normalized away until a critical failure forces a re-evaluation.

Technical Deep Dive

▶ Watch: Historical evolution from pneumatic to digital control systems (4:30)

The technical heart of Marco Ayala's talk lies in illustrating the profound shift from a physically deterministic operational environment to one increasingly reliant on complex digital systems, and the security implications embedded within this transition. The core argument is that as OT environments have digitized, the mechanisms of control and safety have changed in ways that introduce new vectors for "deviancy" and subsequent normalization.

Historically, industrial control was characterized by pneumatic control systems. Ayala provided a glimpse into this past with his personal photographs of Foxboro pneumatic computers and pneumatic chart recorders. These systems operated on air pressure, with physical components directly manipulating valves or recording data. For instance, a change in pressure could directly open or close a valve. Emergency shutdown systems (ESD) in these older plants were often electromechanical shutdowns, relying on physical relays, switches, and direct electrical circuits to rapidly bring a process to a safe state. These systems, while slower and less flexible than modern digital counterparts, possessed a high degree of physical isolation and transparency in their operation. Troubleshooting often involved observing physical components, and failures were typically hardware-centric and localized. Ayala mentioned that even the chart recorders, while pneumatic for data display, still required 120-volt power, indicating discrete power requirements for limited digital components.

The modern paradigm, driven by asset owner demand for efficiency and data, has seen a wholesale migration to digital control systems. This shift is exemplified by the car ignition analogy: from a physical key switch engaging an electromechanical relay to a digital input from a key fob requesting the vehicle to start. In an industrial context, this means that critical commands—such as initiating a process, adjusting parameters, or triggering safety functions like enabling/disabling Anti-lock Braking Systems (ABS) in a vehicle (or analogous safety instrumented functions in a plant)—are now mediated by software, network protocols, and digital signals.

This transition has several profound technical implications:

• Abstraction of Control: Digital systems introduce layers of abstraction. Instead of a direct physical connection, commands are now interpreted by software, communicated over networks, and executed by microprocessors. This abstraction makes the system's behavior less immediately observable and more reliant on the correctness of code and configuration.
• Introduction of Software Bugs: Ayala explicitly states that "systems do have bugs," a lesson he learned from mentors who taught him to "design around bugs in control systems, bugs and safety systems." Unlike mechanical failures, software bugs can introduce subtle, unpredictable behaviors, create vulnerabilities, or lead to logical errors that are difficult to diagnose and can cascade across interconnected systems. The potential for zero-day vulnerabilities or inadvertently introduced flaws is ever-present.
• Increased Connectivity and Attack Surface: Dr. David Clark's early estimate of 10,000 internet-connected devices pales in comparison to the reality of today's interconnected world, where even a single industrial block can host tens of thousands of devices. This exponential increase in connectivity means that OT networks, once air-gapped or physically isolated, are now frequently bridged to corporate IT networks, the internet, or vendor support systems. Each connection point represents a potential attack vector, expanding the attack surface dramatically.
• Software-Defined Safety Logic: Modern Basic Process Control Systems (BPCS) and Emergency Shutdown Systems (ESD) are now largely software-defined. While the fundamental purpose remains "a system composed of sensors, logic solvers, and final control elements for the purpose of taking the process to a safe state when pre-determined conditions are violated," the "logic solvers" are now programmable logic controllers (PLCs) or distributed control systems (DCS) running complex firmware and application code. This introduces the risk of logic errors, malicious code injection, or unauthorized configuration changes that could compromise safety integrity levels (SILs).
• Complex Interdependencies: The digital ecosystem fosters complex interdependencies between various systems, vendors, and even cloud services. A vulnerability or deviation in one component, or a misconfiguration by an integrator, can have unforeseen ripple effects across the entire operational environment.

In summary, the technical deep dive reveals that the shift from pneumatic and electromechanical control to digital, networked systems fundamentally alters the nature of risk in OT. It moves from readily observable physical failures to more abstract, software-driven vulnerabilities and attack surfaces, creating fertile ground for the normalization of deviations that might not manifest as immediate physical catastrophes but silently erode the system's resilience and security.

Demo / Proof of Concept

▶ Watch: Asset owners drove digital adoption, not just vendors (6:00)

This talk did not include a live demonstration or proof of concept, focusing instead on the conceptual framework of normalization of deviance and its historical context within industrial operations and the digital transformation of OT.

Defensive Implications

▶ Watch: Call for collaboration and continuous improvement (6:20)

Understanding the normalization of deviance is not merely an academic exercise; it carries profound defensive implications for organizations operating in the OT space. Marco Ayala's talk provides a clear roadmap for how defenders, from frontline technicians to executive leadership, can actively combat this insidious phenomenon and build more resilient and secure industrial environments.

1. Cultivate a Culture of Questioning and Skepticism: The most critical defensive implication is the need to actively challenge existing norms and practices, especially those that have become routine due to a lack of immediate negative consequences. Ayala stresses, "We need to start questioning things well in advance before there is a catastrophe." This involves fostering an environment where employees are empowered and encouraged to raise concerns about perceived deviations, even if they haven't led to an incident yet. This proactive skepticism is vital to identify and address early warning indicators before they escalate.

1. Embrace Collaboration and Information Sharing: Ayala highlights that "we all need to collaborate, we need to work together." The complexity of modern OT environments demands a collective defense strategy. This means breaking down silos between IT and OT, between different departments (operations, maintenance, security), and across organizations (asset owners, vendors, integrators). Platforms like S4, where professionals can share insights and learn from others' experiences, are crucial for this collaborative effort. Sharing lessons learned from near-misses, vulnerabilities, and even minor incidents can prevent the normalization of risky behaviors across the broader industry.

1. Prioritize Continuous Improvement: The digital landscape is constantly evolving, and so must defensive strategies. Ayala advocates for celebrating "the power of continuous improvement." This isn't just about implementing new technologies but about continually reviewing, refining, and adapting processes, policies, and security controls. Regular audits, vulnerability assessments, penetration testing, and incident response drills should be viewed as opportunities for improvement, not just compliance checkboxes.

1. Actively Learn from Past Deviations and Near-Misses: Instead of dismissing incidents that didn't result in catastrophe, organizations must meticulously analyze them. The "long incubation period" where early warning signs are "misinterpreted, missed or neglected or just completely ignored" must be countered with a rigorous process of incident analysis. Every deviation, no matter how minor, should be treated as a potential signal of a deeper systemic issue, prompting a thorough investigation into its root causes and implications for safety and security. This includes reviewing historical data and incident reports to identify patterns of normalized deviance.

1. Holistic Risk Assessment and Management: Defenders need to understand "where in our process with our companies, our organizations, our technicians, our maintenance staff, our vendors, our integrators, all this piece of it, we need to be able to understand where that is." This calls for a comprehensive, holistic approach to risk assessment that considers not only technical vulnerabilities but also human factors, organizational culture, and the entire supply chain. It means ensuring that security is integrated into every stage of the lifecycle, from design and procurement to operation and maintenance.

1. Educate and Empower the Workforce: Given that the normalization of deviance affects individuals at all levels, continuous education and training are paramount. This includes not just technical cybersecurity training but also awareness programs on the principles of safety culture, human factors, and the risks associated with complacency. Empowering technicians and operators with the knowledge and authority to halt unsafe operations or report security concerns without fear of reprisal is fundamental to breaking the cycle of normalized deviance.

By embracing these defensive strategies, organizations can actively resist the gravitational pull of normalized deviance, ensuring that the pursuit of efficiency and connectivity does not inadvertently compromise the fundamental safety and security of critical industrial operations.

Key Takeaways

• Normalization of Deviance is a Pervasive Threat: The gradual acceptance of lower safety or performance standards due to a lack of immediate catastrophic consequences is a critical, often unrecognized, risk factor in OT/ICS environments.
• Digital Transformation Introduces New, Subtle Risks: The shift from electromechanical and pneumatic control to digital, networked systems, driven by asset owner demand, creates abstract vulnerabilities and attack surfaces that are easily normalized if not proactively challenged.
• Long Incubation Periods Mask Accumulating Dangers: Early warning signs and minor deviations are often overlooked or dismissed, leading to a false sense of security and allowing systemic risks to compound over extended periods until a major incident occurs.
• Proactive Questioning is Essential: Organizations must foster a culture where all stakeholders are encouraged to critically question established practices and raise concerns about deviations, no long before a catastrophe, rather than waiting for failure.
• Collaboration Across the Ecosystem is Paramount: Effective defense against normalized deviance requires robust collaboration and information sharing among asset owners, vendors, integrators, and internal teams (IT, OT, safety, maintenance).
• Continuous Improvement is Non-Negotiable: Given the evolving threat landscape and the inherent nature of complex systems, a commitment to continuous learning, adaptation, and improvement in security and safety practices is vital to maintain resilience.

About the Speaker(s)

Marco Ayala is a seasoned expert in industrial control systems and operational technology security, bringing a wealth of practical experience to the field. He has dedicated almost two decades of his career to working in petrochemical facilities, where he gained firsthand experience with hazardous materials and the critical importance of robust safety systems. With over 20 years of experience conducting risk assessments, field walks, and surveys, Ayala has served in dual capacities as both an asset owner and a consultant, providing him with a comprehensive understanding of OT security challenges from multiple perspectives. He credits his expertise in designing around vulnerabilities to being trained by some of the best mentors in the world, who taught him how to specifically address bugs in control and safety systems. His background includes working on industrial plants built in the 1940s, giving him a unique historical perspective on the evolution from pneumatic industrial control systems to today's complex digital infrastructures.