What Horticulture IoT (and spite) Can Teach Us About Data Science

What Horticulture IoT (and spite) Can Teach Us About Data Science

Speakers: Corey Thuen

Conference: S4

YouTube: https://www.youtube.com/watch?v=VgdqvIl2o78

Overview

Corey Thuen's S4 conference talk, "What Horticulture IoT (and spite) Can Teach Us About Data Science," offers a refreshingly unconventional perspective on data science, security, and the human element. Thuen masterfully weaves an extended analogy between the seemingly mundane task of growing tulips and complex cybersecurity challenges, arguing that fundamental principles of data analysis, critical thinking, and human psychology apply across disparate domains. The talk emphasizes how spite, often considered a negative emotion, can be a powerful motivator for challenging entrenched, unscientific practices and driving innovation, particularly when confronted with the "bias of experience."

This presentation is highly relevant for cybersecurity professionals, data scientists, and anyone grappling with organizational resistance to change. Thuen's narrative dissects how long-held beliefs, even those lacking empirical support, can persist due to human factors like reputation, comfort with the status quo, or even undisclosed conflicts of interest. By demonstrating how data-driven experimentation can debunk myths and reveal more efficient solutions in horticulture, the talk provides a compelling framework for re-evaluating and improving cybersecurity strategies, moving beyond mere compliance to genuine risk reduction.

Ultimately, Thuen's talk serves as a call to action for security practitioners to embrace curiosity, challenge assumptions, and leverage data science to make informed decisions. It highlights the critical need for robust observability, thoughtful metric selection, and an acute awareness of the human dynamics that often dictate the success or failure of security initiatives. The horticultural experiment, while seemingly simple, becomes a powerful metaphor for the perpetual process of securing complex systems in the face of evolving threats and human inertia.

Background

▶ Watch: Introduction: Spite as a powerful motivator (0:00)

The genesis of Thuen's exploration into horticultural data science stems from a friend's frustration at a large institution responsible for planting tens of thousands of tulips annually. For 25 years, these tulips had been covered with straw during winter, a practice initiated after a single year of poor yield. Despite the significant labor involved in applying and removing the moldy straw, and its potential to damage plants, the practice persisted without scientific justification. When Thuen's friend questioned this tradition, he was met with the dismissive response: "Stop asking questions. This is the way that it's been done."

This scenario, Thuen argues, is strikingly similar to common challenges faced in the cybersecurity world. He recounts a personal anecdote from his time at Idaho National Laboratory, where his team was tasked with vulnerability research on vendor equipment. After independently setting up and hacking the equipment, discovering critical vulnerabilities within a week, the vendor's representative dismissed their findings, claiming the team couldn't have possibly gotten the system running without their guidance. This experience, driven by perceived injustice and professional slight, fueled the team to work even harder, ultimately discovering more bugs. Thuen uses this to illustrate his point that "the greatest pleasure in life is doing what others say that you cannot," and how spite can be a potent, albeit double-edged, motivator for rigorous investigation and problem-solving.

A central theme in Thuen's background discussion is the bias of experience. While experience is invaluable, it can lead to complacency and a reluctance to re-evaluate established methods, even when more efficient or effective alternatives emerge. He uses the analogy of a daily commute: if one never re-evaluates their route, they might miss new shortcuts or road improvements. In security, this bias can lead to stagnation. Thuen points to the IBM Cost of a Breach report, noting that the average attacker dwell time—the period an adversary remains undetected in a network—was 300 days in their last report, an increase from previous years. This alarming statistic, coupled with the observation that many organizations retain far fewer than 300 days of logs, suggests a critical gap in observability and a failure to re-evaluate existing security postures. The problem exists because organizations often rely on historical practices or superficial metrics rather than applying data science principles to truly understand and improve their security efficacy.

Key Findings

▶ Watch: The 'stop asking questions' attitude (2:00)

Thuen's horticultural experiment yielded several significant findings, not just about growing tulips, but about the broader application of data science and critical thinking, especially in security contexts.

Firstly, the experiment revealed that while straw did indeed reduce temperature variance at shallow planting depths (2 inches), compost offered a comparable dampening effect, particularly at standard planting depths of 4 and 6 inches. This directly challenged the long-held, unscientific belief that straw was the uniquely superior covering. The data showed that the perceived benefit of straw was accurate in some specific conditions, validating the initial "gut feeling" of experts, but failed to account for other factors.

Secondly, and more critically, the talk highlighted the profound importance of choosing the right Key Performance Indicators (KPIs). If the sole KPI was "greatest reduction in temperature variance," straw would be the clear winner. However, when considering the "greatest impact on organizational mission," compost emerged as the superior solution. This is because compost was already being applied to the beds in spring, meaning a shift to fall application would eliminate the significant labor and cost associated with applying and removing straw, effectively "killing two birds with one stone." This finding underscores that optimizing for a single, potentially narrow, metric can lead to suboptimal overall outcomes, missing out on broader efficiencies and mission alignment.

Finally, the most striking finding related to the human element and hidden forces at work. When Thuen's friend presented the data supporting compost, the proposal was met with resistance. This led to the discovery of a "straw mafia"—the supplier of the straw was a family member of one of the managers. This direct conflict of interest revealed that resistance to change was not based on data or organizational benefit, but on personal financial gain. This anecdote serves as a powerful reminder that technical solutions and data-driven insights can often be derailed by non-technical, human factors, including reputation, fear of losing face, or undisclosed vested interests, a phenomenon equally prevalent in cybersecurity.

Technical Deep Dive

▶ Watch: Idaho National Lab: Vendor denial fuels research (2:25)

The technical core of Thuen's talk revolves around the design, implementation, and analysis of an Internet of Things (IoT)-based horticultural experiment, underpinned by principles of robust logging and data observability.

The initial approach involved seeking commercial garden IoT solutions, but these proved unreliable and "sucked," frequently breaking and failing to deliver promised monitoring capabilities. This led Thuen to adopt a do-it-yourself (DIY) approach, building custom sensor setups. While specific hardware components were not detailed, the implication was a collection of temperature sensors deployed in various configurations. These custom-built devices were assigned whimsical hostnames like Dirty Harry and Soil Sally, emphasizing the hands-on nature of the project.

The experiment was meticulously designed to compare different variables impacting tulip growth. Three primary covering configurations were tested:

1. No covering: Bare soil, serving as a control.
2. Straw covering: Two inches of straw, replicating the existing practice.
3. Compost covering: Two inches of compost, exploring an alternative.

In addition to coverings, three different planting depths were investigated:

1. 2 inches deep: A shallow depth, potentially due to volunteer error or laziness.
2. 4 inches deep: An intermediate depth.
3. 6 inches deep: The recommended depth for tulips.

The data collected primarily consisted of temperature readings from these embedded sensors. To provide a comparative baseline, air temperature data from the National Weather Service for the region was also integrated. Visualizations, presented via a Gravwell interface, clearly showed how air temperature fluctuations influenced soil temperatures. A key observation was the significant dampening effect of both straw and compost on soil temperature volatility compared to uncovered soil. This effect was particularly pronounced at shallower depths, where coverings substantially stabilized temperatures. Interestingly, snow was also identified as a natural insulator, providing a "big blanket" effect, though its presence was limited during the experiment year.

A critical technical consideration was sensor calibration. Thuen noted that even seemingly deterministic digital sensors require calibration, especially when comparing readings across multiple devices. The team calculated baseline differences for each sensor and used this information to enrich the data during analysis, ensuring accurate comparisons.

Beyond the horticultural specifics, Thuen delved into crucial principles of custom logging and observability, which are directly applicable to cybersecurity. He outlined several "dos and don'ts":

Logging "Dos":

• Log atomically: Each log entry should be self-contained and represent a complete event. He cited BIND DNS as a poor example, often requiring correlation of multiple log lines to understand a single request, contrasting it with DNSMasq which logs more comprehensively.
• Timestamp and log more: Always include precise timestamps and strive for comprehensive data capture, as you can't analyze what you don't log.

Logging "Don'ts":

• Don't require context: Log entries should ideally be understandable without external context or prior knowledge.
• Don't be ambiguous: Avoid vague or unclear messages that can lead to misinterpretation.
• Don't assume the use of the log entry: This is a subtle but critical point. Engineers writing logging code often have different analytical goals than security professionals consuming those logs. Logs should be designed to facilitate multiple analytical objectives, rather than assuming a specific use case.

This technical deep dive into IoT deployment, experimental design, and logging best practices underscores the rigorous, data-driven approach Thuen advocates, demonstrating how these principles transcend domain boundaries.

Demo / Proof of Concept

▶ Watch: Lesson 1: Beware the bias of experience (3:45)

While Thuen's talk did not feature a live hack or interactive demonstration in the traditional sense, the "Demo / Proof of Concept" section was effectively realized through the presentation and visualization of the collected horticultural data. The core of the demonstration was the Gravwell interface, a logging analytics and observability platform, which served as the primary tool for analyzing and presenting the temperature data from the custom-built IoT sensors.

Thuen presented several charts, which, despite potential color limitations in the video, clearly illustrated the key findings. These visualizations showed:

• Temperature trends over time: Displaying the fluctuating air temperature (the consistent bottom line across charts) against the soil temperatures at various depths and with different coverings.
• Dampening effect of coverings: Graphs visually demonstrated how both straw and compost significantly reduced the jagged temperature swings observed in uncovered soil, highlighting their insulating properties.
• Comparison of straw vs. compost: The charts allowed for direct visual comparison of the thermal stability provided by straw versus compost at different depths, showcasing that at deeper levels, the performance became comparable.

The physical proof of concept involved the actual deployment of the DIY temperature sensors in the tulip beds, named "Dirty Harry" and "Soil Sally," across the various experimental configurations (different depths and coverings). This physical setup, though not shown in detail, was the foundation for the data collection. The "demo" was therefore a powerful visualization of real-world data collected from a practical IoT implementation, directly supporting the talk's conclusions about the effectiveness of different coverings and the importance of data-driven decision making. It served as concrete evidence of how detailed logging and analytical platforms like Gravwell can transform raw sensor data into actionable insights, even in an unconventional domain like horticulture.

Defensive Implications

▶ Watch: Re-evaluating security: 300-day attacker dwell time (4:40)

Corey Thuen's horticultural analogy offers profound defensive implications for cybersecurity professionals, urging a re-evaluation of entrenched practices and a more data-driven, human-aware approach.

Firstly, the lesson to "beware the bias of experience" is paramount. Just as the straw covering persisted without scientific backing, many security controls and policies are maintained simply because "that's how it's always been done." Defenders must proactively challenge the status quo, re-evaluating their security architecture, tools, and processes. This means embracing new ideas, bringing in fresh perspectives, and being open to the possibility that current methods, like the "commute to work" analogy, may no longer be the most efficient or effective. The 300-day average attacker dwell time from the IBM report is a stark indicator that current defensive strategies are often insufficient, necessitating this critical re-evaluation.

Secondly, the talk underscores that security is a process, not a product. The failure of off-the-shelf garden IoT solutions mirrors the common disappointment with "silver bullet" security products. Organizations are unique, and a one-size-fits-all approach rarely works. Defenders must invest in building and continuously refining a security program that is tailored to their specific environment, threat landscape, and organizational culture. This involves iterative cycles of building, testing, tweaking, and adapting, rather than simply "checking boxes" or buying the latest vendor solution.

Thirdly, the cautionary tale of "being careful what you measure" is critical for security metrics. If the horticultural institution had only measured "reduction in variance," straw would have been the clear winner, missing the broader organizational mission benefits of compost. Similarly, security teams often focus on vanity metrics like "number of attacks blocked" or "vulnerabilities found," which can be easily gamed or fail to reflect true security posture. Defenders must select KPIs that genuinely align with the organization's mission and risk reduction goals. For instance, instead of just counting blocked phishing emails, measure the reduction in successful phishing compromises or the speed of incident response. Optimizing for the wrong KPI can lead to a false sense of security and misallocated resources.

Finally, Thuen highlights the pervasive influence of human factors and hidden forces. The "straw mafia" anecdote serves as a potent reminder that technical solutions can be undermined by human resistance, whether driven by reputation, fear of change, or undisclosed conflicts of interest. In cybersecurity, this manifests in various ways:

• Resistance to reporting incidents: Employees may fear repercussions for falling victim to phishing, leading to underreporting and delayed detection.
• Siloed operations: Teams may resist sharing information or adopting new security practices due to internal politics or perceived threats to their turf.
• Misaligned incentives: Security decisions might be influenced by vendor relationships, internal power struggles, or personal agendas rather than objective risk analysis.

Defenders must develop strong communication and change management skills, understanding the human psychology behind resistance. Building trust, demonstrating clear value, and addressing underlying concerns are as important as technical prowess. A robust observability strategy, encompassing comprehensive logging, is also a critical defensive implication. Adhering to the "dos and don'ts" of custom logging (logging atomically, timestamping, avoiding ambiguity and context dependency, and not assuming log usage) ensures that security teams have the rich, actionable data needed to detect threats, investigate incidents, and measure the true effectiveness of their controls. This proactive, data-driven, and human-aware approach is essential for building resilient and adaptable defensive capabilities.

Key Takeaways

• Challenge the Bias of Experience: Regularly re-evaluate existing security practices and assumptions, as long-held traditions may not be the most effective or efficient.
• Security is a Process, Not a Product: Relying solely on commercial solutions or checking compliance boxes is insufficient; true security requires continuous adaptation, customization, and iterative improvement.
• Carefully Choose and Present Metrics: Focus on KPIs that align with the organization's mission and genuine risk reduction, rather than vanity metrics that can be easily manipulated or lead to suboptimal outcomes.
• Acknowledge Hidden Human Forces: Be aware that resistance to change in security can stem from non-technical factors like reputation, fear, or undisclosed conflicts of interest, and address these human elements proactively.
• Embrace Observability and Quality Logging: Implement comprehensive, atomic, and unambiguous logging practices that provide sufficient context and do not assume specific analytical use cases, enabling better detection and analysis.
• Spite as a Motivator: Harness frustration or a sense of injustice to drive rigorous data collection and analysis, proving what others claim cannot be done, ultimately leading to better solutions.

About the Speaker(s)

Corey Thuen is a seasoned security professional and data scientist with a background in critical infrastructure and vulnerability research. His experience includes working at Idaho National Laboratory, where he engaged in offensive security research, discovering critical vulnerabilities in industrial control systems. Thuen is passionate about logging and observability, advocating for robust data collection and analysis as fundamental to effective cybersecurity. He is also an entrepreneur, having founded companies aimed at solving complex security challenges, and is known for his willingness to challenge the status quo and take on significant competitors. His work consistently emphasizes the importance of data-driven decision-making, critical thinking, and understanding the human element in technology and security.