BoF | Fueling Cloud Native: The Data We Have, the Data We Need - Hilary Carter, SVP Research

BoF | Fueling Cloud Native: The Data We Have, the Data We Need - Hilary Carter, SVP Research

Speakers: Hilary Carter, SVP Research

Conference: KubeCon EU

YouTube: https://www.youtube.com/watch?v=N6W8Ete9vJM

Overview

In this Birds of a Feather (BoF) session at KubeCon EU, Hilary Carter, SVP Research at the Linux Foundation, illuminated the critical role of data-driven insights in navigating the complexities of the open source and cloud-native landscape. Carter introduced LF Research, a dedicated function established four years ago to provide empirical measurements and strategic intelligence for the open source community. The core objective of this initiative is to equip organizations and project communities with the evidence needed to inform strategy, secure buy-in from decision-makers, and ultimately foster a more robust and sustainable open source ecosystem.

The session served as a dual-purpose platform: first, to showcase the extensive library of 78 unique studies already published by LF Research, making this valuable data freely available to the community under the Community Data License Agreement 2.0 (CDLA 2.0) on data.world. Second, and perhaps more crucially, it was a direct appeal to the audience to identify the data gaps and unmet needs that, if addressed, could unlock further value and resolve persistent challenges within their organizations. Carter emphasized that while the utility of open source is widely acknowledged, the tangible Return on Investment (ROI) for upstream contributions remains poorly understood, leading to significant strategic and operational hurdles.

This talk underscores a fundamental truth in the modern technology landscape: open source software is the bedrock of global infrastructure, yet its economic and strategic value is often opaque to those outside technical roles. By systematically collecting and analyzing data on adoption trends, developer journeys, security postures, and the impact of open governance, LF Research aims to provide the empirical ammunition necessary for technologists to advocate effectively within their organizations. The discussion highlighted how robust data can transcend technical jargon, translating the intrinsic value of open source into metrics that resonate with budgeting committees, HR departments, and executive leadership.

Background

▶ Watch: Introduction to LF Research and data availability (0:00)

The Linux Foundation, as a steward of over a thousand diverse open source projects, faces the immense challenge of understanding and measuring the myriad activities and trends within this vast ecosystem. Traditional anecdotal evidence often falls short when attempting to convey the strategic importance and economic impact of open source to non-technical stakeholders. This persistent gap between the technical understanding of open source's utility and the business-centric view of its value led to the establishment of LF Research. The function was founded with the explicit goal of creating a centralized repository of insights and empirical data to inform strategy and decision-making across the open source community.

Prior to LF Research, studies on open source were often ad-hoc or lacked a cohesive framework. Recognizing the inherent value in systematically quantifying trends—from developer participation and industry needs to project velocity and cross-organizational collaboration—the Linux Foundation formalized its research efforts. Since its inception, LF Research has amassed a comprehensive library of 78 unique studies, all freely accessible. This data is published under the CDLA 2.0, an open source license specifically designed for data sharing, making it available on the data.world platform for community exploration and use.

LF Research employs four distinct frameworks to analyze the multifaceted open source landscape:

1. Industry-Specific Analysis: Examining trends within particular sectors, such as networking (e.g., LF Networking Foundation), healthcare, and financial services.
2. Technology Domains: Focusing on specific technology areas like cloud native, AI, edge computing, and IoT.
3. Geographic Analysis: Understanding regional differences in adoption, contribution, and unique market needs across areas like Europe, North America, and Japan.
4. Cross-Industry/Discipline Issues: Investigating pervasive themes that affect all domains, including developer relations, diversity, equity, and inclusion (DEI), cybersecurity, and standards.

Despite these extensive efforts, a significant problem persists: the "contribution conundrum." While open source software constitutes 80-95% of most modern software stacks and underpins the global economy, the value of actively contributing upstream is profoundly misunderstood. This lack of empirical evidence for the monetary ROI of contributions makes it challenging to justify investment in open source teams, leading to scenarios like layoffs. The speaker emphasized that current survey methodologies, while effective at capturing technical insights from qualified contributors, struggle to quantify the direct revenue generated by the products incorporating their open source work, creating a difficult-to-close gap in understanding the full economic impact of contribution.

Key Findings

▶ Watch: Explaining LF Research's four analysis frameworks (3:20)

LF Research has published numerous studies offering critical insights into the cloud-native and broader open source ecosystems, addressing specific technology domains, industry trends, and developer experiences. Hilary Carter highlighted several recent reports that underscore the maturity and pervasive impact of open source:

The 2024 Cloud Native Study, released just prior to the KubeCon EU event, captured a "decade of velocity, code, cloud, and change." This report revealed a high level of cloud-native adoption at 89% among organizations. A definitive finding was the ubiquity of Kubernetes, which "dominates the conversation" and serves as the de facto container orchestration system. The study also observed organizations releasing code at a faster pace and a year-over-year increase in the adoption of GitOps principles and CI/CD (Continuous Integration/Continuous Delivery) approaches, illustrating the continued maturation of the cloud-native ecosystem.

Last year, the Kubernetes Turns 10 report delved into the project's impact on individual developers, startups, and enterprises. It found that 92% of organizations perceive Kubernetes as highly impactful for their deployments and software development practices, significantly boosting deployment speed. Despite this high value, the study also identified challenges, particularly the complexity of Kubernetes. A unique aspect of this research was the collection of qualitative insights through open-text questions, resulting in 30 quotes within the report. These quotes powerfully illustrated how Kubernetes certifications led to new jobs and raises, enabled startups to innovate faster, and provided significant value to enterprises. This mixed-methodology approach emphasizes the community-driven nature of LF Research, relying on non-code contributions from individuals to enrich quantitative data.

The Open Source Software Developer Report, sponsored by Intel, addressed a critical gap in understanding the value of open source events for developer journeys. Countering data that suggested developers don't need events (which was specific to product developers), this study empirically demonstrated the unique value for open source software developers. It confirmed that KubeCon is the most widely attended event, followed by Open Source Summit and PyCon. Crucially, a huge number of respondents reported significant value for their career development, finding opportunities to troubleshoot, problem-solve, gain inspiration, learn best practices, and collaborate on the latest threats and solutions. This data enabled Intel to empirically justify the investment in sending open source software developers to events, proving their tangible return beyond mere networking.

A collaborative study with CNCF, LFAI (Linux Foundation AI & Data Foundation), and the Data Foundation explored the State of Generative AI, particularly its convergence with cloud-native technologies. This research found that 84% of organizations reported at least moderate, high, or very high adoption of generative AI. A key insight was that these GenAI models are predominantly built and trained using cloud-based infrastructure, with Kubernetes appearing in 50% of use cases for inference workloads. The study also highlighted the significant value of a foundation model in establishing trust around emerging technologies, providing license continuity, license certainty, the benefits of open governance, and the crucial importance of neutrality with "many eyes" on data sets and models.

Finally, the Cloud Native Security Study, conducted for the first CloudNative SecurityCon in June, revealed surprisingly positive findings. Compared to other industries, the cloud-native community exhibited a strong "optimism around progress in security," attributed to its size, maturity, and a robust culture of collaboration on critical problems. The study explored the extent of cloud-native application development, challenges for vendors in keeping pace with emerging threats, and the number of security incidents. It underscored the immense importance of CNCF best practices and the community's continuous provision of resources for addressing vulnerabilities, indicating a proactive and collaborative approach to security within the cloud-native domain.

Technical Deep Dive

▶ Watch: Where to find all Linux Foundation research (4:50)

The Linux Foundation's research methodology is a blend of quantitative surveying and qualitative inquiry, designed to capture both the broad trends and the nuanced experiences within the open source community. Data collection heavily relies on surveys, distributed to a mailing list of hundreds of thousands of open source contributors, which the speaker noted remains the most effective method for achieving high engagement and qualified sample sizes. These surveys often include open-text questions to gather qualitative insights, providing rich, anecdotal evidence that complements statistical findings, as exemplified by the 30 quotes in the Kubernetes Turns 10 report. This mixed-methodology approach ensures that research is not conducted in a silo but is "by the community and for the community," leveraging the invaluable non-code contributions of individuals who dedicate their time to participate.

A cornerstone of LF Research's operational philosophy is the open availability of its data. All published studies and underlying data sets are accessible on data.world and governed by the Community Data License Agreement 2.0 (CDLA 2.0), an open source license specifically tailored for data sharing. This commitment to transparency and open data aims to empower organizations to explore, validate, and build upon the existing research.

However, a significant technical and methodological challenge highlighted in the talk is the contribution conundrum—the persistent difficulty in measuring the monetary ROI of upstream open source contributions. While the technical value of open source use is clear, quantifying the direct financial return of contribution is elusive. Survey respondents, typically technical contributors, can articulate technical trends and needs but cannot "ascribe the revenue that is derived from the products that they're building." LF Research is actively working on building a framework to better understand this complex ROI, which includes:

• Linking contribution to high-performance teams: Recent findings show 74% of organizations derive high value from employing maintainers, but more data is needed to solidify this link. Qualitative feedback, such as Mark Tate from JP Morgan stating that open source engagement positions them as a "tech company" and attracts top talent, provides crucial context but requires broader empirical validation.
• Identifying barriers to contribution: Regulatory and policy constraints, particularly in highly regulated industries like financial services (e.g., SEC compliance), are known to suppress contribution rates. Understanding the specific nature of these legal barriers is critical.
• Quantifying the cost of maintaining internal forks: A hypothesis from a leading U.S. bank suggested that organizations are spending "way too many hours" maintaining internally forked projects instead of contributing upstream. This represents a significant, unmeasured operational cost. The challenge lies in first identifying all internal forks and then accurately tallying the hours spent by internal teams on their maintenance. This data is crucial for demonstrating the efficiency gains of upstream engagement.
• Measuring monetary opportunities from open source: This includes quantifying savings from avoiding vendor lock-in and leveraging open source alternatives in contract renegotiations. The speaker emphasized that "money drives a lot of decision-making," and data that translates open source value into tangible savings is paramount.
• Assessing the value of non-code contributions: Beyond direct code, contributions like attending conferences, participating in qualitative interviews, providing thought leadership, and engaging in community discussions generate significant brand value, share of voice, and professional development. Quantifying these less tangible benefits—such as brand visibility on LinkedIn, academic citations, or the impact on employee retention—is an ongoing area of research.
• Standardizing adoption metrics: The current reliance on download data is insufficient for truly understanding open source adoption. The community needs better mechanisms to measure how and for what purpose projects are being used within product lines or for internal operations, moving beyond simple download counts.
• Understanding the value of open governance and the foundation model: Recent license changes in some projects have serendipitously illustrated the value of open governance in mitigating "rugpull" risks. Research into how the foundation model fosters confidence, ensures license continuity, and promotes neutrality is crucial for understanding its long-term strategic benefits.

The speaker also touched upon the strategic value of an employee spending time monitoring a project their organization depends on, participating in its technical direction to prevent undesirable shifts. Similarly, the value of allowing developers to contribute bug fixes (e.g., during a critical event like Log4j) not only addresses immediate threats but also impacts team morale and retention. The frustration of being unable to contribute due to internal policies can lead to talent loss, highlighting the need for data to inform more flexible contribution policies.

In essence, the "technical deep dive" of this talk is less about a specific technology and more about the meta-technology of data collection and analysis itself, focusing on the sophisticated methodologies required to quantify the often-intangible benefits and hidden costs within the vast open source ecosystem.

Demo / Proof of Concept

▶ Watch: Kubernetes 10th anniversary study: impact and challenges (6:40)

This session was structured as a Birds of a Feather (BoF) discussion rather than a traditional presentation featuring a technical demonstration or a live proof of concept. Hilary Carter's "demonstration" primarily involved showcasing the existence and accessibility of LF Research's extensive library of 78 unique studies. She highlighted the data.world platform where these studies and their underlying datasets are published, encouraging attendees to explore them. Furthermore, she provided a QR code for subscribing to the LF Research newsletter and directed attendees to linuxfoundation.org/research, serving as a practical guide for accessing the resources available to the community. The interactive nature of the BoF focused on soliciting audience input regarding future research needs, rather than presenting a new technical tool or exploit.

Defensive Implications

▶ Watch: How community contributions fuel research insights (8:00)

The insights and data provided by LF Research offer profound defensive implications for organizations navigating the open source and cloud-native landscape. These implications extend beyond traditional cybersecurity to encompass strategic planning, talent management, and risk mitigation.

First and foremost, the empirical data provides a robust foundation for strategic justification and resource allocation. Organizations can leverage LF Research's findings, such as the 89% cloud-native adoption rate or the ubiquity of Kubernetes, to justify investments in cloud-native infrastructure, training, and specialized teams. The data on the value of open source events, like KubeCon being the most attended and its impact on career development, directly supports budget requests for employee travel and participation. This evidence is crucial for convincing non-technical decision-makers, translating technical necessity into measurable business value and securing the necessary buy-in for open source initiatives.

Secondly, the research offers critical insights into talent management and retention. The finding that 74% of organizations derive high value from employing maintainers underscores the strategic importance of nurturing internal open source contributors. By allowing employees to contribute upstream, organizations not only gain influence over projects they depend on but also foster an environment that attracts and retains top technical talent. The example of JP Morgan positioning itself as a "tech company" through open source engagement highlights how contribution can enhance employer branding. Data on the frustration of developers unable to contribute bug fixes (e.g., during Log4j events) due to internal policies serves as a warning: restrictive policies risk losing valuable team members to organizations that champion open contribution. LF Research's work on the cost of employee turnover versus upskilling further emphasizes the defensive strategy of investing in existing talent through open source engagement.

Thirdly, the research directly informs risk management and operational efficiency. The ongoing effort to quantify the cost of maintaining internal forks of open source projects is a critical defensive measure. Many organizations, particularly in highly regulated industries, fork projects for perceived safety, but this often leads to significant unmeasured operational overhead. By empirically demonstrating the time and resources wasted on maintaining isolated forks versus the benefits of upstream contribution, organizations can make informed decisions to reduce technical debt, improve efficiency, and benefit from community-driven security patches and feature development. Furthermore, understanding the monetary savings from avoiding vendor lock-in by choosing open source solutions provides leverage in contract renegotiations, a direct financial defense strategy.

Fourthly, the Cloud Native Security Study highlights the importance of proactive security postures. The optimism within the cloud-native community regarding security progress, driven by collaboration and the widespread adoption of CNCF best practices, offers a defensive blueprint. Organizations should actively participate in and contribute to open source security initiatives, leveraging the "many eyes" approach to identify and mitigate vulnerabilities more effectively. The value of open governance and the foundation model in establishing trust and mitigating "rugpull" risks from license changes is a crucial defensive mechanism against unforeseen project instability.

Finally, the data empowers organizations to cultivate a stronger corporate culture and brand reputation. The example of BMO Capital Markets featuring its open source contributions in its annual report demonstrates how supporting open source projects and foundations can position an organization as a "good steward." This brand value extends to thought leadership, enhanced visibility on platforms like LinkedIn, and citations in academic papers, all stemming from non-code contributions. Defensively, this positive brand perception can aid in talent acquisition, customer trust, and overall market standing.

In essence, LF Research provides the intelligence needed for organizations to move from reactive problem-solving to proactive, data-driven strategic planning, securing their technological future, retaining their best talent, and enhancing their market position within the dynamic open source ecosystem.

Key Takeaways

• The Contribution Conundrum: Despite open source powering 80-95% of software stacks, the monetary ROI and strategic value of upstream contributions are poorly understood, leading to underinvestment and potential talent loss.
• Empirical Data for Strategic Decisions: LF Research provides a library of 78 studies, freely available under CDLA 2.0, offering data-driven insights into cloud-native adoption (89%), Kubernetes ubiquity, developer journeys, and GenAI trends (84% adoption).
• Value of Maintainers and Non-Code Contributions: 74% of organizations derive high value from employing maintainers, and non-code contributions (e.g., conference attendance, thought leadership) provide significant brand value and foster talent retention.
• Critical Data Gaps: The community urgently needs better frameworks to measure contribution ROI, quantify the hidden costs of maintaining internal forks, and develop standardized metrics for open source project adoption beyond simple downloads.
• Open Governance and Trust: The foundation model and open governance are crucial for establishing trust, ensuring license certainty, mitigating "rugpull" risks, and providing neutrality in the evolving open source landscape.
• Defensive Strategy through Data: Organizations can leverage LF Research data to justify investments, attract and retain talent, improve security posture (e.g., CNCF best practices), mitigate vendor lock-in, and enhance brand reputation.

About the Speaker(s)

Hilary Carter is the Senior Vice President of Research at the Linux Foundation. She founded LF Research four years prior to this talk with the specific mandate to establish a dedicated research function within the organization. Her mission is to provide a collection of data-driven insights and empirical measurements that help fuel strategy and decision-making across the open source community. Carter has a background in conducting studies that measure trends in open source from various perspectives, including developers, industry needs, and broader technological shifts. She emphasizes that all LF Research data is available to the community, encouraging its use to better understand and navigate the open source ecosystem.