Buildpacks: Pragmatic Solutions To Quick and Secure Image Builds - Juan Bustamante & Aidan Delaney

Buildpacks: Pragmatic Solutions To Quick and Secure Image Builds

Speakers: Juan Bustamante, Platform Maintainer, DB Accessed; Aidan Delaney, Learning Maintainer, Bloomberg

Conference: KubeCon EU

YouTube: https://www.youtube.com/watch?v=Eb9AweCazi8

Overview

In this insightful KubeCon EU presentation, Juan Bustamante and Aidan Delaney, both maintainers of the Cloud Native Buildpacks project, illuminated how this open-source initiative offers a transformative approach to building container images. The talk focused on its pragmatic solutions for achieving both rapid and secure image builds, particularly in complex, multi-language enterprise environments. They emphasized how Buildpacks abstract away the complexities of Dockerfiles, offering a standardized, opinionated, and highly efficient pathway from application source code to production-ready OCI images.

The core message of the session resonated with platform operators, application developers, and even buildpack authors: Buildpacks provide a robust framework that enhances developer experience while simultaneously bolstering an organization's software supply chain security posture. By segmenting the audience into these three key personas, Bustamante and Delaney effectively demonstrated the project's broad utility and the specific benefits it delivers to each group, from streamlined development workflows to granular control over production environments and critical security features like Software Bill of Materials (SBOMs) and rebase capabilities.

The talk underscored the critical importance of secure and efficient image building in the cloud-native landscape, where speed of deployment often clashes with stringent security requirements. Cloud Native Buildpacks emerge as a powerful tool to bridge this gap, offering a maintainable, auditable, and inherently more secure method for container image creation. This presentation served as a comprehensive update on the project's evolution, highlighting recent features, community contributions, and its strategic roadmap for future development.

Background

▶ Watch: Introduction to Cloud Native Buildpacks and project overview (0:00)

The proliferation of containerization, particularly with Docker and Kubernetes, has revolutionized how applications are developed, deployed, and managed. However, this shift introduced new challenges, especially for large organizations managing hundreds or thousands of workloads across diverse development teams and programming languages. The traditional method of building container images using Dockerfiles often becomes a significant operational burden. Each development team might write their own Dockerfiles, leading to inconsistencies, security vulnerabilities due to varying base images or build practices, and a lack of standardization across an organization's container fleet. Platform operators frequently find themselves grappling with "Dockerfiles being a nightmare" due to the sheer volume and variability.

This problem statement formed the genesis of projects like Cloud Native Buildpacks. Originating from Heroku's internal build system, the concept of buildpacks was designed to automatically detect an application's language and framework, then apply a predefined set of build steps to produce a runnable artifact, all without requiring explicit Dockerfile instructions from the developer. This approach inherently enforces consistency and best practices.

The Cloud Native Buildpacks project, now a CNCF (Cloud Native Computing Foundation) initiative, extends this philosophy to the OCI (Open Container Initiative) image specification. It aims to provide a higher-level abstraction for building OCI-compliant images from source code. By defining a clear boundary between application code and underlying infrastructure layers, Buildpacks enable powerful features crucial for modern cloud-native development, such as efficient caching, rapid rebuilds, and robust security patching through image rebasing. The project maintains a formal specification and offers reference implementations and tools like pack and kpack, alongside various community-driven builders like Paketo, Heroku, and Google, which provide pre-configured sets of buildpacks for different languages and frameworks. This ecosystem addresses the critical need for a more standardized, secure, and developer-friendly approach to container image construction.

Key Findings

▶ Watch: Introduction to the powerful rebase feature (4:00)

The Cloud Native Buildpacks project delivers several key findings and contributions that significantly enhance the container image building process:

• Dockerfile-Agnostic Image Creation: A primary finding is the ability to generate production-ready OCI images directly from source code without the need for a Dockerfile. This dramatically simplifies the developer experience and centralizes image build logic.
• Standardized Layering for Optimization and Security: Buildpacks enforce a well-defined, organized set of layers within the OCI image: run image (OS), runtime, dependencies, and application layers. This clear separation facilitates fast rebuilds by only updating changed layers and enables critical security features like rebase.
• Automated Language Detection and Build Processes: The system automatically detects the programming language (e.g., Go, Java, Python, Node.js, Rust) and applies the appropriate buildpacks, abstracting away complex build configurations.
• Efficient Rebase for CVE Remediation: A standout feature is the capability to rebase images. If a vulnerability (CVE) is discovered in a base image (OS or runtime layer), Buildpacks allow platform operators to update the base image in the registry and rebase existing application images onto the new secure base image without a full rebuild. This is a registry-only operation, making security patching incredibly fast and efficient.
• Built-in Software Bill of Materials (SBOM) Generation: All images generated by Cloud Native Buildpacks include an SBOM, providing a comprehensive list of all components, dependencies, and their versions. This is crucial for supply chain security, enabling precise identification of affected images when CVEs are announced.
• Enhanced Software Supply Chain Security: Through integrations with tools like Cosign for image signing and In-toto/Witness for provenance certificates, and by achieving Salsa Level 3 compliance with kpack, Buildpacks offer a robust framework for securing the software supply chain from source to deployment.
• Platform Operator Control and Customization: Buildpacks empower platform operators with granular control over base images, tool versions (e.g., specific JDK or Python versions), supported language families, and build environment variables (e.g., forcing the use of internal dependency mirrors). This ensures organizational policy compliance and enhances security.
• Multi-Architecture Support: Modern buildpacks, especially those from Heroku, now offer multi-architecture support, capable of building images for AMD64 and ARM64 architectures, meeting the demands of diverse deployment environments.
• Community-Driven Innovation: The project thrives on an active community, evidenced by continuous improvements from downstream providers like Paketo (dependency mirrors, UBI/Ubuntu core builders, clearer maintainer status) and Heroku (new .NET buildpack, Poetry support, Debian package installation, multi-arch support).

These findings collectively position Cloud Native Buildpacks as a sophisticated yet practical solution for modern container image management, directly addressing the complexities of speed, security, and scalability in cloud-native ecosystems.

Technical Deep Dive

▶ Watch: Demo: Building a polyglot monorepo application (4:30)

Cloud Native Buildpacks operate on a fundamental principle: transforming application source code into a production-ready OCI image without requiring a Dockerfile. This is achieved through a well-defined, multi-stage process orchestrated by a builder and a series of buildpacks.

At its core, the process begins when a developer or automated system invokes a tool like the pack CLI (for local builds) or the kpack Kubernetes operator (for automated, cluster-based builds). The pack command, such as pack build example-app --builder paketo-builder/jammy-base, points to the application's source code and specifies a builder. A builder is itself an OCI image containing a build image (where the application is built), a run image (the base for the final application image), and a collection of buildpacks.

Upon invocation, the builder first inspects the application's source code. This inspection phase, often handled by a detect buildpack, identifies the programming language and framework (e.g., Go, Java Spring Boot, Node.js, Python). Once detected, the relevant buildpacks within the builder are executed. These buildpacks perform specific tasks:

1. Dependency Resolution: Fetching and installing language-specific dependencies (e.g., Maven dependencies for Java, npm packages for Node.js, pip packages for Python).
2. Compilation/Transpilation: Compiling source code (e.g., Go, Java) or transpiling (e.g., TypeScript to JavaScript).
3. Application Assembly: Packaging the compiled application artifacts.

The output OCI image is meticulously structured into distinct layers, ensuring optimal caching, security, and rebase capabilities:

• Run Image Layer: Contains the operating system (e.g., Ubuntu Jammy, UBI, Ubuntu Core) and fundamental libraries required for the application to execute.
• Runtime Layer: Includes the specific language runtime (e.g., JDK for Java, Node.js engine, Python interpreter).
• Dependencies Layer: Holds all external libraries and frameworks required by the application.
• Application Layer: Contains the compiled or interpreted application code itself.

Crucially, these layers maintain "well-defined boundaries." This separation allows for granular updates. If only the application code changes, only the application layer needs to be rebuilt and pushed. The preceding layers can be cached or reused, leading to significantly faster rebuilds.

For platform operators, Buildpacks offer powerful mechanisms for control and standardization:

• Custom Builders: Operators can create custom builders by defining a builder.toml file. This allows them to specify their own build and run images, ensuring all application images in their organization are based on approved, hardened base images. This is vital for enforcing security policies and compliance.
• Version Control for Toolchains: Custom builders also enable control over the versions of language runtimes and tools. For instance, an organization might standardize on Python 3.12 in production, even if Python 3.13 is the latest, and can configure their builder to only use the approved version of the Paketo or Heroku Python buildpack.
• Language Family Restrictions: Operators can curate which language stacks are supported in production by including only specific buildpacks in their custom builder (e.g., only .NET and Node.js).
• Controlled Build Environment: To prevent supply chain attacks and ensure dependency integrity, platform operators can set non-overridable environment variables in their custom builders. For example, a PIP_INDEX_URL can be configured to point exclusively to an internal dependency mirror, preventing developers from pulling packages directly from public registries like PyPI or npm. This ensures that all dependencies are vetted and hosted internally.

Buildpacks also include advanced features for buildpack authors, such as extensions, which have recently gone GA, providing more flexibility in customizing build behaviors. Furthermore, support for insecure registries streamlines the development loop for buildpack authors testing locally.

The project seamlessly integrates with common CI/CD workflows, providing documentation and examples for CircleCI (via an orb), GitHub Actions, GitLab workflows, and Tekton. While Jenkins integration is highly flexible due to its pipeline customization, pack can be "parachuted in" to any Jenkins pipeline.

Finally, the output images are not just small and optimized; they are also secure. Each image is shipped with a Software Bill of Materials (SBOM), which precisely details all included components. This, combined with features like Salsa Level 3 compliance in kpack, native Cosign support for image signing, and integration with In-toto tooling like Witness for generating provenance certificates, provides a comprehensive security story, crucial for highly regulated environments. The small, focused layers also inherently reduce the attack surface of the final image.

Demo / Proof of Concept

▶ Watch: Demo: Building a Python application with simple commands (8:30)

Juan Bustamante presented three concise, pre-recorded demonstrations showcasing the simplicity and power of Cloud Native Buildpacks, all without touching a Dockerfile. The core command across all demos remained pack build, illustrating the consistent interface for developers regardless of the underlying application language.

1. Monorepo Application (Go Backend, NodeJS Frontend):

• Scenario: A single repository containing a Go backend and a TypeScript NodeJS frontend.
• Process: Juan executed pack build example-app --builder paketo-builder/jammy-base. The pack CLI automatically detected both Go and TypeScript components. The Paketo builder, utilizing its comprehensive collection of buildpacks, compiled the Go backend and processed the NodeJS frontend.
• Output: A single OCI image was produced, containing all necessary layers for both Go and NodeJS runtimes, dependencies, and application code.
• Execution: From this single image, two separate containers were launched using docker run, each with a different entry point: one for the Go backend and another for the NodeJS frontend.
• Highlight: This demo powerfully demonstrated the ability of Buildpacks to handle multi-language applications within a single build process, producing a single, optimized image capable of running multiple services. The web application successfully displayed a message originating from the Go backend, confirming the successful deployment of both components from the unified image.

1. Spring Boot Application (Java):

• Scenario: A common Java Spring Boot application, "Pet Clinic."
• Process: Juan simply ran pack build pet-clinic. In this case, he omitted the --builder flag because his local pack CLI was configured to use the Paketo builder as the default. Buildpacks inspected the code, recognized it as a Java application, and applied the appropriate Java buildpacks from the Paketo collection.
• Output: An optimized OCI image for the Spring Boot application.
• Execution: The image was run using docker run, and the "Pet Clinic" web application was successfully accessed in the browser.
• Highlight: This showcased the ease of use for developers once the environment is configured. A single, intuitive command is all that's needed to build a complex Java application into a production-ready image.

1. Python Application (AI-Generated):

• Scenario: A simple Python application, humorously noted as "AI-generated."
• Process: Following the pattern, Juan used pack build python-app with an additional flag to specify a particular Python version. The Buildpacks automatically handled the Python dependencies and application packaging.
• Output: A runnable OCI image for the Python application.
• Execution: The Python application was successfully run and accessed.
• Highlight: This reinforced the consistency and language-agnostic nature of the pack build command, emphasizing that regardless of the language, the build process remains straightforward and Dockerfile-free.

Across all demonstrations, the core takeaway was the elimination of Dockerfiles, the automatic optimization of application images for production (a benefit inherited from Paketo/Heroku buildpack collections), and the consistent, developer-friendly interface provided by the pack CLI. These demos effectively illustrated how Cloud Native Buildpacks streamline the containerization process, allowing developers to focus on their code rather than on intricate image build configurations.

Defensive Implications

▶ Watch: Summary of optimized, production-ready images (9:30)

Cloud Native Buildpacks offer a significant defensive posture for organizations managing containerized applications, primarily by addressing critical aspects of software supply chain security and operational efficiency. The project's design inherently provides several layers of defense against common vulnerabilities and attacks:

1. Rapid CVE Remediation via Image Rebase: This is perhaps one of the most powerful defensive features. When a Common Vulnerability and Exposure (CVE) is discovered in a base operating system (OS) or language runtime, traditional Dockerfile-based approaches require rebuilding every affected application image from scratch. Buildpacks, however, allow for rebasing. Platform operators can update the underlying run image (containing the OS) or runtime layer (e.g., JDK) in their registry. Then, existing application images can be rebased onto this new, patched base image through a simple registry-only operation, without needing to rebuild the application or dependency layers. This drastically reduces the time and effort required to patch critical vulnerabilities across an entire fleet of applications, minimizing exposure windows.

1. Comprehensive Software Bill of Materials (SBOMs): Every image produced by Cloud Native Buildpacks includes an SBOM. This detailed manifest lists all software components, libraries, and their versions within the image. From a defensive standpoint, SBOMs are invaluable:

• Targeted Vulnerability Scanning: When a new CVE is announced, security teams can quickly query their SBOM database to identify precisely which images and applications are affected, eliminating guesswork.
• Auditing and Compliance: SBOMs provide a clear, auditable record of all components, essential for compliance with regulations that mandate software transparency.
• Operational Visibility: Combined with deployment information, SBOMs can tell operators exactly "where the images with those CVEs are deployed on what pods in which clusters," enabling precise remediation.

1. Strong Software Supply Chain Security: Buildpacks are designed with supply chain security in mind:

• Salsa Level 3 Compliance: kpack, the Kubernetes operator for Buildpacks, achieves Salsa Level 3 compliance. Salsa (Supply-chain Levels for Software Artifacts) is a framework for ensuring the integrity of software artifacts. Level 3 signifies robust protections, including verified source, authenticated builds, and non-falsifiable provenance.
• Image Signing with Cosign: Buildpacks integrate natively with Cosign, a tool for signing and verifying container images. This allows organizations to cryptographically sign their images, ensuring that only trusted, verified images are deployed and that images haven't been tampered with.
• Provenance Certificates with In-toto/Witness: Integration with In-toto tooling like Witness enables the generation of provenance certificates. These certificates provide an unalterable record of how an image was built, including the exact source code, build environment, and tools used, offering a strong defense against build-time tampering.

1. Platform Operator Control over Build Environment: The ability for platform operators to create custom builders provides critical defensive controls:

• Hardened Base Images: Operators can enforce the use of organization-approved, hardened base images (e.g., specific Ubuntu Core or UBI images) for all production builds, reducing the initial attack surface.
• Controlled Tool Versions: By locking down specific versions of language runtimes (e.g., JDKs, Python interpreters), operators can prevent the introduction of vulnerabilities associated with unvetted or outdated toolchains.
• Internal Dependency Mirrors: Enforcing the use of internal, vetted dependency mirrors (via non-overridable environment variables like PIP_INDEX_URL or NPM_CONFIG_REGISTRY) prevents developers from pulling potentially malicious or unvetted packages directly from public repositories, mitigating dependency confusion attacks and other supply chain risks.
• Restricted Language Stacks: Operators can limit the languages and frameworks supported in production, simplifying the security auditing process and reducing the overall attack surface.

1. Small, Focused Layers: The well-defined, granular layering of Buildpack-generated images results in smaller, more focused layers. This reduces the overall size of the image, which in turn reduces the potential attack surface and speeds up vulnerability scanning.

In summary, Cloud Native Buildpacks empower organizations to adopt a proactive and systematic approach to container security. By standardizing the build process, providing granular control to platform operators, and integrating essential supply chain security features, Buildpacks significantly strengthen an organization's defensive capabilities against a wide array of modern threats.

Key Takeaways

• Dockerfile Elimination & Developer Experience: Cloud Native Buildpacks enable the creation of production-ready OCI images directly from source code, completely abstracting away Dockerfiles and simplifying the developer experience across various languages like Go, Java, Python, and Node.js.
• Rapid & Secure CVE Remediation with Rebase: The project's unique layering structure allows for fast and efficient security patching through "rebase." If a CVE is found in a base OS or runtime, images can be updated by simply swapping out the base layer in the registry, avoiding full application rebuilds and drastically reducing exposure time.
• Robust Software Supply Chain Security: Buildpacks natively generate Software Bill of Materials (SBOMs), achieve Salsa Level 3 compliance with kpack, and integrate with Cosign for image signing and In-toto/Witness for provenance, providing comprehensive assurance and traceability for container images.
• Empowering Platform Operators with Granular Control: Platform operators gain extensive control over base images, language tool versions, supported frameworks, and build environments (e.g., enforcing internal dependency mirrors), ensuring consistency, compliance, and enhanced security across the organization's container fleet.
• Optimized & Multi-Architecture Images: Buildpack-generated images are optimized for production with small, well-defined layers for fast rebuilds and reduced attack surface. Modern buildpacks also offer multi-architecture support (AMD64, ARM64), catering to diverse deployment needs.
• Active Community & Downstream Innovation: The Cloud Native Buildpacks ecosystem is vibrant, with continuous development from projects like Paketo (adding dependency mirrors, new base images like UBI/Ubuntu Core) and Heroku (new .NET support, Poetry, Debian package installation), ensuring a rich set of features and ongoing improvements.

About the Speaker(s)

Juan Bustamante is a platform maintainer for the Cloud Native Buildpacks project. He also works for the company DB Accessed, bringing his expertise in platform operations and cloud-native technologies to the community. His contributions focus on ensuring the project provides pragmatic solutions for secure and efficient image builds.

Aidan Delaney is also a maintainer for the Cloud Native Buildpacks project, serving as a learning maintainer. He is employed by Bloomberg, where he applies his deep understanding of software development and containerization to help guide the project's evolution and support its users. Aidan is a vocal advocate for the project's benefits for platform operators and application authors.