David Spielmann
Last Name
Spielmann
First name
David
Email
david.spielmann@unisg.ch
ORCID
Phone
+41 71 224 32 75
8 results
Now showing 1 - 8 of 8
- Some of the metrics are blocked by yourconsent settings
Item type:Publication, Automated Infrastructure as Code Program Testing(2024); ; Automated, "Infrastructure as Code (IaC) enables efficient deployment and operation, which are crucial to releasing software quickly. As setups can be complex, developers implement IaC programs in general-purpose programming languages like TypeScript and Python, using PL-IaC solutions like Pulumi and AWS CDK. The reliability of such IaC programs is even more relevant than in traditional software because a bug in IaC impacts the whole system. Yet, even though testing is a standard development practice, it is rarely used for IaC programs. For instance, in August 2022, less than 1 % of the public Pulumi IaC programs on GitHub implemented tests. Available IaC program testing techniques severely limit the development velocity or require much development effort. To solve these issues, we propose Automated Configuration Testing (ACT), a methodology to test IaC programs in many configurations quickly and with low effort. ACT automatically mocks all resource definitions in the IaC program and uses generator and oracle plugins for test generation and validation. We implement ACT in ProTI, a testing tool for Pulumi TypeScript with a type-based generator and oracle, and support for application specifications. Our evaluation with 6 081 programs from GitHub and artificial benchmarks shows that ProTI can directly be applied to existing IaC programs, quickly finds bugs where current techniques are infeasible, and enables reusing existing generators and oracles thanks to its pluggable architecture.Type:journal articleJournal:IEEE Transactions on Software EngineeringScopus© Citations 13 - Some of the metrics are blocked by yourconsent settings
Item type:Publication, Metamorphic Testing for Infrastructure-as-Code Engines(2026); ; ; Meier, RolandInfrastructure-as-Code (IaC) engines, such as Terraform, OpenTofu, and Pulumi, automate the provisioning and management of cloud resources. They parse IaC specifications and orchestrate the required actions, making them the backbone of modern clouds, and critical to the reliability of both the underlying infrastructure and the software that depends on it. Despite this importance, this class of systems has received little attention: prior work largely targets the correctness of IaC programs rather than the IaC engines themselves. Existing test suites rely on manually written oracles and struggle to expose faults that manifest across multiple executions, leaving a significant reliability gap. We present EMIaC, a metamorphic testing framework for IaC engines. EMIaC defines metamorphic relations as graph-based transformations of IaC programs and checks invariants across executions of the original and transformed programs. A central novelty is our use of e-graphs in software testing, as both a test-input generator and an equivalence oracle. E-graphs compactly represent program equivalences, enabling the systematic generation of large spaces of equivalent IaC programs. To ground these relations, we analyze 43,593 real-world Terraform programs and show that IaC dependency graphs are typically small and sparse, making e-graphs a natural fit. Evaluating EMIaC on Pulumi, Terraform, and OpenTofu, we show that it complements existing test suites by exercising engine-critical code paths and covering 98 previously untested statements in Terraform and 1,313 in Pulumi. EMIaC also uncovers previously unknown issues in all three test suites, improving their adequacy. Three test cases have been merged into Terraform's main branch, and Pulumi has merged a specification fix.Type:conference paper - Some of the metrics are blocked by yourconsent settings
Item type:Publication, TerraDS: A Dataset for Terraform HCL Programs(2025-04-28); ; - Some of the metrics are blocked by yourconsent settings
Item type:Publication, The PIPr Dataset of Public Infrastructure as Code Programs(2024); ; ; With Programming Languages Infrastructure as Code (PL-IaC), developers implement IaC programs in popular imperative programming languages like Python and Typescript. Such programs generate the declarative target state of the deployment, i.e., they describe what to set up, not how to set it up. Despite the popularity of PL-IaC, which has grown more than ten times from 2020 to 2023, we know little about how developers apply it and how IaC programs differ from other software. Such knowledge is essential to effectively use existing software engineering techniques and develop new ones for PL-IaC. To shed light on PL-IaC in practice, we present PIPr, the first systematic PL-IaC dataset. PIPr is based on 37 712 public IaC programs on GitHub from August 2022 and includes initial analyses, assessing the programming languages, testing techniques, and licenses of the IaC programs. Beyond the metadata and analysis results of all IaC programs, PIPr contains the code of all 15 504 IaC programs whose licenses permit redistribution. PIPr sets the ground for future in-depth investigations on PL-IaC in practice.Type:conference paperScopus© Citations 9 - Some of the metrics are blocked by yourconsent settings
Item type:Publication, RDS: FPGA Routing Delay Sensors for Effective Remote Power Analysis Attacks(Universitatsbibliothek der Ruhr-Universitat Bochum, 2023-03-06); Scopus© Citations 33 - Some of the metrics are blocked by yourconsent settings
Item type:Publication, Unleashing the Giants: Enabling Advanced Testing for Infrastructure as Code(2024); ; Infrastructure as Code (IaC) programs are written in imperative programming languages like Python or TypeScript while declaratively defining the target state of software deployments, which the IaC solution then sets up, e.g., Pulumi and AWS CDK. Through a repository mining study and analysis, we noticed that testing IaC programs poses a dilemma: current techniques are either slow and expensive or require prohibitively high development effort. To solve this issue, we introduce Automated Configuration Testing (ACT), enabling efficient testing with low development effort. ACT automates the tedious aspects of unit testing IaC programs and is extensible through a plugin system for test generators and oracles. ACT is already effective with simple type-based plugins, and leveraging existing giants, i.e., advanced test generation and oracle techniques, in new plugins will further boost its effectiveness.Type:conference posterScopus© Citations 1 - Some of the metrics are blocked by yourconsent settings
Item type:Publication, Extensible Testing for Infrastructure as Code(2023); ; Developers automate deployments with Programming Languages Infrastructure as Code (PL-IaC) by implementing IaC programs in popular languages like TypeScript and Python. Yet, systematic testing-well established for high-velocity software development-is rarely applied to IaC programs because IaC testing techniques are either slow or require extensive development e ort. To solve this dilemma, we develop ProTI, a novel IaC unit testing approach, and implement it for Pulumi TypeScript. Our preliminary experiments with simple type-based test case generators and oracles show that ProTI can nd bugs reliably in a short time, often without writing any additional testing code. ProTI's extensible plugin architecture allows combining, adopting, and experimenting with new approaches, opening the discussion about novel generators and oracles for e cient IaC testing.Type:conference posterScopus© Citations 1 - Some of the metrics are blocked by yourconsent settings
Item type:Publication, Creed for Speed: Comprehensive Infrastructure as Code Testing(2023-10-24); ; With Programming Languages Infrastructure as Code (PL-IaC), developers implement imperative IaC programs in one of many general-purpose programming languages, e.g., TypeScript, Python, or Go, to declaratively describe deployments. Using these languages provides access to quality assurance techniques and tools developed for traditional software; however, programmers routinely rely on prohibitively slow integration testing-if they test at all. As a result, even simple bugs are found late, tremendously slowing down the development process. To improve the velocity of PL-IaC development, we propose ProTI, an automated unit testing approach that quickly tests PL-IaC programs in many different configurations. ProTI mocks all cloud resources, replacing them with pluggable oracles that validate all resources' configurations and a generator for realistic test inputs. We implemented ProTI for Pulumi TypeScript with simple generator and oracle plugins. Our experience of testing with ProTI encourages the exploration of more sophisticated oracles and generators, leading to the early detection of more bugs. ProTI enables programmers to rapidly prototype, explore, and plug in new oracles and generators for efficient PL-IaC program testing.Type:conference speech