Using mixed methods to support educational design decisions in ‘bad data’ contexts

Educational research is often conducted in practical contexts such as schools or higher education institutions and aims to contribute to the development of instructional designs or programs. Consequently, there have been repeated attempts to methodologically integrate the design aspect into educational research leading to concepts such as design-based research. Ideally, design decisions are based on clear evidence, obtained through an analysis of data gathered from the intervention. In actual projects, however, design decisions often have to be taken based on data that is incomplete, ambiguous or not meeting established scientific quality standards. We illustrate and discuss examples of an actual design research project where design decisions had to be taken in such ‘bad data’ contexts. Examples include different mixed methods configurations that helped to increase the interpretability and practical significance of the data. We will discuss methodological variants of ‘mixing’ and triangulation such as combining quantitative and qualitative data or applying different analyses on a dataset. In addition, we discuss limitations regarding the consistency between data, design decisions, and resulting effects