The Design-Based Research Collective (2003) proposes that design research, which blends empirical educational research with theory-driven design of learning environments, is an important methodology for understanding how, when, and why educational innovations work in practice. They argue the following:
Design is central in efforts to foster learning, create usable knowledge, and advance theories of learning and teaching in complex settings. Design-based research also may contribute to the growth of human capacity for subsequent educational reform (p. 5).
Design research seeks to increase the impact, transfer and translation of education research into improved practice and emphasises the need for theory building that guides, informs and improves both practice and research in educational contexts (Anderson & Shattuck, 2012). Key features of design research studies include being:
Adjustments to practice are also a key characteristic during the testing of educational ideas for improving learning (Bakker, 2018). Cobb et al. (2003) emphasise that design experiments are not just about ‘what works’ but are conducted to test and develop theories, which is essential for long-term educational improvement. This is in contrast to prominent trends of experimental studies in educational contexts, which do not align with the research worldview that this study takes.
There are multiple reasons for the selection of design research as an ideal approach for this study. These include the complexity of the study that aims to both research and improve technology-enhanced TPD, the use of multiple iterations and multiple sources of data for analysis, the affordance of the part-time nature of the PhD since iterations can last longer than in a full-time PhD, and the focus on scalability which is a central question that this research seeks to explore (i.e. a design theory seeks to suggest how designs may be adapted to new circumstances). In addition, the cycles of designing, trialling and refining activities are central to this study as well as the design and contextualisation of the materials. There is significant conceptual alignment between design research, mechanical MOOCs and building communities of practice through participatory design and contextualisation of learning content and resources. This also includes the need to fail early, often and report on these failures (O’Neill, 2012), which is highly relevant for this particular study. Having in-depth explanations of failures and why the design should have worked enhances reproducibility and scalability so that good ideas do not continue to be implemented poorly. The use of meta, pragmatic and specific design principles also seeks to mitigate this. While the logistics of a design experiment (i.e. aims, timeline and budget) can be considered ambitious, as a part-time PhD study this research benefited from the length of time available for the production of a quality website course structure with no cost and the longer spaces between iterations allowed more time for change to manifest.
Design-based research is not without its challenges and limitations, however, which are important to note and reflect on. These include:
Following consideration of the strengths and weaknesses of a design-based approach, this study prioritised continuous reflection from both the study participants and myself as a researcher. In particular, the study presented an opportunity to reflect on research practice as part of an ongoing process. Research practice, like any practice, requires continuous reflection and articulation, which was incorporated into the research design, building on reflections from my registration report and reflexive accounts during and following each iteration of the intervention.
From the outset of this study, the theoretical framework was intentionally provisional and subject to be changed and informed by emerging data from each iteration of the research. This was a process of hypothesising as opposed to proving cause and effect. Sandoval (2004) refers to this process as developing conjectures, which sit between educational theory and practice, that the designed learning environment embodies. Bakker (2018) describes these as design principles, which echo Sandoval’s high-level conjectures and provide “a general idea of how particular theoretical characteristics when embodied in a design lead to certain mediating processes that then support desirable outcomes” (p. 54). Gutiérrez and Penuel (2014) argue that the use of design principles enhances the validity of the findings through a focus not only on the rigour of the research process, but also their practicality and use. Conjectures / design principles are expected to be refined over time.
My initial and fundamental conjecture for this research study (as articulated in Chapter 3.1 as the theory of change) is that if teachers build a dialogue with one another within an online TPD programme, through reflecting together, sharing resources and supporting one another, they will form a sustainable community of practice, which will enhance their use of dialogic teaching in the classroom. In line with the aims of design research, this research seeks to develop theories about both the process of learning within the context of teacher professional development provided in an online environment and the means that are designed to support that learning (i.e. online course activities, discussion forum, etc.).
Design principles include the what and how behind the proposed intervention, but also offer theoretical explanations to justify these claims (Bakker, 2018). They can be broken down into meta-principles, pragmatic principles and specific principles. The design principles for this study were developed during the first two phases of the research and then used in the first iteration of the intervention. Once they were refined during the first two phases of the research, they were adjusted or removed, as a process of validation and generalisation. Generalisation works through the iterative process of assessing design principles in practice rather than a methodological decision at the outset as in randomised control trials, for example (Ibid). This research also takes the stance that there is a role for intuition in a design-based research study. Where conjectures were not based on established theory in re-iterating the course structure and content, this was documented.
Ultimately, this process yielded a set of design principles for sustainability and scalability of the approach, and insights into the mechanisms by which it supports (or indeed, does not support) practitioners. Chapter 5 presents the iterations of design principles throughout and following the trials of the intervention.