Design and Procedures
Phase I:
This study relies mainly on the advance organizer, which will be used in the next phase as an intervention factor, and since this advance organizer has a great impact on the methodology, this phase will be based on a deep and detailed investigation. Delphi technique is one of the most important tools through which deep data (qualitative) and confirmations (quantitative) can be collected. Also, the Delphi is one of the techniques through which quantitative and/or qualitative data is collected in sequence or convergence in multiple rounds through expert feedback. Based on the advantages and characteristics of the Delphi technique, which are commensurate with what this phase seeks to achieve, it will be used to collect data to answer the first and second questions in this study.
In this technique, five individual steps are employed to validate the results of the 3Ps thematic analysis. The five steps include prepare a knowledge resource nomination worksheet, populating that worksheet with names, first-round contacts, ranking experts by modifications, and inviting experts to the study, where the five-step process will ensure the identification and invitation of the most qualified experts available as outlined by Okoli and Pawlowski (2004). More specifically, Delphi technology will be conducted through two consecutive rounds of validation, and an additional round can be taken as needed, a link to the advance organizer will be sent to all experts. It will include a comprehensive description of the content. The validity of the content is commonly determined by a panel of experts who can judge the representativeness of the elements of the instrument, which will be ensured through a detailed review of each component of the tool by a team of experts.
An initial advance organizer was created based on the result of an extensive review of the research literature through a thematic analysis of hundreds of research references that provided the main elements of the research and its branches, including the relationship between them (Obaid, 2019). Precisely, in this proposed advance organizer as shown in Appendix A, the main elements of the research and its branches are included in the design and highlighted the mechanism of their relationship especially when conducting the research, because the objectives of this study outcomes focus on bridging the knowledge gap that lies in the lack of understanding the relationship between philosophical assumptions and paradigms when conducting research for novice researchers.
In the first part, each instructional design expert will be asked to review the advance organizer and provide comments and feedback on the representation of the applicable multimedia learning principles. —knowing that the experts will investigate the multimedia learning principles that the researcher initially employed and those that relate to still pictures only, as the principles that have to do with sound, animation, and video were excluded. Based on the experts’ comments and feedback provided in the first round, the researcher will make the necessary adjustments to the advance organizer and then re-send it again to them as a second-round with the same nature of questions that were asked in the first round. Also, if a third round is needed, then it will be done as such from the procedure.
In addition to the previous step and meanwhile, an expert panel-reviewed and reliability-tested will be conducted to validate the content of the web-based intervention instrument, including questions, elements, and research scenarios. The expert panel will judge the elements of the instrument and the objectives related to each question; the researcher determined the written objectives and categorized each objective’s behavioral structure according to Bloom’s taxonomy. Since the instrument’s main purpose is to measure researchers’ understanding and performance of philosophical assumptions, paradigms, and praxis, all objectives are classified at the level of understanding and application. Initially, the researcher proposed a list of objectives by examining the literature, interviewed colleagues, considered personal experiences in conducting research, and analyzed existing studies related to students’ understanding of similar contexts, these objectives are found in Appendix B. This list will be subject to validation by an expert panel in the content of the subject who has experience teaching research courses or supervising graduate students.
In both validation phases, whether Delphi technique or expert panel, each member of the validation team will have the opportunity to provide feedback through specific processes. In general, the aim will be to reach a majority agreement on the use of each element of the content that will be validated. Accordingly, feedback forms are designed for each group of validation, and validation rubrics are developed in an online platform that allows experts to access the instrument
independently to provide feedback on each element, this instrument is found in Appendix C. It should be noted that both Delphi technique and research expert panel are similar in that they involve experts for the review process. However, according to Green (2014), Delphi technique involves a group process of surveying and collecting opinions from experts of specific subjects, while expert panel involves the evaluation of specialized input/opinion of experts – addition of expert recommendations in specific subjects (Waltz et al., 2015). The panel of experts will be shared with a link that includes the contents of the post-test, which includes research scenarios and closed questions, and open questions that will be used in the interviews. For the post-test, there will be two scenarios for quantitative and qualitative research, each of which reflects the nature of a particular research approach, as well as a series of questions related to each scenario. The task of the research experts will be to examine each scenario and ensure that it is consistent with the proposed philosophical paradigm and research approach, and the final section is to validate the questions related to the interviews.
Both members of the two groups of experts will be asked to clarify whether each element should be included in one of the three definitive responses: “Accept as described”, “Accept changes” or “Delete item” if the answer is “Accept changes” you will be asked to present the proposed change, and if they Answer “Delete an item”, they will be asked to explain this choice.
A validation rubric that will be used in the Delphi technique process by the instructional design experts to validate the 3Ps advance organizer is in Appendix D and includes learning objectives for the post-test elements shown in Appendix B. Also, the validation rubric that will be used by a panel of experts to validate the content of the post-test and interview questions is found in Appendix E.
The researcher has developed the validation instrument through Qualtrics where experts will be able to access the rubric and instrument through a computer or mobile devices, in addition to that, Qualtrics has many interactive features that allow the researcher to interact with the experts professionally, effectively and quickly. As technology advances, more effective options emerge than the Delphi procedure’s traditional method, as Delphi stages can be completed in a few weeks instead of months (Keeney, Hasson, & McKenna, 2011). However, the researcher realizes that the issue of time depends on several factors, the most important of which is strict adherence to procedural steps, a complete readiness to face challenges and expected risks, and the provision of alternative solutions when needed.
Phase II:
This phase begins after the completion of the previous phase, where the researcher distributes the instrument electronically to homogeneous convenience samples through various means. The instrument has three main sections: demographic, intervention, and post-test. The experimental group will access all the sections, but the control sample will only reach the first and third sections. All participants will be required to complete the consent form (Appendix G) and the necessary instructions (Appendix F) at the beginning of the instrument. Participation will be voluntary, and there will be no requests if they choose not to complete the study. The researcher took into consideration the impact of incentives to join this study, so each participant gets a chance to win a gift card worth $ 25 out of 20 cards.
In the first section, no personal data will be collected from the participants to maintain the confidentiality and privacy of them. Upon reaching the second section, the system will electronically assign the participants into two groups: a treatment group and a control group. The treatment group will be able to access the 3Ps advance regulator and read the instructions accompanying the intervention, and that will take about 10 minutes. After that, they will move to the last section, which is the post-test. Concerning the control group, after they finish the demographic section, they will take the post-test directly without access to the intervention, as both groups will be subject to the same elements of the post-test.
The post-test takes about 20 minutes to complete, which means that the intervention group will take about 30 minutes to complete all sections. In the last step of the instrument, all participants will be invited for an individual interview. A $ 25 gift card will be given to each interviewee chosen as an appreciation of the time they will spend, knowing that this study seeks to interview eight researchers, four of them novices (two from the experimental group and two from the control group), and four expert researchers (two from the experimental group and two from the control group). Each interview will take about an hour. Therefore, after the participants agree to conduct the interview, they will be transferred to another link to provide their contact information. However, their contact information that they will provide will not be linked to the information they provided in the instrument in order to maintain their confidentiality. The Figure 11 below shows the experimental and control sampling process up to the selection of the interview sample.
After obtaining a sufficient number of volunteers to conduct the interview, they will be randomly assigned through a computer-based “random number generators,” then the interview invitation with the required instructions will be sent via email to each person chosen. After receiving their response with approval, the appropriate time for both parties will be determined for an online interview. Questions that will be discussed in the interview for each individual, and the interview will be audio recorded for analysis purposes, the interview questions are found in Appendix H. All records will be kept in a secret file that no one other than the researcher can access, and all data will be destroyed after two years.