Special Research I
Session 1: (1) 7-Step Research Model (Chapter 1)
[Let's Use the 7-Step Research Model]
Have you understood Chapter 1 of the textbook? Let us look through to see how your senior student (say, Bob: imaginary person) have applied the 7-step research model to conduct his research.
1.Selecting a Topic
“1. Instructional analysis and proposal of designing guidelines concerning e-Learning materials for adults”
My job is providing e-Learning materials for adults. I have been looking for some indicators from a perspective of raising the materials' effectiveness. I am interested in this research theme because it is directly related to my job, and I will be able to utilize the results immediately. It will belong to the field of corporate training. The research will deal with the development of e-Learning materials for business education.
2. Identifying the Research Problem
I see that the theme should be more specific. Referring to the example on p.2 of the textbook, the framework for my research will likely be the following:
- Who: Designers of e-Learning materials in my company (it may be too specific if I say “in my company”...)
- When: when designing e-Learning material for business education
- Operation/conditions/intervention: by using designing guidelines
- Expected outcomes: able to formulate more effective instructional material design (if possible, faster; I may be expecting too much...)
3. Conducting a Literature Search
ERIC? When I searched for ERIC, I found both “the International Education Resource & Innovation Center (ERIC)” and “the Education Resources Information Center (ＥＲＩＣ).” It's the latter, which has a collection covers 11 million bibliographic records and 110 thousand full text materials. Wow, it is too much for me. The Educator's Reference Desk, third in the search result list, is also very interesting, but I will look at it later. Let's type in “e-Learning” in the search engine. I found 675 results for it. It is fewer than I thought. “Search within Results” seems to refine my search. I want to look up in the search engine “guideline.” The number of search results for “e-Learning” AND “guideline” is “0”. Probably something is wrong with using “guideline.” Maybe the line is not working well. It takes too long. Let's try another way. (I gave up….for now)
Members of Japan Society for Educational Technology (JSET) can retrieve its past journals and collected papers presented in the national conference and other literature as a member privilege. They can access such literature from the members' site on the JSET's Website.
NII Scholarly and Academic Information Portal (GeNii) provided by the National Institute of Informatics (NII) (service ended on March 31, 2014) is a service that retrieves not only academic papers but also books, magazines, research subjects and outcomes, and specialized academic information. I tried searching for “e-Learning(eラーニング)” and “guideline (ガイドライン)” in Japanese, and four papers, 25529 books, and four research subjects/outcomes were retrieved.
The first on the search result among the papers is titled “Development of e-Mentoring Guideline and its Evaluation”, Japan Journal of Educational Technology 29 (3) (Special Issue on Studies on Practical Use of e-Learning), 239-250. This seems to be research by scholoars at Aoyama Gakuin University. Mentoring is not covered by the instructional design but it is similar to my concern in that a guideline is used to check the effects. Some points in this paper may be useful for exploring the effective process of research implementation. Following a link from the search result a page to the body of the research paper, the full text is displayed in PDF file form. I am grateful for this (I thought it is a member privilege of JSET to be able to see the full text, but it is open to any scholar here at GeNii...).
The number of search results for “e-Learning(eラーニング)” and “designing of materials (教材設計)” in GeNii is “0.” How about “e-Learning(eラーニング)” and “design (デザイン)”? There were 46 results. Some results turned out to be researchers related to Kumamoto University. “checklist (チェックリスト)” seems to be used instead of “guideline.” There are 120 results by searching for “adult education(社会人教育),” but there are only 3 results shown for “adult education(社会人教育)” and “e-Learning(eラーニング).”
It seems necessary to get used to searching trying various ways. It may also be good to ask instructors: “I am interested in such and such. Could you kindly tell me if you know some useful references?” How about trying general search engines? If I know the name of a specific researcher, I may get useful information. I can also try Websites of other academic societies. I have also heard that looking over the list of contents or index of collected papers of recent national conferences will produce something interesting.
4. Stating the Research Questions
Is it time to “Write down a list of my questions (p.5)”?
- Is it possible to design instructional material with higher educational effects using a designing guideline? (Actually, I have found no designing guideline available now.)
- Is there any designing guideline that can be applicable to our company proposed by someone else? (If there is, I can use it. If there is not, then I have to make it. This is why I am talking about “proposing” a guideline.)
- How can a design guideline be proposed? (Can I look for similar ones? Can I combine several guidelines? Can I derive it from the theory? I have to look for the similar research on how to come up with a guideline.)
- What kinds of conditions should be satisfied by highly effective instructional material? (If I know, I can include them in the guidelines.)
- What kinds of improvements are possibly required for the e-Learning materials that our company has already developed? (What are the weaknesses of our materials? Can I make a guideline based on these points to be improved if I get to know it?)
- Who and how (based on what) decides which developed material is satisfactory for our company? (It is probably necessary to review the current situation. Can I explore it through interviewing?)
Wow, I have too many questions to add to the list. I am going to review the list from time to time. As I will come up with new questions before solving those questions, the list will not be shortened for the time being...
5. Determining the Research Design
Do I “know how painstaking research can be (p. 7)?” I don't know yet, but the concept of “feasibility” seems really important. I have to plan research that will be completed in two years, no, I actually have only one year....
I am probably not able to conduct experimental research. It is difficult to compare a case where a guideline is used to a case where it is not used. It may be possible to compare the current situation and the improvements after the introduction of a guideline. I have heard about AB design: A being current situation, and B being situation after introducing the guideline. It seems that if a change in A and B is found between before and after the introduction of a guideline, the guideline would consider to be effective. Experimental research needs two groups to make a comparison, but AB design needs only one, comparing present to the past. It seems troublesome, though, as I need to collect data many times...
6. Determining Methods
I probably have to conduct research in the following order. There may be the feasibility challenge of whether I can finish within two years (actually one year)...
- Look for literature that indicates how to decide on educational effectiveness. Look to see if there are guidelines or checklists. (Probably I cannot find the ones that I just wanted.)
- Get a picture of the current situations of our company. Find the weaknesses, and think of the acceptable level of instructional material. Then, make a draft guideline.
- Try to evaluate the draft guideline referring to the methods adopted by the literature “Development of e-Mentoring Guideline and its Evaluation.”
- If I can make a plausible guideline, I'll propose it to complete the research. (Up to this stage may be all I can do for the master's research. Whether it will be accepted as a thesis for the master's is another question.)
- Investigate changes that occur between before and after the introduction of the guidelines in a company based on the AB design. If the changes continue, the guidelines can be considered effective. (If the guideline is proposed, some companies may possibly adopt it. Is this possible, if I ask some companies to try out the guidelines, not going to the length of asking them to adopt them from the beginning?)
7. Identifying Data Analysis Procedures
I am very relieved to read the phrase “if you are not good at statistics, you do not have to worry (p.11).” However, I will remember that quantitative research is easier. It seems the research I have thought of will not require complicated statistical analyses. I have to ask specialists about this. Reference material I read mentioned about a test of differences between averages in a questionnaire survey (t(44)= 2.79, p<0.01). I have to understand what this means. On the other hand, research that does not require much statistical analysis may be more difficult, which is called qualitative research. I have to check this, too.
