Special Research I Session3: Study Case in ID Field (1)

The research I have been advising is roughly divided into three categories: (1) survey research, (2) developmental research, and (3) experimental research. Survey research involves scrupulous observation and analysis of a given educational practice, or wide and/or deep survey into the actual examples or status-quo of in the field in relation to a given topic. Developmental research involves producing a system, instructional material, or teaching method that have not yet been created. Finally, experimental research involves the examination of the effectiveness of what you have created or improved on. Actually, all of these three aspects are more or less parts of any research. Your research will be categorized depending on which aspect of research methods you would focus on. The following cases represent each category.

A case in survey research

Ichikawa wrote a master's thesis titled “Website content survey of Japanese elementary and secondary schools: Status-quo of the early years”. In his research, he conducted surveys, on three occasions between August 1995 and August 1996, on all school Websites and examined their contents produced by elementary and junior/senior high schools in Japan. He was able to conduct this survey because, in those early days, it was rare for schools to have their own Website. However, even during that period the number of school Websites had risen from 98 to 603. If he were to conduct this kind of research today, it would have to be done using random selection or automated surveys.

In the first survey he conducted, Ichikawa has identified unique and creative Websites, in search of good practices. In the second survey, he has divided the contents into categories, as well as developing a scheme of Website for each school type. In the third survey, he has collected all contents from the target Websites. In his survey, Ichikawa has calculated the number of files and their volume, as well as the information in each category and its progress. In each school type, 50% of the whole volume of information was provided by less than 10% of all schools. He also categorized the purposes for creating a Website, and analyzed the contents of the information. In Senior high schools, it was more for publicity purposes. However, among elementary schools it was mostly outcomes of pupil's work. Among the school subjects, Information Technology, Art and Craft, Science, and Social Studies were most often chosen for publication. The condensed version of this thesis was submitted to a journal after Ichikawa had started his Ph.D. The analysis of the contents was considered unique, however, after the referee process, it was judged that it lacked in originality and was published as “Technical Information” in the Japan Society for Educational Technology Journal (Ichikawa & Suzuki, 1999).

A case in developmental research

Ichinohe submitted a bachelor's thesis in 2001 titled “Development of the i-mode drill ‘Kanji no Sekai' with the review function” ([1] for abstract in Japanese; [2] for thesis in Japanese). The study originated due to her academic interest in creating a more effective learning environment, using the then newly-available mobile phone as a platform. She investigated what sort of educational applications were available and found a relatively well-designed application; Kanji no Sekai (the World of Chinese Characters). She registered and analyzed the application as a user and tried to replicate it. Since she could not gain the source code, she needed to develop the system from scratch. She was studying in a college to become a software engineer, so, it also provided her with an opportunity to improve her programming skills.

Anyway, she then came up with some ideas to improve the application by referring to ID, and designed and developed an extended version of the application. The major functions she added were the features (1) to send drill mails (push technology), (2) to add an option to review only the wrong answers, and (3) to allow users to set the number of practice questions. She also altered the layout of the screen that showed the user decisions. Since she spent most of her research time in developing the application, both replication and expanded edition, the assessment focused mainly on the question of “whether the added functions are workable”. It concluded that the next step to be taken in a future research was the examination of the differences in actual learning effectiveness between the replication and the extended versions after using them for a certain period. After Ichinohe had graduated, Suzuki translated her thesis into English and presented it at an international conference (Ichinohe & Suzuki, 2002).

[1] http://www2.st.gsis.kumamoto-u.ac.jp/study/soturon/1998/abstract/0311998018_abstract.pdf (Japanese only)
[2] http://www2.st.gsis.kumamoto-u.ac.jp/study/soturon/1998/thesis/0311998018_thesis.pdf (Japanese only)

A case in experimental research

Matsumura submitted her bachelor's thesis titled “Improvement of ‘computer virus handling' based on the Goal-based Scenarios” in 2004 ([3] for abstract in Japanese; [4] for thesis in Japanese). In her experimental research, she compared and analyzed two instructional materials; one was a story-type material developed by her senior student, and the other was the altered version of the material she had produced based on the Goal-based Scenarios (GBS), one of the contemporary ID theories. She began by analyzing the senior student's material by applying the GBS, and then clarified what would “implement the theory better”. She then developed an improved version following her plan for alteration, and conducted a user test (formative evaluation). Finally, she asked the volunteer testers to list similarities and differences between the two materials, the strength and weakness of each material, and which they would prefer to use if there were sequels for each material. Since most of the testers preferred the improved version, she concluded that her alteration was effective. To make it a more advanced experimental research, she could have used a larger number of volunteer testers, and statistically examined the differences in learning effectiveness and impression. However, this was mentioned as a consideration for future.

[3] http://www2.st.gsis.kumamoto-u.ac.jp/study/soturon/2004/0312001158.pdf (Japanese only)
[4] http://www2.st.gsis.kumamoto-u.ac.jp/study/soturon/2004/thesis/0312001158.pdf (Japanese only)

Past researches were published as the “Researches of Suzuki Laboratory: ‘Educational Information System'” (http://www2.st.gsis.kumamoto-u.ac.jp/study/) on the Kumamoto University Website, though there may be a broken link due to removals and reconstruction. On the site, we have published as many as we could of the summaries and theses themselves of each research as well as the systems that have been developed by my students, and you are welcome to consult them for your reference, although they are mostly in Japanese.

Finally, it should be mentioned that the undergraduate students studying in the Suzuki-lab were instructed to try to produce a thesis worthy of a master's degree. For various reasons, they do not always reach the level of a master's thesis. However, there should not be much difference in the quality between a bachelor's and master's thesis, except that one was written by an undergraduate and another by a master student. Meanwhile, master's theses may fail to reach the level required for a master's degree, again for various reasons. The category to which each thesis belongs is clearly reflected in the grade I have given to its author, though, I should point out that they are not shown on the Website.

Session 3: Study Case in ID Field (1) (Lecturer: Katsuaki Suzuki)

[Study Case]

A case in survey research

A case in developmental research

A case in experimental research

Reference