Ⅰ. Introduction
A national examination for health care that is conducted to license or qualify for the profession, and each national examination for each profession should evaluate whether it has the ability to solve problems under actual job conditions. The optional test questions currently used in written tests primarily evaluate whether you remember or understand the relevant knowledge required to perform a task rather than these problem-solving skills. It also still takes a lot of effort to become a test that reflects the reality or authentic test. In general, when developing questions for the national examination of health care providers, students are asked to present common or important cases or job situations in text form, or to provide additional questions on the test questions, if necessary. However, it’s has a different between suggest type of problem like as well-structured problem well-defined problem and real world problem about known that there are limitations in predicting the ability of testtakers in real job situations in order to overcome this and evaluate the ability to solve problems in the real world more, NBME (national board of medical examiners) in USA has been using multimedia questions that reinforced the authenticity of cases in the computer test since 2007.
The Korea health personnel licensing examination is converting to smart device-based test all state exams conducted in Korea. Therefore, each major is looking into a testing method for smart device based test (SBT). A first-class emergency exit survey job developed multimedia test questions from 2014 and conducted three mock tests. In addition, the state test was converted into a smart devicebased test system in 2017 and the national test for other health and medical professions will be administered as a smart device-based test using tablet PC as the user interface in 2020 and national optometry licensing examination are scheduled to be implemented in 2021.1,2)
Education institutions in the health care, including nurses, physical therapists, medical laboratory technologist, optometry, dental hygienists, raised fundamental questions about the effectiveness and feasibility of SBT.3-5) In addition, the nurse's national examination is raising a controversy over the feasibility of the current paper test through comparative research with the computerized testing system in the U.S. The medical profession continues to conduct research on the future state-run SBT systems.6) As well as, it is necessary to identify the factors influencing the smart device-based national test that includes multimedia questions. The aim of this research is verifying the effect of smart device utilization on smart device based national test recognition in optometrist national examinations.
Ⅱ. Material and Methods
1. Subject
The evaluation survey included 60 optometry students at E university, Korea. Every participant was informed of the study’s objective in writing, and only those who gave written consent were enrolled in the study. The questionnaire used to measure smart device utilization competency was developed by previous research7), and questions are selected, used to determine which factors influence smart device – based testing awareness. The components of smart device utilization capabilities are two questions of information, two questions of record, two questions of communication, four questions of management, three questions of documentation, and two questions of multimediaization. The tool for measuring awareness of smart devicebased testing has selected and used eight questions from attitude-checking questionnaires used in prior studies studying subjects' attitudes to computers. The reliability of the measuring instrument was indicated by the Cronbach's value of .898 and investigated with a Likert 5 points scale.
2. Statistical analysis
The data analysis in this study was performed using the SPSS Statistics 20(SPSS Inc., Chicago, IL, USA) for Window program in the following manner: 1) Cronbach's coefficient was calculated to verify the capability of smart device utilization and reliability of smart device-based test recognition tools. 2) In order to verify the normality of the study variables, the mean, standard deviation, dwarfism, and kurtosis were calculated from the technical statistics. 3) Pearson's correlation coefficient was calculated to examine the relationship between smart device utilization capacity and smart devicebased national test recognition. 4) Multiple regression analysis was performed to verify the effect of smart device utilization on smart device-based test recognition. Also, the step selection method was applied to select the variables to be included in the regression model to obtain the optimal regression equation for smart device-based test recognition. The significance level was set at 0.050 in all analyses.
Ⅲ. Results
The results showed that among the general characteristics of the subjects were 26 male(43.3%) and 34 females(56.7%). In the case of the grades which was get a last semester, 40 participants showed most frequent of the grade below 2.1~ 3.5. In addition, a survey on the general information related to 48 students has an experience tested by using computers. In the case of the question asking whether or not has an experience with testing using smart device showed 37 students doesn’t have an experience in testing smart devices. Thus, survey question of the who has a currently used smart devices includes tablet PC and cell phone, it’s answered the all participants has an experience. 57 students showed that using smart device more than 7 days per week and 45 students answered to SNS was purpose of the used the smart device. Among the questions asked about overall recognition of smart device-based tests, 22 respondents(36.7%) answered "no difference" as the test type that measures the examinee's ability. When asked how the state-run evaluation system should be changed, 24(40%) answered the written test. Also, among the total national questions, 20 respondents accounted for 1 to 2% of the questions asked for the appropriate ratio of multimedia questions(Table 1).
Table 2 shows the smart device utilization and smart device-based national testing recognition. The average(standard deviation) of the management competencies among the six sub-factors of the smart device utilization competency was 4.29(.735). The average(standard deviation) of the question "Smart-based test can shorten the time required for the test" among the 8 questions about the perception of the smart device-based national test was the highest at 3.73(1.006). In addition, the absolute value of the kurtosis and the kurtosis of the total item did not exceed 3 and 10, so that the scale used in this study has a regularity.8)
The Pearson correlation coefficient was calculated to examine the relationship between the smart device utilization capacity recognized by college students and the national test recognition based on the smart device. The results were as follows: Record(.859***), communication(.775***), management (r.750***). There was a significant positive correlation with the recognition of national test based on smart devices(Table 3).
Table 4 shows the variance analysis of the multiple regression analysis for the recognition of national test based on smart devices. And the multi-collinearity which means high correlation between independent variables. The tolerance for diagnosing multi-collinearity between independent variables is .879 and variance inflation factor (VIF) is 1.690. The correlation between independent variables is not high enough to be a problem. Smart device utilization capability as a result of statistical significance test for the measurement model of the national test recognition based on the smart device with six independent variables, the smart device utilization capability was excluded except for the multimedia capability.
The F statistic of the model with multimedia capability is 8.006 and the significance is .006. The independent variables included in the model are significant at level .050, which explains the recognition of national test based on smart device. 46% of the total change in recognition(43% according to the modified decision coefficient) is explained by the independent variables included in the model. In addition, we examined the contribution and statistical significance of independent variables to dependent variables(Table 5).
In the results of testing the contribution and statistical significance of the multimedia device to the smart device-based national test recognition, the independent variable that significantly influenced the smart device-based national test recognition was the multimedia capability(t = 2.830, p = .006).
Based on these results, an independent variable that describes the recognition of a smart devicebased national test was selected step by step and the model was derived, and follow is the indicative of the relationship of the modify of regression analysis.
According to the regression equation, when the multimedia capability is 0, the average of recognition of the national test based on the smart device is 2.313, and when the multimedia capability is increased by one point, the recognition average is expected to increase by .271.
Ⅳ. Discussion and Conclusions
In recent years, many testing programs transited their tests from paper and pencil(PP) tests to computer-based (CB) tests and smart device based test (SBT) due to a number of advantages that the SBT test offers such as an shorter testing time, flexible scheduling, faster score reporting, opportunity to include innovative items, and reduced costs of test production.9-11) Also, SBT system does not use an offline based wireless network, the size of the equipment being deployed is reduced, making it less likely to fail and simplifying the system.1) The system has been modularized to maximize efficiency of test management so that equipment can be managed as a separate unit in the laboratory, and other functions such as prevention of denial of applicants and other convenience have been developed that were not implemented in the existing system.12,13)
This study is an empirical research study aimed at verifying the impact of the ability to utilize smart devices on the recognition of smart device-based national tests in optometrist' national tests. To that end, 60 students were surveyed. Questions asking whether they have experience using a computer, 48 students(80%) showed that had experience using a computer, 37 students (61.7%) answered for had no experience using a smart device, while if they had a test experience using a smart device. In addition, most of students have computer-based test experiences, but not many have been tested using smart devices. The capability of using smart devices was higher than average, and the recognition level of smart devices based national board exam was moderate. Except for the information and documentation capabilities, the recording, communication, management, and multimedia capabilities showed a significant positive correlation with the recognition of smart devices based national tests. Smart device utilization capability is explained to be about 46% in positive recognition of Smart Device-based national testing, and multimediaization capability among smart device utilization capabilities has been a significant factor in smart device-based national board exam recognition. Previous study reported that SBT could improve the effectiveness of taking a test and reduce marking answers on an OMR sheet.1)
Identifies the impact of smart device capability in the current situation where it is necessary to accept and prepare for the transition to smart device-based national board exam system. This is very meaningful in that it seeks to prevent the difference in capability from affecting negative perception of smart device-based national board exam. Indeed, this study can be expected to compensate for the problems of smart device-based national test systems, and to influence the change in the positive perception that smart device-based national tests, including multimedia questions, are a reliable test for test takers preparing for national board exam. Before the change to smart device-based national board exam, it is very meaningful that differences in test takers' ability to use smart devices were explored to prevent them from affecting the negative perception of smart device-based national board exam.
But the study has a limitation also, it as follows; First, this study is based on a study of optometry students at E University and has limitations in generalizing as a result of the entire population. Therefore, it is necessary to expand the scope of subjects to verify the generalization of research results that affect the recognition of national board exam by smart device-based optometrist. Second, by expanding the scope of the research, the verification of equivalence between the smart device-based test and the paper-based test including the multimedia item, and the examinee should be able to interpret the results regardless of the smart device-based or paper-based test. The results can minimize controversy among members in the transition to smart device-based optometrist national examination system. According to the change of national examination of optometrist, it is considered that the value of basic data to be usefully used when changing to computer based or smart device test is sufficient.
