Ⅰ. Introduction
VR (virtual reality) means any particular environmental situation or technology itself that is similar to reality, but is not real, created using computers to make artificial technologies. Certain virtually created environments or situations may enable users to experience a time and space experience similar to the actual one. In addition, it is possible to interact with the virtual reality by performing a manipulation or command using the existing HMD (head mount display) devices. VR technologies have been implemented for a long time but have not been commercialized. However, VR devices that combine Smart Phone and HMD are taking up the biggest portion in the market as large companies that are related to Smart phones are actively investing into VR industries through development of smartphones.1,2) Compared to the developments in the VR imaging industry, human factors research related to VR is still in complete. A human factor is to ensure the safety and efficiency of users by considering characteristics such as human capabilities and limitations in designing and implementing systems. Cybersickness is the most common symptom when experiencing virtual reality. The symptoms of cybersickness are similar to those of motion sickness, and typical symptoms are nausea, dizziness, and disorientation.
1. Stereoscopic vision
Stereoscopic vision refers to the depth achieved by combining images of objects in the left and the right eye into one in the brain. Each object looks different when viewed with the right eye and the left eye. 3D (three-dimensionality) TVs and VR displays also show images with different angles to the left and right eyes and produce a 3D feeling. The 3D that one feels while watching a video is to create a 3D sensation by looking at a flat screen. The process of combining two different images creates visual fatigue.3,4)
2. Accommodation and convergence discrepancy
Accommodation means that the thickness of the lens in the eye is adjusted to focus. Accommodation and convergence are associated with each other. Looking at closely placed objects, the degree to which the eye is drawn and the degree to which the lens thickens has a definite relationship to each other(Fig. 1).5)
HMD VR technology provides different images in the left eye and in the right eye to simulate objects in the same location as the real one. It gives the viewer the same visual axis and binocular disparity as the real object. However, since it is a screen image, focusing on the screen is problematic, not on the actual distance. In other words, HMD VR technology is technically copied on the visual axis and binocular disparity, but on the other hand, the focus adjustment has a technical limit that is not followed by the simulation. Consequently, accommodation and convergence do not occur in a certain relationship and occur separately. This is confusing as to whether to recognize a virtual object based on convergence or an accommodation. As a result, it causes eyestrain(Fig. 2).5)
3. EEG (electroencephalography)
EEG is a test that measures electrical signals in the brain through electrodes attached to the scalp in many areas. It is measured differently depending on the state of mind and body, and is used as an important indicator of brain activity. Because it is changed by the activity of brain cell, certain patterns change according to conscious and mental activity.
Generally, brain waves are classified into five groups, depending on the frequency band: delta wave(δ, 1~4 Hz), theta wave(θ, 4~8 Hz), alpha wave(α, 8~12 Hz), beta wave(β, 12~30 Hz) and gamma wave(γ, 30~50 Hz).6-8) Delta Wave is a brain wave in the range of 1 to 4 Hz that is dominant in deep sleep conditions where brain functions are completely relaxed.9) Theta wave refers to brainwaves in the range of 4 to 8 Hz and usually means the degree to which the body and consciousness are fuzzy, or between sleepiness and waking. The theta wave is associated with deep internalization and quiet state physical, emotional, and thought activities.10 Alpha Wave is a brain wave in the range of 8 to 12 Hz, a basic pie that reflects the stability of the brain in neurophysiology. Used to determine the functional state of the left and right hemisphere of the brain for human behavior.11) The beta wave is a brainwave of the 12 to 30 Hz band that appears in an awakening, active, and stressed state and is also affected by hearing, tactile, and emotional stimuli.12-14) The gamma wave is a 30 to 50 Hz band, a characteristic brainwave that occurs during mental concentration.12,15,16) Based on the alpha wave, the lower 8 Hz is classified as the slow wave, and the higher 12 Hz is the fast wave. In the study by Lee et al., the results of the analysis of the brain function index and the measurement of the brain wave before and after 3D image viewing with 3D TV showed that the visual fatigue were caused by gamma waves, theta waves, alpha waves, and beta waves.17)
4. Evaluation of visual fatigue
The side effects reported after the experience of 3D video media such as 'Avatar' are cyber sickness, including nausea and dizziness. SSQ (simulator sickness questionnaire) is a self-reporting questionnaire used as a measure of the level of cyber sickness. SSQ consists of 16 questions about symptoms, which are divided into 3 main categories: (1) Nausea: general discomfort, increased salivation, sweating, nausea, difficulty concentrating, stomach awareness, burping. (2) Oculomotor: general discomfort, fatigue, headache, eyestrain, difficulty focusing, difficulty concentrating, blurred vision. (3) Disorientation: difficulty focusing, nausea, fullness of head, blurred vision, dizzy(eyes open), dizzy(eyes closed), vertigo.18)
Ⅱ. Materials and methods
1. Subjects
This study analyzed the 30 subjects(in their 20s to 30s) who use smart devices the most. The subject were chosen as both a college student and an office worker. People with systemic or eye diseases, those taking drugs, alcohol or substance abuse and dependence, pregnant people, and those with corrected visual activity below 1.0 were excluded from the list. They had to have no systemic and eye diseases and their corrected vision were 1.0 or better. If there is anisometropia, it may occur that the image sizes of the right eye and the left eye are different, making it impossible to integrate VR images that are split between the two eyes separately. So people with anisometropia are excluded.
Subjects collected through flyers and online social networking sites. The number of subjects studied were calculated using the GPOWER sample count calculation program(Faul, Erdfelder, Buchner, & Lang, 2009). A total of 25 people were determined at 0.05, 80% of the calibration power, and 0.05, which is the moderate level of a t test. A total of 30 people were calculated in consideration of the failure rate. For qEEG (quantitative EEG) analysis, two were excluded from the survey, and the final sample size were 28. The final sample were composed of an mean age of 24.71±3.63 years, 15(53.7%) males and 13(46.3%) females.
Each subject has signed the agreement based on the information approved by the IRB (institutional review board) at the Inje University's Ilsan Paik Hospital(2017-04-001-002).
2. Visual function measurements
Refraction error were measured using the Phoropter (HDR-7000, Huvitz, Korea) and Chart projector (HDC-9000 N/PF, Huvitz, Korea), (corrected vision 1.0 or better). The far and near phoria test were repeated three times each using Howell cards (3 m, 40 cm) and 6 Δ prism. The push-up test to check near point of accommodation and the NPC (near point of convergence test were repeated three times. AC/A ratio used the Heterophoria method.
3. EEG measurements and qEEG analysis
The subjects sat on the chair in a room that blocked the ambient noise. Resting state qEEG were recorded once before and after the HMD videos are watched, for four minutes interval, with their eyes open and the eyes closed. EEG signals were recorded using a Quick Cap with 62 Ag-AgCl electrodes and NeuroScan SynAmps(Compumedics USA, El Paso, TX, USA). The electrodes are attached in accordance with the extended 10~20 system. The vertical EOG (electrooculogram) were recorded by attaching it above and below the left eye and horizontal EOG were recorded by attaching it to the outer canthus of each eye. We analyzed the resting EEG data with eyes open and eyes closed using CURRY7(Compumedics USA, Charlotte, NC, USA), which is commonly used for EEG preprocessing. The gross artifacts were removed from the recorded data by visual inspection. Eye blink artifacts and eye movement artifacts were removed using the mathematical analysis of CURRY7.19) The power spectral analysis were used to compress the rhythmic information of the brain wave signals and Fourier transformation were performed to analyze it.20) The absolute spectra has different amplitude characteristics for each person, which affects the magnitude of the frequency during Fourier transform.21) To compensate for this disadvantage, the relative spectra were calculated.22)
The band power was calculated into seven frequency bands: delta(1~4 Hz), theta(4~8 Hz), alpha 1(8~ 10 Hz), alpha 2(10~12 Hz), beta 1(12~18 Hz), beta 2(18~30 Hz), gamma(30~50 Hz). The relative power was averaged into three regions: anterior(FP1, F3, F7, Fz, FP2, F4 and F8), middle (T7, C3, Cz, T8 and C4), and posterior(P7, P3, O1, Pz, P8, P4 and O2). The division and selection of these regions were based on previous qEEG studies.23,24)
4. SSQ (simulator sickness questionnaire)
Subjects fill out the simulator sickness questionnaire before watching VR video. After watching VR video, complete the remainder of simulator sickness questionnaire. The computation of the SSQ score follows the Table 1. The degree of symptoms should be set from 0 to 3.18)
5. Virtual reality equipment and virtual reality video
The virtual reality machine used HMD, Samsung New Gear VR(SM-R323, Samsung electronics, Korea) released by Samsung electronics in September 2016. It connects smart phones to realize virtual reality. The smartphone used were Galaxy note 5(SM-N920S, Samsung electronics, Korea), which Samsung electronics launched in August 2015. VR videos had a viewing distance of 6~7 cm when HMD equipment was worn.
The video used in the experiment showed a virtual reality video of a roller coaster, scuba diving, driving, airplane control, skydiving and other situations at random. VR videos(https://youtu.be/nJui9sOlb98) were watched for about 10 minutes.
6. Statistical analysis
Statistical analysis was performed using SPSS version 18.0(SPSS Inc., Chicago, IL, USA) The analysis was performed using a repeated measure ANOVA to compare the qEEG before and after the VR video viewing. Correlation were analyzed by pearson correlation analysis. The mean comparison of the visual function and SSQ score before and after VR video were analyzed using paired t-test.
Ⅲ. Results
1. Characteristic of subjects
The analyzed subjects were 15 males, 13 females and 24.71±3.63 years of age. The mean of the refractive error is OD(-2.42±2.54 D), OS(-2.45 ±2.55 D). Thirteen of the subjects said they had experienced virtual reality, and 15 said they had never experienced virtual reality. The number of subjects according to the correction method is shown in Table 2.
2. Changes in visual function before and after watching the VR video.
1) Far and near phoria
Far phoria after VR video viewing changed on average from -1.44±2.21 to –1.75±2.07 △, but there were no statistically significant differences (t=1.313, p=0.200), (Table 3).
The near phoria after VR video viewing changed on average from –4.26±5.81 to –5.41±5.53 △, and statistically increased exophoria(t=3.127, p=0.004), (Table 3).
2) Near point of accommodation
After watching VR video, the near point of accommodation in the right eye changed from 10.88±2.89 to 11.02±2.53 cm, but without any significant statistical differences (t=-0.748, p=0.461), (Table 4).
After watching the VR video, the near point of accommodation in the left eye changed from 11.05±2.87 to 11.03±2.57 cm, but there were no statistically significant differences(t=0.120, p=0.905), (Table 4).
3) NPC
After watching VR video, the break point of the NPC was significantly increased statistically from 5.93±1.89 to 8.05±1.81cm on average(t= -7.252, p<0.001), (Table 5).
The recovery point increased statistically significantly from 8.29±1.49 to 10.51±1.72 cm on average(t= -14.488, p<0.001), (Table 5).
4) Accommodative convergence/accommodation ratio (AC/A ratio)
After watching VR video, the AC/A ratio significantly decreased from 4.95±2.18 to 4.53±1.88 on average (t=2.414, p=0.023), (Table 6).
3. Changes in qEEG-band relative power before and after watching VR video.
1) Changes in qEEG-band relative power with eyes open
Delta band relative power tends to decrease in anterior, middle and posterior, and were statistically significant(p<0.050). Theta band relative power appeared to decrease in the posterior and were statistically significant(p<0.050). The theta band relative power showed a tendency to decrease in anterior and middle, but there were no statistically significant differences. Alpha 1 band relative power showed significantly higher band power in both anterior, middle, and posterior, and were statistically significant(p<0.050). Alpha 2 band relative power showed significantly higher band power in both anterior, middle, and posterior, and were statistically significant(p<0.050). Beta 1 band relative power tended to increase in the anterior and middle and posterior, but were not statistically significant. Beta 2 band relative power tends to increase in anterior and middle decrease in the posterior, but is not statistically significant. The gamma band relative power showed a tendency to decrease in anterior and middle and posterior, but there were no statistically significant differences(Fig. 3, Table 7).
2) Changes in qEEG band relative power with eyes closed
Delta band relative power showed a tendency to increase in anterior, but were not statistically significant. The delta band relative power appeared to increase in the middle and in the posterior and were statistically significant (p<0.050). The theta band relative power showed a tendency to increase in anterior and middle and posterior, but there were no statistically significant differences. The alpha 1 band relative power showed a tendency to decrease in anterior and middle and posterior, but there were no statistically significant differences. Alpha 2 band relative power showed a tendency to decrease in posterior, and were statistically significant(p<0.050). The alpha 2 band relative power showed a tendency to decrease in anterior and middle, but there were no statistically significant differences. The beta 1 band relative power showed a tendency to decrease in anterior and middle and posterior, but there were no statistically significant differences. Beta 2 band power tended to decrease in anterior and posterior increase in middle, but there were no statistically significant differences. Gamma band power showed a tendency to decrease in anterior and middle increase in posterior, but were not statistically significant (Fig. 5, Table 8).
4. Changes in SSQ score before and after watching VR video.
After watching VR video, the nausea score(p< 0.001), oculomotor score(p<0.001), and disorientation score(p<0.001) significantly increased (Table 9). Fig. 4
5. Correlation analysis
1) The correlation between the visual function and qEEG band relative power with eyes open
In the correlation analysis of the qEEG band relative power with the visual function, alpha 1 band relative power(anterior) and near phoria showed a significant correlation. And alpha 1 band relative power(anterior) and AC/A ratio showed a significant correlation(Table 10).
2) The correlation between the SSQ score and qEEG band relative power.
In the correlation analysis of the SSQ Score with qEEG band relative power alpha 2 band relative power(middle, posterior, global) and nausea score, oculomotor score, disorientation score, total SSQ score show a significant correlation (Table 11).
3) The correlation between the visual function and SSQ score.
There was no correlation between SSQ scores and visual function such as near point of convergence, phoria, accommodation, AC/A ratio.
Ⅳ. Discussion
In this study, we evaluated the change in visual function, changes in qEEG-band power, and correlation between cyber sickness and visual function before and after watching VR video. The results were as follows: (1) After watching VR video, exophoria were significantly increased in near and NPC significantly increased. (2) After watching VR images, with the eyes open, alpha 1 waves increased statistically significantly in the global region. And alpha 2 waves increased statistically significantly in the global. (3) After watching VR videos, SSQ score increased, and all subjects felt cyber sickness. (4) Phoria and alpha 1 waves have a negative correlation. And the AC/A ratio showed negative correlation with the alpha 1 wave. The SSQ score and alpha 2 waves had a negative correlation. There was no significant correlation between SSQ and visual function.
1. Visual function
After watching VR images, near phoria and NPC have significantly changed. Kang et al. reported that NPC is far away after watching VR images using HMD and smartphone.25) VR images using HMD devices are estimated to increase near point of convergence since VR images are viewed at very close distances (6~7 cm). This were similar to the study found in previous studies using smart phones and monitors, where prolonged viewing of smartphones or monitors over a short distance increases the near point of convergence.26) Kang et al.25) said that there was no change in horizontal phoria after VR video viewing. However, the results of this study showed a significant increase in exophoria in near(Table 3). This showed the same result as an increase in exophoria in near when watching computer or smartphone images for a long time in prior studies.26,27)
2. Quantitative EEG
In this study, the alpha waves increased significantly. In particular, the alpha wave increased significantly in the occipital lobe. The alpha waves in the occipital lobe increase when the eyes are closed, which is known to be caused by mechanisms that block visual input. Therefore, the alpha rhythm of the occipital cortex is considered to be the steady vibration of the occipital cortex.21) This can be seen as the steady state of the brain wave with the eyes open after watching VR images. However, Hanslmayr et al. said that it was difficult to perceive the external stimuli when the alpha waves amplitude were high.28) Slobounov et al. measured brain waves by creating a virtual reality room, and reported an increase in theta waves when viewing 3D screens rather than 2D screens. Although theta waves tended to increase in this study, they were not statistically significant.29)
3. Visual fatigue
In this study, when VR images were viewed using HMD devices, all subjects experienced an increase in SSQ scores. That means that all subjects felt cyber motion sickness. This results in the same results as the previous preceding study.30) There are many factors that can cause cybersickness when watching VR images using HMD, such as technical factors and personal factors. Human visual angles are typically between 180° and 200°, and most FOV (field of view) in HMD are less than human visual angle values.31) The FOV for Samsung Gear VR is 101°.32)
4. Correlation analysis
In this study, significant results were found on the correlation between the visual function and the alpha wave, and the correlation between the SSQ score and the alpha wave.
First of all, near phoria and alpha 1 waves have negative correlation. In other words, after watching VR images, exophoria increases at near, the alpha1 wave increases. The alpha 1 waves and AC/A ratio have positive correlation. This means that after watching VR images, the AC/A ratio decreases, the alpha 1 wave increases. So far, very few studies have been done on the relationship between brain waves and visual functions.
Secondly, the alpha 2 waves and SSQ scores showed a negative correlation. Park et al used a driving simulator to analyze the correlation between brainwave and SSQ scores. Park et al. studies reported positive correlation between SSQ and Theta waves.33) In this study, SSQ and theta waves showed positive correlation but were not statistically significant. Research on the correlation between brainwave and SSQ scores using HMD devices has so far been minimal.
Finally, there were no correlation between the SSQ score, the visual function and such as near point of convergence, phoria and accommodation. These results showed similar results to the preceding studuy.34)
Ⅴ. Conclusion
This study was designed to determine how VR video viewing using HMD devices affects visual function, brain waves, and visual fatigue.
-
After watching VR images, the visual function showed a decrease in NPC and an increase in exophoria at near. And the AC/A ratio also significantly decreased. There was no significant difference between the far phoria and accommodation.
-
In the qEEG after watching VR images, alpha 1 band relative power and alpha 2 band relative power were increased in the global region. This can be seen as a steady state of the brain or a condition insensitive to external stimuli.
-
In visual fatigue, cyber sickness was felt by all subjects, and the degree of disorientation was the biggest increase.
-
Correlation analysis showed a positive correlation of the alpha 1 wave (8~10 Hz) in the anterior as exophoria increased at near. The correlation between the brainwave and SSQ showed a tendency for the alpha 1 wave (8~10 Hz) to increase significantly in middle and posterior relative to the total SSQ score. The higher the nausea score, oculomotor score, and disorientation score, the lower the alpha 2 wave (10~12 Hz) tends to increase. No correlation between the function and the SSQ score were shown.