Strategies for enhancing automatic fixation detection in head-mounted eye tracking.

Publisher: Springer Country of Publication: United States NLM ID: 101244316 Publication Model: Print-Electronic Cited Medium: Internet ISSN: 1554-3528 (Electronic) Linking ISSN: 1554351X NLM ISO Abbreviation: Behav Res Methods Subsets: MEDLINE

Publication: 2010- : New York : Springer
Original Publication: Austin, Tex. : Psychonomic Society, c2005-

Eye-Tracking Technology* ; Fixation, Ocular*/physiology ; Algorithms* ; Head Movements*/physiology ; Eye Movements*/physiology; Humans ; Attention/physiology

Moving through a dynamic world, humans need to intermittently stabilize gaze targets on their retina to process visual information. Overt attention being thus split into discrete intervals, the automatic detection of such fixation events is paramount to downstream analysis in many eye-tracking studies. Standard algorithms tackle this challenge in the limiting case of little to no head motion. In this static scenario, which is approximately realized for most remote eye-tracking systems, it amounts to detecting periods of relative eye stillness. In contrast, head-mounted eye trackers allow for experiments with subjects moving naturally in everyday environments. Detecting fixations in these dynamic scenarios is more challenging, since gaze-stabilizing eye movements need to be reliably distinguished from non-fixational gaze shifts. Here, we propose several strategies for enhancing existing algorithms developed for fixation detection in the static case to allow for robust fixation detection in dynamic real-world scenarios recorded with head-mounted eye trackers. Specifically, we consider (i) an optic-flow-based compensation stage explicitly accounting for stabilizing eye movements during head motion, (ii) an adaptive adjustment of algorithm sensitivity according to head-motion intensity, and (iii) a coherent tuning of all algorithm parameters. Introducing a new hand-labeled dataset, recorded with the Pupil Invisible glasses by Pupil Labs, we investigate their individual contributions. The dataset comprises both static and dynamic scenarios and is made publicly available. We show that a combination of all proposed strategies improves standard thresholding algorithms and outperforms previous approaches to fixation detection in head-mounted eye tracking.
(© 2024. The Author(s).)

Agtzidis, I., Startsev, M., & Dorr, M. (2019). 360-degree video gaze behaviour: A Ground-Truth data set and a classification algorithm for eye movements. In: Proceedings of the 27th ACM international conference on multimedia, MM ’19, pages 1007–1015, New York, USA, Oct. 2019. Association for Computing Machinery.
Anantrasirichai, N., Gilchrist, I. D., & Bull, D. R. (2016). Fixation identification for low-sample-rate mobile eye trackers. In: 2016 IEEE International Conference on Image Processing (ICIP), pages 3126–3130.
Andersson, R., Larsson, L., Holmqvist, K., Stridh, M., & Nyström, M. (2017). One algorithm to rule them all? An evaluation and discussion of ten eye movement event-detection algorithms. Behavior Research Methods, 49(2), 616–637. (PMID: 10.3758/s13428-016-0738-927193160)
Baumann, C., & Dierkes, K. (2023). Neon Accuracy Test Report. Pupil Labs. https://doi.org/10.5281/zenodo.10420388.
Bouguet, J.-Y., et al. (2001). Pyramidal implementation of the affine lucas kanade feature tracker description of the algorithm. Intel corporation, 5(1–10), 4.
Dar, A. H., Wagner, A. S., & Hanke, M. (2020). REMoDNaV: Robust eye movement detection for natural viewing. Cold Spring Harbor Laboratory, page 619254.
David, E. J., Gutiérrez, J., Coutrot, A., Da Silva, M. P., & Callet, P. L. (2018). A dataset of head and eye movements for [Formula: see text] videos. In: Proceedings of the 9th ACM multimedia systems conference, MMSys ’18, pages 432–437, New York, USA, June 2018. Association for Computing Machinery.
de Barbaro, K., Chiba, A., & Deák, G. O. (2011). Micro-analysis of infant looking in a naturalistic social setting: Insights from biologically based models of attention. Developmental Science, 14(5), 1150–1160. (PMID: 10.1111/j.1467-7687.2011.01066.x21884330)
Diaz, G., Cooper, J., Rothkopf, C., & Hayhoe, M. (2013). Saccades to future ball location reveal memory-based prediction in a virtual-reality interception task. Journal of Vision, 13(1), 20. (PMID: 10.1167/13.1.20233253473587002)
Duchowski, A. T. (2002). A breadth-first survey of eye-tracking applications. Behavior Research Methods, Instruments, & Computers, 34(4), 455–470. (PMID: 10.3758/BF03195475)
Engbert, R., & Mergenthaler, K. (2006). Microsaccades are triggered by low retinal image slip. Proc. Natl. Acad. Sci. U. S. A., 103(18), 7192–7197. (PMID: 10.1073/pnas.0509557103166326111459039)
Farnebäck, G. (2003). Two-Frame motion estimation based on polynomial expansion. Image Analysis (pp. 363–370). Berlin Heidelberg: Springer. (PMID: 10.1007/3-540-45103-X_50)
Franchak, J. M., Kretch, K. S., Soska, K. C., & Adolph, K. E. (2011). Head-mounted eye tracking: A new method to describe infant looking. Child Development, 82(6), 1738–1750. (PMID: 10.1111/j.1467-8624.2011.01670.x220233103218200)
Hansen, D. W., & Ji, Q. (2010). In the eye of the beholder: A survey of models for eyes and gaze. IEEE Transactions on Pattern Analysis and Machine Intelligence, 32(3), 478–500. (PMID: 10.1109/TPAMI.2009.3020075473)
Hessels, R. S., Hooge, I. T. C., & Kemner, C. (2016). An in-depth look at saccadic search in infancy. Journal of Vision, 16(8), 10. (PMID: 10.1167/16.8.1027299770)
Hessels, R. S., Niehorster, D. C., Kemner, C., & Hooge, I. T. C. (2017). Noise-robust fixation detection in eye movement data: Identification by two-means clustering (I2MC). Behavior Research Methods, 49(5), 1802–1823. (PMID: 10.3758/s13428-016-0822-127800582)
Hessels, R. S., Niehorster, D. C., Nyström, M., Andersson, R., & Hooge, I. T. C. (2018). Is the eye-movement field confused about fixations and saccades? a survey among 124 researchers. Royal Society Open Science, 5(8), 180502. (PMID: 10.1098/rsos.180502302250416124022)
Hooge, I. T. C., Niehorster, D. C., Nyström, M., Andersson, R., & Hessels, R. S. (2018). Is human classification by experienced untrained observers a gold standard in fixation detection? Behavior Research Methods, 50(5), 1864–1881. (PMID: 10.3758/s13428-017-0955-x29052166)
Hooge, I. T. C., Niehorster, D. C., Nyström, M., Andersson, R., & Hessels, R. S. (2022). Fixation classification: How to merge and select fixation candidates. Behavior Research Methods, 54(6), 2765–2776. (PMID: 10.3758/s13428-021-01723-1350230669729319)
Hoppe, S., & Bulling, A. (2016). End-to-End Eye Movement Detection Using Convolutional Neural Networks. arXiv:1609.02452 .
Huang, Z., Zhang, T., Heng, W., Shi, B., & Zhou, S. (2020) RIFE: Real-Time intermediate flow estimation for video frame interpolation. Nov. 2020.
Kasneci, E., Kasneci, G., Kübler, T. C., & Rosenstiel, W. (2015). Online Recognition of Fixations, Saccades, and Smooth Pursuits for Automated Analysis of Traffic Hazard Perception. In P. Koprinkova-Hristova, V. Mladenov, & N. K. Kasabov (Eds.), Artificial Neural Networks, Springer Series in Bio-/Neuroinformatics (pp. 411–434). Cham. Springer International Publishing.
Kassner, M., Patera, W., & Bulling, A. (2014). Pupil: An open source platform for pervasive eye tracking and mobile gaze-based interaction. In: Proceedings of the 2014 ACM international joint conference on pervasive and ubiquitous computing: Adjunct publication, UbiComp ’14 Adjunct, pages 1151–1160, New York, USA, Sept. 2014. Association for Computing Machinery.
Kemner, C., van Ewijk, L., van Engeland, H., & Hooge, I. (2008). Brief report: Eye movements during visual search tasks indicate enhanced stimulus discriminability in subjects with PDD. Journal of Autism and Developmental Disorders, 38(3), 553–557. (PMID: 10.1007/s10803-007-0406-017610058)
Kinsman, T., Evans, K., Sweeney, G., Keane, T., & Pelz, J. (2012). Ego-motion compensation improves fixation detection in wearable eye tracking. In: Proceedings of the symposium on eye tracking research and applications, ETRA ’12, pages 221–224, New York, USA. Association for Computing Machinery.
Klein, C. (2019). Eye Movement Research - An Introduction to its Scientific Foundations and Applications.
Komogortsev, O. V., Gobert, D. V., Jayarathna, S., Koh, D. H., & Gowda, S. M. (2010). Standardization of automated analyses of oculomotor fixation and saccadic behaviors. IEEE Transactions on Biomedical Engineering, 57(11), 2635–2645. (PMID: 10.1109/TBME.2010.2057429)
Komogortsev, O. V., & Karpov, A. (2012). Automated classification and scoring of smooth pursuit eye movements in the presence of fixations and saccades. Behavior Research, 45(1), 203–215. (PMID: 10.3758/s13428-012-0234-9)
Komogortsev, O. V., & Khan, J. I. (2009). Eye movement prediction by oculomotor plant Kalman filter with brainstem control. Journal of Control Theory and Applications, 7(1), 14–22. (PMID: 10.1007/s11768-009-7218-z)
Kothari, R., Yang, Z., Kanan, C., Bailey, R., Pelz, J. B., & Diaz, G. J. (2020). Gaze-in-wild: A dataset for studying eye and head coordination in everyday activities. Scientific Reports, 10(1), 2539. (PMID: 10.1038/s41598-020-59251-5320548847018838)
Larsson, L., Nyström, M., Andersson, R., & Stridh, M. (2015). Detection of fixations and smooth pursuit movements in high-speed eye-tracking data. Biomedical Signal Processing and Control, 18, 145–152. (PMID: 10.1016/j.bspc.2014.12.008)
Martinez-Conde, S., Macknik, S. L., & Hubel, D. H. (2004). The role of fixational eye movements in visual perception. Nature Reviews Neuroscience, 5(3), 229–240. (PMID: 10.1038/nrn134814976522)
Nyström, M., & Holmqvist, K. (2010). An adaptive algorithm for fixation, saccade, and glissade detection in eyetracking data. Behavior Research Methods, 42(1), 188–204. (PMID: 10.3758/BRM.42.1.18820160299)
Olsen, A. (2012). The Tobii I-VT fixation filter. Tobii Technology, 21, 4–19.
Otero-Millan, J., Troncoso, X. G., Macknik, S. L., Serrano-Pedraza, I., & Martinez-Conde, S. (2008). Saccades and microsaccades during visual fixation, exploration, and search: Foundations for a common saccadic generator. Journal of vision, 8(14), 21. (PMID: 10.1167/8.14.21)
Patla, A. E., & Vickers, J. N. (1997). Where and when do we look as we approach and step over an obstacle in the travel path? Neuroreport, 8(17), 3661–3665. (PMID: 10.1097/00001756-199712010-000029427347)
Pérez-Edgar, K., MacNeill, L. A., & Fu, X. (2020). Navigating through the experienced environment: Insights from mobile eye tracking. Current Directions in Psychological Science, 29(3), 286–292. (PMID: 10.1177/0963721420915880336427067909451)
Rai, Y., Gutiérrez, J., & Le Callet, P. (2017). A dataset of head and eye movements for 360 degree images. In: Proceedings of the 8th ACM on multimedia systems conference, MMSys’17, pages 205–210, New York, USA, June 2017. Association for Computing Machinery.
Rayner, K. (1978). Eye movements in reading and information processing. Psychological Bulletin, 85(3):618–660.
Rayner, K. (1978). Eye movements in reading and information processing. Psychological Bulletin, 85(3), 618–660. (PMID: 10.1037/0033-2909.85.3.618353867)
Rayner, K. (1998). Eye movements in reading and information processing: 20 years of research. Psychological Bulletin, 124(3), 372–422. (PMID: 10.1037/0033-2909.124.3.3729849112)
Salvucci, D. D. & Goldberg, J. H. (2000) Identifying fixations and saccades in eye-tracking protocols. In: Proceedings of the 2000 symposium on eye tracking research & applications, ETRA ’00, pages 71–78, New York, USA. Association for Computing Machinery.
Santini, T., Fuhl, W., Kübler, T., & Kasneci, E. (2016). Bayesian identification of fixations, saccades, and smooth pursuits. In: Proceedings of the ninth biennial ACM symposium on eye tracking research & applications, ETRA ’16, pages 163–170, New York, USA, Mar. 2016. Association for Computing Machinery.
Schütz, A. C., Braun, D. I., & Gegenfurtner, K. R. (2011). Eye movements and perception: A selective review. Journal of Vision, 11(5), 9. (PMID: 10.1167/11.5.921917784)
Sparks, D. L. (2002). The brainstem control of saccadic eye movements. Nature Reviews Neuroscience, 3(12), 952–964. (PMID: 10.1038/nrn98612461552)
Startsev, M., Agtzidis, I., & Dorr, M. (2019). 1D CNN with BLSTM for automated classification of fixations, saccades, and smooth pursuits. Behavior Research Methods, 51(2), 556–572. (PMID: 10.3758/s13428-018-1144-230411227)
Startsev, M., Agtzidis, I., & Dorr, M. (2019). 1D CNN with BLSTM for automated classification of fixations, saccades, and smooth pursuits. Behavior Research Methods, 51(2), 556–572.
Startsev, M., & Zemblys, R. (2022). Evaluating eye movement event detection: A review of the state of the art. Behavior Research Methods, 55, 1653–1714. (PMID: 10.3758/s13428-021-01763-735715615)
Steil, J., Huang, M. X., & Bulling, A. (2018). Fixation detection for head-mounted eye tracking based on visual similarity of gaze targets. In: Proceedings of the 2018 ACM symposium on eye tracking research & applications, number Article 23 in ETRA ’18, pages 1–9, New York, USA, June 2018. Association for Computing Machinery.
Tonsen, M., Baumann, C. K., Dierkes, K. (2020). A high-level description and performance evaluation of pupil invisible.
van der Lans, R., Wedel, M., & Pieters, R. (2011). Defining eye-fixation sequences across individuals and tasks: The Binocular-Individual Threshold (BIT) algorithm. Behavior Research Methods, 43(1), 239–257. (PMID: 10.3758/s13428-010-0031-221287116)
Veneri, G., Piu, P., Rosini, F., Federighi, P., Federico, A., & Rufa, A. (2011). Automatic eye fixations identification based on analysis of variance and covariance. Pattern Recognition Letters, 32(13), 1588–1593. (PMID: 10.1016/j.patrec.2011.06.012)
Wan, Q., Kaszowska, A., Panetta, K., Taylor, H.A., & Agaian, S. (2019). A comprehensive head-mounted eye tracking review: Software solutions, applications, and challenges. Electronic Imaging, 2019(3):654–1–654–9.
Zagoruyko, S., & Komodakis N. (2015). Learning to Compare Image Patches via Convolutional Neural Networks. pages 4353–4361.
Zemblys, R. (2017). Eye-movement event detection meets machine learning. Biomedical Engineering, 20(1).
Zemblys, R., Niehorster, D. C., & Holmqvist, K. (2019). Gazenet: End-to-end eye-movement event detection with deep neural networks. Behavior Research Methods, 51(2), 840–864. (PMID: 10.3758/s13428-018-1133-530334148)
Zemblys, R., Niehorster, D. C., Komogortsev, O., & Holmqvist, K. (2018). Using machine learning to detect events in eye-tracking data. Behavior Research Methods, 50(1), 160–181. (PMID: 10.3758/s13428-017-0860-328233250)

Keywords: Automatic detection; Eye movements; Fixation dataset; Fixations; Head movements; Head-mounted eye tracking

Date Created: 20240409 Date Completed: 20240820 Latest Revision: 20241031

20241031

10.3758/s13428-024-02360-0

38594440