Effects of different real-time feedback types on human performance in high-demanding work conditions

Int. J. Hum.-Comput. Stud., Volume 91, 2016, Pages 1-12.

Cited by: 8|Bibtex|Views17|Links
EI
Keywords:
heart rate variabilityreal life crisisTask Set Switcheshuman task performance 1.real time feedbackMore(24+)
Weibo:
The results showed that the performance scores increased when feedback was presented to the participants

Abstract:

Experiencing stress during training is a way to prepare professionals for real-life crises. With the help of feedback tools, professionals can train to recognize and overcome negative effects of stress on task performances. This paper reports two studies that empirically examined the effect of such a feedback system. The system, based on ...More

Code:

Data:

0
Introduction
  • For professionals to be ready for work under stressful circumstances, such as crises, combat or disaster scenarios, they need proper training.
  • Experiencing stress in VR enhances professionals’ performances in real stressful situations (McClernon, McCauley, O'Connor, & Warm, 2010).
  • Adding instructions to such training would provide more advantages, especially for the training of novices (Kirschner, Sweller, & Clark, 2006).
  • Next to the VR training, other training tools are needed to help the trainee learn to perform tasks under stress
Highlights
  • For professionals to be ready for work under stressful circumstances, such as crises, combat or disaster scenarios, they need proper training
  • This paper focuses on real-time feedback that can be used during simulation-based training to learn to cope with stressful situations
  • 2.3 Discussion The first experiment resulted in a set of 8 stressful scenarios that will be used in the experiment
  • The results showed that the performance scores increased when feedback was presented to the participants
  • This feedback system is based on the idea of cognitive tools providing immediate biofeedback, performance feedback and more detailed error-chance feedback (Bouchard, et.al, 2012; Prinsloo et.al., 2013; Gonzalez, 2005; Lerch & Harter, 2001) to decrease negative effects of stress on performances
  • The predictive models were implemented into the feedback system to provide participants with eight different combinations of three types of feedback
Methods
  • Participants were between 18 and 34 years old, with an average age of 25.5 (SD = 4.67) years.
  • Data to calculate the median age was, lost.
  • Fifteen participants were male and all participants were naive with respect to the purpose of the experiment.
  • They were compensated with 25 euros plus travel expenses.
  • A bonus of 20 euros was awarded to the participant with the highest performance score on the experimental task
Results
  • Data from this experiment was used to select the most stressful scenarios. These scenarios were used in the second experiment.
  • The first hypothesis states that immediate feedback in general results in an increase of performance and perceived level of usability.
  • The second hypothesis states that the three separate feedback types increase performance and perceived level of usability.
  • The third hypothesis states that an additional positive effect can be found on top of the effect for the separate feedback types on performance and perceived level of usability.
Conclusion
  • The first experiment resulted in a set of 8 stressful scenarios that will be used in the experiment.
  • Adding error-chance feedback to physiological feedback reduced the perceived usability when comparing it with a situation where only physiological feedback was provided
  • When it came to the effect on performance the findings were inconclusive on this point.
  • The statistical analyses showed that providing participants with immediate feedback resulted in an improvement of performance scores.
  • Analysing the main and interaction effects of the different types of feedback showed an increase of System Usability Scale (SUS) score for physiological feedback over no physiological feedback.
  • The usability data showed that there are good opportunities for this type of feedback to be accepted and processed for performance enhancement
Summary
  • Introduction:

    For professionals to be ready for work under stressful circumstances, such as crises, combat or disaster scenarios, they need proper training.
  • Experiencing stress in VR enhances professionals’ performances in real stressful situations (McClernon, McCauley, O'Connor, & Warm, 2010).
  • Adding instructions to such training would provide more advantages, especially for the training of novices (Kirschner, Sweller, & Clark, 2006).
  • Next to the VR training, other training tools are needed to help the trainee learn to perform tasks under stress
  • Methods:

    Participants were between 18 and 34 years old, with an average age of 25.5 (SD = 4.67) years.
  • Data to calculate the median age was, lost.
  • Fifteen participants were male and all participants were naive with respect to the purpose of the experiment.
  • They were compensated with 25 euros plus travel expenses.
  • A bonus of 20 euros was awarded to the participant with the highest performance score on the experimental task
  • Results:

    Data from this experiment was used to select the most stressful scenarios. These scenarios were used in the second experiment.
  • The first hypothesis states that immediate feedback in general results in an increase of performance and perceived level of usability.
  • The second hypothesis states that the three separate feedback types increase performance and perceived level of usability.
  • The third hypothesis states that an additional positive effect can be found on top of the effect for the separate feedback types on performance and perceived level of usability.
  • Conclusion:

    The first experiment resulted in a set of 8 stressful scenarios that will be used in the experiment.
  • Adding error-chance feedback to physiological feedback reduced the perceived usability when comparing it with a situation where only physiological feedback was provided
  • When it came to the effect on performance the findings were inconclusive on this point.
  • The statistical analyses showed that providing participants with immediate feedback resulted in an improvement of performance scores.
  • Analysing the main and interaction effects of the different types of feedback showed an increase of System Usability Scale (SUS) score for physiological feedback over no physiological feedback.
  • The usability data showed that there are good opportunities for this type of feedback to be accepted and processed for performance enhancement
Tables
  • Table1: Parameter values used to create different scenarios. Orders of experimental conditions. The scenario order did not change during the experiment
  • Table2: Performance scoring scheme for different actions. Participants’ reasons to rate a feedback combination as either most or least pleasant
  • Table3: Predictive performance model. The model consists of 5 variables
  • Table4: Logistic regression model that predicts the chance an error would be made. Errors are weighted with 25:75
  • Table5: Logistic regression model to prediction communication 20 and task allocation errors1
  • Table6: Descriptive statistics for the (relative) performance scores
  • Table7: Likelihood ratio test for models fitting the performance scores
  • Table8: Likelihood ratio test for models fitting the Communication, task allocation and total error variable. Testing if adding effects would improve fit compare to the H0 model
  • Table9: Likelihood ratio test for models fitting the SUS variable
  • Table10: Effects of feedback types on SUS scores; Model 4 including the main effects, 2-way interactions and 3-way interaction
Download tables as Excel
Funding
  • The work presented in this paper is supported by the Dutch FES program: Brain and Cognition: Societal Innovation (project no. 056-22-010)
Reference
  • Anderson, J. R., Corbett, A. T., Koedinger, K. R., & Pelletier, R. (1995). Cognitive tutors: Lessons learned. The journal of the learning sciences, 4(2), 167-207.
    Google ScholarLocate open access versionFindings
  • Andresen, L., Boud, D., & Cohen, R. (2001). Experience-Based learning. In G. Foley (Ed.), Understanding adult education and training (pp. 225-239). Sydney: Allen & Unwin.
    Google ScholarLocate open access versionFindings
  • Beach, L. R., & Lipshitz, R. (1993). Why classical decision theory is an inappropriate standard for evaluating and aiding most human decision making. In G. A. Klein, J. Orasanu, R. Calderwood & C. E. Zsambok (Eds.), Decision making in action: Models and methods (pp. 320). Norwood, New Jersey: Ablex publishing corporation.
    Google ScholarLocate open access versionFindings
  • Beaubien, J. M., & Baker, D. P. (2004). The use of simulation for training teamwork skills in health care: how low can you go?. Quality and safety in health care, 13(suppl 1), i51-i56.
    Google ScholarLocate open access versionFindings
  • Bouchard, S., Bernier, F., Boivin, E., Morin, B., & Robillard, G. (2012). Using biofeedback while immersed in a stressful videogame increases the effectiveness of stress management skills in soldiers. Plos one, 7(4). Brooke, J. (1996). SUS: a quick and dirty usability scale. In P. W. Jordan, B. Thomas, I. L. McClelland & B. Weerdmeester (Eds.), Usability evaluation in industry: CRC Press. 38
    Google ScholarLocate open access versionFindings
  • Brouwer, A.-M., Neerincx, M.A., Kallen, V.L., van der Leer, L. and ten Brinke, M. (2011). EEG alpha asymmetry, heart rate variability and cortisol in response to Virtual Reality induced stress. Journal of CyberTherapy & Rehabilitation, 4(1), 21-34.
    Google ScholarLocate open access versionFindings
  • Busscher, B., Vlieger, D. d., Ling, Y., & Brinkman, W. P. (2011). Physiological measures and selfreport to evaluate neutral virtual reality worlds. Journal of Cybertherapy & Rehabilitation, 4(1), 15-25.
    Google ScholarLocate open access versionFindings
  • Cesta, A., Cortellessa, G., & Benedictis, R. D. (2014). Training for crisis decision making - An approach based on plan adaptation. Knowledgde-based systems, 58, 98-112.
    Google ScholarLocate open access versionFindings
  • Cohen, I. (2015). Improving trainees’ performances while under stress using real-time feedback (Doctoral dissertation, TU Delft, Delft University of Technology).
    Google ScholarFindings
  • Cohen, I., Brinkman, W. P., & Neerincx, M. A. (2012). Assembling a synthetic emotion mediator for quick decision making during acute stress. Paper presented at the Proceedings of the 2012 European Conference on Cognitive Ergonomics, Edinburgh.
    Google ScholarLocate open access versionFindings
  • Cohen, I., Brinkman, W. P., & Neerincx, M. A. (2015). Modelling environmental and cognitive factors to predict performance in a stressful training scenario on a naval ship simulator. Cognition, Technology & Work, 17(4), 503-519.
    Google ScholarLocate open access versionFindings
  • Cohen, I., den Braber, N., Smets, N. J., van Diggelen, J., Brinkman, W. P., & Neerincx, M. A. (2016). Work content influences on cognitive task load, emotional state and performance during a simulated 520-days' Mars mission.Computers in Human Behavior, 55, 642-652.
    Google ScholarLocate open access versionFindings
  • Cohen, M. S. (1993). The bottom line: naturalistic decision aiding. In G. A. Klein, J. Orasanu, R. Calderwood & C. E. Zsambok (Eds.), Decision making in action: Models and methods (pp. 320). Norwood, New Jersey: Ablex publishing corporation.
    Google ScholarLocate open access versionFindings
  • Dörner, D., & Schaub, H. (1994). Errors in planning and decision-making and the nature of human information processing. Applied psychology: an international review, 43(4), 433-453.
    Google ScholarLocate open access versionFindings
  • Driskell, J. E., & Johnston, J. H. (2006). Stress exposure training. In J. A. Cannon-Browers & E. Salas (Eds.), Making decisions under stress (Vol. 3). Washington, DC: American Psychological Association.
    Google ScholarLocate open access versionFindings
  • Gatchel, R. J., Korman, M., Weis, C. B., Smith, D., & Clarke, L. (1978). A Multiple‐ Response 39
    Google ScholarFindings
  • Gohm, C. L., Baumann, M. R., & Sniezek, J. A. (2001). Peronality in extreme situations: thinking (or not) under acute stress. Journal of research in personality, 35, 388-399.
    Google ScholarLocate open access versionFindings
  • Gonzalez, C. (2005). Decision support for real-time, dynamic decision-making tasks. Organizational Behavior and Human Decision Processes, 96, 142–154.
    Google ScholarLocate open access versionFindings
  • Gordon, S. E. (1988). Focusing on the human factor in future expert systems. Paper presented at the Artificial Intelligence and other innovative computer applications in the nuclear industry, Snowbird, Utah. Greeno, J. G. (1994). Gibson’s affordances. Psychological Review, 101(2), 336-342.
    Google ScholarLocate open access versionFindings
  • Grootjen, M., Bierman, E. P. B., & Neerincx, M. A. (2006). Optimizing cognitive task load in naval ship control centres: Design of an adaptive interface. Paper presented at the IEA: 16th World Congress on Ergonomics.
    Google ScholarFindings
  • Hartanto, D., Kampmann, I. L., Morina, N., Emmelkamp, P. G., Neerincx, M. A., & Brinkman, W. P. (2014). Controlling Social Stress in Virtual Reality Environments. Plos one, 9(3), e92804.
    Google ScholarLocate open access versionFindings
  • Hjortskov, N., Rissen, D., Blangsted, A. K., Fallentin, N., Lundberg, U., & Søgaard, K. (2004). The effect of mental stress on heart rate variability and blood pressure during computer work. European Journal Applied Physiology, 92, 84-89.
    Google ScholarLocate open access versionFindings
  • Horsky, J., Schiff, G. D., Johnston, D., Mercincavage, L., Bell, D., & Middleton, B. (2012). Interface design principles for usable decision support: A targeted review of best practices for clinical prescribing interventions. Journal of Biomedical Informatics, 45(6), 1202-1216.
    Google ScholarLocate open access versionFindings
  • Kenealy, P. M. (1997). Mood state-dependent retrieval: The effects of induced mood on memory reconsidered. The Quarterly Journal of Experimental Psychology: Section A, 50(2), 290-317.
    Google ScholarLocate open access versionFindings
  • Kinateder, M. T., Kuligowski, E., Reneke, P. A., & Peacock, R. D. (2014). A review of risk perception in building fire evacuation. NIST Technical Note. Kinateder, M., Ronchi, E., Nilsson, D., Kobes, M., Muller, M., Pauli, P., & Muhlberger, A. (2014, September). Virtual reality for fire evacuation research. In Computer Science and Information Systems (FedCSIS), 2014 Federated Conference on (pp. 313-321). IEEE. 40
    Google ScholarFindings
  • Kirschner, P. A., Sweller, J., & Clark, R. E. (2006). Why minimal guidance during instruction does not work: An analysis of the failure of constructivist, discovery, problem-based, experiential, and inquiry-based teaching. Educational psychologist, 41(2), 75-86.
    Google ScholarLocate open access versionFindings
  • Klein, G. A., Calderwood, R., & Clinton-Cirocco, A. (1986). Rapid decision making on the fire ground. Paper presented at the Proceedings of the Human Factors and Ergonomics Society annual meeting.
    Google ScholarLocate open access versionFindings
  • Kontogiannis, T., & Kossiavelou, Z. (1999). Stress and team performance: principles and challenges for intelligent decision aids. Safety science, 33, 103-128.
    Google ScholarLocate open access versionFindings
  • Krantz, G., Forsman, M., & Lundberg, U. (2004). Consistency in Physiological Stress Responses and Electromyographic Activity during Induced Stress Exposure in Women and Men. Integrative Physiological & Behavioral Science, Vol(2), 105-118.
    Google ScholarLocate open access versionFindings
  • Lerch, F. J., & Harter, D. E. (2001). Cognitive support for real-time dynamic decision making. Information Systems Research, 12(1), 63-82.
    Google ScholarLocate open access versionFindings
  • McClernon, C. K., McCauley, M. E., O'Connor, P. E., & Warm, J. S. (2010). Stress Training Enhances Pilot Performance During a Stressful Flying Task. Paper presented at the Proceedings of the Human Factors and Ergonomics Society Annual Meeting.
    Google ScholarLocate open access versionFindings
  • Mehrabian, A. (1996). Pleasure-arousal-dominance: A general framework for describing and measuring individual differences in temperament. Current Psychology, 14(4), 261-292.
    Google ScholarLocate open access versionFindings
  • Neerincx, M. A. (2003). Cognitive task load design: model, methods and examples. In E. Hollnagel (Ed.), Handbook of Cognitive Task Design (pp. 283-305). Mahwah, New Jersey: Lawrence Erlbaum Associates.
    Google ScholarLocate open access versionFindings
  • Nilsson, D. Design of fire alarms: Selecting appropriate sounds and messages to promote fast evacuation. (2014) Sound, Safety & Society, 33.
    Google ScholarFindings
  • Peeters, M., Van Den Bosch, K., Meyer, J.-J. C., & Neerincx, M. A. (2014). The design and effect of automated directions during scenario-based training. Computers & Education, 70, 173-183.
    Google ScholarLocate open access versionFindings
  • Prinsloo, G. E., Derman, W. E., Lambert, M. I., & Rauch, H. G. L. (2013). The effect of a single session of short duration biofeedback-induced deep breathing on measures of heart rate variability during laboratory-induced cognitive stress: a pilot study. Applied psychophysiology and biofeedback 38, 81-90.
    Google ScholarLocate open access versionFindings
  • Raaijmakers, S. F., Steel, F. W., Goede, M. d., Wouwe, N. C. v., Erp, J. B. F. v., & Brouwer, A.-M. (2013). Heart rate variability and skin conductance biofeedback: A triple-blind randomized controlled study. Paper presented at the Humaine Association Conference on Affective Computing and Intelligent Interaction.
    Google ScholarLocate open access versionFindings
  • Reason, J. (1987). Cognitive aids in process environments: prostheses or tools? International journal of man-machine studies, 27, 463-470.
    Google ScholarLocate open access versionFindings
  • Rock, I. & Palmer, S. E. (1990) The legacy of Gestalt psychology. Scientific American, 263, 84-90.
    Google ScholarLocate open access versionFindings
  • Sasou, K., & Reason, J. (1999). Team errors: definition and taxonomy. Reliability engineering and system safety, 65, 1-9.
    Google ScholarLocate open access versionFindings
  • Schreuder, E. J., & Mioch, T. (2011, October). The effect of time pressure and task completion on the occurrence of cognitive lockup. In Proceedings of the International Workshop on Human Centered Processes 2011 (HCP 2011) (pp. 10-11).
    Google ScholarLocate open access versionFindings
  • Shute, V. J. (2008). Focus on formative feedback. Review of educational research, 78(1), 153-189.
    Google ScholarLocate open access versionFindings
  • Slater, M., Khanna, P., Mortensen, J., & Yu, I. (2009). Visual realism enhances realistic response in an immersive virtual environment. Computer Graphics and Applications, IEEE, 29(3), 76-84.
    Google ScholarLocate open access versionFindings
  • Toups, Z. O., Kerne, A., & Hamilton, W. A. (2011). The Team Coordination Game: Zero-fidelity simulation abstracted from fire emergency response practice. ACM Transactions on Computer-Human Interaction (ToCHI), 18(4), 23.
    Google ScholarLocate open access versionFindings
  • Wickens, C. D., Lee, J., Liu, Y., & Becker, S. G. (2004). An introduction to human factors engineering (second ed.). New Jersey: Pearson Education.
    Google ScholarFindings
Your rating :
0

 

Tags
Comments