Practicing in the Age of AI: What Three Years’ Research Actually Tells Us

Over the past three years, practice-based learning has changed quickly. AI tools can now simulate customers, patients, and coaching conversations on demand. At the same time, organizations are still investing in live role-play, simulations, and practice with facilitators or professional actors.

Lately, I keep hearing the same practical question: Does AI-enabled practice work as well as human-led practice — and when does each make sense? Here’s what the research says.

In technical environments, learners who received AI-generated feedback improved more than those coached by experts.

Where AI-enabled practice is genuinely strong

First, it’s worth saying clearly: AI-enabled practice is not just a budget alternative. In the right conditions, it really works.

A 2025 study of 487 university students examined how generative AI tools affected engagement, motivation, and retention. The findings were straightforward: when used intentionally, AI tools were associated with deeper cognitive engagement, stronger emotional investment, and better retention over time.

In technical environments, the findings are even more tangible. In a randomized surgical training study, learners who received AI-generated feedback improved slightly more than those coached by expert instructors. Why? Because the AI delivered highly specific, metric-based feedback consistently — every time, for every learner.

Other research in medical education shows that AI-powered virtual patients and chatbots can improve clinical reasoning and certain communication skills — particularly when learners can practice repeatedly and reflect between attempts.

Taken together, the findings point to a simple pattern. AI-enabled practice tends to shine when:

• The skill can be clearly defined and broken into observable steps

• Success can easily be measured objectively

• Learners would benefit from repetition and immediate feedback

• Scale and scheduling make human practice hard to deliver consistently

In other words, AI works very well when clarity and repetition matter most.

Where human-led practice still has the edge

Human-led simulation has decades of research behind it, especially in health professions and communication training.

Systematic reviews consistently show that high-fidelity simulations — often involving trained actors or facilitators — lead to better performance and stronger retention than traditional classroom instruction. Learners don’t just understand more. They perform better in realistic situations.

When studies compare actor-based simulations with automated virtual patients, both improve skills. But learners typically report higher satisfaction and stronger perceived communication gains when practicing with real people.

And when communication skills that are more nuanced, such as emotion, resistance, and ambiguity, human counterparts still tend to produce stronger outcomes. Based on my observations since 2005, I would also add that learning retention and application are enhanced by human-led practice scenarios.

This matters because many of the durable skills organizations care most about today are deeply interpersonal:

• Leadership presence

• Coaching conversations

• Navigating emotion

• Building trust under pressure

These skills aren’t about being technically correct. They’re about the impact you have on another person. So where does that leave us?

AI-enabled practice excels when skills are structured, measurable, and repeatable. It offers scale, consistency, and data.

Human-led practice excels when the skill involves nuance, emotion, and confidence in high-stakes conversations.

And increasingly, the strongest designs blend both. Several researchers suggest using AI for foundational skill-building and repetition, then layering in human-led practice for advanced interpersonal work and critical scenarios.

What this means for your learning strategy

If you’re designing practice for procedural or knowledge-heavy skills, AI-enabled tools are often a strong starting point. But, if you’re designing practice for leadership, sales, or client-facing conversations, human-led simulation still carries distinct advantages.

And if the initiative is strategically important — where both precision and presence matter — a blended design is often worth the extra thought.

If you’re curious what this could look like in your organization, the most useful step is usually seeing both approaches side by side.

If it’s helpful, we can offer you a free, immersive demo of both AI-enabled and human-led practice. Experiencing them side by side is more useful than me talking about them in the abstract!

For more detail: https://docs.google.com/document/d/1PnkxBLgEEHPyTbX_WlP306mj_QujU_Hm/edit?usp=sharing&ouid=112987587788372553059&rtpof=true&sd=true

Complete Citations and References (last updated 26-01-2026):

1. [1] Human-Delivered Conversation Versus AI Chatbot Conversations for Health Education. NIH/PMC (2025-10-16). Retrieved from https://pmc.ncbi.nlm.nih.gov/articles/PMC12538107/

2. [2] Comparing AI chatbot simulation and peer role-play for OSCE preparation. JMIR Medical Education; NIH/PMC (2025-11-23). Retrieved from https://pmc.ncbi.nlm.nih.gov/articles/PMC12750692/ ; https://pubmed.ncbi.nlm.nih.gov/41286823/

3. [3] Automated Virtual Patient Simulation Versus Actor-Based Simulation for Teaching Consultation Skills: A Multicenter Randomized Crossover Study. JMIR Formative Research (2025). Retrieved from https://formative.jmir.org/2025/1/e71667

4. [4] Effectiveness of high-fidelity simulation on practice, satisfaction, and psychological outcomes. PMC (2025-05-30). Retrieved from https://pmc.ncbi.nlm.nih.gov/articles/PMC12125790/

5. [5] The Benefits of High-Fidelity Simulation in Healthcare Education. EMS-Works (2025-05-28). Retrieved from https://ems-works.com/blog/content/the-benefits-of-high-fidelity-simulation-in-healthcare-education/

6. [6] AI vs. Human Trainers: The Power of Both in Coaching. TrueCoach (2025-05-01). Retrieved from https://truecoach.co/blog/ai-vs-human-trainers-the-power-of-both-in-coaching/

7. [7] AI Coaching vs. Traditional Management Training. Cloverleaf (2025-11-03). Retrieved from https://cloverleaf.me/blog/ai-coaching-vs-traditional-management-training/

8. [8] How Blended Learning Enhances Training for a Hybrid Workforce. Neovation (2022-12-31). Retrieved from https://www.neovation.com/learn/blended-learning-for-a-hybrid-workforce

9. [9] Effect of Artificial Intelligence Tutoring vs Expert Instruction on Surgical Skill Acquisition. PMC (2022-02-21). Retrieved from https://pmc.ncbi.nlm.nih.gov/articles/PMC8864513/

10. [10] AI-guided feedback outperforms human instructors in neurosurgical training. Medical Xpress (2024-09-03). Retrieved from https://medicalxpress.com/news/2024-09-ai-feedback-outperforms-human-instructors.html

11. [11] Yang, H. (2025). Harnessing Generative AI: Exploring its impact on cognitive engagement, emotional engagement, learning retention, reward sensitivity, and motivation through reinforcement theory. Learning and Motivation, 90, 102136. Retrieved from School of Humanities and Social Science, North University of China.

12. [12] Comparing an AI Chatbot-Based Simulation With Peer Role-Play for Clinical Communication Skills Training: Randomized Controlled Pilot Trial. JMIR Medical Education (2023-01-15). Retrieved from https://pmc.ncbi.nlm.nih.gov/articles/PMC12750692/

13. [13] Unlock Training Success with AI-Driven Training Simulations. Reachum (2025-10-13). Retrieved from https://reachum.com/articles/the-value-of-ai-driven-simulations-for-training-daadr

14. [14] Blended Learning 2.0: Integrating AI with Traditional Learning Strategies. Mindsmith (2025-12-21). Retrieved from https://www.mindsmith.ai/blog/blended-learning-2-0-integrating-ai-with-traditional-learning-strategies

15. [15] Measuring Learning Effectiveness and ROI with AI. SweetRush (2024-12-31). Retrieved from https://www.sweetrush.com/articles/measuring-learning-roi-ai

16. [16] Using Real-time Feedback To Improve Surgical Skill Learning. PMC (2022-10-21). Retrieved from https://pmc.ncbi.nlm.nih.gov/articles/PMC9732447/

17. [17] Impact of blending traditional and digital training methods on ROI. VoreCol (2024-08-27). Retrieved from https://vorecol.com/blogs/blog-impact-of-blending-traditional-and-digital-training-methods-on-roi-10772

18. [18] Why Blended Learning Is the Future of Corporate Training Programs. Aptara Corp (2025-12-10). Retrieved from https://www.aptaracorp.com/why-blended-learning-is-transforming-corporate-training/

19. [19] Effectiveness of Simulation-Based Learning in Health Professions Education: A Review. BMC Medical Education/PMC (2024). Retrieved from https://pmc.ncbi.nlm.nih.gov/articles/PMC11224887

20. [20] The Impact of Generative AI Tools on Student Engagement and Learning Retention. Learning and Motivation (2025). Retrieved from https://www.sciencedirect.com/science/article/abs/pii/S0023969025000438

21. [21] Artificial Intelligence in Simulation-Based Learning: A Systematic Mapping Review. Computers & Education: Artificial Intelligence (2022). Retrieved from https://www.sciencedirect.com/science/article/pii/S2666920X2200042X

22. [22] The Value of AI-Driven Simulations for Training. Reachum (Industry L&D publication). Retrieved from https://reachum.com/articles/the-value-of-ai-driven-simulations-for-training

23. [23] Comparing the Effectiveness of Role-Play Simulation versus Real Patient Training. Advances in Medical Education and Practice (2023-09-30). Retrieved from https://www.tandfonline.com/doi/full/10.2147/AMEP.S410793

24. [24] Artificial Intelligence-Based Feedback Improves Surgical Skill Acquisition Compared With Expert Instruction: A Randomized Controlled Trial. NPJ Digital Medicine (2022). Retrieved from https://www.nature.com/articles/s41746-022-00649-2

25. [25] How does artificial intelligence compare to human feedback? A systematic review and meta-analysis. Learning and Instruction (2025-09-23). Retrieved from https://www.tandfonline.com/doi/full/10.1080/01443410.2025.2553639

26. [26] Exploring the Impact of Artificial Intelligence Chatbots on Human Learning. SAGE Open (2025-05-25). Retrieved from https://journals.sagepub.com/doi/10.1177/21582440251340615

27. [27] The emotional cost of AI chatbots in education: Who benefits and who is left behind. Science Direct (2025). Retrieved from https://www.sciencedirect.com/science/article/pii/S2949882125000659

28. [28] Pedagogical Influence of AI-Chatbots on Learning Outcomes: A Systematic Review. IJEM (2025-10-14). Retrieved from https://www.ijem.com/pedagogical-influence-of-ai-chatbots-on-learning-outcomes-a-systematic-review

29. [29] Artificial Intelligence-Virtual Trainer: Innovative Didactics for Medical Education. PMC (2022-02-23). Retrieved from https://pmc.ncbi.nlm.nih.gov/articles/PMC8865945/

30. [30] Measuring the effectiveness of virtual training: A systematic review. Science Direct (2022). Retrieved from https://www.sciencedirect.com/science/article/pii/S294967802200006X

31. Automated Virtual Patient Simulation Versus Actor-Based Simulation for Teaching Consultation Skills: A Multicenter Randomized Crossover Study — JMIR Formative Research (2025)
    https://formative.jmir.org/2025/1/e71667

32. Comparing an AI Chatbot–Based Simulation With Peer Role-Play for Clinical Communication Skills Training — JMIR MedEd (2023)
    https://mededu.jmir.org/2023/1/e45338

33. AI-Based Feedback Improves Surgical Skill Acquisition Compared With Expert Instruction — NPJ Digital Medicine (2022)
    https://www.nature.com/articles/s41746-022-00649-2

34. Human–AI Interaction Dynamics in Role-Play Conversations — arXiv Preprint (2025)
    https://arxiv.org/abs/2509.01914

35. Effectiveness of Simulation-Based Learning in Health Professions Education: A Review — PMC (2024)
    https://pmc.ncbi.nlm.nih.gov/articles/PMC11224887

36. The Impact of Generative AI Tools on Student Engagement and Learning Retention — Learning & Motivation (2025)
    https://www.sciencedirect.com/science/article/abs/pii/S0023969025000438

37. Artificial Intelligence in Simulation-Based Learning: A Systematic Mapping Review — Computers & Education: AI (2022)
    https://www.sciencedirect.com/science/article/pii/S2666920X2200042X

38. The Value of AI-Driven Simulations for Training — Reachum Industry Article
    https://reachum.com/articles/the-value-of-ai-driven-simulations-for-training