3D Large-Screen Immersive Video Mapping Installations & Interactive Light Game for Conference Seminars

Iwan Awaludin, Muhammad Rizqi Sholahuddin, Trisna Gelar

Abstract


Technological improvements, especially in immersive technologies such as spatial augmented reality (SAR), commonly known as video mapping, have generated opportunities in the arts and performance industry, for example for exhibitions. This study examines the incorporation of interactive video mapping into conference seminars, emphasizing the design of large-screen 3D immersive installations and interactive light games in a college auditorium environment.  The research employs a User-Centered Design (UCD) methodology, integrating user feedback during the design and prototyping phases to develop an engaging and immersive experience for conference participants.  The project entails the development of two immersive themes: Forest and Underwater, utilizing 3D modeling and animation techniques.  MediaPipe incorporates interactive features, enabling participants to participate in the exhibits via hand gestures, augmenting involvement and participation during conference panel sessions and breaks.  The successful performance of these interactive zones and solutions for technical setup and optimization of high-quality, interactive 3D video mapping experiences in conference venues are examined.


Keywords


Video Mapping; Conference Seminars; Immersive; Interactive; Mediapipe

References


L. T. De Paolis, S. Liaci, G. Sumerano, and V. De Luca, “A Video Mapping Performance as an Innovative Tool to Bring to Life and Narrate a Pictorial Cycle,” Inf., vol. 13, no. 3, pp. 1–11, 2022, doi: 10.3390/info13030122

I. P. Sari, A. Juhana, and N. Nurhidayatulloh, “Global trends in the study of projection mapping technology using bibliometric analysis,” J. Print Media Technol. Res., vol. 12, no. 4, pp. 219–229, 2023, doi: 10.14622/JPMTR-2309

R. Yulianto and Rudianto, “Pemanfaatan Video Mapping dalam Kegiatan Bisnis,” Bisnis Teknol. Politek. NSC Surabaya, vol. 1, pp. 42–46, 2014.

A. Pahrulroji and Z. S. Soeteja, “A Case Study on Behavior of Performing Art Appreciators in Public Space: Indonesian Puppet (Wayang) Performance Arts and Video Mapping,” J. Ind. Prod. Des. Res. Stud., vol. 1, no. 2, pp. 65–78, 2023, doi: 10.17509/jipdrs.v1i2.53657

J. Heybert Rompas, S. RUA Sompie, S. D. E Paturusi, T. Elektro Universitas Sam Ratulangi Manado, and J. Kampus Bahu-Unsrat Manado, “Penerapan Video Mapping Multi Proyektor Untuk Mempromosikan Kabupaten Minahasa Selatan,” J. Tek. Inform., vol. 14, no. 04, p. 4, 2019.

I. H. B. Wicaksana, “Application of Video Mapping Technique in Kotak Band Video Music,” Bus. Econ. Commun. Soc. Sci. J., vol. 4, no. 2, pp. 89–96, 2022, doi: 10.21512/becossjournal.v4i2.7963

De Paolis, Lucio Tommaso, Silvia Liaci, Giada Sumerano, and Valerio De Luca. "A Video Mapping Performance as an Innovative Tool to Bring to Life and Narrate a Pictorial Cycle." Information 13, no. 3, 2022. doi: https://doi.org/10.3390/info13030122

F. I. Wijaya, S. P. Ramadina, and A. Simarmata, “Video Mapping Application in Sea Life Experience Interior Design as Education and Recreation Facilities,” Lect. Notes Electr. Eng., vol. 1029 LNEE, pp. 779–789, 2023, doi: 10.1007/978-3-031-29078-7_68

Young, Millie. "Drawn to 360°: How can the aesthetics and qualities of traditional 2D animation storytelling add to the immersive VR projection paradigm?." Animation Practice, Process & Production 7, no. 1, pp. 11-40, 2018.

Hinterwaldner, Inge. The systemic image: A new theory of interactive real-time simulations. MIT Press, 2023.

V. Kriznar, M. Leskovsek, and B. Batagelj, “Use of Computer Vision Based Hand Tracking in Educational Environments,” 2021 44th Int. Conv. Information, Commun. Electron. Technol. MIPRO 2021 - Proc., pp. 804–809, 2021, doi: 10.23919/MIPRO52101.2021.9596976

K. Naganandhini, B. Sowmya, S. Pavish, J. Nitesh, R. Karthika, and E. Prabhu, “Hand Tracking Based Human-Computer Interaction Teaching System,” 2022 3rd Int. Conf. Issues Challenges Intell. Comput. Tech. ICICT 2022, pp. 1–5, 2022, doi: 10.1109/ICICT55121.2022.10064617

C. H. Yeh, W. C. Shen, C. W. Ma, Q. T. Yeh, C. W. Kuo, and J. S. Chen, “Real-time Human Movement Recognition and Interaction in Virtual Fitness using Image Recognition and Motion Analysis,” Proc. 2023 12th Int. Conf. Aware. Sci. Technol. iCAST 2023, pp. 242–246, 2023, doi: 10.1109/iCAST57874.2023.10359266

Q. Huang et al., “A Finger Motion Monitoring Glove for Hand Rehabilitation Training and Assessment Based on Gesture Recognition,” IEEE Sens. J., vol. 23, no. 12, pp. 13789–13796, 2023, doi: 10.1109/JSEN.2023.3264620

N. Zerrouki et al., “Deep Learning for Hand Gesture Recognition in Virtual Museum Using Wearable Vision Sensors,” IEEE Sens. J., vol. 24, no. 6, pp. 8857–8869, 2024, doi: 10.1109/JSEN.2024.3354784

T. Kuflik et al., “A visitor’s guide in an active museum: Presentations, communications, and reflection,” J. Comput. Cult. Herit., vol. 3, no. 3, 2011, doi: 10.1145/1921614.1921618

P. J. Fahy, Media characteristics and online learning technology. Athabasca University Press, 2004.

Mulders, Miriam, J. Buchner, and M. Kerres. "A framework for the use of immersive virtual reality in learning environments." International Journal of Emerging Technologies in Learning (iJET) 15, no. 24, pp. 208-224, 2020. doi: 10.1109/ACCESS.2021.3058735

T. Keinonen, “User-centered design and fundamental need,” ACM Int. Conf. Proceeding Ser., vol. 358, pp. 211–220, 2008, doi: 10.1145/1463160.1463183

E. Duque, G. Fonseca, H. Vieira, G. Gontijo, and L. Ishitani, “A systematic literature review on user centered design and participatory design with older people,” IHC 2019 - Proc. 18th Brazilian Symp. Hum. Factors Comput. Syst., 2019, doi: 10.1145/3357155.3358471

F. Zhang, A. Doroudian, D. Kaufman, S. Hausknecht, J. Jeremic, and H. Owens, “Employing a user-centered design process to create a multiplayer online escape game for older adults,” Lect. Notes Comput. Sci. (including Subser. Lect. Notes Artif. Intell. Lect. Notes Bioinformatics), vol. 10298, pp. 296–307, 2017, doi: 10.1007/978-3-319-58536-9_24

Torin Blankensmith, “MediaPipe TouchDesigner Plugin.” 2023. Available: https://github.com/torinmb/mediapipe-touchdesigner




DOI: https://doi.org/10.17509/coelite.v3i2.76253

Refbacks

  • There are currently no refbacks.


Journal of Computer Engineering, Electronics and Information Technology (COELITE)


is published by UNIVERSITAS PENDIDIKAN INDONESIA (UPI),
and managed by Department of Computer Enginering.
Jl. Dr. Setiabudi No.229, Kota Bandung, Indonesia - 40154
email: coelite@upi.edu
e-ISSN: 2829-4149
p-ISSN: 2829-4157