A team of marine scientists employ extended reality (XR) to revolutionize people experiences in marine education. This UGC-funded project is led by Prof. Sai-Kit Yeung (Integrative Systems & Design) and Prof. Cindy Lam (Department of Ocean Science) at HKUST, with joint collaboration of Prof. Apple Chui (School of Life Sciences) from CUHK, Prof. Jianwen Qiu and Dr. Veronica Chan (Department of Biology) from HKBU, and Prof. Cedric Tan (Environmental and Geographical Sciences) from the University of Nottingham Malaysia.
The project aims to design a game app with immersive Virtual Reality (VR) eLearning tools with AI machine-learning platform “Marine GPT”. The platform contains marine-specific data and information which requires players to learn right before the game starts. This would engage students’ interactions, learning motivation, ideas exchange and gain insight on marine special topics and beyond. The game framework is on level-based, the player must complete certain objectives, such as defeating enemies or solving scenario-based challenges in order to progress to the next level. The game has 4 levels in total, starting from intertidal habitats such as mangroves and mudflats, boulder shore (Level 1) to subtidal coral reefs (Level 2) and then to the deep sea (Level 3) in Hong Kong. After completion of Level 3, players will step into the next level to learn similar coastal marine habitats in Malaysia (Level 4). The game allows the players to recognize marine biodiversity and ecosystems between tropical and subtropical biomes.
In the game, 3D models of marine organisms inhabiting different marine habitats are developed, as well as the game scene of their living habitats. To prepare for the real scene, we will do the data capture of intertidal habitats using air-drone to capture video of overall topography and surface, while the data capture of subtidal habitats using underwater drone with multiple cameras for wide-angle video capture. After that, we can use the 3D reconstruction system to get the 3D models. However, the deep sea scene and 3D models are mostly referred from the reference videos provided by the team members.
We hope our Extended Reality eLearning platform can inspire marine research community to further explore and increase awareness of the general public on marine biodiversity and conservation.
We are hiring! If you are an ocean lover and want to develop new technology to solve different marine related problems, please email [email protected] and [email protected] to join our program.