Leveraging VR in Behavioral Research, How Flies react to VR

Behavioral research has always faced a critical balancing act between ecological validity and experimental control. Traditional methods of studying animal behavior in controlled laboratory environments often compromise naturalistic behavior, while field studies, though authentic, lack the precision needed for mechanistic analyses. The new research included biologists, neuroscientists, and software experts, including Flinders University researchers Dr. Yuri Ogawa, Dr. Richard Leibbrandt, Raymond Aoukar, and Jake Manger from The University of Western Australia and experts from Germany

Enter Virtual Reality (VR), a groundbreaking tool reshaping the landscape of behavioral studies. By creating simulated yet interactive environments, VR combines the best of both worlds, offering precise experimental control alongside more naturalistic behaviors. From tethered flight experiments with hoverflies to trackball-based tests with fiddler crabs, VR is proving to be an invaluable asset in uncovering the intricate dynamics of animal behavior. 

Let’s explore how this transformative technology is enabling researchers to push boundaries while addressing long-standing challenges.

The Role of Vision in Dynamic Animal Behaviors

Animals rely heavily on visual cues to perform a variety of dynamic behaviors such as course stabilization, obstacle avoidance, homing, and predator evasion. Traditional laboratory setups have been instrumental in analyzing these behaviors, but their constrained nature often raises questions about ecological relevance.

Alternatively, studying animals in their natural habitat provides authentic insights but lacks the experimental control needed for detailed analyses. This is where VR stands out. It bridges the gap, offering the freedom to replicate naturalistic behaviors while maintaining control over experimental variables.

Leveraging VR for Behavioral Research

Recent advancements in VR technology have made it possible to conduct highly detailed behavioral studies. In VR, animals experience a simulated world where their actions influence the visual scenery, creating an immersive experience. For instance, a tethered Eristalis tenax can “navigate” through a virtual environment while remaining stationary, simplifying monitoring and quantification.

Key Advantages of VR in Research:

1. Precision and Control: VR allows systematic manipulation of parameters, enabling researchers to pinpoint specific visual triggers for behaviors.
2. Enhanced Safety and Accuracy: With animals tethered, risks are minimized, and responses are easier to quantify.
3. Flexibility in Stimuli Design: Computer-generated visual worlds are easy to modify, accommodating diverse experimental needs.

Tethered Flight Experiments: Insights from Hoverflies

In one study, female hoverflies aged 14–24 days were tethered at a 32° angle using a bee’s wax and resin mixture. Researchers manually provided airflow to encourage flight, placing the hoverflies 10 cm from the center of each monitor in the VR setup.

The hoverflies responded to various visual stimuli, including:

• Sinusoidal gratings (20° wavelength, 5 Hz frequency)
• Starfield Stimulus (yaw rotation at 50°/s)

  

• Bars of (width 3°, varying heights 0.8°–142°)

Using DeepLabCut (DLC), researchers tracked six specific points on the tegula, wing tips, anterior thorax, and anterior abdomen. The trained DLC model used 300,000 maximum iterations and achieved impressive accuracy, with training and testing errors of 1.2 pixels and 1.16 pixels, respectively. This model allowed real-time tracking and detailed quantification of wing beat amplitudes under varying video resolutions.

Trackball Experiments: Fiddler Crabs in Action

The versatility of VR extends to crustaceans as well. Fiddler crabs (Gelasimus dampieri) collected from intertidal mudflats near Broome, Western Australia, were tested in a trackball configuration. These crabs were exposed to two virtual objects:

1. A virtual crab, mimicking a natural conspecific near an invisible burrow.
2. A virtual bird, represented by a 3 cm black sphere, simulating predator threats.

The virtual bird exhibited three behaviors:

• Threatening: Approaching in a straight line at 19.9 cm/s.
• Non-threatening (Stationary): Remaining in the starting position.
• Non-threatening (Arcing): Moving back and forth along a 30° arc.

Each trial lasted 60 seconds, with randomized sequences to prevent order bias. Closed-loop conditions, where the visual stimuli adjusted based on the crab’s movements, provided deeper insights into their responses.

“We developed computer programs that create a virtual reality experience for the animals to move through,” says Dr. Ogawa, a Research Fellow in Neuroscience at the Flinders Health and Medical Research Institute

Overcoming Challenges in VR Implementation

To ensure an immersive VR experience, minimizing delays in the feedback loop is crucial. For humans, delays over 60 ms can cause cybersickness and disrupt the sense of body ownership at 100 ms. Interestingly, Drosophila can tolerate delays of up to 2 seconds, though this threshold remains unquantified for other insects.

The use of Unity as the primary VR platform, integrated with machine vision tools like DLC-live and FicTrac, further enhances usability. Unity’s compatibility with the CAVE interface simplifies experiment design, enabling users to create and analyze detailed visual trials without coding expertise.

The Future of VR in Behavioral Studies

While VR systems have come a long way, there’s still room for improvement. For example, the Antarium, a display optimized for ant vision, highlights the potential of species-specific visual systems. Future developments could include a repository of naturalistic scenes, better perspective correction, and displays tailored to different animals’ color vision.

By making VR tools and interfaces open-source on platforms like GitHub, the research community can collaborate to refine and expand the possibilities of VR in behavioral science.

Transforming Research Through VR

The integration of VR in behavioral research is unlocking new dimensions of understanding. By combining naturalistic behavior with experimental precision, VR empowers researchers to tackle complex questions across species. To learn more, explore our detailed manuals, tutorials, and starting resources on GitHub.