Becoming a Butterfly — Part 2: Flight
In the last post, I covered the creation of the monarch butterfly avatar that players will embody in this VR game/simulation. In this post, I’ll be discussing the development of our avatar’s flight capabilities, using the Nvidia PhysX engine in Unity. As a starting point, it’s important to first understand the basics of how real monarchs — and other butterflies — actually fly.
Butterfly Flight Mechanics, a very brief and simplified primer
Prior to researching this topic, my intuitive sense of how monarchs fly was simple: it’s all about the downstroke. That is, the lift (upward force) and propulsive thrust (forward force) that keep a butterfly suspended in the air and moving forward during standard flight are all generated by the butterfly moving its wings in a fast down- and maybe slightly backward motion, almost like a swimmer generating forward motion by pushing water away in the opposite direction.
This is not how it works. Downstrokes are important, yes, but upstrokes equally so. The basic picture is that like other butterflies, monarchs generate lift through the downstroke, but thrust is achieved primarily through the upstroke. The flexible wings of a butterfly are what make this possible, as they enable what is called the “clap and fling” mechanism. This is where the butterfly cups its wings together during an upstroke, collecting an air pocket between them, and then claps both wings together at the end of the upstroke, forcing the air pocket to shoot out behind the butterfly and thrusting it forward.
You can see these mechanics at work in the video below. Importantly, butterflies do not always use the clap and fling mechanism. For example, during takeoff a butterfly might perform a sequence of powerful downstrokes to generate enough lift to bring its body clear of the ground before using the clap and fling to begin moving forward. And, of course, the amount of force generated varies.
It’s also worth noting that there are many other factors at work here, including, the change in orientation of the butterfly’s body during upstrokes and downstrokes. Moreover, much of this is only true for forward, powered flight, as opposed to hovering, reverse flight, or gliding. I’ll touch on some of this below — particularly gliding, as it is absolutely crucial for migrating monarchs — but this simple forward flight model focused on variegated upstrokes and downstrokes is sufficient for our purposes here.
Building our butterfly avatar’s flight capabilities in Unity based on our simplified flight model
A note to readers: as before, I’m embedding some short video clips of the avatar’s flight capabilities in action — but please keep in mind that the environment that you will see in these videos will bear little-to-no resemblance to the environment we’ll be using in the game. It’s really just for testing and reference at this point. In my next post I’ll talk more specifics about the forthcoming environment, based on the actual habitat of the monarch as well as my research into butterfly perception and how this will be simulated.
We now have a butterfly avatar in Unity with integrated VR controls, and we know the basics of how butterflies fly in standard forward flight. In order to combine the two, to make our avatar fly using VR hand controllers following our simplified flight model, we’ll first add a rigidbody to our avatar in order to make use of Unity’s built-in Nvidia PhysX engine. This allows us to, among other things, control the movement and rotation of our avatar through the application of forces.
Trending AR VR Articles:
1. How VR could bring transhumanism to the masses
2. How Augmented Reality (AR) is Reshaping the Food Service Industry
3. ExpiCulture — Developing an Original World-Traveling VR Experience
We’ll start with lift, essential for taking off and weight support during flight. We know that butterflies create lift primarily through the downstroke. But how do we detect downstrokes in Unity and subsequently use our physics engine to apply forces relative to the velocity of the downstroke (i.e. apply more force for a faster downstroke than a slower one)?
Since our hand controllers are determining the orientation of the avatar’s wings, to code this up we need to measure the changing positions of both controllers while a player is flapping their arms. If those positions are increasing, the controllers are moving upwards; if they’re decreasing, they are moving in a downstroke. To calculate the velocity of the downstroke, and thus the amount of force we should be applying, we can use some fairly simple vector math to calculate the velocity of each controller during the downstroke, add them together, and apply a force to our avatar’s rigidbody in the opposite direction, creating some small amount of lift for our avatar throughout the downstroke.
For the upstroke, our thrust mechanic needs to work slightly differently, since the clap and fling mechanism propels the butterfly forward in a burst at the end of each upstroke. To handle this, we measure the upward velocity of each controller during the upstroke, adding them together until the end of the upstroke when we apply our burst of forward thrust/force to our rigidbody.
We also know from our flight model that butterflies do not always use the clap and fling, so players should, for example, be able to take off from a landed position by performing a series of downstrokes to lift themselves into the air before using the clap and fling to move forward. We can accomplish this by adding a threshold to our upstroke mechanic, such that only upstrokes in which the hand controllers move above the shoulders are capable of propelling the avatar forward.
As noted above, gliding is also important for butterflies in general and absolutely crucial for migrating monarchs, who rely on energy-efficient gliding to complete their long journey to Mexico (at least for Midwestern and Eastern monarch populations). Gliding behavior produces a small amount of lift and also translates some of its downward velocity (due to gravity) into forward velocity.
But to build in a gliding mechanic we first need to distinguish between powered flight and gliding. The powered flight mechanics described above will register even tiny movements in the player’s hand controllers, so we first add a velocity threshold so we’re not counting very small changes in the positions of the controllers. With that added, we can assume that if the player is (i) in the air, (ii) with arms stretched out, and (iii) not flapping their arms/wings, we should be gliding.
One complication that needs to be addressed is rotation, specifically the rotation of the avatar’s forward vector (think of this as the direction the avatar’s body should be facing). Rotation is fairly simple when the butterfly avatar is landed — you are just rotating the avatar around its vertical/Y axis by using the controller joystick, rotating your actual body, or turning your head more than 45 degrees to the right or left. Any of those movements will rotate the avatar’s body so it is facing a new direction.
This is trickier to handle when flying.
The ideal solution would be to track the forward vector of the player’s torso (just imagine an arrow pointing straight out from the center of your chest), which would tell you which direction the player’s body was facing, irrespective of the orientation of the player’s head or hands. For our avatar, this would mean that the player could fly straight forward without rotating while looking around and moving the controllers around arbitrarily — an ideal scenario for our flying butterfly.
Unfortunately, we can’t do this with a standard VR setup — all we’ve got are the positions and orientations of our HMD (head) and controllers (hands).
So, to approximate the ideal scenario, I’ve settled on averaging the forward-pointing vectors of the HMD and controllers. If you imagine a player standing upright with arms stretched straight out to the sides and thumbs pointing forward, the forward direction of this person in my approximation would be based on the direction the player’s head and thumbs are facing, all averaged together.
The result is that you can freely look around while flying without dramatically changing the forward direction of your avatar. To change direction, you can turn your torso, which should also turn your hand controllers. This not only lets you bank to the left or right, but also to dive or turn upward, all while looking around freely.
With these basics coded up, we can begin tweaking our flight and gliding parameters so that the resulting flight of our avatar matches that of real monarchs, as closely as possible. This means that our avatar should be able to fly at normal speeds of ~2–12 mph, but with the ability to fly much faster over short distances (e.g. to avoid predators), or with a nice tailwind. It should also have a glide ratio of between 3:1 to 4:1 as a healthy, full-sized adult, meaning it moves forward 4 units for each unit it moves down due to gravity (for reference, a modern hang glider has a glide ratio of 16:1). The current version flies and glides well, but will likely need some tweaks as we add in more functionality, e.g. feeding behavior, and a more robust environment.
Simulator Sickness & User Comfort
User comfort is an important topic in VR, particularly as VR experiences have a greater tendency to cause simulator sickness in users. It is especially important here, as even with the basic flight mechanics described above, this simulation is already in the “intense” category. While not all researchers agree on the root causes of simulator sickness, for VR developers it is generally accepted that a primary trigger for many users is related to user movement and rotation in the virtual world, particularly when this does not match the users’ movement and/or rotation in the physical world.
In this simulation, players can fly relatively quickly (even 12 mph is quite fast when you consider how small a monarch is), bank around sharp corners, and land and take off from vertical surfaces. This means that there is a good chance it could cause simulator sickness in certain sensitive players.
In an effort to avoid this, I’ve added a vignette mechanic to this prototype that automatically activates when the player is turning. This can be more comfortable for users as it essentially narrows the user’s field of view, masking peripheral visual cues that trigger simulator sickness for many.
The mask I’m using for the vignette is based on a compound eye simulator called toBeeView, which takes images and generates a simulated version as seen through the compound eyes of a bee, butterfly, or other arthropod. (Players will be able to turn this feature off in the simulation.)
That’s all for this month’s post.
Up next: our first public build, and building a new environment as seen by a butterfly
Sources:
Agrawal, Anurag. Monarchs and Milkweed: A Migrating Butterfly, a Poisonous Plant, and Their Remarkable Story of Coevolution. Princeton University Press, 2017. JSTOR, www.jstor.org/stable/j.ctvc775wc. Accessed 29 Apr. 2021.
Dudley R, Srygley R. Flight Physiology Of Neotropical Butterflies: Allometry Of Airspeeds During Natural Free Flight. J Exp Biol. 1994 Jun;191(1):125–39. PMID: 9317473.
Dużmańska N, Strojny P, Strojny A. Can Simulator Sickness Be Avoided? A Review on Temporal Aspects of Simulator Sickness. Front Psychol. 2018;9:2132. Published 2018 Nov 6. doi:10.3389/fpsyg.2018.02132
Fei YH, Yang JT. Enhanced thrust and speed revealed in the forward flight of a butterfly with transient body translation. Phys Rev E Stat Nonlin Soft Matter Phys. 2015 Sep;92(3):033004. doi: 10.1103/PhysRevE.92.033004. Epub 2015 Sep 4. PMID: 26465553.
Gibo, David & Pallett, Megan. (2011). Soaring flight of monarch butterflies, Danaus plexippus (Lepidoptera: Danaidae), during the late summer migration in southern Ontario. Canadian Journal of Zoology. 57. 1393–1401. 10.1139/z79–180.
Johansson L. C. and Henningsson P. 2021 Butterflies fly using efficient propulsive clap mechanism owing to flexible wings J. R. Soc. Interface.182020085420200854 http://doi.org/10.1098/rsif.2020.0854
Monarch Watch : Reading Room : Frequently Asked Questions. https://www.monarchwatch.org/read/faq2.htm. Accessed 28 Apr. 2021.
Srygley, R., Thomas, A. Unconventional lift-generating mechanisms in free-flying butterflies. Nature 420, 660–664 (2002). https://doi.org/10.1038/nature01223
Yokoyama, Naoto & Senda, Kei & Iima, Makoto & Hirai, Norio. (2013). Aerodynamic forces and vortical structures in flapping butterfly’s forward flight. Physics of Fluids. 25. 021902. 10.1063/1.4790882.
Don’t forget to give us your 👏 !
