On March 6th, 2020, the IEEE VR announced a venue change on their website from Atlanta, Georgia to online in virtual reality for the academic conference set to happen between 22-26 March, 2020. The organizers of the IEEE VR conference had two weeks to translate their existing, co-located schedule 1-to-1 into a virtual conference. Luckily Blair MacIntyre had already been planning a supplementary online, VR portion of the conference using a developmental release of Mozilla Hubs Cloud Enterprise that would allow for people to use their phones, tablets, laptops, PCs, or virtual reality devices to have immersive, virtual reality experiences.
I had a chance to attend all five days of the IEEE VR conference while immersed within virtual reality providing real-time feedback and commentary on Twitter, and exploring what virtual reality has to offer the art of gathering in the pandemic age of virtual conferencing. This essay will unpack some of my deeper thoughts, experiences, & reflections from IEEE VR based upon a 90-minute conversation that I just had with a couple of the general chairs of the conference.
The majority of the IEEE VR conference schedule was talks in three different tracks, and they primarily used Zoom meetings that were rebroadcast out to three concurrent Twitch streams, and then were later archived onto their IEEE VR YouTube channel.
They primarily focused on creating virtual spaces to co-watch the live Twitch streams, but this was not as successful or useful as they originally thought it was going to be due to limited time to interact, the unclear architectural affordances that made clear how audio was transmitted, and violating of normative standards of presentations by trying to multiple, concurrent audio streams happening at once. We can really only pay attention to one audio stream at a time, and so users didn’t having as many social interactions in presentation spaces, especially in the absence of clear audio-isolated areas.
The poster sessions were particularly successful, and they were able to generate some open-ended social VR spaces for gathering, but overally it was difficult for the IEEE VR conference to get a critical mass of participation in the online portion or to foster emergent conversations.
I ended up spending a significant amount of time at the IEEE VR conference immersed within the virtual reality experience experimenting and exploring different immersive architectures that would facilitate emergent conversations. I realized that a shared context and shared intention weere key components. There were a number of Birds of a Feather gatherings that happened throughout the conference, and it ended up that setting a specific time to meet with a specific question to be answered or problem to be solved was a great catalyst for gathering within a VR space in Mozilla Hubs in order to facilitate a group conversation.
I had a chance to sit down with two of the general chairs of the IEEE VR conference, Kyle Johnsen and Blair MacIntyre to unpack some of lessons learned from IEEE VR conference. It’s possible to do a 1:1 translation of a co-located schedule into a virtual conference, but the tools to best transmit the content of the talks end up being the existing and well-established, 2D video broadcast teleconferencing tools of Zoom rebroadcast to a Twitch or YouTube livestream. But what about all of the embodied conversations and in between spaces? How can virtual reality start to replicate some of these more emergent hallway conversations? The IEEE VR’s implementation using Mozilla Hubs is a start, and there were additional insights and innovations that came out of the Laval Virtual’s online conference a month later that used the Virbella platform.
LISTEN TO THIS EPISODE OF THE VOICES OF VR PODCAST
There are a lot of tradeoffs and optimizations that need to be made for an online conference that starts with the underlying intentions and purpose of gathering. I’m going to try to break down some of the tradeoffs of virtual conferences through the lens of my experiential design framework that breaks phenomenological experiences down into Social & Mental Presence, Active Presence, Embodied Presence, and Emotional Presence.
MENTAL PRESENCE TRADEOFFS
For me, I tend to break down the essence of a conference gathering between the content of what’s being discussed as well as the relational dynamics that can be cultivated with the people who are there. Most virtual conferences so far have focused 85-90% of their attention on translating and transmitting the content through livestreamed talks, but less effort on how to replicate the relational and social dynamics that naturally emerge when people are sharing physical space with each other with their full and focused attention and intention to be open for new connections and new possibilities for collaboration. So for an academic conference, there are multiple tiers of peer-reviewed content to be shared with the wider community ranging from accepted talks, posters, panels, demos, and workshops.
The poster sessions are able to work particularly well in VR because the poster sets a specific context that is spread out over a space. There were also demo areas, but it’s really impossible to demo emerging VR hardware solutions through the lens of another piece of VR hardware. You have to try it out natively on that original piece of VR hardware, and so demos are one of the hardest thing to translate into a virtual conference. The best that you can do is to record and share a 2D representation of the experience, which can’t ever really fully replicate a direct embodied experience.
In terms of transmitting the schedule and information about the gathering, there was 100% reliance upon the 2D website, and if you were only seeing the experience while immersed within VR, then it was difficult to know when and where things were happening. So I’d like to see more information architecture innovations for thinking about how to communicate when and where events are happening. Sites like The Wave VR have a series of upcoming screenings, AltSpaceVR has a curated listing of upcoming events that are happening that day, and VR Education’s ENGAGE platform all have a scheduled listing of upcoming events. So I’d like to see more thought in terms of translating a schedule into an embodied, immersive space taking inspiration for how SXSW and GDC create large-scale schedules that can be interpreted at a glance for what’s happening at any moment and where.
SOCIAL PRESENCE TRADEOFFS
In terms of social presence, there’s a challenge for how to discover where emergent social gatherings are happening. IEEE VR had a natural discovery mechanism for showing which rooms had clusters of conversations, but Johnsen said that there needs to be better ways of signalling that you’re open to conversation at specific times or when you’re in a specific location or context. I noticed that it takes a lot of dedication to hang out in empty social VR space by yourself at a conference waiting for someone to collide with you in order to seed social conversations. At least once I was able to help seed a conversation that started with just two other people, but soon expanded out to over 20 people in a room.
The sponsor rooms that had videos showing their latest research ended up being a great catalyst for gathering organically emergent conversational clusters that allowed for meeting new people who were 1-degree of separation away or for lurkers to edge their way into existing conversations. The ghost lobby feature in Mozilla Hubs was activated, which facilitated some new dynamics for lukers to do conversational discovery, but it also has some privacy tradeoffs as people could potentially be listening into conversations in a disembodied way. We’re not used to letting people listen in to our conversations in an omniscient way, and this could help facilitate new lurking dynamics for introverts, but there are also risks and new normative standards for the line between public and private conversations within virtual spaces.
IEEE VR used Slack for backchannel for communications, which helped provide a persistent way to send direct messages to people. They didn’t use Discord, and so there isn’t a way to friend people on the communications medium beyond the specific context of this gathering like there is on Discord. I ended up friending many people on Twitter to use Twitter DMs. But Slack does a great job of creating a context-dependent, disposable architecture for backchannel conversations.
Slack ended up providing some interesting, ephemeral direct messaging conversations and coordination since it was difficult to run into people within an embodied virtual environment. My preference is optimize for in-world, embodied exchanges, and so I avoided using the 2D interfaces and language abstractions as I spent most of my time immersed within virtual reality experience and I didn’t use the Slack backchannel very much. So while there was a fair amount of social interactions and engagement on Slack, this wasn’t interesting or particularly new for me as a VR journalist and I tended to avoid it. But I couldn’t avoid it completely as the organizers relied heavily on Slack. They said that about half of the attendees that never logged on to Slack, and so it was easy to miss some of the more emergent aspects of the conference, especially announcements about the Birds of the Feather gatherings that were being dynamically planned throughout the conference.
Virbella has a system wide announcement tool that ends up being a pretty blunt tool that is like a pop-up ad on your 2D experience. Finding ways to send broadcast messages to everyone is an open design problem yet to be fully figured out. With both Mozilla Hubs and Virbella, there were no broadcast messages that were sent when I was in virtual reality.
The discovery of social happenings was a challenge. You had to commit yourself to hanging out into an empty room in order to attract other people to come talk to you. But since most of the rooms were too boring to hang out by yourself, then you’d have people quickly drop in and immediately drop out as there would be no one taking the leap to make themselves available for social interactions. But there was also a bug that required a hard refresh in order for your presence to show up in the master listing of room capacities, which meant that people would have to do a hard refresh every time they went to the master navigation page. I didn’t realize this until many days into the conference, and many people never realized that a hard refresh was required to get an accurate tally of who was in a room, and so the rooms were reporting that they were completely empty even when they weren’t, which only accelerated the problem of cultivating clusters of people.
I’d like to see more consideration to be able to dynamically indicate where emergent clusters of people were gathering by tying an API call to the server gathering the attendance count of rooms to a spatialized object that scales up in size relative to how many people are in a room. This will require some work in making it easier to navigate from room to room, but imagine seeing a hallway of portal doors to different rooms within VR and being able to look determine how popular they are just by looking at it in VR. VR Chat solves this with their 2D menu interface, but it’d be nice to have some of this endemic to the virtual architecture surrounding the portals to navigate between virtual worlds. But there are current limitations of even creating these navigation portals within WebXR.
85/ It seems like going from one immersive VR website to another one should be easy, but there's a long & storied history of GitHub issues on the immersive-web that need to make it safe & secure.
There's an entire repo for it here:https://t.co/iclZYlUsZChttps://t.co/PfRTbSEIWr— Kent Bye (Voices of VR) (@kentbye) March 28, 2020
WebXR Navigation is an open problem at the spec level, and so traversing from page to page is still yet to be full solved. But there’s a WebXR Navigation proposal that can be implemented, which will make traversing from WebXR room to room easier. Once this is figured out on platforms like Mozilla Hubs, then it should be easier to create portals that link different rooms and help with dealing with the native limitation of only being able to have 25 people in the same virtual space at the same time. Hubs does have the ability to traverse from room to room, but the session data isn’t transmitted, and so you have to go back to the browser interface to reconfirm that you’re actually going to the correct location. It’s not an optimal user experience flow, but it works for now to enable a primitive way to transverse from room to room.
It’s still a big open problem for how to best facilitate emergent conversations in virtual spaces. Johnsen suggested that there needs to be a better way to indicate that you’d be open and available for conversation. But this also requires being cognizant of when you are in phase of open to possibility to another phase of being not available and closed. There’s already chat dynamics of marking your status of available or not within VRChat as an example, but also from other 2D chat applications like Discord and Slack.
I suspect that a key to facilitating these types of emergent conversations will be in the establishment of a deeper context in the architectural design of spaces. Imagine that you want to have a conversation about 3DUI, then what would a theme park of 3DUI experiments look like in VR? You could create an entire interactive experience exploring different 3DUI tradeoffs to provide people with a direct experience of some of the 3DUI work that would actually best use the VR affordances.
Johnsen said that most of the content at IEEE VR was focused on 2D posters, images, and videos, and that there was actually very little VR content to directly engage with. What would a conference look like where there was a more participatory engagement with the latest 3DUI research and experiments? Could there be a way to export some of the Unity code into a WebXR experience in order to create some social dynamics around it? Or perhaps future 3DUI research will be conducted within WebXR itself to make it easier to archive and share to the rest of the research community, not only from an experiential perspective but also the perspective of sharing the source code in an open source fashion.
There needs to be more thought about about how research work can be shared in a collaborative WebXR environment like Mozilla Hubs, which then these immersive experiences could become hotspots for emergent collisions at conferences that help to establish a deeper context and more meaningful and focused conversations. How can the ongoing research of VR by academics be used to create spatialized experiences that use the medium of VR to communicate their ideas?
ACTIVE PRESENCE TRADEOFFS
When I think about active presence, then I’m focused on how my agency is being expressed, received, and reflected by the experience. The IDFA DocLab’s Caspar Sonnen told me that “the fundamental character of an interactive piece of work is that the more you put in, then the more you should get out.”
So if I dedicate myself to having a real-time, live, embodied experience at IEEE VR, then what do I get out of it? Ideally the more I put in, then the more that I get out. But the design of virtual conferences still has a long ways to go in order to achieve this. It was hard to get out as much as I was getting in because the quick translation of the conference into a virtualized experience was primarily focused on the experience of collaboratively watching Twitch streams, which didn’t naturally cultivate high-agency interactions. The times that I did feel like I was seeing this type of direct return of getting out what I put in was when I was interacting in real-time with other people.
The creation tools in Mozilla Hubs have the possibility of allowing you to express dynamic agency as you’re able to alter the room by bringing in objects from Sketchfab or Google Poly, to draw, or bring in images. But there were larger performance and security issue concerns that led to disabling these creative features.
The Mozilla Hubs Cloud architecture allows for you to create your own spaces, and it’s this creation of new spaces that could included within the overall conference experience that got me really excited. There’s an opportunity for a Burning Man build-out type of dynamic where attendees have the possibility to create customized spaces that set a specific context for social interactions, embodied exploration, communication of ideas, or the delivery of an immersive experience to be received. I personally wanted to have more interstitial threshold spaces to recreate those hallway conversations, and so I experimented with creating a room with the schedule and a hallway to go from room to room. But then I found that I didn’t want to hang out in those rooms by myself, and I didn’t set a specific time for people to meet me. So it ended up not being a compelling architecture to facilitate serendipitous collisions.
It’s these chance encounters at gatherings that make it feel like your agency of embodied movements are able to facilitate new connections that ultimately help you either solve problems you’re trying to solve or to accomplish a deeper intention with what you’re trying to create and manifest into the world. I thrive off of these types of serendipitous encounters at physical conferences, and it’s one of the most difficult things to abstract into component parts in order to replicate within virtual spaces. The emergent Birds of a Feather gatherings seemed to accomplish this in the best way.
Space Popular just hosted a discussion on civic virtual space in @MozillaHubs as part of the ground breaking @IEEEVR conference. Thank you @kentbye for coming and sharing your thoughts! pic.twitter.com/UqAoBSXwTj
— Space Popular (@SpacePopular) March 26, 2020
Lara Lesmes and Fredrik Hellberg of Space Popular held an amazing Birds of a Feather gathering exploring how to recreate public spaces within immersive architecture, and it ended up facilitating some of the most engaged and interesting conversations of the conference for me because it was able to abstract out some really interesting open problems of what makes a virtual space a “public space” in the sense that there are government-supported places that have a persistence and permanence of serving the larger public interest. Is permanence the key here? Or is it more about matching the emergent intentions of the collective with a dynamic architecture that is able to facilitate cooperation and collaboration?
A big part of the underlying character for active presence are the live elements. What is it about a live gathering that makes it live? Johnsen brought in the analogy of a live sports event where the action is emergent, and that it just isn’t the same if you watch it later. He was surprised by his reaction of not finding the pre-recorded talks as compelling as the ones that were live. I totally agree and have been trying to unpack why that might be.
From a process philosophy perspective, it’s as if there’s an emerging concresence of many things coming together to create an occurance or happening, and in that live moment you feel as if you’re actively participating in that concresence. It’s a participatory dynamic where the engagement of many people combine to create something that’s beyond the contributions of an individual. To me, a pre-recorded talk feels stilted and dead. There isn’t the excitement that there’s something at stake of a live demo, or a presentation where something could go wrong. Or like Johnsen said, that there’s an offhand comment that is very fitting for whatever is emerging at that moment. There’s a giving and a receiving that happens during a talk, and the watching of a live presentation with a live audience can actually change the dynamics of a presentation. It’s a subtle effect, but for me there’s a huge difference between the energy of a live performance in front of 1000 people, and something that was recorded in the privacy of my home office.
But what’s the essence of these live elements? Since many or most of the talks at IEEE VR were pre-recorded, my phenomenological experience was that it wasn’t compelling enough to be in VR to watch a pre-recorded talk. For me, there’s a participatory, experiential element when the audience watching something live. Part of it is having collective focused attention, and some of this can still be replicated within virtual talks with a live audience. It’s almost if these collectively shared moments also help to cultivate a deeper context for the conference that can help to facilitate deeper conversations later in the gathering. Does it matter whether the talk is live or the audience watches live? I can feel a big difference, and I really prefer the live when possible (for some time zones or language barriers make a live transmission more difficult).
But the other big aspect of the “liveness” of live talks is being able to directly engage and interact with the speaker through a question and answer. MacIntyre talked about speaking on a panel at Laval Virtual, and then having people come up to him afterwards to be able to discuss what was talked about. Because IEEE VR was abstracted into 2D livestreams, then there was actually a pretty significant barrier to facilitating those embodied interactions after a talk. Some of the IEEE VR sessions encouraged attendees to enter into a Mozilla Hubs room after a session to facilitate a group discussion or an interactive Q&A, which worked as long as there were less than 25 people who wanted to do that.
I tried to attend the Q&A after Mozilla Chief R&D Officer Sean White’s closing keynote, but the room was flooded with more than 25 people. There were audio connection issues where people would join but couldn’t hear everyone in the room, and then I had to rejoin and the room had exceeded it’s capacity and so I couldn’t get back in. Mozilla Hubs was originally architected for private contexts and to only facilitate groups of people 25 or less, and so these large-scale conference gathering use cases are going to require some fundamentally new architectures in order to facilitate the gathering of a large number of people into a shared space.
AltSpaceVR solved this very early with the ability to re-broadcast presenters into a series of sharded instances in some of their live performances featuring Reggie Watts, and it’s something that Fortnite can achieve at the scale of tens of millions of people at the same virtual event. But it’s an architecture that’s still needs to make it’s way to other social VR experiences like VRChat and potentially eventually Mozilla Hubs if they decide it’s on their product roadmap.
The final point in terms of interaction is that there isn’t currently any way to easily create scriptable or interactive content within Mozilla Hubs. It’s easier with the Mozilla Hubs Cloud instance to be able to add dynamic scripts or customized A-Frame components, but making scriptable and interactive content in Hubs is non-trivial and it’s not intuitively obvious how to bring in levels of interactive content.
There are other social VR experiences like VRChat that allow you to upload an entire Unity scene with their Udon scripting language, or experiences like Rec Room or NeosVR where there’s an entire visual scripting language that facilitates the creation of interactive content. But it’s still very early days for creating interactive social experiences within Hubs, especially due to the many security and safety risks that are introduced with this interactive content. I’d recommend checking out some of what’s happening in VRChat and Rec Room to see some of the frontiers of creating immersive experiences that are focused on games, interaction, and the expression of individual and collective agency.
EMBODIED PRESENCE TRADEOFFS
The IEEE VR wanted to optimize for performance across a broad range of different devices, which meant that the architectural components of the space had to be fairly limited. VRChat has a way to have creators upload different versions of a world that are optimized for the PC and for the Oculus Quest, and then there’s a progressive enhancement of fidelity and interactive components that different based upon what power of a device you’re on. So users can share the same virtual social world across multiple devices where the PC user would have a higher fidelity experience than the Quest user, but that there could be different versions of the world that’s optimized for whatever your system can handle.
This type of progressive enhancement isn’t implemented in Mozilla Hubs or WebXR yet, and so it either ends up being a lowest common denominator experience or an experience that is higher fidelity, but accessibility starts to break down if you don’t have a device powerful enough to handle it. I experienced a lot of IEEE VR in the Oculus Quest, and once you start to have 20-25 continuous audio streams within an experience, then the audio starts to get really choppy, frames start to drop, and it becomes more of a fragmented experience.
But there was hardly anyone using VR headsets throughout IEEE VR, and so most of my embodied interactions ended up being with what felt like a stilted, disembodied zombie representations of folks who were interacting through a 2D computer interface. I was able to have a number of really high-quality conversations despite this, but I find that I have a much higher bandwidth of communication when everyone is immersed in VR and there’s more subtle body language cues that are available for me to get a richer embodied social experience.
The audio falloff settings are hard to always know how they’re set, and this was an architectural element that MacIntyre talked about how there isn’t an intuitive sense for how sound travels in virtual spaces. We have a lot of intuitions about how sound propagates in real life, which actually helps to establish social dynamics in different spaces. Having proper falloff and audio isolation enables for clusters of people to organically gather and cluster. I appreciate a fairly exponential falloff that allows me to quickly get distance from other people, but yet still hear a slight mumble of other people talking.
The absence of good audio spatialization settings in Virbella during Laval Virtual meant that a lot of the emergent social dynamics were clustered around audio isolation bubbles that were indicated by dotted circles on the ground. These type of audio isolation bubbles in Virbella allowed for a type of dynamic sharding, which allowed up to 1200 people to be in the same social space. But the problem with this approach is that the audio was either too loud or too soft. The closing night parties of Laval Virtual had a soccer field of 100 people in there with no audio spatialization, which sounded like 100 people screaming in your ear at the same time at the same audio level. It was impossible to have a conversation unless you got a good distance away from this huge crowd.
The flip side of being too loud is being too silent. In Virbella, if you were in a room with 40 people who were spread out to different audio isolation bubbles with 100% falloff, then there was a distinct uncanniness of feeling like you were in a completely silent room even though there were dozens of people in it having conversations. I found myself preferring the underlying hum of a cocktail party where you can get a sense of the social presence of other people, but with an exponential falloff where it’s difficult to make out the specifics of what people are saying. I’ve been in many situations when I thought that the underlying hum was too loud and that I thought I’d prefer silence, but after experiencing the silent version I realized that I actually like hearing a slight background hum to give me the sense that I’m present with other people.
Proper audio spatialization and falloff is probably one of the hardest open problems in social VR spaces, and there’s still many different contexts where it works and doesn’t work. Mozilla Hubs uses a more, peer-to-peer WebRTC audio stream architecture that can only really handle 20-30 concurrent, spatialized audio streams within the same room. The audio spatialization falloff settings could be tweaked in the version of Mozilla Hubs used during IEEE VR, but this is something that will need a lot of work and further exploration to see how to set the proper falloff settings, ways to potentially scale up to more people, and different architecture of spaces that make it more clear how the sound is or isn’t isolated through the entire environment.
Another embodied presence aspect are the avatars and avatar representations. The range of default avatars in Mozilla Hubs is really quite limited, and there are tools like TryQuilt.io where you can customize the skin of your avatar. The current bar for avatar representation is being set by VRChat, and so all of the other solutions are going to be a subset of options you have in VRChat for embodied avatar representation.
I appreciated the ability to customize my avatar, but generally only about 5-10% of the people will do any type of customization, and so the default settings of avatars will end up dictating the range of identity expression that’s possible. I generally find this range to be quite limited, but I know that it’ll continue to grow and evolve over time.
Being able to add dynamic lip movements to avatar representations will make a huge difference, as right now the entire robot head bounces up and down to indicate someone is talking. Sometimes some of the more humanoid avatars in IEEE VR hubs would have their entire body bounce up and down while talking. I imagine that in the future, we’ll have a sort of early days of social VR nostalgia at these primitive avatar expressions just as we look back on the animated GIFs and Geocities type of web design on the early days of the web.
Another aspect of embodiment is the opportunity for embodied journeys through virtual spaces that allow for embodied collisions. Because most of the locomotion from room to room involved abstracted teleportation that primarily used the 2D interface, then there were very little opportunities for embodied collisions. These tended to happen the most at the sponsor areas during specific break times that were announced on Slack, during the poster and demo sessions, or hanging out within the dedicated social VR spaces. These hallway conversations are a critical part of the conference, and there needs to be more of a deliberate effort to architect spaces that facilitate these embodied interactions, but also time within the schedule where people are expected to have an embodied virtual presence.
The 1:1 translation of the schedule meant that the one-hour breaks were too short. Most people were attending to their lives, email, bio breaks, and resting from Zoom talks, and so there wasn’t a proper rhythm for people to come into the VR portion of the conference. The poster sessions were during lunch and only and hour or two, but again this was following a long block of 2-3 hours of talks. I think there needs to be longer times reflected within the schedule that alternate between the talks and different social events in VR that really use the affordances of embodied conversations. Whether that’s birds of a feather discussions, the poster sessions, the sponsor areas, or creating architectural spaces that are more reflective to contexts and topics of interest for that community.
The sponsor areas ended up doubling as social hang-out spaces, but many of these spaces were not designed to for social gatherings. How can the sponsor area create content that’s endemically interesting on it’s own to experience within VR, while at the same time facilitating emergent conversations? This requires a fundamental understanding of the architecture of emergent conversations, and how to do some deeper experiential design that uses the spatial affordances of the medium to communicate in way that’s deeper than text, photos, or video. There’s a long way to go to accomplish this, but the Virtual Market in VRChat starts to really explore some of these virtual expo dynamics.
The final point that I’d make about embodied presence for now is the relationship between the underlying architecture of a space, and the deeper context that this can help to set. The architecture of most conferences and gatherings is completely generic, and structured around time. Conference rooms are empty boxes that are valuable because they gather people together at a specific time to talk about specific topics. But how can the architecture of a space better serve as being able to set a deeper context across a broad range of different topics?
Rather than structured around time, then what if the conference was structured around space? It’d be more like an interactive theme park, where you’d go to the 3DUI portion of the park to play with the latest 3DUI innovations. Perhaps the social VR portion would have a variety of different social VR experiments to try out. Maybe the collaboration portion would be facilitating people actually collaborating on some topic. So what would theme park or immersive theater version of IEEE VR look like? How could the architecture of that version of a space start to facilitate a deeper context for more meaningful interactions? How do you translate abstract ideas into embodied architectures?
With VR, you have the capability to be a lot more like Burning Man, which creates specific art and architecture that serves the needs of that specific gathering. So dynamic and emergent structures can be built to help to focus attention and intention to help answer the emerging questions and solve the pressing problems.
EMOTIONAL PRESENCE TRADEOFFS
Emotional presence is the hardest thing to cultivate in the absence of a physical, embodied presence. These are intimate moments of being vulnerable with each other, but also cultivating deeper relationships. MacIntyre talks about the late-night karaoke ventures, or mundane conversations that start by getting coffee, but that can lead to meaningful emotional exchange and commitment that could lead to future collaboration or deeper friendships.
A lot of the cultivation of emotional intimacy has to do with communicating to others about your shared experiences of something. The most common shared experience are the tools and your experiences with the virtual world itself. There’s many self-referential conversations that are all about talking about how the conversations and experiences are impacted by that particular platform. I had a lot of conversations about Mozilla Hubs at IEEE VR, and a ton of conversations about Virbella at Laval Virtual.
This seems to be like the equivalent of talking about the weather with strangers when you travel to a place. It seems to serve as a way of accessing the simplest, surface conditions of an environment that are impacting your emotional experience. It’s joyful when things are working well, or it can be cathartic to share your frustrations when the technological limitations are preventing deeper emotional engagement.
It’s often in the in-between, threshold spaces of the hallways where these types of interactions often take place. But there’s usually little to no consideration for how to cultivate these interstitial contexts in virtual conferences to facilitate these types of intimate interactions.
The opening night parties or gatherings at the end of the day at conferences often serve this function, but it’s difficult for people to fully commit entire days for multiple days in order to facilitate these types of interactions, especially after a long day of consuming talks and suffering from Zoom fatigue.
The dynamics of working from home during IEEE VR was a relatively new dynamic for many people who were able to explore remote work for the first time as the shelter-at-home quarantines were just starting to begin. Co-located conferences get people to travel to a new location, which serves enable co-located, embodied presence that’s easy to give your full attention. There are always meetings and virtual distractions that are impossible for some to avoid, but conferences served to gather people in a shared environment where they more or less have their full and complete attention available. People also usually have a certain degree of openness to possibility for emergent and unplanned interactions. How should virtual conferences be structured in order to really optimize for having someone’s complete attention?
Cultivating a context that rewards your full presence and emotional investment with meaningful interactions that cultivate emotional intimacy is something that takes a lot of deliberate design and a broader desire from a community who wants that. This is usually an unintended side effect that organically emerges from the investment and commitment that happens with traveling to a location. But it’s also the most difficult to design for and re-create some of these emergent dynamics virtually.
The social tools and friending tools within VRChat seem to do the best job at being able to cultivate a broad range of private contexts that facilitate increasing levels of intimacy. When creating private instances, then there are rules that dictate how the social dynamics will unfold in that space. There’s Public spaces, hidden Friends+ spaces where friends can bring friends, or friends-only of the room owner, or private Invite+ rooms that require a specific invitation from someone at the party, or a private instance where the owner of the room controls who is there. These facilitate a broad range for how emergent collisions can happen in virtual spaces, and an enable the ability to drop portals from room to room, and to create different contexts that assume different levels of intimate connections with differing abilities to join different rooms.
As an example, the Friends+ dynamic is the equivalent of being at a conference party and joining a friend who is immersed in a conversation with people you don’t know, but they invite you into the conversation and you get introduced to a lot of new people. It’s these types of emergent dynamics that can be formalized within what the rules of a private instance are within VRChat, but are more difficult to emerge within the social dynamics of spaces like Hubs. There is the ability to have private rooms that aren’t publicly listed, and then friends can invite other friends. But once the URL is out, then it’s out. It’s functionally like the Friends+ rooms of VRChat, but with limited ability to make a true invite-only room or friends only room where you’d have to be directly connected to the room owner. The ability to navigate between the architecture of these room privacy settings in VRChat cultivates differing levels of intimate contexts for people to connect to each other.
I see a number of virtual gatherings choosing different social VR platforms for different contexts. So maybe the conference portion in Mozilla Hubs, but maybe there’s an after party that’s in AltSpace or VRChat or TheWave, or SomniumSpace. The Education Summit did this for their after parties. But this creates fragmentation between user identities and friends networks that don’t transfer between, and also diffuses the private and professional context into worlds that are more of a public context. So it’d be like going out to a night on the town in Las Vegas with a small group of friends, rather than going to a specific conference party where there will be the opportunity to make new connections.
I tend to prefer maintaining the professional context of a conference after party, as it quickly gets fragmented into public spaces, but it’s also really difficult to create a persistent URLs to easily allow anyone who has the URL to join. It often requires a backchannel conversation of friending people on the right program, and then receiving a real-time invite. Or the loading screens breaks up the real-time communications channels, which requires something like PlutoVR or Discord audio chat or some other live audio channel outside of VR completely. It’s still very fragmented, and difficult to go from platform and platform and still stay connected.
Conferences like GDC or SXSW have a ton of parties happening after the conference in big fancy locations, while academic conferences tend to use the default hotel party or lobby spaces. But I imagine that the future afterparty scene in VR will start to pull in popular underground locations on a variety of different social VR platforms, and perhaps getting after party invitations will start to feel like a whole game within it’s own like Sundance or SXSW.
The other aspect of immersion into a space is the time zone that you’re in. If you have to timeshift your native time zone, then this will limit your ability to be fully emotionally present for the full range of events. Traveling to a physical location forces people to adjust to that time zone, and it can be really disruptive to fully live into another time zone while you’re still at home.
During Laval Virtual, I ended shifting my schedule to invoke a sort of virtual jet leg. I ended up staying up all night the first day, and then slept during the late afternoon to then start to get up at 1am PST time in order to make the 10a CEST European time zone. I felt like I was traveling to Laval France, and I was 100% dedicated to providing my full attention. But that’s a huge commitment that doesn’t always work between different time zones and, so the concurrency dilemma between multiple time zones is a huge open problem. How can you schedule a conference in order to optimize the critical mass of attendees across multiple time zones?
The fact that it was free and available to more people increases the range of diversity and inclusion that’s possible. There still needs to be a lot of outreach to be done, as well as more consideration for how to create architectures that facilitate options that allow for a broader diversity of participation and inclusion. This not only applies to race, class, gender, but also degrees of introvertedness and extrovertedness as well as the amount of power and privilege that people enjoy.
MacIntyre talks about some of the intention of gathering at conferences at IEEE VR is to promote the “upcoming stars” in the academic research field, but because the amount of resources required to travel to conferences meant that it limited it to established universities and for people who had enough time and resources available to travel. Now that virtual conferences are opening up the range of more inclusive and diverse participants, then what other architectural and cultural dynamics need to be implemented to fully realize this vision of trying to open up the resources and these social dynamics up to a broader range of people?
If it becomes difficult to dedicate your full attention to a virtual conference over the course of 2-5 days, then why not have more frequent gatherings for some of these communities? MacIntyre suggested there could be IEEE VR Mondays that facilitate more focused groups either weekly or once a month. If you distribute out the level of focused attention across, then it creates a dynamic where people could commit their full attention for 1-2 hours at a time over the longer period of a year.
There’s still the forcing function of a deadline where the publication of the IEEE VR Proceedings happen, which serves the peer review academic function of creating archival quality material that advances the knowledge of a community. What Johnsen said is that the conference serves as a re-calibration opportunity to tune into the deeper trends of what’s working and what’s not working so that he could change the direction of future research for the upcoming year. This tuning into the zeitgeist happens for each of these research fields and requires a broad range and diversity of talks to helps dictate what these trends are for the overall community. But it’s not a rational deduction, but more of an emotional, intuitive gut reaction based upon the deeper patterns and trends that you’re feeling are emerging in your field.
For me, this process emerges out of the series of serendipitous collisions I have where I’m able to see these repeating patterns and themes. So attending conferences like IEEE VR help me to determine some of these emerging themes of where things are going. Right now, a hot topic for me and the rest of the VR community is looking at how VR can help with the art of gathering together, being able to help fill in the gaps for when the abstracted 2D Zoom interfaces aren’t satisfactory enough and how embodied representations in virtual spaces help to facilitate organic clusters of conversations that happen from meeting in physical spaces, and also how to architect for and cultivate these serendipitous collisions that help to match the deeper intentions of people to help find potential collaborators or answers to their theoretical questions that they have.
I’ll be continuing to talk with different social VR programs, and event architectures that are exploring these conversations. If you wanted to see some of my embedded reporting with more detailed thoughts and reflections, then be sure to check out my epic 100+ tweet IEEE VR Twitter Thread and Laval Virtual Twitter Thread.
Conclusion
The direct one-to-one translation of the IEEE VR schedule was really informative to see what works and what doesn’t work. There will continue to be different experiments and iterations of virtual conferences in order to get down to the essence of what combination of tradeoffs are required in order to get down the essence of the art of gathering.
- Why do we gather?
- What’s the worth of coming together?
- How do we best share our work and connect with each other?
- How can we formalize our knowledge in rigorous ways?
- How do we facilitate new connections and foster future collaborations?
- How can we form a community to help each other solve our problems?
- How can we facilitate mentorship and deeper professional guidance?
- And how can we explore how embodiment and virtual spaces can facilitate the types of emergent conversations that are difficult to orchestrate through the lens of 2D Zoom technologies?
These are many vital open questions for not only the virtual reality research community, but really everyone who is trying to figure out how to restore the glimmers of what it means to share space with each other in an age of physical distancing. There are some valuable lessons from IEEE VR, but we’re going to need many more iterations and experiments on the level of conferences, but also at the level of architectural experiments that explore different dynamics. I’m looking forward to continuing to explore the latest experiments in social gathering, but also exploring the future of virtual conferences in an ongoing series with platform providers and conference curators.
More podcasts and coverage coming soon. If you’ve read this far and would like to continue the conversation about virtual conferences, then feel free to send me a note at kent@kentbye.com using a subject that includes “Virtual Conferences.”
LISTEN TO THIS EPISODE OF THE VOICES OF VR PODCAST
This is a listener-supported podcast through the Voices of VR Patreon.
Music: Fatality
1/ THREAD👇#IEEEVR2020 conference starts today.
They changed their venue from Atlanta to online in VR.
They kept the exact same schedule 9a-5p EDT Sun-Thu.
Livestreamed free on Twitch & social VR/posters/birds of feather in private instance of @MozillaHubs.
I'll be covering it. pic.twitter.com/7JiYSCB8c3— Kent Bye (Voices of VR) (@kentbye) March 22, 2020
Rough Transcript
[00:00:05.452] Kent Bye: The Voices of VR Podcast. Hello, my name is Kent Bye and welcome to the Voices of VR Podcast. So in today's podcast, we're going to be talking about virtual conferencing. We're specifically going to be doing a deep dive into the IEEE VR conference that happened in between the 22nd and 26th of March, 2020. This is right at the beginning of the quarantine and lockdown and IEEE VR decided to go all virtual, to do a venue change, which they took the entire conference, it's an academic conference with IEEE VR, and they just ported it over one-to-one into having a virtual representation of this conference. Now, a lot of the conference was done through Zoom meetings that were live streamed on Twitch. And so a lot of this interaction was happening in these 2D mediums, but they also had this whole immersive 3D component with Mozilla hubs, which you can go through hubs into this virtual online world, just in the 2D interface, either on a phone, a PC, a tablet, or if you do have a virtual reality headset, then you could be either in PC VR or the Oculus Quest. For me, I actually ended up attending the majority of all the IEEE VR from Oculus Quest and using the different interfaces to be able to engage with the immersive components of the conference. And then about a month later, from the 22nd to 24th, I did the same at Leval Virtual, where I embedded myself into virtual reality and attended probably about 18 hours worth of that conference in VR. But I wanted to just take some time to be able to sit down with them to unpack some of the lessons learned from IEEE VR, taking the existing schedule and to port it over one-to-one. Certainly if they would have architected it from the ground up to just be online, how they start to do that, what different types of trade-offs and decisions might they make. And so we start to unpack that a little bit in terms of what their initial goals and hypotheses were, and a little bit of what they learned from this big experiment of running the IEEE VR. conference within virtual technologies ranging from Twitch, Livestream, Slack, as well as the Mozilla hubs. So that's what we're covering on today's episode of the Voices of VR podcast. So this interview with Blair McIntyre and Kyle Johnson happened on Thursday, May 7th, 2020, while they were in Atlanta, Georgia, and I was in Portland, Oregon. So with that, let's go ahead and dive right in.
[00:02:23.197] Blair Macintyre: I'm Blair McIntyre. I'm a I'm a professor at Georgia Tech in the School of Interactive Computing, and I'm also a principal research scientist at Mozilla, working on WebXR and primarily AR and social VR. At Georgia Tech, I've been doing augmented reality for many years, primarily. Everything from games and design over to software infrastructure. But these days, I've been thinking a lot about social AR and VR.
[00:02:52.645] Kyle Johnsen: I'm Kyle Johnson. I'm an associate professor in the School of Electrical and Computer Engineering here at the University of Georgia. You know, my research is really on applied VR. I did a lot of work with medical applications in trying to build virtual patients early on. And I've since been applying a lot of my work to educational VR. And I do work in the social area of that as well. So a lot of my experience now are collaborative multiplayer experiences.
[00:03:20.695] Kent Bye: Yeah, well, I know that you were both involved with the IEEE VR conference this past year. And my first IEEE VR conference that I attended was in 2015 in Arles, France. And part of my motivation is when I got into VR in 2014, I saw that there was a meeting in Minnesota, but there was no journalists that were there, no coverage that was coming out of it. I felt like with VR coming up with this medium that felt so new for so many people, there was this whole community of people that had been involved with it for year after year since the 90s or even earlier for some folks. So there seemed to be a gap between what was happening in the academic world and what was happening in the consumer market of VR. And I attended my first IEEE VR back in 2015 in Arles, France to help close that gap. But I'm curious if you could each talk about your relationship to the IEEE VR conference and how you see it fits into the overall virtual reality ecosystem.
[00:04:21.458] Blair Macintyre: So I'll actually let Kyle primarily answer that. I've been to VR a few times, had a paper back in the 2000s, done some tutorials, but I only started actually doing VR more seriously in the last few years. Kyle's been involved a lot longer than me.
[00:04:36.243] Kyle Johnsen: Yeah, I've been to almost every VR conference since 2004. Actually, I ironically missed the 2015 conference, but I did meet you for the first time in the 2016 conference. And you're right, yeah, there is a disconnect between industry and academia. And I would say in most academic conferences, that's the case. Very few are successfully able to blend the two. And even when they try, it's sort of two separate tracks. The academics go one way and the industrial folks go the other way. And so I think that everyone starts as a general chair with, we need to involve industry more. And then that the big question becomes how, right? Industry wants to give talks, right? They want to be in front. They want to be promoting their products. And academics, that's distasteful to them for some reason. And I think that's the major challenge. And so you sort of cater to your existing audience. The people are going to show up and then pay the registration fees, right? And your content, which is all of your papers that you've already agreed to publish and promote. And so it's hard to say, we're going to now give speaking time. We're going to give a lot of camera time. industry who doesn't have that mainline content.
[00:05:44.289] Blair Macintyre: I mean, I think it's a challenge. The tension, as Kyle said, this goes across lots of conferences. I've run other conferences in AR and similar. I mean, I think it's the value system and the way of knowing, the way of doing things, right? So in academia, The goal is, you know, that trite way of expressing it. It's like to advance knowledge, to do science, and there's accepted methods and approaches, right? So even Kyle and I will have projects we've done where we feel we've learned something and people should know about it, but, you know, we didn't do a study that conformed to some accepted metrics, so it gets dinged, or we didn't, you know, approach it in the right way, right? Whereas industry, is solving problems within constraints, right? And the things they've learned are valuable both to themselves and to academics, but it's just not following this sort of scientific method. As somebody once put it to me when I was complaining about a paper being rejected, it's like, but how do you know you're right if you don't do that? And it's like, well, define right, right?
[00:06:49.513] Kent Bye: So. Yeah, there was a philosopher named Agnes Callard who was looking at Socrates and the Socratic method and trying to get to the heart of truth and knowledge. And what she said is that it's impossible to believe all truths and avoid all falses at the same time. So you have to have those who have the will to believe putting forth a theory and then those who are critical and really trying to break it down and deconstruct it. they have to each take those perspectives and have this adversarial relationship where it's a, what she calls an adversarial division of epistemic labor. So in order to have knowledge, you have to have those who believe in those who are critical and they come together through this process of the peer review process and the scientific method, put it through the paces. And then at the end, you say, okay, this is what we can stand by. This is what we know and either can be replicated or has gone through a larger review process that is trying to finding these potential blind spots or things that are missing or errors in the methodology. And so, as I think about what the value of something that IEEE VR does is that it can move slowly in terms of being sure for what they know, while the industry can take what's known and try to solve a real problem. And at the end of the day, it's whether or not they're able to solve a problem versus whether or not they're able to come forth with something that can really stand by, that's really backed by this community process. And so when I think about the IEEE VR and the value of that, there's this whole academic community that has been working with each other for decades and decades now, trying to put forth what's known. And so when I think about the process of gathering, the IEEE VR is like this deadline where people are presenting stuff and there's people who are reviewing stuff and there's like this concrescence of everybody's focused attention on, okay, what is the latest things to be presented back to the community through the proceedings? And that seemed to be an underlying aspect of a conference, which makes it different than other conferences. And so I'm just curious if you could sort of reflect upon that underlying process. And then as we're in the age of coronavirus and COVID-19, as we start to meet and be in this Zoom fatigue, as we were talking before we got started, and how, as we are now starting, have the ability to start to gather in VR, how does that start to change this underlying process as we start to reevaluate what the value of meeting is? as to whether or not it's through those presentations or just meeting each other and being able to have those conversations that maybe you couldn't have. So I'm just curious if you could sort of reflect on that a little bit based upon what you saw with trying to translate the IEEE VR over into completely being virtual, but trying to really honor this underlying process for what the IEEE VR as a community is trying to accomplish.
[00:09:37.410] Kyle Johnsen: Yeah, we have sort of maybe a strange perspective as organizers of the conference because we didn't attend the conference probably as much as most because we were sort of putting out fires and in different locations. But I sort of went through an interesting process with my team in sort of what I would normally do in person. We sort of did after the fact at the conference because everything was recorded and we were able to actually rewatch the conference and redo what we normally do, which is sort of recalibrate our work based off of what we see this year. right and say okay what are our future directions what are people doing what seems to be successful right what are the new themes that are emerging you can look at the various aspects of the conference that sort of give you signs of that you know the papers are sort of okay this is the high quality archival work And then you have posters that are sort of showing you the more early stuff, right? A lot of those are the rejected papers, right? The things that aren't quite ready yet. But you have workshops as well, telling you different things and sort of, you know, being able to review that after the fact was really, really nice. Just again, having it recorded as part of that, but that it was recorded in a way that was easy to consume was really nice because everything was online already. It wasn't just like recordings of talks that were given live.
[00:10:50.977] Blair Macintyre: Yeah, I think it's interesting to sort of step back and the other side of what you were asking, I think, is what is the purpose of conferences and what do the online versions actually give that, right? So as Kyle said, one of the big purposes is just to share the latest work and then conversely learn about the latest work, right? I always think it's interesting to contrast CS, computer science conferences, in general with other fields, we're probably much more so than most fields geared to presenting our latest work at conferences, right? Most scientific disciplines, conferences are where you get together and share some abstracts of what you're working on, right? Like a medical conference, you might have 20 presentations, which are like, I'm working on this thing, and I'm working on this thing. And then everything is journals. For us, initially, because of the speed of change, I think, we gravitated toward conferences because journals took so long to get out. So there's those two sides, and I think the online stuff can cater that. And I know we chatted during the conference, Kent, and one of the things you said is you like to watch the videos at high speed and then spend time chatting, right, and talking to people. And I think that's the big part of the conference that is the big unknown right now, which is the, when I was in Osaka for IEEE VR last year, I did what I normally do at conferences, which is I went to some talks. I ended up meeting people in the coffee line between talks and missing entire sessions because I was having a conversation with someone. Then you go out for lunch and you are sort of having this semi-social, semi-work conversations. may do some karaoke at night where you're doing a different kind of conversation, but you're, you know, you're building relationships, you're meeting old friends, old colleagues, making new friends, creating new collaborations. And I think, you know, the reason that those other disciplines that are not Focused on publishing their latest work at conferences still have conferences. It's because of that right there getting together their meeting and that You know, I think is we had some success at that with VR. There was a lot of Holes in terms of that that had to do with just the meeting part, right? It's like how do you find people how to get but it's also the right I think there'll be this really interesting shift when we move either farther into social isolation, right, as Kyle sent me some messages this morning of, oh, this conference has announced they're virtual, this conference has announced they're virtual, but also as we move out of it eventually, hopefully, where the conferences will have to decide, okay, what are we? Because if we get good at the virtual thing over the next few years, there's going to be a question. as to will we want to go out and and it's you know i got into the virtual stuff not because of this i i was interested in both the climate change impact right i don't want to fly to conferences if i can avoid it and the access part right when i had small kids i didn't go to conferences for a few years because it just wasn't feasible and there's lots of people in that boat right and i know when we were at some of the steering committee meetings for the conferences there was some people who were pointing out it's like well this is where we get together and we Support and promote the young upcoming stars, and we help them with networking everything It's like well that means all our young upcoming stars are people at high-ranking Universities and rich countries who can afford the time and money to go to these conferences, and you know what about a meritocracy, right? Why? Aren't the up-and-coming stars in small? Universities or poor countries or remote locations, right? Why is it only the people who can travel and? And the numbers we had for VR, even though it was very last minute, sort of bear out the impact of these kind of changes, right? We had more countries, more people, more women, more minorities.
[00:14:43.355] Kyle Johnsen: And that's without really advertising that or really trying to get the word out to as many people as possible that, hey, anybody can join or anybody here even knowing what was going to be at VR 2020 release.
[00:14:56.025] Blair Macintyre: And I think we have a lot more diversity of background, right? Going back to this initial question of industry versus academia, right? You know what it costs to go to these conferences. You've done a few years. You can only do one or two of those a year at the $1,000 a pop plus hotel plus flight range. But if they're free or $25 or $100 and you don't have to travel, you can do a lot of them, right?
[00:15:22.026] Kent Bye: I'm personally really happy that there's been a movement with the virtual conferences of actually recording the talks and making them available, because I feel like so much of the evolution of virtuality has been in this oral tradition. Because I think very similarly to what you were talking about Blair, even in the mostly the industry side, which is what I've been covering primarily from the Voices of VR, is that there's people who are in the trenches and they're doing the work day after day. And it's the conferences where it's like, OK, I'm going to try to consolidate my knowledge and share what I know. and then they will come and give a talk. But I would find that for me, just personally, it was difficult for me to sift through the signal from the noise, to know what is valuable. From the IEEE VR peer review process, there is a bit of this way of making sure that whatever content's being presented has at least been vetted at some level. But in industry, there's a lot less of that, which means that it's a lot wider variance in terms of if people have knowledge that's interesting and that even if they do, can they have a pedagogical process that they can communicate it in a way that is able to be transmitted in a very efficient way. And I've, I found just from my own preferences that it's, that's a very rare thing. And so I end up just being in a place where there's been a lot of people who have taken the time to consolidate their knowledge into presentations. And then simply just be in the hallway trying to run into those folks and take a very emergent approach of basically who I end up either noticing or know from previous context or running into. It's like a very serendipitous process that's more like a random sample of anything. Just trying to have a series of these conversations that then I share out on my podcast. And so when I go to something like IEEE VR, where those hallway conversations have essentially been abstracted into a 2D realm where people see what the rooms are, and then it eliminates the journey from one location to the next. and they end up going to their final destination. And so part of my frustration has been that the virtual reality technologies can only have about 25 people in the same room, which means that it has meant that you're going from one destination to the next, but eliminated all these interstitial threshold spaces that are the in-between spaces that for me make the value of gathering so valuable, which is those conversations that are happening in those hallways. But that said, as a journalist, that's my primary mode of interacting with people. And for other people who are academics, I can see that having those talks be everything in Zoom and broadcast out in a formalized way, that is satisfying that need for that larger academic purpose. But I guess part of my thing that I have been trying to advocate for after going to dozens of hours at the IEEE VR and then Laval Virtual afterwards, was that it's almost like you have to architect an entire conference from that, from the ground up, if that's what you want to optimize for. If you want to optimize for those types of conversations or interactions or whether it's birds of a feather, which I thought was able to set a time and place for people who are interested in a specific topic to show up and to workshop the types of things that they would do if they were actually gathering. I found that those were the types of conversations that were the closest to what I see happening at a conference than all the other stuff. And the second thing was the poster session that would start to do that as well. And then the demos and then the interstitial spaces and the social spaces that were made available. So for me, my intention is to have those unstructured conversations in the hallway. And I know, Kyle, I've seen you in the hallway in a very similar way in previous conferences. And so just curious, some of your reflections on that, especially when you're trying to satisfy all these different needs and do it within a few weeks. But, you know, kind of like as we move forward, really reflecting upon trying to create architectures that really satisfy all these different needs that people may have.
[00:19:25.877] Blair Macintyre: So Colin and I have chatted about this a lot, both before and during and after the conference. And I know I would split it into sort of three ways you could think about it. One is doing exactly what you said, like figuring out a way to architect the interstitial stuff. And you could do that in a lot of different ways. And I think I wouldn't say that academics are happy with just the talks. I think a lot of us feel exactly the same way as you do. We want both. And so I think there's a lot of ways you could do that, and we can talk more about that. The second way, I want Kyle to share some of his thoughts, because I know he's been thinking about it a lot, which I'll just footnote as ways of changing the conferences to do more of what you want. He was pretty passionate about a few ideas during VR. And the third way, which is radically changing conferences. I think we could get rid of the yearly conferences entirely and do IEEE Mondays, the first Monday afternoon of every month, we'll just do a mini conference, right, and change a lot of that. But I'll let Kyle talk about the sort of ways you could change how the format even works.
[00:20:32.605] Kyle Johnsen: Yeah, I was surprised going into VR. I actually thought that video co-watching was going to be a big deal. And I was disappointed, but at the same time really intrigued at how little I liked that aspect, just because you couldn't pay attention and have a conversation at the same time. And so it was almost useless to be in the same place as somebody watching the talks. You really needed it to be about the conversation that was happening, and there was the conversational ground of something static. Like anything dynamic in these worlds that you have to pay attention to just isn't that useful, it turns out. And I think that we need to divide those two aspects a little bit better in the future. I think we need shorter talks and then longer times to discuss. We still need spacing between the talks. We can't just do these things one after the other. It's just, it's so draining. You need time to process and you need time to have conversations and it's okay if the talks are three minutes long and they really need companion videos in my opinion, but you need the spacing. You could just can't put in so much content that people have to pay attention to because they get exhausted and they don't have those conversations. We also need more just idle time for people to check email and, you know, be at home, go get a cup of coffee, go to the bathroom, whatever, because it was too much. I had to be on all the time. And I think that that to me was the biggest thing that in our defense, what we were trying to do with VR is just replicate the conference experience, right? Like we did not try to go really too far with changing it. Yeah, go ahead, Blair.
[00:22:04.989] Blair Macintyre: Oh, I was going to say, and I would phrase that even more assertively. Like, we made the decision before we even announced the Switched Online that we were going to not do a 24-hour conference. We were going to replicate the In Atlanta thing. And much to the annoyance of our New Zealand and Asian colleagues, we were going to stick to Atlanta time, right? That was actually fine for the Europeans, right? They can get up and deal with it. It's like a five hour, six hour off, but it was really painful for the people on the other side of the world. And that was for the practical reason that we already had a schedule worked out. We already had like this huge, all the pieces were already fit together in a way. And it would have just been too much to add to change that. And also selfishly, I think. This question of can we create social experiences is in everyone's mind, right? Because that's the elephant in the room with online conferencing. And by forcing as many people to be together exactly synchronously, we felt like we would increase the odds that there would be social activity. And if we had recorded and allowed time shifting, we would have essentially said, yeah, we want social activity, but not with the people on the other side of the world, right? Because they're all going to be asleep when we're awake. So I'll let Kyle continue.
[00:23:29.978] Kyle Johnsen: Yeah, I think there's something magical about synchronous, right? It's just like watching sports. It's ruined. If you're watching something on delay, if you know anything about the outcome, it's just sort of ruined. And I think there's something magical about it being synchronous, knowing that the other person is there watching it with you synchronously. It's all going on at the same time. And then knowing you have the opportunity to ask questions of the author, again, synchronously. I think there's another model of this is you just post everything online, right? And that sort of satisfies the publication. You get to watch talks, right? So you get the live video, you get the author's perspective as well as the content. And so that's sort of the academic side. But the synchronous side is just, in my opinion, so important, even for the actual talk delivery. I really felt like the zoom conversations where the authors had a chance of messing up. Right. I was saying something off the cuff, which is so important to me. And I'm just much preferred to watch the live presentations. It's not to say the ones who recorded didn't do a great job. And I think in many cases that helped those authors deliver a clear presentation, but I still just sort of missed it knowing that I just could have watched that at any point in time. Right. And I would've gotten the exact same thing.
[00:24:38.068] Kent Bye: Yeah, I agree. And I really appreciate the fact that you did a one-to-one translation, because I think it's really a lot of things were revealed in that in terms of what works and what doesn't when you start to virtualize it. And so it's a good baseline in terms of this larger dialectic, this iterative process, because I think this is like the first wave of really being forced to go completely virtual. And thank God that, Blair, that you were already working on the infrastructure to be able to do this. and that the timing was such that the quarantine went into effect to the point where of all the conferences to try it, what better conference and community than IEEE VR to try it. And my frustration was that, you know, with the one community that would be out there for the academics that would have access to the virtual reality technologies to really try to innovate, It felt like a lost opportunity when 85 to 90% of all the experience that you were having could be done purely through Twitch or through a 2D screen, eventually on YouTube, but the broadcast meeting of Twitch. And through session to session, I had a similar experience, which was that there was a bit of like a theory that after a talk, there'd be a little bit of a pause and you'd be able to strike up a conversation with people, but What I found in walking around the virtual environment was that it needed a little bit more time, a little bit more context, like to have the architecture of the space really invite that. Because there wasn't audio bubbles that were clearly delineated, it was difficult to know whether or not if you were to have a conversation and the session started, whether or not your conversation would be like, you'd be the person in the movie theater who was talking. It's like there's affordances of being in a theater where it's not cool to talk while the presentation is going on, unless there is clear indication that the people that are watching it, that you're not disturbing them. And the one area that I really enjoyed was the room that had the three presentations happening at once. And then there was the best that I found in terms of a hallway collision was from people going from one of those destinations to the next. They were going from one talk to the next. And maybe they didn't like any of the talks and maybe if you ran into them in that little interstitial space, they'd be open to having a conversation. But the audio fall off of the room was like, I didn't know if like I was having a conversation with Anthony Steed and then I was like, I don't know if people who are watching if people can hear me. And then Ben McNulty came over and was like, no, no, no, you're good. We can't hear you once. This is the line in which you can and cannot be heard. So it's a little bit of like the architecture of the space and the virtual conferencing of knowing that you're in a private bubble and that whatever you're talking about isn't going to be disrupting other people in the space. That's like one way to look at it in terms of the existing architectures of being able to have those types of social conversations and spaces. And then the whole other approach is taking the entire topics of the conference and turning each of those taxonomy tags or whatever they end up being with the themes and create a space that's specifically around that. And then if people are interested in that specific topic, then they would go into that world and then have a discussion around that. What I found is that I was doing a lot of iteration myself of like, I was trying to create a hallway space. And so I was, because it was Mozilla Hubs, I was like, I want to create a portal world where you're in one world and you're going into these other destinations. But what I found was that I didn't want to hang out there because there was nothing to see and nothing to do. And I was the only one there. So I didn't want to be in an empty room with nothing to look at and nothing to see. And so what ended up being some of the more compelling is some of the sponsor areas that actually had videos about this is the research that we're doing and that you could go look at those research videos and then be interested. And then that was actually a thing that was causing other people to come and look at it, too. And then those were a location where some of those collisions were happening. And so really thinking about how to architect those serendipitous collisions through the architecture of these virtual worlds.
[00:28:34.549] Kyle Johnsen: Yeah, that was actually my favorite space. I'm not trying to self-promote, but the UGA space is my favorite place to hang out. Mostly just because we, you know, at some point in time, we were just like, Hey, everybody come to the UGA space. We're going to be hanging out there. You know, 20 people showed up and we were able to have that lots of conversations going on in different places. And there were some people that were just casually browsing videos and there were some people in a circle having conversations and it all just worked. And so those kinds of experiences are just really nice to have a conversation where you've had multiple things going on, but they felt separate and felt like you could just go off and do something else in that space.
[00:29:11.761] Blair Macintyre: Yeah, I feel like I'm laughing because, of course, Kyle and I are at two of the three Georgia flagship universities, and if you're American, you're very aware of sports rivalries and college rivalries. People in the other parts of the world might not be, but our two universities are like the in-state rivals, so I'm laughing. It's like, I hated going to UVA today. It was awful. Too many dogs. Yeah, I mean, there was an interesting bunch of things, right? So I designed very last minute the audio settings on some of these rooms to try to do a little bit of what you were hinting at, Kent. But, you know, I think both with respect to hubs, which was this was the first time we really used it at Mozilla for a big event and tried to sort of architect something like this. So we learned a lot. there, but there was sort of a bunch of things that became much more apparent through this that hadn't been as much of an issue before, right? So, the audio distribution is one. You know, some people talk about how in VR, because it's not like an accurate simulation of reality, you don't have the same intuitions about how audio spreads, right? If I put a big thing that looks like a felt-covered divider in the middle of a space in VR, As a normal person, you would expect that that would dampen the audio, that people on the other side of the wall wouldn't hear you and all that sort of thing, but it doesn't happen at all in most VR systems, right? And we spent a bunch of time before, during, and after the conference talking about, you know, things like, can we visualize audio distribution? Can we visualize audio bubbles? Can we have ways of two people or three people or four people sort of Joining up audio and signaling to the rest of the people there that they're talking and yeah, you can't hear them You can see their heads bobbing maybe and stuff like that so I think things like that will happen both in hubs and in other places over time because It's not just that you and I need to know that we can hear each other We need to know that other people don't hear us, you know There's this back-and-forth right where I need to know that you know that I know that we are all comfortable with what's going on and And then, you know, the other interesting thing that we ran into, which interesting were performance issues, right? One of the reasons why I was willing last year and Kyle and I did stuff last year with other conferences to try to do this with hubs was because it was the first system where we really could assume that anybody on their device could join in somehow, as opposed to, I have to be on a PC or I have to be on a VR device or whatever. So it wasn't exclusionary, but it's still, Problematic because you know if you come on a first gen Macbook air into Kyle's UGA room You're going to crash and burn because it doesn't have the power to run all these videos at the same time display all these Content if you come in on a quest as you discovered and other people discovered the quest is a awesome device for its point in time right but the 835 for whatever chipset it's got inside it is The combination of wireless and rendering and everything, five, six videos and 20 avatars and 20 audio streams, it falls over as well. And on and on. So I think we were both, had the ability to experiment because everybody could join, but you kept running into these situations where the diversity of devices actually also got in the way at times and forced us to make certain choices, right? So the video co-watching rooms, we disabled, most of the rooms, we disabled all of Hubs' content creation features because we worried that if people started dropping videos and PDFs and drawing with the pen and everything, they would destroy other people's experience, right? But a couple of conferences from now when people are used to it, and the systems sort of upgraded and so on, so that, yeah, you can join in your quest, and it's just not going to display the content that will destroy your experience, and it might not show all the avatars, and it might do this and that. So if everybody knows they're going to get a good experience no matter what you put in the room, and people understand the impact of their choices on other people, then you could radically change what can happen in these spaces, which I think will be really exciting, too.
[00:33:30.535] Kyle Johnsen: And I wanted to touch upon a little bit what you said earlier, Kent, that nobody was using VR equipment, which I sort of similarly lamented. At some point, I was just sort of like, let's just have this only for people with VR headsets, right? And let's really explore. And of course, Blair is shaking his head right now. Really, our desire to have everybody in a VR headset really, I think, stems from just what you experience when you're in a VR headset. And you think, this is so great. I can really direct my attention. They know I'm directing attention. I can gesture. I can point. One of the big problems, though, is that there's nothing to point at half the time. And so, like, we weren't in a 3D world that we were trying to explore together. You know, we were in this 3D world that had a lot of 2D content. And yes, there were avatars in there, but it's not sort of like you were pointing at a specific avatar and, like, really gesturing about that. And so we didn't really fully leverage the platform, I would say. I would say mostly because that content wasn't provided by any authors, right? Like, nobody was giving us 3D things to play with. input in space. You know, we had one 360 video at the VR conference. And even that doesn't necessarily call out the need to gesture and things. I think in the future we might see more of that type content, but I don't know if the need is absolutely there. It's really nice and cool to be in a VR headset, but unfortunately is not so comfortable and certainly restricts your ability to do so many other things. And I think for good reason, a lot of people chose not to be in their VR headsets, even though a good number had them.
[00:35:00.168] Kent Bye: Yeah, yeah. And arguably it's a better experience if you're going to be attending the conference for eight hours to not be in VR for the whole time. Both for IEEE VR as well as Laval Virtual. But as a VR journalist, I did that. I was in VR for six, eight, ten hours a day. for both of those conferences for a number of days. And what I found is that when you are immersed in VR, there's certain things you don't want to do. Like, I did not want to sit and watch a video in VR. It's like, that's not what this medium meant for me to be doing, is be immersed in VR, watching a Twitch stream with other people who are not talking to me, who are in a 2D experience that can't even be fully embodied. There's an experience at Laval Virtual that I think was very telling, which was that Laval Virtual used Verbella, and again, most of the people were in 2D, and for Verbella, there's 100% feature complete with 2D, and for VR, it was somewhere between 30 to 60% of the same features, and a lot of it was broken. The 2D features were not only more features, but a better experience. But despite all that, I did it anyway, just to immerse myself. And I was at a lecture with 80 people in the room that were essentially zombies. They were not there. They were not giving it their full attention. Where I was there watching the talk, And then at the end, when they're asking for questions, there was no engagement. There wasn't like anybody who had connected their audio. I just figured there was probably maybe about 10 to 20% of people who were even listening. They could have been parked and just sort of had it as a background as they were doing other things like working on their work. And it was just like a live stream of audio, like they were listening to a podcast while they're doing something else. But That ability to like focus your full attention and when you do have your full attention then what are you doing? And I think that's where you start to get into like if you do start to force people to be in VR Then you start to change the architecture of what is made available because it just doesn't make sense unless it is that highly interactive even this zoom talk that we had, you know, I did an experiment with having different architects talk about the visual aspects of Half-Life Alyx. And so I wanted to see a lot of visual talks. And for me, that makes sense to do as a Zoom call and then to really focus the last half of it as an interactive part. And I think that's the type of thing of like, how do you architect using the strengths of the medium, where maybe the best way to do a Zoom call is to do the pre-recording of whatever content is there. And then when you're there, you're really using the medium to have like, highly interactive discussions. That's the flipped classroom model, which is that you watch the lecture at home, and then when you come in, then you start to engage. But I agree with you, Kyle, at the same time, where there is this magic of concrescence of knowledge that's being transmitted in the moment because the transmission of knowledge, if there's people that are listening live, that transmission changes. And so as I was watching a lot of these pre-recorded videos, it felt stilted and dead in a way that didn't feel like it was alive and happening right now. Even though it may have been recorded earlier that day or yesterday or within the last week, it still had that quality of like, this is not happening right now. This is pre-canned and really produced and it's not messy or it's not, I think maybe subconsciously it's like, well, this is not happening right now. So I can watch it later, or there's nothing that I can, or some of the speakers who had to do that, they're in different time zones and they can't actually respond live to the comments. So this gets back to the time zone dilemma, but there is something about seeing something be presented live and know that you can participate in some ways. And I think that's sort of the value of like, what is the liveness of the live that makes it valuable? And that's why when I saw all these pre-recorded talks, it was like, not only do I not want to watch these in VR, but I don't want to watch them live because it's like, I want to feel that live moment. What is it about that live moment? And how do you really use the affordances of each of those mediums? to their full extent so that you could have intention for why people are gathering? And what is it about once you have people's full attention, and I think that's like the key, that's like their full attention, then what do you do with that full attention?
[00:39:18.974] Blair Macintyre: I think, you know, it's interesting, so much I want to say to what you just said, right? So, You go to a conference, you give a talk, you look at the audience and 75% of the people are heads down on their laptops, right? So that's only worse here, right? Because now the temptation to think you can multitask and check your email and deal with your puppy who's coming over to lick you and all that is so much greater. And so I think there was some of that. We were right at the beginning of the pandemic and people being at home. And so for a lot of people, like one of my colleagues at Mozilla has a one and a half year old, a three year old and a five year old. Right. And so his ability to focus on anything except the absolute most important things are pretty limited right now. And even then, it's like, so I think there's a huge temptation to multitask and put that Twitch video on and whatever. And so I would take Kyle's and your comments about VR further and say, VR prevents me from being distracted, right? I can actually engage in a way that I can't. And if I was at the conference, if people had come to Atlanta, the opportunity to go and have dinner with their family is gone, right? They have to engage with people that are there because what else are you going to do? The Virabella thing is interesting. I gave a talk and was on a panel at Laval Virtual as well, and I had the opposite experience in the panel that you did, which is we had about 50 to 80 people at the panel at any given point in time, and afterwards, Sorry, my son is trying to get the dog out of my office. And so we had all these people, and there was a little bit of motion, but we did get questions. And afterwards, all the panelists went down in front of the stage, just as you know, we had to walk off the stage. And we actually ended up having a conversation for like 15-20 minutes. exactly as you would at a conference, right? Which I thought was really interesting because the sort of Zoom plus hubs plus blah plus blah that we did at VR that other people did didn't facilitate that in an interesting way. Now, the reason Verabella can have so many people in rooms is they're using sort of, I think they're using TeamSpeak as their back end. And so there's no spatialized audio. So if somebody's at the back of the room or on the other side of the massive open field and coughs, they're right beside you, right? Everything is constant. And so you lose some of what Kyle described in the conversations in his room. the ability for people to break off in small groups and have conversations because there's no audio separation. But I think it does point to the need to have sort of some of what you described as the journey, right? Like getting off the stage, the going to whatever. And that's, I think, a lot of what Kyle and I have been talking about in terms of what we want to do with future conferences has focused around, like, I don't think I want to do the 3D VR journey, but we have lots of ideas of if you were going to create stuff to facilitate casual serendipitous interaction, in this world of, I've got Slack and Hubs and Zoom and my web browser and my email, how would you do that, right? And I think that's where I think the most fruitful avenue will be.
[00:42:44.954] Kyle Johnsen: Something that's just desperately needed is this I'm available signal. And we just completely lost that at VR 2020, unfortunately, except for in some of those spaces, like the poster spaces, where if you're alone and your avatar is moving, you're available. And that's sort of a nice signal that you can have. But unfortunately, when you just left it on one screen and you're using your emote over there, your avatar is still sitting there. And that's problematic, obviously, where people just didn't know now, are you available? Can I talk to you? Can you even hear me? Are you muted or whatever? So we really need to find a way to add that I'm available. I'd love to have a conversation maybe about these things, because I'm in a specific room. And if you show up, I'm going to know about it and those kind of things. That would really facilitate, I think, a lot of what you're trying to get at, Ken.
[00:43:35.626] Kent Bye: Well, when I'm at an actual conference, there's sessions that are happening. And if somebody is not in a session, they're telling me in the world that they're available to talk because they're not in a place that something else is happening. And especially if the session has already been going on for 10 minutes. For me, as people are moving and they're moving at a certain speed, that tells me how fast, if they're available or not, if someone's moving. moving really quickly, I know they're trying to go somewhere, but if they're kind of like walking at a certain speed that I know they're available because of what the time is. So knowing what the larger structure of a conference is, um, and this is for me attending nearly a hundred conferences over the last six years and just really trying to break down the art of the serendipitous collision, which is that. The people who are usually the higher in rank and seniority in terms of they've been around the block a few times in terms of the content, or maybe they peer reviewed it and they're already familiar with it, they're usually more interested in talking to their other peers or the people who are presenters. So when I go to a conference and I just hang out in the hallways, the people who are in the hallways are the presenters and the people who are the organizers and the people who are the most senior and have the most knowledge. So it's great because for me as a journalist, it's like I'm able to really cut through the signal from the noise and really focus on both the presenters who have already focused their knowledge in a way that's trying to present it to the larger community in a way that's already formalized. I'm just sort of drawing that knowledge out in a more conversational way because they've already done so much to kind of distill their knowledge down to transmit it into one format. So the presenters are great to be able to talk to in that way. But when everything is virtualized into, I'm going to get to this destination to do this destination, and you go directly there and you essentially teleport, then you eliminate that. And that's one of the things that I really appreciated about Verbella was because they're able to have up to like 1200 people in the same instance. And for most people, they were in 2D. They did have teleport and people were teleporting, but a lot of people were also just locomoting using the 2D interface, which when you're in VR, it becomes more motion sickness inducing when you're locomoting. But the advantage of that is that you have almost a larger persistence of people wanting to go from one destination to the next. And I think what the teleporting does is that the people who are walking super fast and trying to get to the spot they're going to get to, then they're essentially teleporting anyway. They're telling me they're not available. But if you do have this ability to locomote your body through these spaces, then you're able to potentially collide with somebody. The problem that I found with virtual conferences, and also just conferences in general, is that the architecture is completely disposable in absence of any context. The thing that makes conferences so amazing is that you've set a larger context. You know that anybody that's at this conference is interested in this topic and there's often a commitment they have to make in terms of their time and their energy and they have their full attention. So all these things are signals that they're committed to the larger context of whatever the topic of the conference is. But one of the things that I think is the next step for a lot of these virtual conferences is to think about not only how to organize the conference around time, but to organize it around space. Because what you do with setting that larger context with the talks is that you have an empty shell of a room that's just a box usually. And then you say, come here at this time and you're going to get some information. And so at Verbella, when I was going around to all these spaces, I ended up seeing a lot of these empty box rooms that were just like, Well, I clearly don't want to be in this room because there's nothing here. And there's nothing that is going to inspire me to hang out in a place that has really bad architecture and just is not interesting. There's nothing for me to do. It's like the equivalent of hanging out in an empty room, hoping for enough people to kind of happen to show up and have a social happening. So thinking about what is the larger context of these AAA VR, what are the tracks? What are the topics? If you were to design a space that was trying to reflect the architecture of that topic, that would be nice for people just to endemically hang out in, even if there was nothing there. But at the same time, be a point where if you are available and you do want to speak about something and you're really into storytelling or collaboration or education or 3D UI, you would go to those locations and just hang out there and then naturally have the architecture of that space to be able to set a deeper context, to be able to have not only the people who are available for those conversations, but are also interested in those topics, and then have clusters of conversations start to emerge naturally.
[00:48:10.998] Blair Macintyre: Yeah, it's interesting. I remember when Kyle and I were trying to decide how we were going to do the poster sessions, you know, we ended up having 35 rooms with four posters each room. But again, some of the choices we had to make and other people still have to make came back to performance and secondary concerns, right? when we did posters at UIST and SUI last October, we sort of didn't worry too much about that. So the poster rooms had like the poster and videos and ancillary information. And when I co-watched some talks that were broadcast from CHI last April from Glasgow with some friends when I was testing this, we were like pulling in the papers and flipping through the PDFs and going and getting like content off offers, the web pages and tossing it in and having like active conversations during the talk, right? And so the ability to just pull in all this content and then leave it there as this memory of what the conversation was, that stuff is super powerful, but we intentionally couldn't do that because we were worried about performance. But I think you're right, but I will push back and say I think You know, it's the co-evolution of technology as social systems and technology that has enabled you to sort of do that with conferences, right? All these things you understand, like how fast people are moving and what the time is and all that, enabled you to infer things about people's intent, right? I think in 2D we have to, or in online, we have to embrace the fact that at the end of a session, I'm going to go have a conversation with my son to find out what he wanted. I'm going to go make some coffee. I'm going to go to the bathroom. Maybe make banana bread, because we have some bananas that are going bad. As Kyle said, but maybe I want to signal, I'm actually available to talk. I'm not going to sit here and stare and sit in a virtual empty room with nothing to do. Because I'm not going to do that, if there's no other way for me to signal that I'm available to talk, everybody gets the empty room. Kyle and I joked that There's this mental model of the social spaces at VR at the end of the talk, at the beginning of lunch, where there's a sequence of a hundred people popping in, seeing nobody, and leaving, right? Whereas if you were at a physical conference, you would have seen that foot traffic, but here it's like magical teleports, right? Whereas if I could say, Hey, I'm available to talk, take my phone with me, go out into the kitchen and start making banana bread and ping, ping, ping, like three or four people want to talk and be like, yeah, I'm halfway through the banana bread later. Give me 15 minutes. Or if I'm just making coffee, I'll be like, Oh, okay. I'll be back in two minutes. I'll meet you in the UGA room. Right. And just that right. Would just that glue. Let us develop a different social technical relationship with the stuff where now all of a sudden maybe just that maybe just the ability for all 2000 people who registered for VR. to have 500 of them at any point in time say, I'm available to talk, and I've entered the keywords I'm interested in, and I've entered my bio, like most online conferences do. And so now I can have a system where I'm like, I'm available to talk, and then you can go and look and say, who are the people interested in social AR that are available right now? Oh, Blair, and this, and that. Anybody want to go and chat, right?
[00:51:33.764] Kyle Johnsen: Oh, anyway, the forced interactions I thought actually worked pretty well, though. And so like things like the birds of a feather where you said, I am going here to discuss this. That's fine. And I think that actually was some of the more successful uses of hubs that we saw. I'm going here. The general charge will be here or whatever. Right. Blair McIntyre is going to be over in this room right now. Come have a conversation. Like, I think you almost have to force it a little bit with the current technology. And it works. As long as people take that leadership role, it works.
[00:52:04.681] Kent Bye: Well, and I think from my experience as somebody who loves the emergence of that serendipity, I get into this dialectic between the scheduled, planned, set a specific time, meet here to talk about this topic, versus the unscheduled, unplanned, emergent, Kairos time, which is like, we're just going to show up and be embodied and be available and see what happens. I feel like part of what you're talking about, Blair, is still into that structured, like, I am now available. I am demarcating myself as now I've flipped a binary switch and now I'm free. I think there is an openness. When I observe what happens in actual conferences, there's an openness that I see that's part of this serendipity. Like you were talking about, Kyle, is like, I am open for possibility. You know, there's different ways that we signal that with our embodied body language, deeper context of the spaces that we're in. And what I want to see is the architectures of spaces that do that, but also like, you know, this having Slack was actually really important for to have a back channel to be able to have direct messages for people to be able to communicate. In Verbella, if you're in the same shard of a space, you can direct message somebody. There's actually the possibility in Verbella to say, here's Kent Bye, go to him. And you actually locomote and physically move through the space. You don't teleport, you actually physically move through the space. to get to that person to see what's happening. And they have these different ways of architecting those collisions in the virtual space. And I do think there is going to be an element of using a good functional back channel that's high bandwidth that you can have a DMS and have your own persistent identities, whether that's Twitter DMs or Slack or Discord, whatever that ends up being for the conference, to have that back channel to start to facilitate some of that. But for me, the way that I think about and try to define a successful collision is somebody who has an intention, they want to do something, they want to accomplish something, but they can't do it alone. And they need to find other people that have that same intention. And how do those people meet together so that they can cooperate and collaborate to be able to achieve something they couldn't do by themselves? And if I try to abstract out a good collision at a conference, it's exactly that. It's like two people that are interested in the same thing. And they're able to have some exchange, whether if there's an information asymmetry where people have already explored this and they have knowledge, they're just transmitting the knowledge. And that's a part of just spreading the information to people who need it versus the people who need that either knowledge or cooperation for other people to have future embodied actions to be able to accomplish something. But that those types of collaborations that come up, I think it's difficult to really always pin down exactly what it is with down to a taxonomy term. It's more of like, I have this question, this burning question, and maybe like, how do I facilitate getting that answered and having my problem solved? And I feel like with conferences, you have this organic emergent resolution of that. where you're just in the same space and you either you know somebody who just gave a talk about that who you want to be able to talk about or maybe maybe you have no idea that somebody has the answer to your question but you happen to run into each other anyway and you both happen to be open and it just works out and that's the magic of those serendipitous collisions but for me this is the sort of the key of like those shared intentions and the need to either get information or to cooperate and collaborate and how do you get those people to meet with each other And for me, I think there's something key about embodied interactions and being in a deeper context that facilitates that.
[00:55:37.729] Kyle Johnsen: Yeah, especially when you have sort of small groups that are very visible, that really fosters, it's not serendipitous so much as I can observe what's happening, I can listen in, and then I can let others be known that I want to take part, right? Zoom is just awful with this. I can't have a conversation with just Blair right now. And you can't just sort of casually observe that conversation. You're part of it, whether we want you to or not. And you can't just come in and just be a fly on the wall and then move off without bothering us. I really loved that part when I started having a conversation with somebody else in Hubs and somebody else came in. And oftentimes it was a student, right? And it was something that just happens all the time at conferences. Well, someone will just kind of be hanging out in the corner over there and then they get brought into the conversation. That can really only happen with an embodied platform, just because we understand those social signals and people utilize them inside the 3D space.
[00:56:36.241] Blair Macintyre: Yeah, and I think You know, and I'm thinking about the sort of I'm available, come and chat. I'm imagining the context of a larger system like what we set up for VR, right? Like the core problem I think we had at VR and with this kind of architecture was that very few people, as Kyle said, took the effort to sort of go and hang out when nobody was there. And like, at one point, I started doing things like I put my avatar in a room, I'd type like BRB, and I'd turn it into the little text thing and put it above my head, right? So people knew that I was there and available. But what I'm imagining when I think of the okay, I'm available and like, hey, Kent and Kyle, do you want to go and chat? Well, now we're in a room, right? And if that's also visible, then it starts to bootstrap this whole serendipitous interaction where if there's 20 rooms at IEEE VR, each of which had 10 people in them, and you could sort of see at a glance who all these people were and what their interests were. Then you can just pop into rooms and listen. You know, one of the things that struck me about the post panel chat in Verbala, Virtual Laval, was that how much it had that characteristic, like Kyle said, of a regular conference where there were people who came down and sort of moved close to the conversation, but never said anything. I have no idea who they were. Right. And I think about when we go to conferences, the three of us, I don't think any of us have any problem walking up to anybody and chatting. Right. And we're all sort of senior enough and comfortable enough in our roles that we don't feel like it would be a problem. But students, people who are from outside the community, maybe people who don't feel as confident about themselves may not feel comfortable just walking up and saying, hey, you know, famous person, Henry Fuchs or something, I want to chat with you about something. But they could stand five feet back and listen to Henry talking to someone else and kind of slowly edge in. And so I think that sort of stuff is super, super, super popular. But The thing we need in VR, like, I just don't think we're going to be able to get people to commit at the level that they do at a physical conference, right? Both the opportunity to go and do other things, but also the pull. One of our colleagues in Japan pointed out that they actually aren't allowed to skip teaching, to skip meetings, to skip whatever at their university, unless they are actually at a conference that is somewhere else. Right? Attending virtual meetings isn't a thing that they're essentially allowed to do in the way that they would want to. And honestly, I'm not sure I would feel comfortable canceling class to attend a conference if I was here. Right? I would work around it. And it's just, it's nice, right? It's nice to be able to have dinner with your family. It's nice to go out and spend 15 minutes of the 30 minute break chatting with people. Locally, so I think we have to facilitate a different way.
[00:59:42.195] Kyle Johnsen: We can't I think we're gonna be able to force the conference experience the spatial Experience on to a virtual event And I think that the the five-day Conference that's all that's a lot. There's just a lot to consume you don't really get those same nighttime breaks that you get when you attend the conference live and It's just, it's very, very hard, I think, to have this kind of conference in a contiguous stretch. I think shorter conferences, you know, one day conferences are much better suited for an all VR experience, much better. And if I were to sort of run this again, all virtual thing, I think I'd really focus on that timeframe, you know, where I can probably force myself to commit to something for, you know, 12 hours straight, but five days straight is just a lot when you've got these other polls.
[01:00:33.495] Blair Macintyre: VR could have been all March, every Monday.
[01:00:36.319] Kyle Johnsen: Yeah, could have been all March. And I think that might have actually worked in giving people time to get technology and things like that, right? It's like, oh, I need to do that.
[01:00:45.470] Kent Bye: I think this, like I said, it was very informative to discover that after trying to do a one-to-one translation, especially when you have people not having their full attention, split attention, they're not really able to have the same type of social interactions that may be sustaining and fulfilling and re-energizing versus, you know, listening to someone do a one-to-many broadcast model of listening to a talk can be very draining. But I think the point that you were making Blair was very well taken in a sense of a whole range of introverted versus extrovertedness and amount of power and privilege you have and whatever role you have gives you access to different conversations. And I think that one of the architectural things that was maybe it's not actually in the default aspect of Mozilla Hubs, but the ability to go into a room and be present as a ghost and to kind of lurk around actually was on one hand kind of creepy because you don't necessarily know that you're being listened to and so there's a bit of a larger cultural aspect of feeling like when you're in a quote-unquote public space in virtual reality there could be really anybody who's listening to you at any moment. Usually that signal is given when you see an actual embodiment of that person right around but I did think there was something new and different by allowing people to be a ghost and to look into a room. I mean, it's a performance issue where you can't actually have that many people in a same space. And so if you want to open up one of those emergent conversations to people, you can give them the ability to listen as a ghost. But I think it's actually trying to architect systems for lurkers who want to be on the outside of a conversation, but not necessarily be drawn into a conversation. But what better system for an introvert to be able to go to a cocktail party and to literally listen to a dozen different conversations and to maybe go into a room and deliberately target the conversation that you want to be a part of. And I actually did that a few times, listening around to what people were talking about, and then came in as an embodied avatar and joined that conversation. That is an amazing thing you can't do outside of VR, but I think it starts to facilitate these new social dynamics. But thinking about how do you design systems that allow people to lurk, allow people to listen in, and then People, maybe they do have a private conversation. Like there was times at Laval Virtual where I wanted to go have a private conversation, but somebody joined in anyway. And so the culture and the dynamics of what's public, what's private, what's okay to join, what's not okay to join. There needs to be better signaling for that. Whether or not you go off to a private room or you're still in a public space, but you're in a red dot, that means that no one should be entering in it. Maybe it's a yellow dot where you, you're able to have people kind of listen in, but you don't really want them to participate. And then a green dot. But again, this sort of gets into the, yeah. sort of structured and unstructured ways. You don't know how those are going to evolve over time, but there's ways of potentially signaling that. But that ghost-like feature was amazing to be able to sort of facilitate that type of lurking.
[01:03:40.298] Blair Macintyre: Yeah, we don't allow that on hubsmozilla.com. It's a feature you can turn on in sort of the Hubs cloud instances. It's referred to as lobby ghosts, right? So you can be in the lobby where you're just stuck watching the lobby camera, or you can enable rooms to have, like, anybody move around. But it's funny, you know, I was involved in lots of conversations about all of exactly the potential bad issues of that. We put signs in every room reminding people this happened. And even me, I'm sure I engaged in conversations where I wasn't paying attention to see if anybody, because you can tell if people are in that mode by looking at the entry list. I'm sure I engaged in conversations where it was just me and Kyle, and I wasn't paying attention or even looking to see if anybody was in the lobby or paying attention. But I agree. It's like, you know, one of the things that excites me the most about the opportunities we have to explore, given that there's going to be at least, you know, probably at least a half year, maybe another year of lots of virtual conferencing, and the Hubs Cloud stuff finally coming out is that we can hack this stuff up, right? At this point, it's really just a small matter of programming, right? It's one thing to take like... a VR chat or an alt space or something where it's a hosted system that gives you a facilitated set of extension capabilities versus you can install the whole darn thing and hack it. If you know JavaScript, you can hack the heck out of it and build green dots and red dots and purple dots, right? And build like 2D awareness stuff and all that. So I'd love to see, I was joking with Kyle this morning, we were talking, it's like, if I was more startup minded right now, maybe I'm just getting old. There's a bunch of like, you know, we use a company called Presentation Technologies to do AV for our conference and some other people use it. There's a system called PCS, which a lot of conferences use for reviewing. These are sort of startups, right? These are companies that do those sort of things for conferences. Doing like a VR conference startup right now, I think If I had the time, energy, and money, it's like the right time to do it, right? And you could just build these things out and offer these services to conferences and really sort of make this stuff work. I think it's a great opportunity.
[01:05:50.621] Kyle Johnsen: Yeah, there really isn't anything that's custom designed and hackable like Hubs is that anybody can buy right now. It's really unfortunate because I think we kind of have a good sense of what we want now, but we need a little bit more iteration on this. I don't think anyone's going to be able to come into the market right now with a product and sell it to anybody. But yeah, I'm excited. Blair might actually do this, we'll see. But I'm excited about just exploring these concepts and really seeing what the more micro model looks like, as people are willing to do that, really modify the conference to support it or cater to the virtual reality platform.
[01:06:28.272] Kent Bye: Yeah, and I know that Mozilla hubs is really architected for private small conversations with 25 people wasn't meant to facilitate having 1000 people meet all at the same time in the same room, especially I know there's Code like high fidelity that was able to break apart, you know having five different servers have up to 500 people in the same room rubella has their whatever they're doing in terms of Automatically sharding between different people with team speak and having audio isolation bubbles. I love the audio isolation bubble concept because that's great so endemically start to split apart those emergent conversations. And I know that Liam Broza of XYZ Social is taking code from Mozilla Hubs and starting to hack it in different ways. I think there's people that are starting to do it, but again, I think it goes back to designing for 2D interfaces first and then realizing all of the limitations of what you don't have people's full attention and being able to kind of be limited by not having that full access to that attention. But I do think you're right in the sense of this model of small meetups trying to really from the ground up design what those hallway conversations are, what those spaces are, and start to create more of a meetup model where it's the birds of a feather. I think that some of the reason why it was so successful was that exactly that what I was talking about before, which is that you have a problem, you have an interest, you need to have emergent conversations with people who are also interested in the same things and setting a birds of a feather like, hey, let's talk about how you do user studies in the age of COVID-19. It's like, okay, like if part of your research, a key part of it is working with people with VR technologies and public contexts. And like, how do you do that in the age of a global pandemic? And it was a bunch of VR researchers being like, how do we navigate this? How do we do our research? It was such an interesting like problem that so many people had and they all came together to be like, let's talk about this problem. And I feel like that model of either specifically around intentions of making stuff like maker spaces or problems that you're trying to solve or topics that you're interested in and having either more frequent meetings, shorter meetings, but have people's full attention. rather than having 100 people coming all in looking at something and not being able to fully engage, but having that model of like, let's have a working group, let's have a birds of a feather, let's have a small meetup and let's kind of workshop this. It's more of the unconference model that I think is going to be something that you can start to break off of that monolithic conference model and then start to, you know, really from the ground up design for that and then see how you meld these two things together. But as we're sort of wrapping up here, I'm just curious, what's next? What's next for the problems that you still want to solve in this area? Or if you've kind of passed the baton to the next general chairs of the IEEE VR and you've moved on, where do you go from here? I know, Blair, you're doing stuff with Mozilla Hubs and Kyle, you're doing your own research, but what happens next and how do you sort of push this conversation forward to continue to iterate on this?
[01:09:27.801] Kyle Johnsen: I think the model of that conference that is happening live and virtual at the same time is still just completely underexplored, if not completely unexplored. And I'm excited about that in sort of ways that we can have the best of both worlds at some level, where some people are live, and then there's a part of the conference that, or at least a parallel track that is virtual, and somehow they feel still like the same event. And that's really what we were trying to do at VR originally. And that we just didn't get a chance to really unveil that. And I think, I'm not sure I would do things the same way we were planning now having gone through the all virtual. And I think that trying to do the same at the exact same time is probably not going to be the best because I think you do want to get enough critical mass in the VR spaces. for it to work properly. But I'm excited about really thinking about that emerging as we explore all this virtual conferencing over the next six months, and then maybe next year being able to try some hybrid models out.
[01:10:35.275] Blair Macintyre: Yeah, I think the hybrid stuff is really interesting. Even before all this happened, my colleague here at Georgia Tech, Kim Cobb, who does geophysical science stuff, she's concerned about climate change and air quality and so on. And her and some other people in the Fly Less movement and I, so Park Wild up at Tufts led this, we're going to experiment with a three-site conference in the fall in that area, just one day, just to deal with both location, separate locations, and time zones. We're going to team up with some folks on the West Coast at UW. So adding to that will be really exciting, because I think the hybrid model is really good. Expand it out to have multi-site so that, again, going back to the climate change issues, right, so people aren't flying as far. Even in the pandemic world, if you had sites in, you know, a few major locations, even if people, nobody flew, right, if people just drove locally and you had 100 people here, 100 people there, 100 people there, and then 2,000 people online, Dealing with the time change stuff, huge questions, right? So there's some conferences that are doing the 24-hour thing where they're going to record the talks and repeat them on six-hour intervals four times around the clock. I saw some where they're talking about, well, let's actually have, as a presenter, you choose one time zone to present live, but you also commit to doing AMA, Q&A for each of the four time zones. Like, you're going to commit to being up in the middle of the night and every six hours you're going to be there to answer questions. Things like that I think are really interesting, right, to expand out. Kyle and I also just submitted a small NSF grant to explore how we can use similar technologies to support undergraduate research groups that are distributed right now because of COVID, right? So I think conferences are one form of education and knowledge sharing, but there's lots of other things, right? Like my kids' school, you know, K-12 school is now all online, as all are here. Could we use these same technologies to let an entire middle school or high school have all their classes in shared space, right, not just in Zoom, right? So many schools, whether the classes are from 10 to 35 people, those are actually much more amenable to something like Hubs than a conference of 10,000, right? So I think there's a huge amount of places that social VR, especially when it's something web-based where a school that has all Chromebooks or all iPads or something could, in theory, all join in. We have a lot of potential.
[01:13:15.210] Kyle Johnsen: You know, your ability as a teacher to go between group interaction and individual interaction is just severely limited by the platforms that they're trying to use right now for distance education. What you really want is people working somewhat independently or in small groups and the teacher to be able to pop around and, you know, bring people in and focus on someone's work or, you know, highlight that to everybody. And it's just so hard to do that with the platforms we have. We need new platforms.
[01:13:48.696] Kent Bye: Great. And finally, what do you each think is the ultimate potential of virtual reality and immersive technologies and what they might be able to enable?
[01:14:01.441] Blair Macintyre: Well, I'm gonna return to my AR roots, right? I think the ultimate thing that I'm excited about with immersive tech is social AR, right? It's all of us wearing some future HoloLens 12 or Magic Leap 40 or something. And we're having this conversation now, but it's actually your avatars in a shared document, right? we can actually start having all of the things we've talked about, serendipitous interaction. Like when I get up and go to the kitchen to get my coffee between talks, your avatar might follow me to the kitchen, right? But think about the content, think about the mix. The idea of blending the physical and virtual in these sort of immersive social spaces has a massive potential, right? Especially as more and more of what we do moves into the digital domain. we lose a lot of, I think about it from the education viewpoint, lose a lot of the pedagogy that arises from people being able to look around a classroom, like an art studio or architecture studio, and observe what others are doing and have these serendipitous casual interactions, whether it's we're working in an engineering firm or we're middle school students or we're at a conference or we're independent journalists working from home, and it would be great if all of my colleagues could sort of casually interact with me in different ways, right? So I think augmented reality will allow us to pull the physical world in ways that VR isn't going to allow us to. So that's what I'm most excited about.
[01:15:31.890] Kyle Johnsen: I'm really most excited about things getting smoother and trying to make things smoother. Everything is so, there's so much boundaries between platforms and interfaces right now. They're just so frustrating. I can't show you what's going on on my computer in an effective way right now. Not easily. I can do this screen share thing, but I've got three monitors in front of me. I can't show you what's on my desk easily because I have to bring the camera up. Like nothing's being digitized properly. I can't share. You can't jump in and actually be in my space and helping me with something easily. Man, I think VR has that potential in a way that actually AR technologies at least would struggle with because everything's digital. Everything's flexible. And so Blair and I might disagree a little bit on that because I'm more on the augmented virtual reality side. So I'm really excited about just digitizing our world and then being able to flexibly incorporate what you want of the real world. And so it's actually a much harder problem trying to digitize everything, but it does have an immense power of being able to selectively choose how much you want. And everything is opaque. Sorry.
[01:16:42.731] Kent Bye: Yeah. Yeah. Well, I know just spending a lot of time in Zoom meetings. And so, you know, the value of virtual avatar representations to allow you that space that maybe allows you to take a break and still have a presence when you're listening, but not having to be always on that can be very exhausting in this time. So I think, yeah, depending on the context and what I think there's going to be specific on wherever you fall on that mixed reality spectrum, that there's going to be certain use cases that it makes sense to have either one extreme and the Degree of virtual lies NIS of that context and that environment is going to have different use cases. But yeah, is there anything else that's left and said that you'd like to say to the broader immersive community.
[01:17:23.000] Kyle Johnsen: I think just try to join up as many of these conferences as possible. We've not seen a ton of people say, I'm going to make that leap to go and support VR at the conference as a communication platform, social watching platform, poster sessions. We've not really seen a whole lot of people go that route. And as they do, the more people that engage, the more ideas we're going to be able to generate and problems we're going to be able to fix. So I hope more people engage.
[01:17:52.756] Blair Macintyre: Yeah, I'll say just don't be afraid, right? We were lucky. I think in some sense, the bar was pretty low for us. The fact that we didn't fail entirely was a success. But, you know, over the next year, it's like the easy thing to do is to just put the talks online, do some Zoom talks and call it a day. And that's fine if that's what you want to do, but nobody's going to fault you for being ambitious, right? Try to do crazy things, like the thing we just talked about where it's like, why not have your conference every Monday for a month instead of four straight days? If you're going to move it online anyway, why not, right? Think about what you want. What would your ideal experience be? and try to do it, right? Because we have the opportunity now to say, well, I had no choice. I had to try it because we couldn't do the physical conference. So try the things that you want and see if they work.
[01:18:46.098] Kyle Johnsen: Yeah, I don't know if there's ever going to be another time in our lifetimes where we're going to have that ability to have an authentic conference, right? Where we made that choice to go into VR and everybody's going to do it, right? Because when faced with this physical versus virtual, right? Someone's going to fault you if you choose the virtual and it doesn't work out. The physical is just, it's very robust right now, right? And, you know, your risk is very low, but now you can get away with it. And it's going to be, everyone's using that to go to the conference. It is how they're interacting. It's totally authentic. Very hard to have that kind of conference otherwise.
[01:19:24.137] Kent Bye: I think there's going to be a number of waves of, you know, Laval Virtual as well as IEEE VR feels like a part of the first wave of just translating quickly. And then the next wave of AWE, VRTO, there's going to be VR, AR association, and then continued iterations. And so I'm grateful for both of you for being ambitious with these experiments. And I think we learned a lot. We learned a lot in terms of what works, what doesn't work. in sort of doing this one-to-one translation. And really, for me, it was an opportunity to start to really see what the essence of what it means for me to meet at a conference. But I don't have the only use case for all the audiences. And so trying to keep doing these experiments and listening to a broad range of different people, and like Blair, you're saying having more access and ability for different diversity and other ways of being more inclusive. You know, there's a lot of different trade-offs that you're balancing here and seeing where that perfect mix of wherever you fall in those different trade-offs of what you're trying to do and continued exploration and iteration. But I'm just grateful for both of you, Kyle and Blair, for taking that first leap into doing some of these early pioneering explorations and for taking the time to really unpack it and to talk about it and to have other people have the opportunity to learn from these experiences and to see how they can continue to push this forward. So, yeah, I just wanted to thank you each for joining me today on the podcast. So thank you.
[01:20:43.189] Blair Macintyre: You're welcome. Thank you for having us.
[01:20:45.866] Kent Bye: So that was Blair McIntyre. He's a professor at Georgia Tech at the School of Interactive Computing, and he's also a principal research scientist at Mozilla. As well as Kyle Johnson, he's an associate professor in the School of Electrical and Computer Engineering at the University of Georgia. working in applied VR, medical applications, educational VR, and collaborative multiplayer experiences. And they're both general chairs for the IEEE VR conference 2020. So I have a number of different takeaways about this interview is that first of all, well, this is a really interesting conversation for me just to start to unpack some of the deeper intentions of why do we gather at all at conferences? I think the academic conferences, they have many different intentions and motivations that are driving either the advancement of scientific knowledge, or there's also a very pragmatic one of the things you do to be able to advance your career in the academic realm. So you have to publish different papers and get them out there. And so there's a whole wide range of motivations for why these things are the way they are. And so as we start to think about conferencing, I think the approach that was taken here at IEEE VR, I think to me at least showed how some of this one-to-one porting just doesn't work to be able to get to the essence of some of what makes gathering so valuable. I think there's a lot of different trade-offs and a lot of different things that you can try to optimize for. Well, first of all, there's the peer-reviewed content and how do you have this tier of papers and posters and demos, panels and workshops? I think that the poster sessions actually were a huge success of the IEEE VR in terms of content that just really works well. You have 30 to 40 different rooms with four posters each, and then during the poster sessions, You're able to go in and speak to the different authors about their poster and it was just an opportunity to Set a context and to be able to have like a very deep meaningful conversation I found it was difficult to discover all the different posters. There was different videos that they had all produced, but it wasn't Made clear in a video playlist or it wasn't like readily available to go watch all of the posters I love what the SIGGRAPH conference does in terms of their fast-forward which is essentially like each of the people get 15 seconds or so just to give a pitch or promo for what they're working on and why and And even if it was like 30 seconds and you allowed people to give that as a live talk, as a pitch, you know, that would be a fascinating way of giving an overview of what the posters were. They did have the videos there, but the videos weren't incorporated into the rooms and you had to kind of go and find the information. Generally, it was difficult to know where to go find information. It wasn't one centralized space. There was a Slack where there were announcements that were being made and there was stuff that was spread across the website. It was difficult to know where to go to get the information. And actually, because I was embedded within virtual reality, one of the things that I was trying to find is like, can I just go into VR and get all the information that I need? And the answer was a resounding, no, you can't. You have to use all the different existing 2D interfaces and 2D websites. So I'd like to see a little bit more of a consideration that, okay, if we were going to design this as a immersive experience from the ground up, then how would you let people know what's happening and when? But I still think we're still in this hybrid model of doing the Twitch streams and YouTube videos and Zoom talks where they're basically giving a one-to-many talk. which I think there's still a huge value for it, but a lot of them were prerecorded. And I think one of the things that Kyle was saying is that, you know, similarly, it's like you want to know something that is happening now and happening live and to be able to potentially have the messiness of whatever is emerging in that moment, but also the ability to be able to interact with that person in real time. So to be able to ask questions. And so one of the things they're thinking about is to expand out, to be able to do more of those different types of interactions. So a lot of other considerations to be inclusive and diverse. Uh, so not to just focus on virtual reality as a headset, but to look at something like Mozilla hubs, which allows you to do both on PC and on tablets, you know, it's questionable in terms of how much like a Mac book is going to be powered enough to be able to run some of these experiences. And so. you start to run into the lowest common denominator of being able to design an experience that is not able to be able to be progressively enhanced. And I think that's kind of what the goal would be is to, if you do have higher end equipment that you have higher bandwidth type of experience. But there's certainly certain performance issues that even having too many people in the room, having to disable a lot of the creative tools, there was an ability for creators to be able to create their own rooms. And so maybe those private rooms be opportunities for people to go in and start to use some of those interactive and collaborative tools. I'd love to see a lot more audio isolation bubbles that were more clear. The audio fall off isn't necessarily always intuitive as to where that is going to stop and where that ends. And that's a part of what Blair was saying is that there's not architectural affordances, but that makes it intuitively obvious as to whether or not your voice is caring and disrupting other people. There's still lots of pending time zone issues to how to like navigate multiple time zones as you start to do these virtual conferences. And I love the idea of having one talk be rebroadcast every six hours, but also to make the speakers available for an ask me anything questions at specific times that are suitable for each of these different time zones as well. The live versus prerecorded. Of course, I much preferred the liveness of that, uh, the messiness, the concrescence of knowledge and that transmission that happens with a live audience. versus the pre-recorded aspects, which I think does have a little bit of that stiltedness. So what is the liveness of a live event that makes it worthwhile attending in person? And what is it gonna be about that event that makes you commit 100%? Because as these virtual conferences are happening and you kind of drop in and out, then what is it that you get if you give it your full attention? Is it going to actually reward you by giving you an experience that you could only get if you had your full attention? And I think at this point, it's still kind of like a fragmented attention. And actually, as Blair was saying, as the existing conference scene, as you get up and give a talk and everybody's on their laptop, that's not necessarily like a model that's working for anybody already. So we're taking an already kind of broken model in the real world and then transposing it online. And some of those things that don't work start to get amplified in different ways. And that's why I start to advocate for this complete emergent bottom up on conference, designing and structuring architectural affordances for those emergent type of conversations, and then trying to figure out how to blend those together with existing 2d talks that may be watched outside of VR. There is a limit for a lot of people for how long they can spend in VR. For me, I was spending up to eight hours in virtual reality each day going to IEEE and Laval Virtual, but that is an extreme edge case. I was already one of the few in VR and the people who were in VR weren't spending nearly that much time. And honestly, there's not as much stuff that's in there that you're able to do that really justifies being in VR for that long. For me, it was about the immersion in the interactions that you do have your full attention and you can have some really quality interactions, but really takes people committing their full attention and committing to that level of embodiment as well. And like Kyle was saying, there wasn't really any 3d content within the world to be able to point at, to really make it this entire spatial experience, the whole gesturing aspect. And you know, what, what Blair was saying that leading up to this, he was watching some of these Kai lectures with some of his friends and pulling up different PDFs and be able to have these collaborative conversations. But. This was a theory that they had that, you know, a lot of people will be doing that. Well, they, first of all, they disabled a lot of those tools and because of performance considerations, but even if they did have them, I'm not sure if they would have still had that because you still have the social dynamics, which is that if you're in a movie theater or in a room listening to someone talk. Then it's against the normative standards to be able to carry on a lot of side conversations that may be disruptive of that transmission of whatever's being said. There can be ways that you can use text chat and other ways to silently augment those types of conversations. But when you have multiple audio streams happening at the same time, it's really hard to give your full attention to either one of them. And I think they discovered that to try to architect their immersive experiences around that didn't necessarily work out. And so really figuring out how to set a deeper context to have meaningful conversations. So facilitating those serendipitous collisions, uh, how to do that. The audio settings, uh, is a big question. The Q and a as a big part of the different live aspects, you know, how to really facilitate these larger emergent conversations, uh, within the structure of what you're creating, how to create embodied journeys through different spaces and make it worthwhile. Like the room that had like these three presentations. You were actually moving from space to space, but usually when you're in that context, you're actually really interested in listening to the talks and to engage in different sidebar conversations may actually be fairly distracting for some people. Also, you know, having Slack as a back channel for communication was a huge part. I know that another love all virtual, there wasn't any high bandwidth discord or Slack or any other real time chats and that. It tended to be a little bit harder to be able to have ways to connect with people. Although there was an ability to direct message people if they happen to be in world at the time, but thinking about how can existing communications platforms like discord and Slack be integrated. The discovery of social happenings, like where is something happening and you really want to find those happenings. And that can be a little bit difficult, especially if it takes this dilemma of people really dedicating themselves to be able to put themselves into the room. Also just giving these signals that you're open for possibility and open for conversation, whether that's, you know, giving your interests and trying to set time aside, or whether that there's a specific architecture of a place that you go to. I think one interesting idea is to think about the problems you're trying to solve as well as the questions you're trying to answer, and then to frame things around that. And could you start to either put that out into some way where you could start to facilitate different interactions based upon the problems you're trying to solve and the questions that you're trying to answer. especially at an academic conference like the IEEE VR. Because I think a lot of these different interactions, they're trying to facilitate relationships and fostering deeper collaborations. But as Kyle was saying, a big function of these gatherings is to just tap into the zeitgeist of what is happening in the larger research community. What are the trends? What are the patterns? And you start to get that when you not only are seeing what the topics are and what the conversations are, but there's a bit of a buzz that happens when you're at a co located conference. And you just hear different conversations and themes come up over and over again, you get a little bit of that zeitgeist helps to direct you in your future. And When a lot of those hallway conversations are abstracted into 2d teleports and not having those embodied collisions, then it really makes it difficult to tap into that deeper zeitgeist because there just aren't those emergent conversations that are happening to be able to source that from, and then getting people's full attention and to get people to really commit. How do you do that? to pay money and to travel, that is one level of commitment that actually is a forcing function for people's full attention. And in the absence of that, how can you really catalyze people to have your full attention? Is it gonna be moving into smaller bursts, spread out over time, where you have very specific birds of a feather, very specific meeting, where you're meeting at a specific time, or can you have an open-ended day where it's just architecture of a space is just open enough for people to go have those different types of hallway conversations in the absence of any deeper content that's being presented, or if it is being presented, is it a flipped classroom model? So it is made available for people to watch ahead of time, and then they have dedicated social time to be able to go and be able to interact with each other. And then finally, just a point of like, this is a work from home dynamics, uh, just having your family around, uh, the different levels of distraction. It's just very difficult for people to have their full attention. Virtual reality may be able to start to cultivate that, but really thinking about how the deeper architectures of these gatherings can really facilitate. People to have meaningful conversations and have their full attention within these interactions. So there's a lot more from my experiences from IEEE VR, you know, many multiple days, I'm probably end up talking to more people that have been doing different research. And I may end up doing a presentation that tries to really consolidate a little bit more of these different trade-offs, because I do think there is a wide range of different trade-offs as you're trying to design for these different gatherings. whether it's for an academic conference or for an industry gathering. But I really do think that there's a key difference between trying to just replicate what we've been doing before and that there was a natural side effect of getting people to commit and getting there to give a talk. And since they're already there giving a talk, then they're just make themselves available for meetings or make themselves available for those different conversations. How can you, in the absence of that, still get the people to commit to be able to show up and to be able to have A range of conversations or to be really focused as to what conversations you have, because you know, there are already a lot of experiences with social VR that just have generalized chat, but absence of any deeper context. So if you go into rec room or VR chat, there may be a wide range of different people from different contexts, different intentions. But the great thing about a conference is that it really focuses the context and focus the intention on to very specific problems you're trying to solve or a domain that you're in. And you have like a shared context and a culture that you're able to leverage from that, and then be able to find people who are a similar level of commitment in a similar context to be able to dive deep into Like I like to think of different intentions and motivations you want to do in the world, but you can't do it alone. And so when you go to these different gatherings, you're able to find potential collaborators or to find answers to your questions that you're having. So that's all that I have for today, and I just wanted to thank you for listening to the Voices of VR podcast. And if you enjoy the podcast, then please do spread the word, tell your friends, and consider becoming a member of the Patreon. This is a list of support of podcast. And so I do rely upon donations from people like yourself in order to continue to bring you this coverage. So you can become a member and donate today at patreon.com slash Voices of VR. Thanks for listening.
