Hey Y’all,

On Insights, we draw on a wide range of scientific inquiry and structured thinking methods to try and represent players’ needs, desires, behaviors, and thoughts in Riot’s strategic decision-making. In past posts we’ve talked a lot about some of the quantitative analysis stuff we do, but we wanted to start sharing some of the other ways the Insights team supports decision-making at Riot.

In this post, we’ll talk through how observational research labs have contributed to the design and development of the Emote system in LoL. While quantitative analyses are often better-suited to helping us understand the “What” of behavior (analyzing the actions of large groups of players), Labs help us explore more of the “Why” of behavior (getting deeper on discussing how individual players think and feel about their actions or experiences).

Emotes as an Expression System

Our first designs on an “in-the-moment expression” system came from thinking through how we were already, and weren’t yet, serving player needs for expression and recognition in LoL. We realized we had some general in-game expression and customization opportunities with skins, smart pings, and champ mastery, but didn’t have a way to actively represent how you were feeling at any particular moment during a game of League. We also weren’t quite sure if this system should be more about functional communication (“Let’s take Baron!”) or more about emotional expression and recognition (“Damn, nice play dude!”). With this as a starting point, we started to test what such a system could be… and if it would actually feel worth using.

Our main goals in running research labs on the Expression system were:

• Determine where on the spectrum of “expression” to “communication” this system should sit
• Thread the needle of minimizing gameplay interference while making the system memorable and fun
• Observe how players use and interact with this kind of system design
• Get a feel for how interesting or appealing a system like this would be

A couple of the many considerations we kept in mind during the labs were:

• Because the lab can be a somewhat artificial environment, we had to consider that this was a place where we may get a unidirectional signal. If players in the lab didn’t use them at all, then that would give us a pretty good signal that players in the “real world” wouldn’t use them much. If players in the lab used them a ton, then that wouldn’t necessarily mean players in the “real world” would use them a lot or would use them in exactly the same ways.
• This lab was held at our LAX campus. The one with the giant Annie & Tibbers statue right as you walk in. So another thing we have to consider in lab feedback is that often the players coming in are super excited to be there, and to be getting to try stuff that no one else has seen yet. Sometimes that excitement can make things feel cooler or more interesting than they’d actually feel if the player had just come across them naturally in a game of League.

With early Expression system prototypes, we invited several LoL players in to try it out and share their thoughts. Here’s what these labs looked like:

Emote Lab Tasks

First, we talked through the players’ prior experiences with whatever they thought of as “Emotes” in League. We did this first to get a baseline of these players’ perceptions before they were exposed to, and potentially influenced by, any new material.

Then we introduced some system and content prototypes and concepts to get their initial reactions. We guide players to focus their feedback specifically around their own experience, so we can understand how they interact with the product… rather than understand their logic in anticipating how others will experience the product.

Next, we had 5v5 playtests with a prototype Emote system to observe and experience the system during League gameplay.

After playtests, each player filled out a quick individual survey about their experience. We give this survey immediately to get players’ initial feelings, unfiltered and unbiased by post-game discussion with other players or researchers.

Finally, we had a group discussion to understand general or shared impressions. These discussions can also sometimes offer a peek into how community or conversational dynamics can shape—or reshape—a player’s perceptions.

Observations

During those tasks, we also observed and recorded the players (with their express permission, of course!). This helped us to better understand the in-the-moment experience of using the system. Here’s one player experience that increased our confidence that we were on the right path on emphasizing the expression aspects of the system. It happened in a game where one team was getting crushed, and it made the one-sided stomping a bit more light-hearted and fun for both teams:

Discussion

Two memorable quotes that stood out from group discussions were:

Designer Observations

The game designers making the system were also observing the research labs in a separate room, through livestream. Designers get really excited to see players actually seeing and trying and breaking and loving and hating their stuff! A couple of things that came out of live designer observations were:

Screen clutter concerns during teamfights → Led to the ammo system + breakage rules

Usage seemed concentrated around non-gameplay intensive time anyway → Design for these times, like greetings when getting to lane, etc.

We didn’t interpret these observations to confirm solid truths about the player experience, but we used them to form clear hypotheses to test in future research, design, and analysis.

After The Lab

What happens after we’ve finished a lab session? Lots of reviewing, comparing notes and video, forming conclusions, and figuring how to communicate that back to the design and development team.

In this article, we’ve walked through the tasks and observations for only one lab. In totality, Emote design was informed by several research labs with players, countless internal playtests, live testing and analysis (Worlds Team Emotes, Snowdown Emotes, URF Mode Emote, Mastery Emotes)… and that’s just the Insights contribution. Insights work like this is one of many pieces of support for the heavy lifting that all the designers and developers put into building awesome products.

The Emote System Today… and Beyond!

Our system design is informed by lab work, but we still have room to grow in living up to this being a super commonly used expressive tool in League. We only launched with a few pieces of content, but we’re excited to be adding more content into the Emote system, including a variety of new ways for players to engage in this form of expression without spending RP.

We’re always iterating: With the Emote system in the wild now, we can do even more analysis on how players are using the system so we can keep honing and improving it. We’re going to continue building and filling out emotes towards being a robust system for expression!

Why We Lab

The Emotes system design is just one example of how we use in-person player experience and feedback to guide decision-making and amplify the positive player impact of our product design.

While the goal of running labs (and doing any other Insights work) isn’t to tell products what changes to make or how to design their content, they can have a big impact on overall direction. For the Emote system, running labs helped us pinpoint humanizing players and interacting during downtimes in game intensity as key areas for the design team to explore.

We value this direct player feedback so highly, in fact, that we’ve recently completed construction on a brand new lab space that will take us from being able to test with 12 players at a time to over 60! We’ve also leveled up our tech to unblock researchers in the types of testing they’re able to do. For example, our workstations are now equipped with two separate PCs — one strictly for gaming and one strictly for recording and streaming to product teams. This ensures that framerates and gameplay aren’t inhibited by software running in the background.

What’s Next for Labs

Currently, most of our labs run out of our Los Angeles campus, and obviously this isn’t perfectly representative of our global player base. We test some of our biggest changes in other regions—we recently sent a team to China and Korea to test the new Runes Reforged system, and previously went to a range of regions for the League Client update and Summoner’s Rift update—but we don’t yet have a process nailed down that lets us routinely run international labs. This is one of the biggest opportunities for the Labs team to explore in 2018, but for the foreseeable future, Los Angeles will still be the headquarters of gameplay testing.

If you’re thinking: “I live in the LA area and I’m hyped as hell about labs, where do I sign up,” well, we’re trying to figure out how to unlock an opt-in system, but currently there are some major logistical and methodological hurdles that make that impossible. For now, the best way to get involved with player labs is to keep your eyes peeled for surveys hitting your email inbox or game client—all of our recruiting starts with a survey.

Thanks for taking a look into how we think about labs at Riot! Let us know if you’ve got any questions or thoughts in the comments section below! Members of our Lab Ops and Insights teams will be hanging around to nerd out about anything from research methodology and lab space design—to wild speculation about what kinds of emotes we’d like to see in LoL in the future. :)