How a Deep Dive into User Experience Uncovered Hidden Bullying Trends on a Major Platform

Next question. So, uh, in this one, you know, this is largely going to center around your research I'm guessing where you've worked in in in a team environment or even if you were challenged during your dissertation process or or any research process, but the the focus of this question is more to understand how you think. And, and how, you know, the, the layered ability of your thinking, right? And so I want to talk about a time where. You had a problem that you're trying to understand, that required you to go down kind of several levels to get to the actual answer. The the surface level answer was wrong, even the secondary answer was probably wrong, but you had to kind of go down a few levels to really get at the answer. What was it? Um, yeah, I, again, I would refer back to the, I think the first example that I use, um, in terms of how we, I developed that research approach that then ended up being incorporated into our practices. Um, so at the time I, I, I mentioned that we weren't really seeing, uh, bullying and harassment activity on the platform. And so, but we were, we had other data. So for example, we had uh other survey data that showed that users really cared about um their experiences with bullying harassment and then also with other um things as well. And so there was kind of this paradox where users. We're saying that they cared a lot about something that wasn't showing up on the platform and so we obviously knew that users weren't necessarily wrong about their experience, but we didn't have the um we didn't have a robust data capturing system to be able to to find it. And so with that said, um. One thing that I did was embark upon a survey, a global survey, um, that was done on my team's division on their own, to really see how bullying and harassment was manifesting on the platform, and then in tandem because a global survey takes a long time to put together and to administer, etc. um, I worked with our data science team to uh really refine our metrics as to how we were um thinking about bullying harassment, um. And uh thinking about the different ways that we we can capture it as well. And so, um, again, that helped to improve our internal data collection processes. Additionally, we also did focus groups with creators, um, um, so creators that express grievances with bullying harassment, we, um, were able to uh do focus groups with them, and then we found out that a lot of the features so live stream direct messages are where they experience a lot of the bullying harassment rather than solely through videos, etc. So kind of using all of these different um data points to triangulate. The findings around where Bing Crossman is happening and so, um, and then we were able to finally get the results of that global survey that took many months to do it really corroborated a lot of the insights that we were able to get along the way. And so with that said, we were able to take action before the survey results came out knowing that. wouldn't be this big panacea, but then, um, also knowing that it would be important to see how Bros was manifesting globally. So, um, by the time the survey came out, one of the benefits of doing those other, um, research. Um, activities was that we were already taking action on a lot of the major things that we, um, were able to discover in the survey, and then the survey just provided that larger empirical landscape upon which to situate um many of the new features or the new policies that we had developed, um, in light of those other findings that we found as well, um. So I've A question as a. User of the platform as well as other platforms that support video. My observation is based on just complaints from creators and whatnot is that uh as a juxtaposition to your platform, right? YouTube is not. Does not aggressively take down videos. However, uh, they recently made a change where uh they removed the dislike count because that was viewed as a weaponized mechanism from the community to harass somebody into doing more on the on the platform, right? They viewed it as a people could gang up and download a video and make it disappear from the stream. Fine. And so that was, that was the action that that Alphabet took uh to to solve and address that problem and and what I've seen more commonly. On your platform is creators complaining of way too aggressive takedown of content um through you know individuals just spam reporting a video and so it's interesting to me that you mentioned that that your focus group surfaced that they saw the harassing and the bullying coming from the comments and live streams I've seen certainly plenty of that. But I'm wondering, did, did what I just highlight surface. And if it did, in what way? Because what I'm trying to understand is what you researched versus what you surfaced versus how did you prioritize what you worked on. That's what I'm really trying to get at. Right, right, right. So, um, we have seen uh that like, um, where people do kind of reporting where there's a completely innocuous video and then someone doesn't like the user who uploaded the video. How people, yeah, go report the video. We have seen that, but that was not one of the, the more prevalent ways in which people were bullied on the platform. So for example, uh, we did internal research which showed that um For example, younger users were often uh bullied by people that they knew versus older people were uh experienced harassment from people that they didn't know because of, uh, more ideological, uh, arguments that were being held online. Versus with the younger users of this real world conflict that was then being transferred into the digital space, for example, and so, um, a lot of times it was things like, I think, uh, my name's Jonathan Hampton, like I think Jonathan Hampton's ugly like. Uh, go make fun of like, I'm just kidding, a lot of it was, uh, things like that, uh, rather than, uh, Jonathan uploaded a video of him dancing and like go report it. Um, that's interesting. And so, and then another thing again with a livestream, a lot of times live stream, and I guess this would be an example of what you were just referring to a lot of times when people uploaded live streams, people did ask a lot of people to infiltrate someone's live stream and then uh. just make it really silly comments or really hurtful comments, and so, um, but the, the approach that you would take to moderating that type of behavior. It's a lot different than if it were a video that were uploaded and so um we did take steps to um mitigate that type of behavior, for example, allowing users to have more controls in their livestream so before they uploaded, before they began their live stream, they could decide whether they only wanted members of their community or people who followed them back to be able to leave comments, whether the public could, etc. um. And so, um, we were able to provide feature changes in that regard, um, because, yeah, that would be like the most analogous example to the example that you mentioned, but, um, again, a lot of what we were seeing was in comments, and live stream, and then also um. Another thing that we were seeing was with the amorphous. Our very subjective uh policy description around bullying. A lot of moderators weren't capturing it, so we were able to for video policy and so we were able to tighten up those descriptions for video policy and then create features. Um, you might have heard of the rethink feature that came out like uh several months ago where it asked users to rethink a comment. Um I haven't had I haven't had that yet, so I guess I'm not nasty comments. So yeah, basically if the algorithm that you were leaving a hurtful, it would ask you to rethink whether you wanted to post it or not. Um, and so that was one of the approaches that we took for improving, um. Uh, like malicious behavior for comments for live stream we were able to create those other features where it's like, do we, um, do you want everyone to be able to comment or not? So internally, who whose counsel did you seek as you worked through all of this and and Kind of as a, as a follow on, what, what if there was a single thing that proved most valuable, what was the most valuable piece of information? Yeah, so in the role that I was in, I was actually the the global lead for this vertical, so um. I did obviously seek the counsel of my my managers. Oh, you dropped out again. Sought the lead from my manager's last thing you said. We'll give it a second, hopefully you'll come back.

OK, that's one of the things that I tried to do, um, so yeah, I, I sought counsel from my managers, but um I was generally seen as the global lead for this this content area and so um. I, I tried to use again, research findings either that we that were produced internally or produced from leading leading scholars in these areas to support my decisions. And then also um I also sought the counsel of the moderation teams as well, so it's one thing to write a really lofty policy, but then when it needs to be implemented in 5 seconds or 10 seconds, like, is it written well enough for people to understand what you're saying, you know? And so being able to figure out where the um the lapses. And understanding existed among the people that were actually implementing the policy was very important for me as well. And so by being able to incorporate their advice, that was all I feel like that was actually more helpful than getting the advice of my supervisors because they were that people that were actually on the ground implementing the policy. I think I might have froze. Yeah, your video is frozen. OK. Um. But, you know, we can kind of back. You're in, you're out, you're not. I'm guessing that you're, you're, you're on a shared Wi Fi for a building or something like that and it's just, it's complaining. That would be, yeah, I'm on my personal computer and usually I'm on my work computer, so like it might be something related to my computer as well. Right.