Why This Sr. UX Researcher’s Radical Approach to Policy Development Became a Game-Changer

To research problem. OK. So I would say in my current role, um, so I was the, the first person to develop this role. So previously when I joined the organization that I'm at, um, I was working to develop our content moderation policies and so our policies were being developed in a very what I saw to be as a very haphazard way and so I um thought that it was very important for us to have some empirical insights, um, to Undergird our decision-making and so with that said, I uh spoke with many people both within my role and then also with my managers to um petition for this current role that I'm in where I do, um, provide that empirical texture to our decision-making. Um. And um which allows us to better, um, develop our content moderation policies and then also allows us to have more publicly defensible policies because it is rooted in research. Um, I think that, that, the reason why I would categorize that as radicals because, uh, many people weren't thinking about that. A lot of people, um, were taking again like a very arbitrary approach to research and start to not research but to developing a policy and so, um, Yeah, coming in and saying that we do need research to undergird um these decisions, um, was a very radical approach at the time and I think now in light of a lot of the controversy around content moderation and social media in general, um, many people see it as a very, um very positive radical approach as well because uh now we're better prepared to defend our policies publicly.

Yeah, I think um one of the reasons again why I would categorize it as radical is because uh we, the way it, it did slow down our develop our content, our content moderation policymaking process, but it did make the the processes much stronger, it did make the outputs much stronger rather. And so um So yeah, so a lot of times like there was just this idea that we need to get things out really quickly, etc. like we need to print out this policy coverage in this area um without really thinking about a lot of the um underlying ethos around why we were developing the policy and so this was a different approach to that because for example, if we're thinking about disinformation, there's a whole research canon on disinformation. So if we see a video with this Information and just develop a policy to address that single video we're missing the whole range of potential um disinformation videos that can exist on a platform. And so by taking a research approach, it allowed us to Um, defer to the empirical literature and then also defer to experts in these areas to be able to really address the, the broader ethos of what we're trying to address through our policies rather than having a kind of short-sighted solution. And so I would say that, that was probably why it was radical.

OK, yeah, so, uh, for context, I'm also not a UX researcher at the moment. I'm hoping to transition into that. So, um, by outputs being much stronger, what I mean is that we were able to better we were able to better capture violated content on the platform. And so for example, we have a series of data points. So for example, the violation rates, the appeal rates, appeal success rates, etc. So violation rates, for example, would be the, the The general statistics around number of violated videos that exist on the platform, appeal rates would be when someone disagrees with our moderation approach, so they appeal that decision, and then the appeal success rate is how successful is that person in appealing their decision or not. And so, um, kind of with that said, uh, we were able to better capture Violate of content as reflected in the violation rates and then also lower the appeal rates. So um by having clearer policy, many people understood why their videos violated the policy, so they were less inclined to appeal our decision and so um kind of those two numbers in tandem kind of showed that we were doing the right thing and taking the right approach and also better capturing harm on the platform.

Um, can you clarify what you mean?

Yeah, of course. Yeah. So, um, for example, in the, the vertical that I worked on, um, we, when I first started, we had very amorphous policies and so, um, Uh, for context, I was working in bullying and harassment, and so the policies were very amorphous and then also the content is very subjective. So if you were to think about disinformation, well, I guess this information could be subjective as well, but if you think about minor safety or if you think about illegal activity, for example, it's very black and white versus um something like bullying and harassment is very, it's a very gray area because it's also Interactive process and the same statement coming from a different person can have a different meaning. And so uh with that said, um, when I first started, we have very low um incidence of bullying and harassment as reflected in our statistics and so um I think that we saw that bullying harassment only made up 2.5% of all violated content in comparison to all of the issue verticals that we worked with them. However, through clarifying our, our policies and then also um externally messaging them, we were able to increase that number by 300% and so um It bullying harassment then made up roughly 8.5.

so, um,

OK, and it says I have a full connection, so I'm not sure, but I, yeah, I apologize for

OK. Um, so I was saying that, um, yeah, the metric went up 300%, um, and then our appeal rate went down roughly 10%, and I was saying that the appeal rate will never go to zero because bullying and harassment is a very subjective area, so people will always, um, Be saying something in a way that is uh intended differently than it's received either by the user or by the, the moderation team, for example. And so, um, But generally speaking, we were able to again increase the um The capturing of violated content by 300%, which was great. And then also we were able to use that approach so that was thinking solely about video violations and then we were also able to use that approach to think about the different features. So for example, comments or live stream or direct message, etc. And we were able to capture a lot of harm. So for example, in comments, bullying, harassment, we were able to find made up roughly 60% of all violated content because a lot of people bully one another in comments, and that was something that we weren't even thinking about, um. In the past, and so, um, again being able to take that, uh, research-based approach allowed us to not only improve the metrics that we were looking at but then also find other metrics that we weren't looking at and then address those areas as well.