Data Ethics Club meeting 31-05-23, 1pm UK time#

Meeting info#

Description#

This week at Data Ethics Club we are discussing a Twitter thread by Owen Jones about a study where machine learning was used to analyse three million tweets mentioning MPs for ‘toxicity’. You can read more about the study in this BBC News Article.

The basic premise of this sort of sentiment analysis is that you isolate occurrences of certain comments, for example “you are/you’re a disgrace” “you are/you’re a liar” and evaluate the sentiment associated with them. In the case of this study, the sentiment is classification with an associated probability: you classify how likely is something to be toxic.

The BBC’s Shared Data Unit used Perspective, a tool that uses artificial intelligence to spot toxic comments online. Developed by Jigsaw, a research unit within Google, it defines a toxic comment as one which is “rude, disrespectful or unreasonable” and “likely to make someone leave a conversation”.

The team analysed all tweets mentioning MPs from March to Mid-April. The article goes on to discuss the effect of toxic tweets on MPs.

Owen Jones’ twitter thread portrays this study in a different light, taking issues with the conflation of being called a “disgrace” or a “liar” with racist or sexist abuse. For example, “you are a hypocrite” was classified as abuse. The thread goes on to point out that the most negative pushback in a tweet towards an MP was classified as toxic. Where is the line? Should AI be drawing the line?

One Twitter user tested Perspective on phrases associated with Nazi views: “We must secure the existence of our people and a future for white children” which was deemed 34.33% to be toxic, “I have 14 words for you” which was deemed to be 9.48% toxic and finally “You are a poo poo head” which was deemed to be 76.52% toxic.

Discussion points#

  1. The BBC News Article states: “Machine learning algorithms allow researchers and journalists to measure a phenomenon at a scale which would otherwise not be feasible with other methods.” Do you agree with this? Why/why not?

  2. How can we decide if something is toxic? Who/what should be the ones to decide this?

  3. Should the line of what counts as toxic be different if you’re someone in the public eye?