Multimodal Content Filtering using Artificial Intelligence

Sammanfattning

Artificial intelligence (AI) offers a wide range of opportunities for enhancing contentmoderation. This thesis investigates the effectiveness of AI-based content filtering whenbrowsing the internet, doing so through artifact design and evaluation. Using the designscience research methodology, a browser extension was conceptualized, designed, andimplemented. This extension, written in JavaScript, integrates AI content moderation throughAPI services, namely Google’s PerspectiveAPI for text analysis and Sightengine for imageanalysis, with the primary goal is to explore how AI can help in detecting explicit content thatis inappropriate, harmful or distracting and block or limit access to it, so that the students havea safer and more focused study environment. An evaluation was then conducted using variousdatasets, where the APIs were evaluated using a Python script to assess their effectiveness attext and image analysis. The study focuses on two areas: how effective AI is at identifyingdifferent types of explicit content in text and image media, and what ethical and legalconsiderations need to be addressed when implementing AI-based filtering, especiallyregarding bias, privacy, and transparency. Dataset evaluation showed that both services wereeffective at detecting offensive content, especially in terms of recall. For offensive text, recallreached up to 91% at a threshold of 0.50, while maintaining an F-1 score of 71%. At lowerthresholds, the model was able to better identify offensive texts, obtaining an offensive recallscore of 94% and an F-1 Score of 87% at a threshold of 0.3. In real-time browsing, a thresholdof 0.50 resulted in successful filtering of explicit websites. For image filtering, recall was alsohigh, but came with a trade-off in precision, leading to more false positives in complex cases.This reflects the models' cautious approach, which can be useful in educational settings.Additionally, the study briefly states ethical and legal concerns related to scraping websitecontent for filtering, and the public datasets that were used in evaluation. In conclusion, theproject has shown the potential and practicality of pre-built AI services in real-time contentfiltering, as well as identifying challenges such as threshold tuning, bias, and modeltransparency that must be addressed for broader application.