Investigating what AI detection tools consider to be generative AI in code and text

Sammanfattning

Detection tools for artificial intelligence have the purpose of detecting the likelihood that some content is generated by generative AI. Similarly to the generative tools themselves, the detection tools follow complex algorithms to return the results of the likelihood. As such, whether the detection tools are reliable or not, altering parts of the content, even minor changes, can lead to different results. In this thesis, we tried to find out what the effects of making specific changes to parts of text and code could have on three different detection tools, for the purpose of finding out what is considered more AI or not by them. We found that overall in text, using symbols such as ampersands or having choppy sentence lengths caused texts to be considered more likely to be human-written, while heavily simplified text was considered more likely to be AI generated. For C++ code, on the other hand, the introduction of lambda functions was deemed to be less likely to be AI generated. However, this was not consistent in all AI detectors, showing that the detectors do not work the same and are not built the same.