JPlag – Detecting Software Plagiarism

JPlag, like similar plagiarism detectors, is vulnerable to attack. We outline the attack in this paper and show its effectiveness against JPlag and another widely used plagiarism detector, Moss. Note that this was written in 2020, in the pre “CheatGPT” era!

https://arxiv.org/abs/2010.01700

Mossad: Defeating Software Plagiarism Detection

Breanna Devore-McDonald, Emery D. Berger

Automatic software plagiarism detection tools are widely used in educational settings to ensure that submitted work was not copied. These tools have grown in use together with the rise in enrollments in computer science programs and the widespread availability of code on-line. Educators rely on the robustness of plagiarism detection tools; the working assumption is that the effort required to evade detection is as high as that required to actually do the assigned work.

This paper shows this is not the case. It presents an entirely automatic program transformation approach, Mossad, that defeats popular software plagiarism detection tools. Mossad comprises a framework that couples techniques inspired by genetic programming with domain-specific knowledge to effectively undermine plagiarism detectors. Mossad is effective at defeating four plagiarism detectors, including Moss and JPlag. Mossad is both fast and effective: it can, in minutes, generate modified versions of programs that are likely to escape detection. More insidiously, because of its non-deterministic approach, Mossad can, from a single program, generate dozens of variants, which are classified as no more suspicious than legitimate assignments. A detailed study of Mossad across a corpus of real student assignments demonstrates its efficacy at evading detection. A user study shows that graduate student assistants consistently rate Mossad-generated code as just as readable as authentic student code. This work motivates the need for both research on more robust plagiarism detection tools and greater integration of naturally plagiarism-resistant methodologies like code review into computer science education.

I recently had to check code from some of my students at the university as I suspected plagiarism. I discovered JPlag which works like a charm and generates nice reports

I was actually looking for something like this a few days ago!

There’s an open source tool which I love the idea of (basically a tool for declarative integration tests), but I really don’t like it’s implementation. I tried to contribute to improve it, but it’s too much work and it will never fit my ideal.

So I basically decided to "redo it but better", and I’m also tempted to make it a paid, proprietary tool because my implementation diverges enough that I consider it a different codebase altogether (and it would bring legitimate value to companies). I wrote my code from scratch but still had some knowledge of the original code base so I’d be interested in running something like JPlag to make sure I didn’t accidentally plagiarize open source code.

I hope I find a way to make it compare 2 codebases :)

One of the key outcomes of my master's thesis was the development of an extendable solution for Code Clone Detection (CCD), primarily focused on code and tested with undergraduates at my university [1]. Although I didn't have time to complete the adapter for JPlag, I believe it would be highly beneficial.

Interestingly, whenever I discussed my thesis, the first reaction from others often revolved around moral concerns.

[1] https://github.com/studyfair/studyfair

Presumably, this needs a corpus of software to check against. Does it include one, or do you have to bring your own?

Should a plagiarism score be considered when generating code with an infinite monkeys algorithm with selection or better?

Would that result in inability to write code in a clean room, even; because eventually all possible code strings and mutations thereof would already be patented.

For example, are three notes or chords copyrightable?

pro tip: change variable/function names, switch from if/else to switch, invert if/else statements, switch for and while loops, group code differently, create helper functions or collapse helper functions, rewrite loops as streams/ranges/list comprehensions, etc. The ide can do most automatically.

It is pretty much impossible to detect software plagiarism, especially on leetcode style questions as only 1 style or pattern is the most efficient answer.

Though if a student changes it sufficiently enough, they might begin to actually see the invariants and ideas and actually learn the material.

It's funny how we drill the idea that everything must be reimplemented from first principles into students, only to flip that when they join the workforce.

Can I use it to detect Copy & Paste within my company's own codebase?

Is this also effective at detecting code duplication within a codebase?