I need a tool to find duplicates or similar blocks of text in a singular text file or set of text files

Question

I want to automate moving duplicate or similar C code into functions.

This must work under Linux.

Answer 1

See CloneDR, a tool for finding exact copy and near-miss (copy-paste-edit) clones in source code. It uses full language parsers to enable it to find clones according to the language structure, minimizing false positives, and to be completely indendent of how the code is commented or formatted, thereby maximing true detection. The CloneDR will find clones when the cloned block has changed variable, inserted statemens or blocks of code.

It has language front ends for C, C++, COBOL, C#, Java, PHP and a number of other langauges.

You can see sample clone detection reports at the website.