SEO professionals all over the globe scrape search results and information from numerous sources like search engines and social media. It is very much unlikely that this “crud” data will not have duplicates. In-fact most of the times there is just too much duplicate data to deal with. Then again, most of the scrape data are maintained and managed in spreadsheets. Its not very easy to organize them quickly and easily either.

Demo: http://codecanyon.net/item/duplicate-finder-seo-tool-csv-to-mysql/4701831