Towards a Deduplication Framework utilising Apache Spark
Niklas Wilcke
This paper is about a new framework called DeduPlication (DduP). DduP aims to solve large scale deduplication problems on arbitrary data tuples. DduP tries to bridge the gap between big data, high performance and duplicate detection.
Towards a Deduplication Framework utilising Apache
