DRTX: A Duplicate Resolution Tool forXMLRepositories

Abstract

Detecting duplicates in XMLis not trivial due to structural diversityand object dependency. This paper suggests a duplicate detection and resolution tool (DRTX) which is an efficient XMLduplicates detector and resolution that applies two famous techniques of duplicates detection, normal edit distance (NED) and token based damerau-levenshtein distance algorithm (TBED) thencompare the results and suggests thebetter similarity for each of them. DRTX is not only a duplicate detection and resolution system but it also provides two extra services: -first the XMLfile merger which is used to merge XMLdocuments thus solves the structure heterogeneity problem, second dirty XMLgenerator which is used to insert known duplicate problems on clean XMLfile to apply the mentioned algorithms on that file therefore explore how much the system can detect accurately these problems.Tominimize the number of pair-wise element duplicates comparison, a set of filters were used to increase the efficiency of DRTX while its effectiveness is adjustable. Experimental results show that there is no algorithm better than the other but each of them hasits own use ie.NED is better to use at lower threshold similarity values while TBED is better at higher ones