When I came across a cluster of records related to the Space Shuttle Challenger accident, I discovered that the shuttle commander, Francis Dick Scobeeee, was considered a "Person of Exceptional Prominence" by the Archives, and that records of his Air Force service were available for public viewing... at the Archives Military Personnel Records Centerr in St .Louis, Missouri!
We hypothesised that the major cause of 'bad' clusters - the co-clustering of records derived from two different individuals - was the presence of a mistake, perhaps due to typographical error - in a single identifier.
The existence of this type of error detracts from otherwise appealing and efficient exact-match record linkage methods combining records sharing unique identifiers, as the identifier errors cause co-clustering of records from different individuals, referred to here as record collision, compromising overall linkage quality.
There aren't many better ways to spend a rainy Thursday afternoon in London than ducking into the cluster of record shops that populate the enclaves of Soho.
First, the Smith-Waterman algorithm is used to recognize pairs of approximate duplicates, then the union-find algorithm to keep track of clusters of duplicate records incrementally, as pairwise duplicate relationships are discovered.
The feature selection and cluster analysis of recorded impedance spectra were done using PCA and validated using clustering indices.
Testing on five data sources including 9.2 million records indicate that ~99.8% of clusters formed consist of records from 1 individual.
We estimated effects by multiple regression modelling, calculating robust standard errors to take into account clustering of individual records by the common specialties (seven inpatient, 18 outpatient) within the four trusts, resulting in 28 clusters for inpatient data and 72 clusters for outpatient data.
During the first step of pre-clustering of cases into small sub-clusters the initial ordering of records influences the resulting cluster distribution, because the sub-clusters are not resolved during the further analysis.
And the third is to create clusters of coherent related records, using graph drawing, agglomerative clustering methods and partitioning methods.
