record-linkage

Closed. This question is off-topic . It is not currently accepting answers. Want to improve this question? Update the question so it's on-topic for Stack Overflow. Closed 2 years ago . Is there a open source implementation for Fellegi-Sunter? Is this what you are looking for? Wikipedia: Record Linkage Check under "Software implementations" for possible solutions. Here are some open source implementations: http://github.com/dedupeio/dedupe (author of this) https://sourceforge.net/projects/febrl/ https://github.com/larsga/Duke 来源： https://stackoverflow.com/questions/5152217/is-there-a-open

I have several sources of tables with personal data, like this: SOURCE 1 ID, FIRST_NAME, LAST_NAME, FIELD1, ... 1, jhon, gates ... SOURCE 2 ID, FIRST_NAME, LAST_NAME, ANOTHER_FIELD1, ... 1, jon, gate ... SOURCE 3 ID, FIRST_NAME, LAST_NAME, ANOTHER_FIELD1, ... 2, jhon, ballmer ... So, assuming that records with ID 1, from sources 1 and 2, are the same person, my problem is how to determine if a record in every source, represents the same person . Additionally, sure not every records exists in all sources. All the names, are written in spanish, mainly. In this case, the exact matching needs to

I'm trying to use the Duke Fast Deduplication Engine to search for some duplicate records in the database at the company where I work. I run it from the command line like this: java -cp "C:\utils\duke-0.6\duke-0.6.jar;C:\utils\duke-0.6\lucene-core-3.6.1.jar" no.priv.garshol.duke.Duke --showmatches --verbose .\config.xml But I get an error: Exception in thread "main" java.lang.UnsupportedOperationException: Operation no t yet supported at sun.jdbc.odbc.JdbcOdbcResultSet.isClosed(Unknown Source) at no.priv.garshol.duke.datasources.JDBCDataSource$JDBCIterator.close(JD BCDataSource.java:115) at no

Here's an interesting problem. I have an oracle database with name & address information which needs to be kept current. We get data feeds from a number of different gov't sources, and need to figure out matches, and whether or not to update the db with the data, or if a new record needs to be created. There isn't any sort of unique identifier that can be used to tie records together, and the data quality isn't always that good - there will always be typos, people using different names (i.e. Joe vs. Joseph), etc. I'd be interested in hearing from anyone who's worked on this type of problem

I have a large database (potentially in the millions of records) with relatively short strings of text (on the order of street address, names, etc). I am looking for a strategy to remove inexact duplicates, and fuzzy matching seems to be the method of choice. My issue: many articles and SO questions deal with matching a single string against all records in a database. I am looking to deduplicate the entire database at once. The former would be a linear time problem (comparing a value against a million other values, calculating some similarity measure each time). The latter is an exponential