How ignore duplicate rows when insert

Question

I use hibernate-jpa-2.1-api. And I need some functionality.

I parse a file every minute and insert data into MSSQL DB. I need to skip duplicate rows. For e

Answer 1

Database style option

Hibernate does not offer to add an option to its insert into statements. And I don't know if the same option is available for MS SQL.

But if you find such an option, you can intercept the insert statement and add that yourself:

public class IgnoreRowOnDupInterceptor extends EmptyInterceptor {

  public String onPrepareStatement(String sql) {
    if (sql.startsWith("insert into avaya_cm_cdr") {
      return sql.replace("insert into", 
        "insert /*+ ignore_row_on_dupkey_index(avaya_cm_cdr, i_avaya_cm_cdr_nodub) */ into");
    }
    return sql;
  }

} 

You need to declare this interceptor in your persistence.xml:

<property name="hibernate.ejb.interceptor" value="...IgnoreRowOnDupInterceptor" />

JPA style option

You could remember the last line from the last parsing (or retrieve it from the database) and skip the file until that line. In that case you even would save the time to parse every existing item again and again.

From my point of view this is the JPA way, because you usually use the database only as storage and keep the business logic in the (Java) application.