Performance issue while inserting 2000 records using mybatis(3.2.8 version)

问题

I am trying to insert 2000 records in Employee table in batch (using mybatis). My requirements are: 1. To log the error if any of the record fails to insert. 2. To continue with the insertion even if any one of the record fails. 3. Rollback should not happen for other if any one of the record fails. 4. Good performance.

Sample code of Dao implementation: Here I have come up with 2 scenarios.

1. Calling sqlSession.commit() outside the loop.

       SqlSession sqlSession = MyBatisUtil.getSqlSessionFactory().openSession(ExecutorType.BATCH);
       BatchMapper batchMapper = sqlSession.getMapper(BatchMapper.class);
       try
       {
         for(Employee e: empList){
           batchMapper.addEmployee(e);
         }
       }
       catch (Exception ex)
       {
   
       }
       finally{
           sqlSession.commit();
           sqlSession.close();
       }
   

   In this case sqlSession.commit() is outside the for loop. Here insertion is happening for all the records at once after calling sqlSession.commit(). Here the performance is good and it takes 4 seconds to insert 2000 records. But I am not able to log the errors and also it stops the insertion when exception occures.

2. Calling sqlSession.commit() inside the loop.

       SqlSession sqlSession = MyBatisUtil.getSqlSessionFactory().openSession(ExecutorType.BATCH);
       BatchMapper batchMapper = sqlSession.getMapper(BatchMapper.class);
       try
       {
        for(Employee e: empList){
           batchMapper.addEmployee(e);
           sqlSession.commit();
         }
       }
       catch (Exception ex)
       {
   
       }
       finally{
           sqlSession.close();
       }
   

   In this case sqlSession.commit() is inside the for loop. Here insertion happens one by one when we call sqlSession.commit(). Here the performance is not good and it takes 10 minutes to insert 2000 records. But I am able to log the errors and continue with the insertion even if the exception occures for say 100th record.

Please help me with this. Thanks in advance.

回答1:

If you are using java 8, try parallel stream using lambda expressions.

empList.stream().parallel().forEach(s -> {
        try{
            batchMapper.addEmployee(e);
            sqlSession.commit();
        } catch(Exception ex){

        }
    });

来源：https://stackoverflow.com/questions/36036623/performance-issue-while-inserting-2000-records-using-mybatis3-2-8-version