Insert CLOB into Oracle database

Question

My question is: How do you get around the ORA-01704: string literal too long error when inserting (or doing anything in queries) with CLO

Answer 1

You are making it way to complicated.

Use a PreparedStatement and addBatch() for each clob in your list:

String sql = "insert  into " + tempTableName + " values (?)";
PreparedStatement stmt = connection.prepareStatement(sql);
for (String query : readQueries) {
  stmt.setCharacterStream(1, new StringReader(query), query.lenght());
  stmt.addBatch();
}
stmt.exececuteBatch();

No messing around with escaping strings, no problem with the length of the literals, no need to create temporary clobs. And most probably just as fast as using a single INSERT ALL statement.

If you are using a current driver (> 10.2) then I think the setCharacterStream() call and the creation of the Reader is not necessary either. A simple setString(1, query) will most probably work as well.

Answer 2

Check out some CLOB related samples on github.

Answer 3

Me, I like to use the classes from java.sql.* package, not oracle.* stuff. For me the simple approach

Connection con = ...;
try (PreparedStatement pst = con.prepareStatement(
     "insert into tbl (other_fld, clob_fld) values (?,?)", new String[]{"tbl_id"});
     ) {
        Clob clob = con.createClob();
        readIntoClob(clob, inputStream);
        pst.setString(1, "other");
        pst.setClob(2, clob);
        pst.executeUpdate();
        try (ResultSet rst = pst.getGeneratedKeys()) {
            if (rst == null || !rst.next()) {
                throw new Exception("error with getting auto-generated key");
            }
            id = rst.getBigDecimal(1);
        }  

stopped working when testing (current tomcat, jdbc) moved into production (stuck in Tomcat6 for stupid reasons). con.createClob() returns null for reasons unknown in that version, so I had to do this double-take (It took me ages to figure out so I'm sharing here...)

try (PreparedStatement pst = con.prepareStatement(
         "insert into tbl (other_fld) values (?)", new String[]{"tbl_id"});
     PreparedStatement getClob= con.prepareStatement(
         "select clob_fld from tbl where tbl_id = ? for update");
     ) {
        Clob clob = con.createClob();
        readIntoClob(clob, inputStream);
        pst.setString(1, "other");
        pst.executeUpdate();
        try (ResultSet rst = pst.getGeneratedKeys()) {
            if (rst == null || !rst.next()) {
                throw new Exception("error with getting auto-generated key");
            }
            id = rst.getBigDecimal(1);
        }  

        //  fetch back fresh record, with the Clob
        getClob.setBigDecimal(1, id);
        getClob.execute();
        try (ResultSet rst = getClob.getResultSet()) {
            if (rst == null || !rst.next()) {
                throw new Exception("error with fetching back clob");
            }
            Clob c = rst.getClob(1);
            // Fill in data
            readIntoClob(c, stream);
            // that's all 
        }

    } catch (SQLException) {
       ...
    }

for completeness here's

// Read data from an input stream and insert it in to the clob column
private static void readIntoClob(Clob clob, InputStream stream) {
    try (BufferedReader bufferedReader = new BufferedReader(new InputStreamReader(stream))) {
        char[] buffer = new char[CHUNK_BUFFER_SIZE];
        int charsRead;
        try (Writer wr = clob.setCharacterStream(1L)) {
            // Loop for reading of chunk of data and then write into the clob.
            while ((charsRead = bufferedReader.read(buffer)) != -1) {
                wr.write(buffer, 0, charsRead);
            }
        } catch (SQLException | IOException ex) {
            ...
        }

    }
}

which is from elsewhere on SO, thanks.

Answer 4

From Oracle document

You must bear in mind the following automatic switching of the input mode for large data. There are three input modes as follows: Direct binding, Stream binding, and LOB binding.

For PL/SQL statements

The setBytes and setBinary stream methods use direct binding for data less than 32767 bytes.

The setBytes and setBinaryStream methods use LOB binding for data larger than 32766 bytes.

The setString, setCharacterStream, and setAsciiStream methods use direct binding for data smaller than 32767 bytes in the database character set.

The setString, setCharacterStream, and setAsciiStream methods use LOB binding for data larger than 32766 bytes in the database character set.

The setBytesForBlob and setStringForClob methods, present in the oracle.jdbc.OraclePreparedStatement interface, use LOB binding for any data size.

Follow is a example for put a file content into a input CLOB parameter of a PLSQL procedure:

  public int fileToClob( FileItem uploadFileItem ) throws SQLException, IOException
  {
    //for using stmt.setStringForClob method, turn the file to a big String 
    FileItem item = uploadFileItem;
    InputStream inputStream = item.getInputStream();
    InputStreamReader inputStreamReader = new InputStreamReader( inputStream ); 
    BufferedReader bufferedReader = new BufferedReader( inputStreamReader );    
    StringBuffer stringBuffer = new StringBuffer();
    String line = null;

    while((line = bufferedReader.readLine()) != null) {  //Read till end
        stringBuffer.append(line);
        stringBuffer.append("\n");
    }

    String fileString = stringBuffer.toString();

    bufferedReader.close();         
    inputStreamReader.close();
    inputStream.close();
    item.delete();

    OracleCallableStatement stmt;

    String strFunction = "{ call p_file_to_clob( p_in_clob => ? )}";  

    stmt= (OracleCallableStatement)conn.prepareCall(strFunction);    

    try{    
      SasUtility servletUtility = sas.SasUtility.getInstance();

      stmt.setStringForClob(1, fileString );

      stmt.execute();

    } finally {      
      stmt.close();
    }
  }

Answer 5

BLOB (Binary Large Objects ) and CLOB(Character large objects) are special datatypes and can hold the large chunks of data in form of objects or text. Blob and Clob objects persist the data of the objects into the database as a stream.

An example piece of code:

public class TestDB { 
    public static void main(String[] args) { 
        try { 
            /** Loading the driver */ 
            Class.forName("com.oracle.jdbc.Driver"); 

            /** Getting Connection */ 
            Connection con = DriverManager.getConnection("Driver URL","test","test"); 

            PreparedStatement pstmt = con.prepareStatement("insert into Emp(id,name,description)values(?,?,?)"); 
            pstmt.setInt(1,5); 
            pstmt.setString(2,"Das"); 

            // Create a big CLOB value...AND inserting as a CLOB 
            StringBuffer sb = new StringBuffer(400000); 

            sb.append("This is the Example of CLOB .."); 
            String clobValue = sb.toString(); 

            pstmt.setString(3, clobValue); 
            int i = pstmt.executeUpdate(); 
            System.out.println("Done Inserted"); 
            pstmt.close(); 
            con.close(); 

            // Retrive CLOB values 
            Connection con = DriverManager.getConnection("Driver URL","test","test"); 
            PreparedStatement pstmt = con.prepareStatement("select * from Emp where id=5"); 
            ResultSet rs = pstmt.executeQuery(); 
            Reader instream = null; 

            int chunkSize; 
            if (rs.next()) { 
                String name = rs.getString("name"); 
                java.sql.Clob clob = result.getClob("description") 
                StringBuffer sb1 = new StringBuffer(); 

                chunkSize = ((oracle.sql.CLOB)clob).getChunkSize(); 
                instream = clob.getCharacterStream(); 
                BufferedReader in = new BufferedReader(instream); 
                String line = null; 
                while ((line = in.readLine()) != null) { 
                    sb1.append(line); 
                } 

                if (in != null) { 
                    in.close(); 
                } 

                // this is the clob data converted into string
                String clobdata = sb1.toString();  
            } 
        } catch (Exception e) { 
            e.printStackTrace(); 
        } 
    } 
} 

Answer 6

You'll need to use bind variables rather than building a SQL statement using string concatenation. This will be beneficial from a security, performance, and robustness standpoint as well since it will reduce the risk of SQL injection attacks, decrease the amount of time Oracle has to spend doing hard parses of the SQL statement, and will eliminate the potential that there is a special character in the string that causes an invalid SQL statement to get generated (i.e. a single quote).

I would expect that you want something like

private void insertQueries(String tempTableName) throws FileNotFoundException, DataException, SQLException, IOException {
  String preQuery = "  into " + tempTableName + " values (?)" + StringHelper.newline;
  StringBuilder inserts = new StringBuilder("insert all" + StringHelper.newline);
  List<String> readQueries = getDomoQueries();
  for (String query : readQueries) {
    inserts.append(preQuery);
  }
  inserts.append("select * from dual");

  Connection conn = ConnectionPool.getInstance().get(connection);
  PreparedStatement pstmt = conn.prepareStatement(
        inserts);
  int i = 1;
  for (String query : readQueries) {
    Clob clob = CLOB.createTemporary(conn, false, oracle.sql.CLOB.DURATION_SESSION);
    clob.setString(i, query);
    pstmt.setClob(i, clob);
    i = i + 1;
  }
  pstmt.executeUpdate();
}