How can I store large amount of data from a database to XML (memory problem)?

前端 未结 4 661
情书的邮戳
情书的邮戳 2021-02-09 02:18

First, I had a problem with getting the data from the Database, it took too much memory and failed. I\'ve set -Xmx1500M and I\'m using scrolling ResultSet so that was taken care

4条回答
  •  悲&欢浪女
    2021-02-09 02:53

    Ok, so the code is rewritten and I'll include the whole operation:

    //this is the calling/writing function; I have 8 types of "proizvod" which makes 
    //8 XML files. After an XML file is created, it needs to be zipped by a custom zip class
           generateXML(tmpParam,queryRBR,proizvod.getOznaka());
       writeToZip(proizvod.getOznaka());
    
    
    
    //inside writeToZip
    
        ZipEntry ze = new ZipEntry(oznaka + ".xml");
        FileOutputStream fos = new FileOutputStream(new File(zipFolder + oznaka + ".zip"));
        ZipOutputStream zos = new ZipOutputStream(fos);
        zos.putNextEntry(ze);
        FileInputStream fis = new FileInputStream(new File(zipFolder + oznaka + ".xml"));
        final byte[] buffer = new byte[1024];
        int n;
        while ((n = fis.read(buffer)) != -1)
            zos.write(buffer, 0, n);
        zos.closeEntry();
        zos.flush();
        zos.close();
        fis.close();
    
    // inside generateXML
    PrintWriter writer = new PrintWriter(new BufferedOutputStream(new FileOutputStream(zipFolder +oznaka + ".xml")));
            writer.print("\n");
            writer.print("\n");
            stmt = cm.getConnection().createStatement(ResultSet.TYPE_SCROLL_INSENSITIVE, 
                    ResultSet.CONCUR_READ_ONLY);
            String q = "";
            rs = stmt.executeQuery(q);
            if(rs != null){
    
                System.out.println("Početak u : " +Util.nowTime());
                while(rs.next()){
                    writer.print("\n\t");
                    writer.print("\n\t\t" + Util.transformToHTML(rs.getInt("id")) + "");
                    writer.print("\n\t\t" + Util.transformToHTML(rs.getInt("jed_id")) + "");
                  //etc
                  writer.print("\n\t");
                }
                System.out.println("Kraj u : " +Util.nowTime());
            }
            writer.print("\n");
    

    But generateXML part still takes a lot of memory (if I'm guessing correctly, it takes bit by bit as much as it can) and I don't see how I could optimize it (use an alternative way to feed the writer.print function)?

提交回复
热议问题