java - FOP - how to avoid high memory consumption with very high number of page-sequences? -


how can avoid fop consume growing amount of memory when pages not contain forward-references , < page-sequence> blocks small?

here's test java program feeds fop hand made fo repeats on , on same basic page-sequence:

fo2pdf.java

import java.io.filenotfoundexception; import java.io.fileoutputstream; import java.io.outputstream; import java.io.pipedinputstream;  import javax.xml.transform.result; import javax.xml.transform.source; import javax.xml.transform.transformer; import javax.xml.transform.transformerfactory; import javax.xml.transform.sax.saxresult; import javax.xml.transform.stream.streamsource;  import org.apache.fop.apps.fouseragent; import org.apache.fop.apps.fop; import org.apache.fop.apps.fopfactory; import org.apache.fop.apps.mimeconstants; import org.xml.sax.helpers.defaulthandler;  public class fo2pdf implements runnable {  private pipedinputstream in;  public fo2pdf(pipedinputstream in)  {     this.in = in; }   @override public void run() {     // instantiate fop factory     fopfactory fopfactory = fopfactory.newinstance();     fopfactory.setstrictvalidation(false);      // setup output     outputstream out = null;     try {         out = new fileoutputstream("output.pdf");     } catch (filenotfoundexception e) {         e.printstacktrace();     }      try {         // setup user agent         fouseragent useragent = fopfactory.newfouseragent();         useragent.setconservememorypolicy(true);          fop fop = fopfactory.newfop(mimeconstants.mime_pdf, useragent, out);          // setup jaxp using identity transformer         transformerfactory factory = transformerfactory.newinstance();         transformer transformer = factory.newtransformer();           // setup input stream         source src = new streamsource(in);          // resulting sax events (the generated fo) must piped through fop         defaulthandler defaulthandler = (defaulthandler) fop.getdefaulthandler();         result res = new saxresult(defaulthandler);          // start fop processing         transformer.transform(src, res);      } catch (exception e) {         e.printstacktrace();     }     } }

feedfo.java

import java.io.ioexception; import java.io.pipedinputstream; import java.io.pipedoutputstream;   public class feedfo {   public static void main(string args[]) throws ioexception, interruptedexception {      // instantiate , connect pipes     pipedinputstream in = new pipedinputstream();     pipedoutputstream out = new pipedoutputstream(in);      // fo2pdf - instantiate , start consuming stream     fo2pdf fo2pdf = new fo2pdf(in);     thread fo2pdfthread = new thread(fo2pdf, "fo2pdf");     fo2pdfthread.start();      /*      <fo:root xmlns:fo="http://www.w3.org/1999/xsl/format">         <fo:layout-master-set>             <fo:simple-page-master master-name="a4" page-width="210mm" page-height="297mm">                 <fo:region-body/>             </fo:simple-page-master>         </fo:layout-master-set>       */     out.write(("<fo:root xmlns:fo=\"http://www.w3.org/1999/xsl/format\"><fo:layout-master-set>" +             "<fo:simple-page-master master-name=\"a4\" page-width=\"210mm\" page-height=\"297mm\">" +             "<fo:region-body/></fo:simple-page-master></fo:layout-master-set>").getbytes());       for(int i=0; i<100000000; i++) {          // sleep 3 seconds every 50000 page-sequences make sure consumer faster producer         if(i % 50000 == 0) {             thread.currentthread().sleep(3000);         }          /*          <fo:page-sequence xmlns:fo="http://www.w3.org/1999/xsl/format" master-reference="a4">             <fo:flow flow-name="xsl-region-body">                 <fo:block/>             </fo:flow>         </fo:page-sequence>          */         out.write(("<fo:page-sequence xmlns:fo=\"http://www.w3.org/1999/xsl/format\" master-reference=\"a4\"><fo:flow flow-name=\"xsl-region-body\"><fo:block/></fo:flow></fo:page-sequence>").getbytes());     }      out.write("</fo:root>".getbytes());     out.flush();     out.close();      fo2pdfthread.join();      system.out.println("exit"); } }

as notice, fop writes disk pdf page-sequence has been closed. means pages (should?) not being kept memory. but, memory keeps growing , growing. 256mb heap size, generation stops @ 150000 page-sequences.

why happening?

i suspect that, despite sleep call, producer working faster consumer , piped stream filling memory. 2 ways can think of fix this:

option 1 use blockingqueue instead of piped stream.

option 2 add public boolean pipeisfull() method fo2pdf returns true if in.available() exceeds, dunno, 2mb. main loop sleep 500ms or whatever if pipeisfull() true.

also, way reduce memory consumption is

byte[] bytes = ("<fo:page-sequence xmlns:fo=\"http://www.w3.org/1999/xsl/format\" master-reference=\"a4\"><fo:flow flow-name=\"xsl-region-body\"><fo:block/></fo:flow></fo:page-sequence>").getbytes(); for(int i=0; i<100000000; i++) {     ...     out.write(bytes); }

i don't know how significant of impact have (it'll reduce couple gb, that's peanuts compared fo2pdf using), can't hurt.


Comments

Post a Comment

Popular posts from this blog

Why does Ruby on Rails generate add a blank line to the end of a file? -

running rails generate controller foo home results in: class foocontroller < application controller def home end end (there's nothing on line in actual file, blank line) is there purpose blank line? it's common use newline indicate end of line, last line. many editors (like vi, use) add in newline silently. consensus text files (especially in unix world) should end in newline, , there have been problems historically if wasn't present. why should text files end newline? the tool used count lines in file "wc" counts newlines in file, without trailing newline show 3 instead of 4. it improves readability of templates used in generator. consider: https://github.com/rails/rails/blob/master/railties/lib/rails/generators/rails/controller/templates/controller.rb to remove trailing newline, template have last line of: end<% end -%> instead of: end <% end -%> that seems less readable me. the discussion below refers
Read more

keyboard - Smiles and long press feature in Android -

i've created android keyboard. how add features inserting smilies, , using long keypresses display symbols? to create long-press behavior allows select several options pop-up menu, follow guide (specifically pop-up menu section): http://developer.android.com/guide/topics/ui/menus.html smileys (also known emoticons) combination of keys, understood programs , parsed same way. have do, when clicks smiley face on keyboard, simulate pressing of key combination matching emoticon. example, when clicks smiley, should simulate key presses ':' , ')' :). here's list of emoticons: http://en.wikipedia.org/wiki/list_of_emoticons
Read more

node.js - Bad Request - node js ajax post -

i'm using express 3.0 , node v0.11.0. have button submits form , i'm using ajax post data json object node server. client code is: $('#contact').submit(function(e) { console.log('submit called'); var formdata = json.stringify($('form').serializeobject()); console.log(formdata); $.ajax({ url: "http://localhost:3000/savedata/", type: "post", datatype: 'json', data: {objectdata: formdata}, contenttype: "application/json", complete: function() { console.log('process complete'); }, success: function(data) { console.log('process sucess'); }, error: function() { console.log('process error'); }, }); return false; }); then on server: var express = require('express'); var app = express(); app.set('views', __dirname + '/views'); app.set('view engine', 'jade'); app.use(express.bodyparser());
Read more