c# - Cleaning up memory after reading a giant xml element value -


i turn here help, driving me crazy: i'm reading xml file wraps arbitrary number of items, each b64-encoded file (and accompanying metadata it). read whole file xmldocument, while cleaner code, realized there's no limit on size of file, , xmldocument eats lot of memory , can run out if file large enough. rewrote code instead use xmltextreader, works great if issue program sent xml file large number of reasonably-sized attachments... there's still big problem, , that's turn you:

if xml reader @ file element, element contains value that's enormous (say, 500mb), , call reader.readelementcontentasstring(), have string occupies 500mb (or possibly outofmemoryexception). in either case write log, "that file attachment totally way big, we're going ignore , move on", move onto next file. doesn't appear string tried read ever garbage collected, happens string takes ram, , every other file tries read after throws outofmemoryexception, though of files quite small.

recall: @ point, i'm reading element's value local string, have expected eligible garbage collection immediately (and garbage collected, @ latest, when program attempts read next item , discovers has no memory available). i've tried everything, in case: setting string null, calling explicit gc.collect()... no dice, task manager indicates gc collected 40k, of ~500mb requested store string in, , still out of memory exceptions attempting read else.

there doesn't seem way know length of value contained in xml element using xmltextreader without reading element, imagine i'm stuck reading string... missing something, or there no way read giant value xml file without totally destroying program's ability further afterwards? i'm going insane this.

i have read bit c#'s gc, , loh, nothing read have indicated me happen...

let me know if need further information, , thanks!

edit: did realize process running 32-bit process, meant being starved memory bit more should've been. fixed that, becomes less of issue, still behavior i'd fix. (it takes more and/or larger files reach point outofmemoryexception thrown, once thrown, still can't seem reclaim memory in timely fashion.)

i had similiar issue soap service used transfer large files base64 string.

i used xdocument instead of xmldocument then, did trick me.


Comments

Post a Comment

Popular posts from this blog

Why does Ruby on Rails generate add a blank line to the end of a file? -

running rails generate controller foo home results in: class foocontroller < application controller def home end end (there's nothing on line in actual file, blank line) is there purpose blank line? it's common use newline indicate end of line, last line. many editors (like vi, use) add in newline silently. consensus text files (especially in unix world) should end in newline, , there have been problems historically if wasn't present. why should text files end newline? the tool used count lines in file "wc" counts newlines in file, without trailing newline show 3 instead of 4. it improves readability of templates used in generator. consider: https://github.com/rails/rails/blob/master/railties/lib/rails/generators/rails/controller/templates/controller.rb to remove trailing newline, template have last line of: end<% end -%> instead of: end <% end -%> that seems less readable me. the discussion below refers ...
Read more

keyboard - Smiles and long press feature in Android -

i've created android keyboard. how add features inserting smilies, , using long keypresses display symbols? to create long-press behavior allows select several options pop-up menu, follow guide (specifically pop-up menu section): http://developer.android.com/guide/topics/ui/menus.html smileys (also known emoticons) combination of keys, understood programs , parsed same way. have do, when clicks smiley face on keyboard, simulate pressing of key combination matching emoticon. example, when clicks smiley, should simulate key presses ':' , ')' :). here's list of emoticons: http://en.wikipedia.org/wiki/list_of_emoticons
Read more

node.js - Bad Request - node js ajax post -

i'm using express 3.0 , node v0.11.0. have button submits form , i'm using ajax post data json object node server. client code is: $('#contact').submit(function(e) { console.log('submit called'); var formdata = json.stringify($('form').serializeobject()); console.log(formdata); $.ajax({ url: "http://localhost:3000/savedata/", type: "post", datatype: 'json', data: {objectdata: formdata}, contenttype: "application/json", complete: function() { console.log('process complete'); }, success: function(data) { console.log('process sucess'); }, error: function() { console.log('process error'); }, }); return false; }); then on server: var express = require('express'); var app = express(); app.set('views', __dirname + '/views'); app.set('view engine', 'jade'); app.use(express.bodyparser());...
Read more