Re: [Openexr-devel] EXR texture memory overhead

[Karl Rasche]

1. The amount of memory that libIlmImf holds *per open file* as overhead or internal buffers or whatever (I haven't tracked down exactly what it is) is much larger than what libtiff holds as overhead per open file.

I *think* this is related to what you're seeing, at least in part

ImfInputFile.cpp line 678

 

2. libIlmImf seems to have a substantial amount of memory overhead *per thread*, and that can really add up if you have a large thread pool. In contrast, libtiff doesn't have a thread pool (for better or for worse), so there isn't a per-thread component to its memory overhead.

Some of that probably stems from the framebuffer model -- You don't decode directly into the user-provided buffer, but instead into a temp buffer which is copied into the user-provided buffer and reformatted as requested.

That avoids things like tons of extra decodes of a scanline strip of you are walking it scanline by scanline. But in the context of texturing, where you're always reading a full tile into cache, it's just overhead. There might be a sneaky way to flush that backing data, like assigning a null framebuffer or something.