Re: [Gnu3dkit-dev] Scene size limits

[Philippe C . D . Robert]

On Donnerstag, Oktober 10, 2002, at 01:48  Uhr, Brent Gulanowski wrote:
> On Wednesday, October 9, 2002, at 05:28  PM, Philippe C.D. Robert 
> wrote:
> > Incremental loading from disk is the ultimate goal. I do not know how 
> > high its priority really is.
> >
> > > I guess I'm just asking if we want to support incremental loading of 
> > > scenes. I can think of a couple of ways to handle it, depending on 
> > > the design of the G3DScene/G3DSceneRep.
> >
> > What do you have in mind?
>
> Here's some requirements:
>
> 1. A way to put manageable breaks into the file.
> 2. Find a missing section on disk.
> 3. Loading/unloading without affecting the rendering process.
> 4. Attach and detach sections in memory.
> 5. Identify, at the right time, a section that is missing and needed, 
> or one that is loaded but no longer needed.
> 6. Store changed details about a section being unloaded.
>
> Actually, number 6 should be about the same regardless of whether 
> using incremental loading. Now, I've never implemented a state change 
> list, but I imagine it's like I learned in A.I. class -- the pristine 
> scene is the start state, and you keep track of all of the (relevant) 
> changes to the state as they happen. For an incremental scene, you 
> might want the state changes organized in sections, too. This could be 
> left to apps to manage (requiring the app to be aware of different 
> scene sections).
>
> File formats for incremental scenes I won't really get into. If we had 
> such a thing that RenderKit had intimate knowledge of, the sections 
> would be marked for size and location. One tricky thing about an 
> indefinitely large scene stored on disk incrementally is preserving 
> the idea of an entire scene as a single graph. I guess you just add 
> more group structures and push the hierarchy a bit higher. Anyway, if 
> the scenes are sectioned on disk, you would have a way to find a 
> section that you needed.
>
> For 3, whenever I consider one thread issuing aperiodic tasks to 
> another thread (threads seem essential), I think of a queue. If a 
> scene is needed, a request is put on a queue. The loader thread picks 
> this up, loads the section, and when the renderer is not touching the 
> link point, it connects the loaded section into the existing scene. 
> Issue 4 just means synchronizing the two threads for a moment, 
> probably using a Lock.
>
> Issue 5 is, I think, easy or difficult depending on how flexible it 
> has to be. It would vary by app, perhaps dramatically. It's based on 
> how quickly the camera can consume scenery by moving around the scene. 
> If it can tear through huge gobs of scenery, either because it moves 
> quickly in world co-ordinates or because the scene is exceptionally 
> complex, that's going to be really challenging. But if you're running 
> some kind of straightforward architectural walk-through with modest 
> complexity, it would be easy. So, either you leave it up to the app to 
> perform this decision-making, you use one catch-all method, or you 
> have a few and provide a means for the app to give you a hint about 
> which one is best. I'm becoming a fan of the hint pattern in OpenGL. 
> Even if you only have one method of doing something, the user can give 
> you hints and you can ignore them. You can add new methods later.
>
> As for a file containing an indefinite number of scene sections... say 
> it was all archived in native format. You need an alias from the 
> parent node to the separate section, similar to the alias used for 
> multiple references to the same object. But this kind of alias you 
> don't want it to blindly follow. Based on its internal criteria, the 
> app would give go-ahead whether to load a section or not. This logic 
> might go in the node's -initWithCoder:. In fact, no, -initWithCoder: 
> would send a message that it had an alias, and skip over it. The 
> receiver of the message would note the node and the alias. When the 
> section was loaded, the section's root node would be added as a child 
> of the node which held the alias. This is where we need a lock checked 
> and set (it might be the same lock we'd check/set from the 
> state-update thread). [A question for thread management: lock 
> granularity...)
>
> An aliased child node couldn't be lumped in with other children, say 
> in an array, I don't think. If a node keeps an array of "normal" 
> children, it would have to have two separate ivars for a section link: 
> the section identifier (the alias), and a pointer to the section root, 
> set to nil when it is not loaded. Or would you suggest a different 
> class of node to hold an alias? Or perhaps a proxy object, which gets 
> thrown out when the section gets attached? That means fussing about 
> with array members.

I don't want to go into details right now (at this moment). I think 
your suggestions contain some pretty advanced features which we have to 
consider at a later point.

Right now I think it is crucial that we design a flexible 
G3DDatabase/G3DScene ( or how we call it ... ) design which will make 
it possible to implement incremental file loading later.

Anyone?

-Phil
--
Philippe C.D. Robert
http://www.nice.ch/~phip