[Pan-devel] Re: Re: Want to fix memory consumption issues

[Duncan]

Calin A. Culianu posted
<address@hidden>, excerpted
below,  on Mon, 31 May 2004 16:53:54 -0500:


> 
> On Mon, 31 May 2004, Duncan wrote:
> 
>> I'm not a source groker myself, but the consensus has always been []
> 
> Well struct Article eats [] about 300-400 bytes of data per header.  1
> million headers is already 400 megabytes.. so I would say struct Article
> isn't helping matters any..
> 
> 
>> The trouble as I understand it is that PAN uses them as they were never
>> intended to be used, forcing them to scale to entry-counts they were
>> never intended to handle.  The problem is exacerbated by PAN using what
>> is primarily a GUI widget that happens to have minor data-widget
>> capabilities, as a data-widget that happens to be a GUI-widget as well.
> 
> Hmm.. so all headers are loaded into the header pane widget, on TOP of
> being simultaneously stored in a big linked-list of struct Article?!?!
> Eeek!!

I DID say /as/ /I/ /understand/ /it/.  Maybe that's what was changed when
PAN went from bogging down at ~ 200k overviews to ~ 1M overviews..

> I too have been playing lately with sqlite and trying to design some
> reasonable tables to handle the needs of pan.  I have a good amount of
> database programming experience too :).

Cool!  =:^)  Pan needs it.

> I have a lot of professional DB experience.  I am working out now how to
> best design the schema, so that it is quick to extract information such
> as parent/child relationships between articles for threading, and so
> that sorting on any field is quick too.
> 
>> Look in the archives for the previous discussion, and go from there,
>> would be my suggestion.
>> 
>> 
> Okay.. it would be worthwhile to see what other people's thoughts were
> on the db design -- perhaps someone has already worked out a pretty good
> way to organize the data already so I don't re-invent the wheel.
> 
> I will look into it some more and see what I come up with.

You may already be ahead of the work that had been done, but as I said,
see the archives for a bit of discussion.  Charles had previously
mentioned that he had done a bit of preliminary work on it.  I expect it
was just that, rather preliminary, and I'm guessing he didn't do any
testing or anything as you are already doing.  However, if you could get
what he's already done, it's possible you can figure out better where he
was going with it, so "y'all get pulling in the same direction from the
get-go," so to speak.  =:^)

(I'm taking a bit longer to reply now than I might normally, due to
working on Gentoo as a dual boot, taking me away from my already
functioning Mandrake, so..)

-- 
Duncan - List replies preferred.   No HTML msgs.
"They that can give up essential liberty to obtain a little
temporary safety, deserve neither liberty nor safety." --
Benjamin Franklin