Re: [Pan-devel] Re: ancient DB schema

[K. Haley]

Duncan wrote:

> K. Haley posted <address@hidden>, excerpted below,
> on Fri, 04 Jun 2004 18:56:11 -0600:
>
>  
>
> > GROUP
> > A newsgroup.
> > (it looks to me like pan treats folders the same as news groups. `folder`
> > is not needed if seperate FOLDER and FOLDER_ARTICLE tables are used.)
> >    
>
> The "looks like" statement is correct. Charles has stated that it's
> deliberate that folders are treated exactly the same as groups,
> simplifying the logic as there's only one set to deal with, not a separate
> set for folders and for groups.
>
> One gotcha is tracking read and unread articles.  PAN apparently uses the
> server added xref line for that purpose.  Since that's a server-local
> added header, a few server implementations don't add it, and folks have
> from time to time complained that messages on these servers don't get
> marked as read, properly.
>
>  
The DB schema I posted tracks read by article msg-id.  To retrieve an 
article body from the server pan would get the UID assigned by the 
server from the G_S_A table which maps the (server, group, article) 
tuple to the UID.  This way when an article is marked read, it gets 
marked for all groups on all servers that it was posted to.  The UID is 
the only guaranteed way to retrieve an article.

> Additionally, at one point as an performance enhancing experiment, PAN
> tried not actually retrieving user-posted messages, but substituting the
> local copy instead.  This failed with the same mark-as-read problem,
> because the local copy didn't have the server supplied xref header. 
> Additionally, it was I believe decided that it was more important to see
> what actually posted, in case it got mangled or something, than the bit of
> speed saved by not downloading those articles (particularly since PAN
> doesn't do attachments so the articles aren't that long anyway).
>
>  
This is were we run into problems.  The article table currently hold one 
entry per msg-id.  If we keep the folder articles and group articles in 
the same table then what happens with posted articles?  I suppose the 
header info would not get mangled so one entry could still work.  Hmm, I 
suppose we would then need two article caches.  One for the downloaded 
articles and another for the folders.  When viewing an article we would 
check the article cache first then the folder cache, and when viewing 
folder articles we would only have to check the folder cache.  That 
could work.

> Back to folders.  It might be useful to have the ability to add old
> folders back in, at some point.  I wouldn't worry about this functionality
> now, except that we shouldn't break the eventual possibility by limiting
> the database record format.  Basically what I'm saying, is that at some
> point it would be nice to have a folder import function, that would either
> read the messages and build the individual fields from them as possible,
> or substitute reasonable defaults where the data cannot be gathered,
> including if the message doesn't contain a "required" header.
>  
That shouldn't be difficult to implement.

signature.asc
Description: OpenPGP digital signature