lynx-dev Lynx does not open some http sites, forces download

[Mykola Sereda]

        The purpose of this posting is to outline the experience
gathered from the analysis of the error (see Subject: header).

        Summary of my opinion: the error is not related to the
version number. The error is caused by MIME, or more precisely,
the disorder that its implementation introduces in the transmission
of 8 bit text files.
        Summary of my conclusion: best of all, the intervention of
MIME should be abolished and forbidden for the text files; short of
it, all application programs, such as Lynx, ought to provide some
possibility to opt out of MIME. - "Transmit the 8 bit text and mind
your own business!"

        Review. 
        ~~~~~~~
        In the original posting, I said: 

|        The version of Lynx installed on the server (Sun4, SunOS)
|of my ISP is 2.7.1.
|
|        There is a problem with many sites in Ukraine (not all). 
|
|        Instead of opening a www site, the Lynx presents a screen
|with a choice: Download or Cancel. If one cancels, one is dead, the
|session is over before it begins. The Download choice is followed
|with a disposition choice: save on the server, or mail to my local
|post. When saved, the http file looks like any other. At least I
|can not tell any difference.

        What I discovered, with the help of kind netizens, whose
postings I will (partially) comment below, is that there is no
problem, as long as the source site does not specify anything
about its CharSet, or sticks (actually falsely) to some "ISO
Latin 1" choice. Then, their texts, be they in koi8-u or KOI8-R,
or whatever other, pass through like a knife through water.

        The problem begins, when sides must "negotiate" about
what is the contents. And some weird things happen. When I
switched, in Options, to KOI8-R CharSet, I was able ideed to
enter into the sites whose avowed specification was (is) KOI8-R.
But - the KOI8 files were corrupted! They were (and continue to
be, at this moment) subject to some sort of non-reversible, bizarre
transformation. As of now, I have not figured out, who does it,
Lynx or the remote server.

        The only partial work-around I found, is to switch the
Options choice of CharSet during the session. You enter into
the desired site with KOI8-R choice, and switch right after entering
to ISO Latin 1, before opening and saving the desired files. The
text which passes through in the first instance, when KOI8-R choice
is still valid, is busted.

        I do not like this song and dance, to tell you the truth,
and feel like suggesting to hang the inventor of the MIME on the
first tree ...


>From: Philip Webb <address@hidden>
>Subject: Re: lynx-dev Lynx does not open some http sites, forces download
>Date: Fri, 29 Jan 1999 12:38:44 -0500 (EST)

        [ ... ] 

>> @cam % lynx http://www.vechirniykyiv.com/

>i repeated this using 2-8-1rel.1 & got the document without any problem:
>it's a newspaper -- the main articles listed are different today --
>with quasi-Russian text in Latin letters, which i can read sort-of

        Now I understand why you saw Latin letters. I saw them too,
when I opted for KOI8-R. That's garbage. Normally, you should see
the characters above 128, in whatever fashion your terminal or PC
displays them. 

        [ ... ] 

>it looks like a problem with your charset options:
>someone else may have more expert advice -- LP ? -- ,
>but try changing things in the Options screen.

        Right. Thanks for the advice. Of course, after that there was
(is) the problem with some weird transformations ... 

>however, you will need to get 2-8-1 first:
>2-7-1 doesn't have anywhere near as much international character support.

        Give me less support, not more! I beg you. And stick the MIME
in the throat of its inventor. 



>From: "Leonid Pauzner" <address@hidden>
>Date: Sat, 30 Jan 1999 16:24:42 +0300 (MSK)
>Subject: Re: lynx-dev Lynx does not open some http sites, forces download

        [ ... ] 

>>>it does look as if your problem is out-of-date character support (below).

>|       Character support? It seems, in options, I have specified koi8-u
>                                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
>looks like your ISP hacked lynx manually since no koi8-u charset recognized
>by lynx of any version, I am not sure whether they introduce new problems.
>I whonder, however, how could you see koi8 and 1251 pages with lynx 2.7.1,
>I had a similar problem with windows-1251 pages when using lynx before 2.8

        First, please let me use this opportunity and correct myself.
Indeed, I specified koi8-u, but that was in Elm. Sorry.

        Like I said, I always kept the ISO Latin 1 specification for Lynx,
and it worked flawlessly for the transmission of whatever 8 bit text, be
it 1251 or KOI. Until some people started to comply with MIME and specify
KOI8-R on the issuing server. I do not think version makes any difference.


>Resent lynx versions (2.8+) keeps mapping of different character sets and
>allow automatic translation between "document charset" and "display charset"
>if they are different, using latin approximations if the display fonts does
>not have a requested character.

        I wish they would just leave us alone. Why is it anybody's business
to stick their nose into the text being transmitted between two or more
consenting adults? Saving those who are drowning is the basic and primary
responsibility of those who are drowning. 

        MIME recognizes only registered CharSets. Damn it, they ought to
at least allow an option of choosing "8-bit text, whatever it is".

        I know how cumbersome and difficult it is to register a set, 
having been involved somewhat in the registration process of koi8-u.
Also, some time ago, I have put together a charset, which allows the
simultaneous presence (and the utilisation, obviously) of English,
French and Ukrainian texts. Now, the community of those who wish to
communicate using those three languages at the same time in the same
file, is not all that great. However, if it were not for MIME, which
often stands in the way and screws things, we would peacefully talk
to each other, bothering no one.

        One might say - why not use Unicode. Why not. And why not try
and kill a frog with a piece of heavy artillery, or a tractor. 


>I encourage you to try the latest development version since it is most safe
>and bugfixed (compared against 2.8.1), this is not a rule but currently is.

        Well, - but did it solve the problems, described in this thread? 


>Date: Sat, 30 Jan 1999 09:05:01 -0500 (EST)
>From: Ismael Cordeiro <address@hidden>
>Subject: Re: lynx-dev Lynx does not open some http sites, forces download

        [ ... ] 

>>and I never have problems with sites with whatever character sets they use
>>(koi8 or 1251) for that matter.

>Probably because those other sites were not sending "Content-Type:
>text/html; charset=koi8-r" like the one you mentioned.

        Bravo, Ismael! That's what finally solved the mystery!

        After this advice, I finally understood, - and started to play
with my Options. With the results described above. 

        [ ... ] 

>I don't know Russian or Ukranian but it seems that you should get it right.
>When I set "display (C)haracter set" to KOI8-R I get the document displayed.

        Sure. But of course you do not know that it dispays some mangled
garbage, the result of a non-reversible transformation. 

--
        Mykola Sereda
        Ingenieur
        Gerelo Intercontinental
        Ingenieurs-Conseils
        6700, rue Sherbrooke est, CP 358
        Montreal, Quebec H1N 3L3
        CANADA

        Internet: address@hidden