Re: [Lynx-dev] sudden site access failure: realclearpolitics.com

[Rudy Vener]

Hi Travis,

First, to address the screen reader issue, in my vi edit buffer, where
I read all my email, I did:
:s/lynx/catlynx/g

Now as far as the links and catlynx behavior, it may be
that links spoofs the presence of the ad  rendering modules that the 
cloud flare gatekeeper requires, along with active cookies.

Now I normally turn on cookies in catlynx, but they seem to be hit or miss.

Rudy

On Wed, Nov 10, 2021 at 08:20:01PM -0500, Travis Siegel wrote:
> (screen readers will have a tough time with this message, because catlynx with
> a y and links with an I are sounding the same, so apologies in advance for
> any confusion).
> 
> 
> Apparently it isn't the agent header, because on my softcon.com server,
> links works (as it does for you) but catlynx does not.?? Even when I tell 
> catlynx
> to mimic the useragent string sent by links.
> 
> Interesting enough, the first time I tried catlynx with the links headers, I
> got a 408 error, which claimed the server took too long to respond, even
> though it only took a couple seconds to pop up.
> 
> I looked at the man pages for links, and I couldn't find anything that would
> account for the acceptance there and not on catlynx. <shrug>
> 
> 
> On 11/10/2021 3:48 PM, Rudy Vener wrote:
> > Hi Karen,
> > They are probably not blocking by IP or they would have blocked access from 
> > links (the chain) as well.
> > 
> > Thinking they might be blocking by a combination of both IP and User agent 
> > trigger, I changed the user agent
> > for catlynx (the cat) with another UA string I got from the online UA 
> > database.
> > Now I called  realclearpolitics.com with:
> > /usr/local/bin/catlynx-2.9.0-dev9 -useragent="Mozilla/5.0 (Macintosh; U; 
> > PPC Mac OS X Mach-O; XH; rv:8.578.498) fr, Gecko/20121021 Camino/8.723+ 
> > (Firefox compatible)" $*
> > 
> > No joy. Still blocked with catlynx (the cat.)  Links (the chain) still 
> > works.
> > 
> > I don't suppose you could try sending me shellworld's catlynx config file, 
> > could you?
> > 
> > On Wed, Nov 10, 2021 at 02:06:45PM -0500, Karen Lewellen wrote:
> > > Hi Rudy,
> > > Have  you tried this site today?
> > > Granted I am using the Lynx configuration at shellworld, but I just typed
> > > the  address you provided, and reached the page perfectly.
> > > something to consider about cloudflare is that it tracks sometimes by IP
> > > address.
> > > Would love to learn how the fanfiction.net site works with your user agent
> > > though, smiles.
> > > Karen
> > > 
> > > 
> > > 
> > > On Wed, 10 Nov 2021, Rudy Vener wrote:
> > > 
> > > > Hi all,
> > > > 
> > > > Yesterday realclearpolitics.com started returning the following page
> > > > when I tried to access it with catlynx.
> > > > 
> > > >    Please enable JS and disable any ad blocker
> > > > 
> > > > I invoke catlynx as follows:
> > > > /usr/local/bin/catlynx-2.9.0-dev9 -useragent="Mozilla/5.0 (Macintosh; 
> > > > Intel Mac OS X 10_8_0) AppleWebKit/537.1 (KHTML, like Gecko) 
> > > > Chrome/21.0.1180.79 Safari/537.1 catlynx" $*
> > > > 
> > > > The above user agent spoof normally lets me slip the Cloud Flare 
> > > > gatekeepers, but this seems to be something new.
> > > > 
> > > > For what it's worth, the same problem now occurs with w3m, however,
> > > > links (the chain)  can access the site just fine.
> > > > If there is a simple configuration fix for this, I'd really like to 
> > > > know.
> > > > 
> > > > Thank you.
> > > > 
> > > > -- 
> > > > Rudy Vener
> > > > Website: http://www.rudyvener.com
> > > > Twitter: https://twitter.com/RudySalt
> > > > 
> > > > 

-- 
Rudy Vener
Website: http://www.rudyvener.com
Twitter: https://twitter.com/RudySalt