Re: [Lynx-dev] Alternative User-Agent header

[Karen Lewellen]

Travis
I am going to write you privately for help with a test.
as for reaching anyone at fanfiction.net,  Here is what I was told to do, 
since they only read their twitter feed.
Someone suggested sending copies of the messed up captcha to them over and 
over and over again.  It got their attention once for a mac user, who was 
getting the same cloudflare barrier, in safari no less.
Thing is, I am not on twitter.
Anyone here a member?
Karen



On Fri, 25 Jun 2021, Travis Siegel wrote:

> Karen, It's not just you.  I've run into this cloudflare thing on other 
> sites.  I've even jumped through the hoops to get the accessibility cookie, 
> and I still get hit with their captcha at times, so the cookie is no 
> guarantee of access, no matter what they claim to the contrary.  I 
> understand they provide a service many small hosts can't financially handle 
> on their own, but honestly, I don't understand why they work so hard to make 
> problems for users.  Sites are supposed to be browser neutral, and 
> cloudflare takes what might actually be a browser neutral site, and turns it 
> into something that isn't at all what the site designers built, and that 
> doesn't make for good business practices.  It's probably a useless thought, 
> but have you tried getting ahold of someone from fanfiction.net to see if 
> they have any suggestions? It's possible (though not likely) they could 
> provide you an IP address of a server that isn't run through cloudflare.
>
> On 6/25/2021 12:35 AM, Karen Lewellen wrote:
> >  Interestingly enough I do know there are settings in cloudflare that can
> >  allow lynx  to work..a few sites I visit use cloudflare, but lynx is not
> >  blocked. even edbrowse cannot get past the fanfiction.net door, and its
> >  use of JavaScript is more current then links the chain and e-links,
> >  neither of which  work.
> >  Fanfiction.net itself does not require JavaScript, I honestly wish I could
> >  reach the owner and share cloudflare suggestions that still keep them
> >  safe, but open the door.
> >  Your fanfiction server idea sounds like hacking, would not let me use my
> >  years worth of bookmarks again, and as you say produces a lesser result.
> >  In fact I wonder if that is how fanfiction.ws was working until the main
> >  site shut it down.
> >
> >  Do not even get me started on the real problem where the captcha is
> >  concerned, hcaptcha which is the service cloudflare is using that gets run
> >  into with lynx.
> >  Hcaptcha claims they want to be accessible, had a heated discussion with
> >  the
> >   staff a few weeks back.  Their own site  requires JavaScript to get
> >  the accessible cookies, which is counterproductive.
> >  Karen
> >
> >
> >  On Thu, 24 Jun 2021, Travis Siegel wrote:
> >
> > >  fanfiction.net is using cloudflare to help distribute the load of web 
> > >  requests.  Cloudflare apparently requires javascript, cookies, and 
> > >  something in the browser string that identifies the browser as a 
> > >  standard browser.  Two of those you can fix with lynx, but you're never 
> > >  going to solve the javascript issue just using lynx. I've not tried 
> > >  hitting fanfiction.net using one of the various encarnations that can 
> > >  handle javascript, it may or may not work. I also haven't tried changing 
> > >  my identification string, though I doubt highly  that will solve the 
> > >  problem.  I'm guessing that if you could find the ip address of the 
> > >  fanfiction.net actual servers, and not the cloudflare servers they use 
> > >  to spread the workload, then you could bypass all these checks, and then 
> > >  the site may or may not work properly, but at least then you'd have a 
> > >  fighting chance. I'm not convinced cloudflare will let lynx through no 
> > >  matter what you do try to get around it.
> > > 
> > > 
> > >  On 6/24/2021 6:31 PM, Karen Lewellen wrote:
> > > >   Do you mind testing that by visiting
> > > >   Www.fanfiction.net
> > > >   Since you  say you can browse  everywhere?
> > > > 
> > > > 
> > > > 
> > > >   On Thu, 24 Jun 2021, Stef Caunter wrote:
> > > > 
> > > > >   I can browse anywhere using this alias for 'lynx'
> > > > > >   alias lynx='/usr/local/bin/lynx -nopause -tna -useragent=IE . '
> > > > > >   "IE" seems to be the most innocuous choice for the major news 
> > > >  sites. I
> > > > >   have not tried "googlebot" ;)
> > > > > >   /stef
> > > > > >   _______________________________________________
> > > > >   Lynx-dev mailing list
> > > > >   Lynx-dev@nongnu.org
> > > > >   https://lists.nongnu.org/mailman/listinfo/lynx-dev
> > > > > 
> > > >   _______________________________________________
> > > >   Lynx-dev mailing list
> > > >   Lynx-dev@nongnu.org
> > > >   https://lists.nongnu.org/mailman/listinfo/lynx-dev
> > > 
> > >