[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: lynx-dev user agent when running lynx via crontab entry?
From: |
mattack |
Subject: |
Re: lynx-dev user agent when running lynx via crontab entry? |
Date: |
Fri, 9 Jul 1999 10:47:43 -0700 (PDT) |
On Thu, 8 Jul 1999, David Woolley wrote:
>Date: Thu, 8 Jul 1999 08:29:54 +0100 (BST)
>From: David Woolley <address@hidden>
>Reply-To: address@hidden
>To: address@hidden
>Subject: Re: lynx-dev user agent when running lynx via crontab entry?
>
>> /usr/local/bin/lynx -useragent='Mozilla/4.04 [en]' -nolist -dump
>> http://www.imdb.com/StudioBrief | mail address@hidden
>>
>> So, I was getting a "we're not letting you connect" message for the past
>> week or so.. and after actually reading the message, it is being sent back
>> because the useragent is empty.
>>
>
>Based on public statements by the IMDB operator in the UK, and some
>followup email, this rejection is almost certainly to prevent bulk
>fetching of pages by web crawlers. Blocking lynx from automatically
>fetching single pages would probably considered a desirable side effect,
>as that also causes pages to be fetched without the adverts being read.
Strange.. it _did_ work yesterday (my 4pm automated email of StudioBriefing).
..and I had made no further changes after the initial -useragent attempt.
- lynx-dev user agent when running lynx via crontab entry?, mattack, 1999/07/07
- Re: lynx-dev user agent when running lynx via crontab entry?, Kim DeVaughn, 1999/07/08
- Re: lynx-dev user agent when running lynx via crontab entry?, David Woolley, 1999/07/09
- Re: lynx-dev user agent when running lynx via crontab entry?,
mattack <=
- Re: lynx-dev user agent when running lynx via crontab entry?, David Woolley, 1999/07/10
- Re: lynx-dev user agent when running lynx via crontab entry?, Kim DeVaughn, 1999/07/10
- Re: lynx-dev user agent when running lynx via crontab entry?, David Woolley, 1999/07/10
- Re: lynx-dev user agent when running lynx via crontab entry?, Klaus Weide, 1999/07/10
- Re: lynx-dev user agent when running lynx via crontab entry?, mattack, 1999/07/10
- Re: lynx-dev user agent when running lynx via crontab entry?, David Woolley, 1999/07/11
- Re: lynx-dev user agent when running lynx via crontab entry?, mattack, 1999/07/10
- Re: lynx-dev user agent when running lynx via crontab entry?, mattack, 1999/07/10