Re: [Sks-devel] Request for reporting of upstream bandwidth capacity

[kristian . fiskerstrand]

Hi Jeff,

This is basically what the various measurement clients around the world  are 
doing to estimate the first part of the equation, the responsetime, which is 
the factor with the highest loading.

It retrieve my key 0x6b0b9508 that is currently approx 66 KiB on every run, 
measure the conn time, then smoothing is performed with a decay of older 
records. The latest 200 records are used from each of the measuring clients, 
corresponding to just over 8 days as the pool is now updating every hour. 

Then I add weight based on the reported BW and further add weight to rprox 
enabled servers to get to the final SRV w for each of the pools (eu,na,oc,sa)

(aside)
It would be very interesting to receive feedback from users of the various 
geo-pools on whether any changes have been noticed from the client side

Sent from my BlackBerry® smartphone on Telenor

-----Original Message-----
From: Jeffrey Johnson <address@hidden>
Date: Thu, 17 May 2012 16:15:58 
To: Kristian Fiskerstrand<address@hidden>
Cc: <address@hidden>
Subject: Re: [Sks-devel] Request for reporting of upstream bandwidth capacity


On May 16, 2012, at 2:35 PM, Kristian Fiskerstrand wrote:

> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA256
> 
> On 2012-05-16 20:22, Robert J. Hansen wrote:
>> On 05/16/2012 01:12 PM, Kristian Fiskerstrand wrote:
>>> As upstream bandwidth capacity is one of the considerations that
>>> is taken into account in [0] I would appreciate if the server
>>> operators that have not already done so would kindly email me
>>> this information off list (My UIDs in 0x6b0b9508 can be used for
>>> this).
>> 
>> This is sort of a black art.  Depending on which online service is 
>> providing the test, I've seen my upload and download speeds vary by
>> more than two orders of magnitude.  Further, since pretty much all
>> cable ISPs were at one point in league with the Devil before
>> finally usurping Lucifer's throne, I can't even rely on the vague
>> promises made by their tech support imps and balseraphs.
>> 
>> This isn't to say there's no point in what you're doing -- I think 
>> there's a lot of merit to it! -- but if we all use a different
>> service we're going to introduce an awful lot of statistical noise.
>> Is there any particular bandwidth capacity test that you would
>> prefer we use?
>> 
> 
> Hi Robert,
> 
> No, I don't have a particular test in mind. As you say, it might
> introduce some noise. I can however, to some extent, control this by
> the loading given to this particular parameter ($\beta_B$), but the
> more data I get - the more I can play around with this for the actual
> SRV weights.
> 
> Currently the $\beta_B$ is set to 450, and I have total upstream
> recorded at about 4500, so a 100 Mbit/s connection only add 10 to the
> total weight. This is linear, so a Gbit connection would add 100 to
> the weight. This compare to the $\alpha$ (the base weight) of 150 for
> all servers.
> 
> Most of the SRV weight is however determined by the responsetime
> measured from various clients ($\beta_R$….

BTW, there is perhaps another approach to estimating "bandwidth capacity".

I run "continuous integration" buildbots here
        http://harwich.jbj.org:8010
Part of the automation testing is retrieving a "known set" of pub keys
(through multiple crypto stacks) on a path to automating *.rpm
package signing.

I'm currently using the SKS pool URI with this RPM peculiar configuration
        %_hkp_keyserver         hkp://pool.sks-keyservers.net
        %_hkp_keyserver_query   %{_hkp_keyserver}/pks/lookup?op=get&search=

(aside)
Note that I can change this configuration to use my own private SKS key servers
easily if the traffic bothers anyone; I'm hoping to use the pool and SRV weights
eventually as a means to use "fastest" or "nearest" key server.

The buildbot behavior leads to a constant (~every 2h by 5-10 buildbots 
distributed
between 2 geographic points in NA) polling measurement. You might see a trace 
like this
in db.log (I just happened to notice this on "keys.n3npq.net")

Short answer:
        If the IP address from the pool were logged, then the actual
        "bandwidth capacity" of a known set of pubkeys from a single client
        to multiple servers in pool might be retrieved form server logs

I can also easily add the fingerprint of some huge pubkey with
embedded photoid: just name the fingerprint.

This might be an easier way to assess "bandwidth capacity"
objectively, and also start setting up some global polling
points using a cron driven curl script on a known periodic
retrieval that could be grep'ed out of server db.log files and
collected: even a daily e-mail fully automated would be very
KISS and objectively (rather than subjectively) accurate.

hth

73 de Jeff


2012-05-17 15:56:45 /pks/lookup: Get request (0x58e727c4c621be0f)
2012-05-17 15:56:45 /pks/lookup: Get request (0x5d385582001ae622)
2012-05-17 15:56:45 /pks/lookup: Get request (0xa520e8f1cba29bf9)
2012-05-17 15:56:45 /pks/lookup: Get request (0x9ac53d4d)
2012-05-17 15:56:45 /pks/lookup: Get request (0x7ad0becb)
2012-05-17 15:56:46 /pks/lookup: Get request (0x7c611479)
2012-05-17 15:56:46 /pks/lookup: Get request (0x1cfc22f3363deae3)
2012-05-17 15:56:46 /pks/lookup: Get request (0xb873641b2039b291)
2012-05-17 15:56:46 /pks/lookup: Get request (redhat n3npq johnson jeff jbj com 
ars)
2012-05-17 15:56:46 /pks/lookup: Get request (russell com coker au)
2012-05-17 15:56:47 /pks/lookup: Get request (0xd5ca9b04f2c423bc)
2012-05-17 15:56:47 /pks/lookup: Get request (0xbc916af40d001429)
2012-05-17 15:56:48 /pks/lookup: Get request (0x6bddfe8e54a2acf1)
2012-05-17 15:56:48 /pks/lookup: Get request (test software redhat rawhide 
project fedora com)
2012-05-17 15:56:48 /pks/lookup: Get request (us security rpms linux fedora)
2012-05-17 15:56:48 /pks/lookup: Get request (signing redhat rawhide project 
key fedora com build automated 2003)
2012-05-17 15:56:48 /pks/lookup: Get request (signing redhat red rawhide key 
inc hat com build automated 2003)
2012-05-17 15:56:48 /pks/lookup: Get request (0xb44269d04f2a6fd2)
2012-05-17 15:56:48 /pks/lookup: Get request (0x219180cddb42a60e)
2012-05-17 15:56:48 /pks/lookup: Get request (0xfd372689897da07a)
2012-05-17 15:56:49 /pks/lookup: Get request (0x37017186)
2012-05-17 15:56:49 /pks/lookup: Get request (support rhx redhat red key inc 
hat com)
2012-05-17 15:56:49 /pks/lookup: Get request (test software redhat red rawhide 
inc hat com beta)
2012-05-17 15:56:49 /pks/lookup: Get request (security redhat red inc hat com)
2012-05-17 15:56:49 /pks/lookup: Get request (0x219180cddb42a60e)
2012-05-17 15:56:49 /pks/lookup: Get request (security redhat red key inc hat 
com beta 2)
2012-05-17 15:56:49 /pks/lookup: Get request (security release redhat red key 
inc hat com 2)
2012-05-17 15:56:50 /pks/lookup: Get request (org fedoraproject fedora 11)