[Remops] Attn bigapple admin: Rate-limiting woes

[Admin]

Hello Remops,

Saturday, February 4, 2006, 7:13:33 AM, you wrote:

p> Dear bigapple admin:

p> Starting a few days ago, you appear to have introduced strict rate- 
p> limiting on your system.  This includes two things:

p>   - no more than two connections allowed from any host at any time  
p> (transient failure)

p>   - no more than 30 messages allowed from any host in a given period  
p> (permanent failure)

p> The combination is unfortunate.  Here's what happens:

p> Parallel delivery of freshly submitted messages results in a number  
p> of transient failures, because the "two connections at a time" limit  
p> kicks in.  Those are recorded and collected in the queue.  They are  
p> retried every so often, and may fail again due to the same limit.   
p> Eventually, they are moved into a separate queue of aged messages so  
p> as not to hinder delivery of fresher messages.  No problem so far.

p> Every so often, the system will still try to run the queue of aged  
p> messages.  There's a good chance more than 30 messages to bigapple  
p> have accumulated by then.  They are delivered all in a row, not in  
p> parallel (we're down to a single queue runner by then), but within a  
p> very short time.  This time, your second rate-limit kicks in, and  
p> this time, your system indicates it's a permanent failure.  The  
p> message is bounced to mixmaster, then discarded.

p> As I see it, you have four options:

p>   - leave things as they are and live with intermittent failures

p>   - persuade others to use linear instead of parallel delivery

p>   - turn off either of the two rate-limits

p>   - change the error message for the second rate limit to a transient  
p> failure code

p> I think the last option would be easiest and best. :-)

p> Regards,
p> mas

I tried a test yesterday. I created a number of PGP encrypted messages to
myself with the subject test 01,02,03,...50

I used a Mixmaster chain of twisty, bigapple, twisty to a local mail test
account.  

I released them all to the mailer simultaneously.

Some went through and were returned almost immediately, others trickled in
over the course of the day. When I checked early this morning, all had
arrived. The order in which they arrived was not the order I had sent them
in, which I was pleased to see. They were very randomly mixed.

I suspect you are correct, there may be some rate limiting. But in the
initial blast of messages to bigapple, (I have 20 simultaneous outgoing
connections set in Mercury) the initial ones went through too fast for me to
see in the window. I couldn't see anything with regard to simultaneous
connections and if there were more than 2 at a time made. It was just a
quick blur and gone, followed by normal traffic.

I didn't lose any messages. But, in retrospect, my experiment may have
delayed some other people's traffic, so I won't try that again :(  It
sounded like a good experiment at the time :(  Very sheepish look on my
face.

I can't say if Mercury just kept trying to re-send if it couldn't make a
connection or exactly what, I wasn't paying close enough attention nor do I
ever enable logging. If it didn't succeed the first time around, it should
have waited 120 minutes before retrying.

I am not sure if bigapple admin reads this list. When I have a question for
him, I E-mail him direct and he has graciously helped me out on numerous
occasions. If he doesn't answer here, I suggest you E-mail him with your
questions.

Of course I am operating a different setup than you. Basically only Mercury
and my CPU Hog Reliable :)

Just thought you might have found my test results of interest. A 100%
success rate but with the caveat of a lot of latency in the returns.

-- 
Best regards,
 Admin                            mailto:admin at twistycreek.com
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 555 bytes
Desc: not available
Url : http://lists.mixmin.net/pipermail/remops/attachments/20060205/f6f25c4f/attachment.pgp