How would one go about writing a Rate Limiting class? I mean ... I need some 'work' to be done X number of times ... can be sequentially but in parallel would even be a lot better. I will need the Rate Limiting class for quite a few bits of work. The main factor is that I can only do the piece of work X number of times during a period of 60 seconds.
How would one go about writing a Rate Limiting class? I mean ... I need some 'work' to be done X number of times ... can be sequentially but in parallel would even be a lot better. I will need the Rate Limiting class for quite a few bits of work. The main factor is that I can only do the piece of work X number of times during a period of 60 seconds.
In once case the bits of work could take 15 seconds each to finish, so there I would like to execute the bits of work sequentially, but in other cases the bits of work just take a fraction of a second. In those cases the limiting factor would be something like 50 times a second ... not any more.
The actual bits of work are calls to a REST API and execute some work locally based on the JSON I received. Just to be clear.
I know I could just repeat a loop which does the requests as long as I don't have 60 requests within a minute. And that works find if I only have 5 requests to do. But in another spot I have to do almost 1000 requests. So I was hoping there is an easier solution, a solution I could re-use in many spots in my project. But I've just been unable to figure it out yet :-P
Oh and by the way ... the rate limit is imposed on me by the REST API. There is nothing I can do to change that.
In once case the bits of work could take 15 seconds each to finish, so there I would like to execute the bits of work sequentially, but in other cases the bits of work just take a fraction of a second. In those cases the limiting factor would be something like 50 times a second ... not any more.
The actual bits of work are calls to a REST API and execute some work locally based on the JSON I received. Just to be clear.
I know I could just repeat a loop which does the requests as long as I don't have 60 requests within a minute. And that works find if I only have 5 requests to do. But in another spot I have to do almost 1000 requests. So I was hoping there is an easier solution, a solution I could re-use in many spots in my project. But I've just been unable to figure it out yet :-P
Oh and by the way ... the rate limit is imposed on me by the REST API. There is nothing I can do to change that.
Wrap each item in an anonymous method. Pass it to a class whose job it is to do the work. Have that class keep track of the rate. If the rate is being exceeded, then have the worker wait. I'd use a queue with a separate thread pulling work off the queue.
ReplyDeleteDavid Heffernan and make sure to deal with that queue growing too much. Drop the oldest requests, drop every nth, or block pushing new items into the queue. Otherwise the rate problem just turns into a local problem :)
ReplyDeleteSomething I forgot to mention ... Would be nice if the Processes which are limited by the Rate Limiting thingie can be executed in the background (not blocking the main thread).
ReplyDeleteDavid Heffernan Hm ... interesting ... will have to investigate the whole thread pulling work of the queue thingie since that currently sounds like Chinese to me :-P
ReplyDeleteInteresting question. Keep us posted with what you come up with based on what David Heffernan and Wouter van Nifterick suggest.
ReplyDeleteJeroen Wiert Pluimers I will ... if my brain doesn't explode before I find a solution :-P
ReplyDeleteStefaan Lesage Just to not start from scratch: omnithreadlibrary.com - Home - The Ultimate Delphi Threading Library
ReplyDeleteT n T Yeah ... found that during my search as well, posted a message in that Google+ group too.
ReplyDeleteT n T A general method for solving the problem could be using a variable delay in the thread that takes into account the actual time it needed to do the work.
ReplyDeletePseudo-code below just to illustrate the idea. Do note that Sleep() and GetTickCount() are both rather crude. More accurate mechanisms exist.
VAR start, timespent:int64;
...
// This code section takes 5 seconds to run.
start:=GetTickCount;
DoSomeWork();
Timespent:=GetTickCount-Start;
if timespent < 5000 then
Sleep (5000 - TimeSpent);
Arthur Hoornweg Problem is ... I do not know in advance how long a unit of work will take. Heck ... I do not know yet all the units of work I will need to do. Currently I have a few units of work which might take close to a minute and others which take a couple of milliseconds.
ReplyDeletePersonally I've been thinking in the direction of some Queue to which I can add units of work dynamically. The queue will process the first unit of work if it it hasn't processed 60 units already or it has processed 60 units but a minute has been passed since starting the first of 60.
The queue would of course have to know when a unit of work is finished and stuff like that, but I think i will have to look into that way of handling it