Day 38 of 60: Multiple queues, multiple queue runners (pt 1)

I've started to get data about the effect of multiple queues with multiple queue runners.

As before I'm using 1, 5, 10, 20, 30, and 40 queue directories, and I'm instrumenting with queue-run-duration.d. This time I'm starting queue runners with the command sendmail -q30s. This will cause Sendmail to create a new queue runner to process the queue every 30 seconds.

The problem is that, even with 30,000 messages, Sendmail can process the whole lot in about 40 seconds, which doesn't give enough time for more than two queue runners to start.

So I'm using the -w option to smtp-sink(1) to insert a 1 second delay at the DATA stage. So (roughly) the first 30 messages go through at the rate of one per second. Then a second queue runner starts, and messages go at the rate of 2 a second, and so on.

But it's still slow going.

As I write this it occurs to me that I could use DTrace to induce this slowdown, by using chill() to have the process pause for a tenth of a second at the start of every job run. That's something I may look at later. As Thursday is my normal day in London, look for updates on Friday.