I've been looking at different queue sort strategies to see what their overhead is. Since all the messages are going to be delivered to a single host these results aren't necessarily going to be indicative of what you would see on a production server. However, they should serve to illustrate any inherent speed advantages of one sort strategy over another.
Read on for the resuls.
Showing posts with label ministat. Show all posts
Showing posts with label ministat. Show all posts
Day 38 of 60: Multiple queues, one queue runner
Today I'm looking at the results that I've obtained from the latest round of tests. These tests used
sendmail -q to deliver 30,000 messages to a different zone. There were 10 runs to each test, and the different tests collected data on timings for 1, 5, 10, 20, 30, and 40 queue directories.Day 31 of 60: Queues and connections
Back on day 28 I looked at the effect of multiple queue directories with concurrent senders.
These results showed that there was considerable benefit with 10 senders and 10 queue directories. The benefit going to 20 queue directories with 10 senders was negligible.
At the time I wondered whether this was a general rule -- i.e., is anything more than 10 queue directories overkill? Or is there a correlation between the number of queue directories compared to the number of simultaneous sending systems.
These results showed that there was considerable benefit with 10 senders and 10 queue directories. The benefit going to 20 queue directories with 10 senders was negligible.
At the time I wondered whether this was a general rule -- i.e., is anything more than 10 queue directories overkill? Or is there a correlation between the number of queue directories compared to the number of simultaneous sending systems.
Day 30 of 60: What are the single queue directory bottlenecks? (pt 2)
Having established that there's a significant increase in the amount of taken by the
fdsync() and open() system calls when Sendmail creates queue entries with a single queue directory I've set about tracking down what that bottleneck is.Day 29 of 60: What are the single queue directory bottlenecks?
Earlier posts have shown that using a single queue directory imposes a significant bottleneck when processing concurrent connections with Sendmail. Yesterday I posed some questions, and today I've started work on answering the first one.
The first question was:
The first question was:
What is responsible for the dramatic slow down in the single-queue case (test 4)?
Day 28 of 60: Instrumenting Sendmail queue file creation (pt 4)
Yesterday I looked at the effect of multiple queue directories when processing messages over a single connection.
Today I've been looking at how multiple queue directories can help when processing concurrent connections.
The methodology was identical to the previous tests. The only change was to the smtp-source(1) command line. The previous tests were run with
Today I've been looking at how multiple queue directories can help when processing concurrent connections.
The methodology was identical to the previous tests. The only change was to the smtp-source(1) command line. The previous tests were run with
-s 1, indicating one concurrent connection. These tests were run with -s 10, to force 10 concurrent connections.Day 27 of 60: Instrumenting Sendmail queue file creation (pt 3)
I've commited the first sets of results to the repository in the aptly named results/ directory.
To refresh your memory, the question I intended to answer was:
They're quite surprising.
To refresh your memory, the question I intended to answer was:
does the number of queue directories (on a single disk) make a significant impact on the time taken to create new entries in the queue?
They're quite surprising.
Day 19 of 60: M4 issues resolved, ministat updates
The issues with M4 have been resolved. A colleague, Andre Lucas, took up the challenge and worked out a fix which he describes in detail. And ministat's now looking much better. It's grown some useful new options, a lot of documentation, and can now (optionally) generate plots in colour. Look below the fold for two example plots.
Day 17 of 60: ministat
I've spent some of today porting some useful statistics reporting software from C to Perl.
ministat reads in two or more files of data and uses the Student's t test to determine if there is any statistical difference between the means of the datasets. This is especially useful when comparing benchmark results.
For instance, I figure that this will be useful to compare data from several Sendmail runs, where the number of queue directories differ between the runs. It should highlight any benefits between different numbers of queue directories.
Rather than explain more here, I'll point you at the code. There's documentation towards the end of the file. Note that this still needs some work -- there's no proper command line option handling at the moment, some of the documentation needs fleshing out, and I wouldn't use this as an example of good Perl code, as it still looks far too much like a Perl program that's been written in C.
When I've fixed that I'll put it up on CPAN.
ministat reads in two or more files of data and uses the Student's t test to determine if there is any statistical difference between the means of the datasets. This is especially useful when comparing benchmark results.
For instance, I figure that this will be useful to compare data from several Sendmail runs, where the number of queue directories differ between the runs. It should highlight any benefits between different numbers of queue directories.
Rather than explain more here, I'll point you at the code. There's documentation towards the end of the file. Note that this still needs some work -- there's no proper command line option handling at the moment, some of the documentation needs fleshing out, and I wouldn't use this as an example of good Perl code, as it still looks far too much like a Perl program that's been written in C.
When I've fixed that I'll put it up on CPAN.
Subscribe to:
Posts (Atom)