These results showed that there was considerable benefit with 10 senders and 10 queue directories. The benefit going to 20 queue directories with 10 senders was negligible.
At the time I wondered whether this was a general rule -- i.e., is anything more than 10 queue directories overkill? Or is there a correlation between the number of queue directories compared to the number of simultaneous sending systems.
Accordingly, I'm running some additional tests. Each set of tests will use a specific number of queue directories, and different numbers of concurrent senders.
| # queue | # Conns |
|---|---|
| 10 | 20 |
| " | 30 |
| " | 40 |
| 20 | 20 |
| " | 30 |
| " | 40 |
| 30 | 20 |
| " | 30 |
| " | 40 |
| 40 | 20 |
| " | 30 |
| " | 40 |
The message count is still 30,000, and the message size is 13,870 bytes.
Results
View raw data
Here are the results. Each chart compares sending 30,000 messages (so creating 60,000 queue entries) using 20, 30, and 40 concurrent connections. The time taken to create each file in the queue is plotted.
Times with 20 connections are shown in green, 30 connections in red, and 40 connections in yellow.
This chart shows the results with 10 queue directories.
Clearly, here, 20 concurrent connections provides the best performance (and please see the disclaimer before taking these results and applying them to your own systems, especially if those systems have queues spread over multiple disks).
Since there were only 10 queue directories in this test I am expecting that there is still going to be some contention/locking when creating new queue entries. So going to 20 queue directories will (I expect) show an overall decrease in the amount of time required, but the relative differences to 20, 30, and 40 concurrent connections should remain the same.
Here's the chart for the 20 queue directory case.
That does match my expectations, which is reassuring. Here we can see that the extra 10 directories have shaved 8-10ms off all three scenarios, but their relative differences are still similar.
I expect that the next results, with 30 queue directories, might show a significant decrease in the amount of time taken in the 30 connections results. With 20 connections there's going to be spare queue directories, so there shouldn't be that significant a difference.
With 30 connections there will be one queue directory per connection, and the previous chart showed how that helped. So I expect the red and green plots to be closer together this time. The 40 connection plots (yellow) will still be suffering from contention around creating directory entries, so should still be 5-8ms larger than the 30 connection points.
That's quite surprising. All three have continued to benefit from this change, with the 40 connection case seeing the largest drop, while the 20 and 30 connection results show a 2-3ms benefit as well. This would seem to indicate that having more queue directories than maximum concurrent connections would be a good thing, to squeeze maximum performance from your system.
Finally, the 40 queue directory tests. I'm not so sure what to expect now. I think the 20 connection case has probably seen all the benefit it's going to get -- I expect that with 20 concurrent connections 30 queue directories provides enough directories to create queue entries that adding an extra 10 will not be a significant improvement.
The results from the previous test show that 20 connections definitely benefits from 30 queue directories, so I expect that 30 connections will similarly benefit from having 40 queue directories. Since there are more connections the benefit might also be larger.
Finally, the 40 connections case should also show a benefit, perhaps a significant one, as the number of connections finally equals the number of queue directories.
Here's the chart.
Well, I was partially right.
As expected, 20 connections sees no significant benefit with the additional 10 directories. 30 connections does, but it's not as dramatic as I was expecting. And the figures for 40 connections barely move.
This would suggest that the effect of the number of queue directories is no longer significant once the number of concurrent connections climbs over a certain number.
ministat
Running the results through ministat gives some percentages and some statistical rigour to the results. These figures are at a 99.5% confidence interval.
| Queue directories | ||||
|---|---|---|---|---|
| # Conns. | 10 | 20 | 30 | 40 |
| 20 | 28.1ms | -26.7% +/- 5.4% | -31.0% +/- 5.6% | -32.9% +/- 5.8% |
| 30 | 45.3ms | -21.6% +/- 6.2% | -28.4% +/- 5.0% | -31.2% +/- 5.9% |
| 40 | 56.0ms | -16.8% +/- 6.4% | -22.4% +/- 6.6% | -24.1% +/- 6.0% |
For example, this shows that with 30 connections, and 10 queue directories, it took, on average, 45.3ms to create each file. Going to 20 queue directories reduced this time by 21.6%. Going to 30 queue directories reduced this time by 28.4%, and so on. This chart may make things slightly easier to visualise.
Conclusion
Having multiple queue directories provides a significant performance increase when queuing messages if your Sendmail installation will be processing multiple concurrent inbound sessions. It can dramatically decrease the amount of time taken to create new entries in the queue, by 30% or more.
A good rule of thumb would seem to be to ensure that the number of queue directories is at least equal to the number of concurrent sessions you expect to handle, with a few additional directories. You can explicitly limit the number of concurrent sessions that Sendmail will process with the
conncontrol feature.However, they may come a point where the queue entry creation time is no longer significant compared to the other processing that Sendmail has to do. This is apparent in these results where 40 concurrent connections see a benefit of only 2% when going from 30 to 40 queue directories.
Note that these results do not look at the impact of multiple queue directories on other parts of Sendmail. For example, normal and persistent queue runners may see positive or negative benefits from having multiple queue directories, and merit a separate investigation.
No comments:
Post a Comment