RuleStatistics

• The RuleStatistics plugin must be enabled before any statistics are collected. Enable it under Warden -> Settings -> Plugin Settings -> RuleStatistics or with the command warden --task=antispam:plugins --RuleStatistics=1 --reload=yes.
• Rule names containing the patterns WHITELIST, BLACKLIST or GTUBE are not logged as they would skew the contribution scores for the statistics.

This is a custom Warden plugin that logs real-time per-rule statistics for each email, allowing admins to easily analyze rule performance and identify false positives/negatives. The statistics are stored in a dedicated database table and can be viewed under Warden -> Rules -> Rule Statistics and in the Rule Statistics reports.

Each time a rule fires on a scanned message the plugin records the contribution of that rule to the final score. Over time this builds up a picture of how each rule behaves on live traffic:

  • How often the rule fires on messages that end up classified as spam, spammy or ham.
  • The minimum, maximum and average scores the rule contributes.
  • The ham/spam hit ratios, which make it easy to spot rules that are classifying to many messages (false positives) or too few (false negatives).

This makes it easy to decide which rule scores should be tuned when adjusting the filter.

How it works

Whether messages are counted as spam or spammy is determined by the spam level and spam kill level. By default these are synchronized with the server wide policy. Setting rule_statistics_sync_spam_levels to 0 allows custom spam and spam kill levels to be used instead, which is useful for testing the effect of different spam levels without changing them under the server wide policy.

Logging can be limited to specific rules with rule_statistics_log_rule or certain rules can be excluded with rule_statistics_ignore_rule. Statistics for sub-rules (meta rules starting with double underscores) can also be logged, but this adds a lot more database logging so it is normally not recommended on busy servers.

Viewing the statistics

The Rule Statistics view under Warden -> Rules lists every rule that has been seen along with its current score, hit counts and hit ratios. See Rules for a description of each column.

Related Pages