Running the Tool
In this mode, you simply run the tool passing in the API_KEY for querying Abusix Mail Intelligence (you can find this in the User Portal) along with the DNS list that you wish to compare against and the file containing the IP addresses to check. Each lookup will cause the tool to output a CSV (comma-separated values) format log-line for later analysis, so you need to redirect stdout to a CSV file.
$ ./ami_compare_linux --apikey API_KEY --list bb.barracudacentral.com ips_to_test > results.csv Processed 27907 items Processed 28104 items Processed 28261 items Processed 28367 items Processed 28505 items Blocked % Unique % Blocked WL % Abusix Mail Intelligence 21125 74.1 20234 95.8 0 0.0 bb.barracudacentral.org 1139 4.0 248 21.8 0 0.0
As you can see above, when the tool is running, it will periodically output how many lookups have been done so far. Once it is complete, it will output a summary and exit. The summary table which contains the following fields:
The number of IPs that were found to be listed in the blocklist along with the percentage.
The number of IPs where the IP was only found to be listed on one list and not the other along with the percentage.
For the comparison list, this shows the number of listed IPs that were found to be listed in the Abusix Mail Intelligence Whitelist along with the percentage.
The "results.csv" file can be loaded into most Spreadsheet applications and using "Auto Filter", you can analyze all of the results to look for false positives and to investigate the results of the lookups.
One of the drawbacks to using this comparison tool with log files is that whilst it provides a great way to quickly and very simply do a comparison between Abusix Mail Intelligence and another blocklist, it doesn’t provide an exact replica of what would happen if you were to add Abusix Mail Intelligence into your SMTP server, so we added the real-time/streaming mode to provide this.
To be able to use this mode, you need to be able to extract the IP addresses hitting your MTAs in real-time.
For a single node or system that uses a centralized log server, this could be as simple as using UNIX “tail” and “grep” commands. The only requirement is that whatever you use can extract one IP per line.
In real-time/streaming mode you use UNIX pipes to do the necessary extraction and then place the comparison tool at the end of the pipe and provide ‘-’ as the filename to tell the tool that it should read the input from stdin instead of a file.
For example, this is how to extract the necessary data from a Postfix log:
$ tail -f /var/log/mail.log | grep --line-buffered -P '\bconnect from\b' | grep --line-buffered -Po '\d+\.\d+\.\d+\.\d+' | ./ami_compare_linux --apikey API_KEY --list bb.barracudacentral.org - > results.csv Processed 696 items, errors 0, cache hits 0, queue length 115 Count % Unique % Blocked WL % Abusix Mail Intelligence 663 95.3 240 36.2 0 0.0 bb.barracudacentral.org 423 60.8 0 0.0 0 0.0 Not Listed 33 - - - - -