👋 How can we help you?

Setting up your Data Channel using Transaction Feeds

How to use transaction Feeds

Last updated on Invalid Date

Introduction

This article explains how to feed your SMTP transaction data in real-time to Abusix, which helps improve the accuracy of our Abusix Mail Intelligence product.

The protocol is designed as fire-and-forget using UDP for delivery in order to minimise the amount of overhead on the mail system. The data we request is simple to retrieve from most MTAs (Mail Transfer Agents) and does not include any data that would cause any privacy issues.

We supply example code for Exim (see below), and a fully functioning Postfix Policy daemon, written in Node.js which can be used with any version of Postfix, or can be used as example code to write your own integration with any MTA in any programming language.

Please be aware, that this protocol is only applicable to the third wizard set-up option “Send transaction feeds”.

 

About the Protocol

The protocol requires that each data field is separated by a line-feed character (LF \n), without a trailing line-feed, and that all of the data should be sent in a single UDP packet.

This data should be sent once per SMTP transaction which is created on every SMTP MAIL command, so ideally this data should be sent after the MAIL command is received, however sending it afterwards is acceptable, provided it is only sent once per transaction.

All of the fields are required but can be empty if the data is unavailable. The feed receiver will only discard the data if there are not enough line-feeds, the Feed ID is not recognized or the checksum does not match the checksum that we compute on reception.

As this is fire-and-forget, there is no feedback provided as to whether the data being sent is valid and in the correct order or any acknowledgment that the data was received. However, an Abusix representative will work with you to get the feed up and running and will contact you in the event of any subsequent problems.

 
Data
Description
Feed ID
This will be provided to you by Abusix and is used to identify which feed the data is coming from and is used as a lookup key to find the shared key which is used to compute the checksum.
Timestamp
The UNIX epoch time in seconds that the connection was received.
Port
The TCP server port that the connection was received. e.g. 25, 587, 465. This is used so we can distinguish between MUA->MTA traffic or MTA->MTA traffic.
IP Address
IPv4 or IPv6 address of the SMTP client.
rDNS
Reverse DNS name of the SMTP client, multiple names should be separated by commas.
HELO
HELO/EHLO sent by the SMTP client.
ESMTP Flag
‘Y’ if the SMTP client sent EHLO, ‘N’ if the client sent ‘HELO’.
TLS Flag
‘Y’ if the SMTP client used TLS, ‘N’ if the client did not use TLS.
AUTH Flag
‘Y’ if the SMTP client authenticated using SMTP AUTH, ‘N’ if not.
MAIL FROM domain
The right-hand side of the ‘@’ of the SMTP MAIL FROM command. In the case of a null-sender, or an unqualified user e.g. MAIL FROM:<postmaster> or MAIL FROM:<>, the full argument can be sent.
Extended JSON
For future use - this field should be empty.
Checksum
This an MD5 hash computed by taking all of the previous data, including the line-feed separators including a trailing line-feed along with the shared key supplied to you by Abusix added to this. e.g.  checksum = data + “\n” + shared key Once this is computed the following is then sent over the UDP socket to the feed receiver: data + “\n” + checksum

Example

Here is an example of the data received showing the fields and format - the line-feeds are shown as \n for illustration purposes:

txnNNN\n

1540299376001\n

25\n

127.0.0.1\n

localhost\n

this.is.a.test.helo\n

N\n

N\n

N\n

test.com\n

\n

8c86e0ab24415f726e4def79ce9502c5

Feeding the Data to Abusix

After successfully completing all the steps in the app, you can start sending the data in and we will take care of the rest.

You can verify connectivity by running the following command:

$ nc -v -u -z -w 3 smtp-rttf.abusix.com 12211
Connection to smtp-rttf.abusix.com 12211 port [udp/*] succeeded!

Exim Configuration

Configuration for Exim can be very complicated and each distro packages Exim in different ways, so you will need to customize this appropriately for your installation. This what I did on Debian/Ubuntu Linux.

In /etc/exim4/conf.d/acl/00_exim4-config-header - I added the following lines to the top (as macros), these must appear before the "begin acl" directive:

 

ABUSIX_HOST=smtp-rttf.abusix.com

ABUSIX_PORT=12211

ABUSIX_FEED_ID=<Your Feed ID>

ABUSIX_FEED_KEY=<Your Feed Key>

 

Then in 30_exim4_config_check_mail I added this, basically we want the “udpsend” to run for every message, accepted or not:

 

### acl/30_exim4-config_check_mail

##############################

###

# This access control list is used for every MAIL command in an incoming

# SMTP message. The tests are run in order until the address is either

# accepted or denied.

acl_check_mail:

accept

set acl_m_txnfeedraw = ABUSIX_FEED_ID\n${tod_epoch}\n${received_port}\n${sender_host_address}\n${sender_host_name}\n${sender_helo_name}\n${if eq {${substr{0}{1}{${received_protocol}}}}{e} {Y}{N}}\n${if eq {$tls_in_bits}{0} {N}{Y}}\n${if eq {$authenticated_id}{}{N}{Y}}\n${sender_address_domain}\n\n

set acl_m_txnfeedkey = ${acl_m_txnfeedraw}ABUSIX_FEED_KEY

set acl_m_txnfeed = ${acl_m_txnfeedraw}${md5:${acl_m_txnfeedkey}}

udpsend = ABUSIX_HOST:ABUSIX_PORT:${acl_m_txnfeed}

Did this answer your question?
😞
😐
🤩