Submitting MTA Transaction Feeds via UDP to Data Channels

Introduction

This article explains how to feed your MTA’s inbound SMTP transaction data in real-time to Abusix, which helps improve the accuracy of our Guardian Mail product. This protocol only applies to the third wizard set-up option, “Send transaction feeds” in app.abusix.com.

SMTP Transaction Data

The data we request is simple to retrieve from most MTAs and, by design, does not include any data that would cause any privacy issues. Data for each SMTP_ transaction,_ created on every SMTP MAIL command, is what is requested. Ideally, this data should be sent immediately after receiving the MAIL command, provided it is only sent once per transaction. The protocol is designed as fire-and-forget UDP to minimize overhead on your mail system.

Format

Each data field must be separated by a line-feed character (LF \n) without a trailing line-feed, and all data must be sent in a single UDP packet. While all fields are required, they may also be empty if the data is unavailable. The feed receiver will discard data where (a) there are not enough line feeds, (b) the Feed ID is not recognized, or (c) the checksum does not match the checksum we compute on reception. Since this is fire-and-forget UDP, there needs to be feedback on whether the data being sent is valid and in the correct order or any acknowledgment that the data was received.

Data	Description
Feed ID	This will be provided to you by Abusix and is used to identify which feed the data is coming from and is used as a lookup key to find the shared key used to compute the checksum.
Timestamp	The UNIX epoch time in seconds that the connection was received.
Port	The TCP server port that the connection was received. e.g., 25, 587, 465. This is used to distinguish between MUA->MTA traffic and MTA->MTA traffic.
IP Address	IPv4 or IPv6 address of the SMTP client.
rDNS	Reverse DNS name of the SMTP client; multiple names should be separated by commas.
HELO	HELO/EHLO sent by the SMTP client.
ESMTP Flag	‘Y’ if the SMTP client sent EHLO, ‘N’ if the client sent ‘HELO’.
TLS Flag	‘Y’ if the SMTP client used TLS, ‘N’ if the client did not use TLS.
AUTH Flag	‘Y’ if the SMTP client is authenticated using SMTP AUTH, ‘N’ if not.
MAIL FROM domain	The right-hand side of the ‘@’ of the SMTP MAIL FROM command. In the case of a null-sender or an unqualified user, e.g., MAIL FROM:<postmaster> or MAIL FROM:<>, the full argument can be sent.
Extended JSON	For future use – this field should be empty.
Checksum	This is an MD5 hash computed by taking all of the previous data, including the line-feed separators, including a trailing line-feed, along with the shared key supplied to you by Abusix added to this. e.g. checksum = data + “\n” + shared key Once this is computed, the following is then sent over the UDP socket to the feed receiver: data + “\n” + checksum

Example Here is an example of the data showing each field and the format. Line feeds are shown as ”\n” for illustration purposes:

txnNNN\n

1540299376001\n

25\n

127.0.0.1\n

localhost\n

this.is.a.test.helo\n

N\n

N\n

N\n

test.com\n

\n

8c86e0ab24415f726e4def79ce9502c5

Transport

After completing all the Data Channel configuration steps in app.abusix.com, you can send us the data. First, verify connectivity by running the following command:

$ nc -v -u -z -w 3 smtp-rttf.abusix.com 12211
Connection to smtp-rttf.abusix.com 12211 port [udp/*] succeeded!

Example Code

We provide example code for Exim and a fully functioning Postfix Policy daemon, written in Node.js, which can be used with any version of Postfix. These examples may also be used to write an integration with any MTA in any programming language.

Exim

Each distro package in Exim is different, so you will likely need to customize this appropriately for your installation. This is what I did on Debian/Ubuntu Linux. In /etc/exim4/conf.d/acl/00_exim4-config-header – I added the following lines to the top (as macros); these must appear before the “begin acl” directive: ABUSIX_HOST=smtp-rttf.abusix.com ABUSIX_PORT=12211 ABUSIX_FEED_ID=<Your Feed ID> ABUSIX_FEED_KEY=<Your Feed Key> Then in 30_exim4_config_check_mail we want the “udpsend” to run for every message, accepted or not:

### acl/30_exim4-config_check_mail
##############################
###
# This access control list is used for every MAIL command in an incoming
# SMTP message. The tests are run in order until the address is either
# accepted or denied.
acl_check_mail:
accept
    set acl_m_txnfeedraw =
ABUSIX_FEED_ID\n${tod_epoch}\n${received_port}\n${sender_host_address}\n${sender_host_name}\n${sender_helo_name}\n${if eq{${substr{0}{1}{${received_protocol}}}}{e} {Y}{N}}\n${if eq {$tls_in_bits}{0} {N}{Y}}\n${if eq {$authenticated_id}{}{N}{Y}}\n${sender_address_domain}\n\n
    set acl_m_txnfeedkey = ${acl_m_txnfeedraw}ABUSIX_FEED_KEY
    set acl_m_txnfeed = ${acl_m_txnfeedraw}${md5:${acl_m_txnfeedkey}}
    udpsend = ABUSIX_HOST:ABUSIX_PORT:${acl_m_txnfeed}

Postfix

The following linked Postfix Policy daemon is written in Node.js, is fully functional, and can be used with any version of Postfix.

Sending Abusix your “Spamtrap” or “This Is Spam” messages

If you wish to send Abusix, your “Spamtrap” or “This Is Spam” messages, see Submitting data via email to Data Channels and Submitting data via email to Data Channels

Portal

About Data Channels

How to Submit Data

Submitting MTA Transaction Feeds via UDP to Data Channels

Introduction

SMTP Transaction Data

Format

Transport

Example Code

Exim

Postfix

Sending Abusix your “Spamtrap” or “This Is Spam” messages

​Introduction

​SMTP Transaction Data

​Format

​Transport

​Example Code

​Exim

​Postfix

​Sending Abusix your “Spamtrap” or “This Is Spam” messages

Introduction

SMTP Transaction Data

Format

Transport

Example Code

Exim

Postfix

Sending Abusix your “Spamtrap” or “This Is Spam” messages