👋 How can we help you?

Stream specifications

Abusix Intelligence stream specifications

Message stream

Overview

Anti-spam vendors must constantly tune their spam heuristics engines to catch the latest shape-shifting threats.

Abusix’s Spam Threat Intelligence service is a real-time corpus of spam messages. This feed may tune your anti-spam filters and monitor your network or services for bad actors and compromised systems.

For security providers, this is the best solution in the marketplace today, as it provides you with the same data set used by other major security providers.

This is the best solution in the marketplace today for network and service operators, as you can see the start, peak, and end of spam runs that will get your IP addresses blacklisted.

The black portion of this feed is 100% pure spam, false positive free, allowing you to use the data confidently in your automated workflows.

Description

Abusix’s Spam Threat Intelligence Message Stream is a real-time corpus of spam messages designed so that you may use the data with complete confidence in your automated workflows.

Our most complete and standard format is JSON, transported via stream, with identifying attributes such as the message's language, attached file types, and more. The entire message and attachments are also attached. We can provide only files, metadata elements in a stream, and hourly reports.

We offer two message streams of data.

  • Black stream provides 100% false positive free data
  • Black and Grey stream provides a rich mix of spam for hunting.

Ultimately, the depth and versatility of Abusix Intelligence make our data a critical component of any cyber-defense.

Benefits

Using our proprietary sensor network, we provide an unparalleled view of threats through our constant corpus of threat-rich data, which allows you to:

  • identify spam in realtime, within your inbound or outbound spam filters, by using our pure black stream
  • hunt for malicious inbound malware, fraud, and phishing using our grey stream

Format

This feed is available as a meta-data feed, enriched with the transaction, authentication, header, message body, cname, attachment, and associated metadata upon demand.

We distribute the message feed in a JSON structure.

JSON Payload

{
  "smtp_mail_from" : "Anya277@unizentechnologies.com",
  "data_colorcode" : "black",
  "email_attachment_count" : "0",
  "source_ip" : "171.240.245.173",
  "detected_text_language" : null,
  "email_subject" : "hi",
  "email_attachment_count" : 0,
  "email_attachment_content_types" : [ ],
  "email_attachment_file_names" : [ ],
  "email_attachment_hashes_md5" : [ ],
  "email_attachment_tags" : "",
  "data_origin" : "com.abusix.spam.trap",
  "email_urls" : [ ],
  "smtp_timestamp" : "Thu, 18 Jan 2018 13:09:07 +0000",
  "email_headers_raw" : {
    "date" : [ "Thu, 18 Jan 2018 20:09:03 +0700" ],
    "mime-version" : [ "1.0" ],
    "content-transfer-encoding" : [ "8bit" ],
    "x-mailer" : [ "PHPMailer 5.2.22 (https://github.com/PHPMailer/PHPMailer)" ],
    "subject" : [ "hi" ],
    "x-php-originating-script" : [ "853:class-phpmailer.php" ],
    "message-id" : [ "<f7678bee21a5ecec1041bf33f0507707@unizentechnologies.com>" ],
    "received" : [ "from [171.240.245.173] ([171.240.245.173])\r\n\tby example.me (Haraka/2.8.16) with ESMTP id 401F2F97-EE39-4236-9361-760271ACEDD1.1\r\n\tenvelope-from <Anya277@unizentechnologies.com>;\r\n\tThu, 18 Jan 2018 13:09:07 +0000", "by mail.unizentechnologies.com (Postfix, from userid 853) id DB472E03603; Thu, 18 Jan 2018 20:09:02 +0700" ],
    "content-type" : [ "text/html; charset=UTF-8" ],
    "from" : [ "Anya <Anya277@unizentechnologies.com>" ],
    "to" : [ "dumikem@abusix.invalid" ]
  },
  "source_port" : "57505",
  "smtp_rcpt_to" : [ "dumikem@abusix.invalid" ],
  "original_message_base64_encoded" : "UmVjZWl2ZWQ6IGZyb20gWzE3MS4yNDAuMjQ1LjE3M10gKFsxNzEuMjQwLjI0NS4xNzNdKQ0KCWJ5IGV4YW1wbGUubWUgKEhhcmFrYS8yLjguMTYpIHdpdGggRVNNVFAgaWQgNDAxRjJGOTctRUUzOS00MjM2LTkzNjEtNzYwMjcxQUNFREQxLjENCgllbnZlbG9wZS1mcm9tIDxBbnlhMjc3QHVuaXplbnRlY2hub2xvZ2llcy5jb20+Ow0KCVRodSwgMTggSmFuIDIwMTggMTM6MDk6MDcgKzAwMDANClJlY2VpdmVkOiBieSBtYWlsLnVuaXplbnRlY2hub2xvZ2llcy5jb20gKFBvc3RmaXgsIGZyb20gdXNlcmlkIDg1MykgaWQgREI0NzJFMDM2MDM7IFRodSwgMTggSmFuIDIwMTggMjA6MDk6MDIgKzA3MDANClRvOiBkdW1pa2VtQGFidXNpeC5pbnZhbGlkDQpTdWJqZWN0OiBoaQ0KWC1QSFAtT3JpZ2luYXRpbmctU2NyaXB0OiA4NTM6Y2xhc3MtcGhwbWFpbGVyLnBocA0KRGF0ZTogVGh1LCAxOCBKYW4gMjAxOCAyMDowOTowMyArMDcwMA0KRnJvbTogQW55YSA8QW55YTI3N0B1bml6ZW50ZWNobm9sb2dpZXMuY29tPg0KTWVzc2FnZS1JRDogPGY3Njc4YmVlMjFhNWVjZWMxMDQxYmYzM2YwNTA3NzA3QHVuaXplbnRlY2hub2xvZ2llcy5jb20+DQpYLU1haWxlcjogUEhQTWFpbGVyIDUuMi4yMiAoaHR0cHM6Ly9naXRodWIuY29tL1BIUE1haWxlci9QSFBNYWlsZXIpDQpNSU1FLVZlcnNpb246IDEuMA0KQ29udGVudC1UeXBlOiB0ZXh0L2h0bWw7IGNoYXJzZXQ9VVRGLTgNCkNvbnRlbnQtVHJhbnNmZXItRW5jb2Rpbmc6IDhiaXQNCg0KWW91IHNlZW0gbGlrZSBteSB0eXBlIGFuZCBJIHdvdWxkIGxpa2UgdG8ga25vdyB5b3UgbW9yZSENCldyaXRlIG1lIGlmIHlvdSBhcmUgaW50ZXJlc3RlZCwgaGVyZSBpcyBteSBlbWFpbCBkZW5pc2F1cnN1bGEya2VpQHJhbWJsZXIucnUgYW5kLCBpZiB5b3Ugd2FudCwgSSB3aWxsIHNlbmQgc29tZSBvZiBteSBwaG90b3MuDQoNCkh1Z3MsDQpBbnlhDQoNCg=="
}

Volume (as of June 16, 2020)

Abusix processes an ever-increasing 800 million trap hits daily through its infrastructure. The statistics below are for deduped data as of June 16, 2020.

JSON Black Message Stream

All BLACK Messages whole with files- deduped primarily on URL, files (but also includes black text-only messages deduped)

min: 2.01M / day

max: 10.52M / day

avg: 3.83M / day

JSON Black+Grey Message Stream

ALL BLACK+GREY Messages whole with files - deduped similarly (also includes black text-only messages deduped)

min: 3.02M / day

max: 13.50M / day

avg: 7.07M / day

Requirements

To receive reports, you must be able to cURL or use STOMP.

For STOMP script examples, see Getting Started

 

File attachments stream

Overview

The File Stream is a real-time corpus of files derived from 100% spam; the target-rich environment may address real-time short-tail antispam zero-day filtering and long-tail antivirus botnet, command, and control, as well as malware code research.

You decide whether spam messages for heuristics, zero-day edge filtering using our MD5 hashed files, detonating raw files in sandboxes to hunt botnets, command and control servers, or malware code analysis is more critical to your security focus.

This feed is a must-have to complete the suite of feeds you use to filter, hunt, learn and adapt in real-time.

Description

Anti-virus vendors need to gain access to the latest malicious email-borne payloads to the sandbox, detonate and find command and control servers, and analyze malicious code. If you hunt, this feed is a must-have to complete the suite feeds you use to hunt.

Benefits

  • Command and Control server hunters can detonate as many files as possible in sandboxes to track down botnet command and control servers and their proxies.
  • Antivirus researchers find new malicious code in malware, ms-script, and pdf script variants.

Format

File feeds may be sent in RAW or JSON format.

JSON Payload

{
  "smtp_mail_from": "reception@paradisepark.co.uk",
  "content_type": "application/pdf",
  "source_ip": "212.42.162.3",
  "data_origin": "com.abusix.spam.blackhole",
  "smtp_timestamp": "Tue, 14 May 2019 14:02:02 +0000",
  "source_ip_rir": "ripe",
  "source_port": "60299",
  "smtp_rcpt_to": [
    "sales@creativeproducts.co.uk"
  ],
  "source_ip_country_iso": "GB",
  "attachment_base64_encoded": "JVBERi0x[...]"
}

Volume (as of June 16, 2020)

Abusix processes an ever-increasing 800 million trap hits daily through its infrastructure. The statistics below are for deduped data as of June 16, 2020.

Raw Spam Files

Includes ALL URLs in the BLACK+GREY message stream and more (deduped over 60 mins)

min: 89.7K / day

max: 357.7K / day

avg: 268.1K / day

 

Includes:

images avg: 110k / day

text avg: 81k / day

pdfs avg: 70k / day

archive avg: 9k / day

word avg: 6k / day

executable avg: 5k / day

excel avg: 4k / day

web avg: 3k / day

message avg: 2k / day

audio avg: 400 / day

video avg: 250 / day

PowerPoint avg: 250 / day

Requirements

To receive reports, you must be able to cURL or use STOMP.

For STOMP script examples, see Getting Started

 

URL stream

Overview

The URL Stream is designed for AntiVirus and Brand Protection vendors of all types to allow them to constantly hunt for and identify new websites and web pages that are phishing and spoofing brands using trademarks, copyrights, and other intellectual property, hosting drive-by download or malware threats, phish kits, and crime-ware.

Description

The URL streaming service is provided as a script that connects to our stream of the (non-curated) report of URLs, thus allowing you to quickly see new malicious actors hosting phish, spoofing, stealing credentials, defrauding, infecting, and spying on users.

Key Benefits

A service is an ideal place to hunt and identify websites hosting

  • brand phishing
  • generic phishing for user credentials
  • copyrighted images
  • copyrighted intellectual property
  • spoofing
  • drive-by downloads ready to intercept consumer keystrokes for account takeover (ATO)
  • crimeware

Format

  • JSON with URLs and Metadata
  • Raw URLs

The various metadata tags allow you to filter quickly on things such as country, language, etc., to improve the noise-to-signal ratio for your use case, showing you what you care about most.

JSON Payload

{
  "detected_text_language" : "ja",
  "data_origin" : "com.abusix.spam.httprelay",
  "smtp_timestamp" : "Wed, 22 Jul 2020 23:41:22 +0000",
  "source_ip_country_iso" : "TW",
  "url_tld" : "com",
  "url" : "http://csqzvg.re[...]"
}

Volume

Abusix processes an ever-increasing 800 million trap hits daily through its infrastructure. However, counts can vary widely due to the diversity of spam campaigns and the number of URLs used in individual spam campaigns. This feed includes ALL URLs in the BLACK+GREY message stream.

URLs Stream (deduped over 5 mins, as of June 16, 2020)

min: 12.3M / day

max: 168.9M / day

avg: 59M / day

 

Requirements

To receive reports, you must be able to cURL or use STOMP.

For STOMP script examples, see Getting Started

Learn more about Abusix Intelligence

Did this answer your question?
😞
😐
🤩