UPDATE: Please use Data Channels for XARF report submission. The current process listed in this document will soon be disabled.

Introduction

For quite a while now, we have been working on an extended XARF standard using simple JSON syntax. The current state of the schema can be found on GitHub: abusix/xarf. This is also where changes can be proposed and discussed.

Quite a few types can already be used; the list of supported report types looks like this:

• Botnet

• Child abuse

• Copyright

• DDOS

• LoginAttack

• Malware

• Phishing

• Portscan

• RPZ

• Spam

• Trademark

• OpenService

This list, of course, is still small, but it covers the most common types responsible for a big chunk of all reports. If there are report types, you would like to see supported, just message us with the details, and we will see how we can integrate them into XARF and AbuseHQ.

Example reports

The syntax of a report is relatively simple; this, for example, is a copyright report:

{
  "Version": "1",
  "ReporterInfo": {
    "ReporterOrg": "ExampleOrg",
    "ReporterOrgDomain": "example.com",
    "ReporterOrgEmail": "reports@example.com",
    "ReporterContactEmail": "contact@example.com",
    "ReporterContactName": "Mr. Example",
    "ReporterContactPhone": "+ 01 000 1234567"
  },
  "Disclosure": true,
  "Report": {
    "ReportType": "Copyright",
    "ReportClass": "Content",
    "FirstSeen": "2020-07-05T14:17:10Z",
    "Date": "2020-08-03T11:17:10Z",
    "SourceIp": "192.0.2.55",
    "SourcePort": 54321,
    "InfringedMaterial": "Example - Mr. Example",
    "SourceUrl": "http://www.badexample.com/badexapmplesong.mp3",
    "MaterialType": "Music",
    "FileName": "badexamplesong.mp3",
    "CourtOrder": "https://court.example.com/courtorder.pdf",
    "FileSize": 56,
    "FileHash": "somehashhere"
  } 
}

All information about the source of the copyright infringement is easily accessible, as is the reporter's contact information. Of course, not all of these fields have to be present. Look into the schema to determine which fields are required and which are not. More samples can be found in the xarf repository.

Integration into AbuseHQ

The integration into AbuseHQ is straightforward. Just follow the steps below.

1. Getting an API token

To send in reports, you first need an API token. First, head to AbuseHQ, open the settings menu, and select API Keys:

On the top-right menu, click Settings

On the left, click API Keys

You can then create a new key by clicking Create API Key. A key creation dialog will open. Simply chose a name and click Create.

The new key is now created and ready to use!

2. Sending XARF reports to AbuseHQ

All XARF reports delivered by this method are processed like regular reports, including inbound processing and how they are displayed in AbuseHQ. The following pictures result from the copyright report from Example reports, sent using curl. To replicate this, just save the report in a file called copyright.json and run the following command with the API Key created above:

Shell

curl -vX POST https://abusix.abusehq.net/xarf/deliver -d @copyright.json --header "Content-Type: application/json" --header "Authorization: Bearer <API Key>"

3. Result

These are screenshots of the report in AbuseHQ after it was delivered using curl.

New in XARF Version 2: Internal Processing

March 2021: Starting with version 2, XARF allows including information that helps with processing when you use it for internal reporting. You can, for example, include information about the subscriber and contract or add tags. The following is an example of the new structure:

You can view a complete and valid example here:

https://github.com/abusix/xarf/blob/master/samples/positive/2/openservice_sample_optional_api_info.json

Internal Processing is optional and only needed for use cases where you already know certain information at the point where you create the report. This simplifies internal processes because you don’t need to perform additional lookup steps later.

Automatic Parsing and Resolving in AbuseHQ

If you are using AbuseHQ, you can immediately leverage the new XARF version for internal reporting. If you include a subscriber id and potentially some more data about the subscriber and contract, you don’t have to perform additional resolving in the Inbound Processing part of AbuseHQ anymore.

1. There are only two requisites needed for that:

2. You need to send XARF reports, as explained above. They need to include at least a subscriber id. You can send them via API, as explained here, or via email, as explained here.

In your inbound processing configuration, you need to add a filter to check whether that id has already been set and avoid any other resolver components because otherwise, the information would be overwritten.

To setup inbound processing correctly, create a new filter component first:

Then attach this component before any resolvers and ensure that the green/positive output doesn’t route through any resolvers. This is a simplified example:

Don’t forget to save the configuration and take it tlive. From now on, all your XARF reports that contain subscriber (and contract) information won’t need to be resolved during inbound processing anymore. As you can see here, the subscriber information is added to the subscriber and case in AbuseHQ:

To learn more about the new XARF schema, head over to the abusix/xarf repository: abusix/xarf