AIL/ail-framework
7
0
Fork 0
mirror of https://github.com/ail-project/ail-framework.git synced 2024-11-10 00:28:22 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions
ail-framework/doc
History
Terrtia 4c20f58a52
chg: [api] add advanced get item via POST + use same query for each get item
2019-08-01 13:16:57 +02:00
..
logo CIRCL AIL logo added 2016-02-08 11:33:53 +01:00
presentation new: [slides] Added training slides december 2018 2018-12-20 09:17:57 +01:00
screenshots chg: [doc] screenshot added 2019-07-05 16:45:30 +02:00
statistics fix: [Install] add python and package requirements 2018-09-12 19:30:00 +02:00
generate_graph_data.py improve doc graphs display 2018-04-06 14:56:25 +02:00
generate_modules_data_flow_graph.sh improve doc graphs display 2018-04-06 14:56:25 +02:00
README.md chg: [api] add advanced get item via POST + use same query for each get item 2019-08-01 13:16:57 +02:00
SourceCode.info Improve SourceCode, keywords and add description in /doc 2016-03-12 12:30:38 +01:00
X-docker.md chg: [doc] move the Docker junk 2019-07-05 16:35:45 +02:00

README.md

API DOCUMENTATION

General

Automation key

The authentication of the automation is performed via a secure key available in the AIL UI interface. Make sure you keep that key secret. It gives access to the entire database! The API key is available in the Server Management menu under My Profile.

The authorization is performed by using the following header:

Authorization: YOUR_API_KEY

Accept and Content-Type headers

When submitting data in a POST, PUT or DELETE operation you need to specify in what content-type you encoded the payload. This is done by setting the below Content-Type headers:

Content-Type: application/json

Example:

curl --header "Authorization: YOUR_API_KEY" --header "Content-Type: application/json" https://AIL_URL/

Item management

Get item: api/get/item/default/<path:item_id>

Description

Get item default info.

Method : GET

Parameters

  • id
    • item id
    • str - relative item path
    • mandatory

JSON response

  • content
    • item content
    • str
  • id
    • item id
    • str
  • date
    • item date
    • str - YYMMDD
  • tags
    • item tags list
    • list

Example

curl https://127.0.0.1:7000/api/get/item/default/submitted/2019/07/26/3efb8a79-08e9-4776-94ab-615eb370b6d4.gz --header "Authorization: iHc1_ChZxj1aXmiFiF1mkxxQkzawwriEaZpPqyTQj " -H "Content-Type: application/json"

Expected Success Response

HTTP Status Code : 200

  {
    "content": "item content test",
    "date": "20190726",
    "id": "submitted/2019/07/26/3efb8a79-08e9-4776-94ab-615eb370b6d4.gz",
    "tags":
      [
        "misp-galaxy:backdoor=\"Rosenbridge\"",
        "infoleak:automatic-detection=\"pgp-message\"",
        "infoleak:automatic-detection=\"encrypted-private-key\"",
        "infoleak:submission=\"manual\"",
        "misp-galaxy:backdoor=\"SLUB\""
      ]
  }

Expected Fail Response

HTTP Status Code : 400

  {"status": "error", "reason": "Mandatory parameter(s) not provided"}

HTTP Status Code : 404

  {"status": "error", "reason": "Item not found"}

Get item content: api/get/item/content/<path:item_id>

Description

Get a specific item content.

Method : GET

Parameters

  • id
    • item id
    • str - relative item path
    • mandatory

JSON response

  • content
    • item content
    • str
  • id
    • item id
    • str

Example

curl https://127.0.0.1:7000/api/get/item/content/submitted/2019/07/26/3efb8a79-08e9-4776-94ab-615eb370b6d4.gz --header "Authorization: iHc1_ChZxj1aXmiFiF1mkxxQkzawwriEaZpPqyTQj " -H "Content-Type: application/json"

Expected Success Response

HTTP Status Code : 200

  {
    "content": "item content test",
    "id": "submitted/2019/07/26/3efb8a79-08e9-4776-94ab-615eb370b6d4.gz"
  }

Expected Fail Response

HTTP Status Code : 400

  {"status": "error", "reason": "Mandatory parameter(s) not provided"}

HTTP Status Code : 404

  {"status": "error", "reason": "Item not found"}

Get item content: api/get/item/tag/<path:item_id>

Description

Get all tags from an item.

Method : GET

Parameters

  • id
    • item id
    • str - relative item path
    • mandatory

JSON response

  • content
    • item content
    • str
  • tags
    • item tags list
    • list

Example

curl https://127.0.0.1:7000/api/get/item/tag/submitted/2019/07/26/3efb8a79-08e9-4776-94ab-615eb370b6d4.gz --header "Authorization: iHc1_ChZxj1aXmiFiF1mkxxQkzawwriEaZpPqyTQj " -H "Content-Type: application/json"

Expected Success Response

HTTP Status Code : 200

  {
    "id": "submitted/2019/07/26/3efb8a79-08e9-4776-94ab-615eb370b6d4.gz",
    "tags":
      [
        "misp-galaxy:backdoor=\"Rosenbridge\"",
        "infoleak:automatic-detection=\"pgp-message\"",
        "infoleak:automatic-detection=\"encrypted-private-key\"",
        "infoleak:submission=\"manual\"",
        "misp-galaxy:backdoor=\"SLUB\""
      ]
  }

Expected Fail Response

HTTP Status Code : 400

  {"status": "error", "reason": "Mandatory parameter(s) not provided"}

HTTP Status Code : 404

  {"status": "error", "reason": "Item not found"}

Advanced Get item: api/get/item

Description

Get item. Filter requested field.

Method : POST

Parameters

  • id
    • item id
    • str - relative item path
    • mandatory
  • date
    • get item date
    • boolean
    • default: true
  • tags
    • get item tags
    • boolean
    • default: true
  • content
    • get item content
    • boolean
    • default: false
  • size
    • get item size
    • boolean
    • default: false
  • lines
    • get item lines info
    • boolean
    • default: false

JSON response

  • content
    • item content
    • str
  • id
    • item id
    • str
  • date
    • item date
    • str - YYMMDD
  • tags
    • item tags list
    • list
  • size
    • item size (Kb)
    • int
  • lines
    • item lines info
    • {}
      • max_length
        • line max length line
        • int
      • nb
        • nb lines item
        • int

Example

curl https://127.0.0.1:7000/api/get/item --header "Authorization: iHc1_ChZxj1aXmiFiF1mkxxQkzawwriEaZpPqyTQj " -H "Content-Type: application/json" --data @input.json -X POST

input.json Example

{
  "id": "submitted/2019/07/26/3efb8a79-08e9-4776-94ab-615eb370b6d4.gz",
  "content": true,
  "lines_info": true,
  "tags": true,
  "size": true
}

Expected Success Response

HTTP Status Code : 200

  {
    "content": "b'dsvcdsvcdsc vvvv'",
    "date": "20190726",
    "id": "submitted/2019/07/26/3efb8a79-08e9-4776-94ab-615eb370b6d4.gz",
    "lines": {
      "max_length": 19,
      "nb": 1
    },
    "size": 0.03,
    "tags": [
      "misp-galaxy:stealer=\"Vidar\"",
      "infoleak:submission=\"manual\""
    ]
  }

Expected Fail Response

HTTP Status Code : 400

  {"status": "error", "reason": "Mandatory parameter(s) not provided"}

HTTP Status Code : 404

  {"status": "error", "reason": "Item not found"}

add item tags: api/add/item/tag

Description

Add tags to an item.

Method : POST

Parameters

  • id
    • item id
    • str - relative item path
    • mandatory
  • tags
    • list of tags
    • list
    • default: []
  • galaxy
    • list of galaxy
    • list
    • default: []

JSON response

  • id
    • item id
    • str - relative item path
  • tags
    • list of item tags added
    • list

Example

curl https://127.0.0.1:7000/api/import/item --header "Authorization: iHc1_ChZxj1aXmiFiF1mkxxQkzawwriEaZpPqyTQj " -H "Content-Type: application/json" --data @input.json -X POST

input.json Example

  {
    "id": "submitted/2019/07/26/3efb8a79-08e9-4776-94ab-615eb370b6d4.gz",
    "tags": [
      "infoleak:analyst-detection=\"private-key\"",
      "infoleak:analyst-detection=\"api-key\""
    ],
    "galaxy": [
      "misp-galaxy:stealer=\"Vidar\""
    ]
  }

Expected Success Response

HTTP Status Code : 200

  {
    "id": "submitted/2019/07/26/3efb8a79-08e9-4776-94ab-615eb370b6d4.gz",
    "tags": [
      "infoleak:analyst-detection=\"private-key\"",
      "infoleak:analyst-detection=\"api-key\"",
      "misp-galaxy:stealer=\"Vidar\""
    ]
  }

Expected Fail Response

HTTP Status Code : 400

  {"status": "error", "reason": "Item id not found"}
  {"status": "error", "reason": "Tags or Galaxy not specified"}
  {"status": "error", "reason": "Tags or Galaxy not enabled"}

Delete item tags: api/delete/item/tag

Description

Delete tags from an item.

Method : DELETE

Parameters

  • id
    • item id
    • str - relative item path
    • mandatory
  • tags
    • list of tags
    • list
    • default: []

JSON response

  • id
    • item id
    • str - relative item path
  • tags
    • list of item tags deleted
    • list

Example

curl https://127.0.0.1:7000/api/delete/item/tag --header "Authorization: iHc1_ChZxj1aXmiFiF1mkxxQkzawwriEaZpPqyTQj " -H "Content-Type: application/json" --data @input.json -X DELETE

input.json Example

  {
    "id": "submitted/2019/07/26/3efb8a79-08e9-4776-94ab-615eb370b6d4.gz",
    "tags": [
      "infoleak:analyst-detection=\"private-key\"",
      "infoleak:analyst-detection=\"api-key\"",
      "misp-galaxy:stealer=\"Vidar\""
    ]
  }

Expected Success Response

HTTP Status Code : 200

  {
    "id": "submitted/2019/07/26/3efb8a79-08e9-4776-94ab-615eb370b6d4.gz",
    "tags": [
      "infoleak:analyst-detection=\"private-key\"",
      "infoleak:analyst-detection=\"api-key\"",
      "misp-galaxy:stealer=\"Vidar\""
    ]
  }

Expected Fail Response

HTTP Status Code : 400

  {"status": "error", "reason": "Item id not found"}
  {"status": "error", "reason": "No Tag(s) specified"}

Import management

Import item (currently: text only): api/import/item

Description

Allows users to import new items. asynchronous function.

Method : POST

Parameters

  • type
    • import type
    • str
    • default: text
  • text
    • text to import
    • str
    • mandatory if type = text
  • default_tags
    • add default import tag
    • boolean
    • default: True
  • tags
    • list of tags
    • list
    • default: []
  • galaxy
    • list of galaxy
    • list
    • default: []

JSON response

  • uuid
    • import uuid
    • uuid4

Example

curl https://127.0.0.1:7000/api/import/item --header "Authorization: iHc1_ChZxj1aXmiFiF1mkxxQkzawwriEaZpPqyTQj " -H "Content-Type: application/json" --data @input.json -X POST

input.json Example

  {
    "type": "text",
    "tags": [
      "infoleak:analyst-detection=\"private-key\""
    ],
    "text": "text to import"
  }

Expected Success Response

HTTP Status Code : 200

  {
    "uuid": "0c3d7b34-936e-4f01-9cdf-2070184b6016"
  }

Expected Fail Response

HTTP Status Code : 400

  {"status": "error", "reason": "Malformed JSON"}
  {"status": "error", "reason": "No text supplied"}
  {"status": "error", "reason": "Tags or Galaxy not enabled"}
  {"status": "error", "reason": "Size exceeds default"}

GET Import item info: api/import/item/<uuid4>

Description

Get import status and all items imported by uuid

Method : GET

Parameters

  • uuid
    • import uuid
    • uuid4
    • mandatory

JSON response

  • status
    • import status
    • str
    • values: in queue, in progress, imported
  • items
    • list of imported items id
    • list
    • The full list of imported items is not complete until status = "imported"

Example

curl -k https://127.0.0.1:7000/api/import/item/b20a69f1-99ad-4cb3-b212-7ce24b763b50 --header "Authorization: iHc1_ChZxj1aXmiFiF1mkxxQkzawwriEaZpPqyTQj " -H "Content-Type: application/json"

Expected Success Response

HTTP Status Code : 200

  {
    "items": [
      "submitted/2019/07/26/b20a69f1-99ad-4cb3-b212-7ce24b763b50.gz"
    ],
    "status": "imported"
  }

Expected Fail Response

HTTP Status Code : 400

  {"status": "error", "reason": "Invalid uuid"}
  {"status": "error", "reason": "Unknown uuid"}

FUTURE endpoints

Text search by daterange

api/search/textIndexer/item POST

Get all tags list

api/get/tag/all

Get tagged items by daterange

api/search/tag/item POST

Submit a domain to crawl

api/add/crawler/domain POST

Create a term/set/regex tracker

api/add/termTracker/ POST

Get tracker items list

api/get/termTracker/item POST

Check if a tor/regular domain have been crawled

api/get/crawler/domain/ POST

Check if a tor/regular domain have been crawled

api/get/crawler/domain/metadata/ <domain><port> GET

Get domain tags

api/get/crawler/domain/tag/ <domain><port> GET

Get domain history

api/get/crawler/domain/history/ <domain><port> GET

Get domain list of items

api/get/crawler/domain/item/ <domain><port> GET

Create auto-crawlers

api/add/crawler/autoCrawler/ POST

get item by mime type/ decoded type

api/get/decoded POST

Check if a decoded item exists (via sha1)

api/get/decoded/exist/<sha1> GET

Get decoded item metadata

Check if a decoded item exists (via sha1)

api/get/decoded/metadata/<sha1> GET

Get decoded item correlation (1 depth)

api/get/decoded/metadata/<sha1> GET
api/get/cryptocurrency POST

Check if a cryptocurrency address (bitcoin, ..) exists

api/get/cryptocurrency/exist/<bitcoin_address> GET

Get cryptocurrency address metadata

api/get/cryptocurrency/metadata/<bitcoin_address> GET

Item correlation (1 depth)

api/get/item/correlation/ POST

Create MISP event from item

api/export/item/misp POST

Create TheHive case from item

api/export/item/thehive POST