new: [interpol] INTERPOL Dark Web and Virtual Assets Taxonomies

INTERPOL Dark Web and Virtual Assets Taxonomies

.vscode/launch.json

@@ -1,6 +1,16 @@
 {
     "version": "0.2.0",
     "configurations": [
+        {
+            "name": "gen_interpol_dwvat",
+            "type": "debugpy",
+            "request": "launch",
+            "program": "${file}",
+            "console": "integratedTerminal",
+            "args": "-p ../../DW-VA-Taxonomy",
+            "cwd": "${fileDirname}"
+
+        },
         {
             "name": "Python Debugger: Current File",
             "type": "debugpy",

README.md

@@ -6,7 +6,7 @@
 
 MISP galaxy is a simple method to express a large object called cluster that can be attached to MISP events or
 attributes. A cluster can be composed of one or more elements. Elements are expressed as key-values. There
-are default knowledge base (such as Threat Actors, Tools, Ransomware, ATT&CK matrixes) available in MISP galaxy 
+are default knowledge base (such as Threat Actors, Tools, Ransomware, ATT&CK matrixes) available in MISP galaxy
 but those can be overwritten, replaced, updated, forked and shared as you wish.
 
 Existing clusters and vocabularies can be used as-is or as a common knowledge base. MISP distribution can be applied
@@ -63,7 +63,7 @@ Category: *guidelines* - source: *Open Sources* - total: *71* elements
 
 [Backdoor](https://www.misp-project.org/galaxy.html#_backdoor) - A list of backdoor malware.
 
-Category: *tool* - source: *Open Sources* - total: *24* elements
+Category: *tool* - source: *Open Sources* - total: *28* elements
 
 [[HTML](https://www.misp-project.org/galaxy.html#_backdoor)] - [[JSON](https://github.com/MISP/misp-galaxy/blob/main/clusters/backdoor.json)]
 
@@ -211,6 +211,14 @@ Category: *Intelligence Agencies* - source: *https://en.wikipedia.org/wiki/List_
 
 [[HTML](https://www.misp-project.org/galaxy.html#_intelligence_agencies)] - [[JSON](https://github.com/MISP/misp-galaxy/blob/main/clusters/intelligence-agencies.json)]
 
+## INTERPOL DWVA Taxonomy
+
+[INTERPOL DWVA Taxonomy](https://www.misp-project.org/galaxy.html#_interpol_dwva_taxonomy) - This taxonomy defines common forms of abuses and entities that represent real-world actors and service that are part of a larger Darknet- and Cryptoasset Ecosystems.
+
+Category: *dwva* - source: *https://interpol-innovation-centre.github.io/DW-VA-Taxonomy/* - total: *94* elements
+
+[[HTML](https://www.misp-project.org/galaxy.html#_interpol_dwva_taxonomy)] - [[JSON](https://github.com/MISP/misp-galaxy/blob/main/clusters/interpol-dwva.json)]
+
 ## Malpedia
 
 [Malpedia](https://www.misp-project.org/galaxy.html#_malpedia) - Malware galaxy cluster based on Malpedia.
@@ -269,7 +277,7 @@ Category: *course-of-action* - source: *https://github.com/mitre/cti* - total: *
 
 ## mitre-data-component
 
-[mitre-data-component](https://www.misp-project.org/galaxy.html#_mitre-data-component) - Data components are parts of data sources. 
+[mitre-data-component](https://www.misp-project.org/galaxy.html#_mitre-data-component) - Data components are parts of data sources.
 
 Category: *data-component* - source: *https://github.com/mitre/cti* - total: *116* elements
 
@@ -277,7 +285,7 @@ Category: *data-component* - source: *https://github.com/mitre/cti* - total: *11
 
 ## mitre-data-source
 
-[mitre-data-source](https://www.misp-project.org/galaxy.html#_mitre-data-source) - Data sources represent the various subjects/topics of information that can be collected by sensors/logs. 
+[mitre-data-source](https://www.misp-project.org/galaxy.html#_mitre-data-source) - Data sources represent the various subjects/topics of information that can be collected by sensors/logs.
 
 Category: *data-source* - source: *https://github.com/mitre/cti* - total: *40* elements
 
@@ -495,7 +503,7 @@ Category: *actor* - source: *MISP Project* - total: *15* elements
 
 [Ransomware](https://www.misp-project.org/galaxy.html#_ransomware) - Ransomware galaxy based on https://docs.google.com/spreadsheets/d/1TWS238xacAto-fLKh1n5uTsdijWdCEsGIM0Y0Hvmc5g/pubhtml and http://pastebin.com/raw/GHgpWjar
 
-Category: *tool* - source: *Various* - total: *1705* elements
+Category: *tool* - source: *Various* - total: *1706* elements
 
 [[HTML](https://www.misp-project.org/galaxy.html#_ransomware)] - [[JSON](https://github.com/MISP/misp-galaxy/blob/main/clusters/ransomware.json)]
 
@@ -535,7 +543,7 @@ Category: *sector* - source: *CERT-EU* - total: *118* elements
 
 [Sigma-Rules](https://www.misp-project.org/galaxy.html#_sigma-rules) - MISP galaxy cluster based on Sigma Rules.
 
-Category: *rules* - source: *https://github.com/jstnk9/MISP/tree/main/misp-galaxy/sigma* - total: *2840* elements
+Category: *rules* - source: *https://github.com/jstnk9/MISP/tree/main/misp-galaxy/sigma* - total: *2873* elements
 
 [[HTML](https://www.misp-project.org/galaxy.html#_sigma-rules)] - [[JSON](https://github.com/MISP/misp-galaxy/blob/main/clusters/sigma-rules.json)]
 
@@ -575,7 +583,7 @@ Category: *actor* - source: *MISP Project* - total: *50* elements
 
 [Target Information](https://www.misp-project.org/galaxy.html#_target_information) - Description of targets of threat actors.
 
-Category: *target* - source: *Various* - total: *240* elements
+Category: *target* - source: *Various* - total: *241* elements
 
 [[HTML](https://www.misp-project.org/galaxy.html#_target_information)] - [[JSON](https://github.com/MISP/misp-galaxy/blob/main/clusters/target-information.json)]
 
@@ -599,7 +607,7 @@ Category: *tea-matrix* - source: ** - total: *7* elements
 
 [Threat Actor](https://www.misp-project.org/galaxy.html#_threat_actor) - Known or estimated adversary groups targeting organizations and employees. Adversary groups are regularly confused with their initial operation or campaign. threat-actor-classification meta can be used to clarify the understanding of the threat-actor if also considered as operation, campaign or activity group.
 
-Category: *actor* - source: *MISP Project* - total: *644* elements
+Category: *actor* - source: *MISP Project* - total: *671* elements
 
 [[HTML](https://www.misp-project.org/galaxy.html#_threat_actor)] - [[JSON](https://github.com/MISP/misp-galaxy/blob/main/clusters/threat-actor.json)]
 
@@ -663,7 +671,7 @@ Category: *tmss* - source: *https://github.com/microsoft/Threat-matrix-for-stora
 
 [Tool](https://www.misp-project.org/galaxy.html#_tool) - threat-actor-tools is an enumeration of tools used by adversaries. The list includes malware but also common software regularly used by the adversaries.
 
-Category: *tool* - source: *MISP Project* - total: *596* elements
+Category: *tool* - source: *MISP Project* - total: *603* elements
 
 [[HTML](https://www.misp-project.org/galaxy.html#_tool)] - [[JSON](https://github.com/MISP/misp-galaxy/blob/main/clusters/tool.json)]
 
@@ -675,11 +683,17 @@ Category: *military equipment* - source: *Popular Mechanics* - total: *36* eleme
 
 [[HTML](https://www.misp-project.org/galaxy.html#_uavs/ucavs)] - [[JSON](https://github.com/MISP/misp-galaxy/blob/main/clusters/uavs.json)]
 
-[[HTML](https://www.misp-project.org/galaxy.html#_uavs/ucavs)] - [[JSON](https://github.com/MISP/misp-galaxy/blob/main/clusters/uavs.json)]
+## UKHSA Culture Collections
 
-# Online documentation 
+[UKHSA Culture Collections](https://www.misp-project.org/galaxy.html#_ukhsa_culture_collections) - UK Health Security Agency Culture Collections represent deposits of cultures that consist of expertly preserved, authenticated cell lines and microbial strains of known provenance.
 
-The [misp-galaxy.org](https://misp-galaxy.org) website provides an easily navigable resource for all MISP galaxy clusters. 
+Category: *virus* - source: *https://www.culturecollections.org.uk* - total: *6667* elements
+
+[[HTML](https://www.misp-project.org/galaxy.html#_ukhsa_culture_collections)] - [[JSON](https://github.com/MISP/misp-galaxy/blob/main/clusters/ukhsa-culture-collections.json)]
+
+# Online documentation
+
+The [misp-galaxy.org](https://misp-galaxy.org) website provides an easily navigable resource for all MISP galaxy clusters.
 
 A [readable PDF overview of the MISP galaxy is available](https://www.misp.software/galaxy.pdf) or [HTML](https://www.misp.software/galaxy.html) and generated from the JSON.
 

galaxies/interpol-dwva.json

@@ -0,0 +1,27 @@
+{
+  "description": "This taxonomy defines common forms of abuses and entities that represent real-world actors and service that are part of a larger Darknet- and Cryptoasset Ecosystems.",
+  "icon": "user-secret",
+  "kill_chain_order": {
+    "Abuses": [
+      "Concept"
+    ],
+    "Entities": [
+      "Actor",
+      "Asset",
+      "Authorities",
+      "Cryptocurrency",
+      "Dark_Web",
+      "Generic",
+      "Infrastructure",
+      "Process",
+      "Service",
+      "Technology",
+      "Wallet"
+    ]
+  },
+  "name": "INTERPOL DWVA Taxonomy",
+  "namespace": "interpol",
+  "type": "dwva",
+  "uuid": "a375d7fd-0a3e-41cf-a531-ef56033df967",
+  "version": 1
+}

tools/gen_interpol_dwvat.py

@@ -0,0 +1,163 @@
+#!/usr/bin/env python3
+# -*- coding: utf-8 -*-
+#
+#
+#    A simple convertor of the Interpol Dark Web and Virtual Assets Taxonomies to a MISP Galaxy datastructure.
+#    https://github.com/INTERPOL-Innovation-Centre/DW-VA-Taxonomy
+#    Copyright (C) 2024 Christophe Vandeplas
+#
+#    This program is free software: you can redistribute it and/or modify
+#    it under the terms of the GNU Affero General Public License as
+#    published by the Free Software Foundation, either version 3 of the
+#    License, or (at your option) any later version.
+#
+#    This program is distributed in the hope that it will be useful,
+#    but WITHOUT ANY WARRANTY; without even the implied warranty of
+#    MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+#    GNU Affero General Public License for more details.
+#
+#    You should have received a copy of the GNU Affero General Public License
+#    along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+import yaml
+import os
+import uuid
+import re
+import json
+
+import argparse
+
+parser = argparse.ArgumentParser(description='Create/update the Interpol Dark Web and Virtual Assets Taxonomies based on Markdown files.')
+parser.add_argument("-p", "--path", required=True, help="Path of the 'DW-VA-Taxonomy' git clone folder")
+
+args = parser.parse_args()
+
+if not os.path.exists(args.path):
+    exit("ERROR: DW-VA-Taxonomy folder incorrect")
+
+'''
+contains _data folder with
+- abuses.yaml - simple taxonomy
+- entities.yaml - matrix like taxonomy
+'''
+
+try:
+    with open(os.path.join('..', 'galaxies', 'interpol-dwva.json'), 'r') as f:
+        json_galaxy = json.load(f)
+
+except FileNotFoundError:
+    json_galaxy = {
+        'icon': "user-secret",
+        'kill_chain_order': {
+            'Entities': [],
+            'Abuses': ['Concept']
+        },
+        'name': "INTERPOL DWVA Taxonomy",
+        'description': "This taxonomy defines common forms of abuses and entities that represent real-world actors and service that are part of a larger Darknet- and Cryptoasset Ecosystems.",
+        'namespace': "interpol",
+        'type': "dwva",
+        'uuid': "a375d7fd-0a3e-41cf-a531-ef56033df967",
+        'version': 1
+    }
+
+try:
+    with open(os.path.join('..', 'clusters', 'interpol-dwva.json'), 'r') as f:
+        json_cluster = json.load(f)
+except FileNotFoundError:
+    json_cluster = {
+        'authors': ["INTERPOL Darkweb and Virtual Assets Working Group"],
+        'category': 'dwva',
+        'name': "INTERPOL DWVA Taxonomy",
+        'description': "This taxonomy defines common forms of abuses and entities that represent real-world actors and service that are part of a larger Darknet- and Cryptoasset Ecosystems.",
+        'source': 'https://interpol-innovation-centre.github.io/DW-VA-Taxonomy/',
+        'type': "dwva",
+        'uuid': "b15898ba-a923-4916-856c-0dfe8b174196",
+        'values': [],
+        'version': 1
+    }
+
+
+tactics = set()
+clusters_dict = {}
+# FIXME create dict for the existing clusters, so we can update the clusters without losing the relations
+
+#
+# Entities
+#
+with open(os.path.join(args.path, '_data', 'entities.yaml'), 'r') as f:
+    entities_data = yaml.safe_load(f)
+
+# build a broader concept list so we can ignore them later on
+broaders = set()
+for section in entities_data:
+    try:
+        broaders.add(entities_data[section]['broader'])
+    except KeyError:
+        pass
+# the Entities
+for section in entities_data:
+    item = entities_data[section]
+    if item['type'] == 'concept':
+        if item['id'] in broaders:  # skip the broader concepts
+            continue
+        if 'broader' not in item:
+            item['broader'] = 'generic'
+        tactics.add(item['broader'].title())
+        value = item['prefLabel']
+        clusters_dict[value] = {
+            'value': value,
+            'description': item['description'],
+            'uuid': str(uuid.uuid5(uuid.UUID("d0ceebc2-877b-4873-9785-d00f279ccb45"), value)),
+            'meta': {
+                'kill_chain': [f"Entities:{item['broader'].title()}"],
+            }
+        }
+        try:
+            clusters_dict[value]['meta']['refs'] = [item['seeAlso']]
+        except KeyError:
+            pass
+
+#
+# Abuses
+#
+with open(os.path.join(args.path, '_data', 'abuses.yaml'), 'r') as f:
+    entities_data = yaml.safe_load(f)
+for section in entities_data:
+    item = entities_data[section]
+    if item['type'] == 'concept':
+        value = item['prefLabel']
+        clusters_dict[value] = {
+            'value': value,
+            'description': item['description'],
+            'uuid': str(uuid.uuid5(uuid.UUID("d0ceebc2-877b-4873-9785-d00f279ccb45"), value)),
+            'meta': {
+                'kill_chain': [f"Abuses:Concept"],
+            }
+        }
+        try:
+            clusters_dict[value]['meta']['refs'] = [item['seeAlso']]
+        except KeyError:
+            pass
+
+
+#
+# Finally transform dict to list
+#
+clusters = []
+for item in clusters_dict.values():
+    clusters.append(item)
+
+json_cluster['values'] = clusters
+json_galaxy['kill_chain_order']['Entities'] = sorted(list(tactics))
+
+# save the Galaxy and Cluster file
+with open(os.path.join('..', 'galaxies', 'interpol-dwva.json'), 'w') as f:
+    json.dump(json_galaxy, f, indent=2, sort_keys=True, ensure_ascii=False)
+    f.write('\n')  # only needed for the beauty and to be compliant with jq_all_the_things
+
+
+with open(os.path.join('..', 'clusters', 'interpol-dwva.json'), 'w') as f:
+    json.dump(json_cluster, f, indent=2, sort_keys=True, ensure_ascii=False)
+    f.write('\n')  # only needed for the beauty and to be compliant with jq_all_the_things
+
+print("All done, please don't forget to ./jq_all_the_things.sh, commit, and then ./validate_all.sh.")