95 commits

Author	SHA1	Message	Date
terrtia	72f4733242	chg: [crawler] crawl list urls: filter duplicates	2024-10-09 15:37:37 +02:00
terrtia	1505bf0157	chg: [crawler] submit free text of urls to crawl	2024-10-09 15:05:27 +02:00
terrtia	9d26a47c17	chg: [onion module] filter onion v2	2024-10-08 16:26:46 +02:00
terrtia	554897e1d8	chg: [crawler] update tor user agent	2024-10-08 10:27:03 +02:00
terrtia	d9fc014a1d	fix: [crawler] filter lookup tags	2024-10-07 14:53:13 +02:00
terrtia	83e11082b5	fix: [crawler] filter lookup parent + domain daterange	2024-10-07 11:03:56 +02:00
terrtia	d91e14f200	chg: [domain lookup] extract domain from url input	2024-10-04 13:44:50 +02:00
terrtia	5a052c47d9	chg: [api] rename domain lookup	2024-10-04 11:53:55 +02:00
terrtia	483d49fecf	chg: [api] add domain lookup	2024-10-03 14:59:12 +02:00
terrtia	9fb19028fe	fix: [crawler] fix crawler queue stats	2024-09-17 16:59:09 +02:00
terrtia	759d241b75	fix: [crawler] fix crawler queue stats	2024-09-17 16:52:36 +02:00
terrtia	a20b6054e8	fix: [crawler] fix crawler queue stats	2024-09-17 15:36:15 +02:00
terrtia	7b66ff6a8c	fix: [crawler] crawler capture with empty task	2024-09-16 10:50:34 +02:00
terrtia	8ab66e7309	chg: [cookiejars] show organisation	2024-09-06 14:10:38 +02:00
terrtia	a05e1feed6	chg: [acl] refactor acl cookiejars, trackers, retro_hunts, investigation + refactor users roles	2024-09-05 14:41:13 +02:00
terrtia	b466d4766a	chg: [cookiejar] add org level to cookiejar + update acl to support org	2024-08-28 14:32:26 +02:00
terrtia	86f312cbc3	chg: [crawler] add function to delete schedules	2024-05-15 10:21:08 +02:00
terrtia	d5e830c591	chg: [domains] add crawler status stats by domain type pie chart	2024-02-28 14:19:47 +01:00
terrtia	0d55725e28	chg: [crawler] add monthly crawled domains stats	2024-02-27 14:56:48 +01:00
terrtia	fbd7e2236a	fix: [crawlers] fix errored capture start time	2024-01-30 11:24:12 +01:00
terrtia	bd2ca4b319	fix: [crawler] fix api create_task	2024-01-09 09:47:49 +01:00
terrtia	9221e532c4	fix: [crawlers] fix task start	2023-12-12 11:32:33 +01:00
terrtia	235539ea42	fix: [crawler] fix capture start time	2023-12-11 09:30:09 +01:00
terrtia	1c52c187ad	fix: [api] fix add crawler capture return	2023-12-08 10:37:58 +01:00
terrtia	a382b572c6	chg: [crawler] push onion discovery capture_uuid to another AIL	2023-12-07 11:28:35 +01:00
terrtia	c5cef5fd00	chg: [core] merge master + fix object subtype correlation stats	2023-10-12 13:53:00 +02:00
Jean-Louis Huynen	68c17c3fbc	chg: [crawlers] submit cookies to the crawler task API	2023-08-31 16:13:20 +02:00
Jean-Louis Huynen	ed0423118e	chg: [crawlers] submit a single cookie to the crawler task API	2023-08-31 15:42:44 +02:00
Terrtia	b32f110285	chg: [chat + user-account] correlations + usernames timeline	2023-08-28 16:29:38 +02:00
Terrtia	4e3784922c	fix: typo	2023-08-23 11:47:39 +02:00
Terrtia	2145eb7b8a	fix: [title] fix None title	2023-08-23 11:46:37 +02:00
Terrtia	68dffcd26b	chg: [api crawler] fix response + add cookiejar, proxy and frequency parameters	2023-07-25 15:57:11 +02:00
Terrtia	a9485928db	chg: [HHHash] add HHHash object and correlation https://www.foo.be/2023/07/HTTP-Headers-Hashing_HHHash	2023-07-17 15:47:17 +02:00
Terrtia	73bfe614df	chg: [updater] refactor background updater + add v5.2 update	2023-07-12 11:36:47 +02:00
Terrtia	28c647d370	chg: [crawler har] compress HAR	2023-07-10 15:56:34 +02:00
Terrtia	c719990125	fix: [crawler] add timeout to Unknown captures	2023-07-10 11:23:44 +02:00
fukusuket	e35924ec22	fix: [crawler] add exception handing for ping_lacus	2023-07-08 12:11:25 +09:00
Terrtia	450ebdd789	chg: [etag] add new etag object	2023-07-06 11:26:32 +02:00
Terrtia	47e1343187	fix: [crawler] same capture uuid if a domain is already crawled	2023-06-22 16:09:18 +02:00
Terrtia	501d10bbbd	chg: [crawler] auto tag crawled domains	2023-06-20 08:11:44 +02:00
Terrtia	f8fd037bd2	chg: [object cookie-name] add new cookie-name object + correlation	2023-06-16 15:39:13 +02:00
Terrtia	94961f2eba	chg: [favicon object] add favicon object	2023-06-12 16:51:45 +02:00
Terrtia	405d097024	fix: [crawler] fix undefined capture status	2023-05-25 16:26:48 +02:00
Terrtia	c008366f02	chg: [new title object] add new title object + correlation on page title	2023-05-25 14:33:12 +02:00
Terrtia	7669c16c74	fix: [Onion module] fix kvrocks sismeber	2023-05-15 10:42:46 +02:00
Terrtia	54a0bcb022	chg: [crawler] update default user agent	2023-04-04 09:23:52 +02:00
Terrtia	47da4aa62c	chg: [crawle] migrate domains settings	2023-03-31 09:25:06 +02:00
Terrtia	126ecb2e39	fix: [core] fix merge	2023-03-16 16:49:53 +01:00
Terrtia	524a404dc8	chg: [core] merge conflict	2023-03-16 15:50:42 +01:00
Terrtia	925d67a35e	chg: [crawler] add crawler scheduler	2023-03-14 17:36:42 +01:00