858 commits

Author	SHA1	Message	Date
Terrtia	a3167a740a	fix: [Update DB] remove indexErrors	2019-04-12 17:32:17 +02:00
Terrtia	bf92a2f22f	fix: [background update] avoid multiple update on the same key	2019-04-12 16:07:40 +02:00
Terrtia	f2dc3bcbf4	fix: [Global] remove absolute path from item path	2019-04-12 15:13:36 +02:00
Terrtia	3a8631d44a	chg: [Update + Launcher] use background update + add new options in launcher	2019-04-12 15:06:24 +02:00
Terrtia	5c4db5f2f1	chg: [update] add update script Redis ARDB + add background update	2019-04-11 17:49:20 +02:00
Terrtia	3cc614a1ad	chg: [DB] update items tags metadata	2019-04-11 11:58:06 +02:00
Terrtia	d44acea04d	chg: [merge] merge update into advanced_crawler	2019-04-10 16:41:06 +02:00
Terrtia	2589fc2161	chg: [merge] merge master into update branche	2019-04-10 15:43:15 +02:00
Terrtia	b4bee29a33	Merge branch 'master' into advanced_crawler	2019-04-10 15:37:42 +02:00
Terrtia	f6d7d2ae16	chg: [Update] add debug	2019-04-02 09:46:04 +02:00
Thirion Aurélien	976b97bcd1	Merge pull request #338 from CIRCL/pre_update ... Merge Pre update	2019-03-26 16:51:26 +01:00
Terrtia	044933aa33	chg: [Launch] remove old launchers	2019-03-26 16:47:10 +01:00
Terrtia	cee2b2486f	chg: [Launch] remove old launchers	2019-03-26 16:45:01 +01:00
Terrtia	59664efe45	Merge branch 'master' into advanced_crawler	2019-03-26 16:03:42 +01:00
Terrtia	5165a5de2f	fix: [Crawler] fix index + redis history key	2019-03-22 17:14:27 +01:00
Terrtia	f64c385343	chg: [Crawler] handle port: crawling + history	2019-03-22 16:48:07 +01:00
Terrtia	f4cdddbc7f	chg: [UI + item submit] remove browse paste from UI + fix pasteSubmit tagging	2019-03-21 11:50:53 +01:00
Terrtia	c2bc72c72e	chg: [Tags alertHandler] remove alertHandler + add quick tag search sidebar + use tag last_seen to display tagged items	2019-03-21 11:11:53 +01:00
Terrtia	a9e86231ad	chg: [Tags] add backend, daterange selector	2019-03-20 13:25:02 +01:00
Terrtia	65e6eae00d	chg: [UI preview items] b4 migration + new template + better display + decrease item loading time + add basic animation/transition	2019-03-19 13:34:41 +01:00
Terrtia	23d9cb2265	chg: [UI Tags search] bootstrap 4 migration	2019-03-14 17:04:55 +01:00
Alexandre Dulaunoy	85d2ea578c	Merge pull request #335 from 5thColumn/docker ... Add docker-compose support	2019-03-13 21:38:12 +01:00
Jason Hedden	23fb91d326	Add docker-compose support ... This commit separates the AIL Framework services into individual Docker containers that can be managed with docker-compose. It's intended to ease the installation, development and troubleshooting procedures for all-in-one deployments. No changes to architecture or base code are required in this commit. Changes to existing files: .gitignore * Ignore pystemon archive directory Dockerfile: * Add AIL shell environment variables * Install pystemon and crawler pip requirements into AIL virtual environment Docker-compose notes: * All containers are participating in a service network `network_mode: service:flask`. This allows the containers to share the same IP namespace to accommodate hard-coded localhost entries. * By default persistent data is saved to the following directories in the local AIL framework git directory: PASTES, HASHS, CRAWLED_SCREENSHOTS, pystemon/archives This was tested with docker-ce on Ubuntu 16.04 and MacOS. A typical deployment would look like: ``` git clone https://github.com/CIRCL/AIL-framework.git cd AIL-framework cp bin/packages/config.cfg.docker-compose-sample bin/packages/config.cfg (optionally enable activate_crawler in config.cfg) docker-compose build docker-compose up -d ```	2019-03-11 19:46:34 -05:00
Terrtia	d3b3942ad7	chg: [Flask UI] refractor crawler frontend + show crawled domain	2019-03-06 15:05:58 +01:00
Terrtia	c0d72e7d2a	chg: [Crawler UI] Crawler major refractor (end) + basic UI for manual crawler	2019-02-26 14:50:48 +01:00
Terrtia	7b32d7f34e	chg: [Crawler] major refractor	2019-02-25 16:38:50 +01:00
Jason Hedden	9eae25bdf5	Fix current_selected amount and action vars ... This commit fixes issue #268 ``` File "./bin/ModulesInformationV2.py", line 319, in _setValue action = current_selected_action + " " + str(current_selected_amount) + "x" ``` `current_selected_action` was defined twice. The second occurance should be `current_selected_amount`. This also fixes the generic exception from failing when `current_selected_value` is not defined.	2019-02-22 12:39:27 -06:00
Terrtia	60f7645ac1	chg: [Crawler] refractor	2019-02-22 17:00:24 +01:00
Terrtia	3d36ddbc1e	fix: [CVE] fix stat regex	2019-02-21 10:51:05 +01:00
Terrtia	1114aa2ffd	chg: [CVE] add stat script	2019-02-21 09:58:10 +01:00
Terrtia	e5dca268a8	chg: [Crawler] refractor	2019-02-21 09:54:43 +01:00
Terrtia	da78d0552d	chg: [Crawler UI Tags] add tag by day + add crawler status + UI onion blacklist	2019-02-19 11:41:45 +01:00
Terrtia	6a8fa3d5d1	chg: [Tags] tags by day	2019-02-18 14:33:34 +01:00
Terrtia	709d5487b8	fix: [Onion] filter too many subdomain	2019-02-13 16:53:38 +01:00
Terrtia	b87707e8bc	fix: [Crawler] typo	2019-02-12 15:54:42 +01:00
Terrtia	37276e52a3	fix: [Crawler] typo	2019-02-12 15:53:40 +01:00
Terrtia	7cb03fc769	fix: [Crawler] typo	2019-02-12 15:51:19 +01:00
Terrtia	7a4989ce10	fix: [Global Crawler] max filename size	2019-02-12 15:45:58 +01:00
Terrtia	c2885589cf	chg: [UI] basic navbar + sidebar + refractor	2019-02-07 17:22:44 +01:00
Terrtia	516238025f	chg: [Crawler] add bootsrap4 src + refractor crawler	2019-02-05 17:16:44 +01:00
Thirion Aurélien	44c513dcbb	chg: [Crawler] add onion to blacklist	2019-01-31 16:56:45 +01:00
Xavier Mertens	43d08d0d3e	Cleanup	2019-01-30 22:36:10 +01:00
Xavier Mertens	1be91b88bf	Fix: reset alarm timeout	2019-01-30 22:30:07 +01:00
Xavier Mertens	0372a78704	Merge remote-tracking branch 'upstream/master'	2019-01-30 22:27:44 +01:00
Xavier Mertens	b099e2ae4a	Update LAUNCH.sh	2019-01-30 22:27:33 +01:00
Terrtia	92d192238b	fix: [Crawler] change max page crawled	2019-01-29 17:04:45 +01:00
Terrtia	6c7086f4eb	fix: [Crawler] first_seen	2019-01-29 16:54:39 +01:00
Terrtia	ad04cf7bb2	fix: [Onion] typo	2019-01-29 16:27:12 +01:00
Terrtia	88eaaeae93	chg: [Crawler] add priority queue, fix #263	2019-01-29 16:08:59 +01:00
Terrtia	c1b34bd99c	fix: [Crawler] limit max crawled pages	2019-01-29 15:38:00 +01:00
Terrtia	64ffdd52e8	fix: [Onion] loop	2019-01-29 14:39:57 +01:00
Terrtia	2dc0eca4a9	fix: [Crawler] fix crawler cache info	2019-01-29 12:09:19 +01:00
Terrtia	bb301a870c	fix: [Crawler] fix onion blacklist + add crawler info	2019-01-29 12:00:14 +01:00
Terrtia	535f44044d	fix: [Onion] add timeout signal	2019-01-29 09:46:03 +01:00
Thirion Aurélien	f4ba21e492	blacklist onion debian manpages	2019-01-14 11:08:53 +01:00
raggadhub	74c726d5fa	fix path_to_send path ... Hi, Missing slash between pastes_directory and paste in path_to_send = pastes_directory+paste path error example: /opt/AIL-framework/PASTESarchive/pastebin.com_pro/2019/01/07/cAgqsY2U.gz	2019-01-09 13:15:58 +01:00
Terrtia	b3b75ccbea	fix: [Crawler] Restart Splash on failure, limit unbound in memory cache (maxrss)	2019-01-04 15:51:08 +01:00
Terrtia	6ee4b7bf93	Merge branch 'master' into update	2018-12-19 15:23:34 +01:00
Terrtia	ca47764836	fix: [update v1.5]	2018-12-19 15:16:52 +01:00
Terrtia	198ee97d90	chg: [update 1.5] add update bash	2018-12-19 11:41:01 +01:00
Terrtia	2560260ce4	Merge branch 'master' into pre_update	2018-12-19 10:01:52 +01:00
Terrtia	4d04333f54	fix: [Splash server] add debug output	2018-12-19 09:30:24 +01:00
Terrtia	f842194c57	fix: [Crawler] retry when splash is not available	2018-12-17 16:04:12 +01:00
Terrtia	4e08aaa80f	chg: [UI Crawled Domains] show crawled domains by date range (options: domains tags, up domains, down domains) ... todo: Shows domain history	2018-12-17 15:58:48 +01:00
Steve Clement	b60fd55efe	chg: [bash] Small PATH precedence issue.	2018-11-22 16:16:31 +09:00
Steve Clement	f82c899e7b	fix: [bash] fixed log_subscriber path	2018-11-22 15:48:12 +09:00
Steve Clement	88e561aa20	chg: [bash] Replace '' with ""	2018-11-22 15:41:45 +09:00
Steve Clement	33bc23a297	fix: [bash] evoking bash -c and splitting vars in this fashion wants to be changed in future revisions.	2018-11-22 15:22:35 +09:00
Steve Clement	6522cffd57	chg: [doc] Added note that redis-server is bundled during install	2018-11-22 15:10:57 +09:00
Steve Clement	41a34b3519	fix: [bash] As we instantiate from a subdirectory, we need to take this into account.	2018-11-22 15:08:59 +09:00
Steve Clement	e4ced241eb	chg: [python] Added necessary checks in LAUNCH.sh to be able to launch the script without doing bin/activate ... chg: [doc] Added correct pystemon path fix: [zmq] Small ZMQ test fix. fix: [doc] Moved the print statement in feeder so it is less verbose but still useful	2018-11-22 14:57:24 +09:00
Terrtia	31a8dfe0b3	fix: [AIL] use only relative paths ... pastes duplicates are fixed on the fly	2018-11-21 16:45:25 +01:00
Terrtia	cb41c54135	fix: [lib_refine] add dns configuration	2018-11-21 08:49:58 +01:00
Terrtia	4e680aabf0	chg: [Overview] add doc	2018-11-20 14:39:45 +01:00
Terrtia	aaa277b8a0	chg: [Update] add additonal update	2018-11-15 14:26:41 +01:00
Terrtia	f6e86582c8	chg: [Update] generate current version	2018-11-15 13:48:44 +01:00
Terrtia	108fdb868e	chg: update Overview	2018-11-15 10:39:41 +01:00
Terrtia	d9b776d58a	merge master	2018-11-14 15:55:42 +01:00
Terrtia	aed8d65aef	fix: [LAUNCH] add update in helper	2018-11-14 15:17:56 +01:00
Terrtia	912b977bb4	chg: [LAUNCH] update: launch + check BDD	2018-11-13 16:54:39 +01:00
Terrtia	94fcf66d20	fix: [Update] cleaning	2018-11-13 15:52:33 +01:00
Terrtia	347986a271	chg: [LAUNCH] add AIL update by default	2018-11-13 15:45:22 +01:00
Terrtia	7aff45c507	chg [Update] update AIL clone and fork	2018-11-13 14:54:56 +01:00
Terrtia	60ff0b9cf7	chg: [Update] add update script	2018-11-12 17:10:31 +01:00
Terrtia	b31618eb6a	chg: [Mixer] config default feeder name #291	2018-11-09 15:26:26 +01:00
Terrtia	c3903ab203	fix: [MISP_The_Hive_feeder] handle not saved pastes	2018-11-09 13:58:09 +01:00
Sami Mokaddem	62e46b6dc1	Merge pull request #287 from CIRCL/term_management ... Term management upgrade	2018-11-08 14:10:25 +01:00
Terrtia	e3764ca575	fix: [MISP_the_Hive_feeder] force absolute path	2018-11-08 11:22:23 +01:00
Terrtia	979472df15	chg: [Terms] tag tracked terms	2018-11-06 13:38:37 +01:00
Terrtia	7ef5cc3205	fix:[MISP_The_Hive_feeder] typo	2018-11-05 16:40:25 +01:00
Terrtia	85f933fd81	chg: [NotificationHelper] add error traceback	2018-11-05 14:59:40 +01:00
Terrtia	877ac0636e	fix: typo	2018-11-05 14:30:03 +01:00
Terrtia	aa16c52cbf	fix: [NotificationHelper] add config	2018-11-05 14:20:12 +01:00
Terrtia	dad0365c67	fix:[MISP_The_Hive_feeder] set default values	2018-11-05 09:16:18 +01:00
Terrtia	5872cf9196	fix: [Scripts] Remove absolute path	2018-11-02 16:07:27 +01:00
Xavier Mertens	05de39e271	Fixed line counter (count all lines)	2018-11-02 15:55:55 +01:00
Xavier Mertens	ac95dd4ee8	Added timeout for Regex module	2018-11-02 15:50:38 +01:00
Xavier Mertens	9e7ca84581	Added timeout to avoid blocking regexes	2018-11-02 15:49:06 +01:00
Xavier Mertens	9bfb7a5fb6	Added module Regex.py	2018-10-26 17:13:26 +02:00
Terrtia	ea7fd8d54c	fix: [Duplicate] show duplicate with relative path	2018-10-24 15:47:03 +02:00
Terrtia	5afbc3179c	Merge branch 'master' of https://github.com/CIRCL/AIL-framework	2018-10-24 15:16:41 +02:00
Terrtia	8a6ac6be1e	fix: [Paste] create absolute path	2018-10-24 15:16:12 +02:00
Sami Mokaddem	d9bc2d6f44	Merge pull request #266 from kovacsbalu/fix-regexp-timeout ... Stop regexp processing after timeout (60sec)	2018-10-09 14:48:08 +02:00
kovacsbalu	c1c40e3e56	Fix config section name	2018-10-09 11:30:04 +02:00
kovacsbalu	272b0fe960	RegexpForTerm timeout from config ... Critical log removed	2018-10-09 09:32:32 +02:00
kovacsbalu	6d199f0150	Stop regexp processing after timeput (60sec) ... Minor pep8 fixes	2018-10-08 11:25:32 +02:00
Alexandre Dulaunoy	efaba35dec	Common onion addresses	2018-10-05 13:50:17 +02:00
Alexandre Dulaunoy	ed9f9e5dce	Merge pull request #262 from kovacsbalu/feat-smtp-test ... Test notification	2018-10-02 17:27:55 +02:00
Terrtia	44d19fbb80	fix: [pystemon-feeder] paste save path	2018-10-02 16:02:59 +02:00
Alexandre Dulaunoy	59df5f0e12	chg: [config] Tor crawler disabled by default in sample configuration ... (to match HOWTO when enabling it)	2018-10-02 14:17:58 +02:00
kovacsbalu	93a3c5779c	Test notification ... Minor pep8 fixes	2018-10-01 15:56:48 +02:00
Terrtia	6328cc22b7	chg: [Crawler] add domains blacklist	2018-09-28 16:29:09 +02:00
Terrtia	82e6df4b94	chg: [Crawler] domains stats + logs + clean	2018-09-28 15:23:27 +02:00
Terrtia	ecb2857151	chg: [Crawler] css + limit splash RAM	2018-09-27 16:47:48 +02:00
Terrtia	e357dce59b	fix: [Crawler] detect splash connection to proxy error	2018-09-27 15:43:03 +02:00
Terrtia	04b9d9fc1d	chg: [Crawler] add docs	2018-09-27 11:14:29 +02:00
Terrtia	c49e871ba8	chg: [crawler] add infos	2018-09-26 16:34:27 +02:00
Terrtia	e4a5f66a10	Merge branch 'master' into onion_crawler	2018-09-24 16:35:04 +02:00
Terrtia	874824a589	fix: [Crawler] clean	2018-09-24 16:28:55 +02:00
Terrtia	8eca0e0778	fix: [Crawler] clean	2018-09-24 16:24:30 +02:00
Terrtia	50c81773e9	chg: [Crawler] add launcher and install	2018-09-24 16:23:14 +02:00
Terrtia	b575efcc46	fix: [pystemon] pystemon encoding	2018-09-21 14:16:06 +02:00
Terrtia	ead4dba0d8	fix: [MISP feeder] typo filename-config name	2018-09-21 11:02:08 +02:00
Terrtia	6edc1ddbeb	chg: [HiddenServices] get domain related to other domains	2018-09-21 10:34:06 +02:00
Terrtia	5eadc2755f	chg: [Flask] add prefix in config to flask routes	2018-09-20 10:38:19 +02:00
Terrtia	2e763ba0aa	fix: [ModulesInformationV2] fix #244	2018-09-19 15:45:06 +02:00
Terrtia	5732d7b134	fix: [hashDecoded] range_type_json	2018-09-19 15:36:33 +02:00
Terrtia	ebad2af1ae	fix: [import_dir] fix #251, fix paste import ... if the date directory tree don't exist, he will be generated	2018-09-19 13:38:31 +02:00
Terrtia	653463a8c4	fix: [LAUNCH.sh] make pystemon optionnal ... AIL and Pystemon are independent, pystemon shouldn't be launch by default.	2018-09-19 09:52:27 +02:00
Terrtia	5b31b6e853	fix: [Crawler] save domain to crawl on splash error	2018-09-18 16:20:13 +02:00
Terrtia	f5b648d72a	pixelate paste screenshot	2018-09-18 11:03:40 +02:00
Terrtia	6aa1b99c2a	fix: [LAUNCH.sh] remove old decoders	2018-09-17 15:50:27 +02:00
Terrtia	0c63f2f24f	chg: [Crawler] catch server response	2018-09-17 15:35:06 +02:00
Alexandre Dulaunoy	a6c72a677d	Merge pull request #243 from kovacsbalu/docker-with-pystemon ... Docker build with pystemon	2018-09-12 21:50:11 +02:00
Terrtia	a94a0fe1a7	fix: [Install] add python requirements	2018-09-12 19:10:49 +02:00
Terrtia	204e996fc3	chg: [statistics] clean scripts	2018-09-12 11:21:11 +02:00
Terrtia	ea3d2c1977	Merge branch 'master' into statistic	2018-09-12 10:06:53 +02:00
Terrtia	6f0817365a	chg: [Crawler UI] display domain information	2018-09-12 09:55:49 +02:00
kovacsbalu	d19dd72f99	Install pystemon and start pystemon-feeder in docker	2018-09-03 16:01:39 +02:00
Terrtia	ca982e13e1	chg: [Crawled Domain] show crawled pastes by domain	2018-08-27 14:34:08 +02:00
Terrtia	823c33174a	Merge branch 'master' into onion_crawler	2018-08-27 11:20:51 +02:00
Terrtia	5c902640e9	Merge branch 'master' into onion_crawler	2018-08-27 11:05:25 +02:00
Terrtia	d42dd118a4	chg: [Domain crawled] add random screenshot	2018-08-27 11:02:39 +02:00
Terrtia	a586548047	Merge branch 'master' into base64	2018-08-24 10:35:28 +02:00
Terrtia	ced0b1e350	chg: [I2P] add default config	2018-08-24 10:24:03 +02:00
Terrtia	7e24943537	chg: [Crawler] crawler accept all kind of domains	2018-08-24 10:13:56 +02:00
Terrtia	e9580d6775	chg: [Crawler] change BDD, save i2p links	2018-08-21 15:54:53 +02:00
Terrtia	ed559d9f4a	chg: [Showpaste] add screenshot + improve onion db	2018-08-16 17:24:39 +02:00
Terrtia	6f2dbcb4e2	fix: [pystemon-feeder] fix #234	2018-08-13 10:22:20 +02:00
Terrtia	7652089433	chg: [Onion] change onion regex, fix crawler	2018-08-13 09:23:14 +02:00
Terrtia	8b1c10b38c	chg: [Onion] add onion splash crawler	2018-08-09 17:42:21 +02:00
Thirion Aurélien	000d5bf40a	fix: add miss config	2018-08-07 13:07:08 +02:00
Terrtia	d7545a46f0	chg: [Mixer] add submitted paste	2018-08-02 15:29:13 +02:00
Terrtia	449da63fd7	chg: [Keys] typo	2018-08-01 16:05:49 +02:00
Terrtia	67b0d973af	chg: [Keys] detect ssh2 private key	2018-08-01 16:04:06 +02:00
Terrtia	d0135c248d	add iban country statistic	2018-08-01 15:26:29 +02:00
Terrtia	940b96734a	Merge branch 'master' into statistic	2018-08-01 09:29:51 +02:00
Thirion Aurélien	f7f4422ff1	Merge pull request #232 from CIRCL/iban ... Add Iban detection	2018-08-01 08:57:48 +02:00
Terrtia	b5c1f281e6	fix: typo	2018-07-31 10:58:23 +02:00
Terrtia	61e98ed8fb	clean	2018-07-31 10:51:18 +02:00
Terrtia	e455d244d8	chg: [statistics] add domain statistics	2018-07-31 10:36:29 +02:00
Terrtia	5dcb4ebb58	chg: [statistics] add script to generate pie chart png	2018-07-30 16:36:34 +02:00
Terrtia	09fbc363f1	chg: [sglinjection Phone] add tld statistic, fix phone regex	2018-07-30 11:56:50 +02:00
Terrtia	c20e7d5ab4	chg: [Mail Credential] add tld statistic	2018-07-30 10:19:26 +02:00
Terrtia	39f54d4420	chg: [Mail] add country statistic	2018-07-30 09:21:22 +02:00
Terrtia	009f87ee10	[fix: [UI-hash] fix graph node zoom	2018-07-27 11:27:41 +02:00
Terrtia	153f9f38cc	chg: [BankAccount] regex	2018-07-27 10:20:03 +02:00
Terrtia	57b631233b	add: [BankAccount] iban detection	2018-07-26 15:31:58 +02:00
Terrtia	5ad5939bd5	chg: display last logs	2018-07-25 16:48:44 +02:00
Terrtia	03a37af5cf	chg: [Hash] add reprocess regex order	2018-07-25 10:47:36 +02:00
Terrtia	8beb63b6a8	fix: [Queues] module with no subscriber	2018-07-24 15:57:18 +02:00
Terrtia	8c97c70a3f	Merge branch 'master' into base64	2018-07-24 15:21:32 +02:00
Terrtia	4b8c26dec8	fix: [hive auto export] fix #221	2018-07-24 14:46:35 +02:00
Terrtia	54e8750c30	fix: [LAUNCH] script checking	2018-07-24 12:18:41 +02:00
Terrtia	d4f0c14620	fix: [LAUNCH] use envar, add options (launch all, flask, killall)	2018-07-24 10:41:10 +02:00
Terrtia	9c03c40482	chg: [UI-Hash] add decoder pie	2018-07-23 11:11:52 +02:00
Terrtia	1fbd4109c2	chg: [UI-Hash] add decoded type on graph	2018-07-20 15:48:56 +02:00
Terrtia	6d6de3ae54	fix: remove experimental modules	2018-07-20 10:38:04 +02:00
Terrtia	bd5f83f0eb	chg: refractor base64 encoded to hash	2018-07-20 10:32:52 +02:00
Terrtia	9a8e37fb0b	fix: decoder bdd	2018-07-20 09:43:09 +02:00
Terrtia	0e67b56906	add: Decoder module	2018-07-19 16:52:09 +02:00
Terrtia	ad26f016e3	add: Decoder Module, decode binary, hex and base64	2018-07-19 16:50:42 +02:00
Terrtia	280e9d8af8	fix: Binary decoder base2 string to bytes	2018-07-18 13:47:06 +02:00
Terrtia	e25dd52d64	add: Binary decoder	2018-07-18 11:45:19 +02:00
Terrtia	0cac3146f1	fix: remove hardcoded mail url	2018-07-17 15:11:25 +02:00
Terrtia	fa38434e04	add: new mail body and starttls	2018-07-16 15:51:37 +02:00
Terrtia	8b245d17d9	fix: add LibInjection to LAUNCH.sh	2018-07-16 15:45:55 +02:00
Terrtia	6f69da0c0d	fix: paste submit css	2018-07-16 10:32:41 +02:00
Terrtia	fce324e370	fix: release tiemeout alarm, chg:icon graph node	2018-07-13 11:54:14 +02:00
Terrtia	87b7facba8	add: base 64 node graph	2018-07-12 17:07:17 +02:00
bui	bd5c42caeb	add LibInjection Module	2018-07-04 14:28:10 +02:00
bui	362f72fe43	initial LibInjection module	2018-07-04 14:27:54 +02:00
Thirion Aurélien	f8a88cecec	fix: hive_feeder hiveApi declaration	2018-07-02 15:23:21 +02:00
Terrtia	44524b25a4	fix submit paste	2018-06-29 15:30:01 +02:00
Terrtia	a0a639276c	fix submit paste	2018-06-29 15:29:02 +02:00
Terrtia	98d0235fc3	Merge branch 'master' into base64	2018-06-29 14:57:46 +02:00
Terrtia	f5bda70e82	Merge branch 'master' of https://github.com/CIRCL/AIL-framework	2018-06-29 13:51:08 +02:00
Terrtia	72146f938e	change: feeder name separator	2018-06-29 13:50:41 +02:00
Thirion Aurélien	c41a165a21	fix: file submission, content type	2018-06-29 12:08:46 +02:00
Terrtia	407aa14244	fix: catch file error in paste submission	2018-06-29 11:32:52 +02:00
Terrtia	16c700ddb4	new: add base64 display on paste + base64 statistics	2018-06-29 10:02:29 +02:00
Terrtia	225776c69c	fix: tp and fp, errors messages on paste submission	2018-06-20 10:02:26 +02:00
Terrtia	6a89fac309	fix hive and misp error	2018-06-19 16:39:49 +02:00
Terrtia	af9db53171	add tag fp and tp	2018-06-19 15:09:26 +02:00
Terrtia	86c2cfa04b	update ail-leak object	2018-06-18 17:23:41 +02:00
Terrtia	634bf6e552	add img + fix upload api	2018-06-18 17:16:22 +02:00
Terrtia	704e6f0ad1	change default submit tag, event-path map, publish events	2018-06-18 13:58:31 +02:00
Terrtia	aee1b50ef1	fix duplicate export attribute + tag export whitelist	2018-06-15 17:25:43 +02:00
Terrtia	31fc33ab03	misp event creation, auto and manual + the hive alert auto creation, manual case creation	2018-06-14 16:51:06 +02:00
Terrtia	8522d50308	modify navbar + decompress file password	2018-06-11 10:21:11 +02:00
Terrtia	c64588f60e	add feature, user can submit paste on the web interface	2018-06-08 16:49:20 +02:00
Terrtia	6143bc3dce	submit users input paste	2018-06-05 16:58:04 +02:00
Terrtia	6287981668	clean	2018-06-01 09:49:26 +02:00
Terrtia	884d655b0c	taxonomies and galaxies update via LAUNCH	2018-05-31 13:53:29 +02:00
Terrtia	93dcdf4f23	fix duplicate	2018-05-31 09:01:47 +02:00
Terrtia	0d164b6f4c	add tag js and css	2018-05-16 14:48:15 +02:00
Terrtia	b1b7d33c96	tags	2018-05-16 14:39:01 +02:00
Terrtia	f66a528bc2	fix Duplicate, save list of duplicates on disk + prevent empty hash creation	2018-05-15 23:28:47 +02:00
Terrtia	225fe76c96	fix duplicate + get raw paste	2018-05-11 16:07:30 +02:00
Terrtia	d7ba5533be	fix duplicate + fix issue #200	2018-05-11 13:25:45 +02:00
Terrtia	8599d7142d	python 3 upgrade instructions	2018-05-09 13:32:25 +02:00
Terrtia	142c1dc9b0	clean	2018-05-09 13:03:46 +02:00
Terrtia	3841c159a0	change leveldb to ardb	2018-05-07 14:50:40 +02:00
Terrtia	c17b9402f7	clean	2018-05-04 14:25:47 +02:00
Terrtia	96a3df42bc	decode with redis connection	2018-05-04 13:53:29 +02:00
Terrtia	ae8f575879	use feeder name	2018-05-03 16:32:25 +02:00
Terrtia	a900d5f08d	add python 2 to 3 bash + fix Duplicate + clean	2018-05-03 16:21:33 +02:00
Terrtia	c52caebe7c	improve Duplicate + tlsh + add time out handler + debug + clean	2018-05-02 17:07:10 +02:00
Terrtia	92977201fd	create json metadata for base64 module	2018-04-27 15:46:41 +02:00
Terrtia	c3428df5dc	add apiKeys module	2018-04-26 14:42:39 +02:00
Terrtia	29c368250e	create new dashbord alert	2018-04-24 17:03:10 +02:00
Terrtia	8738b7cf75	fix track term and avoid duplicate mail address	2018-04-24 16:44:37 +02:00
Terrtia	a7bd01ed9a	Merge branch 'python3' of https://github.com/CIRCL/AIL-framework into python3	2018-04-23 16:22:51 +02:00
Terrtia	4e55f6ee90	fix Onion module	2018-04-23 16:22:23 +02:00
Thirion Aurélien	ef3c335b62	Delete config.cfg.backup	2018-04-23 14:54:23 +02:00
Terrtia	f93fe9aeb2	add bitcoin module	2018-04-23 14:51:53 +02:00
Terrtia	3fc48db903	add Base64 module	2018-04-20 10:48:44 +02:00
Terrtia	41eb250278	fix python 3 compqtibility issues	2018-04-20 10:42:19 +02:00
Terrtia	8571a86344	frontend python 3.5 upgrade	2018-04-17 16:06:32 +02:00
Terrtia	19c90339e0	clean	2018-04-17 09:43:45 +02:00
Terrtia	d0261233aa	fix ModulesInnformationV2	2018-04-17 09:37:58 +02:00
Terrtia	15ef02fe53	fix python3 types issue	2018-04-16 17:00:44 +02:00
Terrtia	9e07a0ebdb	fix python3 types issue	2018-04-16 15:04:22 +02:00
Terrtia	9c82dd90ec	python 3 backend upgrade	2018-04-16 14:50:04 +02:00
Terrtia	3395b16873	fix incoming encoding file issue	2018-04-13 09:17:56 +02:00
Terrtia	347588fdec	python3.5 Mixer and Global module	2018-04-12 17:06:57 +02:00
Terrtia	4386cbbd34	fix travis	2018-04-11 10:48:22 +02:00
Terrtia	14583f702f	create first test	2018-04-11 10:14:33 +02:00
Terrtia	e9b88b0656	Keys module improvement	2018-04-05 11:40:34 +02:00
CD Nguyen	96259c53bc	refactor NotificationHelper code	2018-04-04 09:41:13 +02:00
CD Nguyen	e9e3f17363	support non TSL SMTP servers	2018-03-30 11:35:37 +02:00
Philipp Schmied	29608ab29b	Fixed e-mail notificatins for sets and regexes	2018-02-28 11:31:16 +01:00
Philipp Schmied	fd329aaba9	config.cfg.sample: Merged overwritten changes from upstream	2018-02-27 16:16:57 +01:00
Philipp Schmied	5b1f0b0212	Implemented email notifications (bin: config.cfg additions, email sending via analyzer scripts; var: Changes to add notifications via terms management); terms_management: Fixed click handlers not being added to all tracked terms.	2018-02-27 15:12:02 +01:00
Alexandre Dulaunoy	abdf41fe23	Merge pull request #164 from mokaddem/showDiff ... Show differences between duplicates	2018-02-21 22:25:16 +01:00
Alexandre Dulaunoy	0263b25da8	Merge pull request #163 from mokaddem/iss160 ... Moving more to config	2018-02-21 22:24:50 +01:00
Sami Mokaddem	cbb19f95e4	fix: asciimatics introduced the '_required_height' property in the Label class. Just updating here to mirror the change	2018-02-16 14:35:09 +01:00
Sami Mokaddem	3e117195a2	fix: Added a backoff time if an IOError happens	2018-01-15 17:45:13 +01:00
Sami Mokaddem	9a66a6111b	Fixing no such file	2018-01-15 17:10:03 +01:00
Sami Mokaddem	c39e537cbe	Merge branch 'iss160' into showDiff ... Preemptively preventing merge-conflict with branch iss160 (moving to configuration)	2017-12-12 11:33:22 +01:00
Sami Mokaddem	05b22e8963	typo: added word + deleted trailing spaces	2017-12-12 11:19:37 +01:00
Sami Mokaddem	ef1bd16a72	typo: added word + deleted trailing spaces	2017-12-12 11:16:40 +01:00
Sami Mokaddem	304213cfa4	update: Possibility to change the authorized max line length in the config for the diff of duplicates	2017-12-12 11:00:26 +01:00
Sami Mokaddem	860acbf7cf	fix: Renamed Credentials into Credential sectiont title ... config change: By default, zmq listen to both local and CRF	2017-12-12 09:29:15 +01:00
Sami Mokaddem	9d0d0b4303	update: Moved filtering operation (thresholds, number of matching in the ... categ file) in the configuration file. It permits to better control the flow of pastes. Also set default mixer duplicate filtering to 3 (Do not filter)	2017-12-11 17:28:34 +01:00
Fabien Mathey	3105192312	Allow for multiple entries like 1 2 3 4 5 ... Should replace the need to type 1<Enter>2<Enter> etc. and allows for 1 2 3 4 5<Enter>	2017-11-28 15:15:12 +01:00
Alexandre Dulaunoy	71922f9def	add: Python script to test if the ZMQ feed works as expected	2017-11-28 09:46:06 +01:00
Sami Mokaddem	83e082e62a	update: removed useless comments	2017-11-24 08:57:41 +01:00
Sami Mokaddem	a0d07b1098	feature: try to send data to MISP if keys are provided, else do nothing	2017-11-23 13:47:02 +01:00
Sami Mokaddem	2a967c4d92	update/feature: Max number of duplicate push to MISP + duplicate are pushed as attachment	2017-11-23 07:13:44 +01:00
Sami Mokaddem	805171a8a5	Added a seemingly working version of ailleak to misp	2017-11-20 14:57:25 +01:00
Sami Mokaddem	e0b188c069	Better support of python3 in Helper.py	2017-11-20 14:55:36 +01:00
Sami Mokaddem	1e76e10c01	Support of add_object to a MISP instance	2017-11-20 12:12:53 +01:00
Sami Mokaddem	6f0227a78a	Starting support of python3	2017-11-20 12:12:06 +01:00
Sami Mokaddem	ee7759be91	Updated alertHandler and ailleakObject	2017-11-16 11:18:13 +01:00
Sami Mokaddem	5c7c07fbea	Merge branch 'launcher_update' into misp_obj	2017-11-16 10:09:05 +01:00
Sami Mokaddem	45a9897020	Added draft support of MISP ail-leak object	2017-11-16 09:52:37 +01:00
Sami Mokaddem	652b00f4b2	Renamed BrowseWarningPaste into alertHandler	2017-11-15 16:15:43 +01:00
Sami Mokaddem	ec7d2d49b2	Added seconds args in import_dir	2017-11-15 16:03:42 +01:00
Sami Mokaddem	13f1c676d6	Added '_AIL' to the end of screen session name to better check an kills sessions. Also, gracefully close redis server + Check 1 time if redis-servers are running.	2017-09-26 16:17:06 +02:00
=	a8dc9231c6	Typo2..	2017-08-24 16:43:42 +02:00
=	b2d4dd4a1f	Typo	2017-08-24 16:43:05 +02:00
=	74ff401e1d	Fixed bug: No CR between indexes in all_index.txt file	2017-08-24 16:41:30 +02:00
Alexandre Dulaunoy	bb128a5e94	Merge pull request #138 from mokaddem/import_dir_update ... Improvement of import_dir	2017-08-24 09:40:52 +02:00
=	429cc96d2a	Improvement of import_dir.py. Added feed name parameter, send the correct path (without potential ../) and gzipEncode it if needed.	2017-08-23 15:10:26 +02:00
=	e7c66d1aac	Updated individual launch scripts	2017-08-23 12:28:12 +02:00
=	d632335760	Automatically start redis_levelDB with the correct date	2017-08-23 10:16:56 +02:00
=	64ec44085d	Dynamically choose correct port for LevelDB in script	2017-08-22 17:52:15 +02:00
=	66d60a1f01	Added LevelDB Curve for 2017 (3017, used for backward compatibility)	2017-08-21 13:54:44 +02:00
=	92adea38d0	Semi-automated directory creation with date as name in LAUNCH and installing_deps	2017-08-21 13:48:02 +02:00
=	bc35747f00	Updated KillAll in LAUNCH	2017-08-21 13:19:26 +02:00
Mokaddem	c3632c7d86	Added operation mode 3 for Mixer (do not filter) and added possibility to recurse through dir for import_dir	2017-08-03 00:34:53 +02:00
Mokaddem	cbb2909156	Added comments	2017-07-20 10:24:48 +02:00
Mokaddem	05417f405c	Finished basic link web interface + solved some bugs	2017-07-20 10:04:30 +02:00
Mokaddem	ef741f85c0	Added similarity checks + started link webInterface	2017-07-19 11:52:06 +02:00
Mokaddem	882ab88b00	Draft module + web link	2017-07-18 16:57:15 +02:00
Mokaddem	9558aafe67	Prevent the loss of the 1 message in import_dir.py	2017-07-17 16:14:27 +02:00
Mokaddem	4d59871f00	Merged with previous changes for PR	2017-07-17 13:48:16 +02:00
Alexandre Dulaunoy	4986352099	Merge pull request #120 from PaulSec/master ... Added Python Phonenumbers module to parse numbers and retrieves stats…	2017-06-08 07:05:52 +02:00
Alexandre Dulaunoy	446a97c47a	Merge pull request #115 from mokaddem/regexInTerms ... New features for terms frequency	2017-06-08 07:04:17 +02:00