672 commits

Author	SHA1	Message	Date
Terrtia	ea3d2c1977	Merge branch 'master' into statistic	2018-09-12 10:06:53 +02:00
Terrtia	6f0817365a	chg: [Crawler UI] display domain information	2018-09-12 09:55:49 +02:00
kovacsbalu	d19dd72f99	Install pystemon and start pystemon-feeder in docker	2018-09-03 16:01:39 +02:00
Terrtia	ca982e13e1	chg: [Crawled Domain] show crawled pastes by domain	2018-08-27 14:34:08 +02:00
Terrtia	823c33174a	Merge branch 'master' into onion_crawler	2018-08-27 11:20:51 +02:00
Terrtia	5c902640e9	Merge branch 'master' into onion_crawler	2018-08-27 11:05:25 +02:00
Terrtia	d42dd118a4	chg: [Domain crawled] add random screenshot	2018-08-27 11:02:39 +02:00
Terrtia	a586548047	Merge branch 'master' into base64	2018-08-24 10:35:28 +02:00
Terrtia	ced0b1e350	chg: [I2P] add default config	2018-08-24 10:24:03 +02:00
Terrtia	7e24943537	chg: [Crawler] crawler accept all kind of domains	2018-08-24 10:13:56 +02:00
Terrtia	e9580d6775	chg: [Crawler] change BDD, save i2p links	2018-08-21 15:54:53 +02:00
Terrtia	ed559d9f4a	chg: [Showpaste] add screenshot + improve onion db	2018-08-16 17:24:39 +02:00
Terrtia	6f2dbcb4e2	fix: [pystemon-feeder] fix #234	2018-08-13 10:22:20 +02:00
Terrtia	7652089433	chg: [Onion] change onion regex, fix crawler	2018-08-13 09:23:14 +02:00
Terrtia	8b1c10b38c	chg: [Onion] add onion splash crawler	2018-08-09 17:42:21 +02:00
Thirion Aurélien	000d5bf40a	fix: add miss config	2018-08-07 13:07:08 +02:00
Terrtia	d7545a46f0	chg: [Mixer] add submitted paste	2018-08-02 15:29:13 +02:00
Terrtia	449da63fd7	chg: [Keys] typo	2018-08-01 16:05:49 +02:00
Terrtia	67b0d973af	chg: [Keys] detect ssh2 private key	2018-08-01 16:04:06 +02:00
Terrtia	d0135c248d	add iban country statistic	2018-08-01 15:26:29 +02:00
Terrtia	940b96734a	Merge branch 'master' into statistic	2018-08-01 09:29:51 +02:00
Thirion Aurélien	f7f4422ff1	Merge pull request #232 from CIRCL/iban ... Add Iban detection	2018-08-01 08:57:48 +02:00
Terrtia	b5c1f281e6	fix: typo	2018-07-31 10:58:23 +02:00
Terrtia	61e98ed8fb	clean	2018-07-31 10:51:18 +02:00
Terrtia	e455d244d8	chg: [statistics] add domain statistics	2018-07-31 10:36:29 +02:00
Terrtia	5dcb4ebb58	chg: [statistics] add script to generate pie chart png	2018-07-30 16:36:34 +02:00
Terrtia	09fbc363f1	chg: [sglinjection Phone] add tld statistic, fix phone regex	2018-07-30 11:56:50 +02:00
Terrtia	c20e7d5ab4	chg: [Mail Credential] add tld statistic	2018-07-30 10:19:26 +02:00
Terrtia	39f54d4420	chg: [Mail] add country statistic	2018-07-30 09:21:22 +02:00
Terrtia	009f87ee10	[fix: [UI-hash] fix graph node zoom	2018-07-27 11:27:41 +02:00
Terrtia	153f9f38cc	chg: [BankAccount] regex	2018-07-27 10:20:03 +02:00
Terrtia	57b631233b	add: [BankAccount] iban detection	2018-07-26 15:31:58 +02:00
Terrtia	5ad5939bd5	chg: display last logs	2018-07-25 16:48:44 +02:00
Terrtia	03a37af5cf	chg: [Hash] add reprocess regex order	2018-07-25 10:47:36 +02:00
Terrtia	8beb63b6a8	fix: [Queues] module with no subscriber	2018-07-24 15:57:18 +02:00
Terrtia	8c97c70a3f	Merge branch 'master' into base64	2018-07-24 15:21:32 +02:00
Terrtia	4b8c26dec8	fix: [hive auto export] fix #221	2018-07-24 14:46:35 +02:00
Terrtia	54e8750c30	fix: [LAUNCH] script checking	2018-07-24 12:18:41 +02:00
Terrtia	d4f0c14620	fix: [LAUNCH] use envar, add options (launch all, flask, killall)	2018-07-24 10:41:10 +02:00
Terrtia	9c03c40482	chg: [UI-Hash] add decoder pie	2018-07-23 11:11:52 +02:00
Terrtia	1fbd4109c2	chg: [UI-Hash] add decoded type on graph	2018-07-20 15:48:56 +02:00
Terrtia	6d6de3ae54	fix: remove experimental modules	2018-07-20 10:38:04 +02:00
Terrtia	bd5f83f0eb	chg: refractor base64 encoded to hash	2018-07-20 10:32:52 +02:00
Terrtia	9a8e37fb0b	fix: decoder bdd	2018-07-20 09:43:09 +02:00
Terrtia	0e67b56906	add: Decoder module	2018-07-19 16:52:09 +02:00
Terrtia	ad26f016e3	add: Decoder Module, decode binary, hex and base64	2018-07-19 16:50:42 +02:00
Terrtia	280e9d8af8	fix: Binary decoder base2 string to bytes	2018-07-18 13:47:06 +02:00
Terrtia	e25dd52d64	add: Binary decoder	2018-07-18 11:45:19 +02:00
Terrtia	0cac3146f1	fix: remove hardcoded mail url	2018-07-17 15:11:25 +02:00
Terrtia	fa38434e04	add: new mail body and starttls	2018-07-16 15:51:37 +02:00
Terrtia	8b245d17d9	fix: add LibInjection to LAUNCH.sh	2018-07-16 15:45:55 +02:00
Terrtia	6f69da0c0d	fix: paste submit css	2018-07-16 10:32:41 +02:00
Terrtia	fce324e370	fix: release tiemeout alarm, chg:icon graph node	2018-07-13 11:54:14 +02:00
Terrtia	87b7facba8	add: base 64 node graph	2018-07-12 17:07:17 +02:00
bui	bd5c42caeb	add LibInjection Module	2018-07-04 14:28:10 +02:00
bui	362f72fe43	initial LibInjection module	2018-07-04 14:27:54 +02:00
Thirion Aurélien	f8a88cecec	fix: hive_feeder hiveApi declaration	2018-07-02 15:23:21 +02:00
Terrtia	44524b25a4	fix submit paste	2018-06-29 15:30:01 +02:00
Terrtia	a0a639276c	fix submit paste	2018-06-29 15:29:02 +02:00
Terrtia	98d0235fc3	Merge branch 'master' into base64	2018-06-29 14:57:46 +02:00
Terrtia	f5bda70e82	Merge branch 'master' of https://github.com/CIRCL/AIL-framework	2018-06-29 13:51:08 +02:00
Terrtia	72146f938e	change: feeder name separator	2018-06-29 13:50:41 +02:00
Thirion Aurélien	c41a165a21	fix: file submission, content type	2018-06-29 12:08:46 +02:00
Terrtia	407aa14244	fix: catch file error in paste submission	2018-06-29 11:32:52 +02:00
Terrtia	16c700ddb4	new: add base64 display on paste + base64 statistics	2018-06-29 10:02:29 +02:00
Terrtia	225776c69c	fix: tp and fp, errors messages on paste submission	2018-06-20 10:02:26 +02:00
Terrtia	6a89fac309	fix hive and misp error	2018-06-19 16:39:49 +02:00
Terrtia	af9db53171	add tag fp and tp	2018-06-19 15:09:26 +02:00
Terrtia	86c2cfa04b	update ail-leak object	2018-06-18 17:23:41 +02:00
Terrtia	634bf6e552	add img + fix upload api	2018-06-18 17:16:22 +02:00
Terrtia	704e6f0ad1	change default submit tag, event-path map, publish events	2018-06-18 13:58:31 +02:00
Terrtia	aee1b50ef1	fix duplicate export attribute + tag export whitelist	2018-06-15 17:25:43 +02:00
Terrtia	31fc33ab03	misp event creation, auto and manual + the hive alert auto creation, manual case creation	2018-06-14 16:51:06 +02:00
Terrtia	8522d50308	modify navbar + decompress file password	2018-06-11 10:21:11 +02:00
Terrtia	c64588f60e	add feature, user can submit paste on the web interface	2018-06-08 16:49:20 +02:00
Terrtia	6143bc3dce	submit users input paste	2018-06-05 16:58:04 +02:00
Terrtia	6287981668	clean	2018-06-01 09:49:26 +02:00
Terrtia	884d655b0c	taxonomies and galaxies update via LAUNCH	2018-05-31 13:53:29 +02:00
Terrtia	93dcdf4f23	fix duplicate	2018-05-31 09:01:47 +02:00
Terrtia	0d164b6f4c	add tag js and css	2018-05-16 14:48:15 +02:00
Terrtia	b1b7d33c96	tags	2018-05-16 14:39:01 +02:00
Terrtia	f66a528bc2	fix Duplicate, save list of duplicates on disk + prevent empty hash creation	2018-05-15 23:28:47 +02:00
Terrtia	225fe76c96	fix duplicate + get raw paste	2018-05-11 16:07:30 +02:00
Terrtia	d7ba5533be	fix duplicate + fix issue #200	2018-05-11 13:25:45 +02:00
Terrtia	8599d7142d	python 3 upgrade instructions	2018-05-09 13:32:25 +02:00
Terrtia	142c1dc9b0	clean	2018-05-09 13:03:46 +02:00
Terrtia	3841c159a0	change leveldb to ardb	2018-05-07 14:50:40 +02:00
Terrtia	c17b9402f7	clean	2018-05-04 14:25:47 +02:00
Terrtia	96a3df42bc	decode with redis connection	2018-05-04 13:53:29 +02:00
Terrtia	ae8f575879	use feeder name	2018-05-03 16:32:25 +02:00
Terrtia	a900d5f08d	add python 2 to 3 bash + fix Duplicate + clean	2018-05-03 16:21:33 +02:00
Terrtia	c52caebe7c	improve Duplicate + tlsh + add time out handler + debug + clean	2018-05-02 17:07:10 +02:00
Terrtia	92977201fd	create json metadata for base64 module	2018-04-27 15:46:41 +02:00
Terrtia	c3428df5dc	add apiKeys module	2018-04-26 14:42:39 +02:00
Terrtia	29c368250e	create new dashbord alert	2018-04-24 17:03:10 +02:00
Terrtia	8738b7cf75	fix track term and avoid duplicate mail address	2018-04-24 16:44:37 +02:00
Terrtia	a7bd01ed9a	Merge branch 'python3' of https://github.com/CIRCL/AIL-framework into python3	2018-04-23 16:22:51 +02:00
Terrtia	4e55f6ee90	fix Onion module	2018-04-23 16:22:23 +02:00
Thirion Aurélien	ef3c335b62	Delete config.cfg.backup	2018-04-23 14:54:23 +02:00
Terrtia	f93fe9aeb2	add bitcoin module	2018-04-23 14:51:53 +02:00
Terrtia	3fc48db903	add Base64 module	2018-04-20 10:48:44 +02:00
Terrtia	41eb250278	fix python 3 compqtibility issues	2018-04-20 10:42:19 +02:00
Terrtia	8571a86344	frontend python 3.5 upgrade	2018-04-17 16:06:32 +02:00
Terrtia	19c90339e0	clean	2018-04-17 09:43:45 +02:00
Terrtia	d0261233aa	fix ModulesInnformationV2	2018-04-17 09:37:58 +02:00
Terrtia	15ef02fe53	fix python3 types issue	2018-04-16 17:00:44 +02:00
Terrtia	9e07a0ebdb	fix python3 types issue	2018-04-16 15:04:22 +02:00
Terrtia	9c82dd90ec	python 3 backend upgrade	2018-04-16 14:50:04 +02:00
Terrtia	3395b16873	fix incoming encoding file issue	2018-04-13 09:17:56 +02:00
Terrtia	347588fdec	python3.5 Mixer and Global module	2018-04-12 17:06:57 +02:00
Terrtia	4386cbbd34	fix travis	2018-04-11 10:48:22 +02:00
Terrtia	14583f702f	create first test	2018-04-11 10:14:33 +02:00
Terrtia	e9b88b0656	Keys module improvement	2018-04-05 11:40:34 +02:00
CD Nguyen	96259c53bc	refactor NotificationHelper code	2018-04-04 09:41:13 +02:00
CD Nguyen	e9e3f17363	support non TSL SMTP servers	2018-03-30 11:35:37 +02:00
Philipp Schmied	29608ab29b	Fixed e-mail notificatins for sets and regexes	2018-02-28 11:31:16 +01:00
Philipp Schmied	fd329aaba9	config.cfg.sample: Merged overwritten changes from upstream	2018-02-27 16:16:57 +01:00
Philipp Schmied	5b1f0b0212	Implemented email notifications (bin: config.cfg additions, email sending via analyzer scripts; var: Changes to add notifications via terms management); terms_management: Fixed click handlers not being added to all tracked terms.	2018-02-27 15:12:02 +01:00
Alexandre Dulaunoy	abdf41fe23	Merge pull request #164 from mokaddem/showDiff ... Show differences between duplicates	2018-02-21 22:25:16 +01:00
Alexandre Dulaunoy	0263b25da8	Merge pull request #163 from mokaddem/iss160 ... Moving more to config	2018-02-21 22:24:50 +01:00
Sami Mokaddem	cbb19f95e4	fix: asciimatics introduced the '_required_height' property in the Label class. Just updating here to mirror the change	2018-02-16 14:35:09 +01:00
Sami Mokaddem	3e117195a2	fix: Added a backoff time if an IOError happens	2018-01-15 17:45:13 +01:00
Sami Mokaddem	9a66a6111b	Fixing no such file	2018-01-15 17:10:03 +01:00
Sami Mokaddem	c39e537cbe	Merge branch 'iss160' into showDiff ... Preemptively preventing merge-conflict with branch iss160 (moving to configuration)	2017-12-12 11:33:22 +01:00
Sami Mokaddem	05b22e8963	typo: added word + deleted trailing spaces	2017-12-12 11:19:37 +01:00
Sami Mokaddem	ef1bd16a72	typo: added word + deleted trailing spaces	2017-12-12 11:16:40 +01:00
Sami Mokaddem	304213cfa4	update: Possibility to change the authorized max line length in the config for the diff of duplicates	2017-12-12 11:00:26 +01:00
Sami Mokaddem	860acbf7cf	fix: Renamed Credentials into Credential sectiont title ... config change: By default, zmq listen to both local and CRF	2017-12-12 09:29:15 +01:00
Sami Mokaddem	9d0d0b4303	update: Moved filtering operation (thresholds, number of matching in the ... categ file) in the configuration file. It permits to better control the flow of pastes. Also set default mixer duplicate filtering to 3 (Do not filter)	2017-12-11 17:28:34 +01:00
Fabien Mathey	3105192312	Allow for multiple entries like 1 2 3 4 5 ... Should replace the need to type 1<Enter>2<Enter> etc. and allows for 1 2 3 4 5<Enter>	2017-11-28 15:15:12 +01:00
Alexandre Dulaunoy	71922f9def	add: Python script to test if the ZMQ feed works as expected	2017-11-28 09:46:06 +01:00
Sami Mokaddem	83e082e62a	update: removed useless comments	2017-11-24 08:57:41 +01:00
Sami Mokaddem	a0d07b1098	feature: try to send data to MISP if keys are provided, else do nothing	2017-11-23 13:47:02 +01:00
Sami Mokaddem	2a967c4d92	update/feature: Max number of duplicate push to MISP + duplicate are pushed as attachment	2017-11-23 07:13:44 +01:00
Sami Mokaddem	805171a8a5	Added a seemingly working version of ailleak to misp	2017-11-20 14:57:25 +01:00
Sami Mokaddem	e0b188c069	Better support of python3 in Helper.py	2017-11-20 14:55:36 +01:00
Sami Mokaddem	1e76e10c01	Support of add_object to a MISP instance	2017-11-20 12:12:53 +01:00
Sami Mokaddem	6f0227a78a	Starting support of python3	2017-11-20 12:12:06 +01:00
Sami Mokaddem	ee7759be91	Updated alertHandler and ailleakObject	2017-11-16 11:18:13 +01:00
Sami Mokaddem	5c7c07fbea	Merge branch 'launcher_update' into misp_obj	2017-11-16 10:09:05 +01:00
Sami Mokaddem	45a9897020	Added draft support of MISP ail-leak object	2017-11-16 09:52:37 +01:00
Sami Mokaddem	652b00f4b2	Renamed BrowseWarningPaste into alertHandler	2017-11-15 16:15:43 +01:00
Sami Mokaddem	ec7d2d49b2	Added seconds args in import_dir	2017-11-15 16:03:42 +01:00
Sami Mokaddem	13f1c676d6	Added '_AIL' to the end of screen session name to better check an kills sessions. Also, gracefully close redis server + Check 1 time if redis-servers are running.	2017-09-26 16:17:06 +02:00
=	a8dc9231c6	Typo2..	2017-08-24 16:43:42 +02:00
=	b2d4dd4a1f	Typo	2017-08-24 16:43:05 +02:00
=	74ff401e1d	Fixed bug: No CR between indexes in all_index.txt file	2017-08-24 16:41:30 +02:00
Alexandre Dulaunoy	bb128a5e94	Merge pull request #138 from mokaddem/import_dir_update ... Improvement of import_dir	2017-08-24 09:40:52 +02:00
=	429cc96d2a	Improvement of import_dir.py. Added feed name parameter, send the correct path (without potential ../) and gzipEncode it if needed.	2017-08-23 15:10:26 +02:00
=	e7c66d1aac	Updated individual launch scripts	2017-08-23 12:28:12 +02:00
=	d632335760	Automatically start redis_levelDB with the correct date	2017-08-23 10:16:56 +02:00
=	64ec44085d	Dynamically choose correct port for LevelDB in script	2017-08-22 17:52:15 +02:00
=	66d60a1f01	Added LevelDB Curve for 2017 (3017, used for backward compatibility)	2017-08-21 13:54:44 +02:00
=	92adea38d0	Semi-automated directory creation with date as name in LAUNCH and installing_deps	2017-08-21 13:48:02 +02:00
=	bc35747f00	Updated KillAll in LAUNCH	2017-08-21 13:19:26 +02:00
Mokaddem	c3632c7d86	Added operation mode 3 for Mixer (do not filter) and added possibility to recurse through dir for import_dir	2017-08-03 00:34:53 +02:00
Mokaddem	cbb2909156	Added comments	2017-07-20 10:24:48 +02:00
Mokaddem	05417f405c	Finished basic link web interface + solved some bugs	2017-07-20 10:04:30 +02:00
Mokaddem	ef741f85c0	Added similarity checks + started link webInterface	2017-07-19 11:52:06 +02:00
Mokaddem	882ab88b00	Draft module + web link	2017-07-18 16:57:15 +02:00
Mokaddem	9558aafe67	Prevent the loss of the 1 message in import_dir.py	2017-07-17 16:14:27 +02:00
Mokaddem	4d59871f00	Merged with previous changes for PR	2017-07-17 13:48:16 +02:00
Alexandre Dulaunoy	4986352099	Merge pull request #120 from PaulSec/master ... Added Python Phonenumbers module to parse numbers and retrieves stats…	2017-06-08 07:05:52 +02:00
Alexandre Dulaunoy	446a97c47a	Merge pull request #115 from mokaddem/regexInTerms ... New features for terms frequency	2017-06-08 07:04:17 +02:00
Mokaddem	3a4dcd691d	Improved description of modules inside the scripts	2017-05-09 11:13:16 +02:00
Paul A	747fff1df1	Added Python Phonenumbers module to parse numbers and retrieves stats from countrycode	2017-05-03 14:25:18 +02:00
Mokaddem	3ea0924120	Improved phone regex to handle end with 4 digits and presence of parenthesis.	2017-04-26 12:41:23 +02:00
Mokaddem	b60e164380	Added support of onion in browseWarningPaste	2017-04-26 11:47:02 +02:00
Mokaddem	318a557c56	Added possibility to keep a backup of the old config + check if config is valid before starting scripts.	2017-04-25 09:32:05 +02:00
Mokaddem	72c1d903eb	Merge branch 'newIndexer' into auto-conf	2017-04-25 08:14:31 +02:00
Mokaddem	eccc8fc88b	modified check_size timeout + utilisation of shutil instead of os.system()	2017-04-19 16:37:04 +02:00
Mokaddem	da081bae42	Merge branch 'regexInTerms' into production	2017-04-18 16:59:40 +02:00
Mokaddem	2d907ae784	Merge branch 'newIndexer' into auto-conf	2017-04-18 16:20:00 +02:00
Mokaddem	b0f1cf84b7	remplaced usage of global variable by a function param	2017-04-18 16:02:22 +02:00
Mokaddem	6e8509a0ba	Removed wierd line	2017-04-18 15:57:07 +02:00
Mokaddem	c8baabd882	Updated TermsFrequency related modules + Fixed bugs (encoding, behaviors, ...)	2017-04-18 15:28:21 +02:00
Mokaddem	2da4c572c7	New feature in Terms: regex and set of words support (draft)	2017-03-28 17:42:44 +02:00
Mokaddem	6e80eb0954	Typo	2017-03-28 09:55:59 +02:00
Mokaddem	c79d4d65c8	Keep original section order while updating	2017-03-28 09:54:24 +02:00
Mokaddem	e225090e07	Added script that checks configuration and may auto-fix it + added missing field in config.cfg.sample	2017-03-15 16:36:51 +01:00
Mokaddem	be93af6f5a	Merge branch 'terms-improvements'	2017-03-15 14:55:44 +01:00
Mokaddem	3261eca836	Fixed all_index newline bug + added print of what's happening	2017-03-15 14:29:49 +01:00
Mokaddem	5b7d047622	Changed os.system by os.mkdir + fixed bug during rotation	2017-03-15 14:05:13 +01:00
Mokaddem	adc37ab50e	Indexer max size now in config	2017-03-15 12:14:41 +01:00
Mokaddem	4005b72180	Added backward support + renamed index with timestamp	2017-03-15 11:51:35 +01:00
Mokaddem	248469d61e	Indexer now create an index_dir when it became too large (search in index not updated yet)	2017-03-14 10:37:31 +01:00
Alexandre Dulaunoy	1540b719ed	Merge pull request #108 from mokaddem/newModuleInformation ... New version of ModuleInformation	2017-03-14 08:29:09 +01:00
Mokaddem	3c9cdac39a	Merge branch 'terms-improvements' into production	2017-02-28 16:51:39 +01:00
Mokaddem	dadc872f9f	Merge branch 'newModuleInformation' into production	2017-02-28 16:51:08 +01:00
Mokaddem	a2849c1bcd	Added a catch for psutils if process get stopped by user	2017-02-28 16:13:01 +01:00
Mokaddem	4071bf2e91	fixed a bug in manage_term (usage of loop.index0) + fixed bug in top_set where graph was not displaying the correct per_paste value	2017-02-28 15:01:48 +01:00
Mokaddem	a1c47d7a13	Merge branch 'production' into terms-improvements	2017-02-28 11:15:56 +01:00
Mokaddem	fd4feb374f	Merge branch 'newModuleInformation' into production	2017-02-28 11:12:06 +01:00
Mokaddem	75b4037519	Fixed bug - If we move the cursor from a line that has dissapear	2017-02-28 11:11:17 +01:00
Mokaddem	d6e02591af	Merge branch 'newModuleInformation' into production	2017-02-28 10:23:45 +01:00
Mokaddem	975c1a37d9	ModuleInformation create the all_modules files if missing	2017-02-28 10:23:22 +01:00
Mokaddem	ccdf256383	Added possibility to show or not in dashboard text that is logged	2017-02-28 10:21:27 +01:00
Mokaddem	39104dea5e	Merge branch 'bugfix_categ' into production	2017-02-28 09:56:25 +01:00
Mokaddem	26eb5fdff0	Fixed bug closing parenthesis in categ + changed behavior of sending to browseWarningPaste in Mail	2017-02-28 09:23:32 +01:00
Mokaddem	4a0dac1fae	Merge branch 'production' into terms-improvements	2017-02-15 16:31:22 +01:00
Mokaddem	a696ccf83b	Merge branch 'newModuleInformation' into production	2017-02-15 16:30:44 +01:00
Mokaddem	85a6f6fa63	Fixed a bug (str+float)	2017-02-15 16:30:31 +01:00
Mokaddem	25adae0104	Added possibility to select in top_terms the data per_paste or not (need module testing)	2017-02-15 16:29:02 +01:00
Mokaddem	e98e92b0d4	Merge branch 'fix-linkPaste-debug' into production	2017-02-15 13:43:03 +01:00
Mokaddem	c552aced42	Merge branch 'newModuleInformation' into production	2017-02-15 13:42:46 +01:00
Mokaddem	1b0d9d9507	Fixed str and int concat + Added sorting by queue name in queue not running	2017-02-15 13:42:21 +01:00
Mokaddem	74e0bc7acb	Merge branch 'newModuleInformation' into production	2017-02-15 11:22:50 +01:00
Mokaddem	7fc9ba18d7	Added support of ENTER for navigation	2017-02-15 11:01:11 +01:00
Mokaddem	b84194542d	Moved logging into a function	2017-02-15 10:57:41 +01:00
Mokaddem	0fce3f5d65	Renamed variables + added comments	2017-02-15 10:37:59 +01:00
Mokaddem	9ca3e79e70	Removed useless comments + commented	2017-02-15 10:19:03 +01:00
Mokaddem	9199ea3f0e	Added comments	2017-02-15 10:04:51 +01:00
Mokaddem	ff1df07b5d	Added comments and moved portion of code for slightly better visibility	2017-02-15 09:43:18 +01:00
Mokaddem	b272b1b07b	Added a way to quit application with 'q' more or less properly	2017-02-15 09:35:03 +01:00
Mokaddem	0254c51164	Added end of paste marker + handling screen resize	2017-02-15 08:26:48 +01:00
Mokaddem	0b9609e4a9	Merge branch 'newModuleInformation' into production	2017-02-14 17:22:06 +01:00
Mokaddem	444f6071b7	Fixed screen bug by adding an input before starting asciimatics + improved path localisation in the logs from modules	2017-02-14 17:21:52 +01:00
Mokaddem	b04bac4731	Merge branch 'newModuleInformation' into production	2017-02-14 16:08:42 +01:00
Mokaddem	d520c45a15	Added support of display of the last paste processed by a module	2017-02-14 16:05:59 +01:00
Mokaddem	d3b4409156	Added path for debug in SQLInjection	2017-02-14 11:23:34 +01:00
Mokaddem	135a1457e0	Merge branch 'fix-linkPaste-debug' into production	2017-02-14 11:12:22 +01:00
Mokaddem	d4da3a316f	Added path of the paste in the log of Categ.py	2017-02-14 10:59:47 +01:00
Mokaddem	303575b2a1	Merge remote-tracking branch 'upstream/master' into production	2017-02-14 09:31:18 +01:00
Alexandre Dulaunoy	2f4e5b9016	Merge branch 'master' of github.com:CIRCL/AIL-framework	2017-01-26 07:16:44 +00:00
Alexandre Dulaunoy	cf903cc212	(partially fix #90 too) using a simple Alarm (SIGNAL) when exec-timeout ... Introducing a timer (in this case 60 seconds) to ensure that the execution time of the analysis takes less than 60 seconds. This is a simple and standard POSIX signal handler. If the timeout is reached, the module will process the next one. This approach fixes the specific issues we have currently with some inputs where the sentiment analysis takes too much time. This fix should be improved and be more generic: - Introducing statistics of content which timeouts. - Keeping a list/queue to further process those files using a different analysis approach. Maybe a set of "dirty" processes to handle the edge cases and to not impact the overall processing and analysis. - Make the timer configurable per module (at least for this one).	2017-01-26 07:11:18 +00:00
Mokaddem	c422db6e6d	Merge remote-tracking branch 'upstream/master' into production	2017-01-17 16:38:48 +01:00
mokaddem	b3a3616b0b	Merge pull request #102 from ptitdoc/bugfix1 ... Bug fixes in Workers and improvement of pystemon-feeder	2017-01-17 16:35:32 +01:00
Mokaddem	ea423fde0f	Added requirement, better behaviors and confirmation box	2017-01-17 15:58:16 +01:00
Mokaddem	692f13d18b	Improved module + Added support of click and keyboard navigation for killing and starting modules	2017-01-17 14:31:06 +01:00
Mokaddem	3b8149d840	Added action button and support of logs	2017-01-16 17:08:48 +01:00
Mokaddem	0ecfaf00f5	Added color in front of queue related to its state	2017-01-16 15:30:33 +01:00
Mokaddem	af3241c008	Overwritted listbox update function to add formating	2017-01-16 14:41:02 +01:00
Mokaddem	30f303cb99	Added new ModuleInformation	2017-01-16 12:18:23 +01:00
Olivier MEDOC	16044d4d36	pystemon-feeder: add the option bind in configuration file instead of address ... This is essentially because multiple feeds can be used in the address configuration variable.	2017-01-13 14:58:18 +01:00
Alexandre Dulaunoy	3b101ea8f5	(partially) Fix #91 using a simple Alarm (SIGNAL) when exec-timeout ... Introducing a timer (in this case 5 seconds) to ensure that the execution time of the tokenizer takes less than 5 seconds. This is a simple and standard POSIX signal handler. This approach fixes the specific issues we have currently with some inputs where the tokenization takes too much time. This fix should be improved and be more generic: - Introducing statistics of content which timeouts. - Keeping a list/queue to further process those files using a different tokenizer approach. Maybe a set of "dirty" processes to handle the edge cases and to not impact the overall processing and analysis. - Make the timer configurable per module (at least for this one).	2017-01-12 07:32:55 +00:00
Olivier MEDOC	9f9c265cb0	SentimentAnalysis: use lexicon file path from the ail configuration file	2017-01-11 11:07:08 +01:00
ptitdoc	c0fac82031	Add pystemon path in configuration sample	2017-01-10 18:33:46 +01:00
Olivier MEDOC	83db40104c	import pystemon-feeder configuration from the central configuration file	2017-01-10 18:26:36 +01:00
Olivier MEDOC	3dc014dad9	fix invalid variable propagation	2017-01-10 18:26:24 +01:00
Olivier MEDOC	68ca20db8f	fix to use the new pycountry API	2017-01-10 18:26:17 +01:00
Alexandre Dulaunoy	1950a2dc0e	Merge pull request #99 from mokaddem/more-feed ... Usage of sha1 to filter paste content in Mixer.py	2017-01-09 17:06:37 +01:00
Mokaddem	500608d0a1	Usage of sha1 to filter paste content in Mixer.py	2017-01-09 17:01:10 +01:00
Alexandre Dulaunoy	f5adb65cb3	Merge pull request #98 from mokaddem/more-feed ... Config 2017	2017-01-09 15:39:53 +01:00
Mokaddem	409c5b8b42	Added 2017	2017-01-09 15:36:50 +01:00
Alexandre Dulaunoy	a96ba40d94	Merge pull request #96 from mokaddem/more-feed ... Handling of multiple feeders	2017-01-09 15:29:28 +01:00
Mokaddem	0e39f516a9	Added sleep in helper for multiple listening interfaces (reduce useless work), Added a simple way to pre-process feed before sending it to the global feed.	2017-01-09 14:12:26 +01:00
Mokaddem	3a298f0c7c	Commented Mixer.py for further multiple feeds support.	2016-12-26 16:23:18 +01:00
Mokaddem	1c2169d3bc	Improved interface for multiple feeds and refresh_script.	2016-12-26 16:16:44 +01:00
Mokaddem	97292e0899	Updated web interface to handle new mixer module. and fixed one dependency bug.	2016-12-23 15:44:46 +01:00
Mokaddem	e70b9cd15c	Added basic mixer with confirugable behavior. It handles muliple feeders and performs some basic stats on them.	2016-12-23 10:31:26 +01:00