Alexandre Dulaunoy 3b101ea8f5 (partially) Fix #91 using a simple Alarm (SIGNAL) when exec-timeout ... Introducing a timer (in this case 5 seconds) to ensure that the execution time of the tokenizer takes less than 5 seconds. This is a simple and standard POSIX signal handler. This approach fixes the specific issues we have currently with some inputs where the tokenization takes too much time. This fix should be improved and be more generic: - Introducing statistics of content which timeouts. - Keeping a list/queue to further process those files using a different tokenizer approach. Maybe a set of "dirty" processes to handle the edge cases and to not impact the overall processing and analysis. - Make the timer configurable per module (at least for this one).		2017-01-12 07:32:55 +00:00
..
feeder	Added pystemon instruction section	2016-10-13 17:13:08 +02:00
packages	Added sleep in helper for multiple listening interfaces (reduce useless work), Added a simple way to pre-process feed before sending it to the global feed.	2017-01-09 14:12:26 +01:00
__init__.py	Initial import of AIL framework - Analysis Information Leak framework	2014-08-06 11:43:40 +02:00
Attributes.py	Added moduleInfo + Renamed modules	2016-08-24 17:28:39 +02:00
BrowseWarningPaste.py	Added support of PID in ModuleInformation and index.html	2016-08-26 15:27:37 +02:00
Categ.py	Getting Started: Initial configuration working on the laptop	2016-06-30 14:36:47 +02:00
Credential.py	Added support of browsing concerned paste in dashboard for all modules + Show in the page what was the concerned module	2016-10-27 11:50:24 +02:00
CreditCards.py	Added support of browsing concerned paste in dashboard for all modules + Show in the page what was the concerned module	2016-10-27 11:50:24 +02:00
Curve.py	Fixed bug in curve preventing to track concerned_pastes. Now all tracked words are put in lower case.	2016-08-23 09:50:03 +02:00
CurveManageTopSets.py	Merge branch 'website-improvement' into production	2016-12-08 10:05:55 +01:00
Cve.py	Fixed bug in CVE (always sending to browse_warning even if there was no match)	2016-08-08 11:54:27 +02:00
Dir.py	Small fixes to make the refactoring production ready	2014-08-22 17:35:40 +02:00
DomClassifier.py	Added support of browsing concerned paste in dashboard for all modules + Show in the page what was the concerned module	2016-10-27 11:50:24 +02:00
DumpValidOnion.py	Fix the onion module, log the valid onions.	2014-09-01 16:18:06 +02:00
Duplicates.py	Added date in Duplicate for better distinguish duplicate. Also, added a timeout for the dataTable in duplicate inside the modal.	2016-11-24 15:05:29 +01:00
Duplicates_old.py	Added moduleInfo + Renamed modules	2016-08-24 17:28:39 +02:00
empty_queue.py	- Modified redis connection (from levelDB to redis).	2016-08-18 15:34:19 +02:00
Global.py	Updated web interface to handle new mixer module. and fixed one dependency bug.	2016-12-23 15:44:46 +01:00
Helper.py	Added sleep in helper for multiple listening interfaces (reduce useless work), Added a simple way to pre-process feed before sending it to the global feed.	2017-01-09 14:12:26 +01:00
import_dir.py	Because 0MQ fails.	2016-03-11 16:16:53 +01:00
Indexer.py	Add script to import from local directory, use local python from env	2016-02-04 15:22:51 +01:00
indexer_lookup.py	-v option added to list the path	2015-12-22 21:37:05 +00:00
Keys.py	Added warning_paste module and created related webpages.	2016-08-08 09:17:44 +02:00
LAUNCH.sh	Added 2017	2017-01-09 15:36:50 +01:00
launch_logs.sh	Add initial Travis file	2016-01-19 11:43:34 +01:00
launch_lvldb.sh	Updated isolated launch scripts + Added switch displaying queue in index	2016-08-17 13:46:01 +02:00
launch_queues.py	Add script to import from local directory, use local python from env	2016-02-04 15:22:51 +01:00
launch_queues.sh	Add initial Travis file	2016-01-19 11:43:34 +01:00
launch_redis.sh	Added new module in individual launch scripts	2016-08-23 16:01:11 +02:00
launch_scripts.sh	Added not running module info + swapped order of module launch	2016-08-26 09:21:02 +02:00
Lines.py	Added moduleInfo + Renamed modules	2016-08-24 17:28:39 +02:00
Mail.py	Added support of local paste view in dashboard for Mails event only	2016-10-27 11:27:26 +02:00
Mixer.py	Usage of sha1 to filter paste content in Mixer.py	2017-01-09 17:01:10 +01:00
ModuleInformation.py	Prepared moduleInfo to handle module click with curses (will come back later).	2016-12-22 08:45:53 +01:00
ModuleStats.py	Added save of paste_baseName in set for searches. This is done in ModuleStats to not surcharge the Global module.	2016-08-23 08:52:54 +02:00
Onion.py	Added support of browsing concerned paste in dashboard for all modules + Show in the page what was the concerned module	2016-10-27 11:50:24 +02:00
Phone.py	Added warning_paste module and created related webpages.	2016-08-08 09:17:44 +02:00
preProcessFeed.py	Added sleep in helper for multiple listening interfaces (reduce useless work), Added a simple way to pre-process feed before sending it to the global feed.	2017-01-09 14:12:26 +01:00
QueueIn.py	Big refactoring, make the queues more flexible	2014-08-29 19:37:56 +02:00
QueueOut.py	Big refactoring, make the queues more flexible	2014-08-29 19:37:56 +02:00
Queues_Monitoring.py	More cleanup	2014-09-05 10:42:01 +02:00
Release.py	Added support of browsing concerned paste in dashboard for all modules + Show in the page what was the concerned module	2016-10-27 11:50:24 +02:00
Repartition_graph.py	Small fixes to make the refactoring production ready	2014-08-22 17:35:40 +02:00
SentimentAnalysis.py	Added moduleInfo + Renamed modules	2016-08-24 17:28:39 +02:00
Shutdown.py	Small fixes to make the refactoring production ready	2014-08-22 17:35:40 +02:00
SourceCode.py	Improve SourceCode, keywords and add description in /doc	2016-03-12 12:30:38 +01:00
SQLInjectionDetection.py	Added support of browsing concerned paste in dashboard for all modules + Show in the page what was the concerned module	2016-10-27 11:50:24 +02:00
template.py	Add template file for writing a new module	2016-02-03 10:33:42 +01:00
Tokenize.py	(partially) Fix #91 using a simple Alarm (SIGNAL) when exec-timeout	2017-01-12 07:32:55 +00:00
tor_fetcher.py	fix onions, cc and domain classifier modules	2014-09-08 16:51:43 +02:00
Web.py	Added support of browsing concerned paste in dashboard for all modules + Show in the page what was the concerned module	2016-10-27 11:50:24 +02:00
WebStats.py	Webstats should correctly updates top_progression_zset (Not fully tested because not enough data. Will be tested latter)	2016-12-08 10:05:07 +01:00