3ce850e093
Merge pull request #94 from mokaddem/website-refacto
...
Website refacto, support and fixes
2016-12-22 22:17:58 +01:00
Mokaddem
9758579753
Merge branch 'module_handling' into production
2016-12-22 10:24:30 +01:00
Mokaddem
7438f16c63
Added new config
2016-12-22 10:06:35 +01:00
Mokaddem
73938cf41b
Prepared moduleInfo to handle module click with curses (will come back later).
2016-12-22 08:45:53 +01:00
Mokaddem
d37b243e1d
Added legend + added more info if restarting disabled
2016-12-09 15:13:47 +01:00
Mokaddem
d3ed1bbf40
Merge branch 'website-improvement' into production
2016-12-08 10:05:55 +01:00
Mokaddem
73d4f9e082
Webstats should correctly updates top_progression_zset (Not fully tested because not enough data. Will be tested latter)
2016-12-08 10:05:07 +01:00
Mokaddem
570324060e
terms top_sets correctly supports blacklisted terms
2016-12-08 09:13:31 +01:00
412b80c44d
Merge pull request #88 from mokaddem/Issue-Feeding-AIL
...
Pystemon tuto
2016-11-26 12:45:28 +01:00
Mokaddem
d831a79614
Do not try to restart not running queue if autokill is not set
2016-11-25 16:00:46 +01:00
Mokaddem
4bf45eea36
ModuleInformation sort by name as well as elapsed time
2016-11-25 14:44:27 +01:00
Mokaddem
866be4e884
Added row coloring in ModuleInformation
2016-11-25 14:12:09 +01:00
Mokaddem
a3255d168c
ModuleInfo correctly handle CurveManageTopSets,
...
Changed queue coloring in dashboard,
ModuleInformation now have a history of executed command and better handle module killing.
2016-11-25 11:54:16 +01:00
Mokaddem
224fbc8084
Better handle stuck modules. Differentiate between not running and no info and tries to restart stuck ones.
2016-11-24 16:58:32 +01:00
Mokaddem
5d269ea1ee
Added date in Duplicate for better distinguish duplicate. Also, added a timeout for the dataTable in duplicate inside the modal.
2016-11-24 15:05:29 +01:00
Mokaddem
ac254e0e63
Added support of browsing concerned paste in dashboard for all modules + Show in the page what was the concerned module
2016-10-27 11:50:24 +02:00
Mokaddem
1826b170ec
Added support of local paste view in dashboard for Mails event only
2016-10-27 11:27:26 +02:00
Mokaddem
5a4ea734a2
Fixed path issues in doc + added dependencies related to ssdeep
2016-10-14 14:26:33 +02:00
Mokaddem
4eafca2936
Added pystemon instruction section
2016-10-13 17:13:08 +02:00
eb14005fcf
Fix: as JSON decode needs a string, the empty list is a string.
2016-08-29 12:08:12 +00:00
3b6e3e156e
Merge branch 'master' of github.com:CIRCL/AIL-framework
2016-08-26 15:57:45 +00:00
Mokaddem
0c760d763b
Added support of PID in ModuleInformation and index.html
2016-08-26 15:27:37 +02:00
758f2c5334
Merge branch 'master' of github.com:CIRCL/AIL-framework
2016-08-26 08:30:11 +00:00
Mokaddem
494980c48e
Added comment in moduleInformation
2016-08-26 10:07:06 +02:00
Mokaddem
8b33b4e7ba
Added not running module info + swapped order of module launch
2016-08-26 09:21:02 +02:00
Mokaddem
813cc7031a
Changed lots of code into zrank
2016-08-25 17:48:12 +02:00
Mokaddem
aaac798db9
updated individual launch script
2016-08-25 16:51:57 +02:00
Mokaddem
78c991539e
Added number of same module running in ModuleInformation
2016-08-25 16:07:47 +02:00
1ab08d4a05
Merge branch 'master' of github.com:CIRCL/AIL-framework
2016-08-25 12:33:31 +00:00
Mokaddem
f9bf00f022
Removed autokill feature if not specified
2016-08-25 11:09:31 +02:00
Mokaddem
2f136595fb
Removed autokill feature if not specified
2016-08-25 11:05:18 +02:00
Mokaddem
9146feab88
Fixed bug in WebStats not deleting correct key + fixed typo in CurveSetManager preventing connecting to redis + modified display in moduleInfo
2016-08-25 10:43:04 +02:00
Mokaddem
3313064e51
removed useless import.
2016-08-24 17:32:50 +02:00
Mokaddem
64bf481480
Added moduleInfo + Renamed modules
2016-08-24 17:28:39 +02:00
Mokaddem
8fb552b7da
ModuleInfo support autokill and auto restart after logging.
2016-08-24 16:52:01 +02:00
Mokaddem
3e88d0911d
ModuleInfo now support logs.
2016-08-24 15:53:00 +02:00
Mokaddem
ff6dcaf456
Fixed typo in creditcard + fixed bug in ModuleInformation (relative to date).
2016-08-24 15:35:23 +02:00
Mokaddem
deeebec2f5
Added new module 'ModuleInformation' which gives real time information on running module
2016-08-24 15:21:41 +02:00
Mokaddem
07856f3119
Added script module-dataflow-graph + fixed bug in module.cfg preventing the credential module to receive data.
2016-08-24 11:32:48 +02:00
bde5f47ed0
Merge branch 'master' of github.com:CIRCL/AIL-framework
...
Conflicts:
bin/launch_scripts.sh
2016-08-23 14:20:39 +00:00
Mokaddem
42b3966499
Added new module in individual launch scripts
2016-08-23 16:01:11 +02:00
Mokaddem
697812dbff
Added config.sample
2016-08-23 09:59:39 +02:00
Mokaddem
f9b27e4013
Fixed bug in curve preventing to track concerned_pastes. Now all tracked words are put in lower case.
2016-08-23 09:50:03 +02:00
Mokaddem
047331df3e
Added save of paste_baseName in set for searches. This is done in ModuleStats to not surcharge the Global module.
2016-08-23 08:52:54 +02:00
Mokaddem
20fd191c20
Suppressed prints
2016-08-22 21:29:59 +02:00
Mokaddem
2fd2c8af39
Removed commented code
2016-08-22 20:59:56 +02:00
Mokaddem
003c72bd7c
Fixed bug with redis where the inserted key was too long, causing a crash + fixed bug taking min instead of max in terms-top-set
2016-08-22 16:01:42 +02:00
Mokaddem
2466f355ab
Fixed bug in manage-terms-top-set + added auto sorting in important-paste-by-module
2016-08-22 14:54:37 +02:00
Mokaddem
d3c0fc5f94
Added terms trending module and web interface:
...
- Curve module has been rewritten
- Added new module curve_manage_top_set
- Added all web-pages
2016-08-22 13:35:49 +02:00
Mokaddem
0b6828fb16
Correctly handle and display add or remove a term in term-frequency + started sketch of terms-plot tool.
2016-08-19 16:53:46 +02:00
Mokaddem
b78effe779
Added draft of terms_management + Added copyright in readme
2016-08-19 13:34:02 +02:00
Mokaddem
52b7124a6b
- Modified redis connection (from levelDB to redis).
...
- Added term frequency in curve.
- Modified ModuleStats and WebStats to use redis special command (incrby instead of get-set and zset) and Flask to perform the correct queries.
- Added panel color in sentiment trending
2016-08-18 15:34:19 +02:00
Mokaddem
902b7e39f4
Probably fixed bug in creditcard (not sending to duplicate)
2016-08-17 14:40:13 +02:00
Mokaddem
c10f28d9df
Updated isolated launch scripts + Added switch displaying queue in index
2016-08-17 13:46:01 +02:00
Mokaddem
dbc164f4e8
modified + into append
2016-08-17 11:26:07 +02:00
Mokaddem
c12e67a35a
Added tooltips + slight gauge modification
2016-08-17 10:43:43 +02:00
Mokaddem
894b9efda9
Added drop of really long line in sentiment-analysis module + Added description of sentiment module. Also, fixed bug in webpage sentiement-trending concerning avg and date range.
2016-08-17 09:46:25 +02:00
Mokaddem
1084e45f1b
Added draft of filter in sentiment analysis (Discard syntaxical languages) + Added nice tooltip for sparkline. Trending displays avg in function of the number of elements processed and not for the complete week + fixed bug in gauge and canvasjs (was performing avg with only 1 graph instead of all 8).
2016-08-16 16:33:02 +02:00
Mokaddem
cdd08d7df0
Added all providers for plot_tool in moduleStats + fixed date range bug
2016-08-16 11:07:36 +02:00
Mokaddem
1a5158b081
Added sentiment analyser module (draft)
2016-08-13 15:24:57 +02:00
Mokaddem
c028ca80da
Separated options from the plot funcion for sparkline.
2016-08-11 17:57:23 +02:00
95a22abe9c
Merge branch 'master' of github.com:CIRCL/AIL-framework
...
Conflicts:
bin/packages/modules.cfg
2016-08-11 08:06:57 +00:00
Mokaddem
cd74e6d1c5
Added module sentimentAnalyser
2016-08-11 09:40:42 +02:00
Mokaddem
64fe32b29b
Fixed small bug in SQLInjection module
2016-08-10 14:23:36 +02:00
Mokaddem
feafa8ffe2
Slight refacto + sql comments are now placed in injection suspision.
2016-08-10 09:06:37 +02:00
Mokaddem
943016d6ca
Fixed one of the small display bug in DUplicate_ssdeep_v2 (was displaying [tlsh, tlsh] instead of simply tlsh)
2016-08-09 15:06:55 +02:00
Mokaddem
a4595010c0
Added whole config in config.sample
2016-08-09 14:23:36 +02:00
Mokaddem
a80f6ca0c2
Bug fix related with redis:
...
Fixed typo key in redis for module creditcard and sqlinjection
Modified Curve redisLvlDb server
Modified Url.py so that it forwards name of protocol from saved protocolsfile
Added Cache control in Flask
Modified key-tab name into keys-tab
2016-08-09 11:59:36 +02:00
Mokaddem
60512a30a2
Fixeed bug in Duplicate (The comparison value was not saved correctly in redis) + Modified to progression detection algo
2016-08-08 15:28:26 +02:00
Mokaddem
aedb3198cd
Fixed bug in CVE (always sending to browse_warning even if there was no match)
2016-08-08 11:54:27 +02:00
Mokaddem
14062a549c
Fixed bug in credential (not sending correctly to ModuleStat) + Fixed date bug in ModuleStats
2016-08-08 11:37:18 +02:00
Mokaddem
4e56f76f62
Fixed bug CVE,
...
CVE was not sending its data to BrowseWarningPaste.
2016-08-08 10:05:21 +02:00
Mokaddem
1c40ad522f
Merge branch 'master' into production
2016-08-08 09:26:58 +02:00
Mokaddem
78c611fead
Added warning_paste module and created related webpages.
...
Fixed a Faup bug in credential (multiple instanciation) and added correc populate_set_out in concerned modules (creditcard, credential, ...).
Linked browse_warning_paste module and Flask function with redis (created new sets).
2016-08-08 09:17:44 +02:00
Mokaddem
3523d79a05
Started browse important paste feature. Still need to link with redis. + Fixed a bug in Flask (String to list parser)
2016-08-04 16:57:34 +02:00
Mokaddem
55ef917ef1
Added few comments
2016-08-04 12:10:56 +02:00
Mokaddem
d9316771cd
Draft: added new duplicate hash comparison - tlsh
2016-08-04 11:55:38 +02:00
Mokaddem
455911fc26
Better display printing
2016-08-02 16:21:52 +02:00
Mokaddem
3169dc27b0
Added SQLInjectionDetection module
2016-08-02 15:43:11 +02:00
Mokaddem
a1b2329a9b
Fixed bug in case of null elements arrising because of a new month
2016-08-01 10:32:24 +02:00
Mokaddem
a9637ec7cf
removed prints
2016-07-29 16:54:25 +02:00
Mokaddem
feb1717dc6
fixed size bug, which in some cases display the total size instead of the average + few cleanup + commented out log scale in moduleStats.js and added large number division in case of div by 0
2016-07-29 16:52:50 +02:00
Mokaddem
e8fae17cfd
Fixed small race condition bug in ModuleStat.py and Added highlight of bar belonging to the same serie in moduleStats.js
2016-07-29 10:36:44 +02:00
Mokaddem
240bdd9cca
Added number of paste by provider in module stats
2016-07-27 09:27:32 +02:00
Mokaddem
baa042827d
Removed useless comments + prevent trying to add an already present element in the set
2016-07-26 11:06:31 +02:00
Mokaddem
cc1b49baaf
Fixed variable bug in moduleStats and added draft of credential stats
2016-07-26 10:45:02 +02:00
Mokaddem
06be1f129a
Added Draft of ModuleStats + Paste size average
2016-07-25 16:38:57 +02:00
d013546643
Merge branch 'master' of github.com:CIRCL/AIL-framework
...
Conflicts:
bin/packages/modules.cfg
2016-07-25 07:56:46 +00:00
722634adf4
Prod config
2016-07-25 07:49:14 +00:00
Raphaël Vinot
4aa484a91f
Merge pull request #63 from mokaddem/uptodate-duplicate-module
...
Uptodate duplicate module
2016-07-25 09:44:49 +02:00
Mokaddem
34a00abdc2
Merge branch 'uptodate-duplicate-module' into production
2016-07-22 10:31:10 +02:00
Mokaddem
c686f69ca6
Clean not compatible ssdeep hash encountered
2016-07-22 10:04:58 +02:00
Raphaël Vinot
ea8703e608
Fix path to tldsfile
2016-07-22 09:32:13 +02:00
Mokaddem
dad05073fd
Fixed bug introduced in merge-conflict
2016-07-22 09:25:05 +02:00
Mokaddem
7c4f4a2f79
Merge branch 'statistics-v2' into uptodate-statistics-v2
2016-07-21 15:16:03 +02:00
Mokaddem
b685f81752
Merge branch 'duplicate-module' into uptodate-duplicate-module
2016-07-21 14:59:52 +02:00
Mokaddem
0ab7560477
Added top_progression chart for tld, domain and scheme + Small modification in config file.
2016-07-21 13:44:22 +02:00
Raphaël Vinot
ede46ec799
Fix default config.
2016-07-21 09:19:13 +02:00
Raphaël Vinot
781b441bb8
Fix path in config
2016-07-20 14:12:50 +02:00
Raphaël Vinot
620d69467f
Add WebStats in launcher
2016-07-20 14:12:37 +02:00
