Mokaddem
a4595010c0
Added whole config in config.sample
2016-08-09 14:23:36 +02:00
Mokaddem
a80f6ca0c2
Bug fix related with redis:
...
Fixed typo key in redis for module creditcard and sqlinjection
Modified Curve redisLvlDb server
Modified Url.py so that it forwards name of protocol from saved protocolsfile
Added Cache control in Flask
Modified key-tab name into keys-tab
2016-08-09 11:59:36 +02:00
Mokaddem
60512a30a2
Fixeed bug in Duplicate (The comparison value was not saved correctly in redis) + Modified to progression detection algo
2016-08-08 15:28:26 +02:00
Mokaddem
aedb3198cd
Fixed bug in CVE (always sending to browse_warning even if there was no match)
2016-08-08 11:54:27 +02:00
Mokaddem
14062a549c
Fixed bug in credential (not sending correctly to ModuleStat) + Fixed date bug in ModuleStats
2016-08-08 11:37:18 +02:00
Mokaddem
4e56f76f62
Fixed bug CVE,
...
CVE was not sending its data to BrowseWarningPaste.
2016-08-08 10:05:21 +02:00
Mokaddem
1c40ad522f
Merge branch 'master' into production
2016-08-08 09:26:58 +02:00
Mokaddem
78c611fead
Added warning_paste module and created related webpages.
...
Fixed a Faup bug in credential (multiple instanciation) and added correc populate_set_out in concerned modules (creditcard, credential, ...).
Linked browse_warning_paste module and Flask function with redis (created new sets).
2016-08-08 09:17:44 +02:00
Mokaddem
3523d79a05
Started browse important paste feature. Still need to link with redis. + Fixed a bug in Flask (String to list parser)
2016-08-04 16:57:34 +02:00
Mokaddem
55ef917ef1
Added few comments
2016-08-04 12:10:56 +02:00
Mokaddem
d9316771cd
Draft: added new duplicate hash comparison - tlsh
2016-08-04 11:55:38 +02:00
Mokaddem
455911fc26
Better display printing
2016-08-02 16:21:52 +02:00
Mokaddem
3169dc27b0
Added SQLInjectionDetection module
2016-08-02 15:43:11 +02:00
Mokaddem
a1b2329a9b
Fixed bug in case of null elements arrising because of a new month
2016-08-01 10:32:24 +02:00
Mokaddem
a9637ec7cf
removed prints
2016-07-29 16:54:25 +02:00
Mokaddem
feb1717dc6
fixed size bug, which in some cases display the total size instead of the average + few cleanup + commented out log scale in moduleStats.js and added large number division in case of div by 0
2016-07-29 16:52:50 +02:00
Mokaddem
e8fae17cfd
Fixed small race condition bug in ModuleStat.py and Added highlight of bar belonging to the same serie in moduleStats.js
2016-07-29 10:36:44 +02:00
Mokaddem
240bdd9cca
Added number of paste by provider in module stats
2016-07-27 09:27:32 +02:00
Mokaddem
baa042827d
Removed useless comments + prevent trying to add an already present element in the set
2016-07-26 11:06:31 +02:00
Mokaddem
cc1b49baaf
Fixed variable bug in moduleStats and added draft of credential stats
2016-07-26 10:45:02 +02:00
Mokaddem
06be1f129a
Added Draft of ModuleStats + Paste size average
2016-07-25 16:38:57 +02:00
d013546643
Merge branch 'master' of github.com:CIRCL/AIL-framework
...
Conflicts:
bin/packages/modules.cfg
2016-07-25 07:56:46 +00:00
722634adf4
Prod config
2016-07-25 07:49:14 +00:00
Raphaël Vinot
4aa484a91f
Merge pull request #63 from mokaddem/uptodate-duplicate-module
...
Uptodate duplicate module
2016-07-25 09:44:49 +02:00
Mokaddem
34a00abdc2
Merge branch 'uptodate-duplicate-module' into production
2016-07-22 10:31:10 +02:00
Mokaddem
c686f69ca6
Clean not compatible ssdeep hash encountered
2016-07-22 10:04:58 +02:00
Raphaël Vinot
ea8703e608
Fix path to tldsfile
2016-07-22 09:32:13 +02:00
Mokaddem
dad05073fd
Fixed bug introduced in merge-conflict
2016-07-22 09:25:05 +02:00
Mokaddem
7c4f4a2f79
Merge branch 'statistics-v2' into uptodate-statistics-v2
2016-07-21 15:16:03 +02:00
Mokaddem
b685f81752
Merge branch 'duplicate-module' into uptodate-duplicate-module
2016-07-21 14:59:52 +02:00
Mokaddem
0ab7560477
Added top_progression chart for tld, domain and scheme + Small modification in config file.
2016-07-21 13:44:22 +02:00
Raphaël Vinot
ede46ec799
Fix default config.
2016-07-21 09:19:13 +02:00
Raphaël Vinot
781b441bb8
Fix path in config
2016-07-20 14:12:50 +02:00
Raphaël Vinot
620d69467f
Add WebStats in launcher
2016-07-20 14:12:37 +02:00
Raphaël Vinot
62eef44ca8
Fix pep8
2016-07-20 14:12:18 +02:00
Raphaël Vinot
8a69110c8e
Merge branch 'webpage-splitting' of https://github.com/mokaddem/AIL-framework into mokaddem-webpage-splitting
2016-07-19 16:35:31 +02:00
Mokaddem
a6996c0b23
Added related functions and display of duplicated paste in search.py
2016-07-19 10:48:44 +02:00
Mokaddem
996c0e02de
Duplicate module takes its messages from other modules and no more from Global.
2016-07-18 16:22:33 +02:00
Mokaddem
6805ed6488
Added default config
2016-07-18 15:52:53 +02:00
Mokaddem
4f6813350b
Added two new version of duplicate module.
...
One with hashes are saved in json on disk
The other with only leveldb
2016-07-18 15:50:41 +02:00
Mokaddem
14e9850dd6
Added new module for Duplicate paste. Seems working but has some small bug (re-check same paste twice)
2016-07-15 16:58:48 +02:00
Mokaddem
2383db022f
Added default configuration
2016-07-15 09:10:44 +02:00
Mokaddem
9a9e07f600
Added default configuration
2016-07-15 09:08:38 +02:00
Mokaddem
0332f23579
Added SimHash library
2016-07-15 08:56:16 +02:00
Mokaddem
fba14bfb4b
In index: Added number of processed pastes chart
2016-07-14 11:16:07 +02:00
Mokaddem
ab61e32399
Commented out get_language because it adds too much overhead
2016-07-14 11:15:15 +02:00
Mokaddem
594d2def35
In index: Added number of processed pastes chart
2016-07-13 15:57:33 +02:00
Mokaddem
56b6659d8b
Commented out get_language because it adds too much overhead
2016-07-13 08:59:48 +02:00
Mokaddem
c51bdec8aa
Merge branch 'mokaddem-testing'
2016-07-12 11:53:24 +02:00
Mokaddem
8a1247cf5d
modified variable name str
2016-07-12 11:52:19 +02:00
Mokaddem
7e5ce0f17f
Optimized create_plot and removed test commemts
2016-07-12 11:47:51 +02:00
Mokaddem
465244e1ce
Added dynamic table sorting in search page. (Still need to add dependencies)
2016-07-07 16:38:00 +02:00
Mokaddem
7ff9b9a583
Added DomainTrending seems working.
...
Started search features with related html pages, not finish yet.
2016-07-05 16:53:03 +02:00
Mokaddem
5a9eca9291
Added few comments
2016-07-04 09:18:23 +02:00
Mokaddem
4b3101b7b6
Added template tld. Modified URL using Faup and refactored WebStats.
2016-07-01 16:59:08 +02:00
Mokaddem
beeeb76de9
Added new modules and started WebTrending web interface
2016-06-30 14:38:28 +02:00
Mokaddem
3dc356dc5e
Getting Started: Initial configuration working on the laptop
2016-06-30 14:36:47 +02:00
MaximeStor
ab66cd255a
Improve SourceCode, keywords and add description in /doc
2016-03-12 12:30:38 +01:00
MaximeStor
701d771aa5
Add first version of Source Code
2016-03-12 11:21:29 +01:00
Raphaël Vinot
be86737ca7
Because 0MQ fails.
2016-03-11 16:16:53 +01:00
Raphaël Vinot
f6e4ea2270
Fix logging, fix URL regex
2016-02-11 12:19:03 +01:00
Raphaël Vinot
d160e4a2c8
Add Credential in the scripts to launch
2016-02-10 17:33:16 +01:00
Raphaël Vinot
90e1b25426
Split filepath and count in credential module
2016-02-10 17:31:52 +01:00
Raphaël Vinot
4895ee9fa2
Add new category (Credential)
2016-02-10 16:39:56 +01:00
Raphaël Vinot
1da8675750
Refactoring on Credential, Phone and Release
2016-02-10 16:39:06 +01:00
c68136b04b
Merge branch 'pgp' of https://github.com/Rafiot/AIL-framework
...
Conflicts:
bin/packages/modules.cfg
2016-02-08 10:13:44 +01:00
MaximeStor
b7d2b64a86
Merge branch 'master' of https://github.com/CIRCL/AIL-framework into module
2016-02-06 11:28:48 +01:00
192ee7f0ec
Merge pull request #49 from Alainfou/master
...
Phone module added (regex needs optimization)
2016-02-06 11:20:46 +01:00
Alain
ea52fd1068
Phone regex updated
...
Might still need to be fixed / optimized, in case of maths or random numbers starting with a 0. Do not capture dates, hours, coordinates anymore. Captured formats are: e.g. +331234567890 ; 09 12 34 56 78 ; +4177/123.45.69 ; +352(0)6-23-23-23...
2016-02-05 20:58:02 +01:00
Alain
fabbfd8ae9
Update module.cfg (adding Keys and Phone section)
2016-02-05 14:00:41 -05:00
Alain
43b3556588
Starting Phone number recognition
2016-02-05 13:58:21 -05:00
MaximeStor
07513a5b37
Add modules Credential and Release
2016-02-05 16:15:09 +01:00
Raphaël Vinot
9171d5b118
Add module to find PGP encrypted blobs
2016-02-05 16:03:37 +01:00
Raphaël Vinot
aef8ab0411
Listen locally for 0MQ
2016-02-04 15:32:50 +01:00
Raphaël Vinot
5ca13c42eb
Launch redis and leveldb from local directory
2016-02-04 15:24:39 +01:00
Raphaël Vinot
12aca6b760
Add script to import from local directory, use local python from env
2016-02-04 15:22:51 +01:00
Raphaël Vinot
315cb48117
Add template file for writing a new module
2016-02-03 10:33:42 +01:00
Raphaël Vinot
0d6adc2063
Add initial Travis file
2016-01-19 11:43:34 +01:00
cdd0725e88
-v option added to list the path
2015-12-22 21:37:05 +00:00
e3971ac93a
Onion fetching loop deactivated by default
2014-12-22 16:06:38 +00:00
Raphaël Vinot
08ceefc375
Re-add config option
2014-12-22 16:50:25 +01:00
Raphaël Vinot
50369c6706
Revert changes on the config file due to merging messup
2014-12-22 16:29:05 +01:00
Raphaël Vinot
f717f9fe89
Merge branch 'master' of github.com:CIRCL/AIL-framework
2014-12-22 15:32:48 +01:00
Raphaël Vinot
9ee61db2cf
Add hotfixes
2014-12-22 15:27:02 +01:00
Raphaël Vinot
8803c8447a
Publish the fetched onions on a ZMQ feed.
2014-09-30 16:55:16 +02:00
25757b0fff
A simple feeder script feeding data from pystemon to AIL.
...
The configuration matches the default Redis parameters used
in the pystemon configuration.
https://github.com/cvandeplas/pystemon/blob/master/pystemon.yaml#L16
2014-09-19 14:03:05 +02:00
Raphaël Vinot
65b9a01644
Add config file for DomainClassifier, proper reporting
2014-09-17 17:22:56 +02:00
27b134ec03
Add proper publisher for classified domains/hostnames
2014-09-10 09:27:47 +02:00
Raphaël Vinot
f017680365
fix onions, cc and domain classifier modules
2014-09-08 16:51:43 +02:00
de6e21d5a7
DomainClassifier sample configuration added
2014-09-08 16:44:05 +02:00
246621f663
First version of the DomainClassifier
2014-09-08 16:43:21 +02:00
1397db9691
Global queue for DomainClassifier
2014-09-08 11:07:45 +02:00
Raphaël Vinot
e983c839ad
Categ now listen to the Global queue
2014-09-05 17:05:45 +02:00
Raphaël Vinot
46f27ada4e
More cleanup
2014-09-05 10:42:01 +02:00
Raphaël Vinot
fca00beed9
Add Domain Classifier module.
...
Cleanup in the config files.
2014-09-05 10:41:00 +02:00
Raphaël Vinot
b7c9e489c9
Fix the exceptions
2014-09-04 11:46:07 +02:00
Raphaël Vinot
9e8611a42d
stop killing the disk when creating the word curve
2014-09-02 18:20:28 +02:00
Raphaël Vinot
7542eaf739
Update starting script.
2014-09-02 15:21:36 +02:00
Raphaël Vinot
0c6b09f379
Fix the onion module, log the valid onions.
2014-09-01 16:18:06 +02:00
Raphaël Vinot
f4b89669fc
The onion module now fetches the URLs it finds.
2014-08-31 22:42:12 +02:00
