Commit graph

112 commits

Author SHA1 Message Date
Mokaddem
4f6813350b Added two new version of duplicate module.
One with hashes are saved in json on disk
The other with only leveldb
2016-07-18 15:50:41 +02:00
Mokaddem
14e9850dd6 Added new module for Duplicate paste. Seems working but has some small bug (re-check same paste twice) 2016-07-15 16:58:48 +02:00
Mokaddem
0332f23579 Added SimHash library 2016-07-15 08:56:16 +02:00
Mokaddem
594d2def35 In index: Added number of processed pastes chart 2016-07-13 15:57:33 +02:00
Mokaddem
56b6659d8b Commented out get_language because it adds too much overhead 2016-07-13 08:59:48 +02:00
Mokaddem
c51bdec8aa Merge branch 'mokaddem-testing' 2016-07-12 11:53:24 +02:00
Mokaddem
8a1247cf5d modified variable name str 2016-07-12 11:52:19 +02:00
Mokaddem
7e5ce0f17f Optimized create_plot and removed test commemts 2016-07-12 11:47:51 +02:00
Mokaddem
465244e1ce Added dynamic table sorting in search page. (Still need to add dependencies) 2016-07-07 16:38:00 +02:00
Mokaddem
7ff9b9a583 Added DomainTrending seems working.
Started search features with related html pages, not finish yet.
2016-07-05 16:53:03 +02:00
Mokaddem
5a9eca9291 Added few comments 2016-07-04 09:18:23 +02:00
Mokaddem
4b3101b7b6 Added template tld. Modified URL using Faup and refactored WebStats. 2016-07-01 16:59:08 +02:00
Mokaddem
beeeb76de9 Added new modules and started WebTrending web interface 2016-06-30 14:38:28 +02:00
Mokaddem
3dc356dc5e Getting Started: Initial configuration working on the laptop 2016-06-30 14:36:47 +02:00
MaximeStor
ab66cd255a Improve SourceCode, keywords and add description in /doc 2016-03-12 12:30:38 +01:00
MaximeStor
701d771aa5 Add first version of Source Code 2016-03-12 11:21:29 +01:00
Raphaël Vinot
be86737ca7 Because 0MQ fails. 2016-03-11 16:16:53 +01:00
Raphaël Vinot
f6e4ea2270 Fix logging, fix URL regex 2016-02-11 12:19:03 +01:00
Raphaël Vinot
d160e4a2c8 Add Credential in the scripts to launch 2016-02-10 17:33:16 +01:00
Raphaël Vinot
90e1b25426 Split filepath and count in credential module 2016-02-10 17:31:52 +01:00
Raphaël Vinot
4895ee9fa2 Add new category (Credential) 2016-02-10 16:39:56 +01:00
Raphaël Vinot
1da8675750 Refactoring on Credential, Phone and Release 2016-02-10 16:39:06 +01:00
c68136b04b Merge branch 'pgp' of https://github.com/Rafiot/AIL-framework
Conflicts:
	bin/packages/modules.cfg
2016-02-08 10:13:44 +01:00
MaximeStor
b7d2b64a86 Merge branch 'master' of https://github.com/CIRCL/AIL-framework into module 2016-02-06 11:28:48 +01:00
192ee7f0ec Merge pull request #49 from Alainfou/master
Phone module added (regex needs optimization)
2016-02-06 11:20:46 +01:00
Alain
ea52fd1068 Phone regex updated
Might still need to be fixed / optimized, in case of maths or random numbers starting with a 0. Do not capture dates, hours, coordinates anymore. Captured formats are: e.g. +331234567890 ; 09 12 34 56 78 ; +4177/123.45.69 ; +352(0)6-23-23-23...
2016-02-05 20:58:02 +01:00
Alain
fabbfd8ae9 Update module.cfg (adding Keys and Phone section) 2016-02-05 14:00:41 -05:00
Alain
43b3556588 Starting Phone number recognition 2016-02-05 13:58:21 -05:00
MaximeStor
07513a5b37 Add modules Credential and Release 2016-02-05 16:15:09 +01:00
Raphaël Vinot
9171d5b118 Add module to find PGP encrypted blobs 2016-02-05 16:03:37 +01:00
Raphaël Vinot
aef8ab0411 Listen locally for 0MQ 2016-02-04 15:32:50 +01:00
Raphaël Vinot
5ca13c42eb Launch redis and leveldb from local directory 2016-02-04 15:24:39 +01:00
Raphaël Vinot
12aca6b760 Add script to import from local directory, use local python from env 2016-02-04 15:22:51 +01:00
Raphaël Vinot
315cb48117 Add template file for writing a new module 2016-02-03 10:33:42 +01:00
Raphaël Vinot
0d6adc2063 Add initial Travis file 2016-01-19 11:43:34 +01:00
cdd0725e88 -v option added to list the path 2015-12-22 21:37:05 +00:00
e3971ac93a Onion fetching loop deactivated by default 2014-12-22 16:06:38 +00:00
Raphaël Vinot
08ceefc375 Re-add config option 2014-12-22 16:50:25 +01:00
Raphaël Vinot
50369c6706 Revert changes on the config file due to merging messup 2014-12-22 16:29:05 +01:00
Raphaël Vinot
f717f9fe89 Merge branch 'master' of github.com:CIRCL/AIL-framework 2014-12-22 15:32:48 +01:00
Raphaël Vinot
9ee61db2cf Add hotfixes 2014-12-22 15:27:02 +01:00
Raphaël Vinot
8803c8447a Publish the fetched onions on a ZMQ feed. 2014-09-30 16:55:16 +02:00
25757b0fff A simple feeder script feeding data from pystemon to AIL.
The configuration matches the default Redis parameters used
in the pystemon configuration.

https://github.com/cvandeplas/pystemon/blob/master/pystemon.yaml#L16
2014-09-19 14:03:05 +02:00
Raphaël Vinot
65b9a01644 Add config file for DomainClassifier, proper reporting 2014-09-17 17:22:56 +02:00
27b134ec03 Add proper publisher for classified domains/hostnames 2014-09-10 09:27:47 +02:00
Raphaël Vinot
f017680365 fix onions, cc and domain classifier modules 2014-09-08 16:51:43 +02:00
de6e21d5a7 DomainClassifier sample configuration added 2014-09-08 16:44:05 +02:00
246621f663 First version of the DomainClassifier 2014-09-08 16:43:21 +02:00
1397db9691 Global queue for DomainClassifier 2014-09-08 11:07:45 +02:00
Raphaël Vinot
e983c839ad Categ now listen to the Global queue 2014-09-05 17:05:45 +02:00