AIL/ail-framework
7
0
Fork 0
mirror of https://github.com/ail-project/ail-framework.git synced 2024-11-27 00:07:16 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions
204 commits 8 branches 43 tags 99 MiB
Commit graph

97 commits

Author SHA1 Message Date
MaximeStor
701d771aa5 Add first version of Source Code 2016-03-12 11:21:29 +01:00
Raphaël Vinot
be86737ca7 Because 0MQ fails. 2016-03-11 16:16:53 +01:00
Raphaël Vinot
f6e4ea2270 Fix logging, fix URL regex 2016-02-11 12:19:03 +01:00
Raphaël Vinot
d160e4a2c8 Add Credential in the scripts to launch 2016-02-10 17:33:16 +01:00
Raphaël Vinot
90e1b25426 Split filepath and count in credential module 2016-02-10 17:31:52 +01:00
Raphaël Vinot
4895ee9fa2 Add new category (Credential) 2016-02-10 16:39:56 +01:00
Raphaël Vinot
1da8675750 Refactoring on Credential, Phone and Release 2016-02-10 16:39:06 +01:00
Alexandre Dulaunoy
c68136b04b Merge branch 'pgp' of https://github.com/Rafiot/AIL-framework 
Conflicts:
	bin/packages/modules.cfg
 2016-02-08 10:13:44 +01:00
MaximeStor
b7d2b64a86 Merge branch 'master' of https://github.com/CIRCL/AIL-framework into module 2016-02-06 11:28:48 +01:00
Alexandre Dulaunoy
192ee7f0ec Merge pull request #49 from Alainfou/master 
Phone module added (regex needs optimization)
 2016-02-06 11:20:46 +01:00
Alain
ea52fd1068 Phone regex updated 
Might still need to be fixed / optimized, in case of maths or random numbers starting with a 0. Do not capture dates, hours, coordinates anymore. Captured formats are: e.g. +331234567890 ; 09 12 34 56 78 ; +4177/123.45.69 ; +352(0)6-23-23-23...
 2016-02-05 20:58:02 +01:00
Alain
fabbfd8ae9 Update module.cfg (adding Keys and Phone section) 2016-02-05 14:00:41 -05:00
Alain
43b3556588 Starting Phone number recognition 2016-02-05 13:58:21 -05:00
MaximeStor
07513a5b37 Add modules Credential and Release 2016-02-05 16:15:09 +01:00
Raphaël Vinot
9171d5b118 Add module to find PGP encrypted blobs 2016-02-05 16:03:37 +01:00
Raphaël Vinot
aef8ab0411 Listen locally for 0MQ 2016-02-04 15:32:50 +01:00
Raphaël Vinot
5ca13c42eb Launch redis and leveldb from local directory 2016-02-04 15:24:39 +01:00
Raphaël Vinot
12aca6b760 Add script to import from local directory, use local python from env 2016-02-04 15:22:51 +01:00
Raphaël Vinot
315cb48117 Add template file for writing a new module 2016-02-03 10:33:42 +01:00
Raphaël Vinot
0d6adc2063 Add initial Travis file 2016-01-19 11:43:34 +01:00
Alexandre Dulaunoy
cdd0725e88 -v option added to list the path 2015-12-22 21:37:05 +00:00
Alexandre Dulaunoy
e3971ac93a Onion fetching loop deactivated by default 2014-12-22 16:06:38 +00:00
Raphaël Vinot
08ceefc375 Re-add config option 2014-12-22 16:50:25 +01:00
Raphaël Vinot
50369c6706 Revert changes on the config file due to merging messup 2014-12-22 16:29:05 +01:00
Raphaël Vinot
f717f9fe89 Merge branch 'master' of github.com:CIRCL/AIL-framework 2014-12-22 15:32:48 +01:00
Raphaël Vinot
9ee61db2cf Add hotfixes 2014-12-22 15:27:02 +01:00
Raphaël Vinot
8803c8447a Publish the fetched onions on a ZMQ feed. 2014-09-30 16:55:16 +02:00
Alexandre Dulaunoy
25757b0fff A simple feeder script feeding data from pystemon to AIL. 
The configuration matches the default Redis parameters used
in the pystemon configuration.

https://github.com/cvandeplas/pystemon/blob/master/pystemon.yaml#L16
 2014-09-19 14:03:05 +02:00
Raphaël Vinot
65b9a01644 Add config file for DomainClassifier, proper reporting 2014-09-17 17:22:56 +02:00
Alexandre Dulaunoy
27b134ec03 Add proper publisher for classified domains/hostnames 2014-09-10 09:27:47 +02:00
Raphaël Vinot
f017680365 fix onions, cc and domain classifier modules 2014-09-08 16:51:43 +02:00
Alexandre Dulaunoy
de6e21d5a7 DomainClassifier sample configuration added 2014-09-08 16:44:05 +02:00
Alexandre Dulaunoy
246621f663 First version of the DomainClassifier 2014-09-08 16:43:21 +02:00
Alexandre Dulaunoy
1397db9691 Global queue for DomainClassifier 2014-09-08 11:07:45 +02:00
Raphaël Vinot
e983c839ad Categ now listen to the Global queue 2014-09-05 17:05:45 +02:00
Raphaël Vinot
46f27ada4e More cleanup 2014-09-05 10:42:01 +02:00
Raphaël Vinot
fca00beed9 Add Domain Classifier module. 
Cleanup in the config files.
 2014-09-05 10:41:00 +02:00
Raphaël Vinot
b7c9e489c9 Fix the exceptions 2014-09-04 11:46:07 +02:00
Raphaël Vinot
9e8611a42d stop killing the disk when creating the word curve 2014-09-02 18:20:28 +02:00
Raphaël Vinot
7542eaf739 Update starting script. 2014-09-02 15:21:36 +02:00
Raphaël Vinot
0c6b09f379 Fix the onion module, log the valid onions. 2014-09-01 16:18:06 +02:00
Raphaël Vinot
f4b89669fc The onion module now fetches the URLs it finds. 2014-08-31 22:42:12 +02:00
Raphaël Vinot
abfe13436b Big refactoring, make the queues more flexible 2014-08-29 19:37:56 +02:00
Raphaël Vinot
623e876f3b Cleanup. 
* Remove useless subscriber
* Fix typo in the config file
* Update Helper accordingly
 2014-08-26 17:36:57 +02:00
Alexandre Dulaunoy
3b499a2ec8 ZMQ Publisher removed 
ZMQ Publisher removed to allow concurrent use of the scripts.
In short term, we would replace all publishing part within AIL
into pub-sub Redis to avoid ZMQ limitation.
 2014-08-26 14:38:49 +02:00
Alexandre Dulaunoy
f070ac2005 cymruwhois uses dotted decimal format 2014-08-25 10:05:36 +02:00
Raphaël Vinot
3886d1b834 Small fixes to make the refactoring production ready 
* the port for the logging is 6380
* use os.environ properly
* fix typos
 2014-08-22 17:35:40 +02:00
Raphaël Vinot
78125db4ea Use env variables everywhere 2014-08-22 14:52:02 +02:00
Raphaël Vinot
277d138a5d cleanup, add FIXME 2014-08-21 14:39:17 +02:00
Raphaël Vinot
63b29176c1 move Redis_Data_Merging to Paste 2014-08-21 12:22:07 +02:00