f3d1ca052e
Return the number of indexed documents
2014-08-11 14:50:35 +02:00
611d2a466f
Configuration that should not be there...
2014-08-11 14:24:27 +02:00
2b8f2689bf
Indexer queue and script added to "BBS-like" LAUNCH script
2014-08-11 14:06:52 +02:00
9657c6bf80
Merge branch 'master' of https://github.com/CIRCL/AIL-framework
2014-08-11 13:46:37 +02:00
b1053af3cd
Indexer module: script to query the index
...
Test script to query the index generated from the Indexer module.
python indexer_lookup.py -q Visa -q Mastercard
2014-08-11 12:03:27 +02:00
Starow
079db6f80c
Hardcoded path from ZMQ_Curve are now referring correctly in config.cfg.sample fix #6
2014-08-11 11:33:18 +02:00
7bdd4a41a5
Indexer module added - initial version with Whoosh full-text indexer
...
The indexer module indexes all the pastes using Whoosh. The module
can be extended to support additional full-text indexers in the future.
2014-08-11 11:04:09 +02:00
Starow
d1d4b2ebe0
Importing dns.exeption fix #4 fix #7
2014-08-11 09:27:50 +02:00
Starow
192074e569
Merge branch 'master' of https://github.com/CIRCL/AIL-framework
2014-08-11 09:21:09 +02:00
Starow
a5c1d59d29
Catching the exception dns.exception.Timeout fix #7
2014-08-11 09:18:55 +02:00
Starow
54091a2174
Catching the exception dns.exception.Timeout fix #4
2014-08-11 09:08:28 +02:00
Starow
eb603e8762
Fixing a bug about caching paste inside Redis :)
2014-08-08 17:23:51 +02:00
Starow
7a1db94f9e
Adding a letter (s)
2014-08-08 17:19:42 +02:00
Starow
043800287a
adding a .
2014-08-08 17:18:03 +02:00
Starow
bf682c4b44
Fixing last commit ...
2014-08-08 17:13:18 +02:00
Starow
503c23ca3b
Fixing last commit
2014-08-08 17:08:41 +02:00
Starow
c9e1eaf182
Improving cache code
2014-08-08 17:04:25 +02:00
Starow
44addf1afe
Redis cache added fix #5
...
The paste will be add in Redis during 5min and also saved on disk.
Now if a module want to get the paste for further processing, it will first try to get it in the cache
instead of getting it directly on the disk and wasting I/O.
2014-08-08 16:48:02 +02:00
Starow
97f3a3df9e
update pubsublogger with the last version
2014-08-07 14:49:34 +02:00
Starow
c10003a630
Changing ZMQ Curve Module comment
2014-08-07 14:46:43 +02:00
Starow
1379ef705a
Initial import of AIL framework - Analysis Information Leak framework
...
AIL is a modular framework to analyse potential information leak from unstructured data source like pastes from Past
ebin or similar services. AIL framework is flexible and can be extended to support other functionalities to mine sen
sitive information
2014-08-06 11:43:40 +02:00
