By default, the index is not storing the vector of the document (Whoosh
document schema). It won't work if you don't change the schema of the
index for the content. It depends of your storage strategy.
usage: indexer_lookup.py [-h] [-q Q] [-n] [-t] [-l]
Fulltext search for AIL
optional arguments:
-h, --help show this help message and exit
-q Q query to lookup (one or more)
-n return number of indexed documents
-t dump top 500 terms
-l dump all terms encountered in indexed documents
The paste will be add in Redis during 5min and also saved on disk.
Now if a module want to get the paste for further processing, it will first try to get it in the cache
instead of getting it directly on the disk and wasting I/O.
AIL is a modular framework to analyse potential information leak from unstructured data source like pastes from Past
ebin or similar services. AIL framework is flexible and can be extended to support other functionalities to mine sen
sitive information
