Jules
52ea36ee21
Merge pull request #12 from adulau/master
...
Indexer script updated to dump documents and find similar documents
2014-08-12 13:45:58 +02:00
0b4a80b7ea
-s option added to find similar documents
...
By default, the index is not storing the vector of the document (Whoosh
document schema). It won't work if you don't change the schema of the
index for the content. It depends of your storage strategy.
2014-08-12 13:42:26 +02:00
fd6e1a8436
-f option added: dump full document for each match
2014-08-12 13:26:56 +02:00
6d196e5022
Merge branch 'master' of https://github.com/CIRCL/AIL-framework
2014-08-12 11:45:17 +02:00
Starow
ef6bece82c
Major javascript update: Queues curves are now being set dynamically.
...
Now the javacript part is a little bit less disgusting than before.
2014-08-12 11:41:56 +02:00
7d396b5649
Merge branch 'master' of https://github.com/CIRCL/AIL-framework
2014-08-11 16:57:48 +02:00
Starow
60cfc9789f
Adding logging proc time inside the table
2014-08-11 16:29:38 +02:00
Starow
a4302b54ad
Updating the screenshot
2014-08-11 15:54:48 +02:00
Starow
e96be56b82
Checkbox default checked at true.
2014-08-11 15:30:44 +02:00
f28ad491e4
Merge branch 'master' of https://github.com/CIRCL/AIL-framework
2014-08-11 15:18:35 +02:00
Jules
802804ea6d
Merge pull request #11 from adulau/master
...
Indexer tool - statistics added
2014-08-11 15:09:23 +02:00
e0ffdfdff5
Merge branch 'master' of https://github.com/CIRCL/AIL-framework
2014-08-11 15:09:03 +02:00
Starow
f02cb10cac
Merge branch 'master' of https://github.com/CIRCL/AIL-framework
2014-08-11 15:08:16 +02:00
Starow
25cebf314c
Index.html log table modification
2014-08-11 15:07:55 +02:00
0a6664ffba
Indexer: Some index statistics added
...
usage: indexer_lookup.py [-h] [-q Q] [-n] [-t] [-l]
Fulltext search for AIL
optional arguments:
-h, --help show this help message and exit
-q Q query to lookup (one or more)
-n return number of indexed documents
-t dump top 500 terms
-l dump all terms encountered in indexed documents
2014-08-11 15:07:12 +02:00
Starow
7cf05f959d
Javascript modification dashboard log table
2014-08-11 15:06:38 +02:00
f65a94d47b
-l added -> dumping all terms indexed
2014-08-11 14:56:15 +02:00
f3d1ca052e
Return the number of indexed documents
2014-08-11 14:50:35 +02:00
Jules
12dd16daf9
Merge pull request #10 from adulau/master
...
Whoosh requirements added for Indexer modules
2014-08-11 14:27:43 +02:00
611d2a466f
Configuration that should not be there...
2014-08-11 14:24:27 +02:00
38860db284
Whoosh requirements added
2014-08-11 14:13:36 +02:00
72b232366b
Whoosh added for the indexer modules
2014-08-11 14:13:09 +02:00
2b8f2689bf
Indexer queue and script added to "BBS-like" LAUNCH script
2014-08-11 14:06:52 +02:00
006d4b130e
Merge branch 'master' of https://github.com/CIRCL/AIL-framework
2014-08-11 13:48:32 +02:00
Jules
d48ad07145
Merge pull request #9 from adulau/master
...
Indexer module: script to query the index
2014-08-11 13:47:34 +02:00
9657c6bf80
Merge branch 'master' of https://github.com/CIRCL/AIL-framework
2014-08-11 13:46:37 +02:00
Starow
80fe77da26
Making bigger the graph
2014-08-11 13:46:02 +02:00
Starow
a7679787b7
Removing "Dashboard" Title to win some place for displaying infos..
2014-08-11 13:41:58 +02:00
b1053af3cd
Indexer module: script to query the index
...
Test script to query the index generated from the Indexer module.
python indexer_lookup.py -q Visa -q Mastercard
2014-08-11 12:03:27 +02:00
Starow
079db6f80c
Hardcoded path from ZMQ_Curve are now referring correctly in config.cfg.sample fix #6
2014-08-11 11:33:18 +02:00
Jules
30640ab3c6
Merge pull request #8 from adulau/master
...
Indexer module added - initial version with Whoosh full-text indexer
2014-08-11 11:16:44 +02:00
Starow
edd4ea9624
Queue name tab moved to left side of the dashboard
2014-08-11 11:15:44 +02:00
7bdd4a41a5
Indexer module added - initial version with Whoosh full-text indexer
...
The indexer module indexes all the pastes using Whoosh. The module
can be extended to support additional full-text indexers in the future.
2014-08-11 11:04:09 +02:00
Starow
2306783882
Updating dashboard navigation menu list
2014-08-11 10:48:03 +02:00
Starow
f3eb02c5b7
Updating installing script + adding another required package to the list
2014-08-11 10:41:50 +02:00
Starow
5bc0f154f9
Update of installing_deps.sh and pip_requirement.txt
2014-08-11 09:36:28 +02:00
Starow
d1d4b2ebe0
Importing dns.exeption fix #4 fix #7
2014-08-11 09:27:50 +02:00
Starow
192074e569
Merge branch 'master' of https://github.com/CIRCL/AIL-framework
2014-08-11 09:21:09 +02:00
Starow
a5c1d59d29
Catching the exception dns.exception.Timeout fix #7
2014-08-11 09:18:55 +02:00
Starow
54091a2174
Catching the exception dns.exception.Timeout fix #4
2014-08-11 09:08:28 +02:00
Starow
eb603e8762
Fixing a bug about caching paste inside Redis :)
2014-08-08 17:23:51 +02:00
Starow
7a1db94f9e
Adding a letter (s)
2014-08-08 17:19:42 +02:00
Starow
043800287a
adding a .
2014-08-08 17:18:03 +02:00
Starow
bf682c4b44
Fixing last commit ...
2014-08-08 17:13:18 +02:00
Starow
503c23ca3b
Fixing last commit
2014-08-08 17:08:41 +02:00
Starow
c9e1eaf182
Improving cache code
2014-08-08 17:04:25 +02:00
Starow
0a503db815
Merge branch 'master' of https://github.com/CIRCL/AIL-framework
2014-08-08 16:52:42 +02:00
Starow
44addf1afe
Redis cache added fix #5
...
The paste will be add in Redis during 5min and also saved on disk.
Now if a module want to get the paste for further processing, it will first try to get it in the cache
instead of getting it directly on the disk and wasting I/O.
2014-08-08 16:48:02 +02:00
b21e7ea3c8
Merge branch 'master' of github.com:CIRCL/AIL-framework
2014-08-08 14:03:30 +02:00
81ea9cfb3c
Clarification for the env variable
2014-08-08 14:03:00 +02:00
