Commit graph

2623 commits

Author SHA1 Message Date
Starow
04a8f1bdf2 maxi cleanup old code :'( 2014-08-14 11:48:46 +02:00
Starow
7a3c216787 Adding a js library for the tag cloud (to install with update_thirdparty.sh 2014-08-14 10:04:39 +02:00
Starow
6cce9f7dac Javascript modif (Tagclouds) ... 2014-08-13 17:32:24 +02:00
Starow
29b24b6466 printing set of domain for debugging 2014-08-13 16:35:27 +02:00
Starow
663cc75369 Merge branch 'master' of https://github.com/CIRCL/AIL-framework 2014-08-13 16:33:14 +02:00
Raphaël Vinot
ece3bc173e Cleanup of main Paste module 2014-08-13 11:56:22 +02:00
Raphaël Vinot
5b17d416c8 remove script installed by pubsublogger 2014-08-13 11:55:59 +02:00
Starow
0d25b960af logs display modifications 2014-08-13 10:59:47 +02:00
Raphaël Vinot
935e51c961 Remove 3rd party code (pubsublogger), add it in the deps. 2014-08-13 10:19:43 +02:00
Starow
37033ca3a6 Minor logs modifications 2014-08-13 10:08:44 +02:00
Starow
1df2ffd094 Splitting the graph in two one for words, one for paste monitoring 2014-08-12 16:44:35 +02:00
Starow
731141c723 Padding log hours + adding some super nice bootstrap icon :) 2014-08-12 16:11:11 +02:00
Starow
5a4218035a minor javascript update 2014-08-12 15:48:53 +02:00
Starow
6aa4d7cb7d Harmonising logs messages + Changing some dygraph options 2014-08-12 15:42:16 +02:00
Starow
c3948867ad Adding an option to have better view of the curves 2014-08-12 14:51:03 +02:00
Starow
088574a048 Merge branch 'master' of https://github.com/CIRCL/AIL-framework 2014-08-12 14:01:58 +02:00
Starow
8d6c2f4288 Improving javascript and Fixing minor curves drawing bug
Everything now is fully "dynamic"
2014-08-12 14:00:00 +02:00
Jules
52ea36ee21 Merge pull request #12 from adulau/master
Indexer script updated to dump documents and find similar documents
2014-08-12 13:45:58 +02:00
0b4a80b7ea -s option added to find similar documents
By default, the index is not storing the vector of the document (Whoosh
document schema). It won't work if you don't change the schema of the
index for the content. It depends of your storage strategy.
2014-08-12 13:42:26 +02:00
fd6e1a8436 -f option added: dump full document for each match 2014-08-12 13:26:56 +02:00
6d196e5022 Merge branch 'master' of https://github.com/CIRCL/AIL-framework 2014-08-12 11:45:17 +02:00
Starow
ef6bece82c Major javascript update: Queues curves are now being set dynamically.
Now the javacript part is a little bit less disgusting than before.
2014-08-12 11:41:56 +02:00
7d396b5649 Merge branch 'master' of https://github.com/CIRCL/AIL-framework 2014-08-11 16:57:48 +02:00
Starow
60cfc9789f Adding logging proc time inside the table 2014-08-11 16:29:38 +02:00
Starow
a4302b54ad Updating the screenshot 2014-08-11 15:54:48 +02:00
Starow
e96be56b82 Checkbox default checked at true. 2014-08-11 15:30:44 +02:00
f28ad491e4 Merge branch 'master' of https://github.com/CIRCL/AIL-framework 2014-08-11 15:18:35 +02:00
Jules
802804ea6d Merge pull request #11 from adulau/master
Indexer tool - statistics added
2014-08-11 15:09:23 +02:00
e0ffdfdff5 Merge branch 'master' of https://github.com/CIRCL/AIL-framework 2014-08-11 15:09:03 +02:00
Starow
f02cb10cac Merge branch 'master' of https://github.com/CIRCL/AIL-framework 2014-08-11 15:08:16 +02:00
Starow
25cebf314c Index.html log table modification 2014-08-11 15:07:55 +02:00
0a6664ffba Indexer: Some index statistics added
usage: indexer_lookup.py [-h] [-q Q] [-n] [-t] [-l]

Fulltext search for AIL

optional arguments:
  -h, --help  show this help message and exit
  -q Q        query to lookup (one or more)
  -n          return number of indexed documents
  -t          dump top 500 terms
  -l          dump all terms encountered in indexed documents
2014-08-11 15:07:12 +02:00
Starow
7cf05f959d Javascript modification dashboard log table 2014-08-11 15:06:38 +02:00
f65a94d47b -l added -> dumping all terms indexed 2014-08-11 14:56:15 +02:00
f3d1ca052e Return the number of indexed documents 2014-08-11 14:50:35 +02:00
Jules
12dd16daf9 Merge pull request #10 from adulau/master
Whoosh requirements added for Indexer modules
2014-08-11 14:27:43 +02:00
611d2a466f Configuration that should not be there... 2014-08-11 14:24:27 +02:00
38860db284 Whoosh requirements added 2014-08-11 14:13:36 +02:00
72b232366b Whoosh added for the indexer modules 2014-08-11 14:13:09 +02:00
2b8f2689bf Indexer queue and script added to "BBS-like" LAUNCH script 2014-08-11 14:06:52 +02:00
006d4b130e Merge branch 'master' of https://github.com/CIRCL/AIL-framework 2014-08-11 13:48:32 +02:00
Jules
d48ad07145 Merge pull request #9 from adulau/master
Indexer module: script to query the index
2014-08-11 13:47:34 +02:00
9657c6bf80 Merge branch 'master' of https://github.com/CIRCL/AIL-framework 2014-08-11 13:46:37 +02:00
Starow
80fe77da26 Making bigger the graph 2014-08-11 13:46:02 +02:00
Starow
a7679787b7 Removing "Dashboard" Title to win some place for displaying infos.. 2014-08-11 13:41:58 +02:00
b1053af3cd Indexer module: script to query the index
Test script to query the index generated from the Indexer module.

python indexer_lookup.py -q Visa -q Mastercard
2014-08-11 12:03:27 +02:00
Starow
079db6f80c Hardcoded path from ZMQ_Curve are now referring correctly in config.cfg.sample fix #6 2014-08-11 11:33:18 +02:00
Jules
30640ab3c6 Merge pull request #8 from adulau/master
Indexer module added - initial version with Whoosh full-text indexer
2014-08-11 11:16:44 +02:00
Starow
edd4ea9624 Queue name tab moved to left side of the dashboard 2014-08-11 11:15:44 +02:00
7bdd4a41a5 Indexer module added - initial version with Whoosh full-text indexer
The indexer module indexes all the pastes using Whoosh. The module
can be extended to support additional full-text indexers in the future.
2014-08-11 11:04:09 +02:00