Commit graph

2800 commits

Author SHA1 Message Date
Raphaël Vinot
277d138a5d cleanup, add FIXME 2014-08-21 14:39:17 +02:00
Raphaël Vinot
63b29176c1 move Redis_Data_Merging to Paste 2014-08-21 12:22:07 +02:00
Raphaël Vinot
50cfac857e Update config
Make all paths in the config file relative to the home directory.
2014-08-20 16:00:56 +02:00
Raphaël Vinot
a68f5b6a0e fix subscriber names, update default config 2014-08-20 15:54:21 +02:00
17a9185404 Merge pull request #19 from adulau/master
Redis explanation added
2014-08-20 15:35:16 +02:00
5096230f05 Markdown fixed 2014-08-20 15:33:26 +02:00
a030f939d9 Overview of Redis database added 2014-08-20 15:31:10 +02:00
Raphaël Vinot
2485ba5df2 Merge remote-tracking branch 'origin/master' into testing
Conflicts:
	bin/ZMQ_Sub_Urls.py
2014-08-20 15:24:10 +02:00
Raphaël Vinot
cf814468e7 Merge branch 'rewrite_zmq' into testing
Conflicts:
	bin/indexer_lookup.py
2014-08-20 15:18:02 +02:00
Raphaël Vinot
99c8cc7941 completely remove ZMQ_PubSub.py 2014-08-20 15:14:57 +02:00
a84ffc9ef7 Merge pull request #18 from adulau/master
Fix one assignment before use and remove duplicate call to libmagic
2014-08-20 10:25:57 +02:00
1d64dc44c8 MIME type guessing - removed one duplicate call to libmagic 2014-08-20 10:22:33 +02:00
Raphaël Vinot
8d9ffbaa53 Do not create a ZMQ sub if it is not required. 2014-08-19 19:53:33 +02:00
Raphaël Vinot
45b0bf3983 Improve the cleanup. Still some to do. 2014-08-19 19:07:07 +02:00
Raphaël Vinot
f1753d67c6 Cleanup the queues. 2014-08-19 16:05:37 +02:00
e8fcea6cd6 Remove undeclared variable 2014-08-18 16:17:36 +02:00
7d8ee102a3 Assignment before use (if Enumerate fails) 2014-08-18 15:58:06 +02:00
Raphaël Vinot
ebef13fa5b Merge branch 'master' of github.com:CIRCL/AIL-framework 2014-08-18 13:40:22 +02:00
Raphaël Vinot
3f066ddaec fix install script 2014-08-18 13:40:07 +02:00
e01ac78538 Merge pull request #15 from adulau/master
Configuration path fixed
2014-08-18 13:23:21 +02:00
4304c6858e Configuration path fixed 2014-08-18 09:02:08 +02:00
Raphaël Vinot
078c8ea836 Big cleanup, pep8 2014-08-14 18:07:18 +02:00
Jules
715adb4546 Merge pull request #14 from adulau/master
Requirements updated for CC/ASN lookup in URL module
2014-08-14 15:07:40 +02:00
0d23a3952c Requirements updated for CC/ASN lookup in URL module 2014-08-14 14:56:33 +02:00
Jules
ab6765315e Merge pull request #13 from adulau/master
Log where URLs are hosted - cc_critical option added
2014-08-14 14:28:01 +02:00
762def3a23 Log where URLs are hosted - cc_critical option added
It logs where the hostname of the URL is hosted (ASN and geographic location).
A simple option cc_critical added to set the country code to log as critical.
2014-08-14 14:22:11 +02:00
Raphaël Vinot
4a1f300a1a Cleanup (remove unused imports, more pep8 compatible) 2014-08-14 14:11:07 +02:00
Starow
04a8f1bdf2 maxi cleanup old code :'( 2014-08-14 11:48:46 +02:00
Starow
7a3c216787 Adding a js library for the tag cloud (to install with update_thirdparty.sh 2014-08-14 10:04:39 +02:00
Starow
6cce9f7dac Javascript modif (Tagclouds) ... 2014-08-13 17:32:24 +02:00
Starow
29b24b6466 printing set of domain for debugging 2014-08-13 16:35:27 +02:00
Starow
663cc75369 Merge branch 'master' of https://github.com/CIRCL/AIL-framework 2014-08-13 16:33:14 +02:00
Raphaël Vinot
ece3bc173e Cleanup of main Paste module 2014-08-13 11:56:22 +02:00
Raphaël Vinot
5b17d416c8 remove script installed by pubsublogger 2014-08-13 11:55:59 +02:00
Starow
0d25b960af logs display modifications 2014-08-13 10:59:47 +02:00
Raphaël Vinot
935e51c961 Remove 3rd party code (pubsublogger), add it in the deps. 2014-08-13 10:19:43 +02:00
Starow
37033ca3a6 Minor logs modifications 2014-08-13 10:08:44 +02:00
Starow
1df2ffd094 Splitting the graph in two one for words, one for paste monitoring 2014-08-12 16:44:35 +02:00
Starow
731141c723 Padding log hours + adding some super nice bootstrap icon :) 2014-08-12 16:11:11 +02:00
Starow
5a4218035a minor javascript update 2014-08-12 15:48:53 +02:00
Starow
6aa4d7cb7d Harmonising logs messages + Changing some dygraph options 2014-08-12 15:42:16 +02:00
Starow
c3948867ad Adding an option to have better view of the curves 2014-08-12 14:51:03 +02:00
Starow
088574a048 Merge branch 'master' of https://github.com/CIRCL/AIL-framework 2014-08-12 14:01:58 +02:00
Starow
8d6c2f4288 Improving javascript and Fixing minor curves drawing bug
Everything now is fully "dynamic"
2014-08-12 14:00:00 +02:00
Jules
52ea36ee21 Merge pull request #12 from adulau/master
Indexer script updated to dump documents and find similar documents
2014-08-12 13:45:58 +02:00
0b4a80b7ea -s option added to find similar documents
By default, the index is not storing the vector of the document (Whoosh
document schema). It won't work if you don't change the schema of the
index for the content. It depends of your storage strategy.
2014-08-12 13:42:26 +02:00
fd6e1a8436 -f option added: dump full document for each match 2014-08-12 13:26:56 +02:00
6d196e5022 Merge branch 'master' of https://github.com/CIRCL/AIL-framework 2014-08-12 11:45:17 +02:00
Starow
ef6bece82c Major javascript update: Queues curves are now being set dynamically.
Now the javacript part is a little bit less disgusting than before.
2014-08-12 11:41:56 +02:00
7d396b5649 Merge branch 'master' of https://github.com/CIRCL/AIL-framework 2014-08-11 16:57:48 +02:00