858 commits

Author	SHA1	Message	Date
Terrtia	823c33174a	Merge branch 'master' into onion_crawler	2018-08-27 11:20:51 +02:00
Terrtia	d42dd118a4	chg: [Domain crawled] add random screenshot	2018-08-27 11:02:39 +02:00
Terrtia	6c981ea5e9	fix: [HashDecoded] typo	2018-08-24 11:58:54 +02:00
Terrtia	ab452841b1	fix: [HashDecoded] change template title	2018-08-24 11:51:16 +02:00
Terrtia	e6b583ee7a	chg: [Hash] add Help on graph node	2018-08-24 11:40:09 +02:00
Terrtia	a586548047	Merge branch 'master' into base64	2018-08-24 10:35:28 +02:00
Terrtia	7e24943537	chg: [Crawler] crawler accept all kind of domains	2018-08-24 10:13:56 +02:00
Terrtia	e9580d6775	chg: [Crawler] change BDD, save i2p links	2018-08-21 15:54:53 +02:00
Terrtia	ed559d9f4a	chg: [Showpaste] add screenshot + improve onion db	2018-08-16 17:24:39 +02:00
Terrtia	54cc4f3723	fix: [UI-Showpaste] show raw paste, force text/plain mimetype	2018-08-03 09:15:34 +02:00
Terrtia	911cff70da	chg: [UI] add AIL icon, fix li links	2018-08-02 10:44:47 +02:00
Terrtia	deaebaa4fd	fix: [UI-Hash] pie chart json	2018-07-27 13:51:29 +02:00
Terrtia	009f87ee10	[fix: [UI-hash] fix graph node zoom	2018-07-27 11:27:41 +02:00
Terrtia	b02cbee179	chg: [UI-dashboard] add logs number configuration	2018-07-26 11:35:54 +02:00
Terrtia	0a82482db4	chg: [UI-dashboard] convert time to local	2018-07-26 10:34:43 +02:00
Terrtia	5ad5939bd5	chg: display last logs	2018-07-25 16:48:44 +02:00
Terrtia	03a37af5cf	chg: [Hash] add reprocess regex order	2018-07-25 10:47:36 +02:00
Terrtia	8c97c70a3f	Merge branch 'master' into base64	2018-07-24 15:21:32 +02:00
Terrtia	4b8c26dec8	fix: [hive auto export] fix #221	2018-07-24 14:46:35 +02:00
Terrtia	3479923f96	fix: [UI-Hash] pie decoder	2018-07-23 11:43:48 +02:00
Terrtia	9c03c40482	chg: [UI-Hash] add decoder pie	2018-07-23 11:11:52 +02:00
Terrtia	1fbd4109c2	chg: [UI-Hash] add decoded type on graph	2018-07-20 15:48:56 +02:00
Terrtia	bd5f83f0eb	chg: refractor base64 encoded to hash	2018-07-20 10:32:52 +02:00
Terrtia	e25dd52d64	add: Binary decoder	2018-07-18 11:45:19 +02:00
Terrtia	524d5fe604	add: hash download	2018-07-18 10:09:35 +02:00
Terrtia	d9c924c38d	chg: use json to submit hash	2018-07-17 14:23:49 +02:00
Terrtia	412b012ddf	add: hash line_graph (nb/day)	2018-07-16 16:45:36 +02:00
Terrtia	6f69da0c0d	fix: paste submit css	2018-07-16 10:32:41 +02:00
Terrtia	fce324e370	fix: release tiemeout alarm, chg:icon graph node	2018-07-13 11:54:14 +02:00
Terrtia	87b7facba8	add: base 64 node graph	2018-07-12 17:07:17 +02:00
Alexandre Dulaunoy	9dc211dbe9	chg: [UI] Typo fixed as requested by our favourite German.	2018-07-10 10:49:02 +02:00
Terrtia	fd02085495	add: show hash info	2018-07-09 17:07:59 +02:00
Terrtia	c2a976e907	add: stacked barchart, resize chart	2018-07-09 11:47:04 +02:00
Terrtia	0e5a7d8d47	add date_range stacked bar	2018-07-06 16:54:43 +02:00
Terrtia	3b4e2a18aa	fix: date format https://github.com/CIRCL/AIL-framework/issues/219	2018-07-05 14:45:34 +02:00
Terrtia	cd154be6f9	fix: term_management UI bug ,https://github.com/CIRCL/AIL-framework/issues/214	2018-07-05 14:12:47 +02:00
Terrtia	f7d38bea16	add: base64 date and type selector	2018-07-05 13:58:40 +02:00
Terrtia	b6d855b963	change import d3.js to d3.min.js	2018-06-29 15:58:24 +02:00
Terrtia	adfc8cf260	add d3 and daterange js	2018-06-29 15:52:13 +02:00
Terrtia	a0a639276c	fix submit paste	2018-06-29 15:29:02 +02:00
Terrtia	98d0235fc3	Merge branch 'master' into base64	2018-06-29 14:57:46 +02:00
Terrtia	5908710b34	fix: update submitted tags link	2018-06-29 11:29:52 +02:00
Terrtia	cb807bf2c6	chg: change base64 icon	2018-06-29 11:28:35 +02:00
Terrtia	16c700ddb4	new: add base64 display on paste + base64 statistics	2018-06-29 10:02:29 +02:00
Terrtia	225776c69c	fix: tp and fp, errors messages on paste submission	2018-06-20 10:02:26 +02:00
Terrtia	6a89fac309	fix hive and misp error	2018-06-19 16:39:49 +02:00
Terrtia	af9db53171	add tag fp and tp	2018-06-19 15:09:26 +02:00
Terrtia	bf72c27711	fix path-event id map	2018-06-19 13:43:26 +02:00
Terrtia	cd4c099685	fix db init	2018-06-19 11:54:34 +02:00
Terrtia	103787a07d	fix	2018-06-19 11:43:16 +02:00
Terrtia	bc40bb3c38	fix hive connection + add keys sample	2018-06-19 11:42:06 +02:00
Terrtia	0ee36e484f	fix hive connection + submitted dir creation	2018-06-19 11:31:30 +02:00
Terrtia	634bf6e552	add img + fix upload api	2018-06-18 17:16:22 +02:00
Terrtia	704e6f0ad1	change default submit tag, event-path map, publish events	2018-06-18 13:58:31 +02:00
Terrtia	aee1b50ef1	fix duplicate export attribute + tag export whitelist	2018-06-15 17:25:43 +02:00
Terrtia	31fc33ab03	misp event creation, auto and manual + the hive alert auto creation, manual case creation	2018-06-14 16:51:06 +02:00
Terrtia	8522d50308	modify navbar + decompress file password	2018-06-11 10:21:11 +02:00
Terrtia	c64588f60e	add feature, user can submit paste on the web interface	2018-06-08 16:49:20 +02:00
Terrtia	8e9c77b364	verify file upload extention	2018-06-06 10:05:25 +02:00
Terrtia	6143bc3dce	submit users input paste	2018-06-05 16:58:04 +02:00
Terrtia	c41a56e4a2	clean Flask_tags + gitignore	2018-06-01 13:36:08 +02:00
Terrtia	2b310d4367	clean	2018-06-01 11:26:45 +02:00
Terrtia	cd6734f704	fix install	2018-05-31 15:35:20 +02:00
Terrtia	884d655b0c	taxonomies and galaxies update via LAUNCH	2018-05-31 13:53:29 +02:00
Terrtia	5c50de45de	add taxonomies and galaxies update and install	2018-05-31 10:07:23 +02:00
Terrtia	8a87b7b082	galaxy tag info + fix	2018-05-30 16:18:58 +02:00
Terrtia	dbe67e094b	galaxies synonyms	2018-05-29 17:17:02 +02:00
Terrtia	f3f6b76428	galaxy description and tags	2018-05-29 08:47:46 +02:00
Terrtia	f5cae0d99c	taxonomie + add tags + tags display	2018-05-23 16:58:56 +02:00
Terrtia	b9eb3ed9ba	add and delete tags features	2018-05-17 15:33:06 +02:00
Terrtia	4fbbf8bbd2	display tag on important paste and search	2018-05-17 11:00:05 +02:00
Terrtia	0d164b6f4c	add tag js and css	2018-05-16 14:48:15 +02:00
Terrtia	b1b7d33c96	tags	2018-05-16 14:39:01 +02:00
Terrtia	f66a528bc2	fix Duplicate, save list of duplicates on disk + prevent empty hash creation	2018-05-15 23:28:47 +02:00
Terrtia	225fe76c96	fix duplicate + get raw paste	2018-05-11 16:07:30 +02:00
Terrtia	142c1dc9b0	clean	2018-05-09 13:03:46 +02:00
Terrtia	3841c159a0	change leveldb to ardb	2018-05-07 14:50:40 +02:00
Terrtia	c17b9402f7	clean	2018-05-04 14:25:47 +02:00
Terrtia	96a3df42bc	decode with redis connection	2018-05-04 13:53:29 +02:00
Terrtia	a900d5f08d	add python 2 to 3 bash + fix Duplicate + clean	2018-05-03 16:21:33 +02:00
Terrtia	c52caebe7c	improve Duplicate + tlsh + add time out handler + debug + clean	2018-05-02 17:07:10 +02:00
Terrtia	c3428df5dc	add apiKeys module	2018-04-26 14:42:39 +02:00
Terrtia	8738b7cf75	fix track term and avoid duplicate mail address	2018-04-24 16:44:37 +02:00
Terrtia	f93fe9aeb2	add bitcoin module	2018-04-23 14:51:53 +02:00
Terrtia	41eb250278	fix python 3 compqtibility issues	2018-04-20 10:42:19 +02:00
Terrtia	8571a86344	frontend python 3.5 upgrade	2018-04-17 16:06:32 +02:00
Philipp Schmied	4835589ff4	Fixed typo	2018-03-01 08:50:27 +01:00
Philipp Schmied	29608ab29b	Fixed e-mail notificatins for sets and regexes	2018-02-28 11:31:16 +01:00
Philipp Schmied	b6fcf51309	Flask_terms.py: Fixed mapping of notifcation e-mails to terms with uppercase letters	2018-02-28 09:19:27 +01:00
Philipp Schmied	5b1f0b0212	Implemented email notifications (bin: config.cfg additions, email sending via analyzer scripts; var: Changes to add notifications via terms management); terms_management: Fixed click handlers not being added to all tracked terms.	2018-02-27 15:12:02 +01:00
Alexandre Dulaunoy	abdf41fe23	Merge pull request #164 from mokaddem/showDiff ... Show differences between duplicates	2018-02-21 22:25:16 +01:00
Sami Mokaddem	9125930a77	fix: canvasjs changed its folder hierarchy	2017-12-14 11:47:11 +01:00
Sami Mokaddem	304213cfa4	update: Possibility to change the authorized max line length in the config for the diff of duplicates	2017-12-12 11:00:26 +01:00
Sami Mokaddem	3fe7ecf75a	feature: diff in the web interface for duplicates pastes.	2017-12-12 10:48:21 +01:00
mokaddem	9cab76cf88	Merge pull request #162 from mokaddem/sentimentBugfix ... Fix: bug between front-end and back-end date representation	2017-12-11 16:35:38 +01:00
Paul A	4a4a1d478b	Quick refactor for the JavaScript	2017-10-22 18:48:54 +02:00
Paul A	f6b6777095	Fixed some code for the JavaScript (coding-style)	2017-10-20 15:44:09 +02:00
Paul A	48fc8e3464	Removed duplicate function	2017-10-20 15:32:58 +02:00
Simon Tran	a41c5d55c5	Migrated Dockerfile to Ubuntu 16.04 LTS ... Fixed AIL Web dependencies installation Added Docker documentation	2017-10-01 02:23:32 +02:00
Sami Mokaddem	2cfb8c173b	Fixed from front-end date representation bug	2017-09-27 10:46:20 +02:00
=	b9bd651033	Removed useless import	2017-08-23 14:06:59 +02:00
=	3b3f3aa89c	Flask: Added support of LevelDB database depending of the year	2017-08-23 12:20:39 +02:00
=	d632335760	Automatically start redis_levelDB with the correct date	2017-08-23 10:16:56 +02:00
=	bd46c83fb6	BrowseImportantPaste: Fixed a bug where onClickListener was not set when the table was only 1 page long	2017-08-21 11:55:01 +02:00
=	409eb25a64	Deleted data-flow graph and header.html from the index + Added fixed typo where 10min was in fact 3min	2017-08-21 10:51:56 +02:00
Mokaddem	fa52292029	Added comments, cleanup and added credentials search progress in Flask	2017-07-20 10:50:24 +02:00
Mokaddem	cbb2909156	Added comments	2017-07-20 10:24:48 +02:00
Mokaddem	05417f405c	Finished basic link web interface + solved some bugs	2017-07-20 10:04:30 +02:00
Mokaddem	ef741f85c0	Added similarity checks + started link webInterface	2017-07-19 11:52:06 +02:00
Mokaddem	882ab88b00	Draft module + web link	2017-07-18 16:57:15 +02:00
Mokaddem	9b8570e40b	Dirty draft webpage credential_tracking	2017-07-17 17:26:19 +02:00
Mokaddem	f1a1e75c32	Restaured old example webpage	2017-07-17 16:18:21 +02:00
Mokaddem	9558aafe67	Prevent the loss of the 1 message in import_dir.py	2017-07-17 16:14:27 +02:00
Mokaddem	b60e164380	Added support of onion in browseWarningPaste	2017-04-26 11:47:02 +02:00
Mokaddem	5e4d0170d4	Bug fixes, jinja2 iter0 and typo	2017-04-26 11:31:08 +02:00
Mokaddem	e614da4748	Deleted print + Added curve fill for Mixer graph in Index	2017-04-25 15:52:41 +02:00
Mokaddem	71b2757ed2	New script create_new_web_module permits to rapidly create a new pre-configured webpage from a template.	2017-04-25 15:22:04 +02:00
Mokaddem	6afbf77af0	Modified skeleton template name into rawSkeleton + Added possibility to hide modules in the header.	2017-04-25 12:18:08 +02:00
Mokaddem	f6e8582d9d	Auto organisation of the navbar -> Web page are auto-inserted. Created skeleton webpage	2017-04-19 15:14:20 +02:00
Mokaddem	32906965d8	Draft of new organisation of the web interface. Usage of Flask's blueprint	2017-04-19 11:02:03 +02:00
Mokaddem	da081bae42	Merge branch 'regexInTerms' into production	2017-04-18 16:59:40 +02:00
Mokaddem	c8baabd882	Updated TermsFrequency related modules + Fixed bugs (encoding, behaviors, ...)	2017-04-18 15:28:21 +02:00
Mokaddem	2da4c572c7	New feature in Terms: regex and set of words support (draft)	2017-03-28 17:42:44 +02:00
Mokaddem	3261eca836	Fixed all_index newline bug + added print of what's happening	2017-03-15 14:29:49 +01:00
Mokaddem	5b7d047622	Changed os.system by os.mkdir + fixed bug during rotation	2017-03-15 14:05:13 +01:00
Mokaddem	4005b72180	Added backward support + renamed index with timestamp	2017-03-15 11:51:35 +01:00
Mokaddem	381e72ee99	Added display of number of elements inside the index + changed variables names	2017-03-15 10:07:46 +01:00
Mokaddem	e4757f5ceb	Added possibility to choose the index in search + Updated search page with jinja2 iter0	2017-03-15 09:39:48 +01:00
Mokaddem	41132fe0bc	Added support of per_paste for term-management and plot-term	2017-02-28 15:54:39 +01:00
Mokaddem	4071bf2e91	fixed a bug in manage_term (usage of loop.index0) + fixed bug in top_set where graph was not displaying the correct per_paste value	2017-02-28 15:01:48 +01:00
Mokaddem	25adae0104	Added possibility to select in top_terms the data per_paste or not (need module testing)	2017-02-15 16:29:02 +01:00
Mokaddem	d4da3a316f	Added path of the paste in the log of Categ.py	2017-02-14 10:59:47 +01:00
Mokaddem	29241eb49f	Added a line separator between the two graphs in dashboard.	2017-01-09 14:44:14 +01:00
Mokaddem	0e39f516a9	Added sleep in helper for multiple listening interfaces (reduce useless work), Added a simple way to pre-process feed before sending it to the global feed.	2017-01-09 14:12:26 +01:00
Mokaddem	24c5621d29	BUG FIX in Flask_trendingmodule, Fixed member_set when get_top_relevant_data is None.	2016-12-26 16:33:10 +01:00
Mokaddem	1c2169d3bc	Improved interface for multiple feeds and refresh_script.	2016-12-26 16:16:44 +01:00
Mokaddem	a18c046dea	Fixed bug multiple refresh instances and harmonized interface.	2016-12-23 16:15:05 +01:00
Mokaddem	03dddbc359	Reduced refresh rate of processed_pastes, synchro graphs and adjusted max on each graphs.	2016-12-23 16:01:30 +01:00
Mokaddem	97292e0899	Updated web interface to handle new mixer module. and fixed one dependency bug.	2016-12-23 15:44:46 +01:00
Mokaddem	faa8f8302d	Removed typo in the number of remaining pastes to display in browseImportantPaste	2016-12-22 09:28:55 +01:00
Mokaddem	9df12cdae8	Added sub-flask in a directory	2016-12-22 08:57:45 +01:00
Mokaddem	d30f3ca6c7	Added missing depency datetime in dashboard	2016-12-09 13:53:57 +01:00
Mokaddem	0054be5bab	Added missing json dependency in dashboard	2016-12-09 08:50:36 +01:00
Mokaddem	8daa72789e	Splitted Flask_server into module related to website sections	2016-12-09 08:46:37 +01:00
Mokaddem	d3ed1bbf40	Merge branch 'website-improvement' into production	2016-12-08 10:05:55 +01:00
Mokaddem	1abba4dcf9	Added support of re-plotting the plotted terms	2016-12-08 08:44:10 +01:00
Mokaddem	84dcd5586f	Started plotting same data with different dates	2016-11-25 16:42:22 +01:00
Mokaddem	a3255d168c	ModuleInfo correctly handle CurveManageTopSets, ... Changed queue coloring in dashboard, ModuleInformation now have a history of executed command and better handle module killing.	2016-11-25 11:54:16 +01:00
Mokaddem	5d269ea1ee	Added date in Duplicate for better distinguish duplicate. Also, added a timeout for the dataTable in duplicate inside the modal.	2016-11-24 15:05:29 +01:00
Mokaddem	df8d978088	Added dynamic data loading in dataTable in search.html	2016-11-24 13:31:31 +01:00
Mokaddem	80ca49f018	Prevent browser to request reconnection for the eventStream + Added dynamic data loading in the dataTable of BrowseImportantPastes	2016-11-11 16:51:39 +01:00
Mokaddem	768fb6f2df	removed server overhead for paste preview data	2016-11-10 16:41:07 +01:00
Mokaddem	699c819387	fixed bug in show concerned pastes previex en (terms managements)	2016-11-10 16:33:05 +01:00
Mokaddem	94e3034e7d	Fixed bun in terms management, where action_button was not bind in other dataTable pages	2016-11-10 15:53:22 +01:00
Mokaddem	7763bfb4c7	Pastes dataTable now display only the latest clicked paste even if there were discarded queries	2016-11-10 15:39:45 +01:00
Mokaddem	2dd69fbb25	Fixed modal background stacking bug + button leading to saved paste bug fixed in search	2016-11-10 14:56:25 +01:00
Mokaddem	317005c2b4	tryed to fix html display problem in tooltip + fixed problem button showPaste in modal not pointing to the paste	2016-11-10 10:30:29 +01:00
Mokaddem	34221b083f	Partially solved the browser freezing problem + tootlip is also binded at a new page generation of the DataTable	2016-11-10 09:28:55 +01:00
Mokaddem	acdd1367a3	Added event source for getImportantPaste	2016-10-28 16:55:56 +02:00
Mokaddem	c95000866d	Dynamic table in search now load all the data + fixed bugs where tooltip where not shown on other than the first page displayed and ... tooltip interpret html (not supposed to...)	2016-10-28 14:21:08 +02:00
Mokaddem	f51808d914	Added support of html formatting in search result dynamic loading table	2016-10-28 09:48:22 +02:00
Mokaddem	4be62a4e2d	Added draft of dynamic update table with search result	2016-10-28 09:28:42 +02:00
Mokaddem	340624fe1e	Fixed bug in modal show paste (show more button not showing)	2016-10-27 16:18:49 +02:00
Mokaddem	7e7e679ab6	Added dataTable for duplicate in show saved paste	2016-10-27 15:53:45 +02:00
Mokaddem	59a7000527	Bootstrap support in show saved paste	2016-10-27 12:38:39 +02:00
Mokaddem	ac254e0e63	Added support of browsing concerned paste in dashboard for all modules + Show in the page what was the concerned module	2016-10-27 11:50:24 +02:00
Mokaddem	1826b170ec	Added support of local paste view in dashboard for Mails event only	2016-10-27 11:27:26 +02:00
Mokaddem	7c8d414948	modified provider name into no data when there is no data	2016-10-14 16:47:22 +02:00
Mokaddem	91678179fd	Added msgs when queues or feed is not running/coming	2016-10-14 16:33:54 +02:00
Alexandre Dulaunoy	2781a2f0b7	Stuck not stucked	2016-09-21 08:35:20 +02:00
Jerome St-Hilaire	ea13f87c2c	Fixed dependency fetching for the web interface	2016-09-15 12:14:55 -07:00
Raphaël Vinot	acfe9c1336	Update Flask_server.py	2016-08-26 18:06:48 +02:00
Raphaël Vinot	3290b51c91	Update update_thirdparty.sh	2016-08-26 18:05:18 +02:00
Mokaddem	79be8ab934	Fixed introduced bug relative to chart in index webpage.	2016-08-26 15:59:53 +02:00
Mokaddem	0c760d763b	Added support of PID in ModuleInformation and index.html	2016-08-26 15:27:37 +02:00
Mokaddem	5c565068a6	Removed auto-reload in dashboard if window in undefined	2016-08-26 11:25:11 +02:00
Mokaddem	181c372131	Updated dependencies	2016-08-26 11:10:39 +02:00
Mokaddem	4cb3b672be	Added comments	2016-08-26 10:28:35 +02:00
Mokaddem	86b655100e	Deleted useless parsing function	2016-08-26 10:01:14 +02:00
Mokaddem	07d7559060	Added dataTable support in browse paste modal.	2016-08-26 08:30:09 +02:00
Mokaddem	813cc7031a	Changed lots of code into zrank	2016-08-25 17:48:12 +02:00
Mokaddem	991176fdfd	Suppressed metismenu import.	2016-08-25 16:40:13 +02:00
Mokaddem	fb22816236	Added module number info in index webpage	2016-08-25 16:27:32 +02:00
Mokaddem	0b5a3e0cd2	Auto generate graph div + possibility to query all sentiments data.	2016-08-25 15:06:18 +02:00
Mokaddem	21f2c43572	Individualized sentiments-graph loading	2016-08-25 13:11:16 +02:00
Mokaddem	f9bf00f022	Removed autokill feature if not specified	2016-08-25 11:09:31 +02:00
Mokaddem	611454ba82	Added color in web-index reprenting modules states.	2016-08-24 18:00:05 +02:00
Mokaddem	e3daa84196	Added icon in header	2016-08-23 16:09:34 +02:00
Mokaddem	e12244fb53	Added show/hide curve with legend in terms-top-set.	2016-08-23 15:39:13 +02:00
Mokaddem	e56187eb7e	Added position of each top term in the other sets + highligth ones not present.	2016-08-23 14:52:02 +02:00
Mokaddem	a64d447b7b	Auto hide button add_to_chart when no data plotted.	2016-08-23 12:09:41 +02:00
Mokaddem	e59b7c4e47	Added tooltip in graph for terms-plot-tool	2016-08-23 12:03:23 +02:00
Mokaddem	0eba419651	Added multiple terms plot in terms-plot-tool.	2016-08-23 11:56:32 +02:00
Mokaddem	9f2d672ec8	harmonized tooltip and popover display	2016-08-23 11:20:48 +02:00
Mokaddem	8c956c22b1	Added autohide curves in terms-top-set.	2016-08-23 11:02:36 +02:00
Mokaddem	32dfd2b181	Moved legend place in term-plot-top	2016-08-23 10:16:33 +02:00
Mokaddem	f9b27e4013	Fixed bug in curve preventing to track concerned_pastes. Now all tracked words are put in lower case.	2016-08-23 09:50:03 +02:00
Mokaddem	1766ced9e6	Fixed potential XSS problem with pastes display in 'search' and 'important_paste_by_module'.	2016-08-23 09:08:17 +02:00
Mokaddem	047331df3e	Added save of paste_baseName in set for searches. This is done in ModuleStats to not surcharge the Global module.	2016-08-23 08:52:54 +02:00
Mokaddem	6be557890c	Replaced negative hour by 24h based.	2016-08-23 07:19:51 +02:00
Mokaddem	85ab0e36ff	In sentiment analysis. if today top provider set is empty, try to take yesterday data and then takes from all_providers.	2016-08-23 07:11:03 +02:00
Mokaddem	8359a6668f	Suppressed useless comms	2016-08-22 21:36:19 +02:00
Mokaddem	2fd2c8af39	Removed commented code	2016-08-22 20:59:56 +02:00
Mokaddem	27ae577e00	Fixed stupid line remove in index.html...	2016-08-22 16:25:52 +02:00
Mokaddem	a424069c64	Fixed bug in flask with zrange (was taking the bottom of the set instead of the top leading to get only the min instead of max)	2016-08-22 16:12:20 +02:00
Mokaddem	003c72bd7c	Fixed bug with redis where the inserted key was too long, causing a crash + fixed bug taking min instead of max in terms-top-set	2016-08-22 16:01:42 +02:00
Mokaddem	2466f355ab	Fixed bug in manage-terms-top-set + added auto sorting in important-paste-by-module	2016-08-22 14:54:37 +02:00
Mokaddem	e144e8e1c6	Fixed bug in terms-management-interface (only display one paste) + removed get_language of all pastes due to the huge overhead.	2016-08-22 13:59:12 +02:00
Mokaddem	d3c0fc5f94	Added terms trending module and web interface: ... - Curve module has been rewritten - Added new module curve_manage_top_set - Added all web-pages	2016-08-22 13:35:49 +02:00
Mokaddem	841e87eb16	Fixed bug in sentiment trending, if there was no data for the avg -> there was one NaN leading to no display in the gauge.	2016-08-20 18:48:24 +02:00
Mokaddem	ee8f26c7c4	In sentiment trending, if there are no providers in the today\'s set, get data from random providers	2016-08-20 18:14:19 +02:00
Mokaddem	2ed60cb9bf	Fixed typo	2016-08-20 16:44:36 +02:00
Mokaddem	3f1cf705fc	Deleted alert()	2016-08-20 16:41:49 +02:00
Mokaddem	1c96eac2e4	Enter simulate button click on input field for terms management and plot tool	2016-08-20 16:38:51 +02:00
Mokaddem	465d18fc73	Added terms-plot tool	2016-08-20 16:23:06 +02:00
Mokaddem	fb3dea6bd2	Finnished terms-plot-top-set. Also added possibility to add terms to track or blacklist.	2016-08-20 15:59:22 +02:00
Mokaddem	a5117d7cad	Added top set webpage	2016-08-20 11:46:22 +02:00
Mokaddem	0b6828fb16	Correctly handle and display add or remove a term in term-frequency + started sketch of terms-plot tool.	2016-08-19 16:53:46 +02:00
Mokaddem	8d41b109e7	Added query of the data for the new tracked word	2016-08-19 14:33:01 +02:00
Mokaddem	b78effe779	Added draft of terms_management + Added copyright in readme	2016-08-19 13:34:02 +02:00
Mokaddem	52b7124a6b	- Modified redis connection (from levelDB to redis). ... - Added term frequency in curve. - Modified ModuleStats and WebStats to use redis special command (incrby instead of get-set and zset) and Flask to perform the correct queries. - Added panel color in sentiment trending	2016-08-18 15:34:19 +02:00
Mokaddem	24072ece36	Added support of pasteName in search + added header in search.html + supressed mixed indent and trailing in Flask.py	2016-08-17 14:21:31 +02:00
Mokaddem	c10f28d9df	Updated isolated launch scripts + Added switch displaying queue in index	2016-08-17 13:46:01 +02:00
Mokaddem	23b65eddca	Auto unchecked info checkbox in index	2016-08-17 11:25:05 +02:00
Mokaddem	4d09ab0f6b	Modified icon sentiment in web-header.	2016-08-17 11:13:37 +02:00
Mokaddem	44f54b7b81	Added dependencies	2016-08-17 11:04:48 +02:00
Mokaddem	f0b527167a	Added script flexgauge	2016-08-17 10:50:30 +02:00
Mokaddem	c12e67a35a	Added tooltips + slight gauge modification	2016-08-17 10:43:43 +02:00
Mokaddem	894b9efda9	Added drop of really long line in sentiment-analysis module + Added description of sentiment module. Also, fixed bug in webpage sentiement-trending concerning avg and date range.	2016-08-17 09:46:25 +02:00
Mokaddem	1084e45f1b	Added draft of filter in sentiment analysis (Discard syntaxical languages) + Added nice tooltip for sparkline. Trending displays avg in function of the number of elements processed and not for the complete week + fixed bug in gauge and canvasjs (was performing avg with only 1 graph instead of all 8).	2016-08-16 16:33:02 +02:00
Mokaddem	ecd834ffb6	Added autoreload in sentiment-trending.	2016-08-16 11:27:49 +02:00
Mokaddem	bb677b9061	Added avg text in sentiment-trending + fixed date range bug	2016-08-16 11:23:14 +02:00
Mokaddem	cdd08d7df0	Added all providers for plot_tool in moduleStats + fixed date range bug	2016-08-16 11:07:36 +02:00
Mokaddem	8fa3c2b147	Added stacked choice or not for plot-tool (sentiments)	2016-08-16 10:49:05 +02:00
Mokaddem	4baac0b994	Tool-plot for sentiment working + Splitted script and webpage	2016-08-16 10:31:18 +02:00
Mokaddem	1e7547d16c	Created draft of search for plot tool	2016-08-15 16:32:02 +02:00
Mokaddem	88ed8a239d	Linked plot button with server	2016-08-15 15:54:53 +02:00
Mokaddem	82f70e61f4	Added provider checkbox	2016-08-15 15:16:31 +02:00
Mokaddem	f90987d284	Added date on slider	2016-08-15 14:53:46 +02:00
Mokaddem	d8c84ec674	Added link to data in analysis trending + fixed tooltip bug in sparkline	2016-08-15 12:57:39 +02:00
Mokaddem	6c7f660052	Added support of week and day average.	2016-08-13 22:17:11 +02:00
Mokaddem	1a5158b081	Added sentiment analyser module (draft)	2016-08-13 15:24:57 +02:00
Mokaddem	0431ecf434	Modified webpage display and added js.	2016-08-12 10:22:24 +02:00
Mokaddem	9c3b573af2	Splited js from html + Added some ID + small display modification.	2016-08-12 07:49:24 +02:00
Mokaddem	87a86b38d1	Added page Plot-tool	2016-08-11 21:08:43 +02:00
Mokaddem	c028ca80da	Separated options from the plot funcion for sparkline.	2016-08-11 17:57:23 +02:00
Mokaddem	98c4cacb98	Finished first panel in sentiment-today.	2016-08-11 17:22:50 +02:00
Mokaddem	8682fcc29c	Added empty webpages for sentiments modules.	2016-08-11 11:34:22 +02:00
Mokaddem	5612a77af9	Separated webpage header code from all webpage	2016-08-11 10:55:51 +02:00
Mokaddem	0ba5637e06	Added bar chart hover binder in trending charts	2016-08-10 11:16:56 +02:00
Mokaddem	24318f6a86	Added the part 'other' in Trending chartscd	2016-08-10 10:42:06 +02:00
Mokaddem	1ca8cf69f6	Added trendingchart.js	2016-08-10 09:38:15 +02:00
Mokaddem	b249676fe9	Removed console log	2016-08-09 16:54:53 +02:00
Mokaddem	507c83e06d	In index webpage, reset value graph if no paste received	2016-08-09 16:48:57 +02:00
Mokaddem	943016d6ca	Fixed one of the small display bug in DUplicate_ssdeep_v2 (was displaying [tlsh, tlsh] instead of simply tlsh)	2016-08-09 15:06:55 +02:00
Mokaddem	0d16a8beb7	Fixed bug where Listener was not placed if the row was not displayed (BrowseWarningPaste)	2016-08-09 12:30:40 +02:00
Mokaddem	e2ba4060ee	restaured loading gif when new tab is shown (BrowseWarningPaste)	2016-08-09 12:07:20 +02:00
Mokaddem	a80f6ca0c2	Bug fix related with redis: ... Fixed typo key in redis for module creditcard and sqlinjection Modified Curve redisLvlDb server Modified Url.py so that it forwards name of protocol from saved protocolsfile Added Cache control in Flask Modified key-tab name into keys-tab	2016-08-09 11:59:36 +02:00
Mokaddem	084737dd26	Fixed bug in flask not correctly displaying the number of paste/providers	2016-08-08 11:39:17 +02:00
Mokaddem	78c611fead	Added warning_paste module and created related webpages. ... Fixed a Faup bug in credential (multiple instanciation) and added correc populate_set_out in concerned modules (creditcard, credential, ...). Linked browse_warning_paste module and Flask function with redis (created new sets).	2016-08-08 09:17:44 +02:00
Mokaddem	3523d79a05	Started browse important paste feature. Still need to link with redis. + Fixed a bug in Flask (String to list parser)	2016-08-04 16:57:34 +02:00
Mokaddem	55ef917ef1	Added few comments	2016-08-04 12:10:56 +02:00
Mokaddem	d9316771cd	Draft: added new duplicate hash comparison - tlsh	2016-08-04 11:55:38 +02:00
Mokaddem	50d2848a40	Tryed to fix the glob_tabvar bug	2016-08-02 16:39:57 +02:00
Mokaddem	9e82ec82b7	Fixed a bug in the hover of provider graph (happened when there was an 'other' in other graphs)	2016-08-02 16:17:25 +02:00
Mokaddem	9d37ebc860	Fixed a division by 0 in Flask + Harmonized webpage item positioning	2016-08-02 16:00:31 +02:00
Mokaddem	f3cf2f853a	Fixed a bug in the hover of providers graph. ... When we click on a part in providers graphs, we hover an item, then we click on a random part not belonging to providers graphs, and we re-hover one part of the providers graphs, there was an undefined variable (The random click overrided it).	2016-08-01 11:08:53 +02:00
Mokaddem	feb1717dc6	fixed size bug, which in some cases display the total size instead of the average + few cleanup + commented out log scale in moduleStats.js and added large number division in case of div by 0	2016-07-29 16:52:50 +02:00
Mokaddem	a827e8ca3a	Big refactoring and added lot of comments + fixed bug attaching multiple binder on barChart	2016-07-29 14:30:41 +02:00
Mokaddem	e8fae17cfd	Fixed small race condition bug in ModuleStat.py and Added highlight of bar belonging to the same serie in moduleStats.js	2016-07-29 10:36:44 +02:00
Mokaddem	c2bf2ba630	Added tooltip in moduleStat above each bar chart + Added highlight of the selected bar. (Still not working for category 'Other')	2016-07-28 16:35:30 +02:00
Mokaddem	09fdaa3e39	Added tooltip for moduleStats bar chart	2016-07-28 12:06:54 +02:00
Mokaddem	acec508f53	fixed multiple binding bug in moduleStats + Added multiple bar display for 'other pie part' + Added ModuleStats tab in all other pages	2016-07-28 09:52:07 +02:00
Mokaddem	5a02a7ffce	Clean up and refactoring for back-end moduleStats	2016-07-27 11:55:57 +02:00
Mokaddem	27e0d5c578	Clean up and refactoring for front-end moduleStats	2016-07-27 11:26:56 +02:00
Mokaddem	240bdd9cca	Added number of paste by provider in module stats	2016-07-27 09:27:32 +02:00
Mokaddem	63774cd160	In moduleStats, if there is data which have small percentage, display it in an 'other' part + Added stacked chart for them (still draft)	2016-07-26 16:56:52 +02:00
Mokaddem	5bab1a6f17	Added the click on a part in module stats	2016-07-26 12:12:26 +02:00
Mokaddem	c07c12088d	In moduleStat, display old data if there is no data today and notify the user.	2016-07-26 12:07:52 +02:00
Mokaddem	baa042827d	Removed useless comments + prevent trying to add an already present element in the set	2016-07-26 11:06:31 +02:00
Mokaddem	cc1b49baaf	Fixed variable bug in moduleStats and added draft of credential stats	2016-07-26 10:45:02 +02:00
Mokaddem	06be1f129a	Added Draft of ModuleStats + Paste size average	2016-07-25 16:38:57 +02:00
Mokaddem	b076765cf4	Added dependency flot.time	2016-07-21 16:35:49 +02:00
Mokaddem	7c4f4a2f79	Merge branch 'statistics-v2' into uptodate-statistics-v2	2016-07-21 15:16:03 +02:00
Mokaddem	b685f81752	Merge branch 'duplicate-module' into uptodate-duplicate-module	2016-07-21 14:59:52 +02:00
Mokaddem	9a34a587cc	bug global_tabvar seems solved: ... Caused by a race condition, Just switched two functions execution	2016-07-21 14:53:34 +02:00
Mokaddem	6f4bfeb4ef	restored deleted part of a failed merge-conflict	2016-07-21 14:45:41 +02:00
Mokaddem	37557da8a2	Merge remote-tracking branch 'upstream/master'	2016-07-21 14:04:18 +02:00
Mokaddem	ef76d7aba3	re-ordered tab in trending charts	2016-07-21 13:48:44 +02:00
Mokaddem	0ab7560477	Added top_progression chart for tld, domain and scheme + Small modification in config file.	2016-07-21 13:44:22 +02:00
Raphaël Vinot	d2557aae9a	Update web thirdparty modules	2016-07-20 14:11:39 +02:00
Raphaël Vinot	a0a4d0dff3	Cleanup and remove useless deps	2016-07-19 18:07:29 +02:00
mokaddem	7e5cf7cc55	Added dependency for flot chart	2016-07-19 16:51:29 +02:00
mokaddem	4bc84a2580	Added dependency for flot chart	2016-07-19 16:49:57 +02:00
Raphaël Vinot	8a69110c8e	Merge branch 'webpage-splitting' of https://github.com/mokaddem/AIL-framework into mokaddem-webpage-splitting	2016-07-19 16:35:31 +02:00
Mokaddem	a6996c0b23	Added related functions and display of duplicated paste in search.py	2016-07-19 10:48:44 +02:00
Mokaddem	4ecb9a4b44	Fixed a flexibility issue	2016-07-15 09:47:00 +02:00
Mokaddem	321064bf0d	Fixed a bug in processed_pastes graph	2016-07-14 11:17:08 +02:00
Mokaddem	fba14bfb4b	In index: Added number of processed pastes chart	2016-07-14 11:16:07 +02:00
Mokaddem	60552bca4d	Fixed a bug in processed_pastes graph	2016-07-14 10:31:47 +02:00
Mokaddem	594d2def35	In index: Added number of processed pastes chart	2016-07-13 15:57:33 +02:00
Mokaddem	921da8aeb0	Display the correct active tab in navbar.	2016-07-12 14:44:59 +02:00
Mokaddem	4e4dfb2782	Searching now works in trending graphs.	2016-07-12 14:38:22 +02:00
Mokaddem	6305f1eba7	Deleted redundant code and separeted all trendings chart from the main page.	2016-07-12 14:33:54 +02:00
Mokaddem	7e5ce0f17f	Optimized create_plot and removed test commemts	2016-07-12 11:47:51 +02:00
Mokaddem	4ac448e71e	restored working search>html + Added max curves display to all graphs.	2016-07-12 10:02:01 +02:00
Mokaddem	8cd36982b8	Added a dynamic limit on the number of elements to display. Not fully tested. (recovered old doc, searching not working)	2016-07-11 16:57:45 +02:00
Mokaddem	f6c7917149	Graphs now displayed the highest value. Plotting and filtering is done in its own script.	2016-07-11 15:05:04 +02:00
Mokaddem	880359265c	Renamed tldstrending into trending and added graphics into their respective pannels. Separated graph generation from html source code for wordtrending and protocoltrending	2016-07-08 15:08:37 +02:00
Mokaddem	0174d46788	Added loading GIF when searching and fixed small display bug in modal	2016-07-08 12:09:33 +02:00
Mokaddem	f475f18f06	Added comments and renamed variables and separeted chunks of codes into function	2016-07-08 10:19:24 +02:00
Mokaddem	f846255d84	fixed small sorting problem (date)	2016-07-08 09:16:43 +02:00
Mokaddem	6d760b8dfc	Added dataTable resources in update-third-party.sh	2016-07-08 09:04:19 +02:00
Mokaddem	e206d27f63	Added query search display	2016-07-07 16:43:18 +02:00
Mokaddem	465244e1ce	Added dynamic table sorting in search page. (Still need to add dependencies)	2016-07-07 16:38:00 +02:00
Mokaddem	f1979f6dcb	Search preview modal more dynamic and responsive	2016-07-07 15:34:08 +02:00
Mokaddem	74b337f1a1	search preview modal do not duplicate anymore but still needs improvements	2016-07-07 09:31:04 +02:00
Mokaddem	f29620cf96	Added load more content in search modal preview. Works fine but may duplicate content (need to fix).	2016-07-06 16:54:27 +02:00
Mokaddem	3345124219	Added pannel in search modal + small fix related to config	2016-07-06 14:48:27 +02:00
Mokaddem	9209d48e30	searching now working with preview, details and redirection to the paste display page.	2016-07-06 14:30:32 +02:00
Mokaddem	7ff9b9a583	Added DomainTrending seems working. ... Started search features with related html pages, not finish yet.	2016-07-05 16:53:03 +02:00
Mokaddem	8c1eeea6e6	created new graph pannel for further url stats and dissociate graph script from html	2016-07-04 11:43:38 +02:00
Mokaddem	5a9eca9291	Added few comments	2016-07-04 09:18:23 +02:00
Mokaddem	4b3101b7b6	Added template tld. Modified URL using Faup and refactored WebStats.	2016-07-01 16:59:08 +02:00
Mokaddem	beeeb76de9	Added new modules and started WebTrending web interface	2016-06-30 14:38:28 +02:00
Mokaddem	3dc356dc5e	Getting Started: Initial configuration working on the laptop	2016-06-30 14:36:47 +02:00
Alexandre Dulaunoy	d89ae365f9	web: basic search functionality added	2014-12-24 14:42:20 +00:00
Alexandre Dulaunoy	9b7e329888	Font awesome link fixed	2014-12-22 20:51:05 +00:00
Alexandre Dulaunoy	c2d4e8b437	logo added and layout changed	2014-12-22 17:08:02 +00:00
Alexandre Dulaunoy	dc57454d82	Update start-bootstrap urls	2014-12-22 15:15:56 +00:00
Alexandre Dulaunoy	f15caa70f5	ultag removed	2014-09-17 17:24:08 +02:00
Alexandre Dulaunoy	b36cf76889	Removed ultag	2014-09-17 17:24:08 +02:00
Raphaël Vinot	7542eaf739	Update starting script.	2014-09-02 15:21:36 +02:00
Raphaël Vinot	abfe13436b	Big refactoring, make the queues more flexible	2014-08-29 19:37:56 +02:00
Raphaël Vinot	d7a7da4814	Make sure the webserver fails properly if there is no config file. ... Create the queue list in a more pythonesque fashion.	2014-08-26 17:33:28 +02:00
Alexandre Dulaunoy	1567e452f5	Cleanup: removed unused parameter	2014-08-26 09:25:28 +02:00
Raphaël Vinot	078c8ea836	Big cleanup, pep8	2014-08-14 18:07:18 +02:00
Starow	7a3c216787	Adding a js library for the tag cloud (to install with update_thirdparty.sh	2014-08-14 10:04:39 +02:00
Starow	6cce9f7dac	Javascript modif (Tagclouds) ...	2014-08-13 17:32:24 +02:00
Starow	0d25b960af	logs display modifications	2014-08-13 10:59:47 +02:00
Starow	1df2ffd094	Splitting the graph in two one for words, one for paste monitoring	2014-08-12 16:44:35 +02:00
Starow	731141c723	Padding log hours + adding some super nice bootstrap icon :)	2014-08-12 16:11:11 +02:00
Starow	5a4218035a	minor javascript update	2014-08-12 15:48:53 +02:00
Starow	6aa4d7cb7d	Harmonising logs messages + Changing some dygraph options	2014-08-12 15:42:16 +02:00
Starow	c3948867ad	Adding an option to have better view of the curves	2014-08-12 14:51:03 +02:00
Starow	8d6c2f4288	Improving javascript and Fixing minor curves drawing bug ... Everything now is fully "dynamic"	2014-08-12 14:00:00 +02:00
Starow	ef6bece82c	Major javascript update: Queues curves are now being set dynamically. ... Now the javacript part is a little bit less disgusting than before.	2014-08-12 11:41:56 +02:00
Starow	60cfc9789f	Adding logging proc time inside the table	2014-08-11 16:29:38 +02:00
Starow	e96be56b82	Checkbox default checked at true.	2014-08-11 15:30:44 +02:00
Starow	f02cb10cac	Merge branch 'master' of https://github.com/CIRCL/AIL-framework	2014-08-11 15:08:16 +02:00
Starow	25cebf314c	Index.html log table modification	2014-08-11 15:07:55 +02:00
Starow	7cf05f959d	Javascript modification dashboard log table	2014-08-11 15:06:38 +02:00
Starow	80fe77da26	Making bigger the graph	2014-08-11 13:46:02 +02:00
Starow	a7679787b7	Removing "Dashboard" Title to win some place for displaying infos..	2014-08-11 13:41:58 +02:00
Starow	edd4ea9624	Queue name tab moved to left side of the dashboard	2014-08-11 11:15:44 +02:00
Starow	2306783882	Updating dashboard navigation menu list	2014-08-11 10:48:03 +02:00
Starow	af827a3402	Removing the hardcoded IP adress	2014-08-08 13:49:38 +02:00
Starow	1379ef705a	Initial import of AIL framework - Analysis Information Leak framework ... AIL is a modular framework to analyse potential information leak from unstructured data source like pastes from Past ebin or similar services. AIL framework is flexible and can be extended to support other functionalities to mine sen sitive information	2014-08-06 11:43:40 +02:00