AIL/ail-framework
7
0
Fork 0
mirror of https://github.com/ail-project/ail-framework.git synced 2024-11-27 00:07:16 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions
2088 commits 8 branches 43 tags 99 MiB
Commit graph

745 commits

Author SHA1 Message Date
Terrtia
a43467a0f2
chg: [Update v1.5] update tracked terms path 2019-04-15 13:27:46 +02:00
Terrtia
fc1a04336c
fix: [dynamic update v1.5] make sure updates are excuted in the correct order + fix nb_seen_hash dynamic update 2019-04-15 11:01:33 +02:00
Terrtia
1ed9468881
fix: [UI showpaste] min preview 2019-04-11 13:44:20 +02:00
Terrtia
3cc614a1ad
chg: [DB] update items tags metadata 2019-04-11 11:58:06 +02:00
Terrtia
e83174327a
chg: [DB] remove absolute path 2019-04-10 17:47:40 +02:00
Terrtia
d44acea04d
chg: [merge] merge update into advanced_crawler 2019-04-10 16:41:06 +02:00
Terrtia
2589fc2161
chg: [merge] merge master into update branche 2019-04-10 15:43:15 +02:00
Terrtia
b4bee29a33
Merge branch 'master' into advanced_crawler 2019-04-10 15:37:42 +02:00
Terrtia
6165987ec3
chg: [DB Migration] add hash dynamic update 2019-04-10 15:36:41 +02:00
Terrtia
870fa1981c
fix: [update_thirdparty] detect and use virtualenv 2019-04-02 09:50:52 +02:00
Terrtia
f64c385343
chg: [Crawler] handle port: crawling + history 2019-03-22 16:48:07 +01:00
Terrtia
f4cdddbc7f
chg: [UI + item submit] remove browse paste from UI + fix pasteSubmit tagging 2019-03-21 11:50:53 +01:00
Terrtia
51eb951a47
fix: [UI terms] update tags endpoint 2019-03-21 11:34:21 +01:00
Terrtia
c2bc72c72e
chg: [Tags alertHandler] remove alertHandler + add quick tag search sidebar + use tag last_seen to display tagged items 2019-03-21 11:11:53 +01:00
Terrtia
ce6475e96e
chg: [tags] remove/add tags by daterange + improve item loading time 2019-03-20 15:57:55 +01:00
Terrtia
8b5266706e
fix: [Tags] cast nb_max_page to int 2019-03-20 13:33:12 +01:00
Terrtia
a9e86231ad
chg: [Tags] add backend, daterange selector 2019-03-20 13:25:02 +01:00
Terrtia
65e6eae00d
chg: [UI preview items] b4 migration + new template + better display + decrease item loading time + add basic animation/transition 2019-03-19 13:34:41 +01:00
Terrtia
e9bac098c9
fix: [UI Tags] paste preview 2019-03-15 11:16:44 +01:00
Terrtia
23d9cb2265
chg: [UI Tags search] bootstrap 4 migration 2019-03-14 17:04:55 +01:00
Terrtia
9e4813ca4b
chg: [UI Tags] start b4 migration + add date range selector 2019-03-12 13:37:15 +01:00
Terrtia
d3b3942ad7
chg: [Flask UI] refractor crawler frontend + show crawled domain 2019-03-06 15:05:58 +01:00
Terrtia
6f5f880b7d
chg: [Crawler UI] add basic tooltips 2019-03-01 11:40:43 +01:00
Terrtia
93c7db86e4
chg: [Crawler UI Flask] refractor functions, templates 2019-02-28 14:56:14 +01:00
Terrtia
460f70fee6
chg: [css] add basic toggle sidebar 2019-02-27 16:58:17 +01:00
Terrtia
aaf4712877
fix: [UI crawler manual] fix min input 2019-02-26 15:39:52 +01:00
Terrtia
c0d72e7d2a
chg: [Crawler UI] Crawler major refractor (end) + basic UI for manual crawler 2019-02-26 14:50:48 +01:00
Terrtia
e5dca268a8
chg: [Crawler] refractor 2019-02-21 09:54:43 +01:00
Terrtia
0832784f7a
chg: [thirdparty_update] get b4 datatable 2019-02-19 11:56:08 +01:00
Terrtia
da78d0552d
chg: [Crawler UI Tags] add tag by day + add crawler status + UI onion blacklist 2019-02-19 11:41:45 +01:00
Terrtia
95d63daaf4
chg: [Crawler UI] fix css + bootstrap4 migration 2019-02-15 17:00:07 +01:00
Terrtia
674a9a2591
chg: [UI] font icon v5 + navbar + sidebar 2019-02-08 17:16:58 +01:00
Terrtia
c2885589cf
chg: [UI] basic navbar + sidebar + refractor 2019-02-07 17:22:44 +01:00
Terrtia
516238025f
chg: [Crawler] add bootsrap4 src + refractor crawler 2019-02-05 17:16:44 +01:00
Terrtia
423c7b1455
fix: [UI] fix flot js install 2019-02-04 14:56:54 +01:00
Terrtia
655646b7c5
fix: [hiddenServices] catch empty data 2019-01-29 16:43:56 +01:00
Terrtia
c1b34bd99c
fix: [Crawler] limit max crawled pages 2019-01-29 15:38:00 +01:00
Terrtia
7a80683f13
chg: [hiddenServices] crawlers status, dynamic update 2019-01-29 14:27:50 +01:00
Terrtia
b9572e0085
fix: [Crawler] typo 2019-01-29 12:19:10 +01:00
Terrtia
bb301a870c
fix: [Crawler] fix onion blacklist + add crawler info 2019-01-29 12:00:14 +01:00
Terrtia
c4c3810037
fix: [hiddenServices] typo 2019-01-08 15:51:39 +01:00
Terrtia
46c1105483
chg: [hiddenServices] auto update last crawled domains 2019-01-08 15:48:40 +01:00
Terrtia
77211c5496
chg: [hashDecoded] add top5 types piechart (v0.1) 2018-12-24 11:42:55 +01:00
Terrtia
f023f41ae5
fix: cleaning ... 2018-12-21 15:21:08 +01:00
Terrtia
e93a0b637c
chg: [hashDecoded] add option to display decoded files, fix #271 2018-12-21 15:18:07 +01:00
Terrtia
306815a31d
fix: [hiddenServices UI] fix none type 2018-12-20 09:08:14 +01:00
Terrtia
6ee4b7bf93
Merge branch 'master' into update 2018-12-19 15:23:34 +01:00
Terrtia
4e08aaa80f
chg: [UI Crawled Domains] show crawled domains by date range (options: domains tags, up domains, down domains) 
todo: Shows domain history
 2018-12-17 15:58:48 +01:00
Terrtia
31a8dfe0b3
fix: [AIL] use only relative paths 
pastes duplicates are fixed on the fly
 2018-11-21 16:45:25 +01:00
Terrtia
d9b776d58a
merge master 2018-11-14 15:55:42 +01:00
Sami Mokaddem
62e46b6dc1
Merge pull request #287 from CIRCL/term_management 
Term management upgrade
 2018-11-08 14:10:25 +01:00
Terrtia
173c0a15c6
fix: [term_management] delete tags 2018-11-08 12:01:40 +01:00
Terrtia
4e906ff841
fix: [term_management] delete emails 2018-11-08 10:41:37 +01:00
Terrtia
0adb8b4624
chg: [terms_management] delete terms tags 2018-11-07 15:37:25 +01:00
Terrtia
46614c97c8
chg: [terms_management] delete email 2018-11-07 11:42:31 +01:00
Terrtia
2c46c7f2ab
fix: [tag export] limit custom tag lenght 2018-11-07 10:17:57 +01:00
Terrtia
f91aba5f6f
chg: [tag export] UI, add custom tags 2018-11-07 09:41:58 +01:00
Terrtia
0ad80a22ca
fix: [update_thirdparty] upgrade github source 2018-11-06 17:04:02 +01:00
Terrtia
a3fa5a4dcf
chg: [auto push tag] delete unwanted tags 2018-11-06 16:08:58 +01:00
Terrtia
979472df15
chg: [Terms] tag tracked terms 2018-11-06 13:38:37 +01:00
Terrtia
af1ff6225e
fix: [Search] tags display 2018-11-05 10:36:58 +01:00
Terrtia
5c67297dc8
fix: [Search] Improve search performance 
note: search by paste name is temporary disabled
 2018-11-05 10:13:48 +01:00
Terrtia
5872cf9196
fix: [Scripts] Remove absolute path 2018-11-02 16:07:27 +01:00
Terrtia
ea7fd8d54c
fix: [Duplicate] show duplicate with relative path 2018-10-24 15:47:03 +02:00
Terrtia
6328cc22b7
chg: [Crawler] add domains blacklist 2018-09-28 16:29:09 +02:00
Terrtia
b3a6dc8487
fix: [Crawler] remove hardcoded url 2018-09-28 15:42:06 +02:00
Terrtia
3dd20af4d1
Merge branch 'master' into onion_crawler 2018-09-28 15:29:24 +02:00
Terrtia
82e6df4b94
chg: [Crawler] domains stats + logs + clean 2018-09-28 15:23:27 +02:00
Terrtia
ecb2857151
chg: [Crawler] css + limit splash RAM 2018-09-27 16:47:48 +02:00
Terrtia
ff0affc679
fix: [Flask] remove hardcoded url 2018-09-26 11:59:51 +02:00
Terrtia
3a72f1548d
fix: [Flask] remove hardcoded url 2018-09-25 16:46:06 +02:00
Terrtia
b3923428e5
fix: [Flask] remove hardcoded url 2018-09-25 14:37:14 +02:00
Terrtia
e4a5f66a10
Merge branch 'master' into onion_crawler 2018-09-24 16:35:04 +02:00
Terrtia
874824a589
fix: [Crawler] clean 2018-09-24 16:28:55 +02:00
Terrtia
50c81773e9
chg: [Crawler] add launcher and install 2018-09-24 16:23:14 +02:00
Terrtia
6edc1ddbeb
chg: [HiddenServices] get domain related to other domains 2018-09-21 10:34:06 +02:00
Terrtia
61b4c9e579
chg: [Flask] add prefix in config to flask routes 2018-09-20 10:56:29 +02:00
Terrtia
5eadc2755f
chg: [Flask] add prefix in config to flask routes 2018-09-20 10:38:19 +02:00
Terrtia
5732d7b134
fix: [hashDecoded] range_type_json 2018-09-19 15:36:33 +02:00
Terrtia
912f2352ab
fix: [crawled screenshot] fix img error 2018-09-18 16:58:49 +02:00
Terrtia
ce63d81878
chg: [UI] pixelate crawled screenshot by default 2018-09-18 11:51:21 +02:00
Terrtia
f5b648d72a
pixelate paste screenshot 2018-09-18 11:03:40 +02:00
Terrtia
094b211259
fix: [showpaste] path transversal 2018-09-17 15:42:22 +02:00
Terrtia
0c63f2f24f
chg: [Crawler] catch server response 2018-09-17 15:35:06 +02:00
Terrtia
6f0817365a
chg: [Crawler UI] display domain information 2018-09-12 09:55:49 +02:00
Terrtia
ca982e13e1
chg: [Crawled Domain] show crawled pastes by domain 2018-08-27 14:34:08 +02:00
Terrtia
40772a5732
fix: merge 2018-08-27 11:30:19 +02:00
Terrtia
823c33174a
Merge branch 'master' into onion_crawler 2018-08-27 11:20:51 +02:00
Terrtia
d42dd118a4
chg: [Domain crawled] add random screenshot 2018-08-27 11:02:39 +02:00
Terrtia
6c981ea5e9
fix: [HashDecoded] typo 2018-08-24 11:58:54 +02:00
Terrtia
ab452841b1
fix: [HashDecoded] change template title 2018-08-24 11:51:16 +02:00
Terrtia
e6b583ee7a
chg: [Hash] add Help on graph node 2018-08-24 11:40:09 +02:00
Terrtia
a586548047
Merge branch 'master' into base64 2018-08-24 10:35:28 +02:00
Terrtia
7e24943537
chg: [Crawler] crawler accept all kind of domains 2018-08-24 10:13:56 +02:00
Terrtia
e9580d6775
chg: [Crawler] change BDD, save i2p links 2018-08-21 15:54:53 +02:00
Terrtia
ed559d9f4a
chg: [Showpaste] add screenshot + improve onion db 2018-08-16 17:24:39 +02:00
Terrtia
54cc4f3723
fix: [UI-Showpaste] show raw paste, force text/plain mimetype 2018-08-03 09:15:34 +02:00
Terrtia
911cff70da
chg: [UI] add AIL icon, fix li links 2018-08-02 10:44:47 +02:00
Terrtia
deaebaa4fd
fix: [UI-Hash] pie chart json 2018-07-27 13:51:29 +02:00
Terrtia
009f87ee10
[fix: [UI-hash] fix graph node zoom 2018-07-27 11:27:41 +02:00
Terrtia
b02cbee179
chg: [UI-dashboard] add logs number configuration 2018-07-26 11:35:54 +02:00
Terrtia
0a82482db4
chg: [UI-dashboard] convert time to local 2018-07-26 10:34:43 +02:00
Terrtia
5ad5939bd5
chg: display last logs 2018-07-25 16:48:44 +02:00
Terrtia
03a37af5cf
chg: [Hash] add reprocess regex order 2018-07-25 10:47:36 +02:00
Terrtia
8c97c70a3f
Merge branch 'master' into base64 2018-07-24 15:21:32 +02:00
Terrtia
4b8c26dec8
fix: [hive auto export] fix #221 2018-07-24 14:46:35 +02:00
Terrtia
3479923f96
fix: [UI-Hash] pie decoder 2018-07-23 11:43:48 +02:00
Terrtia
9c03c40482
chg: [UI-Hash] add decoder pie 2018-07-23 11:11:52 +02:00
Terrtia
1fbd4109c2
chg: [UI-Hash] add decoded type on graph 2018-07-20 15:48:56 +02:00
Terrtia
bd5f83f0eb
chg: refractor base64 encoded to hash 2018-07-20 10:32:52 +02:00
Terrtia
e25dd52d64
add: Binary decoder 2018-07-18 11:45:19 +02:00
Terrtia
524d5fe604
add: hash download 2018-07-18 10:09:35 +02:00
Terrtia
d9c924c38d
chg: use json to submit hash 2018-07-17 14:23:49 +02:00
Terrtia
412b012ddf
add: hash line_graph (nb/day) 2018-07-16 16:45:36 +02:00
Terrtia
6f69da0c0d
fix: paste submit css 2018-07-16 10:32:41 +02:00
Terrtia
fce324e370
fix: release tiemeout alarm, chg:icon graph node 2018-07-13 11:54:14 +02:00
Terrtia
87b7facba8
add: base 64 node graph 2018-07-12 17:07:17 +02:00
Alexandre Dulaunoy
9dc211dbe9
chg: [UI] Typo fixed as requested by our favourite German. 2018-07-10 10:49:02 +02:00
Terrtia
fd02085495
add: show hash info 2018-07-09 17:07:59 +02:00
Terrtia
c2a976e907
add: stacked barchart, resize chart 2018-07-09 11:47:04 +02:00
Terrtia
0e5a7d8d47
add date_range stacked bar 2018-07-06 16:54:43 +02:00
Terrtia
3b4e2a18aa
fix: date format https://github.com/CIRCL/AIL-framework/issues/219 2018-07-05 14:45:34 +02:00
Terrtia
cd154be6f9
fix: term_management UI bug ,https://github.com/CIRCL/AIL-framework/issues/214 2018-07-05 14:12:47 +02:00
Terrtia
f7d38bea16
add: base64 date and type selector 2018-07-05 13:58:40 +02:00
Terrtia
b6d855b963
change import d3.js to d3.min.js 2018-06-29 15:58:24 +02:00
Terrtia
adfc8cf260
add d3 and daterange js 2018-06-29 15:52:13 +02:00
Terrtia
a0a639276c
fix submit paste 2018-06-29 15:29:02 +02:00
Terrtia
98d0235fc3
Merge branch 'master' into base64 2018-06-29 14:57:46 +02:00
Terrtia
5908710b34
fix: update submitted tags link 2018-06-29 11:29:52 +02:00
Terrtia
cb807bf2c6
chg: change base64 icon 2018-06-29 11:28:35 +02:00
Terrtia
16c700ddb4
new: add base64 display on paste + base64 statistics 2018-06-29 10:02:29 +02:00
Terrtia
225776c69c
fix: tp and fp, errors messages on paste submission 2018-06-20 10:02:26 +02:00
Terrtia
6a89fac309
fix hive and misp error 2018-06-19 16:39:49 +02:00
Terrtia
af9db53171
add tag fp and tp 2018-06-19 15:09:26 +02:00
Terrtia
bf72c27711
fix path-event id map 2018-06-19 13:43:26 +02:00
Terrtia
cd4c099685
fix db init 2018-06-19 11:54:34 +02:00
Terrtia
103787a07d
fix 2018-06-19 11:43:16 +02:00
Terrtia
bc40bb3c38
fix hive connection + add keys sample 2018-06-19 11:42:06 +02:00
Terrtia
0ee36e484f
fix hive connection + submitted dir creation 2018-06-19 11:31:30 +02:00
Terrtia
634bf6e552
add img + fix upload api 2018-06-18 17:16:22 +02:00
Terrtia
704e6f0ad1
change default submit tag, event-path map, publish events 2018-06-18 13:58:31 +02:00
Terrtia
aee1b50ef1
fix duplicate export attribute + tag export whitelist 2018-06-15 17:25:43 +02:00
Terrtia
31fc33ab03
misp event creation, auto and manual + the hive alert auto creation, manual case creation 2018-06-14 16:51:06 +02:00
Terrtia
8522d50308
modify navbar + decompress file password 2018-06-11 10:21:11 +02:00
Terrtia
c64588f60e
add feature, user can submit paste on the web interface 2018-06-08 16:49:20 +02:00
Terrtia
8e9c77b364
verify file upload extention 2018-06-06 10:05:25 +02:00
Terrtia
6143bc3dce
submit users input paste 2018-06-05 16:58:04 +02:00
Terrtia
c41a56e4a2
clean Flask_tags + gitignore 2018-06-01 13:36:08 +02:00
Terrtia
2b310d4367
clean 2018-06-01 11:26:45 +02:00
Terrtia
cd6734f704
fix install 2018-05-31 15:35:20 +02:00
Terrtia
884d655b0c
taxonomies and galaxies update via LAUNCH 2018-05-31 13:53:29 +02:00
Terrtia
5c50de45de
add taxonomies and galaxies update and install 2018-05-31 10:07:23 +02:00
Terrtia
8a87b7b082
galaxy tag info + fix 2018-05-30 16:18:58 +02:00
Terrtia
dbe67e094b
galaxies synonyms 2018-05-29 17:17:02 +02:00
Terrtia
f3f6b76428
galaxy description and tags 2018-05-29 08:47:46 +02:00
Terrtia
f5cae0d99c
taxonomie + add tags + tags display 2018-05-23 16:58:56 +02:00
Terrtia
b9eb3ed9ba
add and delete tags features 2018-05-17 15:33:06 +02:00
Terrtia
4fbbf8bbd2
display tag on important paste and search 2018-05-17 11:00:05 +02:00
Terrtia
0d164b6f4c
add tag js and css 2018-05-16 14:48:15 +02:00
Terrtia
b1b7d33c96
tags 2018-05-16 14:39:01 +02:00
Terrtia
f66a528bc2
fix Duplicate, save list of duplicates on disk + prevent empty hash creation 2018-05-15 23:28:47 +02:00
Terrtia
225fe76c96
fix duplicate + get raw paste 2018-05-11 16:07:30 +02:00
Terrtia
142c1dc9b0
clean 2018-05-09 13:03:46 +02:00
Terrtia
3841c159a0
change leveldb to ardb 2018-05-07 14:50:40 +02:00
Terrtia
c17b9402f7
clean 2018-05-04 14:25:47 +02:00
Terrtia
96a3df42bc
decode with redis connection 2018-05-04 13:53:29 +02:00
Terrtia
a900d5f08d
add python 2 to 3 bash + fix Duplicate + clean 2018-05-03 16:21:33 +02:00
Terrtia
c52caebe7c
improve Duplicate + tlsh + add time out handler + debug + clean 2018-05-02 17:07:10 +02:00
Terrtia
c3428df5dc add apiKeys module 2018-04-26 14:42:39 +02:00
Terrtia
8738b7cf75 fix track term and avoid duplicate mail address 2018-04-24 16:44:37 +02:00
Terrtia
f93fe9aeb2 add bitcoin module 2018-04-23 14:51:53 +02:00
Terrtia
41eb250278 fix python 3 compqtibility issues 2018-04-20 10:42:19 +02:00
Terrtia
8571a86344 frontend python 3.5 upgrade 2018-04-17 16:06:32 +02:00
Philipp Schmied
4835589ff4 Fixed typo 2018-03-01 08:50:27 +01:00
Philipp Schmied
29608ab29b Fixed e-mail notificatins for sets and regexes 2018-02-28 11:31:16 +01:00
Philipp Schmied
b6fcf51309 Flask_terms.py: Fixed mapping of notifcation e-mails to terms with uppercase letters 2018-02-28 09:19:27 +01:00
Philipp Schmied
5b1f0b0212 Implemented email notifications (bin: config.cfg additions, email sending via analyzer scripts; var: Changes to add notifications via terms management); terms_management: Fixed click handlers not being added to all tracked terms. 2018-02-27 15:12:02 +01:00
Alexandre Dulaunoy
abdf41fe23
Merge pull request #164 from mokaddem/showDiff 
Show differences between duplicates
 2018-02-21 22:25:16 +01:00
Sami Mokaddem
9125930a77 fix: canvasjs changed its folder hierarchy 2017-12-14 11:47:11 +01:00
Sami Mokaddem
304213cfa4 update: Possibility to change the authorized max line length in the config for the diff of duplicates 2017-12-12 11:00:26 +01:00
Sami Mokaddem
3fe7ecf75a feature: diff in the web interface for duplicates pastes. 2017-12-12 10:48:21 +01:00
mokaddem
9cab76cf88
Merge pull request #162 from mokaddem/sentimentBugfix 
Fix: bug between front-end and back-end date representation
 2017-12-11 16:35:38 +01:00
Paul A
4a4a1d478b Quick refactor for the JavaScript 2017-10-22 18:48:54 +02:00
Paul A
f6b6777095 Fixed some code for the JavaScript (coding-style) 2017-10-20 15:44:09 +02:00
Paul A
48fc8e3464 Removed duplicate function 2017-10-20 15:32:58 +02:00
Simon Tran
a41c5d55c5 Migrated Dockerfile to Ubuntu 16.04 LTS 
Fixed AIL Web dependencies installation
Added Docker documentation
 2017-10-01 02:23:32 +02:00
Sami Mokaddem
2cfb8c173b Fixed from front-end date representation bug 2017-09-27 10:46:20 +02:00
=
b9bd651033 Removed useless import 2017-08-23 14:06:59 +02:00
=
3b3f3aa89c Flask: Added support of LevelDB database depending of the year 2017-08-23 12:20:39 +02:00
=
d632335760 Automatically start redis_levelDB with the correct date 2017-08-23 10:16:56 +02:00
=
bd46c83fb6 BrowseImportantPaste: Fixed a bug where onClickListener was not set when the table was only 1 page long 2017-08-21 11:55:01 +02:00
=
409eb25a64 Deleted data-flow graph and header.html from the index + Added fixed typo where 10min was in fact 3min 2017-08-21 10:51:56 +02:00
Mokaddem
fa52292029 Added comments, cleanup and added credentials search progress in Flask 2017-07-20 10:50:24 +02:00
Mokaddem
cbb2909156 Added comments 2017-07-20 10:24:48 +02:00
Mokaddem
05417f405c Finished basic link web interface + solved some bugs 2017-07-20 10:04:30 +02:00
Mokaddem
ef741f85c0 Added similarity checks + started link webInterface 2017-07-19 11:52:06 +02:00
Mokaddem
882ab88b00 Draft module + web link 2017-07-18 16:57:15 +02:00
Mokaddem
9b8570e40b Dirty draft webpage credential_tracking 2017-07-17 17:26:19 +02:00
Mokaddem
f1a1e75c32 Restaured old example webpage 2017-07-17 16:18:21 +02:00
Mokaddem
9558aafe67 Prevent the loss of the 1 message in import_dir.py 2017-07-17 16:14:27 +02:00
Mokaddem
b60e164380 Added support of onion in browseWarningPaste 2017-04-26 11:47:02 +02:00
Mokaddem
5e4d0170d4 Bug fixes, jinja2 iter0 and typo 2017-04-26 11:31:08 +02:00
Mokaddem
e614da4748 Deleted print + Added curve fill for Mixer graph in Index 2017-04-25 15:52:41 +02:00
Mokaddem
71b2757ed2 New script create_new_web_module permits to rapidly create a new pre-configured webpage from a template. 2017-04-25 15:22:04 +02:00
Mokaddem
6afbf77af0 Modified skeleton template name into rawSkeleton + Added possibility to hide modules in the header. 2017-04-25 12:18:08 +02:00
Mokaddem
f6e8582d9d Auto organisation of the navbar -> Web page are auto-inserted. Created skeleton webpage 2017-04-19 15:14:20 +02:00
Mokaddem
32906965d8 Draft of new organisation of the web interface. Usage of Flask's blueprint 2017-04-19 11:02:03 +02:00
Mokaddem
da081bae42 Merge branch 'regexInTerms' into production 2017-04-18 16:59:40 +02:00
Mokaddem
c8baabd882 Updated TermsFrequency related modules + Fixed bugs (encoding, behaviors, ...) 2017-04-18 15:28:21 +02:00
Mokaddem
2da4c572c7 New feature in Terms: regex and set of words support (draft) 2017-03-28 17:42:44 +02:00
Mokaddem
3261eca836 Fixed all_index newline bug + added print of what's happening 2017-03-15 14:29:49 +01:00
Mokaddem
5b7d047622 Changed os.system by os.mkdir + fixed bug during rotation 2017-03-15 14:05:13 +01:00
Mokaddem
4005b72180 Added backward support + renamed index with timestamp 2017-03-15 11:51:35 +01:00
Mokaddem
381e72ee99 Added display of number of elements inside the index + changed variables names 2017-03-15 10:07:46 +01:00
Mokaddem
e4757f5ceb Added possibility to choose the index in search + Updated search page with jinja2 iter0 2017-03-15 09:39:48 +01:00
Mokaddem
41132fe0bc Added support of per_paste for term-management and plot-term 2017-02-28 15:54:39 +01:00
Mokaddem
4071bf2e91 fixed a bug in manage_term (usage of loop.index0) + fixed bug in top_set where graph was not displaying the correct per_paste value 2017-02-28 15:01:48 +01:00
Mokaddem
25adae0104 Added possibility to select in top_terms the data per_paste or not (need module testing) 2017-02-15 16:29:02 +01:00
Mokaddem
d4da3a316f Added path of the paste in the log of Categ.py 2017-02-14 10:59:47 +01:00
Mokaddem
29241eb49f Added a line separator between the two graphs in dashboard. 2017-01-09 14:44:14 +01:00
Mokaddem
0e39f516a9 Added sleep in helper for multiple listening interfaces (reduce useless work), Added a simple way to pre-process feed before sending it to the global feed. 2017-01-09 14:12:26 +01:00
Mokaddem
24c5621d29 BUG FIX in Flask_trendingmodule, Fixed member_set when get_top_relevant_data is None. 2016-12-26 16:33:10 +01:00
Mokaddem
1c2169d3bc Improved interface for multiple feeds and refresh_script. 2016-12-26 16:16:44 +01:00
Mokaddem
a18c046dea Fixed bug multiple refresh instances and harmonized interface. 2016-12-23 16:15:05 +01:00
Mokaddem
03dddbc359 Reduced refresh rate of processed_pastes, synchro graphs and adjusted max on each graphs. 2016-12-23 16:01:30 +01:00
Mokaddem
97292e0899 Updated web interface to handle new mixer module. and fixed one dependency bug. 2016-12-23 15:44:46 +01:00
Mokaddem
faa8f8302d Removed typo in the number of remaining pastes to display in browseImportantPaste 2016-12-22 09:28:55 +01:00
Mokaddem
9df12cdae8 Added sub-flask in a directory 2016-12-22 08:57:45 +01:00
Mokaddem
d30f3ca6c7 Added missing depency datetime in dashboard 2016-12-09 13:53:57 +01:00
Mokaddem
0054be5bab Added missing json dependency in dashboard 2016-12-09 08:50:36 +01:00
Mokaddem
8daa72789e Splitted Flask_server into module related to website sections 2016-12-09 08:46:37 +01:00
Mokaddem
d3ed1bbf40 Merge branch 'website-improvement' into production 2016-12-08 10:05:55 +01:00
Mokaddem
1abba4dcf9 Added support of re-plotting the plotted terms 2016-12-08 08:44:10 +01:00
Mokaddem
84dcd5586f Started plotting same data with different dates 2016-11-25 16:42:22 +01:00
Mokaddem
a3255d168c ModuleInfo correctly handle CurveManageTopSets, 
Changed queue coloring in dashboard,
ModuleInformation now have a history of executed command and better handle module killing.
 2016-11-25 11:54:16 +01:00
Mokaddem
5d269ea1ee Added date in Duplicate for better distinguish duplicate. Also, added a timeout for the dataTable in duplicate inside the modal. 2016-11-24 15:05:29 +01:00
Mokaddem
df8d978088 Added dynamic data loading in dataTable in search.html 2016-11-24 13:31:31 +01:00
Mokaddem
80ca49f018 Prevent browser to request reconnection for the eventStream + Added dynamic data loading in the dataTable of BrowseImportantPastes 2016-11-11 16:51:39 +01:00
Mokaddem
768fb6f2df removed server overhead for paste preview data 2016-11-10 16:41:07 +01:00
Mokaddem
699c819387 fixed bug in show concerned pastes previex en (terms managements) 2016-11-10 16:33:05 +01:00
Mokaddem
94e3034e7d Fixed bun in terms management, where action_button was not bind in other dataTable pages 2016-11-10 15:53:22 +01:00
Mokaddem
7763bfb4c7 Pastes dataTable now display only the latest clicked paste even if there were discarded queries 2016-11-10 15:39:45 +01:00
Mokaddem
2dd69fbb25 Fixed modal background stacking bug + button leading to saved paste bug fixed in search 2016-11-10 14:56:25 +01:00
Mokaddem
317005c2b4 tryed to fix html display problem in tooltip + fixed problem button showPaste in modal not pointing to the paste 2016-11-10 10:30:29 +01:00
Mokaddem
34221b083f Partially solved the browser freezing problem + tootlip is also binded at a new page generation of the DataTable 2016-11-10 09:28:55 +01:00
Mokaddem
acdd1367a3 Added event source for getImportantPaste 2016-10-28 16:55:56 +02:00
Mokaddem
c95000866d Dynamic table in search now load all the data + fixed bugs where tooltip where not shown on other than the first page displayed and 
tooltip interpret html (not supposed to...)
 2016-10-28 14:21:08 +02:00
Mokaddem
f51808d914 Added support of html formatting in search result dynamic loading table 2016-10-28 09:48:22 +02:00
Mokaddem
4be62a4e2d Added draft of dynamic update table with search result 2016-10-28 09:28:42 +02:00
Mokaddem
340624fe1e Fixed bug in modal show paste (show more button not showing) 2016-10-27 16:18:49 +02:00
Mokaddem
7e7e679ab6 Added dataTable for duplicate in show saved paste 2016-10-27 15:53:45 +02:00
Mokaddem
59a7000527 Bootstrap support in show saved paste 2016-10-27 12:38:39 +02:00
Mokaddem
ac254e0e63 Added support of browsing concerned paste in dashboard for all modules + Show in the page what was the concerned module 2016-10-27 11:50:24 +02:00
Mokaddem
1826b170ec Added support of local paste view in dashboard for Mails event only 2016-10-27 11:27:26 +02:00
Mokaddem
7c8d414948 modified provider name into no data when there is no data 2016-10-14 16:47:22 +02:00
Mokaddem
91678179fd Added msgs when queues or feed is not running/coming 2016-10-14 16:33:54 +02:00
Alexandre Dulaunoy
2781a2f0b7 Stuck not stucked 2016-09-21 08:35:20 +02:00
Jerome St-Hilaire
ea13f87c2c Fixed dependency fetching for the web interface 2016-09-15 12:14:55 -07:00
Raphaël Vinot
acfe9c1336 Update Flask_server.py 2016-08-26 18:06:48 +02:00
Raphaël Vinot
3290b51c91 Update update_thirdparty.sh 2016-08-26 18:05:18 +02:00
Mokaddem
79be8ab934 Fixed introduced bug relative to chart in index webpage. 2016-08-26 15:59:53 +02:00
Mokaddem
0c760d763b Added support of PID in ModuleInformation and index.html 2016-08-26 15:27:37 +02:00
Mokaddem
5c565068a6 Removed auto-reload in dashboard if window in undefined 2016-08-26 11:25:11 +02:00
Mokaddem
181c372131 Updated dependencies 2016-08-26 11:10:39 +02:00
Mokaddem
4cb3b672be Added comments 2016-08-26 10:28:35 +02:00
Mokaddem
86b655100e Deleted useless parsing function 2016-08-26 10:01:14 +02:00
Mokaddem
07d7559060 Added dataTable support in browse paste modal. 2016-08-26 08:30:09 +02:00
Mokaddem
813cc7031a Changed lots of code into zrank 2016-08-25 17:48:12 +02:00
Mokaddem
991176fdfd Suppressed metismenu import. 2016-08-25 16:40:13 +02:00
Mokaddem
fb22816236 Added module number info in index webpage 2016-08-25 16:27:32 +02:00
Mokaddem
0b5a3e0cd2 Auto generate graph div + possibility to query all sentiments data. 2016-08-25 15:06:18 +02:00
Mokaddem
21f2c43572 Individualized sentiments-graph loading 2016-08-25 13:11:16 +02:00
Mokaddem
f9bf00f022 Removed autokill feature if not specified 2016-08-25 11:09:31 +02:00
Mokaddem
611454ba82 Added color in web-index reprenting modules states. 2016-08-24 18:00:05 +02:00
Mokaddem
e3daa84196 Added icon in header 2016-08-23 16:09:34 +02:00
Mokaddem
e12244fb53 Added show/hide curve with legend in terms-top-set. 2016-08-23 15:39:13 +02:00
Mokaddem
e56187eb7e Added position of each top term in the other sets + highligth ones not present. 2016-08-23 14:52:02 +02:00
Mokaddem
a64d447b7b Auto hide button add_to_chart when no data plotted. 2016-08-23 12:09:41 +02:00
Mokaddem
e59b7c4e47 Added tooltip in graph for terms-plot-tool 2016-08-23 12:03:23 +02:00
Mokaddem
0eba419651 Added multiple terms plot in terms-plot-tool. 2016-08-23 11:56:32 +02:00
Mokaddem
9f2d672ec8 harmonized tooltip and popover display 2016-08-23 11:20:48 +02:00
Mokaddem
8c956c22b1 Added autohide curves in terms-top-set. 2016-08-23 11:02:36 +02:00
Mokaddem
32dfd2b181 Moved legend place in term-plot-top 2016-08-23 10:16:33 +02:00
Mokaddem
f9b27e4013 Fixed bug in curve preventing to track concerned_pastes. Now all tracked words are put in lower case. 2016-08-23 09:50:03 +02:00
Mokaddem
1766ced9e6 Fixed potential XSS problem with pastes display in 'search' and 'important_paste_by_module'. 2016-08-23 09:08:17 +02:00
Mokaddem
047331df3e Added save of paste_baseName in set for searches. This is done in ModuleStats to not surcharge the Global module. 2016-08-23 08:52:54 +02:00
Mokaddem
6be557890c Replaced negative hour by 24h based. 2016-08-23 07:19:51 +02:00
Mokaddem
85ab0e36ff In sentiment analysis. if today top provider set is empty, try to take yesterday data and then takes from all_providers. 2016-08-23 07:11:03 +02:00
Mokaddem
8359a6668f Suppressed useless comms 2016-08-22 21:36:19 +02:00
Mokaddem
2fd2c8af39 Removed commented code 2016-08-22 20:59:56 +02:00
Mokaddem
27ae577e00 Fixed stupid line remove in index.html... 2016-08-22 16:25:52 +02:00
Mokaddem
a424069c64 Fixed bug in flask with zrange (was taking the bottom of the set instead of the top leading to get only the min instead of max) 2016-08-22 16:12:20 +02:00
Mokaddem
003c72bd7c Fixed bug with redis where the inserted key was too long, causing a crash + fixed bug taking min instead of max in terms-top-set 2016-08-22 16:01:42 +02:00
Mokaddem
2466f355ab Fixed bug in manage-terms-top-set + added auto sorting in important-paste-by-module 2016-08-22 14:54:37 +02:00
Mokaddem
e144e8e1c6 Fixed bug in terms-management-interface (only display one paste) + removed get_language of all pastes due to the huge overhead. 2016-08-22 13:59:12 +02:00
Mokaddem
d3c0fc5f94 Added terms trending module and web interface: 
- Curve module has been rewritten
- Added new module curve_manage_top_set
- Added all web-pages
 2016-08-22 13:35:49 +02:00
Mokaddem
841e87eb16 Fixed bug in sentiment trending, if there was no data for the avg -> there was one NaN leading to no display in the gauge. 2016-08-20 18:48:24 +02:00
Mokaddem
ee8f26c7c4 In sentiment trending, if there are no providers in the today\'s set, get data from random providers 2016-08-20 18:14:19 +02:00
Mokaddem
2ed60cb9bf Fixed typo 2016-08-20 16:44:36 +02:00
Mokaddem
3f1cf705fc Deleted alert() 2016-08-20 16:41:49 +02:00
Mokaddem
1c96eac2e4 Enter simulate button click on input field for terms management and plot tool 2016-08-20 16:38:51 +02:00
Mokaddem
465d18fc73 Added terms-plot tool 2016-08-20 16:23:06 +02:00
Mokaddem
fb3dea6bd2 Finnished terms-plot-top-set. Also added possibility to add terms to track or blacklist. 2016-08-20 15:59:22 +02:00
Mokaddem
a5117d7cad Added top set webpage 2016-08-20 11:46:22 +02:00
Mokaddem
0b6828fb16 Correctly handle and display add or remove a term in term-frequency + started sketch of terms-plot tool. 2016-08-19 16:53:46 +02:00
Mokaddem
8d41b109e7 Added query of the data for the new tracked word 2016-08-19 14:33:01 +02:00
Mokaddem
b78effe779 Added draft of terms_management + Added copyright in readme 2016-08-19 13:34:02 +02:00
Mokaddem
52b7124a6b - Modified redis connection (from levelDB to redis). 
- Added term frequency in curve.
- Modified ModuleStats and WebStats to use redis special command (incrby instead of get-set and zset) and Flask to perform the correct queries.
- Added panel color in sentiment trending
 2016-08-18 15:34:19 +02:00
Mokaddem
24072ece36 Added support of pasteName in search + added header in search.html + supressed mixed indent and trailing in Flask.py 2016-08-17 14:21:31 +02:00
Mokaddem
c10f28d9df Updated isolated launch scripts + Added switch displaying queue in index 2016-08-17 13:46:01 +02:00
Mokaddem
23b65eddca Auto unchecked info checkbox in index 2016-08-17 11:25:05 +02:00
Mokaddem
4d09ab0f6b Modified icon sentiment in web-header. 2016-08-17 11:13:37 +02:00
Mokaddem
44f54b7b81 Added dependencies 2016-08-17 11:04:48 +02:00
Mokaddem
f0b527167a Added script flexgauge 2016-08-17 10:50:30 +02:00
Mokaddem
c12e67a35a Added tooltips + slight gauge modification 2016-08-17 10:43:43 +02:00
Mokaddem
894b9efda9 Added drop of really long line in sentiment-analysis module + Added description of sentiment module. Also, fixed bug in webpage sentiement-trending concerning avg and date range. 2016-08-17 09:46:25 +02:00
Mokaddem
1084e45f1b Added draft of filter in sentiment analysis (Discard syntaxical languages) + Added nice tooltip for sparkline. Trending displays avg in function of the number of elements processed and not for the complete week + fixed bug in gauge and canvasjs (was performing avg with only 1 graph instead of all 8). 2016-08-16 16:33:02 +02:00
Mokaddem
ecd834ffb6 Added autoreload in sentiment-trending. 2016-08-16 11:27:49 +02:00
Mokaddem
bb677b9061 Added avg text in sentiment-trending + fixed date range bug 2016-08-16 11:23:14 +02:00
Mokaddem
cdd08d7df0 Added all providers for plot_tool in moduleStats + fixed date range bug 2016-08-16 11:07:36 +02:00
Mokaddem
8fa3c2b147 Added stacked choice or not for plot-tool (sentiments) 2016-08-16 10:49:05 +02:00
Mokaddem
4baac0b994 Tool-plot for sentiment working + Splitted script and webpage 2016-08-16 10:31:18 +02:00
Mokaddem
1e7547d16c Created draft of search for plot tool 2016-08-15 16:32:02 +02:00
Mokaddem
88ed8a239d Linked plot button with server 2016-08-15 15:54:53 +02:00
Mokaddem
82f70e61f4 Added provider checkbox 2016-08-15 15:16:31 +02:00
Mokaddem
f90987d284 Added date on slider 2016-08-15 14:53:46 +02:00
Mokaddem
d8c84ec674 Added link to data in analysis trending + fixed tooltip bug in sparkline 2016-08-15 12:57:39 +02:00
Mokaddem
6c7f660052 Added support of week and day average. 2016-08-13 22:17:11 +02:00
Mokaddem
1a5158b081 Added sentiment analyser module (draft) 2016-08-13 15:24:57 +02:00
Mokaddem
0431ecf434 Modified webpage display and added js. 2016-08-12 10:22:24 +02:00
Mokaddem
9c3b573af2 Splited js from html + Added some ID + small display modification. 2016-08-12 07:49:24 +02:00
Mokaddem
87a86b38d1 Added page Plot-tool 2016-08-11 21:08:43 +02:00
Mokaddem
c028ca80da Separated options from the plot funcion for sparkline. 2016-08-11 17:57:23 +02:00
Mokaddem
98c4cacb98 Finished first panel in sentiment-today. 2016-08-11 17:22:50 +02:00
Mokaddem
8682fcc29c Added empty webpages for sentiments modules. 2016-08-11 11:34:22 +02:00
Mokaddem
5612a77af9 Separated webpage header code from all webpage 2016-08-11 10:55:51 +02:00
Mokaddem
0ba5637e06 Added bar chart hover binder in trending charts 2016-08-10 11:16:56 +02:00
Mokaddem
24318f6a86 Added the part 'other' in Trending chartscd 2016-08-10 10:42:06 +02:00
Mokaddem
1ca8cf69f6 Added trendingchart.js 2016-08-10 09:38:15 +02:00
Mokaddem
b249676fe9 Removed console log 2016-08-09 16:54:53 +02:00
Mokaddem
507c83e06d In index webpage, reset value graph if no paste received 2016-08-09 16:48:57 +02:00
Mokaddem
943016d6ca Fixed one of the small display bug in DUplicate_ssdeep_v2 (was displaying [tlsh, tlsh] instead of simply tlsh) 2016-08-09 15:06:55 +02:00
Mokaddem
0d16a8beb7 Fixed bug where Listener was not placed if the row was not displayed (BrowseWarningPaste) 2016-08-09 12:30:40 +02:00
Mokaddem
e2ba4060ee restaured loading gif when new tab is shown (BrowseWarningPaste) 2016-08-09 12:07:20 +02:00
Mokaddem
a80f6ca0c2 Bug fix related with redis: 
Fixed typo key in redis for module creditcard and sqlinjection
Modified Curve redisLvlDb server
Modified Url.py so that it forwards name of protocol from saved protocolsfile
Added Cache control in Flask
Modified key-tab name into keys-tab
 2016-08-09 11:59:36 +02:00
Mokaddem
084737dd26 Fixed bug in flask not correctly displaying the number of paste/providers 2016-08-08 11:39:17 +02:00
Mokaddem
78c611fead Added warning_paste module and created related webpages. 
Fixed a Faup bug in credential (multiple instanciation) and added correc populate_set_out in concerned modules (creditcard, credential, ...).
Linked browse_warning_paste module and Flask function with redis (created new sets).
 2016-08-08 09:17:44 +02:00
Mokaddem
3523d79a05 Started browse important paste feature. Still need to link with redis. + Fixed a bug in Flask (String to list parser) 2016-08-04 16:57:34 +02:00
Mokaddem
55ef917ef1 Added few comments 2016-08-04 12:10:56 +02:00
Mokaddem
d9316771cd Draft: added new duplicate hash comparison - tlsh 2016-08-04 11:55:38 +02:00