Merge pull request #359 from CIRCL/user_management

User/role management

.gitignore

@@ -20,6 +20,8 @@ indexdir/
 logs/
 old/
 
+DEFAULT_PASSWORD
+
 # Webstuff
 var/www/static/
 !var/www/static/css/dygraph_gallery.css

OVERVIEW.md

@@ -30,6 +30,14 @@ Redis and ARDB overview
 
 # Database Map:
 
+### Redis cache
+
+##### Brute force protection:
+| Set Key | Value |
+| ------ | ------ |
+| failed_login_ip:**ip**           | **nb login failed** | TTL
+| failed_login_user_id:**user_id** | **nb login failed** | TTL
+
 ## DB0 - Core:
 
 ##### Update keys:
@@ -42,8 +50,6 @@ Redis and ARDB overview
 | | **background update name** |
 | | **...** |
 | | |
-| ail:update_date_v1.5 | **update date** |
-| | |
 | ail:update_error | **update message error** |
 | | |
 | ail:update_in_progress | **update version in progress** |
@@ -52,6 +58,30 @@ Redis and ARDB overview
 | ail:current_background_script | **name of the background script currently executed** |
 | ail:current_background_script_stat | **progress in % of the background script** |
 
+| Hset Key | Field | Value |
+| ------ | ------ | ------ |
+| ail:update_date | **update tag** | **update date**  |
+
+##### User Management:
+| Hset Key | Field | Value |
+| ------ | ------ | ------ |
+| user:all | **user id** | **password hash**  |
+| | | |
+| user:tokens | **token** | **user id** |
+| | | |
+| user_metadata:**user id** | token | **token**  |
+|                           | change_passwd  | **boolean** |
+|                           | role  | **role** |
+
+| Set Key | Value |
+| ------ | ------ |
+| user_role:**role** | **user id** |
+
+
+| Zrank Key | Field | Value |
+| ------ | ------ | ------ |
+| ail:all_role | **role** | **int, role priority (1=admin)** |
+
 ## DB2 - TermFreq:
 
 ##### Set:

README.md

@@ -78,14 +78,13 @@ Type these command lines for a fully automated installation and start AIL framew
 git clone https://github.com/CIRCL/AIL-framework.git
 cd AIL-framework
 ./installing_deps.sh
+
 cd ~/AIL-framework/
-. ./AILENV/bin/activate
 cd bin/
-./LAUNCH.sh
+./LAUNCH.sh -l
 ```
 
-The default [installing_deps.sh](./installing_deps.sh) is for Debian and Ubuntu based distributions. For Arch
+The default [installing_deps.sh](./installing_deps.sh) is for Debian and Ubuntu based distributions.
-linux based distributions, you can replace it with [installing_deps_archlinux.sh](./installing_deps_archlinux.sh).
 
 There is also a [Travis file](.travis.yml) used for automating the installation that can be used to build and install AIL on other systems.
 
@@ -143,23 +142,12 @@ Install using Ansible
 
 Please check the [Ansible readme](ansible/README.md).
 
-Starting AIL web interface
+Starting AIL
 --------------------------
 
-To start the web interface, you first need to fetch the required JavaScript/CSS files:
-
 ```bash
-cd $AILENV
+cd bin/
-cd var/www/
+./LAUNCH -l
-bash update_thirdparty.sh
-```
-
-and then you can start the web interface python script:
-
-```bash
-cd $AILENV
-cd var/www/
-./Flask_server.py
 ```
 
 Eventually you can browse the status of the AIL framework website at the following URL:
@@ -168,6 +156,8 @@ Eventually you can browse the status of the AIL framework website at the followi
 http://localhost:7000/
 ```
 
+The default credentials for the web interface are located in ``DEFAULT_PASSWORD``. This file is removed when you change your password.
+
 Training
 --------
 

bin/helper/gen_cert/README.md

@@ -0,0 +1,14 @@
+Usage
+=====
+These scripts are useful to generate the server's self-signed certificate.
+## Root Certification Authority ##
+`gen_root.sh`
+This will create a rootCA.crt to verify the server's cert.
+## Server certificate ##
+### Configuration Files ###
+Look into these files if you plan to verify the certificate:
+  * san.cnf holds basic information about the certificate.
+  * ext.3 holds in particular the 'subjectAltNames** option that is for the verification to succeed.
+
+ ### Generation ###
+`gen_root.sh`

bin/helper/gen_cert/ext3.cnf

@@ -0,0 +1,5 @@
+keyUsage = digitalSignature, nonRepudiation, keyEncipherment, dataEncipherment
+subjectAltName = @alt_names
+
+[alt_names]
+IP.1 = 127.0.0.1

bin/helper/gen_cert/gen_cert.sh

@@ -0,0 +1,9 @@
+#!/usr/bin/env bash
+# Create Server key
+openssl genrsa -out server.key 4096
+# Create the Server Signing Request - non interactive, config in san.cnf
+openssl req -sha256 -new -key server.key -out server.csr -config san.cnf
+# Create the server certificate by rootCA, with ext3 subjectAltName in ext3.cnf
+openssl x509 -req -in server.csr -CA rootCA.crt -CAkey rootCA.key -CAcreateserial -out server.crt -days 500 -sha256 -extfile ext3.cnf
+# Concat in pem
+cat server.crt server.key > server.pem

bin/helper/gen_cert/gen_root.sh

@@ -0,0 +1,5 @@
+#!/usr/bin/env bash 
+# Create Root key
+openssl genrsa -out rootCA.key 4096
+# Create and Sign the Root CA Certificate
+openssl req -x509 -new -nodes -key rootCA.key -sha256 -days 1024 -out rootCA.crt -config san.cnf

bin/helper/gen_cert/san.cnf

@@ -0,0 +1,11 @@
+[req]
+default_bits = 4096
+prompt = no
+default_md = sha256
+distinguished_name = dn
+
+[dn]
+C=LU
+L=Luxembourg
+O=AIL-Default
+CN = localhost

bin/packages/User.py

@@ -0,0 +1,72 @@
+#!/usr/bin/env python3
+# -*-coding:UTF-8 -*
+
+import os
+import redis
+import bcrypt
+import configparser
+
+from flask_login import UserMixin
+
+class User(UserMixin):
+
+    def __init__(self, id):
+
+        configfile = os.path.join(os.environ['AIL_BIN'], 'packages/config.cfg')
+        if not os.path.exists(configfile):
+            raise Exception('Unable to find the configuration file. \
+                            Did you set environment variables? \
+                            Or activate the virtualenv.')
+
+        cfg = configparser.ConfigParser()
+        cfg.read(configfile)
+
+        self.r_serv_db = redis.StrictRedis(
+            host=cfg.get("ARDB_DB", "host"),
+            port=cfg.getint("ARDB_DB", "port"),
+            db=cfg.getint("ARDB_DB", "db"),
+            decode_responses=True)
+
+        if self.r_serv_db.hexists('user:all', id):
+            self.id = id
+        else:
+            self.id = "__anonymous__"
+
+    # return True or False
+    #def is_authenticated():
+
+    # return True or False
+    #def is_anonymous():
+
+    @classmethod
+    def get(self_class, id):
+        return self_class(id)
+
+    def user_is_anonymous(self):
+        if self.id == "__anonymous__":
+            return True
+        else:
+            return False
+
+    def check_password(self, password):
+        if self.user_is_anonymous():
+            return False
+
+        password = password.encode()
+        hashed_password = self.r_serv_db.hget('user:all', self.id).encode()
+        if bcrypt.checkpw(password, hashed_password):
+            return True
+        else:
+            return False
+
+    def request_password_change(self):
+        if self.r_serv_db.hget('user_metadata:{}'.format(self.id), 'change_passwd') == 'True':
+            return True
+        else:
+            return False
+
+    def is_in_role(self, role):
+        if self.r_serv_db.sismember('user_role:{}'.format(role), self.id):
+            return True
+        else:
+            return False

installing_deps.sh

@@ -99,7 +99,17 @@ if [ -z "$VIRTUAL_ENV" ]; then
 
 fi
 
-pushd var/www/
+pushd ${AIL_BIN}helper/gen_cert
+./gen_root.sh
+wait
+./gen_cert.sh
+wait
+popd
+
+cp ${AIL_BIN}helper/gen_cert/server.crt ${AIL_FLASK}server.crt
+cp ${AIL_BIN}helper/gen_cert/server.key ${AIL_FLASK}server.key
+
+pushd ${AIL_FLASK}
 ./update_thirdparty.sh
 popd
 
@@ -123,6 +133,26 @@ python3 setup.py install
 HOME=$(pwd) python3 -m textblob.download_corpora
 python3 -m nltk.downloader vader_lexicon
 python3 -m nltk.downloader punkt
+popd
 
 #Create the file all_module and update the graph in doc
 $AIL_HOME/doc/generate_modules_data_flow_graph.sh
+
+#### DB SETUP ####
+
+# init update version
+git describe --abbrev=0 --tags | tr -d '\n' > ${AIL_HOME}/update/current_version
+
+# LAUNCH ARDB
+bash ${AIL_BIN}LAUNCH.sh -lav &
+wait
+echo ""
+
+# create default user
+pushd ${AIL_FLASK}
+python3 create_default_user.py
+popd
+
+bash ${AIL_BIN}LAUNCH.sh -k &
+wait
+echo ""

installing_deps_archlinux.sh

@@ -1,108 +0,0 @@
-#!/bin/bash
-
-
-echo "Currently unmaintained, continue at your own risk of not having a working AIL at the end :( Will be merged into main install deps later on."
-exit 1
-
-set -e
-set -x
-
-sudo pacman -Syu
-
-sudo pacman -S python2-pip screen gcc unzip freetype2 python2 git --needed
-sudo yaourt -S snappy --needed
-sudo pip2 install virtualenv
-
-#Needed for bloom filters
-sudo pacman -S openssl python2-numpy --needed
-
-# DNS deps
-sudo pacman -S adns --needed
-
-#Needed for redis-lvlDB
-sudo pacman -S libev gmp --needed
-
-#needed for mathplotlib
-test ! -L /usr/include/ft2build.h && sudo ln -s freetype2/ft2build.h /usr/include/
-sudo easy_install-2.7 -U distribute
-
-# REDIS #
-test ! -d redis/ && git clone https://github.com/antirez/redis.git
-pushd redis/
-git checkout 2.8
-make
-popd
-
-# REDIS LEVEL DB #
-test ! -d redis-leveldb/ && git clone https://github.com/KDr2/redis-leveldb.git
-pushd redis-leveldb/
-git submodule init
-git submodule update
-make
-popd
-
-# Faup
-test ! -d faup/ && git clone https://github.com/stricaud/faup.git
-pushd faup/
-test ! -d build && mkdir build
-cd build
-cmake .. && make
-sudo make install
-echo '/usr/local/lib' | sudo tee -a /etc/ld.so.conf.d/faup.conf
-sudo ldconfig
-popd
-
-# tlsh
-test ! -d tlsh && git clone git://github.com/trendmicro/tlsh.git
-pushd tlsh/
-./make.sh
-pushd build/release/
-sudo make install
-sudo ldconfig
-popd
-popd
-
-
-
-if [ ! -f bin/packages/config.cfg ]; then
-    cp bin/packages/config.cfg.sample bin/packages/config.cfg
-fi
-
-pushd var/www/
-./update_thirdparty.sh
-popd
-
-virtualenv AILENV
-
-echo export AIL_HOME=$(pwd) >> ./AILENV/bin/activate
-echo export AIL_BIN=$(pwd)/bin/ >> ./AILENV/bin/activate
-echo export AIL_FLASK=$(pwd)/var/www/ >> ./AILENV/bin/activate
-echo export AIL_REDIS=$(pwd)/redis/src/ >> ./AILENV/bin/activate
-echo export AIL_LEVELDB=$(pwd)/redis-leveldb/ >> ./AILENV/bin/activate
-
-. ./AILENV/bin/activate
-
-mkdir -p $AIL_HOME/{PASTES,Blooms,dumps}
-mkdir -p $AIL_HOME/LEVEL_DB_DATA/2017
-mkdir -p $AIL_HOME/LEVEL_DB_DATA/3017
-
-pip install -U pip
-pip install -U -r pip_packages_requirement.txt
-
-# Pyfaup
-pushd faup/src/lib/bindings/python/
-python setup.py install
-popd
-
-# Py tlsh
-pushd tlsh/py_ext
-python setup.py build
-python setup.py install
-
-# Download the necessary NLTK corpora and sentiment vader
-HOME=$(pwd) python -m textblob.download_corpora
-python -m nltk.downloader vader_lexicon
-python -m nltk.downloader punkt
-
-#Create the file all_module and update the graph in doc
-$AIL_HOME/doc/generate_modules_data_flow_graph.sh

requirements.txt

@@ -43,9 +43,12 @@ psutil
 phonenumbers
 
 ipython
-flask
 texttable
 
+flask
+flask-login
+bcrypt
+
 #DomainClassifier
 DomainClassifier
 #Indexer requirements

update/v2.0/Update.py

@@ -0,0 +1,41 @@
+#!/usr/bin/env python3
+# -*-coding:UTF-8 -*
+
+import os
+import sys
+import time
+import redis
+import datetime
+import configparser
+
+if __name__ == '__main__':
+
+    start_deb = time.time()
+
+    configfile = os.path.join(os.environ['AIL_BIN'], 'packages/config.cfg')
+    if not os.path.exists(configfile):
+        raise Exception('Unable to find the configuration file. \
+                        Did you set environment variables? \
+                        Or activate the virtualenv.')
+    cfg = configparser.ConfigParser()
+    cfg.read(configfile)
+
+    r_serv = redis.StrictRedis(
+        host=cfg.get("ARDB_DB", "host"),
+        port=cfg.getint("ARDB_DB", "port"),
+        db=cfg.getint("ARDB_DB", "db"),
+        decode_responses=True)
+
+    #Set current ail version
+    r_serv.set('ail:version', 'v2.0')
+
+    # use new update_date format
+    date_tag_to_replace = ['v1.5', 'v1.7']
+    for tag in date_tag_to_replace:
+        if r_serv.exists('ail:update_date_{}'.format(tag)):
+            date_tag = r_serv.get('ail:update_date_{}'.format(tag))
+            r_serv.hset('ail:update_date', tag, date_tag)
+            r_serv.delete('ail:update_date_{}'.format(tag))
+
+    #Set current ail version
+    r_serv.hset('ail:update_date', 'v2.0', datetime.datetime.now().strftime("%Y%m%d"))

update/v2.0/Update.sh

@@ -0,0 +1,75 @@
+#!/bin/bash
+
+[ -z "$AIL_HOME" ] && echo "Needs the env var AIL_HOME. Run the script from the virtual environment." && exit 1;
+[ -z "$AIL_REDIS" ] && echo "Needs the env var AIL_REDIS. Run the script from the virtual environment." && exit 1;
+[ -z "$AIL_ARDB" ] && echo "Needs the env var AIL_ARDB. Run the script from the virtual environment." && exit 1;
+[ -z "$AIL_BIN" ] && echo "Needs the env var AIL_ARDB. Run the script from the virtual environment." && exit 1;
+[ -z "$AIL_FLASK" ] && echo "Needs the env var AIL_FLASK. Run the script from the virtual environment." && exit 1;
+
+export PATH=$AIL_HOME:$PATH
+export PATH=$AIL_REDIS:$PATH
+export PATH=$AIL_ARDB:$PATH
+export PATH=$AIL_BIN:$PATH
+export PATH=$AIL_FLASK:$PATH
+
+GREEN="\\033[1;32m"
+DEFAULT="\\033[0;39m"
+
+echo -e $GREEN"Shutting down AIL ..."$DEFAULT
+bash ${AIL_BIN}/LAUNCH.sh -k
+wait
+
+echo ""
+echo -e $GREEN"Create Self-Signed Certificate"$DEFAULT
+echo ""
+pushd ${AIL_BIN}/helper/gen_cert
+bash gen_root.sh
+wait
+bash gen_cert.sh
+wait
+popd
+
+cp ${AIL_BIN}/helper/gen_cert/server.crt ${AIL_FLASK}/server.crt
+cp ${AIL_BIN}/helper/gen_cert/server.key ${AIL_FLASK}/server.key
+
+echo ""
+echo -e $GREEN"Update requirement"$DEFAULT
+echo ""
+pip3 install flask-login
+wait
+echo ""
+pip3 install bcrypt
+wait
+echo ""
+echo ""
+
+bash ${AIL_BIN}/LAUNCH.sh -lav &
+wait
+echo ""
+
+echo ""
+echo -e $GREEN"Updating AIL VERSION ..."$DEFAULT
+echo ""
+python ${AIL_HOME}/update/v2.0/Update.py
+wait
+echo ""
+echo ""
+
+echo ""
+echo -e $GREEN"Update thirdparty ..."$DEFAULT
+bash ${AIL_BIN}/LAUNCH.sh -t
+wait
+echo ""
+
+echo ""
+echo -e $GREEN"Create Default User"$DEFAULT
+echo ""
+python3 ${AIL_FLASK}create_default_user.py
+
+
+echo ""
+echo -e $GREEN"Shutting down ARDB ..."$DEFAULT
+bash ${AIL_BIN}/LAUNCH.sh -k
+wait
+
+exit 0

var/www/Flask_server.py

@@ -1,28 +1,42 @@
 #!/usr/bin/env python3
 # -*-coding:UTF-8 -*
 
-import redis
+import os
-import configparser
+import re
-import json
+import sys
-import datetime
+import ssl
 import time
-import calendar
+
-from flask import Flask, render_template, jsonify, request, Request
+import redis
+import random
+import logging
+import logging.handlers
+import configparser
+
+from flask import Flask, render_template, jsonify, request, Request, session, redirect, url_for
+from flask_login import LoginManager, current_user, login_user, logout_user, login_required
+
+import bcrypt
+
 import flask
 import importlib
-import os
 from os.path import join
-import sys
 sys.path.append(os.path.join(os.environ['AIL_BIN'], 'packages/'))
 sys.path.append('./modules/')
 import Paste
 from Date import Date
 
+from User import User
+
 from pytaxonomies import Taxonomies
 
 # Import config
 import Flask_config
 
+# Import Role_Manager
+from Role_Manager import create_user_db, check_password_strength, check_user_role_integrity
+from Role_Manager import login_admin, login_analyst
+
 # CONFIG #
 cfg = Flask_config.cfg
 baseUrl = cfg.get("Flask", "baseurl")
@@ -30,10 +44,63 @@ baseUrl = baseUrl.replace('/', '')
 if baseUrl != '':
     baseUrl = '/'+baseUrl
 
+# ========= REDIS =========#
+r_serv_db = redis.StrictRedis(
+    host=cfg.get("ARDB_DB", "host"),
+    port=cfg.getint("ARDB_DB", "port"),
+    db=cfg.getint("ARDB_DB", "db"),
+    decode_responses=True)
+r_serv_tags = redis.StrictRedis(
+    host=cfg.get("ARDB_Tags", "host"),
+    port=cfg.getint("ARDB_Tags", "port"),
+    db=cfg.getint("ARDB_Tags", "db"),
+    decode_responses=True)
+
+r_cache = redis.StrictRedis(
+    host=cfg.get("Redis_Cache", "host"),
+    port=cfg.getint("Redis_Cache", "port"),
+    db=cfg.getint("Redis_Cache", "db"),
+    decode_responses=True)
+
+# logs
+log_dir = os.path.join(os.environ['AIL_HOME'], 'logs')
+if not os.path.isdir(log_dir):
+    os.makedirs(logs_dir)
+
+log_filename = os.path.join(log_dir, 'flask_server.logs')
+logger = logging.getLogger()
+formatter = logging.Formatter('%(asctime)s - %(levelname)s - %(message)s')
+handler_log = logging.handlers.TimedRotatingFileHandler(log_filename, when="midnight", interval=1)
+handler_log.suffix = '%Y-%m-%d.log'
+handler_log.setFormatter(formatter)
+handler_log.setLevel(30)
+logger.addHandler(handler_log)
+logger.setLevel(30)
+
+# =========       =========#
+
+# =========  TLS  =========#
+ssl_context = ssl.SSLContext(ssl.PROTOCOL_TLSv1_2)
+ssl_context.load_cert_chain(certfile='server.crt', keyfile='server.key')
+#print(ssl_context.get_ciphers())
+# =========       =========#
+
 Flask_config.app = Flask(__name__, static_url_path=baseUrl+'/static/')
 app = Flask_config.app
 app.config['MAX_CONTENT_LENGTH'] = 900 * 1024 * 1024
 
+# ========= session ========
+app.secret_key = str(random.getrandbits(256))
+login_manager = LoginManager()
+login_manager.login_view = 'login'
+login_manager.init_app(app)
+
+# ========= LOGIN MANAGER ========
+
+@login_manager.user_loader
+def load_user(user_id):
+    return User.get(user_id)
+
 # ========= HEADER GENERATION ========
 
 # Get headers items that should be ignored (not displayed)
@@ -95,7 +162,6 @@ modified_header = modified_header.replace('<!--insert here-->', '\n'.join(to_add
 with open('templates/header.html', 'w') as f:
     f.write(modified_header)
 
-
 # ========= JINJA2 FUNCTIONS ========
 def list_len(s):
     return len(s)
@@ -113,18 +179,123 @@ def add_header(response):
     response.headers['Cache-Control'] = 'public, max-age=0'
     return response
 
+# @app.route('/test', methods=['GET'])
+# def test():
+#     for rule in app.url_map.iter_rules():
+#         print(rule)
+#     return 'o'
+
 # ========== ROUTES ============
+@app.route('/login', methods=['POST', 'GET'])
+def login():
+
+    current_ip = request.remote_addr
+    login_failed_ip = r_cache.get('failed_login_ip:{}'.format(current_ip))
+
+    # brute force by ip
+    if login_failed_ip:
+        login_failed_ip = int(login_failed_ip)
+        if login_failed_ip >= 5:
+            error = 'Max Connection Attempts reached, Please wait {}s'.format(r_cache.ttl('failed_login_ip:{}'.format(current_ip)))
+            return render_template("login.html", error=error)
+
+    if request.method == 'POST':
+        username = request.form.get('username')
+        password = request.form.get('password')
+        #next_page = request.form.get('next_page')
+
+        if username is not None:
+            user = User.get(username)
+            login_failed_user_id = r_cache.get('failed_login_user_id:{}'.format(username))
+            # brute force by user_id
+            if login_failed_user_id:
+                login_failed_user_id = int(login_failed_user_id)
+                if login_failed_user_id >= 5:
+                    error = 'Max Connection Attempts reached, Please wait {}s'.format(r_cache.ttl('failed_login_user_id:{}'.format(username)))
+                    return render_template("login.html", error=error)
+
+            if user and user.check_password(password):
+                if not check_user_role_integrity(user.get_id()):
+                    error = 'Incorrect User ACL, Please contact your administrator'
+                    return render_template("login.html", error=error)
+                login_user(user) ## TODO: use remember me ?
+                if user.request_password_change():
+                    return redirect(url_for('change_password'))
+                else:
+                    return redirect(url_for('dashboard.index'))
+            # login failed
+            else:
+                # set brute force protection
+                logger.warning("Login failed, ip={}, username={}".format(current_ip, username))
+                r_cache.incr('failed_login_ip:{}'.format(current_ip))
+                r_cache.expire('failed_login_ip:{}'.format(current_ip), 300)
+                r_cache.incr('failed_login_user_id:{}'.format(username))
+                r_cache.expire('failed_login_user_id:{}'.format(username), 300)
+                #
+
+                error = 'Password Incorrect'
+                return render_template("login.html", error=error)
+
+        return 'please provide a valid username'
+
+    else:
+        #next_page = request.args.get('next')
+        error = request.args.get('error')
+        return render_template("login.html" , error=error)
+
+@app.route('/change_password', methods=['POST', 'GET'])
+@login_required
+def change_password():
+    password1 = request.form.get('password1')
+    password2 = request.form.get('password2')
+    error = request.args.get('error')
+
+    if error:
+        return render_template("change_password.html", error=error)
+
+    if current_user.is_authenticated and password1!=None:
+        if password1==password2:
+            if check_password_strength(password1):
+                user_id = current_user.get_id()
+                create_user_db(user_id , password1, update=True)
+                return redirect(url_for('dashboard.index'))
+            else:
+                error = 'Incorrect password'
+                return render_template("change_password.html", error=error)
+        else:
+            error = "Passwords don't match"
+            return render_template("change_password.html", error=error)
+    else:
+        error = 'Please choose a new password'
+        return render_template("change_password.html", error=error)
+
+@app.route('/logout')
+@login_required
+def logout():
+    logout_user()
+    return redirect(url_for('login'))
+
+# role error template
+@app.route('/role', methods=['POST', 'GET'])
+@login_required
+def role():
+    return render_template("error/403.html"), 403
+
 @app.route('/searchbox/')
+@login_required
+@login_analyst
 def searchbox():
     return render_template("searchbox.html")
 
+# ========== ERROR HANDLER ============
+
+@app.errorhandler(404)
+@login_required
+def page_not_found(e):
+    # avoid endpoint enumeration
+    return render_template('error/404.html'), 404
 
 # ========== INITIAL taxonomies ============
-r_serv_tags = redis.StrictRedis(
-    host=cfg.get("ARDB_Tags", "host"),
-    port=cfg.getint("ARDB_Tags", "port"),
-    db=cfg.getint("ARDB_Tags", "db"),
-    decode_responses=True)
 # add default ail taxonomies
 r_serv_tags.sadd('active_taxonomies', 'infoleak')
 r_serv_tags.sadd('active_taxonomies', 'gdpr')
@@ -139,11 +310,6 @@ for tag in taxonomies.get('fpf').machinetags():
     r_serv_tags.sadd('active_tag_fpf', tag)
 
 # ========== INITIAL tags auto export ============
-r_serv_db = redis.StrictRedis(
-    host=cfg.get("ARDB_DB", "host"),
-    port=cfg.getint("ARDB_DB", "port"),
-    db=cfg.getint("ARDB_DB", "db"),
-    decode_responses=True)
 infoleak_tags = taxonomies.get('infoleak').machinetags()
 infoleak_automatic_tags = []
 for tag in taxonomies.get('infoleak').machinetags():
@@ -154,4 +320,4 @@ r_serv_db.sadd('list_export_tags', 'infoleak:submission="manual"')
 # ============ MAIN ============
 
 if __name__ == "__main__":
-    app.run(host='0.0.0.0', port=7000, threaded=True)
+    app.run(host='0.0.0.0', port=7000, threaded=True, ssl_context=ssl_context)

var/www/create_default_user.py

@@ -0,0 +1,53 @@
+#!/usr/bin/env python3
+# -*-coding:UTF-8 -*
+
+import os
+import sys
+import redis
+import secrets
+import configparser
+
+sys.path.append(os.path.join(os.environ['AIL_FLASK'], 'modules'))
+
+from Role_Manager import create_user_db, edit_user_db, get_default_admin_token
+
+
+
+configfile = os.path.join(os.environ['AIL_BIN'], 'packages/config.cfg')
+if not os.path.exists(configfile):
+    raise Exception('Unable to find the configuration file. \
+                    Did you set environment variables? \
+                    Or activate the virtualenv.')
+cfg = configparser.ConfigParser()
+cfg.read(configfile)
+
+r_serv = redis.StrictRedis(
+    host=cfg.get("ARDB_DB", "host"),
+    port=cfg.getint("ARDB_DB", "port"),
+    db=cfg.getint("ARDB_DB", "db"),
+    decode_responses=True)
+
+
+if __name__ == "__main__":
+
+    # create role_list
+    if not r_serv.exists('ail:all_role'):
+        r_serv.zadd('ail:all_role', 1, 'admin')
+        r_serv.zadd('ail:all_role', 2, 'analyst')
+
+    username = 'admin@admin.test'
+    password = secrets.token_urlsafe()
+    if r_serv.exists('user_metadata:admin@admin.test'):
+        edit_user_db(username, password=password, role='admin')
+    else:
+        create_user_db(username, password, role='admin', default=True)
+    token = get_default_admin_token()
+
+    default_passwd_file = os.path.join(os.environ['AIL_HOME'], 'DEFAULT_PASSWORD')
+    to_write_str = '# Password Generated by default\n# This file is deleted after the first login\n#\nemail=admin@admin.test\npassword='
+    to_write_str = to_write_str + password + '\nAPI_Key=' + token
+    with open(default_passwd_file, 'w') as f:
+        f.write(to_write_str)
+
+    print('new user created: {}'.format(username))
+    print('password: {}'.format(password))

var/www/modules/Flask_config.py

@@ -7,6 +7,7 @@
 import configparser
 import redis
 import os
+import re
 import sys
 
 # FLASK #
@@ -175,6 +176,9 @@ max_dashboard_logs = int(cfg.get("Flask", "max_dashboard_logs"))
 
 crawler_enabled = cfg.getboolean("Crawler", "activate_crawler")
 
+email_regex = r'[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,6}'
+email_regex = re.compile(email_regex)
+
 # VT
 try:
     from virusTotalKEYS import vt_key

var/www/modules/PasteSubmit/Flask_PasteSubmit.py

@@ -7,6 +7,9 @@
 import redis
 from flask import Flask, render_template, jsonify, request, Blueprint, url_for, redirect
 
+from Role_Manager import login_admin, login_analyst
+from flask_login import login_required
+
 import unicodedata
 import string
 import subprocess
@@ -273,6 +276,8 @@ def hive_create_case(hive_tlp, threat_level, hive_description, hive_case_title,
 # ============= ROUTES ==============
 
 @PasteSubmit.route("/PasteSubmit/", methods=['GET'])
+@login_required
+@login_analyst
 def PasteSubmit_page():
     #active taxonomies
     active_taxonomies = r_serv_tags.smembers('active_taxonomies')
@@ -285,6 +290,8 @@ def PasteSubmit_page():
                             active_galaxies = active_galaxies)
 
 @PasteSubmit.route("/PasteSubmit/submit", methods=['POST'])
+@login_required
+@login_analyst
 def submit():
 
     #paste_name = request.form['paste_name']
@@ -394,6 +401,8 @@ def submit():
     return PasteSubmit_page()
 
 @PasteSubmit.route("/PasteSubmit/submit_status", methods=['GET'])
+@login_required
+@login_analyst
 def submit_status():
     UUID = request.args.get('UUID')
 
@@ -460,6 +469,8 @@ def submit_status():
 
 
 @PasteSubmit.route("/PasteSubmit/create_misp_event", methods=['POST'])
+@login_required
+@login_analyst
 def create_misp_event():
 
     distribution = int(request.form['misp_data[Event][distribution]'])
@@ -482,6 +493,8 @@ def create_misp_event():
     return 'error0'
 
 @PasteSubmit.route("/PasteSubmit/create_hive_case", methods=['POST'])
+@login_required
+@login_analyst
 def create_hive_case():
 
     hive_tlp = int(request.form['hive_tlp'])
@@ -504,6 +517,8 @@ def create_hive_case():
     return 'error'
 
 @PasteSubmit.route("/PasteSubmit/edit_tag_export")
+@login_required
+@login_analyst
 def edit_tag_export():
     misp_auto_events = r_serv_db.get('misp:auto-events')
     hive_auto_alerts = r_serv_db.get('hive:auto-alerts')
@@ -568,6 +583,8 @@ def edit_tag_export():
                             flag_hive=flag_hive)
 
 @PasteSubmit.route("/PasteSubmit/tag_export_edited", methods=['POST'])
+@login_required
+@login_analyst
 def tag_export_edited():
     tag_enabled_misp = request.form.getlist('tag_enabled_misp')
     tag_enabled_hive = request.form.getlist('tag_enabled_hive')
@@ -592,26 +609,36 @@ def tag_export_edited():
     return redirect(url_for('PasteSubmit.edit_tag_export'))
 
 @PasteSubmit.route("/PasteSubmit/enable_misp_auto_event")
+@login_required
+@login_analyst
 def enable_misp_auto_event():
     r_serv_db.set('misp:auto-events', 1)
     return edit_tag_export()
 
 @PasteSubmit.route("/PasteSubmit/disable_misp_auto_event")
+@login_required
+@login_analyst
 def disable_misp_auto_event():
     r_serv_db.set('misp:auto-events', 0)
     return edit_tag_export()
 
 @PasteSubmit.route("/PasteSubmit/enable_hive_auto_alert")
+@login_required
+@login_analyst
 def enable_hive_auto_alert():
     r_serv_db.set('hive:auto-alerts', 1)
     return edit_tag_export()
 
 @PasteSubmit.route("/PasteSubmit/disable_hive_auto_alert")
+@login_required
+@login_analyst
 def disable_hive_auto_alert():
     r_serv_db.set('hive:auto-alerts', 0)
     return edit_tag_export()
 
 @PasteSubmit.route("/PasteSubmit/add_push_tag")
+@login_required
+@login_analyst
 def add_push_tag():
     tag = request.args.get('tag')
     if tag is not None:
@@ -629,6 +656,8 @@ def add_push_tag():
         return 'None args', 400
 
 @PasteSubmit.route("/PasteSubmit/delete_push_tag")
+@login_required
+@login_analyst
 def delete_push_tag():
     tag = request.args.get('tag')
 

var/www/modules/Role_Manager.py

@@ -0,0 +1,184 @@
+#!/usr/bin/env python3
+# -*-coding:UTF-8 -*
+
+import os
+import re
+import redis
+import bcrypt
+import secrets
+import configparser
+
+from functools import wraps
+from flask_login import LoginManager, current_user, login_user, logout_user, login_required
+
+from flask import request, current_app
+
+login_manager = LoginManager()
+login_manager.login_view = 'role'
+
+# CONFIG #
+configfile = os.path.join(os.environ['AIL_BIN'], 'packages/config.cfg')
+if not os.path.exists(configfile):
+    raise Exception('Unable to find the configuration file. \
+                    Did you set environment variables? \
+                    Or activate the virtualenv.')
+
+cfg = configparser.ConfigParser()
+cfg.read(configfile)
+
+r_serv_db = redis.StrictRedis(
+    host=cfg.get("ARDB_DB", "host"),
+    port=cfg.getint("ARDB_DB", "port"),
+    db=cfg.getint("ARDB_DB", "db"),
+    decode_responses=True)
+
+default_passwd_file = os.path.join(os.environ['AIL_HOME'], 'DEFAULT_PASSWORD')
+
+regex_password = r'^(?=(.*\d){2})(?=.*[a-z])(?=.*[A-Z]).{10,100}$'
+regex_password = re.compile(regex_password)
+
+###############################################################
+###############       CHECK ROLE ACCESS      ##################
+###############################################################
+
+def login_admin(func):
+    @wraps(func)
+    def decorated_view(*args, **kwargs):
+        if not current_user.is_authenticated:
+            return login_manager.unauthorized()
+        elif (not current_user.is_in_role('admin')):
+            return login_manager.unauthorized()
+        return func(*args, **kwargs)
+    return decorated_view
+
+def login_analyst(func):
+    @wraps(func)
+    def decorated_view(*args, **kwargs):
+        if not current_user.is_authenticated:
+            return login_manager.unauthorized()
+        elif (not current_user.is_in_role('analyst')):
+            return login_manager.unauthorized()
+        return func(*args, **kwargs)
+    return decorated_view
+
+
+
+###############################################################
+###############################################################
+###############################################################
+
+def generate_new_token(user_id):
+    # create user token
+    current_token = r_serv_db.hget('user_metadata:{}'.format(user_id), 'token')
+    r_serv_db.hdel('user:tokens', current_token)
+    token = secrets.token_urlsafe(41)
+    r_serv_db.hset('user:tokens', token, user_id)
+    r_serv_db.hset('user_metadata:{}'.format(user_id), 'token', token)
+
+def get_default_admin_token():
+    if r_serv_db.exists('user_metadata:admin@admin.test'):
+        return r_serv_db.hget('user_metadata:admin@admin.test', 'token')
+    else:
+        return ''
+
+def create_user_db(username_id , password, default=False, role=None, update=False):
+    password = password.encode()
+    password_hash = hashing_password(password)
+
+    # create user token
+    generate_new_token(username_id)
+
+    if update:
+        r_serv_db.hdel('user_metadata:{}'.format(username_id), 'change_passwd')
+        # remove default user password file
+        if username_id=='admin@admin.test':
+            os.remove(default_passwd_file)
+    else:
+        if default:
+            r_serv_db.hset('user_metadata:{}'.format(username_id), 'change_passwd', True)
+        if role:
+            if role in get_all_role():
+                for role_to_add in get_all_user_role(role):
+                    r_serv_db.sadd('user_role:{}'.format(role_to_add), username_id)
+                r_serv_db.hset('user_metadata:{}'.format(username_id), 'role', role)
+
+    r_serv_db.hset('user:all', username_id, password_hash)
+
+def edit_user_db(user_id, role, password=None):
+    if password:
+        password_hash = hashing_password(password.encode())
+        r_serv_db.hset('user:all', user_id, password_hash)
+
+    current_role = r_serv_db.hget('user_metadata:{}'.format(user_id), 'role')
+    if role != current_role:
+        request_level = get_role_level(role)
+        current_role = get_role_level(current_role)
+
+        if current_role < request_level:
+            role_to_remove = get_user_role_by_range(current_role -1, request_level - 2)
+            for role_id in role_to_remove:
+                r_serv_db.srem('user_role:{}'.format(role_id), user_id)
+            r_serv_db.hset('user_metadata:{}'.format(user_id), 'role', role)
+        else:
+            role_to_add = get_user_role_by_range(request_level -1, current_role)
+            for role_id in role_to_add:
+                r_serv_db.sadd('user_role:{}'.format(role_id), user_id)
+            r_serv_db.hset('user_metadata:{}'.format(user_id), 'role', role)
+
+def delete_user_db(user_id):
+    if r_serv_db.exists('user_metadata:{}'.format(user_id)):
+        role_to_remove =get_all_role()
+        for role_id in role_to_remove:
+            r_serv_db.srem('user_role:{}'.format(role_id), user_id)
+        user_token = r_serv_db.hget('user_metadata:{}'.format(user_id), 'token')
+        r_serv_db.hdel('user:tokens', user_token)
+        r_serv_db.delete('user_metadata:{}'.format(user_id))
+        r_serv_db.hdel('user:all', user_id)
+
+def hashing_password(bytes_password):
+    hashed = bcrypt.hashpw(bytes_password, bcrypt.gensalt())
+    return hashed
+
+def check_password_strength(password):
+    result = regex_password.match(password)
+    if result:
+        return True
+    else:
+        return False
+
+def get_all_role():
+    return r_serv_db.zrange('ail:all_role', 0, -1)
+
+def get_role_level(role):
+    return int(r_serv_db.zscore('ail:all_role', role))
+
+def get_all_user_role(user_role):
+    current_role_val = get_role_level(user_role)
+    return r_serv_db.zrange('ail:all_role', current_role_val -1, -1)
+
+def get_all_user_upper_role(user_role):
+    current_role_val = get_role_level(user_role)
+    # remove one rank
+    if current_role_val > 1:
+        return r_serv_db.zrange('ail:all_role', 0, current_role_val -2)
+    else:
+        return []
+
+def get_user_role_by_range(inf, sup):
+    return r_serv_db.zrange('ail:all_role', inf, sup)
+
+def get_user_role(user_id):
+    return r_serv_db.hget('user_metadata:{}'.format(user_id), 'role')
+
+def check_user_role_integrity(user_id):
+    user_role = get_user_role(user_id)
+    all_user_role = get_all_user_role(user_role)
+    res = True
+    for role in all_user_role:
+        if not r_serv_db.sismember('user_role:{}'.format(role), user_id):
+            res = False
+    upper_role = get_all_user_upper_role(user_role)
+    for role in upper_role:
+        if r_serv_db.sismember('user_role:{}'.format(role), user_id):
+            res = False
+    return res

var/www/modules/Tags/Flask_Tags.py

@@ -7,6 +7,9 @@
 import redis
 from flask import Flask, render_template, jsonify, request, Blueprint, redirect, url_for
 
+from Role_Manager import login_admin, login_analyst
+from flask_login import login_required
+
 import json
 import datetime
 
@@ -218,6 +221,8 @@ def update_tag_last_seen(tag, tag_first_seen, tag_last_seen):
 # ============= ROUTES ==============
 
 @Tags.route("/tags/", methods=['GET'])
+@login_required
+@login_analyst
 def Tags_page():
     date_from = request.args.get('date_from')
     date_to = request.args.get('date_to')
@@ -351,6 +356,8 @@ def Tags_page():
 
 
 @Tags.route("/Tags/get_all_tags")
+@login_required
+@login_analyst
 def get_all_tags():
 
     all_tags = r_serv_tags.smembers('list_tags')
@@ -373,6 +380,8 @@ def get_all_tags():
     return jsonify(list_tags)
 
 @Tags.route("/Tags/get_all_tags_taxonomies")
+@login_required
+@login_analyst
 def get_all_tags_taxonomies():
 
     taxonomies = Taxonomies()
@@ -390,6 +399,8 @@ def get_all_tags_taxonomies():
     return jsonify(list_tags)
 
 @Tags.route("/Tags/get_all_tags_galaxies")
+@login_required
+@login_analyst
 def get_all_tags_galaxy():
 
     active_galaxies = r_serv_tags.smembers('active_galaxies')
@@ -403,6 +414,8 @@ def get_all_tags_galaxy():
     return jsonify(list_tags)
 
 @Tags.route("/Tags/get_tags_taxonomie")
+@login_required
+@login_analyst
 def get_tags_taxonomie():
 
     taxonomie = request.args.get('taxonomie')
@@ -429,6 +442,8 @@ def get_tags_taxonomie():
         return 'INCORRECT INPUT'
 
 @Tags.route("/Tags/get_tags_galaxy")
+@login_required
+@login_analyst
 def get_tags_galaxy():
 
     galaxy = request.args.get('galaxy')
@@ -449,6 +464,8 @@ def get_tags_galaxy():
         return 'this galaxy is disable'
 
 @Tags.route("/Tags/remove_tag")
+@login_required
+@login_analyst
 def remove_tag():
 
     #TODO verify input
@@ -460,6 +477,8 @@ def remove_tag():
     return redirect(url_for('showsavedpastes.showsavedpaste', paste=path))
 
 @Tags.route("/Tags/confirm_tag")
+@login_required
+@login_analyst
 def confirm_tag():
 
     #TODO verify input
@@ -478,6 +497,8 @@ def confirm_tag():
     return 'incompatible tag'
 
 @Tags.route("/Tags/tag_validation")
+@login_required
+@login_analyst
 def tag_validation():
 
     path = request.args.get('paste')
@@ -498,6 +519,8 @@ def tag_validation():
         return 'input error'
 
 @Tags.route("/Tags/addTags")
+@login_required
+@login_analyst
 def addTags():
 
     tags = request.args.get('tags')
@@ -547,6 +570,8 @@ def addTags():
 
 
 @Tags.route("/Tags/taxonomies")
+@login_required
+@login_analyst
 def taxonomies():
 
     active_taxonomies = r_serv_tags.smembers('active_taxonomies')
@@ -583,6 +608,8 @@ def taxonomies():
                             n_tags=n_tags)
 
 @Tags.route("/Tags/edit_taxonomie")
+@login_required
+@login_analyst
 def edit_taxonomie():
 
     taxonomies = Taxonomies()
@@ -631,6 +658,8 @@ def edit_taxonomie():
         return 'INVALID TAXONOMIE'
 
 @Tags.route("/Tags/disable_taxonomie")
+@login_required
+@login_analyst
 def disable_taxonomie():
 
     taxonomies = Taxonomies()
@@ -651,6 +680,8 @@ def disable_taxonomie():
 
 
 @Tags.route("/Tags/active_taxonomie")
+@login_required
+@login_analyst
 def active_taxonomie():
 
     taxonomies = Taxonomies()
@@ -670,6 +701,8 @@ def active_taxonomie():
         return "INCORRECT INPUT"
 
 @Tags.route("/Tags/edit_taxonomie_tag")
+@login_required
+@login_analyst
 def edit_taxonomie_tag():
 
     taxonomies = Taxonomies()
@@ -712,6 +745,8 @@ def edit_taxonomie_tag():
         return "INCORRECT INPUT"
 
 @Tags.route("/Tags/galaxies")
+@login_required
+@login_analyst
 def galaxies():
 
     active_galaxies = r_serv_tags.smembers('active_galaxies')
@@ -758,6 +793,8 @@ def galaxies():
 
 
 @Tags.route("/Tags/edit_galaxy")
+@login_required
+@login_analyst
 def edit_galaxy():
 
     id = request.args.get('galaxy')
@@ -825,6 +862,8 @@ def edit_galaxy():
 
 
 @Tags.route("/Tags/active_galaxy")
+@login_required
+@login_analyst
 def active_galaxy():
 
     id = request.args.get('galaxy')
@@ -869,6 +908,8 @@ def active_galaxy():
 
 
 @Tags.route("/Tags/disable_galaxy")
+@login_required
+@login_analyst
 def disable_galaxy():
 
     id = request.args.get('galaxy')
@@ -889,6 +930,8 @@ def disable_galaxy():
 
 
 @Tags.route("/Tags/edit_galaxy_tag")
+@login_required
+@login_analyst
 def edit_galaxy_tag():
 
     arg1 = request.args.getlist('tag_enabled')
@@ -961,6 +1004,8 @@ def edit_galaxy_tag():
         return "INCORRECT INPUT"
 
 @Tags.route("/Tags/tag_galaxy_info")
+@login_required
+@login_analyst
 def tag_galaxy_info():
 
     galaxy = request.args.get('galaxy')

var/www/modules/dashboard/Flask_dashboard.py

@@ -14,6 +14,9 @@ from Date import Date
 
 from flask import Flask, render_template, jsonify, request, Blueprint, url_for
 
+from Role_Manager import login_admin, login_analyst
+from flask_login import login_required
+
 # ============ VARIABLES ============
 import Flask_config
 
@@ -109,10 +112,14 @@ def datetime_from_utc_to_local(utc_str):
 # ============ ROUTES ============
 
 @dashboard.route("/_logs")
+@login_required
+@login_analyst
 def logs():
     return flask.Response(event_stream(), mimetype="text/event-stream")
 
 @dashboard.route("/_get_last_logs_json")
+@login_required
+@login_analyst
 def get_last_logs_json():
     date = datetime.datetime.now().strftime("%Y%m%d")
 
@@ -154,11 +161,15 @@ def get_last_logs_json():
 
 
 @dashboard.route("/_stuff", methods=['GET'])
+@login_required
+@login_analyst
 def stuff():
     return jsonify(row1=get_queues(r_serv))
 
 
 @dashboard.route("/")
+@login_required
+@login_analyst
 def index():
     default_minute = cfg.get("Flask", "minute_processed_paste")
     threshold_stucked_module = cfg.getint("Module_ModuleInformation", "threshold_stucked_module")

var/www/modules/hashDecoded/Flask_hashDecoded.py

@@ -17,6 +17,8 @@ from hashlib import sha256
 
 import requests
 from flask import Flask, render_template, jsonify, request, Blueprint, redirect, url_for, send_file
+from Role_Manager import login_admin, login_analyst
+from flask_login import login_required
 
 # ============ VARIABLES ============
 import Flask_config
@@ -474,6 +476,8 @@ def correlation_graph_node_json(correlation_type, type_id, key_id):
 
 # ============= ROUTES ==============
 @hashDecoded.route("/hashDecoded/all_hash_search", methods=['POST'])
+@login_required
+@login_analyst
 def all_hash_search():
     date_from = request.form.get('date_from')
     date_to = request.form.get('date_to')
@@ -483,6 +487,8 @@ def all_hash_search():
     return redirect(url_for('hashDecoded.hashDecoded_page', date_from=date_from, date_to=date_to, type=type, encoding=encoding, show_decoded_files=show_decoded_files))
 
 @hashDecoded.route("/hashDecoded/", methods=['GET'])
+@login_required
+@login_analyst
 def hashDecoded_page():
     date_from = request.args.get('date_from')
     date_to = request.args.get('date_to')
@@ -600,6 +606,8 @@ def hashDecoded_page():
 
 
 @hashDecoded.route('/hashDecoded/hash_by_type')
+@login_required
+@login_analyst
 def hash_by_type():
     type = request.args.get('type')
     type = 'text/plain'
@@ -607,12 +615,16 @@ def hash_by_type():
 
 
 @hashDecoded.route('/hashDecoded/hash_hash')
+@login_required
+@login_analyst
 def hash_hash():
     hash = request.args.get('hash')
     return render_template('hash_hash.html')
 
 
 @hashDecoded.route('/hashDecoded/showHash')
+@login_required
+@login_analyst
 def showHash():
     hash = request.args.get('hash')
     #hash = 'e02055d3efaad5d656345f6a8b1b6be4fe8cb5ea'
@@ -666,6 +678,8 @@ def showHash():
 
 
 @hashDecoded.route('/hashDecoded/downloadHash')
+@login_required
+@login_analyst
 def downloadHash():
     hash = request.args.get('hash')
     # sanitize hash
@@ -702,6 +716,8 @@ def downloadHash():
 
 
 @hashDecoded.route('/hashDecoded/hash_by_type_json')
+@login_required
+@login_analyst
 def hash_by_type_json():
     type = request.args.get('type')
 
@@ -735,6 +751,8 @@ def hash_by_type_json():
 
 
 @hashDecoded.route('/hashDecoded/decoder_type_json')
+@login_required
+@login_analyst
 def decoder_type_json():
     date_from = request.args.get('date_from')
     date_to = request.args.get('date_to')
@@ -790,6 +808,8 @@ def decoder_type_json():
 
 
 @hashDecoded.route('/hashDecoded/top5_type_json')
+@login_required
+@login_analyst
 def top5_type_json():
     date_from = request.args.get('date_from')
     date_to = request.args.get('date_to')
@@ -848,6 +868,8 @@ def top5_type_json():
 
 
 @hashDecoded.route('/hashDecoded/daily_type_json')
+@login_required
+@login_analyst
 def daily_type_json():
     date = request.args.get('date')
 
@@ -867,6 +889,8 @@ def daily_type_json():
 
 
 @hashDecoded.route('/hashDecoded/range_type_json')
+@login_required
+@login_analyst
 def range_type_json():
     date_from = request.args.get('date_from')
     date_to = request.args.get('date_to')
@@ -923,6 +947,8 @@ def range_type_json():
 
 
 @hashDecoded.route('/hashDecoded/hash_graph_line_json')
+@login_required
+@login_analyst
 def hash_graph_line_json():
     hash = request.args.get('hash')
     date_from = request.args.get('date_from')
@@ -952,6 +978,8 @@ def hash_graph_line_json():
 
 
 @hashDecoded.route('/hashDecoded/hash_graph_node_json')
+@login_required
+@login_analyst
 def hash_graph_node_json():
     hash = request.args.get('hash')
 
@@ -1019,6 +1047,8 @@ def hash_graph_node_json():
 
 
 @hashDecoded.route('/hashDecoded/hash_types')
+@login_required
+@login_analyst
 def hash_types():
     date_from = 20180701
     date_to = 20180706
@@ -1026,6 +1056,8 @@ def hash_types():
 
 
 @hashDecoded.route('/hashDecoded/send_file_to_vt_js')
+@login_required
+@login_analyst
 def send_file_to_vt_js():
     hash = request.args.get('hash')
 
@@ -1049,6 +1081,8 @@ def send_file_to_vt_js():
 
 
 @hashDecoded.route('/hashDecoded/update_vt_result')
+@login_required
+@login_analyst
 def update_vt_result():
     hash = request.args.get('hash')
 
@@ -1085,6 +1119,8 @@ def update_vt_result():
 ############################ PGPDump ############################
 
 @hashDecoded.route('/decoded/pgp_by_type_json') ## TODO: REFRACTOR
+@login_required
+@login_analyst
 def pgp_by_type_json():
     type_id = request.args.get('type_id')
     date_from = request.args.get('date_from')
@@ -1129,6 +1165,8 @@ def pgp_by_type_json():
 
 ############################ Correlation ############################
 @hashDecoded.route("/correlation/pgpdump", methods=['GET'])
+@login_required
+@login_analyst
 def pgpdump_page():
     date_from = request.args.get('date_from')
     date_to = request.args.get('date_to')
@@ -1139,6 +1177,8 @@ def pgpdump_page():
     return res
 
 @hashDecoded.route("/correlation/cryptocurrency", methods=['GET'])
+@login_required
+@login_analyst
 def cryptocurrency_page():
     date_from = request.args.get('date_from')
     date_to = request.args.get('date_to')
@@ -1149,6 +1189,8 @@ def cryptocurrency_page():
     return res
 
 @hashDecoded.route("/correlation/all_pgpdump_search", methods=['POST'])
+@login_required
+@login_analyst
 def all_pgpdump_search():
     date_from = request.form.get('date_from')
     date_to = request.form.get('date_to')
@@ -1157,6 +1199,8 @@ def all_pgpdump_search():
     return redirect(url_for('hashDecoded.pgpdump_page', date_from=date_from, date_to=date_to, type_id=type_id, show_decoded_files=show_decoded_files))
 
 @hashDecoded.route("/correlation/all_cryptocurrency_search", methods=['POST'])
+@login_required
+@login_analyst
 def all_cryptocurrency_search():
     date_from = request.form.get('date_from')
     date_to = request.form.get('date_to')
@@ -1165,6 +1209,8 @@ def all_cryptocurrency_search():
     return redirect(url_for('hashDecoded.cryptocurrency_page', date_from=date_from, date_to=date_to, type_id=type_id, show_decoded_files=show_decoded_files))
 
 @hashDecoded.route('/correlation/show_pgpdump')
+@login_required
+@login_analyst
 def show_pgpdump():
     type_id = request.args.get('type_id')
     key_id = request.args.get('key_id')
@@ -1172,36 +1218,48 @@ def show_pgpdump():
 
 
 @hashDecoded.route('/correlation/show_cryptocurrency')
+@login_required
+@login_analyst
 def show_cryptocurrency():
     type_id = request.args.get('type_id')
     key_id = request.args.get('key_id')
     return show_correlation('cryptocurrency', type_id, key_id)
 
 @hashDecoded.route('/correlation/cryptocurrency_range_type_json')
+@login_required
+@login_analyst
 def cryptocurrency_range_type_json():
     date_from = request.args.get('date_from')
     date_to = request.args.get('date_to')
     return correlation_type_range_type_json('cryptocurrency', date_from, date_to)
 
 @hashDecoded.route('/correlation/pgpdump_range_type_json')
+@login_required
+@login_analyst
 def pgpdump_range_type_json():
     date_from = request.args.get('date_from')
     date_to = request.args.get('date_to')
     return correlation_type_range_type_json('pgpdump', date_from, date_to)
 
 @hashDecoded.route('/correlation/pgpdump_graph_node_json')
+@login_required
+@login_analyst
 def pgpdump_graph_node_json():
     type_id = request.args.get('type_id')
     key_id = request.args.get('key_id')
     return correlation_graph_node_json('pgpdump', type_id, key_id)
 
 @hashDecoded.route('/correlation/cryptocurrency_graph_node_json')
+@login_required
+@login_analyst
 def cryptocurrency_graph_node_json():
     type_id = request.args.get('type_id')
     key_id = request.args.get('key_id')
     return correlation_graph_node_json('cryptocurrency', type_id, key_id)
 
 @hashDecoded.route('/correlation/pgpdump_graph_line_json')
+@login_required
+@login_analyst
 def pgpdump_graph_line_json():
     type_id = request.args.get('type_id')
     key_id = request.args.get('key_id')
@@ -1234,6 +1292,8 @@ def correlation_graph_line_json(correlation_type, type_id, key_id, date_from, da
         return jsonify()
 
 @hashDecoded.route('/correlation/cryptocurrency_graph_line_json')
+@login_required
+@login_analyst
 def cryptocurrency_graph_line_json():
     type_id = request.args.get('type_id')
     key_id = request.args.get('key_id')

var/www/modules/hiddenServices/Flask_hiddenServices.py

@@ -13,6 +13,9 @@ import json
 from pyfaup.faup import Faup
 from flask import Flask, render_template, jsonify, request, send_file, Blueprint, redirect, url_for
 
+from Role_Manager import login_admin, login_analyst
+from flask_login import login_required
+
 from Date import Date
 from HiddenServices import HiddenServices
 
@@ -239,6 +242,8 @@ def delete_auto_crawler(url):
 # ============= ROUTES ==============
 
 @hiddenServices.route("/crawlers/", methods=['GET'])
+@login_required
+@login_analyst
 def dashboard():
     crawler_metadata_onion = get_crawler_splash_status('onion')
     crawler_metadata_regular = get_crawler_splash_status('regular')
@@ -253,15 +258,15 @@ def dashboard():
                                 crawler_metadata_regular=crawler_metadata_regular,
                                 statDomains_onion=statDomains_onion, statDomains_regular=statDomains_regular)
 
-@hiddenServices.route("/hiddenServices/2", methods=['GET'])
-def hiddenServices_page_test():
-    return render_template("Crawler_index.html")
-
 @hiddenServices.route("/crawlers/manual", methods=['GET'])
+@login_required
+@login_analyst
 def manual():
     return render_template("Crawler_Splash_manual.html", crawler_enabled=crawler_enabled)
 
 @hiddenServices.route("/crawlers/crawler_splash_onion", methods=['GET'])
+@login_required
+@login_analyst
 def crawler_splash_onion():
     type = 'onion'
     last_onions = get_last_domains_crawled(type)
@@ -279,6 +284,8 @@ def crawler_splash_onion():
                             crawler_metadata=crawler_metadata, date_from=date_string, date_to=date_string)
 
 @hiddenServices.route("/crawlers/Crawler_Splash_last_by_type", methods=['GET'])
+@login_required
+@login_analyst
 def Crawler_Splash_last_by_type():
     type = request.args.get('type')
     # verify user input
@@ -302,6 +309,8 @@ def Crawler_Splash_last_by_type():
                             crawler_metadata=crawler_metadata, date_from=date_string, date_to=date_string)
 
 @hiddenServices.route("/crawlers/blacklisted_domains", methods=['GET'])
+@login_required
+@login_analyst
 def blacklisted_domains():
     blacklist_domain = request.args.get('blacklist_domain')
     unblacklist_domain = request.args.get('unblacklist_domain')
@@ -336,6 +345,8 @@ def blacklisted_domains():
         return 'Incorrect Type'
 
 @hiddenServices.route("/crawler/blacklist_domain", methods=['GET'])
+@login_required
+@login_analyst
 def blacklist_domain():
     domain = request.args.get('domain')
     type = request.args.get('type')
@@ -357,6 +368,8 @@ def blacklist_domain():
         return 'Incorrect type'
 
 @hiddenServices.route("/crawler/unblacklist_domain", methods=['GET'])
+@login_required
+@login_analyst
 def unblacklist_domain():
     domain = request.args.get('domain')
     type = request.args.get('type')
@@ -378,6 +391,8 @@ def unblacklist_domain():
         return 'Incorrect type'
 
 @hiddenServices.route("/crawlers/create_spider_splash", methods=['POST'])
+@login_required
+@login_analyst
 def create_spider_splash():
     url = request.form.get('url_to_crawl')
     automatic = request.form.get('crawler_type')
@@ -464,6 +479,8 @@ def create_spider_splash():
     return redirect(url_for('hiddenServices.manual'))
 
 @hiddenServices.route("/crawlers/auto_crawler", methods=['GET'])
+@login_required
+@login_analyst
 def auto_crawler():
     nb_element_to_display = 100
     try:
@@ -516,6 +533,8 @@ def auto_crawler():
                                 auto_crawler_domain_regular_metadata=auto_crawler_domain_regular_metadata)
 
 @hiddenServices.route("/crawlers/remove_auto_crawler", methods=['GET'])
+@login_required
+@login_analyst
 def remove_auto_crawler():
     url = request.args.get('url')
     page = request.args.get('page')
@@ -525,6 +544,8 @@ def remove_auto_crawler():
     return redirect(url_for('hiddenServices.auto_crawler', page=page))
 
 @hiddenServices.route("/crawlers/crawler_dashboard_json", methods=['GET'])
+@login_required
+@login_analyst
 def crawler_dashboard_json():
 
     crawler_metadata_onion = get_crawler_splash_status('onion')
@@ -541,6 +562,8 @@ def crawler_dashboard_json():
 
 # # TODO: refractor
 @hiddenServices.route("/hiddenServices/last_crawled_domains_with_stats_json", methods=['GET'])
+@login_required
+@login_analyst
 def last_crawled_domains_with_stats_json():
     last_onions = r_serv_onion.lrange('last_onion', 0 ,-1)
     list_onion = []
@@ -590,6 +613,8 @@ def last_crawled_domains_with_stats_json():
     return jsonify({'last_onions': list_onion, 'statDomains': statDomains, 'crawler_metadata':crawler_metadata})
 
 @hiddenServices.route("/hiddenServices/get_onions_by_daterange", methods=['POST'])
+@login_required
+@login_analyst
 def get_onions_by_daterange():
     date_from = request.form.get('date_from')
     date_to = request.form.get('date_to')
@@ -601,6 +626,8 @@ def get_onions_by_daterange():
     return redirect(url_for('hiddenServices.show_domains_by_daterange', date_from=date_from, date_to=date_to, service_type=service_type, domains_up=domains_up, domains_down=domains_down, domains_tags=domains_tags))
 
 @hiddenServices.route("/hiddenServices/show_domains_by_daterange", methods=['GET'])
+@login_required
+@login_analyst
 def show_domains_by_daterange():
     date_from = request.args.get('date_from')
     date_to = request.args.get('date_to')
@@ -705,6 +732,8 @@ def show_domains_by_daterange():
                                 domains_tags=domains_tags, type=service_type, bootstrap_label=bootstrap_label)
 
 @hiddenServices.route("/crawlers/show_domain", methods=['GET'])
+@login_required
+@login_analyst
 def show_domain():
     domain = request.args.get('domain')
     epoch = request.args.get('epoch')
@@ -788,6 +817,8 @@ def show_domain():
                             domain_tags=domain_tags, screenshot=screenshot)
 
 @hiddenServices.route("/crawlers/download_domain", methods=['GET'])
+@login_required
+@login_analyst
 def download_domain():
     domain = request.args.get('domain')
     epoch = request.args.get('epoch')
@@ -839,6 +870,8 @@ def download_domain():
 
 
 @hiddenServices.route("/hiddenServices/onion_son", methods=['GET'])
+@login_required
+@login_analyst
 def onion_son():
     onion_domain = request.args.get('onion_domain')
 
@@ -849,6 +882,8 @@ def onion_son():
 
 # ============= JSON ==============
 @hiddenServices.route("/hiddenServices/domain_crawled_7days_json", methods=['GET'])
+@login_required
+@login_analyst
 def domain_crawled_7days_json():
     type = 'onion'
         ## TODO: # FIXME: 404 error
@@ -867,6 +902,8 @@ def domain_crawled_7days_json():
     return jsonify(json_domain_stats)
 
 @hiddenServices.route('/hiddenServices/domain_crawled_by_type_json')
+@login_required
+@login_analyst
 def domain_crawled_by_type_json():
     current_date = request.args.get('date')
     type = request.args.get('type')

var/www/modules/rawSkeleton/Flask_rawSkeleton.py

@@ -7,6 +7,9 @@
 import redis
 from flask import Flask, render_template, jsonify, request, Blueprint
 
+from Role_Manager import login_admin, login_analyst
+from flask_login import login_required
+
 # ============ VARIABLES ============
 import Flask_config
 
@@ -22,6 +25,8 @@ def one():
 # ============= ROUTES ==============
 
 @rawSkeleton.route("/rawSkeleton/", methods=['GET'])
+@login_required
+@login_analyst
 def skeleton_page():
     return render_template("rawSkeleton.html")
 

var/www/modules/restApi/Flask_restApi.py

@@ -0,0 +1,109 @@
+#!/usr/bin/env python3
+# -*-coding:UTF-8 -*
+
+'''
+    Flask functions and routes for the rest api
+'''
+
+import os
+import re
+import sys
+import json
+import redis
+import datetime
+
+from flask import Flask, render_template, jsonify, request, Blueprint, redirect, url_for, Response
+from flask_login import login_required
+
+from functools import wraps
+
+# ============ VARIABLES ============
+import Flask_config
+
+app = Flask_config.app
+cfg = Flask_config.cfg
+baseUrl = Flask_config.baseUrl
+r_cache = Flask_config.r_cache
+r_serv_db = Flask_config.r_serv_db
+r_serv_onion = Flask_config.r_serv_onion
+r_serv_metadata = Flask_config.r_serv_metadata
+
+restApi = Blueprint('restApi', __name__, template_folder='templates')
+
+# ============ AUTH FUNCTIONS ============
+
+def check_token_format(strg, search=re.compile(r'[^a-zA-Z0-9_-]').search):
+    return not bool(search(strg))
+
+def verify_token(token):
+    if len(token) != 55:
+        return False
+
+    if not check_token_format(token):
+        return False
+
+    if r_serv_db.hexists('user:tokens', token):
+        return True
+    else:
+        return False
+
+# ============ DECORATOR ============
+
+def token_required(funct):
+    @wraps(funct)
+    def api_token(*args, **kwargs):
+        data = authErrors()
+        if data:
+            return Response(json.dumps(data[0], indent=2, sort_keys=True), mimetype='application/json'), data[1]
+        else:
+            return funct(*args, **kwargs)
+    return api_token
+
+def get_auth_from_header():
+    token = request.headers.get('Authorization').replace(' ', '') # remove space
+    return token
+
+def authErrors():
+    # Check auth
+    if not request.headers.get('Authorization'):
+        return ({'status': 'error', 'reason': 'Authentication needed'}, 401)
+    token = get_auth_from_header()
+    data = None
+    # verify token format
+
+    try:
+        authenticated = False
+        if verify_token(token):
+            authenticated = True
+
+        if not authenticated:
+            data = ({'status': 'error', 'reason': 'Authentication failed'}, 401)
+    except Exception as e:
+        print(e)
+        data = ({'status': 'error', 'reason': 'Malformed Authentication String'}, 400)
+    if data:
+        return data
+    else:
+        return None
+
+# ============ FUNCTIONS ============
+
+def one():
+    return 1
+
+# ============= ROUTES ==============
+
+# @restApi.route("/api", methods=['GET'])
+# @login_required
+# def api():
+#     return 'api doc'
+
+@restApi.route("api/items", methods=['POST'])
+@token_required
+def items():
+    item = request.args.get('id')
+
+    return Response(json.dumps({'test': 2}), mimetype='application/json')
+
+# ========= REGISTRATION =========
+app.register_blueprint(restApi, url_prefix=baseUrl)

var/www/modules/restApi/templates/api_default.html

@@ -0,0 +1,35 @@
+<!DOCTYPE html>
+
+<html>
+<head>
+	<title>AIL-Framework</title>
+	<link rel="icon" href="{{ url_for('static', filename='image/ail-icon.png')}}">
+	<!-- Core CSS -->
+	<link href="{{ url_for('static', filename='css/bootstrap4.min.css') }}" rel="stylesheet">
+	<link href="{{ url_for('static', filename='css/font-awesome.min.css') }}" rel="stylesheet">
+
+	<!-- JS -->
+	<script src="{{ url_for('static', filename='js/jquery.js')}}"></script>
+	<script src="{{ url_for('static', filename='js/bootstrap4.min.js')}}"></script>
+
+
+</head>
+
+<body>
+
+	{% include 'nav_bar.html' %}
+
+	<div class="container-fluid">
+		<div class="row">
+
+			{% include 'crawler/menu_sidebar.html' %}
+
+			<div class="col-12 col-lg-10" id="core_content">
+
+
+			</div>
+		</div>
+	</div>
+
+
+</body>

var/www/modules/search/Flask_search.py

@@ -11,6 +11,9 @@ import datetime
 import flask
 from flask import Flask, render_template, jsonify, request, Blueprint
 
+from Role_Manager import login_admin, login_analyst
+from flask_login import login_required
+
 import Paste
 from whoosh import index
 from whoosh.fields import Schema, TEXT, ID
@@ -93,6 +96,8 @@ def to_iso_date(timestamp):
 # ============ ROUTES ============
 
 @searches.route("/search", methods=['POST'])
+@login_required
+@login_analyst
 def search():
     query = request.form['query']
     q = []
@@ -180,6 +185,8 @@ def search():
 
 
 @searches.route("/get_more_search_result", methods=['POST'])
+@login_required
+@login_analyst
 def get_more_search_result():
     query = request.form['query']
     q = []

var/www/modules/sentiment/Flask_sentiment.py

@@ -11,6 +11,9 @@ from Date import Date
 import flask
 from flask import Flask, render_template, jsonify, request, Blueprint
 
+from Role_Manager import login_admin, login_analyst
+from flask_login import login_required
+
 import Paste
 
 # ============ VARIABLES ============
@@ -39,11 +42,15 @@ def get_date_range(num_day):
 # ============ ROUTES ============
 
 @sentiments.route("/sentiment_analysis_trending/")
+@login_required
+@login_analyst
 def sentiment_analysis_trending():
     return render_template("sentiment_analysis_trending.html")
 
 
 @sentiments.route("/sentiment_analysis_getplotdata/", methods=['GET'])
+@login_required
+@login_analyst
 def sentiment_analysis_getplotdata():
     # Get the top providers based on number of pastes
     oneHour = 60*60
@@ -94,12 +101,16 @@ def sentiment_analysis_getplotdata():
 
 
 @sentiments.route("/sentiment_analysis_plot_tool/")
+@login_required
+@login_analyst
 def sentiment_analysis_plot_tool():
     return render_template("sentiment_analysis_plot_tool.html")
 
 
 
 @sentiments.route("/sentiment_analysis_plot_tool_getdata/", methods=['GET'])
+@login_required
+@login_analyst
 def sentiment_analysis_plot_tool_getdata():
     getProviders = request.args.get('getProviders')
 

var/www/modules/settings/Flask_settings.py

@@ -5,8 +5,13 @@
     Flask functions and routes for the settings modules page
 '''
 from flask import Flask, render_template, jsonify, request, Blueprint, redirect, url_for
+from flask_login import login_required, current_user
+
+from Role_Manager import login_admin, login_analyst
+from Role_Manager import create_user_db, edit_user_db, delete_user_db, check_password_strength, generate_new_token
 
 import json
+import secrets
 import datetime
 
 import git_status
@@ -22,6 +27,7 @@ max_preview_char = Flask_config.max_preview_char
 max_preview_modal = Flask_config.max_preview_modal
 REPO_ORIGIN = Flask_config.REPO_ORIGIN
 dict_update_description = Flask_config.dict_update_description
+email_regex = Flask_config.email_regex
 
 settings = Blueprint('settings', __name__, template_folder='templates')
 
@@ -31,8 +37,12 @@ settings = Blueprint('settings', __name__, template_folder='templates')
 def one():
     return 1
 
-#def get_v1.5_update_tags_backgroud_status():
+def check_email(email):
-#    return '38%'
+    result = email_regex.match(email)
+    if result:
+        return True
+    else:
+        return False
 
 def get_git_metadata():
     dict_git = {}
@@ -71,20 +81,161 @@ def get_update_metadata():
         dict_update['current_background_script_stat'] = r_serv_db.get('ail:current_background_script_stat')
 
     return dict_update
+
+def get_user_metadata(user_id):
+    user_metadata = {}
+    user_metadata['email'] = user_id
+    user_metadata['role'] = r_serv_db.hget('user_metadata:{}'.format(user_id), 'role')
+    user_metadata['api_key'] = r_serv_db.hget('user_metadata:{}'.format(user_id), 'token')
+    return user_metadata
+
+def get_users_metadata(list_users):
+    users = []
+    for user in list_users:
+        users.append(get_user_metadata(user))
+    return users
+
+def get_all_users():
+    return r_serv_db.hkeys('user:all')
+
+def get_all_roles():
+    return r_serv_db.zrange('ail:all_role', 0, -1)
+
 # ============= ROUTES ==============
 
 @settings.route("/settings/", methods=['GET'])
+@login_required
+@login_analyst
 def settings_page():
     git_metadata = get_git_metadata()
     current_version = r_serv_db.get('ail:version')
     update_metadata = get_update_metadata()
 
+    admin_level = current_user.is_in_role('admin')
 
     return render_template("settings_index.html", git_metadata=git_metadata,
+                            admin_level=admin_level,
                             current_version=current_version)
 
+@settings.route("/settings/edit_profile", methods=['GET'])
+@login_required
+@login_analyst
+def edit_profile():
+    user_metadata = get_user_metadata(current_user.get_id())
+    admin_level = current_user.is_in_role('admin')
+    return render_template("edit_profile.html", user_metadata=user_metadata,
+                            admin_level=admin_level)
+
+@settings.route("/settings/new_token", methods=['GET'])
+@login_required
+@login_analyst
+def new_token():
+    generate_new_token(current_user.get_id())
+    return redirect(url_for('settings.edit_profile'))
+
+@settings.route("/settings/new_token_user", methods=['GET'])
+@login_required
+@login_admin
+def new_token_user():
+    user_id = request.args.get('user_id')
+    if r_serv_db.exists('user_metadata:{}'.format(user_id)):
+        generate_new_token(user_id)
+    return redirect(url_for('settings.users_list'))
+
+@settings.route("/settings/create_user", methods=['GET'])
+@login_required
+@login_admin
+def create_user():
+    user_id = request.args.get('user_id')
+    error = request.args.get('error')
+    error_mail = request.args.get('error_mail')
+    role = None
+    if r_serv_db.exists('user_metadata:{}'.format(user_id)):
+        role = r_serv_db.hget('user_metadata:{}'.format(user_id), 'role')
+    else:
+        user_id = None
+    all_roles = get_all_roles()
+    return render_template("create_user.html", all_roles=all_roles, user_id=user_id, user_role=role,
+                                        error=error, error_mail=error_mail,
+                                        admin_level=True)
+
+@settings.route("/settings/create_user_post", methods=['POST'])
+@login_required
+@login_admin
+def create_user_post():
+    email = request.form.get('username')
+    role = request.form.get('user_role')
+    password1 = request.form.get('password1')
+    password2 = request.form.get('password2')
+
+    all_roles = get_all_roles()
+
+    if email and len(email)< 300 and check_email(email) and role:
+        if role in all_roles:
+            # password set
+            if password1 and password2:
+                if password1==password2:
+                    if check_password_strength(password1):
+                        password = password1
+                    else:
+                        return render_template("create_user.html", all_roles=all_roles, error="Incorrect Password", admin_level=True)
+                else:
+                    return render_template("create_user.html", all_roles=all_roles, error="Passwords don't match", admin_level=True)
+            # generate password
+            else:
+                password = secrets.token_urlsafe()
+
+            if current_user.is_in_role('admin'):
+                # edit user
+                if r_serv_db.exists('user_metadata:{}'.format(email)):
+                    if password1 and password2:
+                        edit_user_db(email, password=password, role=role)
+                        return redirect(url_for('settings.users_list', new_user=email, new_user_password=password, new_user_edited=True))
+                    else:
+                        edit_user_db(email, role=role)
+                        return redirect(url_for('settings.users_list', new_user=email, new_user_password='Password not changed', new_user_edited=True))
+                # create user
+                else:
+                    create_user_db(email, password, default=True, role=role)
+                    return redirect(url_for('settings.users_list', new_user=email, new_user_password=password, new_user_edited=False))
+
+        else:
+            return render_template("create_user.html", all_roles=all_roles, admin_level=True)
+    else:
+        return render_template("create_user.html", all_roles=all_roles, error_mail=True, admin_level=True)
+
+@settings.route("/settings/users_list", methods=['GET'])
+@login_required
+@login_admin
+def users_list():
+    all_users = get_users_metadata(get_all_users())
+    new_user = request.args.get('new_user')
+    new_user_dict = {}
+    if new_user:
+        new_user_dict['email'] = new_user
+        new_user_dict['edited'] = request.args.get('new_user_edited')
+        new_user_dict['password'] = request.args.get('new_user_password')
+    return render_template("users_list.html", all_users=all_users, new_user=new_user_dict, admin_level=True)
+
+@settings.route("/settings/edit_user", methods=['GET'])
+@login_required
+@login_admin
+def edit_user():
+    user_id = request.args.get('user_id')
+    return redirect(url_for('settings.create_user', user_id=user_id))
+
+@settings.route("/settings/delete_user", methods=['GET'])
+@login_required
+@login_admin
+def delete_user():
+    user_id = request.args.get('user_id')
+    delete_user_db(user_id)
+    return redirect(url_for('settings.users_list'))
+
 
 @settings.route("/settings/get_background_update_stats_json", methods=['GET'])
+@login_required
+@login_analyst
 def get_background_update_stats_json():
     # handle :end, error
     update_stats = {}

var/www/modules/settings/templates/create_user.html

@@ -0,0 +1,153 @@
+<!DOCTYPE html>
+<html>
+
+<head>
+  <title>Server Management - AIL</title>
+	<link rel="icon" href="{{ url_for('static', filename='image/ail-icon.png') }}">
+
+  <!-- Core CSS -->
+	<link href="{{ url_for('static', filename='css/bootstrap4.min.css') }}" rel="stylesheet">
+	<link href="{{ url_for('static', filename='css/font-awesome.min.css') }}" rel="stylesheet">
+	<link href="{{ url_for('static', filename='css/dataTables.bootstrap4.min.css') }}" rel="stylesheet">
+
+  <!-- JS -->
+	<script src="{{ url_for('static', filename='js/jquery.js')}}"></script>
+	<script src="{{ url_for('static', filename='js/popper.min.js')}}"></script>
+	<script src="{{ url_for('static', filename='js/bootstrap4.min.js')}}"></script>
+	<script src="{{ url_for('static', filename='js/jquery.dataTables.min.js')}}"></script>
+	<script src="{{ url_for('static', filename='js/dataTables.bootstrap.min.js')}}"></script>
+
+</head>
+<body>
+
+  {% include 'nav_bar.html' %}
+
+	<div class="container-fluid">
+		<div class="row">
+
+			{% include 'settings/menu_sidebar.html' %}
+
+      <div class="col-12 col-lg-10" id="core_content">
+
+        <form class="form-signin" action="{{ url_for('settings.create_user_post')}}" autocomplete="off" method="post">
+
+          <h1 class="h3 mt-1 mb-3 text-center text-secondary">Create User</h1>
+          <label for="inputEmail" class="sr-only">Email address</label>
+          <input type="email" id="inputEmail" name="username" class="form-control {% if error_mail %}is-invalid{% endif %}" placeholder="Email address" autocomplete="off" required {% if user_id %}value="{{user_id}}"{% else %}{% endif %}>
+          {% if error_mail %}
+            <div class="invalid-feedback">
+              Please provide a valid email address
+            </div>
+          {% endif %}
+
+          <label  class="mt-3" for="role_selector">User Role</label>
+          <select class="custom-select" id="role_selector" name="user_role">
+            {% for role in all_roles %}
+              {% if role ==  user_role %}
+                <option value="{{role}}" selected>{{role}}</option>
+              {% else %}
+                <option value="{{role}}">{{role}}</option>
+              {% endif %}
+            {% endfor %}
+          </select>
+
+          <div class="custom-control custom-switch mt-4 mb-3">
+  				  <input type="checkbox" class="custom-control-input" id="set_manual_password" value="" onclick="toggle_password_fields();">
+  				  <label class="custom-control-label" for="set_manual_password">Set Password</label>
+				  </div>
+
+          <div id="password-section">
+    	      <h1 class="h3 mb-3 text-center text-secondary">Create Password</h1>
+    	      <label for="inputPassword1" class="sr-only">Password</label>
+    	      <input type="password" id="inputPassword1" name="password1" class="form-control {% if error %}is-invalid{% endif %}" placeholder="Password" autocomplete="new-password">
+    	      <label for="inputPassword2" class="sr-only">Confirm Password</label>
+    	      <input type="password" id="inputPassword2" name="password2" class="form-control {% if error %}is-invalid{% endif %}" placeholder="Confirm Password"  value="" autocomplete="new-password">
+            {% if error %}
+  						<div class="invalid-feedback">
+  							{{error}}
+  						</div>
+  					{% endif %}
+
+          </div>
+          <button class="btn btn-lg btn-primary btn-block mt-3" type="submit">Submit</button>
+
+          <div id="password-section-info">
+    				<br>
+    				<br>
+    				<br>
+    				<h5 class="h3 mb-3 text-center text-secondary">Password Requirements</h5>
+    				<ul class="list-group">
+    				 <li class="list-group-item d-flex justify-content-between align-items-center">
+    					 Minimal length
+    					 <span class="badge badge-primary badge-pill">10</span>
+    				 </li>
+    				 <li class="list-group-item d-flex justify-content-between align-items-center">
+    					 Upper characters: A-Z
+    					 <span class="badge badge-primary badge-pill">1</span>
+    				 </li>
+    				 <li class="list-group-item d-flex justify-content-between align-items-center">
+    					 Lower characters: a-z
+    					 <span class="badge badge-primary badge-pill">1</span>
+    				 </li>
+    				 <li class="list-group-item d-flex justify-content-between align-items-center">
+    					 Digits: 0-9
+    					 <span class="badge badge-primary badge-pill">2</span>
+    				 </li>
+             <li class="list-group-item d-flex justify-content-between align-items-center">
+  						 Maximum length
+  						 <span class="badge badge-primary badge-pill">100</span>
+  					 </li>
+    				</ul>
+          </div>
+
+  		  </form>
+
+    	</div>
+		</div>
+	</div>
+
+</body>
+
+<script>
+$(document).ready(function(){
+  $("#password-section").hide();
+  $("#password-section-info").hide();
+  $("#nav_create_user").addClass("active");
+	$("#nav_user_management").removeClass("text-muted");
+
+  {% if error %}
+    toggle_password_fields();
+  {% endif %}
+} );
+
+function toggle_sidebar(){
+	if($('#nav_menu').is(':visible')){
+		$('#nav_menu').hide();
+		$('#side_menu').removeClass('border-right')
+		$('#side_menu').removeClass('col-lg-2')
+		$('#core_content').removeClass('col-lg-10')
+	}else{
+		$('#nav_menu').show();
+		$('#side_menu').addClass('border-right')
+		$('#side_menu').addClass('col-lg-2')
+		$('#core_content').addClass('col-lg-10')
+	}
+}
+
+function toggle_password_fields() {
+  var password_div = $("#password-section");
+  if(password_div.is(":visible")){
+    $("#password-section").hide();
+    $("#password-section-info").hide();
+    $("#inputPassword1").prop('required',false);
+    $("#inputPassword2").prop('required',false);
+  } else {
+    $("#password-section").show();
+    $("#password-section-info").show();
+    $("#inputPassword1").prop('required',true);
+    $("#inputPassword2").prop('required',true);
+  }
+}
+</script>
+
+</html>

var/www/modules/settings/templates/edit_profile.html

@@ -0,0 +1,96 @@
+<!DOCTYPE html>
+<html>
+
+<head>
+  <title>Server Management - AIL</title>
+	<link rel="icon" href="{{ url_for('static', filename='image/ail-icon.png') }}">
+
+  <!-- Core CSS -->
+	<link href="{{ url_for('static', filename='css/bootstrap4.min.css') }}" rel="stylesheet">
+	<link href="{{ url_for('static', filename='css/font-awesome.min.css') }}" rel="stylesheet">
+	<link href="{{ url_for('static', filename='css/dataTables.bootstrap4.min.css') }}" rel="stylesheet">
+
+  <!-- JS -->
+	<script src="{{ url_for('static', filename='js/jquery.js')}}"></script>
+	<script src="{{ url_for('static', filename='js/popper.min.js')}}"></script>
+	<script src="{{ url_for('static', filename='js/bootstrap4.min.js')}}"></script>
+	<script src="{{ url_for('static', filename='js/jquery.dataTables.min.js')}}"></script>
+	<script src="{{ url_for('static', filename='js/dataTables.bootstrap.min.js')}}"></script>
+
+</head>
+<body>
+
+  {% include 'nav_bar.html' %}
+
+	<div class="container-fluid">
+		<div class="row">
+
+			{% include 'settings/menu_sidebar.html' %}
+
+			<div class="col-12 col-lg-10" id="core_content">
+
+				<div class="card mb-3 mt-1">
+					<div class="card-header text-white bg-dark pb-1">
+						<h5 class="card-title">My Profile :</h5>
+					</div>
+					<div class="card-body">
+
+            <div class="row">
+              <div class="col-xl-6">
+                <div class="card text-center border-secondary">
+                  <div class="card-body px-1 py-0">
+                    <table class="table table-sm">
+                      <tbody>
+                        <tr>
+                          <td>Email</td>
+                          <td>{{user_metadata['email']}}</td>
+                        </tr>
+                        <tr>
+                          <td>Role</td>
+                          <td>{{user_metadata['role']}}</td>
+                        </tr>
+                        <tr>
+                          <td>API Key</td>
+                          <td>
+                            {{user_metadata['api_key']}}
+                            <a class="ml-3" href="{{url_for('settings.new_token')}}"><i class="fa fa-random"></i></a>
+                          </td>
+                        </tr>
+                      </tbody>
+                    </table>
+                  </div>
+                </div>
+              </div>
+            </div>
+
+					</div>
+				</div>
+
+    	</div>
+		</div>
+	</div>
+
+</body>
+
+<script>
+$(document).ready(function(){
+  $("#nav_edit_profile").addClass("active");
+	$("#nav_my_profile").removeClass("text-muted");
+} );
+
+function toggle_sidebar(){
+	if($('#nav_menu').is(':visible')){
+		$('#nav_menu').hide();
+		$('#side_menu').removeClass('border-right')
+		$('#side_menu').removeClass('col-lg-2')
+		$('#core_content').removeClass('col-lg-10')
+	}else{
+		$('#nav_menu').show();
+		$('#side_menu').addClass('border-right')
+		$('#side_menu').addClass('col-lg-2')
+		$('#core_content').addClass('col-lg-10')
+	}
+}
+</script>
+
+</html>

var/www/modules/settings/templates/settings_index.html

@@ -142,7 +142,8 @@
 
 <script>
 $(document).ready(function(){
-	$("#page-options").addClass("active");
+  $("#nav_server_status").addClass("active");
+	$("#nav_server").removeClass("text-muted");
 } );
 
 function toggle_sidebar(){
@@ -191,6 +192,7 @@ update_progress();
 var progress_interval = setInterval(function(){
 	update_progress()
 }, 4000);
+
 </script>
 
 </html>

var/www/modules/settings/templates/users_list.html

@@ -0,0 +1,122 @@
+<!DOCTYPE html>
+<html>
+
+<head>
+  <title>Server Management - AIL</title>
+	<link rel="icon" href="{{ url_for('static', filename='image/ail-icon.png') }}">
+
+  <!-- Core CSS -->
+	<link href="{{ url_for('static', filename='css/bootstrap4.min.css') }}" rel="stylesheet">
+	<link href="{{ url_for('static', filename='css/font-awesome.min.css') }}" rel="stylesheet">
+	<link href="{{ url_for('static', filename='css/dataTables.bootstrap4.min.css') }}" rel="stylesheet">
+
+  <!-- JS -->
+	<script src="{{ url_for('static', filename='js/jquery.js')}}"></script>
+	<script src="{{ url_for('static', filename='js/popper.min.js')}}"></script>
+	<script src="{{ url_for('static', filename='js/bootstrap4.min.js')}}"></script>
+	<script src="{{ url_for('static', filename='js/jquery.dataTables.min.js')}}"></script>
+	<script src="{{ url_for('static', filename='js/dataTables.bootstrap.min.js')}}"></script>
+
+  <style>
+	.edit_icon:hover{
+		cursor: pointer;
+		color: #17a2b8;
+	}
+  .trash_icon:hover{
+		cursor: pointer;
+		color: #c82333;
+	}
+	</style>
+
+</head>
+<body>
+
+  {% include 'nav_bar.html' %}
+
+	<div class="container-fluid">
+		<div class="row">
+
+			{% include 'settings/menu_sidebar.html' %}
+
+			<div class="col-12 col-lg-10" id="core_content">
+
+        {% if new_user %}
+        <div class="text-center my-3 ">
+          <div class="card">
+            <div class="card-header">
+              {% if new_user['edited']=='True' %}
+                <h5 class="card-title">User Edited</h5>
+              {% else %}
+                <h5 class="card-title">User Created</h5>
+              {% endif %}
+            </div>
+            <div class="card-body">
+              <p>User: {{new_user['email']}}</p>
+              <p>Password: {{new_user['password']}}</p>
+              <a href="{{url_for('settings.users_list')}}" class="btn btn-primary"><i class="fas fa-eye-slash"></i> Hide</a>
+            </div>
+          </div>
+        </div>
+        {% endif %}
+
+        <div class="table-responsive mt-1 table-hover table-borderless table-striped">
+          <table class="table">
+            <thead class="thead-dark">
+                <tr>
+                  <th>Email</th>
+                  <th>Role</th>
+                  <th>Api Key</th>
+                  <th>Actions</th>
+                </tr>
+            </thead>
+            <tbody id="tbody_last_crawled">
+                {% for user in all_users %}
+                  <tr>
+                    <td>{{user['email']}}</td>
+                    <td>{{user['role']}}</td>
+                    <td>
+                      {{user['api_key']}}
+                      <a class="ml-3" href="{{url_for('settings.new_token_user')}}?user_id={{user['email']}}"><i class="fa fa-random"></i></a>
+                    </td>
+                    <td>
+                      <a href="{{ url_for('settings.edit_user')}}?user_id={{user['email']}}">
+                          <i class="fas fa-pencil-alt edit_icon"></i>
+											</a>
+                      <a href="{{ url_for('settings.delete_user')}}?user_id={{user['email']}}" class="ml-4">
+                          <i class="fas fa-trash-alt trash_icon"></i>
+											</a>
+                    </td>
+                  </tr>
+                {% endfor %}
+            </tbody>
+          </table>
+        </div>
+
+    	</div>
+		</div>
+	</div>
+
+</body>
+
+<script>
+$(document).ready(function(){
+  $("#nav_users_list").addClass("active");
+	$("#nav_user_management").removeClass("text-muted");
+} );
+
+function toggle_sidebar(){
+	if($('#nav_menu').is(':visible')){
+		$('#nav_menu').hide();
+		$('#side_menu').removeClass('border-right')
+		$('#side_menu').removeClass('col-lg-2')
+		$('#core_content').removeClass('col-lg-10')
+	}else{
+		$('#nav_menu').show();
+		$('#side_menu').addClass('border-right')
+		$('#side_menu').addClass('col-lg-2')
+		$('#core_content').addClass('col-lg-10')
+	}
+}
+</script>
+
+</html>

var/www/modules/showpaste/Flask_showpaste.py

@@ -9,6 +9,10 @@ import json
 import os
 import flask
 from flask import Flask, render_template, jsonify, request, Blueprint, make_response, Response, send_from_directory, redirect, url_for
+
+from Role_Manager import login_admin, login_analyst
+from flask_login import login_required
+
 import difflib
 import ssdeep
 
@@ -380,16 +384,22 @@ def show_item_min(requested_path , content_range=0):
 # ============ ROUTES ============
 
 @showsavedpastes.route("/showsavedpaste/") #completely shows the paste in a new tab
+@login_required
+@login_analyst
 def showsavedpaste():
     requested_path = request.args.get('paste', '')
     return showpaste(0, requested_path)
 
 @showsavedpastes.route("/showsaveditem_min/") #completely shows the paste in a new tab
+@login_required
+@login_analyst
 def showsaveditem_min():
     requested_path = request.args.get('paste', '')
     return show_item_min(requested_path)
 
 @showsavedpastes.route("/showsavedrawpaste/") #shows raw
+@login_required
+@login_analyst
 def showsavedrawpaste():
     requested_path = request.args.get('paste', '')
     paste = Paste.Paste(requested_path)
@@ -397,6 +407,8 @@ def showsavedrawpaste():
     return Response(content, mimetype='text/plain')
 
 @showsavedpastes.route("/showpreviewpaste/")
+@login_required
+@login_analyst
 def showpreviewpaste():
     num = request.args.get('num', '')
     requested_path = request.args.get('paste', '')
@@ -404,6 +416,8 @@ def showpreviewpaste():
 
 
 @showsavedpastes.route("/getmoredata/")
+@login_required
+@login_analyst
 def getmoredata():
     requested_path = request.args.get('paste', '')
     paste = Paste.Paste(requested_path)
@@ -412,6 +426,8 @@ def getmoredata():
     return to_return
 
 @showsavedpastes.route("/showDiff/")
+@login_required
+@login_analyst
 def showDiff():
     s1 = request.args.get('s1', '')
     s2 = request.args.get('s2', '')
@@ -428,10 +444,14 @@ def showDiff():
     return the_html
 
 @showsavedpastes.route('/screenshot/<path:filename>')
+@login_required
+@login_analyst
 def screenshot(filename):
     return send_from_directory(SCREENSHOT_FOLDER, filename+'.png', as_attachment=True)
 
 @showsavedpastes.route('/send_file_to_vt/', methods=['POST'])
+@login_required
+@login_analyst
 def send_file_to_vt():
     b64_path = request.form['b64_path']
     paste = request.form['paste']

var/www/modules/terms/Flask_terms.py

@@ -11,6 +11,10 @@ import datetime
 import calendar
 import flask
 from flask import Flask, render_template, jsonify, request, Blueprint, url_for, redirect
+
+from Role_Manager import login_admin, login_analyst
+from flask_login import login_required
+
 import re
 import Paste
 from pprint import pprint
@@ -143,6 +147,8 @@ def save_tag_to_auto_push(list_tag):
 # ============ ROUTES ============
 
 @terms.route("/terms_management/")
+@login_required
+@login_analyst
 def terms_management():
     per_paste = request.args.get('per_paste')
     if per_paste == "1" or per_paste is None:
@@ -261,6 +267,8 @@ def terms_management():
 
 
 @terms.route("/terms_management_query_paste/")
+@login_required
+@login_analyst
 def terms_management_query_paste():
     term =  request.args.get('term')
     paste_info = []
@@ -293,6 +301,8 @@ def terms_management_query_paste():
 
 
 @terms.route("/terms_management_query/")
+@login_required
+@login_analyst
 def terms_management_query():
     TrackedTermsDate_Name = "TrackedTermDate"
     BlackListTermsDate_Name = "BlackListTermDate"
@@ -315,6 +325,8 @@ def terms_management_query():
 
 
 @terms.route("/terms_management_action/", methods=['GET'])
+@login_required
+@login_analyst
 def terms_management_action():
     today = datetime.datetime.now()
     today = today.replace(microsecond=0)
@@ -440,6 +452,8 @@ def terms_management_action():
         return jsonify(to_return)
 
 @terms.route("/terms_management/delete_terms_tags", methods=['POST'])
+@login_required
+@login_analyst
 def delete_terms_tags():
     term = request.form.get('term')
     tags_to_delete = request.form.getlist('tags_to_delete')
@@ -452,6 +466,8 @@ def delete_terms_tags():
         return 'None args', 400
 
 @terms.route("/terms_management/delete_terms_email", methods=['GET'])
+@login_required
+@login_analyst
 def delete_terms_email():
     term =  request.args.get('term')
     email =  request.args.get('email')
@@ -464,6 +480,8 @@ def delete_terms_email():
 
 
 @terms.route("/terms_plot_tool/")
+@login_required
+@login_analyst
 def terms_plot_tool():
     term =  request.args.get('term')
     if term is not None:
@@ -473,6 +491,8 @@ def terms_plot_tool():
 
 
 @terms.route("/terms_plot_tool_data/")
+@login_required
+@login_analyst
 def terms_plot_tool_data():
     oneDay = 60*60*24
     range_start =  datetime.datetime.utcfromtimestamp(int(float(request.args.get('range_start')))) if request.args.get('range_start') is not None else 0;
@@ -503,6 +523,8 @@ def terms_plot_tool_data():
 
 
 @terms.route("/terms_plot_top/")
+@login_required
+@login_analyst
 def terms_plot_top():
     per_paste = request.args.get('per_paste')
     per_paste = per_paste if per_paste is not None else 1
@@ -510,6 +532,8 @@ def terms_plot_top():
 
 
 @terms.route("/terms_plot_top_data/")
+@login_required
+@login_analyst
 def terms_plot_top_data():
     oneDay = 60*60*24
     today = datetime.datetime.now()
@@ -556,10 +580,14 @@ def terms_plot_top_data():
 
 
 @terms.route("/credentials_tracker/")
+@login_required
+@login_analyst
 def credentials_tracker():
     return render_template("credentials_tracker.html")
 
 @terms.route("/credentials_management_query_paste/", methods=['GET', 'POST'])
+@login_required
+@login_analyst
 def credentials_management_query_paste():
     cred =  request.args.get('cred')
     allPath = request.json['allPath']
@@ -583,6 +611,8 @@ def credentials_management_query_paste():
     return jsonify(paste_info)
 
 @terms.route("/credentials_management_action/", methods=['GET'])
+@login_required
+@login_analyst
 def cred_management_action():
 
     supplied =  request.args.get('term')

var/www/modules/trendingcharts/Flask_trendingcharts.py

@@ -10,6 +10,9 @@ from Date import Date
 import flask
 from flask import Flask, render_template, jsonify, request, Blueprint
 
+from Role_Manager import login_admin, login_analyst
+from flask_login import login_required
+
 # ============ VARIABLES ============
 import Flask_config
 
@@ -36,6 +39,8 @@ def get_date_range(num_day):
 # ============ ROUTES ============
 
 @trendings.route("/_progressionCharts", methods=['GET'])
+@login_required
+@login_analyst
 def progressionCharts():
     attribute_name = request.args.get('attributeName')
     trending_name = request.args.get('trendingName')
@@ -61,18 +66,24 @@ def progressionCharts():
         return jsonify(keyw_value)
 
 @trendings.route("/wordstrending/")
+@login_required
+@login_analyst
 def wordstrending():
     default_display = cfg.get("Flask", "default_display")
     return render_template("Wordstrending.html", default_display = default_display)
 
 
 @trendings.route("/protocolstrending/")
+@login_required
+@login_analyst
 def protocolstrending():
     default_display = cfg.get("Flask", "default_display")
     return render_template("Protocolstrending.html", default_display = default_display)
 
 
 @trendings.route("/trending/")
+@login_required
+@login_analyst
 def trending():
     default_display = cfg.get("Flask", "default_display")
     return render_template("Trending.html", default_display = default_display)

var/www/modules/trendingmodules/Flask_trendingmodules.py

@@ -10,6 +10,9 @@ from Date import Date
 import flask
 from flask import Flask, render_template, jsonify, request, Blueprint
 
+from Role_Manager import login_admin, login_analyst
+from flask_login import login_required
+
 # ============ VARIABLES ============
 import Flask_config
 
@@ -49,6 +52,8 @@ def get_date_range(num_day):
 # ============ ROUTES ============
 
 @trendingmodules.route("/_moduleCharts", methods=['GET'])
+@login_required
+@login_analyst
 def modulesCharts():
     keyword_name = request.args.get('keywordName')
     module_name = request.args.get('moduleName')
@@ -75,6 +80,8 @@ def modulesCharts():
 
 
 @trendingmodules.route("/_providersChart", methods=['GET'])
+@login_required
+@login_analyst
 def providersChart():
     keyword_name = request.args.get('keywordName')
     module_name = request.args.get('moduleName')
@@ -121,6 +128,8 @@ def providersChart():
 
 
 @trendingmodules.route("/moduletrending/")
+@login_required
+@login_analyst
 def moduletrending():
     return render_template("Moduletrending.html")
 

var/www/templates/change_password.html

@@ -0,0 +1,108 @@
+<!DOCTYPE html>
+
+<html>
+<head>
+	 <meta name="viewport" content="width=device-width, initial-scale=1, shrink-to-fit=no">
+	<title>AIL-Framework</title>
+	<link rel="icon" href="{{ url_for('static', filename='image/ail-icon.png')}}">
+	<!-- Core CSS -->
+	<link href="{{ url_for('static', filename='css/bootstrap4.min.css') }}" rel="stylesheet">
+	<link href="{{ url_for('static', filename='css/font-awesome.min.css') }}" rel="stylesheet">
+
+	<!-- JS -->
+	<script src="{{ url_for('static', filename='js/jquery.js')}}"></script>
+	<script src="{{ url_for('static', filename='js/bootstrap4.min.js')}}"></script>
+
+
+	<style>
+	html,
+	body {
+	height: 100%;
+	}
+
+	body {
+	display: -ms-flexbox;
+	display: flex;
+	-ms-flex-align: center;
+	align-items: center;
+	padding-top: 40px;
+	padding-bottom: 40px;
+	background-color: #f5f5f5;
+	}
+
+	.form-signin {
+	width: 100%;
+	max-width: 330px;
+	padding: 15px;
+	margin: auto;
+	}
+	.form-signin .checkbox {
+	font-weight: 400;
+	}
+	.form-signin .form-control {
+	position: relative;
+	box-sizing: border-box;
+	height: auto;
+	padding: 10px;
+	font-size: 16px;
+	}
+	.form-signin .form-control:focus {
+	z-index: 2;
+	}
+	.form-signin input[type="password"] {
+	margin-bottom: 10px;
+	border-top-left-radius: 0;
+	border-top-right-radius: 0;
+	}
+	</style>
+
+</head>
+
+<body class="text-center">
+
+
+				<form class="form-signin" action="{{ url_for('change_password')}}" autocomplete="off" method="post">
+		      <img class="mb-4" src="{{ url_for('static', filename='image/AIL-logo.png')}}" width="300">
+		      <h1 class="h3 mb-3 text-secondary">Change Password</h1>
+		      <label for="inputPassword1" class="sr-only">Password</label>
+		      <input type="password" id="inputPassword1" name="password1" class="form-control {% if error %}is-invalid{% endif %}" placeholder="Password" autocomplete="new-password" required autofocus>
+		      <label for="inputPassword2" class="sr-only">Confirm Password</label>
+		      <input type="password" id="inputPassword2" name="password2" class="form-control {% if error %}is-invalid{% endif %}" placeholder="Confirm Password"  value="" autocomplete="new-password" required>
+					{% if error %}
+						<div class="invalid-feedback">
+							{{error}}
+						</div>
+					{% endif %}
+					<button class="btn btn-lg btn-primary btn-block" type="submit">Submit</button>
+
+					<br>
+					<br>
+					<br>
+					<h5 class="h3 mb-3 text-secondary">Password Requirements</h5>
+					<ul class="list-group">
+					 <li class="list-group-item d-flex justify-content-between align-items-center">
+						 Minimal length
+						 <span class="badge badge-primary badge-pill">10</span>
+					 </li>
+					 <li class="list-group-item d-flex justify-content-between align-items-center">
+						 Upper characters: A-Z
+						 <span class="badge badge-primary badge-pill">1</span>
+					 </li>
+					 <li class="list-group-item d-flex justify-content-between align-items-center">
+						 Lower characters: a-z
+						 <span class="badge badge-primary badge-pill">1</span>
+					 </li>
+					 <li class="list-group-item d-flex justify-content-between align-items-center">
+						 Digits: 0-9
+						 <span class="badge badge-primary badge-pill">2</span>
+					 </li>
+					 <li class="list-group-item d-flex justify-content-between align-items-center">
+						 Maximum length
+						 <span class="badge badge-primary badge-pill">100</span>
+					 </li>
+					</ul>
+
+		    </form>
+
+
+</body>

var/www/templates/error/403.html

@@ -0,0 +1,50 @@
+<!DOCTYPE html>
+<html>
+
+<head>
+  <title>403 - AIL</title>
+  <link rel="icon" href="{{ url_for('static', filename='image/ail-icon.png') }}">
+
+  <!-- Core CSS -->
+	<link href="{{ url_for('static', filename='css/bootstrap4.min.css') }}" rel="stylesheet">
+  <link href="{{ url_for('static', filename='css/font-awesome.min.css') }}" rel="stylesheet">
+</head>
+
+<body>
+
+{% include 'nav_bar.html' %}
+
+<div>
+  <br>
+  <br>
+  <h1 class="text-center">403 Forbidden</h1>
+</div>
+<br>
+<br>
+<br>
+<br>
+<div class="d-flex justify-content-center">
+<pre>
+                                  ,d8       ,a8888a,     ad888888b,
+                                ,d888     ,8P"'  `"Y8,  d8"     "88
+                              ,d8" 88    ,8P        Y8,         a8P
+                            ,d8"   88    88          88      aad8"
+                          ,d8"     88    88          88      ""Y8,
+                          8888888888888  `8b        d8'         "8b
+                                   88     `8ba,  ,ad8'  Y8,     a88
+                                   88       "Y8888P"     "Y888888P'
+
+88888888888                       88           88           88           88
+88                                88           ""           88           88
+88                                88                        88           88
+88aaaaa   ,adPPYba,   8b,dPPYba,  88,dPPYba,   88   ,adPPYb,88   ,adPPYb,88   ,adPPYba,  8b,dPPYba,
+88"""""  a8"     "8a  88P'   "Y8  88P'    "8a  88  a8"    `Y88  a8"    `Y88  a8P_____88  88P'   `"8a
+88       8b       d8  88          88       d8  88  8b       88  8b       88  8PP"""""""  88       88
+88       "8a,   ,a8"  88          88b,   ,a8"  88  "8a,   ,d88  "8a,   ,d88  "8b,   ,aa  88       88
+88        `"YbbdP"'   88          8Y"Ybbd8"'   88   `"8bbdP"Y8   `"8bbdP"Y8   `"Ybbd8"'  88       88
+</pre>
+</div>
+
+<body>
+
+</html>

var/www/templates/error/404.html

@@ -0,0 +1,41 @@
+<!DOCTYPE html>
+<html>
+
+<head>
+  <title>404 - AIL</title>
+  <link rel="icon" href="{{ url_for('static', filename='image/ail-icon.png') }}">
+
+  <!-- Core CSS -->
+	<link href="{{ url_for('static', filename='css/bootstrap4.min.css') }}" rel="stylesheet">
+  <link href="{{ url_for('static', filename='css/font-awesome.min.css') }}" rel="stylesheet">
+</head>
+
+<body>
+
+{% include 'nav_bar.html' %}
+
+<div>
+  <br>
+  <br>
+  <h1 class="text-center">404 Not Found</h1>
+</div>
+<br>
+<br>
+<br>
+<br>
+<div class="d-flex justify-content-center">
+<pre>
+        ,d8       ,a8888a,              ,d8
+      ,d888     ,8P"'  `"Y8,          ,d888
+    ,d8" 88    ,8P        Y8,       ,d8" 88
+  ,d8"   88    88          88     ,d8"   88
+,d8"     88    88          88   ,d8"     88
+8888888888888  `8b        d8'   8888888888888
+         88     `8ba,  ,ad8'             88
+         88       "Y8888P"               88
+</pre>
+</div>
+
+<body>
+
+</html>

var/www/templates/login.html

@@ -0,0 +1,85 @@
+<!DOCTYPE html>
+
+<html>
+<head>
+	 <meta name="viewport" content="width=device-width, initial-scale=1, shrink-to-fit=no">
+	<title>AIL-Framework</title>
+	<link rel="icon" href="{{ url_for('static', filename='image/ail-icon.png')}}">
+	<!-- Core CSS -->
+	<link href="{{ url_for('static', filename='css/bootstrap4.min.css') }}" rel="stylesheet">
+	<link href="{{ url_for('static', filename='css/font-awesome.min.css') }}" rel="stylesheet">
+
+	<!-- JS -->
+	<script src="{{ url_for('static', filename='js/jquery.js')}}"></script>
+	<script src="{{ url_for('static', filename='js/bootstrap4.min.js')}}"></script>
+
+
+	<style>
+	html,
+	body {
+	height: 100%;
+	}
+
+	body {
+	display: -ms-flexbox;
+	display: flex;
+	-ms-flex-align: center;
+	align-items: center;
+	padding-top: 40px;
+	padding-bottom: 40px;
+	background-color: #f5f5f5;
+	}
+
+	.form-signin {
+	width: 100%;
+	max-width: 330px;
+	padding: 15px;
+	margin: auto;
+	}
+	.form-signin .checkbox {
+	font-weight: 400;
+	}
+	.form-signin .form-control {
+	position: relative;
+	box-sizing: border-box;
+	height: auto;
+	padding: 10px;
+	font-size: 16px;
+	}
+	.form-signin .form-control:focus {
+	z-index: 2;
+	}
+	.form-signin input[type="email"] {
+	margin-bottom: -1px;
+	border-bottom-right-radius: 0;
+	border-bottom-left-radius: 0;
+	}
+	.form-signin input[type="password"] {
+	margin-bottom: 10px;
+	border-top-left-radius: 0;
+	border-top-right-radius: 0;
+	}
+	</style>
+
+</head>
+
+<body class="text-center">
+
+
+				<form class="form-signin" action="{{ url_for('login')}}"  method="post">
+		      <img class="mb-4" src="{{ url_for('static', filename='image/AIL-logo.png')}}" width="300">
+		      <h1 class="h3 mb-3 text-secondary">Please sign in</h1>
+		      <label for="inputEmail" class="sr-only">Email address</label>
+		      <input type="email" id="inputEmail" name="username" class="form-control" placeholder="Email address" required autofocus>
+		      <label for="inputPassword" class="sr-only">Password</label>
+		      <input type="password" id="inputPassword" name="password" class="form-control {% if error %}is-invalid{% endif %}" placeholder="Password" required>
+					{% if error %}
+						<div class="invalid-feedback">
+							{{error}}
+						</div>
+					{% endif %}
+					<button class="btn btn-lg btn-primary btn-block" type="submit">Sign in</button>
+		    </form>
+
+
+</body>

var/www/templates/nav_bar.html

@@ -33,6 +33,9 @@
       <li class="nav-item mr-3">
         <a class="nav-link" id="page-options" href="{{ url_for('settings.settings_page') }}" aria-disabled="true"><i class="fas fa-cog"></i> Server Management</a>
       </li>
+      <li class="nav-item mr-3">
+        <a class="nav-link" id="page-options" href="{{ url_for('logout') }}" aria-disabled="true"><i class="fas fa-sign-out-alt"></i> Log Out</a>
+      </li>
     </ul>
 
     <form class="form-inline my-2 my-lg-0 ml-auto justify-content-center" action="{{ url_for('searches.search') }}" method=POST>

var/www/templates/settings/menu_sidebar.html

@@ -1,13 +1,58 @@
       <div class="col-12 col-lg-2 p-0 bg-light border-right" id="side_menu">
 
-      	<button type="button" class="btn btn-outline-secondary mt-1 ml-3" onclick="toggle_sidebar()">
+      	<button type="button" class="btn btn-outline-secondary mt-1 ml-3" onclick="toggle_sidebar();">
           <i class="fas fa-align-left"></i>
       	  <span>Toggle Sidebar</span>
       	</button>
 
       	<nav class="navbar navbar-expand navbar-light bg-light flex-md-column flex-row align-items-start py-2" id="nav_menu">
-      		<h5 class="d-flex text-muted w-100">
+      		<h5 class="d-flex text-muted w-100" id="nav_server">
       		  <span>Diagnostic</span>
       		</h5>
-      	</nav>
+          <ul class="nav flex-md-column flex-row navbar-nav justify-content-between w-100"> <!--nav-pills-->
+      	    <li class="nav-item">
+      	      <a class="nav-link" href="{{url_for('settings.settings_page')}}" id="nav_server_status">
+      	        <i class="fas fa-tools"></i>
+      	        <span>Server Status</span>
+      	      </a>
+      	    </li>
+          </ul>
+          <h5 class="d-flex text-muted w-100 py-2" id="nav_my_profile">
+            <span>My Profile</span>
+          </h5>
+          <ul class="nav flex-md-column flex-row navbar-nav justify-content-between w-100"> <!--nav-pills-->
+      	    <li class="nav-item">
+      	      <a class="nav-link" href="{{url_for('settings.edit_profile')}}" id="nav_edit_profile">
+      	        <i class="fas fa-user-edit"></i>
+      	        <span>My Profile</span>
+      	      </a>
+      	    </li>
+      	    <li class="nav-item">
+      	      <a class="nav-link" href="{{url_for('change_password')}}" id="nav_dashboard">
+      	        <i class="fas fa-key"></i>
+      	        <span>Change Password</span>
+      	      </a>
+      	    </li>
+          </ul>
+        {% if admin_level %}
+        		<h5 class="d-flex text-muted w-100 py-2" id="nav_user_management">
+        		  <span>User Management</span>
+        		</h5>
+            <ul class="nav flex-md-column flex-row navbar-nav justify-content-between w-100"> <!--nav-pills-->
+        	    <li class="nav-item">
+        	      <a class="nav-link" href="{{url_for('settings.create_user')}}" id="nav_create_user">
+        	        <i class="fas fa-user-plus"></i>
+        	        <span>Create User</span>
+        	      </a>
+        	    </li>
+              <li class="nav-item">
+        	      <a class="nav-link" href="{{url_for('settings.users_list')}}" id="nav_users_list">
+        	        <i class="fas fa-users"></i>
+        	        <span>Users List</span>
+        	      </a>
+        	    </li>
+            </ul>
+        {% endif %}
+
+        </nav>
       </div>

var/www/update_thirdparty.sh

@@ -102,6 +102,7 @@ rm -rf temp
 mkdir -p ./static/image
 pushd static/image
 wget -q https://www.circl.lu/assets/images/logos/AIL.png -O AIL.png
+wget -q https://www.circl.lu/assets/images/logos/AIL-logo.png -O AIL-logo.png
 popd
 
 if ! [[ -n "$AIL_HOME" ]]