This commit separates the AIL Framework services into individual Docker
containers that can be managed with docker-compose. It's intended to ease the
installation, development and troubleshooting procedures for all-in-one
deployments.
No changes to architecture or base code are required in this commit.
Changes to existing files:
.gitignore
* Ignore pystemon archive directory
Dockerfile:
* Add AIL shell environment variables
* Install pystemon and crawler pip requirements into AIL virtual environment
Docker-compose notes:
* All containers are participating in a service network `network_mode:
service:flask`. This allows the containers to share the same IP namespace
to accommodate hard-coded localhost entries.
* By default persistent data is saved to the following directories in the
local AIL framework git directory: PASTES, HASHS, CRAWLED_SCREENSHOTS,
pystemon/archives
This was tested with docker-ce on Ubuntu 16.04 and MacOS. A typical deployment
would look like:
```
git clone https://github.com/CIRCL/AIL-framework.git
cd AIL-framework
cp bin/packages/config.cfg.docker-compose-sample bin/packages/config.cfg
(optionally enable activate_crawler in config.cfg)
docker-compose build
docker-compose up -d
```
The python module DomainClassifier requires pybgpranking which is not
available in the community PyPI repository. This commit installs pybgpranking
from git using the same version required by DomainClassifier https://github.com/adulau/DomainClassifier
Fixes the following installation error when using installing_deps.sh:
```
Could not find a version that satisfies the requirement pybgpranking (from DomainClassifier->-r pip3_packages_requirement.txt (line 50)) (from versions: )
No matching distribution found for pybgpranking (from DomainClassifier->-r pip3_packages_requirement.txt (line 50))
```
