AIL/ail-framework
7
0
Fork 0
mirror of https://github.com/ail-project/ail-framework.git synced 2024-11-10 08:38:28 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions
ail-framework/bin
History
Jason Hedden 23fb91d326 Add docker-compose support 
This commit separates the AIL Framework services into individual Docker
containers that can be managed with docker-compose. It's intended to ease the
installation, development and troubleshooting procedures for all-in-one
deployments.

No changes to architecture or base code are required in this commit.

Changes to existing files:
.gitignore
  * Ignore pystemon archive directory

Dockerfile:
  * Add AIL shell environment variables
  * Install pystemon and crawler pip requirements into AIL virtual environment

Docker-compose notes:
  * All containers are participating in a service network `network_mode:
  service:flask`.  This allows the containers to share the same IP namespace
  to accommodate hard-coded localhost entries.
  * By default persistent data is saved to the following directories in the
  local AIL framework git directory: PASTES, HASHS, CRAWLED_SCREENSHOTS,
  pystemon/archives

This was tested with docker-ce on Ubuntu 16.04 and MacOS. A typical deployment
would look like:
```
git clone https://github.com/CIRCL/AIL-framework.git
cd AIL-framework
cp bin/packages/config.cfg.docker-compose-sample bin/packages/config.cfg
	(optionally enable activate_crawler in config.cfg)
docker-compose build
docker-compose up -d
```
2019-03-11 19:46:34 -05:00
..
feeder fix path_to_send path 2019-01-09 13:15:58 +01:00
packages Add docker-compose support 2019-03-11 19:46:34 -05:00
torcrawler fix: [Crawler] typo 2019-02-12 15:54:42 +01:00
__init__.py Initial import of AIL framework - Analysis Information Leak framework 2014-08-06 11:43:40 +02:00
ailleakObject.py update ail-leak object 2018-06-18 17:23:41 +02:00
alertHandler.py misp event creation, auto and manual + the hive alert auto creation, manual case creation 2018-06-14 16:51:06 +02:00
ApiKey.py new: add base64 display on paste + base64 statistics 2018-06-29 10:02:29 +02:00
Attributes.py decode with redis connection 2018-05-04 13:53:29 +02:00
BankAccount.py chg: [statistics] clean scripts 2018-09-12 11:21:11 +02:00
Bitcoin.py chg: [Crawler UI] display domain information 2018-09-12 09:55:49 +02:00
Categ.py decode with redis connection 2018-05-04 13:53:29 +02:00
Crawler.py fix: [Crawler] change max page crawled 2019-01-29 17:04:45 +01:00
Credential.py Merge branch 'master' into statistic 2018-08-01 09:29:51 +02:00
CreditCards.py tags 2018-05-16 14:39:01 +02:00
Curve.py chg: [Terms] tag tracked terms 2018-11-06 13:38:37 +01:00
CurveManageTopSets.py change leveldb to ardb 2018-05-07 14:50:40 +02:00
Cve.py tags 2018-05-16 14:39:01 +02:00
CVE_check.py fix: [CVE] fix stat regex 2019-02-21 10:51:05 +01:00
Decoder.py fix: [hashDecoded] range_type_json 2018-09-19 15:36:33 +02:00
Dir.py decode with redis connection 2018-05-04 13:53:29 +02:00
DomClassifier.py chg: [statistics] clean scripts 2018-09-12 11:21:11 +02:00
DumpValidOnion.py decode with redis connection 2018-05-04 13:53:29 +02:00
Duplicates.py fix Duplicate, save list of duplicates on disk + prevent empty hash creation 2018-05-15 23:28:47 +02:00
empty_queue.py decode with redis connection 2018-05-04 13:53:29 +02:00
Global.py fix: [Global Crawler] max filename size 2019-02-12 15:45:58 +01:00
Helper.py Merge branch 'master' into statistic 2018-09-12 10:06:53 +02:00
import_dir.py fix: [import_dir] fix #251, fix paste import 2018-09-19 13:38:31 +02:00
Indexer.py decode with redis connection 2018-05-04 13:53:29 +02:00
indexer_lookup.py decode with redis connection 2018-05-04 13:53:29 +02:00
Keys.py chg: [Keys] typo 2018-08-01 16:05:49 +02:00
LAUNCH.sh Update LAUNCH.sh 2019-01-30 22:27:33 +01:00
launch_logs.sh Updated individual launch scripts 2017-08-23 12:28:12 +02:00
launch_lvldb.sh Updated individual launch scripts 2017-08-23 12:28:12 +02:00
launch_queues.py decode with redis connection 2018-05-04 13:53:29 +02:00
launch_queues.sh Updated individual launch scripts 2017-08-23 12:28:12 +02:00
launch_redis.sh Updated individual launch scripts 2017-08-23 12:28:12 +02:00
launch_scripts.sh Renamed BrowseWarningPaste into alertHandler 2017-11-15 16:15:43 +01:00
LibInjection.py chg: [sglinjection Phone] add tld statistic, fix phone regex 2018-07-30 11:56:50 +02:00
Lines.py decode with redis connection 2018-05-04 13:53:29 +02:00
Mail.py chg: [statistics] add script to generate pie chart png 2018-07-30 16:36:34 +02:00
MISP_The_Hive_feeder.py fix: [MISP_The_Hive_feeder] handle not saved pastes 2018-11-09 13:58:09 +01:00
Mixer.py chg: [Mixer] config default feeder name #291 2018-11-09 15:26:26 +01:00
ModuleInformation.py decode with redis connection 2018-05-04 13:53:29 +02:00
ModulesInformationV2.py fix: [ModulesInformationV2] fix #244 2018-09-19 15:45:06 +02:00
ModuleStats.py change leveldb to ardb 2018-05-07 14:50:40 +02:00
NotificationHelper.py chg: [NotificationHelper] add error traceback 2018-11-05 14:59:40 +01:00
Onion.py fix: [Onion] filter too many subdomain 2019-02-13 16:53:38 +01:00
Phone.py chg: [statistics] clean scripts 2018-09-12 11:21:11 +02:00
PreProcessFeed.py.sample fix: [MISP feeder] typo filename-config name 2018-09-21 11:02:08 +02:00
QueueIn.py decode with redis connection 2018-05-04 13:53:29 +02:00
QueueOut.py decode with redis connection 2018-05-04 13:53:29 +02:00
Queues_Monitoring.py decode with redis connection 2018-05-04 13:53:29 +02:00
RegexForTermsFrequency.py chg: [Terms] tag tracked terms 2018-11-06 13:38:37 +01:00
Release.py fix: release tiemeout alarm, chg:icon graph node 2018-07-13 11:54:14 +02:00
Repartition_graph.py change leveldb to ardb 2018-05-07 14:50:40 +02:00
SentimentAnalysis.py new: add base64 display on paste + base64 statistics 2018-06-29 10:02:29 +02:00
SetForTermsFrequency.py chg: [Terms] tag tracked terms 2018-11-06 13:38:37 +01:00
Shutdown.py decode with redis connection 2018-05-04 13:53:29 +02:00
SourceCode.py decode with redis connection 2018-05-04 13:53:29 +02:00
SQLInjectionDetection.py chg: [sglinjection Phone] add tld statistic, fix phone regex 2018-07-30 11:56:50 +02:00
submit_paste.py Merge branch 'master' into base64 2018-08-24 10:35:28 +02:00
Tags.py new: add base64 display on paste + base64 statistics 2018-06-29 10:02:29 +02:00
template.py decode with redis connection 2018-05-04 13:53:29 +02:00
Tokenize.py new: add base64 display on paste + base64 statistics 2018-06-29 10:02:29 +02:00
tor_fetcher.py decode with redis connection 2018-05-04 13:53:29 +02:00
Update-conf.py fix: [LAUNCH] use envar, add options (launch all, flask, killall) 2018-07-24 10:41:10 +02:00
Web.py add: Decoder Module, decode binary, hex and base64 2018-07-19 16:50:42 +02:00
WebStats.py change leveldb to ardb 2018-05-07 14:50:40 +02:00