{% include 'nav_bar.html' %}
{% include 'crawler/menu_sidebar.html' %}
{% include 'crawler/crawler_disabled.html' %}
Crawl a Domain

Enter an url or a domain and choose what kind of option you want.

{% if urls %} {% for url in urls %}
{% endfor %} {% else %}
{% endif %}
 Crawler Type  
 Manual  
 Adjust crawling interval as needed
Months
Weeks
Days
Hours
Minutes
Tags
{% include 'tags/block_tags_selector.html' %}
Depth Limit
{# TEMPORARY DISABLED #} {#
#} {#
#} {#  #} {#
#} {# #} {#
#} {# Max Pages#} {#
#} {#
#}
Cookiejar: