Merge branch 'development' into new/rev-server

.editorconfig

@@ -1,4 +1,4 @@
-# EditorConfig is awesome: http://EditorConfig.org
+# EditorConfig is awesome: https://editorconfig.org/
 
 # top-most EditorConfig file
 root = true

.github/PULL_REQUEST_TEMPLATE.md

@@ -10,7 +10,7 @@
 - [ ] It is compatible with the [EUPL 1.2 license](https://opensource.org/licenses/EUPL-1.1)
 - [ ] I have squashed any insignificant commits. ([`git rebase`](http://gitready.com/advanced/2009/02/10/squashing-commits-with-rebase.html))
 
-Please make sure you [Sign Off](https://github.com/pi-hole/pi-hole/wiki/How-to-signoff-your-commits.) all commits. Pi-hole enforces the [DCO](https://github.com/pi-hole/pi-hole/wiki/Contributing-to-the-project).
+Please make sure you [Sign Off](https://docs.pi-hole.net/guides/github/how-to-signoff/) all commits. Pi-hole enforces the [DCO](https://docs.pi-hole.net/guides/github/contributing/).
 
 ---
 **What does this PR aim to accomplish?:**

.stickler.yml

@@ -2,5 +2,4 @@ linters:
   shellcheck:
     shell: bash
   phpcs:
-  csslint:
   flake8:

README.md

@@ -66,12 +66,10 @@ Sending a donation using our links below is **extremely helpful** in offsetting
 ### Alternative support
 If you'd rather not [donate](https://pi-hole.net/donate/) (_which is okay!_), there are other ways you can help support us:
 - [Patreon](https://patreon.com/pihole) _Become a patron for rewards_
-- [Digital Ocean](http://www.digitalocean.com/?refcode=344d234950e1) _affiliate link_
+- [Digital Ocean](https://www.digitalocean.com/?refcode=344d234950e1) _affiliate link_
 - [Stickermule](https://www.stickermule.com/unlock?ref_id=9127301701&utm_medium=link&utm_source=invite) _earn a $10 credit after your first purchase_
-- [Pi-hole Swag Store](https://pi-hole.net/shop/) _affiliate link_
+- [Amazon](https://www.amazon.com/exec/obidos/redirect-home/pihole09-20) _affiliate link_
-- [Amazon](http://www.amazon.com/exec/obidos/redirect-home/pihole09-20) _affiliate link_
 - [DNS Made Easy](https://cp.dnsmadeeasy.com/u/133706) _affiliate link_
-- [Vultr](http://www.vultr.com/?ref=7190426) _affiliate link_
 - Spreading the word about our software, and how you have benefited from it
 
 ### Contributing via GitHub
@@ -97,7 +95,6 @@ While we are primarily reachable on our <a href="https://discourse.pi-hole.net/"
   <li><a href="https://discourse.pi-hole.net/c/feature-requests?order=votes">Feature Requests</a></li>
   <li><a href="https://discourse.pi-hole.net/">Discourse User Forum</a></li>
   <li><a href="https://www.reddit.com/r/pihole/">Reddit</a></li>
-  <li><a href="https://gitter.im/pi-hole/pi-hole">Gitter</a> (Real-time chat)</li>
   <li><a href="https://twitter.com/The_Pi_Hole">Twitter</a></li>
   <li><a href="https://www.youtube.com/channel/UCT5kq9w0wSjogzJb81C9U0w">YouTube</a></li>
   <li><a href="https://www.facebook.com/ThePiHole/">Facebook</a></li>
@@ -165,48 +162,22 @@ The API can be accessed via [`telnet`](https://github.com/pi-hole/FTL), the Web
 Pi-hole being an **advertising-aware DNS/Web server**, makes use of the following technologies:
 
 * [`dnsmasq`](http://www.thekelleys.org.uk/dnsmasq/doc.html) - a lightweight DNS and DHCP server
-* [`curl`](https://curl.haxx.se) - A command line tool for transferring data with URL syntax
+* [`curl`](https://curl.haxx.se/) - A command line tool for transferring data with URL syntax
-* [`lighttpd`](https://www.lighttpd.net) - web server designed and optimized for high performance
+* [`lighttpd`](https://www.lighttpd.net/) - web server designed and optimized for high performance
-* [`php`](https://secure.php.net) - a popular general-purpose web scripting language
+* [`php`](https://www.php.net/) - a popular general-purpose web scripting language
-* [AdminLTE Dashboard](https://github.com/almasaeed2010/AdminLTE) - premium admin control panel based on Bootstrap 3.x
+* [AdminLTE Dashboard](https://github.com/ColorlibHQ/AdminLTE) - premium admin control panel based on Bootstrap 3.x
 
 While quite outdated at this point, [this original blog post about Pi-hole](https://jacobsalmela.com/2015/06/16/block-millions-ads-network-wide-with-a-raspberry-pi-hole-2-0/) goes into **great detail** about how Pi-hole was originally set up and how it works. Syntactically, it's no longer accurate, but the same basic principles and logic still apply to Pi-hole's current state.
 
 -----
 
-## Coverage
+## [Coverage](https://docs.pi-hole.net/main/coverage/)
-- [Lifehacker: Turn A Raspberry Pi Into An Ad Blocker With A Single Command](https://www.lifehacker.com.au/2015/02/turn-a-raspberry-pi-into-an-ad-blocker-with-a-single-command/) (February, 2015)
-- [MakeUseOf: Adblock Everywhere: The Raspberry Pi-Hole Way](http://www.makeuseof.com/tag/adblock-everywhere-raspberry-pi-hole-way/) (March, 2015)
-- [Catchpoint: Ad-Blocking on Apple iOS9: Valuing the End User Experience](http://blog.catchpoint.com/2015/09/14/ad-blocking-apple/) (September, 2015)
-- [Security Now Netcast: Pi-hole](https://www.youtube.com/watch?v=p7-osq_y8i8&t=100m26s) (October, 2015)
-- [TekThing: Raspberry Pi-Hole Makes Ads Disappear!](https://youtu.be/8Co59HU2gY0?t=2m) (December, 2015)
-- [Foolish Tech Show](https://youtu.be/bYyena0I9yc?t=2m4s) (December, 2015)
-- [Block Ads on All Home Devices for $53.18](https://medium.com/@robleathern/block-ads-on-all-home-devices-for-53-18-a5f1ec139693#.gj1xpgr5d) (December, 2015)
-- [Pi-Hole for Ubuntu 14.04](http://www.boyter.org/2015/12/pi-hole-ubuntu-14-04/) (December, 2015)
-- [MacObserver Podcast 585](https://www.macobserver.com/tmo/podcast/macgeekgab-585) (December, 2015)
-- [The Defrag Show: Endoscope USB Camera, The Final [HoloLens] Vote, Adblock Pi and more](https://channel9.msdn.com/Shows/The-Defrag-Show/Defrag-Endoscope-USB-Camera-The-Final-HoloLens-Vote-Adblock-Pi-and-more?WT.mc_id=dlvr_twitter_ch9#time=20m39s) (January, 2016)
-- [Adafruit: Pi-hole is a black hole for internet ads](https://blog.adafruit.com/2016/03/04/pi-hole-is-a-black-hole-for-internet-ads-piday-raspberrypi-raspberry_pi/) (March, 2016)
-- [Digital Trends: 5 Fun, Easy Projects You Can Try With a $35 Raspberry Pi](https://youtu.be/QwrKlyC2kdM?t=1m42s) (March, 2016)
-- [Adafruit: Raspberry Pi Quick Look at Pi Hole ad blocking server with Tony D](https://www.youtube.com/watch?v=eg4u2j1HYlI) (June, 2016)
-- [Devacron: OrangePi Zero as an Ad-Block server with Pi-Hole](http://www.devacron.com/orangepi-zero-as-an-ad-block-server-with-pi-hole/) (December, 2016)
-- [Linux Pro: The Hole Truth](http://www.linuxpromagazine.com/Issues/2017/200/The-sysadmin-s-daily-grind-Pi-hole) (July, 2017)
-- [Adafruit: installing Pi-hole on a Pi Zero W](https://learn.adafruit.com/pi-hole-ad-blocker-with-pi-zero-w/install-pi-hole) (August, 2017)
-- [CryptoAUSTRALIA: How We Tried 5 Privacy Focused Raspberry Pi Projects](https://blog.cryptoaustralia.org.au/2017/10/05/5-privacy-focused-raspberry-pi-projects/) (October, 2017)
-- [CryptoAUSTRALIA: Pi-hole Workshop](https://blog.cryptoaustralia.org.au/2017/11/02/pi-hole-network-wide-ad-blocker/) (November, 2017)
-- [Know How 355: Killing ads with a Raspberry Pi-Hole!](https://www.twit.tv/shows/know-how/episodes/355) (November, 2017)
-- [Hobohouse: Block Advertising on your Network with Pi-hole and Raspberry Pi](https://hobo.house/2018/02/27/block-advertising-with-pi-hole-and-raspberry-pi/) (March, 2018)
-- [Scott Helme: Securing DNS across all of my devices with Pi-Hole + DNS-over-HTTPS + 1.1.1.1](https://scotthelme.co.uk/securing-dns-across-all-of-my-devices-with-pihole-dns-over-https-1-1-1-1/) (April, 2018)
-- [Scott Helme: Catching and dealing with naughty devices on my home network](https://scotthelme.co.uk/catching-naughty-devices-on-my-home-network/) (April, 2018)
-- [Bloomberg Business Week: Brotherhood of the Ad blockers](https://www.bloomberg.com/news/features/2018-05-10/inside-the-brotherhood-of-pi-hole-ad-blockers) (May, 2018)
-- [Software Engineering Daily: Interview with the creator of Pi-hole](https://softwareengineeringdaily.com/2018/05/29/pi-hole-ad-blocker-hardware-with-jacob-salmela/) (May, 2018)
-- [Raspberry Pi: Block ads at home using Pi-hole and a Raspberry Pi](https://www.raspberrypi.org/blog/pi-hole-raspberry-pi/) (July, 2018)
-- [Troy Hunt: Mmm... Pi-hole...](https://www.troyhunt.com/mmm-pi-hole/) (September, 2018)
-- [PEBKAK Podcast: Interview With Jacob Salmela](https://www.jerseystudios.net/2018/10/11/150-pi-hole/) (October, 2018)
 
 -----
 
 ## Pi-hole Projects
-- [The Big Blocklist Collection](https://wally3k.github.io)
+
+- [The Big Blocklist Collection](https://firebog.net/)
 - [Pie in the Sky-Hole](https://dlaa.me/blog/post/skyhole)
 - [Copernicus: Windows Tray Application](https://github.com/goldbattle/copernicus)
 - [Magic Mirror with DNS Filtering](https://zonksec.com/blog/magic-mirror-dns-filtering/#dnssoftware)

advanced/Scripts/chronometer.sh

@@ -13,7 +13,7 @@ LC_NUMERIC=C
 
 # Retrieve stats from FTL engine
 pihole-FTL() {
-    ftl_port=$(cat /var/run/pihole-FTL.port 2> /dev/null)
+    ftl_port=$(cat /run/pihole-FTL.port 2> /dev/null)
     if [[ -n "$ftl_port" ]]; then
         # Open connection to FTL
         exec 3<>"/dev/tcp/127.0.0.1/$ftl_port"
@@ -153,7 +153,7 @@ get_init_stats() {
 
         sys_throttle_raw=$(vgt=$(sudo vcgencmd get_throttled); echo "${vgt##*x}")
 
-        # Active Throttle Notice: http://bit.ly/2gnunOo
+        # Active Throttle Notice: https://bit.ly/2gnunOo
         if [[ "$sys_throttle_raw" != "0" ]]; then
             case "$sys_throttle_raw" in
                 *0001) thr_type="${COL_YELLOW}Under Voltage";;

advanced/Scripts/database_migration/gravity-db.sh

@@ -104,4 +104,10 @@ upgrade_gravityDB(){
 		sqlite3 "${database}" < "${scriptPath}/10_to_11.sql"
 		version=11
 	fi
+	if [[ "$version" == "11" ]]; then
+		# Rename group 0 from "Unassociated" to "Default"
+		echo -e "  ${INFO} Upgrading gravity database from version 11 to 12"
+		sqlite3 "${database}" < "${scriptPath}/11_to_12.sql"
+		version=12
+	fi
 }

advanced/Scripts/database_migration/gravity/11_to_12.sql

@@ -0,0 +1,19 @@
+.timeout 30000
+
+PRAGMA FOREIGN_KEYS=OFF;
+
+BEGIN TRANSACTION;
+
+UPDATE "group" SET name = 'Default' WHERE id = 0;
+UPDATE "group" SET description = 'The default group' WHERE id = 0;
+
+DROP TRIGGER IF EXISTS tr_group_zero;
+
+CREATE TRIGGER tr_group_zero AFTER DELETE ON "group"
+    BEGIN
+      INSERT OR IGNORE INTO "group" (id,enabled,name,description) VALUES (0,1,'Default','The default group');
+    END;
+
+UPDATE info SET value = 12 WHERE property = 'version';
+
+COMMIT;

advanced/Scripts/list.sh

@@ -22,6 +22,9 @@ web=false
 domList=()
 
 typeId=""
+comment=""
+declare -i domaincount
+domaincount=0
 
 colfile="/opt/pihole/COL_TABLE"
 source ${colfile}
@@ -97,10 +100,12 @@ ValidateDomain() {
     fi
 
     if [[ -n "${validDomain}" ]]; then
-        domList=("${domList[@]}" ${validDomain})
+        domList=("${domList[@]}" "${validDomain}")
     else
         echo -e "  ${CROSS} ${domain} is not a valid argument or domain name!"
     fi
+
+    domaincount=$((domaincount+1))
 }
 
 ProcessDomainList() {
@@ -151,7 +156,12 @@ AddDomain() {
     reload=true
     # Insert only the domain here. The enabled and date_added fields will be filled
     # with their default values (enabled = true, date_added = current timestamp)
-    sqlite3 "${gravityDBfile}" "INSERT INTO domainlist (domain,type) VALUES ('${domain}',${typeId});"
+    if [[ -z "${comment}" ]]; then
+        sqlite3 "${gravityDBfile}" "INSERT INTO domainlist (domain,type) VALUES ('${domain}',${typeId});"
+    else
+        # also add comment when variable has been set through the "--comment" option
+        sqlite3 "${gravityDBfile}" "INSERT INTO domainlist (domain,type,comment) VALUES ('${domain}',${typeId},'${comment}');"
+    fi
 }
 
 RemoveDomain() {
@@ -224,8 +234,16 @@ NukeList() {
     sqlite3 "${gravityDBfile}" "DELETE FROM domainlist WHERE type = ${typeId};"
 }
 
-for var in "$@"; do
+GetComment() {
-    case "${var}" in
+    comment="$1"
+    if [[ "${comment}" =~ [^a-zA-Z0-9_\#:/\.,\ -] ]]; then
+      echo "  ${CROSS} Found invalid characters in domain comment!"
+      exit
+    fi
+}
+
+while (( "$#" )); do
+    case "${1}" in
         "-w" | "whitelist"   ) typeId=0;;
         "-b" | "blacklist"   ) typeId=1;;
         "--white-regex" | "white-regex" ) typeId=2;;
@@ -239,13 +257,15 @@ for var in "$@"; do
         "-l" | "--list"      ) Displaylist;;
         "--nuke"             ) NukeList;;
         "--web"              ) web=true;;
-        *                    ) ValidateDomain "${var}";;
+        "--comment"          ) GetComment "${2}"; shift;;
+        *                    ) ValidateDomain "${1}";;
     esac
+    shift
 done
 
 shift
 
-if [[ $# = 0 ]]; then
+if [[ ${domaincount} == 0 ]]; then
     helpFunc
 fi
 

advanced/Scripts/piholeCheckout.sh

@@ -3,7 +3,7 @@
 # (c) 2017 Pi-hole, LLC (https://pi-hole.net)
 # Network-wide ad blocking via your own hardware.
 #
-# Switch Pi-hole subsystems to a different Github branch.
+# Switch Pi-hole subsystems to a different GitHub branch.
 #
 # This file is copyright under the latest version of the EUPL.
 # Please see LICENSE file for your rights under this license.

advanced/Scripts/piholeDebug.sh

@@ -87,7 +87,7 @@ PIHOLE_DHCP_CONFIG_FILE="${DNSMASQ_D_DIRECTORY}/02-pihole-dhcp.conf"
 PIHOLE_WILDCARD_CONFIG_FILE="${DNSMASQ_D_DIRECTORY}/03-wildcard.conf"
 
 WEB_SERVER_CONFIG_FILE="${WEB_SERVER_CONFIG_DIRECTORY}/lighttpd.conf"
-#WEB_SERVER_CUSTOM_CONFIG_FILE="${WEB_SERVER_CONFIG_DIRECTORY}/external.conf"
+WEB_SERVER_CUSTOM_CONFIG_FILE="${WEB_SERVER_CONFIG_DIRECTORY}/external.conf"
 
 PIHOLE_INSTALL_LOG_FILE="${PIHOLE_DIRECTORY}/install.log"
 PIHOLE_RAW_BLOCKLIST_FILES="${PIHOLE_DIRECTORY}/list.*"
@@ -166,11 +166,13 @@ REQUIRED_FILES=("${PIHOLE_CRON_FILE}"
 "${PIHOLE_DHCP_CONFIG_FILE}"
 "${PIHOLE_WILDCARD_CONFIG_FILE}"
 "${WEB_SERVER_CONFIG_FILE}"
+"${WEB_SERVER_CUSTOM_CONFIG_FILE}"
 "${PIHOLE_INSTALL_LOG_FILE}"
 "${PIHOLE_RAW_BLOCKLIST_FILES}"
 "${PIHOLE_LOCAL_HOSTS_FILE}"
 "${PIHOLE_LOGROTATE_FILE}"
 "${PIHOLE_SETUP_VARS_FILE}"
+"${PIHOLE_FTL_CONF_FILE}"
 "${PIHOLE_COMMAND}"
 "${PIHOLE_COLTABLE_FILE}"
 "${FTL_PID}"
@@ -296,7 +298,11 @@ compare_local_version_to_git_version() {
                 log_write "${INFO} ${pihole_component}: ${COL_YELLOW}${remote_version:-Untagged}${COL_NC} (${FAQ_UPDATE_PI_HOLE})"
             fi
 
-            # If the repo is on the master branch, they are on the stable codebase
+            # Print the repo upstreams
+            remotes=$(git remote -v)
+            log_write "${INFO} Remotes: ${remotes//$'\n'/'\n             '}"
+
+            # If the repo is on the master branchs, they are on the stable codebase
             if [[ "${remote_branch}" == "master" ]]; then
                 # so the color of the text is green
                 log_write "${INFO} Branch: ${COL_GREEN}${remote_branch}${COL_NC}"
@@ -1107,22 +1113,19 @@ show_db_entries() {
 }
 
 show_groups() {
-    show_db_entries "Groups" "SELECT id,name,enabled,datetime(date_added,'unixepoch','localtime') date_added,datetime(date_modified,'unixepoch','localtime') date_modified,description FROM \"group\"" "4 50 7 19 19 50"
+    show_db_entries "Groups" "SELECT id,CASE enabled WHEN '0' THEN '   0' WHEN '1' THEN '      1' ELSE enabled END enabled,name,datetime(date_added,'unixepoch','localtime') date_added,datetime(date_modified,'unixepoch','localtime') date_modified,description FROM \"group\"" "4 7 50 19 19 50"
 }
 
 show_adlists() {
-    show_db_entries "Adlists" "SELECT id,address,enabled,datetime(date_added,'unixepoch','localtime') date_added,datetime(date_modified,'unixepoch','localtime') date_modified,comment FROM adlist" "4 100 7 19 19 50"
+    show_db_entries "Adlists" "SELECT id,CASE enabled WHEN '0' THEN '   0' WHEN '1' THEN '      1' ELSE enabled END enabled,GROUP_CONCAT(adlist_by_group.group_id) group_ids,address,datetime(date_added,'unixepoch','localtime') date_added,datetime(date_modified,'unixepoch','localtime') date_modified,comment FROM adlist LEFT JOIN adlist_by_group ON adlist.id = adlist_by_group.adlist_id GROUP BY id;" "4 7 12 100 19 19 50"
-    show_db_entries "Adlist groups" "SELECT * FROM adlist_by_group" "4 4"
 }
 
 show_domainlist() {
-    show_db_entries "Domainlist (0/1 = exact white-/blacklist, 2/3 = regex white-/blacklist)" "SELECT id,type,domain,enabled,datetime(date_added,'unixepoch','localtime') date_added,datetime(date_modified,'unixepoch','localtime') date_modified,comment FROM domainlist" "4 4 100 7 19 19 50"
+    show_db_entries "Domainlist (0/1 = exact white-/blacklist, 2/3 = regex white-/blacklist)" "SELECT id,CASE type WHEN '0' THEN '0   ' WHEN '1' THEN ' 1  ' WHEN '2' THEN '  2 ' WHEN '3' THEN '   3' ELSE type END type,CASE enabled WHEN '0' THEN '   0' WHEN '1' THEN '      1' ELSE enabled END enabled,GROUP_CONCAT(domainlist_by_group.group_id) group_ids,domain,datetime(date_added,'unixepoch','localtime') date_added,datetime(date_modified,'unixepoch','localtime') date_modified,comment FROM domainlist LEFT JOIN domainlist_by_group ON domainlist.id = domainlist_by_group.domainlist_id GROUP BY id;" "4 4 7 12 100 19 19 50"
-    show_db_entries "Domainlist groups" "SELECT * FROM domainlist_by_group" "10 10"
 }
 
 show_clients() {
-    show_db_entries "Clients" "SELECT id,ip,datetime(date_added,'unixepoch','localtime') date_added,datetime(date_modified,'unixepoch','localtime') date_modified,comment FROM client" "4 100 19 19 50"
+    show_db_entries "Clients" "SELECT id,GROUP_CONCAT(client_by_group.group_id) group_ids,ip,datetime(date_added,'unixepoch','localtime') date_added,datetime(date_modified,'unixepoch','localtime') date_modified,comment FROM client LEFT JOIN client_by_group ON client.id = client_by_group.client_id GROUP BY id;" "4 12 100 19 19 50"
-    show_db_entries "Client groups" "SELECT * FROM client_by_group" "10 10"
 }
 
 analyze_gravity_list() {

advanced/Scripts/query.sh

@@ -29,7 +29,7 @@ scanList(){
     # Prevent grep from printing file path
     cd "$piholeDir" || exit 1
 
-    # Prevent grep -i matching slowly: http://bit.ly/2xFXtUX
+    # Prevent grep -i matching slowly: https://bit.ly/2xFXtUX
     export LC_CTYPE=C
 
     # /dev/null forces filename to be printed when only one list has been generated

advanced/Scripts/setupLCD.sh

@@ -70,5 +70,5 @@ setupcon
 reboot
 
 # Start showing the stats on the screen by running the command on another tty:
-# http://unix.stackexchange.com/questions/170063/start-a-process-on-a-different-tty
+# https://unix.stackexchange.com/questions/170063/start-a-process-on-a-different-tty
 #setsid sh -c 'exec /usr/local/bin/chronometer.sh <> /dev/tty1 >&0 2>&1'

advanced/Scripts/version.sh

@@ -84,6 +84,21 @@ getRemoteVersion(){
     # Get the version from the remote origin
     local daemon="${1}"
     local version
+    local cachedVersions
+    local arrCache
+    cachedVersions="/etc/pihole/GitHubVersions"
+
+    #If the above file exists, then we can read from that. Prevents overuse of GitHub API
+    if [[ -f "$cachedVersions" ]]; then
+        IFS=' ' read -r -a arrCache < "$cachedVersions"
+        case $daemon in
+          "pi-hole"   )  echo "${arrCache[0]}";;
+          "AdminLTE"  )  echo "${arrCache[1]}";;
+          "FTL"       )  echo "${arrCache[2]}";;
+        esac
+
+        return 0
+    fi
 
     version=$(curl --silent --fail "https://api.github.com/repos/pi-hole/${daemon}/releases/latest" | \
         awk -F: '$1 ~/tag_name/ { print $2 }' | \
@@ -97,22 +112,48 @@ getRemoteVersion(){
     return 0
 }
 
+getLocalBranch(){
+    # Get the checked out branch of the local directory
+    local directory="${1}"
+    local branch
+
+     # Local FTL btranch is stored in /etc/pihole/ftlbranch
+    if [[ "$1" == "FTL" ]]; then
+        branch="$(pihole-FTL branch)"
+    else
+        cd "${directory}" 2> /dev/null || { echo "${DEFAULT}"; return 1; }
+        branch=$(git rev-parse --abbrev-ref HEAD || echo "$DEFAULT")
+    fi
+    if [[ ! "${branch}" =~ ^v ]]; then
+        if [[ "${branch}" == "master" ]]; then
+            echo ""
+        elif [[ "${branch}" == "HEAD" ]]; then
+            echo "in detached HEAD state at "
+        else
+            echo "${branch} "
+        fi
+    else
+        # Branch started in "v"
+        echo "release "
+    fi
+    return 0
+}
+
 versionOutput() {
     [[ "$1" == "pi-hole" ]] && GITDIR=$COREGITDIR
     [[ "$1" == "AdminLTE" ]] && GITDIR=$WEBGITDIR
     [[ "$1" == "FTL" ]] && GITDIR="FTL"
 
-    [[ "$2" == "-c" ]] || [[ "$2" == "--current" ]] || [[ -z "$2" ]] && current=$(getLocalVersion $GITDIR)
+    [[ "$2" == "-c" ]] || [[ "$2" == "--current" ]] || [[ -z "$2" ]] && current=$(getLocalVersion $GITDIR) && branch=$(getLocalBranch $GITDIR)
     [[ "$2" == "-l" ]] || [[ "$2" == "--latest" ]] || [[ -z "$2" ]] && latest=$(getRemoteVersion "$1")
     if [[ "$2" == "-h" ]] || [[ "$2" == "--hash" ]]; then
-        [[ "$3" == "-c" ]] || [[ "$3" == "--current" ]] || [[ -z "$3" ]] && curHash=$(getLocalHash "$GITDIR")
+        [[ "$3" == "-c" ]] || [[ "$3" == "--current" ]] || [[ -z "$3" ]] && curHash=$(getLocalHash "$GITDIR") && branch=$(getLocalBranch $GITDIR)
         [[ "$3" == "-l" ]] || [[ "$3" == "--latest" ]] || [[ -z "$3" ]] && latHash=$(getRemoteHash "$1" "$(cd "$GITDIR" 2> /dev/null && git rev-parse --abbrev-ref HEAD)")
     fi
-
     if [[ -n "$current" ]] && [[ -n "$latest" ]]; then
-        output="${1^} version is $current (Latest: $latest)"
+        output="${1^} version is $branch$current (Latest: $latest)"
     elif [[ -n "$current" ]] && [[ -z "$latest" ]]; then
-        output="Current ${1^} version is $current"
+        output="Current ${1^} version is $branch$current."
     elif [[ -z "$current" ]] && [[ -n "$latest" ]]; then
         output="Latest ${1^} version is $latest"
     elif [[ "$curHash" == "N/A" ]] || [[ "$latHash" == "N/A" ]]; then
@@ -162,7 +203,7 @@ Repositories:
 Options:
   -c, --current        Return the current version
   -l, --latest         Return the latest version
-  --hash               Return the Github hash from your local repositories
+  --hash               Return the GitHub hash from your local repositories
   -h, --help           Show this help dialog"
   exit 0
 }

advanced/Scripts/webpage.sh

@@ -18,6 +18,7 @@ readonly FTLconf="/etc/pihole/pihole-FTL.conf"
 readonly dhcpstaticconfig="/etc/dnsmasq.d/04-pihole-static-dhcp.conf"
 readonly PI_HOLE_BIN_DIR="/usr/local/bin"
 readonly dnscustomfile="/etc/pihole/custom.list"
+readonly dnscustomcnamefile="/etc/dnsmasq.d/05-pihole-custom-cname.conf"
 
 readonly gravityDBfile="/etc/pihole/gravity.db"
 
@@ -36,7 +37,6 @@ Options:
   -c, celsius         Set Celsius as preferred temperature unit
   -f, fahrenheit      Set Fahrenheit as preferred temperature unit
   -k, kelvin          Set Kelvin as preferred temperature unit
-  -r, hostrecord      Add a name to the DNS associated to an IPv4/IPv6 address
   -e, email           Set an administrative contact address for the Block Page
   -h, --help          Show this help dialog
   -i, interface       Specify dnsmasq's interface listening behavior
@@ -432,6 +432,10 @@ SetWebUILayout() {
     change_setting "WEBUIBOXEDLAYOUT" "${args[2]}"
 }
 
+SetWebUITheme() {
+    change_setting "WEBTHEME" "${args[2]}"
+}
+
 CheckUrl(){
     local regex
     # Check for characters NOT allowed in URLs
@@ -509,32 +513,6 @@ RemoveDHCPStaticAddress() {
     sed -i "/dhcp-host=${mac}.*/d" "${dhcpstaticconfig}"
 }
 
-SetHostRecord() {
-    if [[ "${1}" == "-h" ]] || [[ "${1}" == "--help" ]]; then
-        echo "Usage: pihole -a hostrecord <domain> [IPv4-address],[IPv6-address]
-Example: 'pihole -a hostrecord home.domain.com 192.168.1.1,2001:db8:a0b:12f0::1'
-Add a name to the DNS associated to an IPv4/IPv6 address
-
-Options:
-  \"\"                  Empty: Remove host record
-  -h, --help          Show this help dialog"
-        exit 0
-    fi
-
-    if [[ -n "${args[3]}" ]]; then
-        change_setting "HOSTRECORD" "${args[2]},${args[3]}"
-        echo -e "  ${TICK} Setting host record for ${args[2]} to ${args[3]}"
-    else
-        change_setting "HOSTRECORD" ""
-        echo -e "  ${TICK} Removing host record"
-    fi
-
-    ProcessDNSSettings
-
-    # Restart dnsmasq to load new configuration
-    RestartDNS
-}
-
 SetAdminEmail() {
     if [[ "${1}" == "-h" ]] || [[ "${1}" == "--help" ]]; then
         echo "Usage: pihole -a email <address>
@@ -550,7 +528,10 @@ Options:
     if [[ -n "${args[2]}" ]]; then
 
         # Sanitize email address in case of security issues
-        if [[ ! "${args[2]}" =~ ^[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\.[A-Za-z]{2,4}$  ]]; then
+        # Regex from https://stackoverflow.com/a/2138832/4065967
+        local regex
+        regex="^[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\.[A-Za-z]{2,}\$"
+        if [[ ! "${args[2]}" =~ ${regex} ]]; then
             echo -e "  ${CROSS} Invalid email address"
             exit 0
         fi
@@ -650,6 +631,7 @@ SetPrivacyLevel() {
     # Set privacy level. Minimum is 0, maximum is 4
     if [ "${args[2]}" -ge 0 ] && [ "${args[2]}" -le 4 ]; then
         changeFTLsetting "PRIVACYLEVEL" "${args[2]}"
+        pihole restartdns reload-lists
     fi
 }
 
@@ -675,6 +657,28 @@ RemoveCustomDNSAddress() {
     RestartDNS
 }
 
+AddCustomCNAMERecord() {
+    echo -e "  ${TICK} Adding custom CNAME record..."
+
+    domain="${args[2]}"
+    target="${args[3]}"
+    echo "cname=${domain},${target}" >> "${dnscustomcnamefile}"
+
+    # Restart dnsmasq to load new custom CNAME records
+    RestartDNS
+}
+
+RemoveCustomCNAMERecord() {
+    echo -e "  ${TICK} Removing custom CNAME record..."
+
+    domain="${args[2]}"
+    target="${args[3]}"
+    sed -i "/cname=${domain},${target}/d" "${dnscustomcnamefile}"
+
+    # Restart dnsmasq to update removed custom CNAME records
+    RestartDNS
+}
+
 main() {
     args=("$@")
 
@@ -693,12 +697,12 @@ main() {
         "enabledhcp"          ) EnableDHCP;;
         "disabledhcp"         ) DisableDHCP;;
         "layout"              ) SetWebUILayout;;
+        "theme"               ) SetWebUITheme;;
         "-h" | "--help"       ) helpFunc;;
         "privacymode"         ) SetPrivacyMode;;
         "resolve"             ) ResolutionSettings;;
         "addstaticdhcp"       ) AddDHCPStaticAddress;;
         "removestaticdhcp"    ) RemoveDHCPStaticAddress;;
-        "-r" | "hostrecord"   ) SetHostRecord "$3";;
         "-e" | "email"        ) SetAdminEmail "$3";;
         "-i" | "interface"    ) SetListeningMode "$@";;
         "-t" | "teleporter"   ) Teleporter;;
@@ -708,6 +712,8 @@ main() {
         "-l" | "privacylevel" ) SetPrivacyLevel;;
         "addcustomdns"        ) AddCustomDNSAddress;;
         "removecustomdns"     ) RemoveCustomDNSAddress;;
+        "addcustomcname"      ) AddCustomCNAMERecord;;
+        "removecustomcname"   ) RemoveCustomCNAMERecord;;
         *                     ) helpFunc;;
     esac
 

advanced/Templates/gravity.db.sql

@@ -10,17 +10,18 @@ CREATE TABLE "group"
 	date_modified INTEGER NOT NULL DEFAULT (cast(strftime('%s', 'now') as int)),
 	description TEXT
 );
-INSERT INTO "group" (id,enabled,name) VALUES (0,1,'Unassociated');
+INSERT INTO "group" (id,enabled,name,description) VALUES (0,1,'Default','The default group');
 
 CREATE TABLE domainlist
 (
 	id INTEGER PRIMARY KEY AUTOINCREMENT,
 	type INTEGER NOT NULL DEFAULT 0,
-	domain TEXT UNIQUE NOT NULL,
+	domain TEXT NOT NULL,
 	enabled BOOLEAN NOT NULL DEFAULT 1,
 	date_added INTEGER NOT NULL DEFAULT (cast(strftime('%s', 'now') as int)),
 	date_modified INTEGER NOT NULL DEFAULT (cast(strftime('%s', 'now') as int)),
-	comment TEXT
+	comment TEXT,
+	UNIQUE(domain, type)
 );
 
 CREATE TABLE adlist
@@ -52,7 +53,7 @@ CREATE TABLE info
 	value TEXT NOT NULL
 );
 
-INSERT INTO "info" VALUES('version','11');
+INSERT INTO "info" VALUES('version','12');
 
 CREATE TABLE domain_audit
 (
@@ -167,7 +168,7 @@ CREATE TRIGGER tr_group_update AFTER UPDATE ON "group"
 
 CREATE TRIGGER tr_group_zero AFTER DELETE ON "group"
     BEGIN
-      INSERT OR IGNORE INTO "group" (id,enabled,name) VALUES (0,1,'Unassociated');
+      INSERT OR IGNORE INTO "group" (id,enabled,name) VALUES (0,1,'Default');
     END;
 
 CREATE TRIGGER tr_domainlist_delete AFTER DELETE ON domainlist

advanced/Templates/pihole-FTL.service

@@ -1,8 +1,8 @@
 #!/usr/bin/env bash
 ### BEGIN INIT INFO
 # Provides:          pihole-FTL
-# Required-Start:    $remote_fs $syslog
+# Required-Start:    $remote_fs $syslog $network
-# Required-Stop:     $remote_fs $syslog
+# Required-Stop:     $remote_fs $syslog $network
 # Default-Start:     2 3 4 5
 # Default-Stop:      0 1 6
 # Short-Description: pihole-FTL daemon
@@ -10,7 +10,7 @@
 ### END INIT INFO
 
 FTLUSER=pihole
-PIDFILE=/var/run/pihole-FTL.pid
+PIDFILE=/run/pihole-FTL.pid
 
 get_pid() {
     # First, try to obtain PID from PIDFILE
@@ -37,12 +37,12 @@ start() {
     touch /var/log/pihole-FTL.log /var/log/pihole.log
     touch /run/pihole-FTL.pid /run/pihole-FTL.port
     touch /etc/pihole/dhcp.leases
-    mkdir -p /var/run/pihole
+    mkdir -p /run/pihole
     mkdir -p /var/log/pihole
-    chown pihole:pihole /var/run/pihole /var/log/pihole
+    chown pihole:pihole /run/pihole /var/log/pihole
     # Remove possible leftovers from previous pihole-FTL processes
     rm -f /dev/shm/FTL-* 2> /dev/null
-    rm /var/run/pihole/FTL.sock 2> /dev/null
+    rm /run/pihole/FTL.sock 2> /dev/null
     # Ensure that permissions are set so that pihole-FTL can edit all necessary files
     chown pihole:pihole /run/pihole-FTL.pid /run/pihole-FTL.port
     chown pihole:pihole /etc/pihole /etc/pihole/dhcp.leases 2> /dev/null
@@ -50,7 +50,7 @@ start() {
     chmod 0644 /var/log/pihole-FTL.log /run/pihole-FTL.pid /run/pihole-FTL.port /var/log/pihole.log
     # Chown database files to the user FTL runs as. We ignore errors as the files may not (yet) exist
     chown pihole:pihole /etc/pihole/pihole-FTL.db /etc/pihole/gravity.db 2> /dev/null
-    if setcap CAP_NET_BIND_SERVICE,CAP_NET_RAW,CAP_NET_ADMIN+eip "$(which pihole-FTL)"; then
+    if setcap CAP_NET_BIND_SERVICE,CAP_NET_RAW,CAP_NET_ADMIN,CAP_SYS_NICE+eip "$(which pihole-FTL)"; then
       su -s /bin/sh -c "/usr/bin/pihole-FTL" "$FTLUSER"
     else
       echo "Warning: Starting pihole-FTL as root because setting capabilities is not supported on this system"

advanced/Templates/pihole.cron

@@ -10,7 +10,7 @@
 #
 #
 # This file is under source-control of the Pi-hole installation and update
-# scripts, any changes made to this file will be overwritten when the softare
+# scripts, any changes made to this file will be overwritten when the software
 # is updated or re-installed. Please make any changes to the appropriate crontab
 # or other cron file snippets.
 

advanced/bash-completion/pihole

@@ -15,7 +15,7 @@ _pihole() {
 			COMPREPLY=( $(compgen -W "${opts_lists}" -- ${cur}) )
 		;;
 		"admin")
-			opts_admin="celsius email fahrenheit hostrecord interface kelvin password privacylevel"
+			opts_admin="celsius email fahrenheit interface kelvin password privacylevel"
 			COMPREPLY=( $(compgen -W "${opts_admin}" -- ${cur}) )
 		;;
 		"checkout")

advanced/index.php

@@ -6,8 +6,8 @@
 *  This file is copyright under the latest version of the EUPL.
 *  Please see LICENSE file for your rights under this license. */
 
-// Sanitize HTTP_HOST output
+// Sanitize SERVER_NAME output
-$serverName = htmlspecialchars($_SERVER["HTTP_HOST"]);
+$serverName = htmlspecialchars($_SERVER["SERVER_NAME"]);
 // Remove external ipv6 brackets if any
 $serverName = preg_replace('/^\[(.*)\]$/', '${1}', $serverName);
 
@@ -50,16 +50,24 @@ function setHeader($type = "x") {
 }
 
 // Determine block page type
-if ($serverName === "pi.hole") {
+if ($serverName === "pi.hole"
+    || (!empty($_SERVER["VIRTUAL_HOST"]) && $serverName === $_SERVER["VIRTUAL_HOST"])) {
     // Redirect to Web Interface
     exit(header("Location: /admin"));
 } elseif (filter_var($serverName, FILTER_VALIDATE_IP) || in_array($serverName, $authorizedHosts)) {
     // Set Splash Page output
     $splashPage = "
-    <html><head>
+    <html>
+      <head>
         $viewPort
-        <link rel='stylesheet' href='/pihole/blockingpage.css' type='text/css'/>
+        <link rel='stylesheet' href='pihole/blockingpage.css' type='text/css'/>
-    </head><body id='splashpage'><img src='/admin/img/logo.svg'/><br/>Pi-<b>hole</b>: Your black hole for Internet advertisements<br><a href='/admin'>Did you mean to go to the admin panel?</a></body></html>
+      </head>
+      <body id='splashpage'>
+        <img src='admin/img/logo.svg'/><br/>
+        Pi-<b>hole</b>: Your black hole for Internet advertisements<br/>
+        <a href='/admin'>Did you mean to go to the admin panel?</a>
+      </body>
+    </html>
     ";
 
     // Set splash/landing page based off presence of $landPage
@@ -131,7 +139,12 @@ ini_set("default_socket_timeout", 3);
 function queryAds($serverName) {
     // Determine the time it takes while querying adlists
     $preQueryTime = microtime(true)-$_SERVER["REQUEST_TIME_FLOAT"];
-    $queryAds = file("http://127.0.0.1/admin/scripts/pi-hole/php/queryads.php?domain=$serverName&bp", FILE_IGNORE_NEW_LINES);
+    $queryAdsURL = sprintf(
+        "http://127.0.0.1:%s/admin/scripts/pi-hole/php/queryads.php?domain=%s&bp",
+        $_SERVER["SERVER_PORT"],
+        $serverName
+    );
+    $queryAds = file($queryAdsURL, FILE_IGNORE_NEW_LINES);
     $queryAds = array_values(array_filter(preg_replace("/data:\s+/", "", $queryAds)));
     $queryTime = sprintf("%.0f", (microtime(true)-$_SERVER["REQUEST_TIME_FLOAT"]) - $preQueryTime);
 
@@ -226,10 +239,10 @@ setHeader();
   <?=$viewPort ?>
   <meta name="robots" content="noindex,nofollow"/>
   <meta http-equiv="x-dns-prefetch-control" content="off">
-  <link rel="shortcut icon" href="//pi.hole/admin/img/favicon.png" type="image/x-icon"/>
+  <link rel="shortcut icon" href="admin/img/favicon.png" type="image/x-icon"/>
-  <link rel="stylesheet" href="//pi.hole/pihole/blockingpage.css" type="text/css"/>
+  <link rel="stylesheet" href="pihole/blockingpage.css" type="text/css"/>
   <title>● <?=$serverName ?></title>
-  <script src="//pi.hole/admin/scripts/vendor/jquery.min.js"></script>
+  <script src="admin/scripts/vendor/jquery.min.js"></script>
   <script>
     window.onload = function () {
       <?php

advanced/lighttpd.conf.debian

@@ -30,7 +30,7 @@ server.document-root        = "/var/www/html"
 server.error-handler-404    = "/pihole/index.php"
 server.upload-dirs          = ( "/var/cache/lighttpd/uploads" )
 server.errorlog             = "/var/log/lighttpd/error.log"
-server.pid-file             = "/var/run/lighttpd.pid"
+server.pid-file             = "/run/lighttpd.pid"
 server.username             = "www-data"
 server.groupname            = "www-data"
 server.port                 = 80
@@ -49,7 +49,7 @@ mimetype.assign   = ( ".png"  => "image/png",
                       ".jpeg" => "image/jpeg",
                       ".html" => "text/html",
                       ".css" => "text/css; charset=utf-8",
-                      ".js" => "application/javascript",
+                      ".js" => "application/javascript; charset=utf-8",
                       ".json" => "application/json",
                       ".txt"  => "text/plain",
                       ".svg"  => "image/svg+xml" )

advanced/lighttpd.conf.fedora

@@ -31,7 +31,7 @@ server.document-root        = "/var/www/html"
 server.error-handler-404    = "/pihole/index.php"
 server.upload-dirs          = ( "/var/cache/lighttpd/uploads" )
 server.errorlog             = "/var/log/lighttpd/error.log"
-server.pid-file             = "/var/run/lighttpd.pid"
+server.pid-file             = "/run/lighttpd.pid"
 server.username             = "lighttpd"
 server.groupname            = "lighttpd"
 server.port                 = 80
@@ -51,7 +51,7 @@ mimetype.assign   = ( ".png"  => "image/png",
                       ".jpeg" => "image/jpeg",
                       ".html" => "text/html",
                       ".css" => "text/css; charset=utf-8",
-                      ".js" => "application/javascript",
+                      ".js" => "application/javascript; charset=utf-8",
                       ".json" => "application/json",
                       ".txt"  => "text/plain",
                       ".svg"  => "image/svg+xml" )

automated install/basic-install.sh

@@ -184,7 +184,7 @@ if is_command apt-get ; then
     # A variable to store the command used to update the package cache
     UPDATE_PKG_CACHE="${PKG_MANAGER} update"
     # An array for something...
-    PKG_INSTALL=("${PKG_MANAGER}" --yes --no-install-recommends install)
+    PKG_INSTALL=("${PKG_MANAGER}" -qq --no-install-recommends install)
     # grep -c will return 1 retVal on 0 matches, block this throwing the set -e with an OR TRUE
     PKG_COUNT="${PKG_MANAGER} -s -o Debug::NoLocking=true upgrade | grep -c ^Inst || true"
     # Some distros vary slightly so these fixes for dependencies may apply
@@ -192,8 +192,8 @@ if is_command apt-get ; then
     APT_SOURCES="/etc/apt/sources.list"
     if awk 'BEGIN{a=1;b=0}/bionic main/{a=0}/bionic.*universe/{b=1}END{exit a + b}' ${APT_SOURCES}; then
         if ! whiptail --defaultno --title "Dependencies Require Update to Allowed Repositories" --yesno "Would you like to enable 'universe' repository?\\n\\nThis repository is required by the following packages:\\n\\n- dhcpcd5" "${r}" "${c}"; then
-            printf "  %b Aborting installation: dependencies could not be installed.\\n" "${CROSS}"
+            printf "  %b Aborting installation: Dependencies could not be installed.\\n" "${CROSS}"
-            exit # exit the installer
+            exit 1 # exit the installer
         else
             printf "  %b Enabling universe package repository for Ubuntu Bionic\\n" "${INFO}"
             cp -p ${APT_SOURCES} ${APT_SOURCES}.backup # Backup current repo list
@@ -202,14 +202,18 @@ if is_command apt-get ; then
             printf "  %b Enabled %s\\n" "${TICK}" "'universe' repository"
         fi
     fi
-    # Debian 7 doesn't have iproute2 so if the dry run install is successful,
+    # Update package cache. This is required already here to assure apt-cache calls have package lists available.
-    if "${PKG_MANAGER}" install --dry-run iproute2 > /dev/null 2>&1; then
+    update_package_cache || exit 1
-        # we can install it
+    # Debian 7 doesn't have iproute2 so check if it's available first
+    if apt-cache show iproute2 > /dev/null 2>&1; then
         iproute_pkg="iproute2"
-    # Otherwise,
+    # Otherwise, check if iproute is available
-    else
+    elif apt-cache show iproute > /dev/null 2>&1; then
-        # use iproute
         iproute_pkg="iproute"
+    # Else print error and exit
+    else
+        printf "  %b Aborting installation: iproute2 and iproute packages were not found in APT repository.\\n" "${CROSS}"
+        exit 1
     fi
     # Check for and determine version number (major and minor) of current php install
     if is_command php ; then
@@ -224,21 +228,28 @@ if is_command apt-get ; then
     # Check if installed php is v 7.0, or newer to determine packages to install
     if [[ "$phpInsNewer" != true ]]; then
         # Prefer the php metapackage if it's there
-        if "${PKG_MANAGER}" install --dry-run php > /dev/null 2>&1; then
+        if apt-cache show php > /dev/null 2>&1; then
             phpVer="php"
-        # fall back on the php5 packages
+        # Else fall back on the php5 package if it's there
-        else
+        elif apt-cache show php5 > /dev/null 2>&1; then
             phpVer="php5"
+        # Else print error and exit
+        else
+            printf "  %b Aborting installation: No PHP packages were found in APT repository.\\n" "${CROSS}"
+            exit 1
         fi
     else
         # Newer php is installed, its common, cgi & sqlite counterparts are deps
         phpVer="php$phpInsMajor.$phpInsMinor"
     fi
     # We also need the correct version for `php-sqlite` (which differs across distros)
-    if "${PKG_MANAGER}" install --dry-run "${phpVer}-sqlite3" > /dev/null 2>&1; then
+    if apt-cache show "${phpVer}-sqlite3" > /dev/null 2>&1; then
         phpSqlite="sqlite3"
-    else
+    elif apt-cache show "${phpVer}-sqlite" > /dev/null 2>&1; then
         phpSqlite="sqlite"
+    else
+        printf "  %b Aborting installation: No SQLite PHP module was found in APT repository.\\n" "${CROSS}"
+        exit 1
     fi
     # Since our install script is so large, we need several other programs to successfully get a machine provisioned
     # These programs are stored in an array so they can be looped through later
@@ -247,7 +258,7 @@ if is_command apt-get ; then
     PIHOLE_DEPS=(cron curl dnsutils iputils-ping lsof netcat psmisc sudo unzip wget idn2 sqlite3 libcap2-bin dns-root-data libcap2)
     # The Web dashboard has some that also need to be installed
     # It's useful to separate the two since our repos are also setup as "Core" code and "Web" code
-    PIHOLE_WEB_DEPS=(lighttpd "${phpVer}-common" "${phpVer}-cgi" "${phpVer}-${phpSqlite}" "${phpVer}-xml" "php-intl")
+    PIHOLE_WEB_DEPS=(lighttpd "${phpVer}-common" "${phpVer}-cgi" "${phpVer}-${phpSqlite}" "${phpVer}-xml" "${phpVer}-intl")
     # The Web server user,
     LIGHTTPD_USER="www-data"
     # group,
@@ -281,8 +292,6 @@ elif is_command rpm ; then
         PKG_MANAGER="yum"
     fi
 
-    # Fedora and family update cache on every PKG_INSTALL call, no need for a separate update.
-    UPDATE_PKG_CACHE=":"
     PKG_INSTALL=("${PKG_MANAGER}" install -y)
     PKG_COUNT="${PKG_MANAGER} check-update | egrep '(.i686|.x86|.noarch|.arm|.src)' | wc -l"
     INSTALLER_DEPS=(git iproute newt procps-ng which chkconfig)
@@ -426,19 +435,18 @@ make_repo() {
     fi
     # Clone the repo and return the return code from this command
     git clone -q --depth 20 "${remoteRepo}" "${directory}" &> /dev/null || return $?
-    # Data in the repositories is public anyway so we can make it readable by everyone (+r to keep executable permission if already set by git)
-    chmod -R a+rX "${directory}"
     # Move into the directory that was passed as an argument
     pushd "${directory}" &> /dev/null || return 1
     # Check current branch. If it is master, then reset to the latest available tag.
-    # In case extra commits have been added after tagging/release (i.e in case of metadata updates/README.MD tweaks)    
+    # In case extra commits have been added after tagging/release (i.e in case of metadata updates/README.MD tweaks)
     curBranch=$(git rev-parse --abbrev-ref HEAD)
     if [[ "${curBranch}" == "master" ]]; then #If we're calling make_repo() then it should always be master, we may not need to check.
          git reset --hard "$(git describe --abbrev=0 --tags)" || return $?
     fi
     # Show a colored message showing it's status
     printf "%b  %b %s\\n" "${OVER}" "${TICK}" "${str}"
-
+    # Data in the repositories is public anyway so we can make it readable by everyone (+r to keep executable permission if already set by git)
+    chmod -R a+rX "${directory}"
     # Move back into the original directory
     popd &> /dev/null || return 1
     return 0
@@ -467,7 +475,7 @@ update_repo() {
     # Pull the latest commits
     git pull --quiet &> /dev/null || return $?
     # Check current branch. If it is master, then reset to the latest available tag.
-    # In case extra commits have been added after tagging/release (i.e in case of metadata updates/README.MD tweaks)    
+    # In case extra commits have been added after tagging/release (i.e in case of metadata updates/README.MD tweaks)
     curBranch=$(git rev-parse --abbrev-ref HEAD)
     if [[ "${curBranch}" == "master" ]]; then
          git reset --hard "$(git describe --abbrev=0 --tags)" || return $?
@@ -573,7 +581,7 @@ welcomeDialogs() {
     whiptail --msgbox --backtitle "Welcome" --title "Pi-hole automated installer" "\\n\\nThis installer will transform your device into a network-wide ad blocker!" "${r}" "${c}"
 
     # Request that users donate if they enjoy the software since we all work on it in our free time
-    whiptail --msgbox --backtitle "Plea" --title "Free and open source" "\\n\\nThe Pi-hole is free, but powered by your donations:  http://pi-hole.net/donate" "${r}" "${c}"
+    whiptail --msgbox --backtitle "Plea" --title "Free and open source" "\\n\\nThe Pi-hole is free, but powered by your donations:  https://pi-hole.net/donate/" "${r}" "${c}"
 
     # Explain the need for a static address
     whiptail --msgbox --backtitle "Initiating network interface" --title "Static IP Needed" "\\n\\nThe Pi-hole is a SERVER so it needs a STATIC IP ADDRESS to function properly.
@@ -665,7 +673,7 @@ chooseInterface() {
         # Feed the available interfaces into this while loop
         done <<< "${availableInterfaces}"
         # The whiptail command that will be run, stored in a variable
-        chooseInterfaceCmd=(whiptail --separate-output --radiolist "Choose An Interface (press space to select)" "${r}" "${c}" "${interfaceCount}")
+        chooseInterfaceCmd=(whiptail --separate-output --radiolist "Choose An Interface (press space to toggle selection)" "${r}" "${c}" "${interfaceCount}")
         # Now run the command using the interfaces saved into the array
         chooseInterfaceOptions=$("${chooseInterfaceCmd[@]}" "${interfacesArray[@]}" 2>&1 >/dev/tty) || \
         # If the user chooses Cancel, exit
@@ -755,8 +763,8 @@ use4andor6() {
     # Named local variables
     local useIPv4
     local useIPv6
-    # Let use select IPv4 and/or IPv6 via a checklist
+    # Let user choose IPv4 and/or IPv6 via a checklist
-    cmd=(whiptail --separate-output --checklist "Select Protocols (press space to select)" "${r}" "${c}" 2)
+    cmd=(whiptail --separate-output --checklist "Select Protocols (press space to toggle selection)" "${r}" "${c}" 2)
     # In an array, show the options available:
     # IPv4 (on by default)
     options=(IPv4 "Block ads over IPv4" on
@@ -1206,21 +1214,21 @@ chooseBlocklists() {
         mv "${adlistFile}" "${adlistFile}.old"
     fi
     # Let user select (or not) blocklists via a checklist
-    cmd=(whiptail --separate-output --checklist "Pi-hole relies on third party lists in order to block ads.\\n\\nYou can use the suggestions below, and/or add your own after installation\\n\\nTo deselect any list, use the arrow keys and spacebar" "${r}" "${c}" 6)
+    cmd=(whiptail --separate-output --checklist "Pi-hole relies on third party lists in order to block ads.\\n\\nYou can use the suggestions below, and/or add your own after installation\\n\\nTo deselect any list, use the arrow keys and spacebar" "${r}" "${c}" 5)
     # In an array, show the options available (all off by default):
     options=(StevenBlack "StevenBlack's Unified Hosts List" on
-        MalwareDom "MalwareDomains" on
+        MalwareDom "MalwareDomains" on)
-        Cameleon "Cameleon" on
-        DisconTrack "Disconnect.me Tracking" on
-        DisconAd "Disconnect.me Ads" on)
 
     # In a variable, show the choices available; exit if Cancel is selected
     choices=$("${cmd[@]}" "${options[@]}" 2>&1 >/dev/tty) || { printf "  %bCancel was selected, exiting installer%b\\n" "${COL_LIGHT_RED}" "${COL_NC}"; rm "${adlistFile}" ;exit 1; }
-    # For each choice available,
+    # create empty adlist file if no list was selected
+    : > "${adlistFile}"
+    # For each choice available
     for choice in ${choices}
     do
         appendToListsFile "${choice}"
     done
+    touch "${adlistFile}"
     chmod 644 "${adlistFile}"
 }
 
@@ -1231,9 +1239,6 @@ appendToListsFile() {
     case $1 in
         StevenBlack  )  echo "https://raw.githubusercontent.com/StevenBlack/hosts/master/hosts" >> "${adlistFile}";;
         MalwareDom   )  echo "https://mirror1.malwaredomains.com/files/justdomains" >> "${adlistFile}";;
-        Cameleon     )  echo "https://sysctl.org/cameleon/hosts" >> "${adlistFile}";;
-        DisconTrack  )  echo "https://s3.amazonaws.com/lists.disconnect.me/simple_tracking.txt" >> "${adlistFile}";;
-        DisconAd     )  echo "https://s3.amazonaws.com/lists.disconnect.me/simple_ad.txt" >> "${adlistFile}";;
     esac
 }
 
@@ -1247,10 +1252,8 @@ installDefaultBlocklists() {
     fi
     appendToListsFile StevenBlack
     appendToListsFile MalwareDom
-    appendToListsFile Cameleon
     appendToListsFile DisconTrack
     appendToListsFile DisconAd
-    appendToListsFile HostsFile
 }
 
 # Check if /etc/dnsmasq.conf is from pi-hole.  If so replace with an original and install new in .d directory
@@ -1427,8 +1430,8 @@ installConfigs() {
             sed -i 's/^\(server\.error-handler-404\s*=\s*\).*$/\1"pihole\/custom\.php"/' /etc/lighttpd/lighttpd.conf
         fi
         # Make the directories if they do not exist and set the owners
-        mkdir -p /var/run/lighttpd
+        mkdir -p /run/lighttpd
-        chown ${LIGHTTPD_USER}:${LIGHTTPD_GROUP} /var/run/lighttpd
+        chown ${LIGHTTPD_USER}:${LIGHTTPD_GROUP} /run/lighttpd
         mkdir -p /var/cache/lighttpd/compress
         chown ${LIGHTTPD_USER}:${LIGHTTPD_GROUP} /var/cache/lighttpd/compress
         mkdir -p /var/cache/lighttpd/uploads
@@ -1768,61 +1771,53 @@ create_pihole_user() {
     printf "  %b %s..." "${INFO}" "${str}"
     # If the user pihole exists,
     if id -u pihole &> /dev/null; then
-        # just show a success
+        # if group exists
-        printf "%b  %b %s\\n" "${OVER}" "${TICK}" "${str}"
+        if getent group pihole > /dev/null 2>&1; then
+            # just show a success
+            printf "%b  %b %s\\n" "${OVER}" "${TICK}" "${str}"
+        else
+            local str="Checking for group 'pihole'"
+            printf "  %b %s..." "${INFO}" "${str}"
+            local str="Creating group 'pihole'"
+            # if group can be created
+            if groupadd pihole; then
+                printf "%b  %b %s\\n" "${OVER}" "${TICK}" "${str}"
+                local str="Adding user 'pihole' to group 'pihole'"
+                printf "  %b %s..." "${INFO}" "${str}"
+                # if pihole user can be added to group pihole
+                if usermod -g pihole pihole; then
+                    printf "%b  %b %s\\n" "${OVER}" "${TICK}" "${str}"
+                else
+                    printf "%b  %b %s\\n" "${OVER}" "${CROSS}" "${str}"
+                fi
+            else
+                printf "%b  %b %s\\n" "${OVER}" "${CROSS}" "${str}"
+            fi
+        fi
     # Otherwise,
     else
         printf "%b  %b %s" "${OVER}" "${CROSS}" "${str}"
         local str="Creating user 'pihole'"
         printf "%b  %b %s..." "${OVER}" "${INFO}" "${str}"
         # create her with the useradd command
-        if useradd -r -s /usr/sbin/nologin pihole; then
+        if getent group pihole > /dev/null 2>&1; then
-          printf "%b  %b %s\\n" "${OVER}" "${TICK}" "${str}"
+            # add primary group pihole as it already exists
+            if useradd -r --no-user-group -g pihole -s /usr/sbin/nologin pihole; then
+                printf "%b  %b %s\\n" "${OVER}" "${TICK}" "${str}"
+            else
+                printf "%b  %b %s\\n" "${OVER}" "${CROSS}" "${str}"
+            fi
         else
-          printf "%b  %b %s\\n" "${OVER}" "${CROSS}" "${str}"
+            # add user pihole with default group settings
+            if useradd -r -s /usr/sbin/nologin pihole; then
+                printf "%b  %b %s\\n" "${OVER}" "${TICK}" "${str}"
+            else
+                printf "%b  %b %s\\n" "${OVER}" "${CROSS}" "${str}"
+            fi
         fi
     fi
 }
 
-# Allow HTTP and DNS traffic
-configureFirewall() {
-    printf "\\n"
-    # If a firewall is running,
-    if firewall-cmd --state &> /dev/null; then
-        # ask if the user wants to install Pi-hole's default firewall rules
-        whiptail --title "Firewall in use" --yesno "We have detected a running firewall\\n\\nPi-hole currently requires HTTP and DNS port access.\\n\\n\\n\\nInstall Pi-hole default firewall rules?" "${r}" "${c}" || \
-        { printf "  %b Not installing firewall rulesets.\\n" "${INFO}"; return 0; }
-        printf "  %b Configuring FirewallD for httpd and pihole-FTL\\n" "${TICK}"
-        # Allow HTTP and DNS traffic
-        firewall-cmd --permanent --add-service=http --add-service=dns
-        # Reload the firewall to apply these changes
-        firewall-cmd --reload
-        return 0
-        # Check for proper kernel modules to prevent failure
-    elif modinfo ip_tables &> /dev/null && is_command iptables ; then
-    # If chain Policy is not ACCEPT or last Rule is not ACCEPT
-    # then check and insert our Rules above the DROP/REJECT Rule.
-        if iptables -S INPUT | head -n1 | grep -qv '^-P.*ACCEPT$' || iptables -S INPUT | tail -n1 | grep -qv '^-\(A\|P\).*ACCEPT$'; then
-            whiptail --title "Firewall in use" --yesno "We have detected a running firewall\\n\\nPi-hole currently requires HTTP and DNS port access.\\n\\n\\n\\nInstall Pi-hole default firewall rules?" "${r}" "${c}" || \
-            { printf "  %b Not installing firewall rulesets.\\n" "${INFO}"; return 0; }
-            printf "  %b Installing new IPTables firewall rulesets\\n" "${TICK}"
-            # Check chain first, otherwise a new rule will duplicate old ones
-            iptables -C INPUT -p tcp -m tcp --dport 80 -j ACCEPT &> /dev/null || iptables -I INPUT 1 -p tcp -m tcp --dport 80 -j ACCEPT
-            iptables -C INPUT -p tcp -m tcp --dport 53 -j ACCEPT &> /dev/null || iptables -I INPUT 1 -p tcp -m tcp --dport 53 -j ACCEPT
-            iptables -C INPUT -p udp -m udp --dport 53 -j ACCEPT &> /dev/null || iptables -I INPUT 1 -p udp -m udp --dport 53 -j ACCEPT
-            iptables -C INPUT -p tcp -m tcp --dport 4711:4720 -i lo -j ACCEPT &> /dev/null || iptables -I INPUT 1 -p tcp -m tcp --dport 4711:4720 -i lo -j ACCEPT
-            return 0
-        fi
-    # Otherwise,
-    else
-        # no firewall is running
-        printf "  %b No active firewall detected.. skipping firewall configuration\\n" "${INFO}"
-        # so just exit
-        return 0
-    fi
-    printf "  %b Skipping firewall configuration\\n" "${INFO}"
-}
-
 #
 finalExports() {
     # If the Web interface is not set to be installed,
@@ -1931,8 +1926,6 @@ installPihole() {
             # Repair permissions if /var/www/html is not world readable
             chmod a+rx /var/www
             chmod a+rx /var/www/html
-            # Give pihole access to the Web server group
-            usermod -a -G ${LIGHTTPD_GROUP} pihole
             # Give lighttpd access to the pihole group so the web interface can
             # manage the gravity.db database
             usermod -a -G pihole ${LIGHTTPD_USER}
@@ -1973,10 +1966,6 @@ installPihole() {
     # Check if dnsmasq is present. If so, disable it and back up any possible
     # config file
     disable_dnsmasq
-    # Configure the firewall
-    if [[ "${useUpdateVars}" == false ]]; then
-        configureFirewall
-    fi
 
     # install a man page entry for pihole
     install_manpage
@@ -1990,9 +1979,9 @@ checkSelinux() {
     local DEFAULT_SELINUX
     local CURRENT_SELINUX
     local SELINUX_ENFORCING=0
-    # Check if a SELinux configuration file exists
+    # Check for SELinux configuration file and getenforce command
-    if [[ -f /etc/selinux/config ]]; then
+    if [[ -f /etc/selinux/config ]] && command -v getenforce &> /dev/null; then
-        # If a SELinux configuration file was found, check the default SELinux mode.
+        # Check the default SELinux mode
         DEFAULT_SELINUX=$(awk -F= '/^SELINUX=/ {print $2}' /etc/selinux/config)
         case "${DEFAULT_SELINUX,,}" in
             enforcing)
@@ -2357,11 +2346,11 @@ get_binary_name() {
             if [[ -f "/.dockerenv" ]]; then
                 printf "%b  %b Detected ARM architecture in docker\\n" "${OVER}" "${TICK}"
                 # set the binary to be used
-                binary="pihole-FTL-armel-native"
+                l_binary="pihole-FTL-armel-native"
             else
                 printf "%b  %b Detected ARM architecture\\n" "${OVER}" "${TICK}"
                 # set the binary to be used
-                binary="pihole-FTL-arm-linux-gnueabi"
+                l_binary="pihole-FTL-arm-linux-gnueabi"
             fi
         fi
     elif [[ "${machine}" == "x86_64" ]]; then
@@ -2583,9 +2572,6 @@ main() {
         verifyFreeDiskSpace
     fi
 
-    # Update package cache
-    update_package_cache || exit 1
-
     # Notify user of package availability
     notify_package_updates_available
 

automated install/uninstall.sh

@@ -188,9 +188,17 @@ removeNoPurge() {
             echo -e "  ${CROSS} Unable to remove 'pihole' user"
         fi
     fi
+    # If the pihole group exists, then remove
+    if getent group "pihole" &> /dev/null; then
+        if ${SUDO} groupdel pihole 2> /dev/null; then
+            echo -e "  ${TICK} Removed 'pihole' group"
+        else
+            echo -e "  ${CROSS} Unable to remove 'pihole' group"
+        fi
+    fi
 
     echo -e "\\n   We're sorry to see you go, but thanks for checking out Pi-hole!
-       If you need help, reach out to us on Github, Discourse, Reddit or Twitter
+       If you need help, reach out to us on GitHub, Discourse, Reddit or Twitter
        Reinstall at any time: ${COL_WHITE}curl -sSL https://install.pi-hole.net | bash${COL_NC}
 
       ${COL_LIGHT_RED}Please reset the DNS on your router/clients to restore internet connectivity

block hulu ads/lighttpd.conf

@@ -1,43 +0,0 @@
-# Pi-hole: A black hole for Internet advertisements
-# (c) 2015, 2016 by Jacob Salmela
-# Network-wide ad blocking via your Raspberry Pi
-# http://pi-hole.net
-# Lighttpd config file for Pi-hole
-#
-# Pi-hole is free software: you can redistribute it and/or modify
-# it under the terms of the GNU General Public License as published by
-# the Free Software Foundation, either version 2 of the License, or
-# (at your option) any later version.
-
-server.modules = (
-	"mod_access",
-	"mod_alias",
-	"mod_compress",
- 	"mod_redirect",
-   "mod_rewrite"
-)
-
-server.document-root        = "/var/www"
-server.upload-dirs          = ( "/var/cache/lighttpd/uploads" )
-server.errorlog             = "/var/log/lighttpd/error.log"
-server.pid-file             = "/var/run/lighttpd.pid"
-server.username             = "www-data"
-server.groupname            = "www-data"
-server.port                 = 80
-
-
-index-file.names            = ( "index.php", "index.html", "index.lighttpd.html" )
-url.access-deny             = ( "~", ".inc" )
-static-file.exclude-extensions = ( ".php", ".pl", ".fcgi" )
-
-compress.cache-dir          = "/var/cache/lighttpd/compress/"
-compress.filetype           = ( "application/javascript", "text/css", "text/html", "text/plain" )
-
-# default listening port for IPv6 falls back to the IPv4 port
-include_shell "/usr/share/lighttpd/use-ipv6.pl " + server.port
-include_shell "/usr/share/lighttpd/create-mime.assign.pl"
-include_shell "/usr/share/lighttpd/include-conf-enabled.pl"
-
-$HTTP["host"] =~ "ads.hulu.com|ads-v-darwin.hulu.com|ads-e-darwin.hulu.com" {
-	url.redirect = ( ".*" => "http://192.168.1.101:8200/MediaItems/19.mov")
-}

block hulu ads/minidlna.conf

@@ -1,17 +0,0 @@
-# Pi-hole: A black hole for Internet advertisements
-# (c) 2015, 2016 by Jacob Salmela
-# Network-wide ad blocking via your Raspberry Pi
-# http://pi-hole.net
-# MiniDLNA config file for Pi-hole
-#
-# Pi-hole is free software: you can redistribute it and/or modify
-# it under the terms of the GNU General Public License as published by
-# the Free Software Foundation, either version 2 of the License, or
-# (at your option) any later version.
-
-media_dir=V,/var/lib/minidlna/videos/
-port=8200
-friendly_name=pihole
-serial=12345678
-model_number=1
-inotify=yes

gravity.sh

@@ -376,7 +376,7 @@ gravity_DownloadBlocklists() {
     echo -e "  ${INFO} Target: ${url}"
     local regex
     # Check for characters NOT allowed in URLs
-    regex="[^a-zA-Z0-9:/?&%=~._-]"
+    regex="[^a-zA-Z0-9:/?&%=~._()-;]"
     if [[ "${url}" =~ ${regex} ]]; then
         echo -e "  ${CROSS} Invalid Target"
     else
@@ -573,12 +573,14 @@ gravity_ParseFileIntoDomains() {
     # It also helps with debugging so each stage of the script can be researched more in depth
     # 1) Remove carriage returns
     # 2) Convert all characters to lowercase
-    # 3) Remove lines containing "#" or "/"
+    # 3) Remove comments (text starting with "#", include possible spaces before the hash sign)
-    # 4) Remove leading tabs, spaces, etc.
+    # 4) Remove lines containing "/"
-    # 5) Delete lines not matching domain names
+    # 5) Remove leading tabs, spaces, etc.
+    # 6) Delete lines not matching domain names
     < "${source}" tr -d '\r' | \
     tr '[:upper:]' '[:lower:]' | \
-    sed -r '/(\/|#).*$/d' | \
+    sed 's/\s*#.*//g' | \
+    sed -r '/(\/).*$/d' | \
     sed -r 's/^.*\s+//g' | \
     sed -r '/([^\.]+\.)+[^\.]{2,}/!d' >  "${destination}"
     chmod 644 "${destination}"
@@ -638,7 +640,7 @@ gravity_Table_Count() {
   if [[ "${table}" == "vw_gravity" ]]; then
     local unique
     unique="$(sqlite3 "${gravityDBfile}" "SELECT COUNT(DISTINCT domain) FROM ${table};")"
-    echo -e "  ${INFO} Number of ${str}: ${num} (${unique} unique domains)"
+    echo -e "  ${INFO} Number of ${str}: ${num} (${COL_BOLD}${unique} unique domains${COL_NC})"
     sqlite3 "${gravityDBfile}" "INSERT OR REPLACE INTO info (property,value) VALUES ('gravity_count',${unique});"
   else
     echo -e "  ${INFO} Number of ${str}: ${num}"

manpages/pihole.8

@@ -1,4 +1,4 @@
-.TH "Pi-hole" "8" "Pi-hole" "Pi-hole" "May 2018"
+.TH "Pi-hole" "8" "Pi-hole" "Pi-hole" "April 2020"
 .SH "NAME"
 
 Pi-hole : A black-hole for internet advertisements
@@ -11,8 +11,6 @@ Pi-hole : A black-hole for internet advertisements
 .br
 \fBpihole -a\fR (\fB-c|-f|-k\fR)
 .br
-\fBpihole -a\fR [\fB-r\fR hostrecord]
-.br
 \fBpihole -a -e\fR email
 .br
 \fBpihole -a -i\fR interface
@@ -43,7 +41,7 @@ pihole -g\fR
 .br
 pihole status
 .br
-pihole restartdns\fR
+pihole restartdns\fR [options]
 .br
 \fBpihole\fR (\fBenable\fR|\fBdisable\fR [time])
 .br
@@ -134,9 +132,6 @@ Available commands and options:
       -f, fahrenheit    Set Fahrenheit as preferred temperature unit
 .br
       -k, kelvin        Set Kelvin as preferred temperature unit
-.br
-      -r, hostrecord    Add a name to the DNS associated to an
-                        IPv4/IPv6 address
 .br
       -e, email         Set an administrative contact address for the
                         Block Page
@@ -229,7 +224,7 @@ Available commands and options:
 .br
       -l, --latest      Return the latest version
 .br
-      --hash            Return the Github hash from your local
+      --hash            Return the GitHub hash from your local
                         repositories
 .br
 
@@ -260,14 +255,21 @@ Available commands and options:
       #m                Disable Pi-hole functionality for # minute(s)
 .br
 
-\fBrestartdns\fR
+\fBrestartdns\fR [options]
 .br
-    Restart Pi-hole subsystems
+    Full restart Pi-hole subsystems
+.br
+
+    (restart options):
+.br
+      reload            Updates the lists and flushes DNS cache
+.br
+      reload-lists      Updates the lists WITHOUT flushing the DNS cache
 .br
 
 \fBcheckout\fR [repo] [branch]
 .br
-    Switch Pi-hole subsystems to a different Github branch
+    Switch Pi-hole subsystems to a different GitHub branch
 .br
 
     (repo options):

pihole

@@ -317,7 +317,7 @@ piholeCheckoutFunc() {
   if [[ "$2" == "-h" ]] || [[ "$2" == "--help" ]]; then
     echo "Usage: pihole checkout [repo] [branch]
 Example: 'pihole checkout master' or 'pihole checkout core dev'
-Switch Pi-hole subsystems to a different Github branch
+Switch Pi-hole subsystems to a different GitHub branch
 
 Repositories:
   core [branch]       Change the branch of Pi-hole's core subsystem
@@ -413,8 +413,10 @@ Options:
   enable              Enable Pi-hole subsystems
   disable             Disable Pi-hole subsystems
                         Add '-h' for more info on disable usage
-  restartdns          Restart Pi-hole subsystems
+  restartdns          Full restart Pi-hole subsystems
-  checkout            Switch Pi-hole subsystems to a different Github branch
+                        Add 'reload' to update the lists and flush the cache without restarting the DNS server
+                        Add 'reload-lists' to only update the lists WITHOUT flushing the cache or restarting the DNS server
+  checkout            Switch Pi-hole subsystems to a different GitHub branch
                         Add '-h' for more info on checkout usage
   arpflush            Flush information stored in Pi-hole's network tables";
   exit 0

test/test_automated_install.py

@@ -92,168 +92,6 @@ def test_setupVars_saved_to_file(Pihole):
         assert "{}={}".format(k, v) in output
 
 
-def test_configureFirewall_firewalld_running_no_errors(Pihole):
-    '''
-    confirms firewalld rules are applied when firewallD is running
-    '''
-    # firewallD returns 'running' as status
-    mock_command('firewall-cmd', {'*': ('running', 0)}, Pihole)
-    # Whiptail dialog returns Ok for user prompt
-    mock_command('whiptail', {'*': ('', 0)}, Pihole)
-    configureFirewall = Pihole.run('''
-    source /opt/pihole/basic-install.sh
-    configureFirewall
-    ''')
-    expected_stdout = 'Configuring FirewallD for httpd and pihole-FTL'
-    assert expected_stdout in configureFirewall.stdout
-    firewall_calls = Pihole.run('cat /var/log/firewall-cmd').stdout
-    assert 'firewall-cmd --state' in firewall_calls
-    assert ('firewall-cmd '
-            '--permanent '
-            '--add-service=http '
-            '--add-service=dns') in firewall_calls
-    assert 'firewall-cmd --reload' in firewall_calls
-
-
-def test_configureFirewall_firewalld_disabled_no_errors(Pihole):
-    '''
-    confirms firewalld rules are not applied when firewallD is not running
-    '''
-    # firewallD returns non-running status
-    mock_command('firewall-cmd', {'*': ('not running', '1')}, Pihole)
-    configureFirewall = Pihole.run('''
-    source /opt/pihole/basic-install.sh
-    configureFirewall
-    ''')
-    expected_stdout = ('No active firewall detected.. '
-                       'skipping firewall configuration')
-    assert expected_stdout in configureFirewall.stdout
-
-
-def test_configureFirewall_firewalld_enabled_declined_no_errors(Pihole):
-    '''
-    confirms firewalld rules are not applied when firewallD is running, user
-    declines ruleset
-    '''
-    # firewallD returns running status
-    mock_command('firewall-cmd', {'*': ('running', 0)}, Pihole)
-    # Whiptail dialog returns Cancel for user prompt
-    mock_command('whiptail', {'*': ('', 1)}, Pihole)
-    configureFirewall = Pihole.run('''
-    source /opt/pihole/basic-install.sh
-    configureFirewall
-    ''')
-    expected_stdout = 'Not installing firewall rulesets.'
-    assert expected_stdout in configureFirewall.stdout
-
-
-def test_configureFirewall_no_firewall(Pihole):
-    ''' confirms firewall skipped no daemon is running '''
-    configureFirewall = Pihole.run('''
-    source /opt/pihole/basic-install.sh
-    configureFirewall
-    ''')
-    expected_stdout = 'No active firewall detected'
-    assert expected_stdout in configureFirewall.stdout
-
-
-def test_configureFirewall_IPTables_enabled_declined_no_errors(Pihole):
-    '''
-    confirms IPTables rules are not applied when IPTables is running, user
-    declines ruleset
-    '''
-    # iptables command exists
-    mock_command('iptables', {'*': ('', '0')}, Pihole)
-    # modinfo returns always true (ip_tables module check)
-    mock_command('modinfo', {'*': ('', '0')}, Pihole)
-    # Whiptail dialog returns Cancel for user prompt
-    mock_command('whiptail', {'*': ('', '1')}, Pihole)
-    configureFirewall = Pihole.run('''
-    source /opt/pihole/basic-install.sh
-    configureFirewall
-    ''')
-    expected_stdout = 'Not installing firewall rulesets.'
-    assert expected_stdout in configureFirewall.stdout
-
-
-def test_configureFirewall_IPTables_enabled_rules_exist_no_errors(Pihole):
-    '''
-    confirms IPTables rules are not applied when IPTables is running and rules
-    exist
-    '''
-    # iptables command exists and returns 0 on calls
-    # (should return 0 on iptables -C)
-    mock_command('iptables', {'-S': ('-P INPUT DENY', '0')}, Pihole)
-    # modinfo returns always true (ip_tables module check)
-    mock_command('modinfo', {'*': ('', '0')}, Pihole)
-    # Whiptail dialog returns Cancel for user prompt
-    mock_command('whiptail', {'*': ('', '0')}, Pihole)
-    configureFirewall = Pihole.run('''
-    source /opt/pihole/basic-install.sh
-    configureFirewall
-    ''')
-    expected_stdout = 'Installing new IPTables firewall rulesets'
-    assert expected_stdout in configureFirewall.stdout
-    firewall_calls = Pihole.run('cat /var/log/iptables').stdout
-    # General call type occurrences
-    assert len(re.findall(r'iptables -S', firewall_calls)) == 1
-    assert len(re.findall(r'iptables -C', firewall_calls)) == 4
-    assert len(re.findall(r'iptables -I', firewall_calls)) == 0
-
-    # Specific port call occurrences
-    assert len(re.findall(r'tcp --dport 80', firewall_calls)) == 1
-    assert len(re.findall(r'tcp --dport 53', firewall_calls)) == 1
-    assert len(re.findall(r'udp --dport 53', firewall_calls)) == 1
-    assert len(re.findall(r'tcp --dport 4711:4720', firewall_calls)) == 1
-
-
-def test_configureFirewall_IPTables_enabled_not_exist_no_errors(Pihole):
-    '''
-    confirms IPTables rules are applied when IPTables is running and rules do
-    not exist
-    '''
-    # iptables command and returns 0 on calls (should return 1 on iptables -C)
-    mock_command(
-        'iptables',
-        {
-            '-S': (
-                '-P INPUT DENY',
-                '0'
-            ),
-            '-C': (
-                '',
-                1
-            ),
-            '-I': (
-                '',
-                0
-            )
-        },
-        Pihole
-    )
-    # modinfo returns always true (ip_tables module check)
-    mock_command('modinfo', {'*': ('', '0')}, Pihole)
-    # Whiptail dialog returns Cancel for user prompt
-    mock_command('whiptail', {'*': ('', '0')}, Pihole)
-    configureFirewall = Pihole.run('''
-    source /opt/pihole/basic-install.sh
-    configureFirewall
-    ''')
-    expected_stdout = 'Installing new IPTables firewall rulesets'
-    assert expected_stdout in configureFirewall.stdout
-    firewall_calls = Pihole.run('cat /var/log/iptables').stdout
-    # General call type occurrences
-    assert len(re.findall(r'iptables -S', firewall_calls)) == 1
-    assert len(re.findall(r'iptables -C', firewall_calls)) == 4
-    assert len(re.findall(r'iptables -I', firewall_calls)) == 4
-
-    # Specific port call occurrences
-    assert len(re.findall(r'tcp --dport 80', firewall_calls)) == 2
-    assert len(re.findall(r'tcp --dport 53', firewall_calls)) == 2
-    assert len(re.findall(r'udp --dport 53', firewall_calls)) == 2
-    assert len(re.findall(r'tcp --dport 4711:4720', firewall_calls)) == 2
-
-
 def test_selinux_not_detected(Pihole):
     '''
     confirms installer continues when SELinux configuration file does not exist