diff --git a/.github/workflows/test.yml b/.github/workflows/test.yml index 8e19e550..d19df5ab 100644 --- a/.github/workflows/test.yml +++ b/.github/workflows/test.yml @@ -10,7 +10,7 @@ jobs: runs-on: ubuntu-latest strategy: matrix: - distro: [debian_9, debian_10, ubuntu_16, ubuntu_18, ubuntu_20, centos_7, centos_8, fedora_31, fedora_32] + distro: [debian_9, debian_10, ubuntu_16, ubuntu_18, ubuntu_20, centos_7, centos_8, fedora_32, fedora_33] env: DISTRO: ${{matrix.distro}} steps: diff --git a/README.md b/README.md index 6d7e5b5e..57dee16d 100644 --- a/README.md +++ b/README.md @@ -1,5 +1,7 @@ +# +

Pi-hole @@ -9,8 +11,6 @@

-# - The Pi-hole® is a [DNS sinkhole](https://en.wikipedia.org/wiki/DNS_Sinkhole) that protects your devices from unwanted content, without installing any client-side software. - **Easy-to-install**: our versatile installer walks you through the process, and takes less than ten minutes @@ -26,8 +26,6 @@ The Pi-hole® is a [DNS sinkhole](https://en.wikipedia.org/wiki/DNS_Sinkhole) th ----- -Master [![Build Status](https://travis-ci.com/pi-hole/pi-hole.svg?branch=master)](https://travis-ci.com/pi-hole/pi-hole) Development [![Build Status](https://travis-ci.com/pi-hole/pi-hole.svg?branch=development)](https://travis-ci.com/pi-hole/pi-hole) - ## One-Step Automated Install Those who want to get started quickly and conveniently may install Pi-hole using the following command: @@ -71,16 +69,18 @@ Make no mistake: **your support is absolutely vital to help keep us innovating!* ### [Donations](https://pi-hole.net/donate) -Sending a donation using our Sponsor Button is **extremely helpful** in offsetting a portion of our monthly expenses: +Sending a donation using our Sponsor Button is **extremely helpful** in offsetting a portion of our monthly expenses and rewarding our dedicated development team: ### Alternative support If you'd rather not donate (_which is okay!_), there are other ways you can help support us: -- [Patreon](https://patreon.com/pihole) _Become a patron for rewards_ +- [GitHub Sponsors](https://github.com/sponsors/pi-hole/) +- [Patreon](https://patreon.com/pihole) +- [Hetzner Cloud](https://hetzner.cloud/?ref=7aceisRX3AzA) _affiliate link_ - [Digital Ocean](https://www.digitalocean.com/?refcode=344d234950e1) _affiliate link_ - [Stickermule](https://www.stickermule.com/unlock?ref_id=9127301701&utm_medium=link&utm_source=invite) _earn a $10 credit after your first purchase_ -- [Amazon](http://www.amazon.com/exec/obidos/redirect-home/pihole09-20) _affiliate link_ +- [Amazon US](http://www.amazon.com/exec/obidos/redirect-home/pihole09-20) _affiliate link_ - Spreading the word about our software, and how you have benefited from it ### Contributing via GitHub @@ -95,7 +95,9 @@ You'll find that the [install script](https://github.com/pi-hole/pi-hole/blob/ma ## Getting in touch with us -While we are primarily reachable on our [Discourse User Forum](https://discourse.pi-hole.net/), we can also be found on a variety of social media outlets. **Please be sure to check the FAQ's** before starting a new discussion, as we do not have the spare time to reply to every request for assistance. +While we are primarily reachable on our [Discourse User Forum](https://discourse.pi-hole.net/), we can also be found on a variety of social media outlets. + +**Please be sure to check the FAQ's** before starting a new discussion. Many user questions already have answers and can be solved without any additional assistance. - [Frequently Asked Questions](https://discourse.pi-hole.net/c/faqs) - [Feature Requests](https://discourse.pi-hole.net/c/feature-requests?order=votes) @@ -106,12 +108,27 @@ While we are primarily reachable on our [Discourse User Forum](https://discourse ## Breakdown of Features +### [Faster-than-light Engine](https://github.com/pi-hole/ftl) + +[FTLDNS](https://github.com/pi-hole/ftl) is a lightweight, purpose-built daemon used to provide statistics needed for the Web Interface, and its API can be easily integrated into your own projects. As the name implies, FTLDNS does this all *very quickly*! + +Some of the statistics you can integrate include: + +- Total number of domains being blocked +- Total number of DNS queries today +- Total number of ads blocked today +- Percentage of ads blocked +- Unique domains +- Queries forwarded (to your chosen upstream DNS server) +- Queries cached +- Unique clients + +The API can be accessed via [`telnet`](https://github.com/pi-hole/FTL), the Web (`admin/api.php`) and Command Line (`pihole -c -j`). You can find out [more details over here](https://discourse.pi-hole.net/t/pi-hole-api/1863). + ### The Command Line Interface The [pihole](https://docs.pi-hole.net/core/pihole-command/) command has all the functionality necessary to be able to fully administer the Pi-hole, without the need of the Web Interface. It's fast, user-friendly, and auditable by anyone with an understanding of `bash`. -![Pi-hole Blacklist Demo](https://pi-hole.github.io/graphics/Screenshots/blacklist-cli.gif) - Some notable features include: - [Whitelisting, Blacklisting and Regex](https://docs.pi-hole.net/core/pihole-command/#whitelisting-blacklisting-and-regex) @@ -128,8 +145,6 @@ You can read our [Core Feature Breakdown](https://docs.pi-hole.net/core/pihole-c This [optional dashboard](https://github.com/pi-hole/AdminLTE) allows you to view stats, change settings, and configure your Pi-hole. It's the power of the Command Line Interface, with none of the learning curve! -![Pi-hole Dashboard](https://pi-hole.github.io/graphics/Screenshots/pihole-dashboard.png) - Some notable features include: - Mobile friendly interface @@ -145,21 +160,3 @@ There are several ways to [access the dashboard](https://discourse.pi-hole.net/t 1. `http://pi.hole/admin/` (when using Pi-hole as your DNS server) 2. `http:///admin/` -3. `http://pi.hole/` (when using Pi-hole as your DNS server) - -## Faster-than-light Engine - -FTLDNS is a lightweight, purpose-built daemon used to provide statistics needed for the Web Interface, and its API can be easily integrated into your own projects. As the name implies, FTLDNS does this all *very quickly*! - -Some of the statistics you can integrate include: - -- Total number of domains being blocked -- Total number of DNS queries today -- Total number of ads blocked today -- Percentage of ads blocked -- Unique domains -- Queries forwarded (to your chosen upstream DNS server) -- Queries cached -- Unique clients - -The API can be accessed via [`telnet`](https://github.com/pi-hole/FTL), the Web (`admin/api.php`) and Command Line (`pihole -c -j`). You can find out [more details over here](https://discourse.pi-hole.net/t/pi-hole-api/1863). diff --git a/advanced/Scripts/chronometer.sh b/advanced/Scripts/chronometer.sh index 3126ed25..4f9ea59a 100755 --- a/advanced/Scripts/chronometer.sh +++ b/advanced/Scripts/chronometer.sh @@ -559,7 +559,7 @@ Calculates stats and displays to an LCD Options: -j, --json Output stats as JSON formatted string -r, --refresh Set update frequency (in seconds) - -e, --exit Output stats and exit witout refreshing + -e, --exit Output stats and exit without refreshing -h, --help Display this help text" fi diff --git a/advanced/Scripts/database_migration/gravity-db.sh b/advanced/Scripts/database_migration/gravity-db.sh index 282ea07b..22f241dd 100644 --- a/advanced/Scripts/database_migration/gravity-db.sh +++ b/advanced/Scripts/database_migration/gravity-db.sh @@ -111,9 +111,15 @@ upgrade_gravityDB(){ version=12 fi if [[ "$version" == "12" ]]; then - # Add column date_updated to alist table + # Add column date_updated to adlist table echo -e " ${INFO} Upgrading gravity database from version 12 to 13" sqlite3 "${database}" < "${scriptPath}/12_to_13.sql" version=13 fi + if [[ "$version" == "13" ]]; then + # Add columns number and status to adlist table + echo -e " ${INFO} Upgrading gravity database from version 13 to 14" + sqlite3 "${database}" < "${scriptPath}/13_to_14.sql" + version=14 + fi } diff --git a/advanced/Scripts/database_migration/gravity/13_to_14.sql b/advanced/Scripts/database_migration/gravity/13_to_14.sql new file mode 100644 index 00000000..fa230865 --- /dev/null +++ b/advanced/Scripts/database_migration/gravity/13_to_14.sql @@ -0,0 +1,13 @@ +.timeout 30000 + +PRAGMA FOREIGN_KEYS=OFF; + +BEGIN TRANSACTION; + +ALTER TABLE adlist ADD COLUMN number INTEGER NOT NULL DEFAULT 0; +ALTER TABLE adlist ADD COLUMN invalid_domains INTEGER NOT NULL DEFAULT 0; +ALTER TABLE adlist ADD COLUMN status INTEGER NOT NULL DEFAULT 0; + +UPDATE info SET value = 14 WHERE property = 'version'; + +COMMIT; \ No newline at end of file diff --git a/advanced/Scripts/list.sh b/advanced/Scripts/list.sh index 16bb2001..e213b014 100755 --- a/advanced/Scripts/list.sh +++ b/advanced/Scripts/list.sh @@ -1,4 +1,6 @@ #!/usr/bin/env bash +# shellcheck disable=SC1090 + # Pi-hole: A black hole for Internet advertisements # (c) 2017 Pi-hole, LLC (https://pi-hole.net) # Network-wide ad blocking via your own hardware. @@ -9,9 +11,17 @@ # Please see LICENSE file for your rights under this license. # Globals -basename=pihole -piholeDir=/etc/"${basename}" -gravityDBfile="${piholeDir}/gravity.db" +piholeDir="/etc/pihole" +GRAVITYDB="${piholeDir}/gravity.db" +# Source pihole-FTL from install script +pihole_FTL="${piholeDir}/pihole-FTL.conf" +if [[ -f "${pihole_FTL}" ]]; then + source "${pihole_FTL}" +fi + +# Set this only after sourcing pihole-FTL.conf as the gravity database path may +# have changed +gravityDBfile="${GRAVITYDB}" reload=false addmode=true @@ -112,7 +122,7 @@ ProcessDomainList() { for dom in "${domList[@]}"; do # Format domain into regex filter if requested if [[ "${wildcard}" == true ]]; then - dom="(^|\\.)${dom//\./\\.}$" + dom="(\\.|^)${dom//\./\\.}$" fi # Logic: If addmode then add to desired list and remove from the other; diff --git a/advanced/Scripts/piholeARPTable.sh b/advanced/Scripts/piholeARPTable.sh index b6b552c9..66d05bf9 100755 --- a/advanced/Scripts/piholeARPTable.sh +++ b/advanced/Scripts/piholeARPTable.sh @@ -38,7 +38,7 @@ flushARP(){ # Truncate network_addresses table in pihole-FTL.db # This needs to be done before we can truncate the network table due to - # foreign key contraints + # foreign key constraints if ! output=$(sqlite3 "${DBFILE}" "DELETE FROM network_addresses" 2>&1); then echo -e "${OVER} ${CROSS} Failed to truncate network_addresses table" echo " Database location: ${DBFILE}" diff --git a/advanced/Scripts/piholeDebug.sh b/advanced/Scripts/piholeDebug.sh index d8d86563..13a886f1 100755 --- a/advanced/Scripts/piholeDebug.sh +++ b/advanced/Scripts/piholeDebug.sh @@ -244,7 +244,7 @@ initialize_debug() { log_write "${INFO} $(date "+%Y-%m-%d:%H:%M:%S") debug log has been initialized." } -# This is a function for visually displaying the curent test that is being run. +# This is a function for visually displaying the current test that is being run. # Accepts one variable: the name of what is being diagnosed # Colors do not show in the dasboard, but the icons do: [i], [✓], and [✗] echo_current_diagnostic() { @@ -379,7 +379,7 @@ get_program_version() { # Create a local variable so this function can be safely reused local program_version echo_current_diagnostic "${program_name} version" - # Evalutate the program we are checking, if it is any of the ones below, show the version + # Evaluate the program we are checking, if it is any of the ones below, show the version case "${program_name}" in "lighttpd") program_version="$(${program_name} -v 2> /dev/null | head -n1 | cut -d '/' -f2 | cut -d ' ' -f1)" ;; @@ -641,7 +641,7 @@ detect_ip_addresses() { # First argument should be a 4 or a 6 local protocol=${1} # Use ip to show the addresses for the chosen protocol - # Store the values in an arry so they can be looped through + # Store the values in an array so they can be looped through # Get the lines that are in the file(s) and store them in an array for parsing later mapfile -t ip_addr_list < <(ip -"${protocol}" addr show dev "${PIHOLE_INTERFACE}" | awk -F ' ' '{ for(i=1;i<=NF;i++) if ($i ~ '/^inet/') print $(i+1) }') @@ -695,7 +695,7 @@ ping_gateway() { # Check if we are using IPv4 or IPv6 # Find the default gateway using IPv4 or IPv6 local gateway - gateway="$(ip -"${protocol}" route | grep default | cut -d ' ' -f 3)" + gateway="$(ip -"${protocol}" route | grep default | grep "${PIHOLE_INTERFACE}" | cut -d ' ' -f 3)" # If the gateway variable has a value (meaning a gateway was found), if [[ -n "${gateway}" ]]; then @@ -823,7 +823,7 @@ check_x_headers() { # Do it for the dashboard as well, as the header is different than above local dashboard dashboard=$(curl -Is localhost/admin/ | awk '/X-Pi-hole/' | tr -d '\r') - # Store what the X-Header shoud be in variables for comparison later + # Store what the X-Header should be in variables for comparison later local block_page_working block_page_working="X-Pi-hole: A black hole for Internet advertisements." local dashboard_working @@ -842,12 +842,12 @@ check_x_headers() { log_write "${COL_RED}${full_curl_output_block_page}${COL_NC}" fi - # Same logic applies to the dashbord as above, if the X-Header matches what a working system shoud have, + # Same logic applies to the dashboard as above, if the X-Header matches what a working system should have, if [[ $dashboard == "$dashboard_working" ]]; then # then we can show a success log_write "$TICK Web interface X-Header: ${COL_GREEN}${dashboard}${COL_NC}" else - # Othewise, it's a failure since the X-Headers either don't exist or have been modified in some way + # Otherwise, it's a failure since the X-Headers either don't exist or have been modified in some way log_write "$CROSS Web interface X-Header: ${COL_RED}X-Header does not match or could not be retrieved.${COL_NC}" log_write "${COL_RED}${full_curl_output_dashboard}${COL_NC}" fi @@ -877,7 +877,7 @@ dig_at() { local pihole_address="${IP}" local remote_address="2001:4860:4860::8888" local record_type="AAAA" - # Othwerwise, it should be 4 + # Otherwise, it should be 4 else # so use the IPv4 values local local_address="127.0.0.1" @@ -911,7 +911,7 @@ dig_at() { # show a success log_write "${TICK} ${random_url} ${COL_GREEN}is ${pihole_dig}${COL_NC} via ${COL_CYAN}Pi-hole${COL_NC} (${pihole_address})" else - # Othewise, show a failure + # Otherwise, show a failure log_write "${CROSS} ${COL_RED}Failed to resolve${COL_NC} ${random_url} via ${COL_RED}Pi-hole${COL_NC} (${pihole_address})" fi @@ -1044,7 +1044,7 @@ parse_file() { } check_name_resolution() { - # Check name resoltion from localhost, Pi-hole's IP, and Google's name severs + # Check name resolution from localhost, Pi-hole's IP, and Google's name severs # using the function we created earlier dig_at 4 "${IPV4_ADDRESS%/*}" # If IPv6 enabled, diff --git a/advanced/Scripts/query.sh b/advanced/Scripts/query.sh index fe9b8ebf..26b4508e 100755 --- a/advanced/Scripts/query.sh +++ b/advanced/Scripts/query.sh @@ -1,5 +1,6 @@ #!/usr/bin/env bash # shellcheck disable=SC1090 + # Pi-hole: A black hole for Internet advertisements # (c) 2018 Pi-hole, LLC (https://pi-hole.net) # Network-wide ad blocking via your own hardware. @@ -11,12 +12,21 @@ # Globals piholeDir="/etc/pihole" -gravityDBfile="${piholeDir}/gravity.db" +GRAVITYDB="${piholeDir}/gravity.db" options="$*" all="" exact="" blockpage="" matchType="match" +# Source pihole-FTL from install script +pihole_FTL="${piholeDir}/pihole-FTL.conf" +if [[ -f "${pihole_FTL}" ]]; then + source "${pihole_FTL}" +fi + +# Set this only after sourcing pihole-FTL.conf as the gravity database path may +# have changed +gravityDBfile="${GRAVITYDB}" colfile="/opt/pihole/COL_TABLE" source "${colfile}" diff --git a/advanced/Scripts/webpage.sh b/advanced/Scripts/webpage.sh index da2afb0f..8ef4d940 100755 --- a/advanced/Scripts/webpage.sh +++ b/advanced/Scripts/webpage.sh @@ -238,18 +238,18 @@ trust-anchor=.,20326,8,2,E06D44B80B8F1D39A95C0B0D7C65D08458E880409BBC68345710423 # 168.192.in-addr.arpa to 192.168.0.0/16 # 192.in-addr.arpa to 192.0.0.0/8 if [[ "${CONDITIONAL_FORWARDING_REVERSE}" == *"in-addr.arpa" ]];then - arrRev=("${CONDITIONAL_FORWARDING_REVERSE//./ }") - case ${#arrRev[@]} in + arrRev=("${CONDITIONAL_FORWARDING_REVERSE//./ }") + case ${#arrRev[@]} in 6 ) REV_SERVER_CIDR="${arrRev[3]}.${arrRev[2]}.${arrRev[1]}.${arrRev[0]}/32";; 5 ) REV_SERVER_CIDR="${arrRev[2]}.${arrRev[1]}.${arrRev[0]}.0/24";; 4 ) REV_SERVER_CIDR="${arrRev[1]}.${arrRev[0]}.0.0/16";; - 3 ) REV_SERVER_CIDR="${arrRev[0]}.0.0.0/8";; + 3 ) REV_SERVER_CIDR="${arrRev[0]}.0.0.0/8";; esac else # Set REV_SERVER_CIDR to whatever value it was set to REV_SERVER_CIDR="${CONDITIONAL_FORWARDING_REVERSE}" fi - + # If REV_SERVER_CIDR is not converted by the above, then use the REV_SERVER_TARGET variable to derive it if [ -z "${REV_SERVER_CIDR}" ]; then # Convert existing input to /24 subnet (preserves legacy behavior) @@ -564,7 +564,13 @@ AddDHCPStaticAddress() { RemoveDHCPStaticAddress() { mac="${args[2]}" - sed -i "/dhcp-host=${mac}.*/d" "${dhcpstaticconfig}" + if [[ "$mac" =~ ^([0-9A-Fa-f]{2}[:-]){5}([0-9A-Fa-f]{2})$ ]]; then + sed -i "/dhcp-host=${mac}.*/d" "${dhcpstaticconfig}" + else + echo " ${CROSS} Invalid Mac Passed!" + exit 1 + fi + } SetAdminEmail() { @@ -636,8 +642,11 @@ Interfaces: Teleporter() { local datetimestamp + local host datetimestamp=$(date "+%Y-%m-%d_%H-%M-%S") - php /var/www/html/admin/scripts/pi-hole/php/teleporter.php > "pi-hole-teleporter_${datetimestamp}.tar.gz" + host=$(hostname) + host="${host//./_}" + php /var/www/html/admin/scripts/pi-hole/php/teleporter.php > "pi-hole-${host:-noname}-teleporter_${datetimestamp}.tar.gz" } checkDomain() @@ -705,7 +714,13 @@ RemoveCustomDNSAddress() { ip="${args[2]}" host="${args[3]}" - sed -i "/${ip} ${host}/d" "${dnscustomfile}" + + if valid_ip "${ip}" || valid_ip6 "${ip}" ; then + sed -i "/${ip} ${host}/d" "${dnscustomfile}" + else + echo -e " ${CROSS} Invalid IP has been passed" + exit 1 + fi # Restart dnsmasq to update removed custom DNS entries RestartDNS @@ -716,6 +731,7 @@ AddCustomCNAMERecord() { domain="${args[2]}" target="${args[3]}" + echo "cname=${domain},${target}" >> "${dnscustomcnamefile}" # Restart dnsmasq to load new custom CNAME records @@ -727,7 +743,20 @@ RemoveCustomCNAMERecord() { domain="${args[2]}" target="${args[3]}" - sed -i "/cname=${domain},${target}/d" "${dnscustomcnamefile}" + + validDomain="$(checkDomain "${domain}")" + if [[ -n "${validDomain}" ]]; then + validTarget="$(checkDomain "${target}")" + if [[ -n "${validDomain}" ]]; then + sed -i "/cname=${validDomain},${validTarget}/d" "${dnscustomcnamefile}" + else + echo " ${CROSS} Invalid Target Passed!" + exit 1 + fi + else + echo " ${CROSS} Invalid Domain passed!" + exit 1 + fi # Restart dnsmasq to update removed custom CNAME records RestartDNS diff --git a/advanced/Templates/gravity.db.sql b/advanced/Templates/gravity.db.sql index 26b030c8..5d7bafa9 100644 --- a/advanced/Templates/gravity.db.sql +++ b/advanced/Templates/gravity.db.sql @@ -32,7 +32,10 @@ CREATE TABLE adlist date_added INTEGER NOT NULL DEFAULT (cast(strftime('%s', 'now') as int)), date_modified INTEGER NOT NULL DEFAULT (cast(strftime('%s', 'now') as int)), comment TEXT, - date_updated INTEGER + date_updated INTEGER, + number INTEGER NOT NULL DEFAULT 0, + invalid_domains INTEGER NOT NULL DEFAULT 0, + status INTEGER NOT NULL DEFAULT 0 ); CREATE TABLE adlist_by_group @@ -54,7 +57,7 @@ CREATE TABLE info value TEXT NOT NULL ); -INSERT INTO "info" VALUES('version','13'); +INSERT INTO "info" VALUES('version','14'); CREATE TABLE domain_audit ( diff --git a/advanced/blockingpage.css b/advanced/blockingpage.css index 9f82a27e..0cc7a65c 100644 --- a/advanced/blockingpage.css +++ b/advanced/blockingpage.css @@ -145,7 +145,17 @@ body { } /* User is greeted with a splash page when browsing to Pi-hole IP address */ -#splashpage { background: #222; color: rgba(255, 255, 255, 0.7); text-align: center; } +#splashpage { + background: #222; + color: rgba(255, 255, 255, 0.7); + text-align: center; + width: 100%; + height: 100%; + display: flex; + align-items: center; + justify-content: center; +} + #splashpage img { margin: 5px; width: 256px; } #splashpage b { color: inherit; } @@ -196,6 +206,26 @@ header #bpAlt label { display: block; } +html, body { + height: 100%; +} + +#pihole_card { + width: 400px; + height: auto; + max-width: 400px; +} + + #pihole_card p, #pihole_card a { + font-size: 13pt; + text-align: center; + } + +#pihole_logo_splash { + height: auto; + width: 100%; +} + /* Click anywhere else on screen to hide #bpAbout */ #bpAboutToggle:checked { display: block; @@ -382,12 +412,44 @@ footer { /* Responsive Content */ @media only screen and (max-width: 500px) { - h1 a { font-size: 1.8rem; min-width: 170px; } - footer span::before { content: "Generated "; } - footer span { display: block; } + h1 a { + font-size: 1.8rem; + min-width: 170px; + } + + footer span::before { + content: "Generated "; + } + + footer span { + display: block; + } } @media only screen and (min-width: 1251px) { - #bpWrapper, footer { border-radius: 0 0 5px 5px; } - #bpAbout { border-right-width: 1px; } + #bpWrapper, footer { + border-radius: 0 0 5px 5px; + } + + #bpAbout { + border-right-width: 1px; + } +} + +@media only screen and (max-width: 400px) { + #pihole_card { + width: 100%; + height: auto; + } + + #pihole_card p, #pihole_card a { + font-size: 100%; + } +} + +@media only screen and (max-width: 256px) { + #pihole_logo_splash { + width: 90% !important; + height: auto; + } } diff --git a/advanced/index.php b/advanced/index.php index ca8c35e4..8f33cf5d 100644 --- a/advanced/index.php +++ b/advanced/index.php @@ -58,14 +58,15 @@ if ($serverName === "pi.hole" // When directly browsing via IP or authorized hostname // Render splash/landing page based off presence of $landPage file // Unset variables so as to not be included in $landPage or $splashPage - unset($serverName, $svPasswd, $svEmail, $authorizedHosts, $validExtTypes, $currentUrlExt, $viewPort); + unset($svPasswd, $svEmail, $authorizedHosts, $validExtTypes, $currentUrlExt); // If $landPage file is present if (is_file(getcwd()."/$landPage")) { + unset($serverName, $viewPort); // unset extra variables not to be included in $landpage include $landPage; exit(); } // If $landPage file was not present, Set Splash Page output - $splashPage = " + $splashPage = << @@ -76,13 +77,14 @@ if ($serverName === "pi.hole" - Pi-hole logo -
-

Pi-hole: Your black hole for Internet advertisements

- Did you mean to go to the admin panel? +
+ Pi-hole logo id="pihole_logo_splash" /> + <p>Pi-<strong>hole</strong>: Your black hole for Internet advertisements</p> + <a href=Did you mean to go to the admin panel? +
- "; +EOT; exit($splashPage); } elseif ($currentUrlExt === "js") { // Serve Pi-hole JavaScript for blocked domains requesting JS diff --git a/advanced/lighttpd.conf.debian b/advanced/lighttpd.conf.debian index 6020dfd8..cd6d7737 100644 --- a/advanced/lighttpd.conf.debian +++ b/advanced/lighttpd.conf.debian @@ -81,6 +81,10 @@ mimetype.assign = ( ".woff2" => "font/woff2" ) +# Add user chosen options held in external file +# This uses include_shell instead of an include wildcard for compatibility +include_shell "cat external.conf 2>/dev/null" + # default listening port for IPv6 falls back to the IPv4 port include_shell "/usr/share/lighttpd/use-ipv6.pl " + server.port @@ -109,7 +113,3 @@ $HTTP["url"] =~ "^/admin/\.(.*)" { # Default expire header expire.url = ( "" => "access plus 0 seconds" ) - -# Add user chosen options held in external file -# This uses include_shell instead of an include wildcard for compatibility -include_shell "cat external.conf 2>/dev/null" diff --git a/advanced/lighttpd.conf.fedora b/advanced/lighttpd.conf.fedora index 8e3bddc4..64428617 100644 --- a/advanced/lighttpd.conf.fedora +++ b/advanced/lighttpd.conf.fedora @@ -82,6 +82,10 @@ mimetype.assign = ( ".woff2" => "font/woff2" ) +# Add user chosen options held in external file +# This uses include_shell instead of an include wildcard for compatibility +include_shell "cat external.conf 2>/dev/null" + # default listening port for IPv6 falls back to the IPv4 port #include_shell "/usr/share/lighttpd/use-ipv6.pl " + server.port #include_shell "/usr/share/lighttpd/create-mime.assign.pl" @@ -117,7 +121,3 @@ $HTTP["url"] =~ "^/admin/\.(.*)" { # Default expire header expire.url = ( "" => "access plus 0 seconds" ) - -# Add user chosen options held in external file -# This uses include_shell instead of an include wildcard for compatibility -include_shell "cat external.conf 2>/dev/null" diff --git a/automated install/basic-install.sh b/automated install/basic-install.sh index 4ab5c0e2..07683204 100755 --- a/automated install/basic-install.sh +++ b/automated install/basic-install.sh @@ -21,8 +21,8 @@ # instead of continuing the installation with something broken set -e -# Set PATH to a usual default to assure that all basic commands are available. -# When using "su" an uncomplete PATH could be passed: https://github.com/pi-hole/pi-hole/issues/3209 +# Append common folders to the PATH to ensure that all basic commands are available. +# When using "su" an incomplete PATH could be passed: https://github.com/pi-hole/pi-hole/issues/3209 export PATH+=':/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin' ######## VARIABLES ######### @@ -59,12 +59,17 @@ coltable=/opt/pihole/COL_TABLE # Root of the web server webroot="/var/www/html" -# We store several other directories and + +# We clone (or update) two git repositories during the install. This helps to make sure that we always have the latest versions of the relevant files. +# AdminLTE is used to set up the Web admin interface. +# Pi-hole contains various setup scripts and files which are critical to the installation. +# Search for "PI_HOLE_LOCAL_REPO" in this file to see all such scripts. +# Two notable scripts are gravity.sh (used to generate the HOSTS file) and advanced/Scripts/webpage.sh (used to install the Web admin interface) webInterfaceGitUrl="https://github.com/pi-hole/AdminLTE.git" webInterfaceDir="${webroot}/admin" piholeGitUrl="https://github.com/pi-hole/pi-hole.git" PI_HOLE_LOCAL_REPO="/etc/.pihole" -# These are the names of pi-holes files, stored in an array +# List of pihole scripts, stored in an array PI_HOLE_FILES=(chronometer list piholeDebug piholeLogFlush setupLCD update version gravity uninstall webpage) # This directory is where the Pi-hole scripts will be installed PI_HOLE_INSTALL_DIR="/opt/pihole" @@ -76,11 +81,10 @@ if [ -z "$useUpdateVars" ]; then fi adlistFile="/etc/pihole/adlists.list" -# Pi-hole needs an IP address; to begin, these variables are empty since we don't know what the IP is until -# this script can run +# Pi-hole needs an IP address; to begin, these variables are empty since we don't know what the IP is until this script can run IPV4_ADDRESS=${IPV4_ADDRESS} IPV6_ADDRESS=${IPV6_ADDRESS} -# By default, query logging is enabled and the dashboard is set to be installed +# Give settings their default values. These may be changed by prompts later in the script. QUERY_LOGGING=true INSTALL_WEB_INTERFACE=true PRIVACY_LEVEL=0 @@ -98,9 +102,8 @@ if [ -t 0 ] ; then else screen_size="24 80" fi -# Set rows variable to contain first number +# Determine terminal rows and columns by parsing screen_size printf -v rows '%d' "${screen_size%% *}" -# Set columns variable to contain second number printf -v columns '%d' "${screen_size##* }" # Divide by two so the dialogs take up half of the screen, which looks nice. @@ -171,9 +174,8 @@ show_ascii_berry() { } is_command() { - # Checks for existence of string passed in as only function argument. - # Exit value of 0 when exists, 1 if not exists. Value is the result - # of the `command` shell built-in call. + # Checks to see if the given command (passed as a string argument) exists on the system. + # The function returns 0 (success) if the command exists, and 1 if it doesn't. local check_command="$1" command -v "${check_command}" >/dev/null 2>&1 @@ -190,15 +192,15 @@ os_check() { detected_version=$(grep VERSION_ID /etc/os-release | cut -d '=' -f2 | tr -d '"') cmdResult="$(dig +short -t txt ${remote_os_domain} @ns1.pi-hole.net 2>&1; echo $?)" - #Get the return code of the previous command (last line) + # Gets the return code of the previous command (last line) digReturnCode="${cmdResult##*$'\n'}" if [ ! "${digReturnCode}" == "0" ]; then valid_response=false else - # Dig returned 0 code, so get the actual response, and loop through it to determine if the detected variables above are valid + # Dig returned 0 (success), so get the actual response, and loop through it to determine if the detected variables above are valid response="${cmdResult%%$'\n'*}" - # If the value of ${result} is a single 0, then this is the return code, not the response. Response is blank + # If the value of ${response} is a single 0, then this is the return code, not an actual response. if [ "${response}" == 0 ]; then valid_response=false fi @@ -209,6 +211,7 @@ os_check() { distro_part="${distro_and_versions%%=*}" versions_part="${distro_and_versions##*=}" + # If the distro part is a (case-insensistive) substring of the computer OS if [[ "${detected_os^^}" =~ ${distro_part^^} ]]; then valid_os=true IFS="," read -r -a supportedVer <<<"${versions_part}" @@ -281,9 +284,9 @@ if is_command apt-get ; then PKG_MANAGER="apt-get" # A variable to store the command used to update the package cache UPDATE_PKG_CACHE="${PKG_MANAGER} update" - # An array for something... + # The command we will use to actually install packages PKG_INSTALL=("${PKG_MANAGER}" -qq --no-install-recommends install) - # grep -c will return 1 retVal on 0 matches, block this throwing the set -e with an OR TRUE + # grep -c will return 1 if there are no matches. This is an acceptable condition, so we OR TRUE to prevent set -e exiting the script. PKG_COUNT="${PKG_MANAGER} -s -o Debug::NoLocking=true upgrade | grep -c ^Inst || true" # Some distros vary slightly so these fixes for dependencies may apply # on Ubuntu 18.04.1 LTS we need to add the universe repository to gain access to dhcpcd5 @@ -291,7 +294,7 @@ if is_command apt-get ; then if awk 'BEGIN{a=1;b=0}/bionic main/{a=0}/bionic.*universe/{b=1}END{exit a + b}' ${APT_SOURCES}; then if ! whiptail --defaultno --title "Dependencies Require Update to Allowed Repositories" --yesno "Would you like to enable 'universe' repository?\\n\\nThis repository is required by the following packages:\\n\\n- dhcpcd5" "${r}" "${c}"; then printf " %b Aborting installation: Dependencies could not be installed.\\n" "${CROSS}" - exit 1 # exit the installer + exit 1 else printf " %b Enabling universe package repository for Ubuntu Bionic\\n" "${INFO}" cp -p ${APT_SOURCES} ${APT_SOURCES}.backup # Backup current repo list @@ -323,7 +326,8 @@ if is_command apt-get ; then phpInsNewer=true fi fi - # Check if installed php is v 7.0, or newer to determine packages to install + # Several other packages depend on the version of PHP. If PHP is not installed, or an insufficient version, + # those packages should fall back to the default (latest?) if [[ "$phpInsNewer" != true ]]; then # Prefer the php metapackage if it's there if apt-cache show php > /dev/null 2>&1; then @@ -337,7 +341,8 @@ if is_command apt-get ; then exit 1 fi else - # Newer php is installed, its common, cgi & sqlite counterparts are deps + # Else, PHP is already installed at a version beyond v7.0, so the additional packages + # should match version with the current PHP version. phpVer="php$phpInsMajor.$phpInsMinor" fi # We also need the correct version for `php-sqlite` (which differs across distros) @@ -349,13 +354,12 @@ if is_command apt-get ; then printf " %b Aborting installation: No SQLite PHP module was found in APT repository.\\n" "${CROSS}" exit 1 fi - # Since our install script is so large, we need several other programs to successfully get a machine provisioned - # These programs are stored in an array so they can be looped through later + # Packages required to run this install script (stored as an array) INSTALLER_DEPS=(dhcpcd5 git "${iproute_pkg}" whiptail dnsutils) - # Pi-hole itself has several dependencies that also need to be installed + # Packages required to run Pi-hole (stored as an array) PIHOLE_DEPS=(cron curl iputils-ping lsof netcat psmisc sudo unzip wget idn2 sqlite3 libcap2-bin dns-root-data libcap2) - # The Web dashboard has some that also need to be installed - # It's useful to separate the two since our repos are also setup as "Core" code and "Web" code + # Packages required for the Web admin interface (stored as an array) + # It's useful to separate this from Pi-hole, since the two repos are also setup separately PIHOLE_WEB_DEPS=(lighttpd "${phpVer}-common" "${phpVer}-cgi" "${phpVer}-${phpSqlite}" "${phpVer}-xml" "${phpVer}-json" "${phpVer}-intl") # The Web server user, LIGHTTPD_USER="www-data" @@ -364,20 +368,19 @@ if is_command apt-get ; then # and config file LIGHTTPD_CFG="lighttpd.conf.debian" - # A function to check... + # This function waits for dpkg to unlock, which signals that the previous apt-get command has finished. test_dpkg_lock() { - # An iterator used for counting loop iterations i=0 # fuser is a program to show which processes use the named files, sockets, or filesystems - # So while the command is true - while fuser /var/lib/dpkg/lock >/dev/null 2>&1 ; do - # Wait half a second + # So while the lock is held, + while fuser /var/lib/dpkg/lock >/dev/null 2>&1 + do + # we wait half a second, sleep 0.5 - # and increase the iterator + # increase the iterator, ((i=i+1)) done - # Always return success, since we only return if there is no - # lock (anymore) + # and then report success once dpkg is unlocked. return 0 } @@ -390,6 +393,7 @@ elif is_command rpm ; then PKG_MANAGER="yum" fi + # These variable names match the ones in the Debian family. See above for an explanation of what they are for. PKG_INSTALL=("${PKG_MANAGER}" install -y) PKG_COUNT="${PKG_MANAGER} check-update | egrep '(.i686|.x86|.noarch|.arm|.src)' | wc -l" INSTALLER_DEPS=(git iproute newt procps-ng which chkconfig bind-utils) @@ -476,7 +480,7 @@ elif is_command rpm ; then # Warn user of unsupported version of Fedora or CentOS if ! whiptail --defaultno --title "Unsupported RPM based distribution" --yesno "Would you like to continue installation on an unsupported RPM based distribution?\\n\\nPlease ensure the following packages have been installed manually:\\n\\n- lighttpd\\n- lighttpd-fastcgi\\n- PHP version 7+" "${r}" "${c}"; then printf " %b Aborting installation due to unsupported RPM based distribution\\n" "${CROSS}" - exit # exit the installer + exit else printf " %b Continuing installation with unsupported RPM based distribution\\n" "${INFO}" fi @@ -540,8 +544,9 @@ make_repo() { # Check current branch. If it is master, then reset to the latest available tag. # In case extra commits have been added after tagging/release (i.e in case of metadata updates/README.MD tweaks) curBranch=$(git rev-parse --abbrev-ref HEAD) - if [[ "${curBranch}" == "master" ]]; then #If we're calling make_repo() then it should always be master, we may not need to check. - git reset --hard "$(git describe --abbrev=0 --tags)" || return $? + if [[ "${curBranch}" == "master" ]]; then + # If we're calling make_repo() then it should always be master, we may not need to check. + git reset --hard "$(git describe --abbrev=0 --tags)" || return $? fi # Show a colored message showing it's status printf "%b %b %s\\n" "${OVER}" "${TICK}" "${str}" @@ -589,7 +594,7 @@ update_repo() { return 0 } -# A function that combines the functions previously made +# A function that combines the previous git functions to update or clone a repo getGitFiles() { # Setup named variables for the git repos # We need the directory @@ -613,9 +618,8 @@ getGitFiles() { # Attempt to make the repository, showing an error on failure make_repo "${directory}" "${remoteRepo}" || { printf "\\n %bError: Could not update local repository. Contact support.%b\\n" "${COL_LIGHT_RED}" "${COL_NC}"; exit 1; } fi - # echo a blank line echo "" - # and return success? + # Success via one of the two branches, as the commands would exit if they failed. return 0 } @@ -637,7 +641,7 @@ resetRepo() { printf "%b %b %s\\n" "${OVER}" "${TICK}" "${str}" # Return to where we came from popd &> /dev/null || return 1 - # Returning success anyway? + # Function succeeded, as "git reset" would have triggered a return earlier if it failed return 0 } @@ -689,7 +693,7 @@ welcomeDialogs() { In the next section, you can choose to use your current network settings (DHCP) or to manually edit them." "${r}" "${c}" } -# A function that let's the user pick an interface to use with Pi-hole +# A function that lets the user pick an interface to use with Pi-hole chooseInterface() { # Turn the available interfaces into an array so it can be used with a whiptail dialog local interfacesArray=() @@ -713,7 +717,7 @@ chooseInterface() { else # While reading through the available interfaces while read -r line; do - # use a variable to set the option as OFF to begin with + # Use a variable to set the option as OFF to begin with mode="OFF" # If it's the first loop, if [[ "${firstLoop}" -eq 1 ]]; then @@ -747,9 +751,9 @@ chooseInterface() { testIPv6() { # first will contain fda2 (ULA) printf -v first "%s" "${1%%:*}" - # value1 will contain 253 which is the decimal value corresponding to 0xfd + # value1 will contain 253 which is the decimal value corresponding to 0xFD value1=$(( (0x$first)/256 )) - # will contain 162 which is the decimal value corresponding to 0xa2 + # value2 will contain 162 which is the decimal value corresponding to 0xA2 value2=$(( (0x$first)%256 )) # the ULA test is testing for fc00::/7 according to RFC 4193 if (( (value1&254)==252 )); then @@ -779,8 +783,9 @@ useIPv6dialog() { result=$(testIPv6 "$i") # If it's a ULA address, use it and store it as a global variable [[ "${result}" == "ULA" ]] && ULA_ADDRESS="${i%/*}" - # If it's a GUA address, we can still use it si store it as a global variable + # If it's a GUA address, use it and store it as a global variable [[ "${result}" == "GUA" ]] && GUA_ADDRESS="${i%/*}" + # Else if it's a Link-local address, we cannot use it, so just continue done # Determine which address to be used: Prefer ULA over GUA or don't use any if none found @@ -858,7 +863,6 @@ use4andor6() { fi } -# getStaticIPv4Settings() { # Local, named variables local ipSettingsCorrect @@ -905,9 +909,9 @@ It is also possible to use a DHCP reservation, but if you are going to do that, fi } -# configure networking via dhcpcd +# Configure networking via dhcpcd setDHCPCD() { - # check if the IP is already in the file + # Check if the IP is already in the file if grep -q "${IPV4_ADDRESS}" /etc/dhcpcd.conf; then printf " %b Static IP already configured\\n" "${INFO}" # If it's not, @@ -925,8 +929,8 @@ setDHCPCD() { fi } -# configure networking ifcfg-xxxx file found at /etc/sysconfig/network-scripts/ -# this function requires the full path of an ifcfg file passed as an argument +# Configure networking ifcfg-xxxx file found at /etc/sysconfig/network-scripts/ +# This function requires the full path of an ifcfg file passed as an argument setIFCFG() { # Local, named variables local IFCFG_FILE @@ -934,12 +938,11 @@ setIFCFG() { local CIDR IFCFG_FILE=$1 printf -v IPADDR "%s" "${IPV4_ADDRESS%%/*}" - # check if the desired IP is already set + # Check if the desired IP is already set if grep -Eq "${IPADDR}(\\b|\\/)" "${IFCFG_FILE}"; then printf " %b Static IP already configured\\n" "${INFO}" - # Otherwise, else - # Put the IP in variables without the CIDR notation + # Otherwise, put the IP in variables without the CIDR notation printf -v CIDR "%s" "${IPV4_ADDRESS##*/}" # Backup existing interface configuration: cp -p "${IFCFG_FILE}" "${IFCFG_FILE}".pihole.orig @@ -981,25 +984,29 @@ setStaticIPv4() { return 0 fi fi - # For the Debian family, if dhcpcd.conf exists, + # For the Debian family, if dhcpcd.conf exists then we can just configure using DHCPD. if [[ -f "/etc/dhcpcd.conf" ]]; then - # configure networking via dhcpcd setDHCPCD return 0 fi - # If a DHCPCD config file was not found, check for an ifcfg config file based on interface name + # If a DHCPCD config file was not found, check for an ifcfg config file based on the interface name if [[ -f "/etc/sysconfig/network-scripts/ifcfg-${PIHOLE_INTERFACE}" ]];then - # If it exists, + # If it exists, then we can configure using IFCFG IFCFG_FILE=/etc/sysconfig/network-scripts/ifcfg-${PIHOLE_INTERFACE} setIFCFG "${IFCFG_FILE}" return 0 fi - # if an ifcfg config does not exists for the interface name, try the connection name via network manager + # If an ifcfg config does not exists for the interface name, search for one based on the connection name via network manager if is_command nmcli && nmcli general status &> /dev/null; then CONNECTION_NAME=$(nmcli dev show "${PIHOLE_INTERFACE}" | grep 'GENERAL.CONNECTION' | cut -d: -f2 | sed 's/^System//' | xargs | tr ' ' '_') - if [[ -f "/etc/sysconfig/network-scripts/ifcfg-${CONNECTION_NAME}" ]];then + IFCFG_FILE=/etc/sysconfig/network-scripts/ifcfg-${CONNECTION_NAME} + if [[ -f "${IFCFG_FILE}" ]];then # If it exists, - IFCFG_FILE=/etc/sysconfig/network-scripts/ifcfg-${CONNECTION_NAME} + setIFCFG "${IFCFG_FILE}" + return 0 + else + printf " %b Warning: sysconfig network script not found. Creating ${IFCFG_FILE}\\n" "${INFO}" + touch "${IFCFG_FILE}" setIFCFG "${IFCFG_FILE}" return 0 fi @@ -1015,17 +1022,19 @@ valid_ip() { local ip=${1} local stat=1 - # One IPv4 element is 8bit: 0 - 256 + # Regex matching one IPv4 component, i.e. an integer from 0 to 255. + # See https://tools.ietf.org/html/rfc1340 local ipv4elem="(25[0-5]|2[0-4][0-9]|1[0-9][0-9]|[1-9][0-9]?|0)"; - # optional port number starting '#' with range of 1-65536 - local portelem="(#([1-9]|[1-8][0-9]|9[0-9]|[1-8][0-9]{2}|9[0-8][0-9]|99[0-9]|[1-8][0-9]{3}|9[0-8][0-9]{2}|99[0-8][0-9]|999[0-9]|[1-5][0-9]{4}|6[0-4][0-9]{3}|65[0-4][0-9]{2}|655[0-2][0-9]|6553[0-6]))?" - # build a full regex string from the above parts + # Regex matching an optional port beginning with : from 0 to 65535 + # See https://tools.ietf.org/html/rfc1340#page-33 + local portelem="(:(6553[0-5]|655[0-2][0-9]|65[0-4][0-9]{2}|6[0-4][0-9]{3}|[1-5][0-9]{4}|[1-9][0-9]{0,3}|0))?"; + # Build a full IPv4 regex from the above subexpressions local regex="^${ipv4elem}\.${ipv4elem}\.${ipv4elem}\.${ipv4elem}${portelem}$" + # Evaluate the regex, and return the result [[ $ip =~ ${regex} ]] stat=$? - # Return the exit code return "${stat}" } @@ -1033,19 +1042,19 @@ valid_ip6() { local ip=${1} local stat=1 - # One IPv6 element is 16bit: 0000 - FFFF + # Regex matching one IPv6 element, i.e. a hex value from 0000 to FFFF local ipv6elem="[0-9a-fA-F]{1,4}" - # CIDR for IPv6 is 1- 128 bit + # Regex matching an IPv6 CIDR, i.e. 1 to 128 local v6cidr="(\\/([1-9]|[1-9][0-9]|1[0-1][0-9]|12[0-8])){0,1}" - # optional port number starting '#' with range of 1-65536 + # Regex matching an optional port beginning with # matching optional port number starting '#' with range of 1-65536 local portelem="(#([1-9]|[1-8][0-9]|9[0-9]|[1-8][0-9]{2}|9[0-8][0-9]|99[0-9]|[1-8][0-9]{3}|9[0-8][0-9]{2}|99[0-8][0-9]|999[0-9]|[1-5][0-9]{4}|6[0-4][0-9]{3}|65[0-4][0-9]{2}|655[0-2][0-9]|6553[0-6]))?" - # build a full regex string from the above parts + # Build a full IPv6 regex from the above subexpressions local regex="^(((${ipv6elem}))*((:${ipv6elem}))*::((${ipv6elem}))*((:${ipv6elem}))*|((${ipv6elem}))((:${ipv6elem})){7})${v6cidr}${portelem}$" + # Evaluate the regex, and return the result [[ ${ip} =~ ${regex} ]] stat=$? - # Return the exit code return "${stat}" } @@ -1057,7 +1066,7 @@ setDNS() { # In an array, list the available upstream providers DNSChooseOptions=() local DNSServerCount=0 - # Save the old Internal Field Separator in a variable + # Save the old Internal Field Separator in a variable, OIFS=$IFS # and set the new one to newline IFS=$'\n' @@ -1078,23 +1087,22 @@ setDNS() { # In a whiptail dialog, show the options DNSchoices=$(whiptail --separate-output --menu "Select Upstream DNS Provider. To use your own, select Custom." "${r}" "${c}" 7 \ "${DNSChooseOptions[@]}" 2>&1 >/dev/tty) || \ - # exit if Cancel is selected + # Exit if the user selects "Cancel" { printf " %bCancel was selected, exiting installer%b\\n" "${COL_LIGHT_RED}" "${COL_NC}"; exit 1; } - # Depending on the user's choice, set the GLOBAl variables to the IP of the respective provider + # Depending on the user's choice, set the GLOBAL variables to the IP of the respective provider if [[ "${DNSchoices}" == "Custom" ]] then - # Until the DNS settings are selected, + # Loop until we have a valid DNS setting until [[ "${DNSSettingsCorrect}" = True ]]; do - # + # Signal value, to be used if the user inputs an invalid IP address strInvalid="Invalid" - # If the first if [[ ! "${PIHOLE_DNS_1}" ]]; then - # and second upstream servers do not exist if [[ ! "${PIHOLE_DNS_2}" ]]; then + # If the first and second upstream servers do not exist, do not prepopulate an IP address prePopulate="" - # Otherwise, else + # Otherwise, prepopulate the whiptail dialogue with the appropriate DNS value(s) prePopulate=", ${PIHOLE_DNS_2}" fi elif [[ "${PIHOLE_DNS_1}" ]] && [[ ! "${PIHOLE_DNS_2}" ]]; then @@ -1103,52 +1111,48 @@ setDNS() { prePopulate="${PIHOLE_DNS_1}, ${PIHOLE_DNS_2}" fi - # Dialog for the user to enter custom upstream servers + # Prompt the user to enter custom upstream servers piholeDNS=$(whiptail --backtitle "Specify Upstream DNS Provider(s)" --inputbox "Enter your desired upstream DNS provider(s), separated by a comma.\\n\\nFor example '8.8.8.8, 8.8.4.4'" "${r}" "${c}" "${prePopulate}" 3>&1 1>&2 2>&3) || \ { printf " %bCancel was selected, exiting installer%b\\n" "${COL_LIGHT_RED}" "${COL_NC}"; exit 1; } # Clean user input and replace whitespace with comma. piholeDNS=$(sed 's/[, \t]\+/,/g' <<< "${piholeDNS}") + # Separate the user input into the two DNS values (separated by a comma) printf -v PIHOLE_DNS_1 "%s" "${piholeDNS%%,*}" printf -v PIHOLE_DNS_2 "%s" "${piholeDNS##*,}" - # If the IP is valid, + # If the first DNS value is invalid or empty, this if statement will be true and we will set PIHOLE_DNS_1="Invalid" if ! valid_ip "${PIHOLE_DNS_1}" || [[ ! "${PIHOLE_DNS_1}" ]]; then - # store it in the variable so we can use it PIHOLE_DNS_1=${strInvalid} fi - # Do the same for the secondary server + # If the second DNS value is invalid or empty, this if statement will be true and we will set PIHOLE_DNS_2="Invalid" if ! valid_ip "${PIHOLE_DNS_2}" && [[ "${PIHOLE_DNS_2}" ]]; then PIHOLE_DNS_2=${strInvalid} fi # If either of the DNS servers are invalid, if [[ "${PIHOLE_DNS_1}" == "${strInvalid}" ]] || [[ "${PIHOLE_DNS_2}" == "${strInvalid}" ]]; then - # explain this to the user + # explain this to the user, whiptail --msgbox --backtitle "Invalid IP" --title "Invalid IP" "One or both entered IP addresses were invalid. Please try again.\\n\\n DNS Server 1: $PIHOLE_DNS_1\\n DNS Server 2: ${PIHOLE_DNS_2}" ${r} ${c} - # and set the variables back to nothing + # set the variables back to nothing, if [[ "${PIHOLE_DNS_1}" == "${strInvalid}" ]]; then PIHOLE_DNS_1="" fi if [[ "${PIHOLE_DNS_2}" == "${strInvalid}" ]]; then PIHOLE_DNS_2="" fi - # Since the settings will not work, stay in the loop + # and continue the loop. DNSSettingsCorrect=False - # Otherwise, else - # Show the settings + # Otherwise, show the DNS setting to the user, and break the loop if they confirm them. if (whiptail --backtitle "Specify Upstream DNS Provider(s)" --title "Upstream DNS Provider(s)" --yesno "Are these settings correct?\\n DNS Server 1: $PIHOLE_DNS_1\\n DNS Server 2: ${PIHOLE_DNS_2}" "${r}" "${c}"); then - # and break from the loop since the servers are valid DNSSettingsCorrect=True - # Otherwise, else - # If the settings are wrong, the loop continues DNSSettingsCorrect=False fi fi done else - # Save the old Internal Field Separator in a variable + # Save the old Internal Field Separator in a variable, OIFS=$IFS # and set the new one to newline IFS=$'\n' @@ -1187,16 +1191,16 @@ setLogging() { # Get the user's choice LogChoices=$("${LogToggleCommand[@]}" "${LogChooseOptions[@]}" 2>&1 >/dev/tty) || (printf " %bCancel was selected, exiting installer%b\\n" "${COL_LIGHT_RED}" "${COL_NC}" && exit 1) case ${LogChoices} in - # If it's on + # If it's on, "On (Recommended)") printf " %b Logging On.\\n" "${INFO}" - # Set the GLOBAL variable to true so we know what they selected + # set the GLOBAL variable setting to true QUERY_LOGGING=true ;; # Otherwise, it's off, Off) printf " %b Logging Off.\\n" "${INFO}" - # So set it to false + # set the GLOBAL variable setting to false QUERY_LOGGING=false ;; esac @@ -1252,7 +1256,7 @@ setAdminFlag() { ;; esac - # Request user to install web server, if it has not been deselected before (INSTALL_WEB_SERVER=true is default). + # If the user wants to install the Web admin interface (i.e. it has not been deselected above) if [[ "${INSTALL_WEB_SERVER}" == true ]]; then # Get list of required PHP modules, excluding base package (common) and handler (cgi) local i php_modules @@ -1291,18 +1295,18 @@ chooseBlocklists() { # In a variable, show the choices available; exit if Cancel is selected choices=$("${cmd[@]}" "${options[@]}" 2>&1 >/dev/tty) || { printf " %bCancel was selected, exiting installer%b\\n" "${COL_LIGHT_RED}" "${COL_NC}"; rm "${adlistFile}" ;exit 1; } - # For each choice available, + # Add all selected choices to the lists file for choice in ${choices} do appendToListsFile "${choice}" done + # Create an empty adList file with appropriate permissions. touch "${adlistFile}" chmod 644 "${adlistFile}" } # Accept a string parameter, it must be one of the default lists -# This function allow to not duplicate code in chooseBlocklists and -# in installDefaultBlocklists +# This function saves duplication between chooseBlocklists and installDefaultBlocklists appendToListsFile() { case $1 in StevenBlack ) echo "https://raw.githubusercontent.com/StevenBlack/hosts/master/hosts" >> "${adlistFile}";; @@ -1339,16 +1343,15 @@ version_check_dnsmasq() { grep -q "${dnsmasq_pihole_id_string2}" "${dnsmasq_conf}"; then printf " it is from a previous Pi-hole install.\\n" printf " %b Backing up dnsmasq.conf to dnsmasq.conf.orig..." "${INFO}" - # so backup the original file + # so backup the original file, mv -f "${dnsmasq_conf}" "${dnsmasq_conf_orig}" printf "%b %b Backing up dnsmasq.conf to dnsmasq.conf.orig...\\n" "${OVER}" "${TICK}" printf " %b Restoring default dnsmasq.conf..." "${INFO}" # and replace it with the default install -D -m 644 -T "${dnsmasq_original_config}" "${dnsmasq_conf}" printf "%b %b Restoring default dnsmasq.conf...\\n" "${OVER}" "${TICK}" - # Otherwise, else - # Don't to anything + # Otherwise, don't to anything printf " it is not a Pi-hole file, leaving alone!\\n" fi else @@ -1368,36 +1371,33 @@ version_check_dnsmasq() { install -D -m 644 -T "${dnsmasq_pihole_01_snippet}" "${dnsmasq_pihole_01_location}" printf "%b %b Copying 01-pihole.conf to /etc/dnsmasq.d/01-pihole.conf\\n" "${OVER}" "${TICK}" # Replace our placeholder values with the GLOBAL DNS variables that we populated earlier - # First, swap in the interface to listen on + # First, swap in the interface to listen on, sed -i "s/@INT@/$PIHOLE_INTERFACE/" "${dnsmasq_pihole_01_location}" if [[ "${PIHOLE_DNS_1}" != "" ]]; then - # Then swap in the primary DNS server + # then swap in the primary DNS server. sed -i "s/@DNS1@/$PIHOLE_DNS_1/" "${dnsmasq_pihole_01_location}" else - # + # Otherwise, remove the line which sets DNS1. sed -i '/^server=@DNS1@/d' "${dnsmasq_pihole_01_location}" fi + # Ditto if DNS2 is not empty if [[ "${PIHOLE_DNS_2}" != "" ]]; then - # Then swap in the primary DNS server sed -i "s/@DNS2@/$PIHOLE_DNS_2/" "${dnsmasq_pihole_01_location}" else - # sed -i '/^server=@DNS2@/d' "${dnsmasq_pihole_01_location}" fi # Set the cache size sed -i "s/@CACHE_SIZE@/$CACHE_SIZE/" ${dnsmasq_pihole_01_location} - # sed -i 's/^#conf-dir=\/etc\/dnsmasq.d$/conf-dir=\/etc\/dnsmasq.d/' "${dnsmasq_conf}" # If the user does not want to enable logging, if [[ "${QUERY_LOGGING}" == false ]] ; then - # Disable it by commenting out the directive in the DNS config file + # disable it by commenting out the directive in the DNS config file sed -i 's/^log-queries/#log-queries/' "${dnsmasq_pihole_01_location}" - # Otherwise, else - # enable it by uncommenting the directive in the DNS config file + # Otherwise, enable it by uncommenting the directive in the DNS config file sed -i 's/^#log-queries/log-queries/' "${dnsmasq_pihole_01_location}" fi } @@ -1407,14 +1407,13 @@ clean_existing() { # Local, named variables # ${1} Directory to clean local clean_directory="${1}" - # Make ${2} the new one? + # Pop the first argument, and shift all addresses down by one (i.e. ${2} becomes ${1}) shift - # ${2} Array of files to remove + # Then, we can access all arguments ($@) without including the directory to clean local old_files=( "$@" ) - # For each script found in the old files array + # Remove each script in the old_files array for script in "${old_files[@]}"; do - # Remove them rm -f "${clean_directory}/${script}.sh" done } @@ -1447,9 +1446,8 @@ installScripts() { install -Dm644 ./advanced/bash-completion/pihole /etc/bash_completion.d/pihole printf "%b %b %s\\n" "${OVER}" "${TICK}" "${str}" - # Otherwise, else - # Show an error and exit + # Otherwise, show an error and exit printf "%b %b %s\\n" "${OVER}" "${CROSS}" "${str}" printf "\\t\\t%bError: Local repo %s not found, exiting installer%b\\n" "${COL_LIGHT_RED}" "${PI_HOLE_LOCAL_REPO}" "${COL_NC}" return 1 @@ -1501,7 +1499,7 @@ installConfigs() { # Make sure the external.conf file exists, as lighttpd v1.4.50 crashes without it touch /etc/lighttpd/external.conf chmod 644 /etc/lighttpd/external.conf - # if there is a custom block page in the html/pihole directory, replace 404 handler in lighttpd config + # If there is a custom block page in the html/pihole directory, replace 404 handler in lighttpd config if [[ -f "${PI_HOLE_BLOCKPAGE_DIR}/custom.php" ]]; then sed -i 's/^\(server\.error-handler-404\s*=\s*\).*$/\1"pihole\/custom\.php"/' /etc/lighttpd/lighttpd.conf fi @@ -1575,9 +1573,8 @@ restart_service() { if is_command systemctl ; then # use that to restart the service systemctl restart "${1}" &> /dev/null - # Otherwise, else - # fall back to the service command + # Otherwise, fall back to the service command service "${1}" restart &> /dev/null fi printf "%b %b %s...\\n" "${OVER}" "${TICK}" "${str}" @@ -1592,9 +1589,8 @@ enable_service() { if is_command systemctl ; then # use that to enable the service systemctl enable "${1}" &> /dev/null - # Otherwise, else - # use update-rc.d to accomplish this + # Otherwise, use update-rc.d to accomplish this update-rc.d "${1}" defaults &> /dev/null fi printf "%b %b %s...\\n" "${OVER}" "${TICK}" "${str}" @@ -1609,9 +1605,8 @@ disable_service() { if is_command systemctl ; then # use that to disable the service systemctl disable "${1}" &> /dev/null - # Otherwise, else - # use update-rc.d to accomplish this + # Otherwise, use update-rc.d to accomplish this update-rc.d "${1}" disable &> /dev/null fi printf "%b %b %s...\\n" "${OVER}" "${TICK}" "${str}" @@ -1622,9 +1617,8 @@ check_service_active() { if is_command systemctl ; then # use that to check the status of the service systemctl is-enabled "${1}" &> /dev/null - # Otherwise, else - # fall back to service command + # Otherwise, fall back to service command service "${1}" status &> /dev/null fi } @@ -1666,9 +1660,8 @@ update_package_cache() { # Create a command from the package cache variable if eval "${UPDATE_PKG_CACHE}" &> /dev/null; then printf "%b %b %s\\n" "${OVER}" "${TICK}" "${str}" - # Otherwise, else - # show an error and exit + # Otherwise, show an error and exit printf "%b %b %s\\n" "${OVER}" "${CROSS}" "${str}" printf " %bError: Unable to update package cache. Please try \"%s\"%b" "${COL_LIGHT_RED}" "${UPDATE_PKG_CACHE}" "${COL_NC}" return 1 @@ -1697,19 +1690,18 @@ notify_package_updates_available() { fi } -# What's this doing outside of a function in the middle of nowhere? +# This counter is outside of install_dependent_packages so that it can count the number of times the function is called. counter=0 install_dependent_packages() { # Local, named variables should be used here, especially for an iterator # Add one to the counter counter=$((counter+1)) - # If it equals 1, if [[ "${counter}" == 1 ]]; then - # + # On the first loop, print a special message printf " %b Installer Dependency checks...\\n" "${INFO}" else - # + # On all subsequent loops, print a generic message. printf " %b Main Dependency checks...\\n" "${INFO}" fi @@ -1723,7 +1715,7 @@ install_dependent_packages() { # NOTE: We may be able to use this installArray in the future to create a list of package that were # installed by us, and remove only the installed packages, and not the entire list. if is_command apt-get ; then - # For each package, + # For each package, check if it's already installed (and if so, don't add it to the installArray) for i in "$@"; do printf " %b Checking for %s..." "${INFO}" "${i}" if dpkg-query -W -f='${Status}' "${i}" 2>/dev/null | grep "ok installed" &> /dev/null; then @@ -1733,6 +1725,7 @@ install_dependent_packages() { installArray+=("${i}") fi done + # If there's anything to install, install everything in the list. if [[ "${#installArray[@]}" -gt 0 ]]; then test_dpkg_lock printf " %b Processing %s install(s) for: %s, please wait...\\n" "${INFO}" "${PKG_MANAGER}" "${installArray[*]}" @@ -1747,6 +1740,7 @@ install_dependent_packages() { # Install Fedora/CentOS packages for i in "$@"; do + # For each package, check if it's already installed (and if so, don't add it to the installArray) printf " %b Checking for %s..." "${INFO}" "${i}" if "${PKG_MANAGER}" -q list installed "${i}" &> /dev/null; then printf "%b %b Checking for %s\\n" "${OVER}" "${TICK}" "${i}" @@ -1755,6 +1749,7 @@ install_dependent_packages() { installArray+=("${i}") fi done + # If there's anything to install, install everything in the list. if [[ "${#installArray[@]}" -gt 0 ]]; then printf " %b Processing %s install(s) for: %s, please wait...\\n" "${INFO}" "${PKG_MANAGER}" "${installArray[*]}" printf '%*s\n' "$columns" '' | tr " " -; @@ -1772,7 +1767,7 @@ installPiholeWeb() { local str="Creating directory for blocking page, and copying files" printf " %b %s..." "${INFO}" "${str}" - # Install the directory + # Install the directory, install -d -m 0755 ${PI_HOLE_BLOCKPAGE_DIR} # and the blockpage install -D -m 644 ${PI_HOLE_LOCAL_REPO}/advanced/{index,blockingpage}.* ${PI_HOLE_BLOCKPAGE_DIR}/ @@ -1791,9 +1786,8 @@ installPiholeWeb() { # back it up mv ${webroot}/index.lighttpd.html ${webroot}/index.lighttpd.orig printf "%b %b %s\\n" "${OVER}" "${TICK}" "${str}" - # Otherwise, else - # don't do anything + # Otherwise, don't do anything printf "%b %b %s\\n" "${OVER}" "${INFO}" "${str}" printf " No default index.lighttpd.html file found... not backing up\\n" fi @@ -1801,7 +1795,7 @@ installPiholeWeb() { # Install Sudoers file local str="Installing sudoer file" printf "\\n %b %s..." "${INFO}" "${str}" - # Make the .d directory if it doesn't exist + # Make the .d directory if it doesn't exist, install -d -m 755 /etc/sudoers.d/ # and copy in the pihole sudoers file install -m 0640 ${PI_HOLE_LOCAL_REPO}/advanced/Templates/pihole.sudo /etc/sudoers.d/pihole @@ -1845,11 +1839,11 @@ runGravity() { create_pihole_user() { local str="Checking for user 'pihole'" printf " %b %s..." "${INFO}" "${str}" - # If the user pihole exists, + # If the pihole user exists, if id -u pihole &> /dev/null; then - # if group exists + # and if the pihole group exists, if getent group pihole > /dev/null 2>&1; then - # just show a success + # succeed printf "%b %b %s\\n" "${OVER}" "${TICK}" "${str}" else local str="Checking for group 'pihole'" @@ -1870,14 +1864,14 @@ create_pihole_user() { printf "%b %b %s\\n" "${OVER}" "${CROSS}" "${str}" fi fi - # Otherwise, else + # If the pihole user doesn't exist, printf "%b %b %s" "${OVER}" "${CROSS}" "${str}" local str="Creating user 'pihole'" printf "%b %b %s..." "${OVER}" "${INFO}" "${str}" - # create her with the useradd command + # create her with the useradd command, if getent group pihole > /dev/null 2>&1; then - # add primary group pihole as it already exists + # then add her to the pihole group (as it already exists) if useradd -r --no-user-group -g pihole -s /usr/sbin/nologin pihole; then printf "%b %b %s\\n" "${OVER}" "${TICK}" "${str}" else @@ -1894,7 +1888,7 @@ create_pihole_user() { fi } -# +# This function saves any changes to the setup variables into the setupvars.conf file for future runs finalExports() { # If the Web interface is not set to be installed, if [[ "${INSTALL_WEB_INTERFACE}" == false ]]; then @@ -1946,7 +1940,6 @@ finalExports() { # Install the logrotate script installLogrotate() { - local str="Installing latest logrotate script" printf "\\n %b %s..." "${INFO}" "${str}" # Copy the file over from the local repo @@ -1958,9 +1951,9 @@ installLogrotate() { # customize the logrotate script here in order to reflect # the local properties of the /var/log directory logusergroup="$(stat -c '%U %G' /var/log)" - # If the variable has a value, + # If there is a usergroup for log rotation, if [[ ! -z "${logusergroup}" ]]; then - # + # replace the line in the logrotate script with that usergroup. sed -i "s/# su #/su ${logusergroup}/g;" /etc/pihole/logrotate fi printf "%b %b %s\\n" "${OVER}" "${TICK}" "${str}" @@ -2099,15 +2092,15 @@ checkSelinux() { # Installation complete message with instructions for the user displayFinalMessage() { - # If + # If the number of arguments is > 0, if [[ "${#1}" -gt 0 ]] ; then + # set the password to the first argument. pwstring="$1" - # else, if the dashboard password in the setup variables exists, elif [[ $(grep 'WEBPASSWORD' -c /etc/pihole/setupVars.conf) -gt 0 ]]; then - # set a variable for evaluation later + # Else if the password exists from previous setup, we'll load it later pwstring="unchanged" else - # set a variable for evaluation later + # Else, inform the user that there is no set password. pwstring="NOT SET" fi # If the user wants to install the dashboard, @@ -2138,9 +2131,8 @@ update_dialogs() { opt1a="Repair" opt1b="This will retain existing settings" strAdd="You will remain on the same version" - # Otherwise, else - # set some variables with different values + # Otherwise, set some variables with different values opt1a="Update" opt1b="This will retain existing settings." strAdd="You will be updated to the latest version." @@ -2291,7 +2283,6 @@ clone_or_update_repos() { # Disable directive for SC2120 a value _can_ be passed to this function, but it is passed from an external script that sources this one # shellcheck disable=SC2120 FTLinstall() { - # Local, named variables local latesttag local str="Downloading and Installing FTL" @@ -2322,9 +2313,8 @@ FTLinstall() { url="https://ftl.pi-hole.net/${ftlBranch}" fi - # If the download worked, if curl -sSL --fail "${url}/${binary}" -o "${binary}"; then - # get sha1 of the binary we just downloaded for verification. + # If the download worked, get sha1 of the binary we just downloaded for verification. curl -sSL --fail "${url}/${binary}.sha1" -o "${binary}.sha1" # If we downloaded binary file (as opposed to text), @@ -2348,16 +2338,15 @@ FTLinstall() { # Installed the FTL service printf "%b %b %s\\n" "${OVER}" "${TICK}" "${str}" return 0 - # Otherwise, else - # the download failed, so just go back to the original directory + # Otherwise, the hash download failed, so print and exit. popd > /dev/null || { printf "Unable to return to original directory after FTL binary download.\\n"; return 1; } printf "%b %b %s\\n" "${OVER}" "${CROSS}" "${str}" printf " %bError: Download of %s/%s failed (checksum error)%b\\n" "${COL_LIGHT_RED}" "${url}" "${binary}" "${COL_NC}" return 1 fi - # Otherwise, else + # Otherwise, the download failed, so print and exit. popd > /dev/null || { printf "Unable to return to original directory after FTL binary download.\\n"; return 1; } printf "%b %b %s\\n" "${OVER}" "${CROSS}" "${str}" # The URL could not be found @@ -2400,18 +2389,14 @@ get_binary_name() { # If the machine is arm or aarch if [[ "${machine}" == "arm"* || "${machine}" == *"aarch"* ]]; then # ARM - # local rev rev=$(uname -m | sed "s/[^0-9]//g;") - # local lib lib=$(ldd /bin/ls | grep -E '^\s*/lib' | awk '{ print $1 }') - # if [[ "${lib}" == "/lib/ld-linux-aarch64.so.1" ]]; then printf "%b %b Detected AArch64 (64 Bit ARM) processor\\n" "${OVER}" "${TICK}" # set the binary to be used l_binary="pihole-FTL-aarch64-linux-gnu" - # elif [[ "${lib}" == "/lib/ld-linux-armhf.so.3" ]]; then # Hard-float available: Use gnueabihf binaries # If ARMv8 or higher is found (e.g., BCM2837 as found in Raspberry Pi Model 3B) @@ -2419,13 +2404,13 @@ get_binary_name() { printf "%b %b Detected ARMv8 (or newer) processor\\n" "${OVER}" "${TICK}" # set the binary to be used l_binary="pihole-FTL-armv8-linux-gnueabihf" - # Otherwise, if ARMv7 is found (e.g., BCM2836 as found in Raspberry Pi Model 2) elif [[ "${rev}" -eq 7 ]]; then + # Otherwise, if ARMv7 is found (e.g., BCM2836 as found in Raspberry Pi Model 2) printf "%b %b Detected ARMv7 processor (with hard-float support)\\n" "${OVER}" "${TICK}" # set the binary to be used l_binary="pihole-FTL-armv7-linux-gnueabihf" - # Otherwise, use the ARMv6 binary (e.g., BCM2835 as found in Raspberry Pi Zero and Model 1) else + # Otherwise, use the ARMv6 binary (e.g., BCM2835 as found in Raspberry Pi Zero and Model 1) printf "%b %b Detected ARMv6 processor (with hard-float support)\\n" "${OVER}" "${TICK}" # set the binary to be used l_binary="pihole-FTL-armv6-linux-gnueabihf" @@ -2475,6 +2460,7 @@ get_binary_name() { l_binary="pihole-FTL-linux-x86_32" fi + # Returning a string value via echo echo ${l_binary} } @@ -2611,9 +2597,8 @@ main() { # Show the Pi-hole logo so people know it's genuine since the logo and name are trademarked show_ascii_berry make_temporary_log - # Otherwise, else - # They do not have enough privileges, so let the user know + # Otherwise, they do not have enough privileges, so let the user know printf " %b %s\\n" "${INFO}" "${str}" printf " %b %bScript called with non-root privileges%b\\n" "${INFO}" "${COL_LIGHT_RED}" "${COL_NC}" printf " The Pi-hole requires elevated privileges to install and run\\n" @@ -2621,7 +2606,7 @@ main() { printf " Make sure to download this script from a trusted source\\n\\n" printf " %b Sudo utility check" "${INFO}" - # If the sudo command exists, + # If the sudo command exists, try rerunning as admin if is_command sudo ; then printf "%b %b Sudo utility check\\n" "${OVER}" "${TICK}" @@ -2635,9 +2620,8 @@ main() { fi exit $? - # Otherwise, else - # Let them know they need to run it as root + # Otherwise, tell the user they need to run the script as root, and bail printf "%b %b Sudo utility check\\n" "${OVER}" "${CROSS}" printf " %b Sudo is needed for the Web Interface to run pihole commands\\n\\n" "${INFO}" printf " %b %bPlease re-run this installer as root${COL_NC}\\n" "${INFO}" "${COL_LIGHT_RED}" @@ -2657,9 +2641,8 @@ main() { useUpdateVars=true # also disable debconf-apt-progress dialogs export DEBIAN_FRONTEND="noninteractive" - # Otherwise, else - # show the available options (repair/reconfigure) + # If running attended, show the available options (repair/reconfigure) update_dialogs fi fi @@ -2713,13 +2696,13 @@ main() { PRIVACY_LEVEL="${PRIVACY_LEVEL:-0}" fi fi - # Clone/Update the repos + # Download or update the scripts by updating the appropriate git repos clone_or_update_repos # Install the Core dependencies local dep_install_list=("${PIHOLE_DEPS[@]}") if [[ "${INSTALL_WEB_SERVER}" == true ]]; then - # Install the Web dependencies + # And, if the setting says so, install the Web admin interface dependencies dep_install_list+=("${PIHOLE_WEB_DEPS[@]}") fi @@ -2727,7 +2710,7 @@ main() { unset dep_install_list # On some systems, lighttpd is not enabled on first install. We need to enable it here if the user - # has chosen to install the web interface, else the `LIGHTTPD_ENABLED` check will fail + # has chosen to install the web interface, else the LIGHTTPD_ENABLED check will fail if [[ "${INSTALL_WEB_SERVER}" == true ]]; then enable_service lighttpd fi @@ -2779,7 +2762,6 @@ main() { # If the Web server was installed, if [[ "${INSTALL_WEB_SERVER}" == true ]]; then - if [[ "${LIGHTTPD_ENABLED}" == true ]]; then restart_service lighttpd enable_service lighttpd diff --git a/automated install/uninstall.sh b/automated install/uninstall.sh index 6e36d16b..a0d3b108 100755 --- a/automated install/uninstall.sh +++ b/automated install/uninstall.sh @@ -31,7 +31,7 @@ else else echo -e " ${CROSS} ${str} Script called with non-root privileges - The Pi-hole requires elevated privleges to uninstall" + The Pi-hole requires elevated privileges to uninstall" exit 1 fi fi diff --git a/gravity.sh b/gravity.sh index dfb863e9..24a41c48 100755 --- a/gravity.sh +++ b/gravity.sh @@ -35,8 +35,9 @@ localList="${piholeDir}/local.list" VPNList="/etc/openvpn/ipp.txt" piholeGitDir="/etc/.pihole" -gravityDBfile="${piholeDir}/gravity.db" -gravityTEMPfile="${piholeDir}/gravity_temp.db" +gravityDBfile_default="${piholeDir}/gravity.db" +# GRAVITYDB may be overwritten by source pihole-FTL.conf below +GRAVITYDB="${gravityDBfile_default}" gravityDBschema="${piholeGitDir}/advanced/Templates/gravity.db.sql" gravityDBcopy="${piholeGitDir}/advanced/Templates/gravity_copy.sql" @@ -68,6 +69,11 @@ if [[ -f "${pihole_FTL}" ]]; then source "${pihole_FTL}" fi +# Set this only after sourcing pihole-FTL.conf as the gravity database path may +# have changed +gravityDBfile="${GRAVITYDB}" +gravityTEMPfile="${GRAVITYDB}_temp" + if [[ -z "${BLOCKINGMODE}" ]] ; then BLOCKINGMODE="NULL" fi @@ -84,11 +90,11 @@ generate_gravity_database() { # Copy data from old to new database file and swap them gravity_swap_databases() { - local str + local str copyGravity str="Building tree" echo -ne " ${INFO} ${str}..." - # The index is intentionally not UNIQUE as prro quality adlists may contain domains more than once + # The index is intentionally not UNIQUE as poor quality adlists may contain domains more than once output=$( { sqlite3 "${gravityTEMPfile}" "CREATE INDEX idx_gravity ON gravity (domain, adlist_id);"; } 2>&1 ) status="$?" @@ -101,7 +107,14 @@ gravity_swap_databases() { str="Swapping databases" echo -ne " ${INFO} ${str}..." - output=$( { sqlite3 "${gravityTEMPfile}" < "${gravityDBcopy}"; } 2>&1 ) + # Gravity copying SQL script + copyGravity="$(cat "${gravityDBcopy}")" + if [[ "${gravityDBfile}" != "${gravityDBfile_default}" ]]; then + # Replace default gravity script location by custom location + copyGravity="${copyGravity//"${gravityDBfile_default}"/"${gravityDBfile}"}" + fi + + output=$( { sqlite3 "${gravityTEMPfile}" <<< "${copyGravity}"; } 2>&1 ) status="$?" if [[ "${status}" -ne 0 ]]; then @@ -176,7 +189,7 @@ database_table_from_file() { echo "${rowid},\"${domain}\",${timestamp}" >> "${tmpFile}" elif [[ "${table}" == "adlist" ]]; then # Adlist table format - echo "${rowid},\"${domain}\",1,${timestamp},${timestamp},\"Migrated from ${source}\"," >> "${tmpFile}" + echo "${rowid},\"${domain}\",1,${timestamp},${timestamp},\"Migrated from ${source}\",,0,0,0" >> "${tmpFile}" else # White-, black-, and regexlist table format echo "${rowid},${type},\"${domain}\",1,${timestamp},${timestamp},\"Migrated from ${source}\"" >> "${tmpFile}" @@ -217,6 +230,48 @@ database_adlist_updated() { fi } +# Check if a column with name ${2} exists in gravity table with name ${1} +gravity_column_exists() { + output=$( { printf ".timeout 30000\\nSELECT EXISTS(SELECT * FROM pragma_table_info('%s') WHERE name='%s');\\n" "${1}" "${2}" | sqlite3 "${gravityDBfile}"; } 2>&1 ) + if [[ "${output}" == "1" ]]; then + return 0 # Bash 0 is success + fi + + return 1 # Bash non-0 is failure +} + +# Update number of domain on this list. We store this in the "old" database as all values in the new database will later be overwritten +database_adlist_number() { + # Only try to set number of domains when this field exists in the gravity database + if ! gravity_column_exists "adlist" "number"; then + return; + fi + + output=$( { printf ".timeout 30000\\nUPDATE adlist SET number = %i, invalid_domains = %i WHERE id = %i;\\n" "${num_lines}" "${num_invalid}" "${1}" | sqlite3 "${gravityDBfile}"; } 2>&1 ) + status="$?" + + if [[ "${status}" -ne 0 ]]; then + echo -e "\\n ${CROSS} Unable to update number of domains in adlist with ID ${1} in database ${gravityDBfile}\\n ${output}" + gravity_Cleanup "error" + fi +} + +# Update status of this list. We store this in the "old" database as all values in the new database will later be overwritten +database_adlist_status() { + # Only try to set the status when this field exists in the gravity database + if ! gravity_column_exists "adlist" "status"; then + return; + fi + + output=$( { printf ".timeout 30000\\nUPDATE adlist SET status = %i WHERE id = %i;\\n" "${2}" "${1}" | sqlite3 "${gravityDBfile}"; } 2>&1 ) + status="$?" + + if [[ "${status}" -ne 0 ]]; then + echo -e "\\n ${CROSS} Unable to update status of adlist with ID ${1} in database ${gravityDBfile}\\n ${output}" + gravity_Cleanup "error" + fi +} + # Migrate pre-v5.0 list files to database-based Pi-hole versions migrate_to_database() { # Create database file only if not present @@ -317,6 +372,10 @@ gravity_CheckDNSResolutionAvailable() { gravity_DownloadBlocklists() { echo -e " ${INFO} ${COL_BOLD}Neutrino emissions detected${COL_NC}..." + if [[ "${gravityDBfile}" != "${gravityDBfile_default}" ]]; then + echo -e " ${INFO} Storing gravity database in ${COL_BOLD}${gravityDBfile}${COL_NC}" + fi + # Retrieve source URLs from gravity database # We source only enabled adlists, sqlite3 stores boolean values as 0 (false) or 1 (true) mapfile -t sources <<< "$(sqlite3 "${gravityDBfile}" "SELECT address FROM vw_adlist;" 2> /dev/null)" @@ -363,7 +422,7 @@ gravity_DownloadBlocklists() { target="$(mktemp -p "/tmp" --suffix=".gravity")" - # Use compression to reduce the amount of data that is transfered + # Use compression to reduce the amount of data that is transferred # between the Pi-hole and the ad list provider. Use this feature # only if it is supported by the locally available version of curl if curl -V | grep -q "Features:.* libz"; then @@ -444,6 +503,8 @@ gravity_DownloadBlocklists() { } total_num=0 +num_lines=0 +num_invalid=0 parseList() { local adlistID="${1}" src="${2}" target="${3}" incorrect_lines # This sed does the following things: @@ -454,7 +515,7 @@ parseList() { # Find (up to) five domains containing invalid characters (see above) incorrect_lines="$(sed -e "/[^a-zA-Z0-9.\_-]/!d" "${src}" | head -n 5)" - local num_lines num_target_lines num_correct_lines num_invalid + local num_target_lines num_correct_lines num_invalid # Get number of lines in source file num_lines="$(grep -c "^" "${src}")" # Get number of lines in destination file @@ -463,9 +524,9 @@ parseList() { total_num="$num_target_lines" num_invalid="$(( num_lines-num_correct_lines ))" if [[ "${num_invalid}" -eq 0 ]]; then - echo " ${INFO} Received ${num_lines} domains" + echo " ${INFO} Analyzed ${num_lines} domains" else - echo " ${INFO} Received ${num_lines} domains, ${num_invalid} domains invalid!" + echo " ${INFO} Analyzed ${num_lines} domains, ${num_invalid} domains invalid!" fi # Display sample of invalid lines if we found some @@ -476,6 +537,29 @@ parseList() { done <<< "${incorrect_lines}" fi } +compareLists() { + local adlistID="${1}" target="${2}" + + # Verify checksum when an older checksum exists + if [[ -s "${target}.sha1" ]]; then + if ! sha1sum --check --status --strict "${target}.sha1"; then + # The list changed upstream, we need to update the checksum + sha1sum "${target}" > "${target}.sha1" + echo " ${INFO} List has been updated" + database_adlist_status "${adlistID}" "1" + database_adlist_updated "${adlistID}" + else + echo " ${INFO} List stayed unchanged" + database_adlist_status "${adlistID}" "2" + fi + else + # No checksum available, create one for comparing on the next run + sha1sum "${target}" > "${target}.sha1" + # We assume here it was changed upstream + database_adlist_status "${adlistID}" "1" + database_adlist_updated "${adlistID}" + fi +} # Download specified URL and perform checks on HTTP status and file content gravity_DownloadBlocklistFromUrl() { @@ -559,31 +643,49 @@ gravity_DownloadBlocklistFromUrl() { esac;; esac + local done="false" # Determine if the blocklist was downloaded and saved correctly if [[ "${success}" == true ]]; then if [[ "${httpCode}" == "304" ]]; then # Add domains to database table file parseList "${adlistID}" "${saveLocation}" "${target}" + database_adlist_status "${adlistID}" "2" + database_adlist_number "${adlistID}" + done="true" # Check if $patternbuffer is a non-zero length file elif [[ -s "${patternBuffer}" ]]; then # Determine if blocklist is non-standard and parse as appropriate gravity_ParseFileIntoDomains "${patternBuffer}" "${saveLocation}" # Add domains to database table file parseList "${adlistID}" "${saveLocation}" "${target}" - # Update date_updated field in gravity database table - database_adlist_updated "${adlistID}" + # Compare lists, are they identical? + compareLists "${adlistID}" "${saveLocation}" + # Update gravity database table (status and updated timestamp are set in + # compareLists) + database_adlist_number "${adlistID}" + done="true" else # Fall back to previously cached list if $patternBuffer is empty - echo -e " ${INFO} Received empty file: ${COL_LIGHT_GREEN}using previously cached list${COL_NC}" + echo -e " ${INFO} Received empty file" fi - else + fi + + # Do we need to fall back to a cached list (if available)? + if [[ "${done}" != "true" ]]; then # Determine if cached list has read permission if [[ -r "${saveLocation}" ]]; then echo -e " ${CROSS} List download failed: ${COL_LIGHT_GREEN}using previously cached list${COL_NC}" # Add domains to database table file parseList "${adlistID}" "${saveLocation}" "${target}" + database_adlist_number "${adlistID}" + database_adlist_status "${adlistID}" "3" else echo -e " ${CROSS} List download failed: ${COL_LIGHT_RED}no cached list available${COL_NC}" + # Manually reset these two numbers because we do not call parseList here + num_lines=0 + num_invalid=0 + database_adlist_number "${adlistID}" + database_adlist_status "${adlistID}" "4" fi fi } @@ -595,7 +697,7 @@ gravity_ParseFileIntoDomains() { # Determine if we are parsing a consolidated list #if [[ "${source}" == "${piholeDir}/${matterAndLight}" ]]; then # Remove comments and print only the domain name - # Most of the lists downloaded are already in hosts file format but the spacing/formating is not contiguous + # Most of the lists downloaded are already in hosts file format but the spacing/formatting is not contiguous # This helps with that and makes it easier to read # It also helps with debugging so each stage of the script can be researched more in depth # 1) Remove carriage returns diff --git a/manpages/pihole.8 b/manpages/pihole.8 index a57eb05e..4ba0e0f7 100644 --- a/manpages/pihole.8 +++ b/manpages/pihole.8 @@ -153,7 +153,7 @@ Available commands and options: .br -r, --refresh Set update frequency (in seconds) .br - -e, --exit Output stats and exit witout refreshing + -e, --exit Output stats and exit without refreshing .br \fB-g, updateGravity\fR diff --git a/pihole b/pihole index 2b2b7bc2..f8085c8e 100755 --- a/pihole +++ b/pihole @@ -16,6 +16,7 @@ readonly PI_HOLE_SCRIPT_DIR="/opt/pihole" # error due to modifying a readonly variable. setupVars="/etc/pihole/setupVars.conf" PI_HOLE_BIN_DIR="/usr/local/bin" +readonly FTL_PID_FILE="/run/pihole-FTL.pid" readonly colfile="${PI_HOLE_SCRIPT_DIR}/COL_TABLE" source "${colfile}" @@ -98,8 +99,25 @@ versionFunc() { exit 0 } +# Get PID of main pihole-FTL process +getFTLPID() { + local pid + + if [ -s "${FTL_PID_FILE}" ]; then + # -s: FILE exists and has a size greater than zero + pid="$(<"$FTL_PID_FILE")" + # Exploit prevention: unset the variable if there is malicious content + # Verify that the value read from the file is numeric + [[ "$pid" =~ [^[:digit:]] ]] && unset pid + fi + + # If FTL is not running, or the PID file contains malicious stuff, substitute + # negative PID to signal this to the caller + echo "${pid:=-1}" +} + restartDNS() { - local svcOption svc str output status + local svcOption svc str output status pid icon svcOption="${1:-restart}" # Determine if we should reload or restart @@ -108,17 +126,34 @@ restartDNS() { # Note 1: This will NOT re-read any *.conf files # Note 2: We cannot use killall here as it does # not know about real-time signals - svc="pkill -RTMIN pihole-FTL" - str="Reloading DNS lists" + pid="$(getFTLPID)" + if [[ "$pid" -eq "-1" ]]; then + svc="true" + str="FTL is not running" + icon="${INFO}" + else + svc="kill -RTMIN ${pid}" + str="Reloading DNS lists" + icon="${TICK}" + fi elif [[ "${svcOption}" =~ "reload" ]]; then # Reloading of the DNS cache has been requested # Note: This will NOT re-read any *.conf files - svc="pkill -HUP pihole-FTL" - str="Flushing DNS cache" + pid="$(getFTLPID)" + if [[ "$pid" -eq "-1" ]]; then + svc="true" + str="FTL is not running" + icon="${INFO}" + else + svc="kill -HUP ${pid}" + str="Flushing DNS cache" + icon="${TICK}" + fi else # A full restart has been requested svc="service pihole-FTL restart" str="Restarting DNS server" + icon="${TICK}" fi # Print output to Terminal, but not to Web Admin @@ -128,7 +163,7 @@ restartDNS() { status="$?" if [[ "${status}" -eq 0 ]]; then - [[ -t 1 ]] && echo -e "${OVER} ${TICK} ${str}" + [[ -t 1 ]] && echo -e "${OVER} ${icon} ${str}" return 0 else [[ ! -t 1 ]] && local OVER="" diff --git a/supportedos.txt b/supportedos.txt index 1eb1fde6..6d579132 100644 --- a/supportedos.txt +++ b/supportedos.txt @@ -1,5 +1,5 @@ Raspbian=9,10 Ubuntu=16,18,20 Debian=9,10 -Fedora=31,32 +Fedora=32,33 CentOS=7,8 \ No newline at end of file diff --git a/test/_fedora_31.Dockerfile b/test/_fedora_33.Dockerfile similarity index 96% rename from test/_fedora_31.Dockerfile rename to test/_fedora_33.Dockerfile index 02dcb733..0a1ac59c 100644 --- a/test/_fedora_31.Dockerfile +++ b/test/_fedora_33.Dockerfile @@ -1,4 +1,4 @@ -FROM fedora:31 +FROM fedora:33 ENV GITDIR /etc/.pihole ENV SCRIPTDIR /opt/pihole diff --git a/test/test_automated_install.py b/test/test_automated_install.py index f6b5a87e..1266f46b 100644 --- a/test/test_automated_install.py +++ b/test/test_automated_install.py @@ -524,43 +524,45 @@ def test_IPv6_ULA_GUA_test(Pihole): assert expected_stdout in detectPlatform.stdout -def test_validate_ip_valid(Pihole): - ''' - Given a valid IP address, valid_ip returns success - ''' - - output = Pihole.run(''' - source /opt/pihole/basic-install.sh - valid_ip "192.168.1.1" - ''') - - assert output.rc == 0 - - -def test_validate_ip_invalid_octet(Pihole): - ''' - Given an invalid IP address (large octet), valid_ip returns an error - ''' - - output = Pihole.run(''' - source /opt/pihole/basic-install.sh - valid_ip "1092.168.1.1" - ''') - - assert output.rc == 1 - - -def test_validate_ip_invalid_letters(Pihole): - ''' - Given an invalid IP address (contains letters), valid_ip returns an error - ''' - - output = Pihole.run(''' - source /opt/pihole/basic-install.sh - valid_ip "not an IP" - ''') - - assert output.rc == 1 +def test_validate_ip(Pihole): + ''' + Tests valid_ip for various IP addresses + ''' + + def test_address(addr, success=True): + output = Pihole.run(''' + source /opt/pihole/basic-install.sh + valid_ip "{addr}" + '''.format(addr=addr)) + + assert output.rc == 0 if success else 1 + + test_address('192.168.1.1') + test_address('127.0.0.1') + test_address('255.255.255.255') + test_address('255.255.255.256', False) + test_address('255.255.256.255', False) + test_address('255.256.255.255', False) + test_address('256.255.255.255', False) + test_address('1092.168.1.1', False) + test_address('not an IP', False) + test_address('8.8.8.8:', False) + test_address('8.8.8.8:0') + test_address('8.8.8.8:1') + test_address('8.8.8.8:42') + test_address('8.8.8.8:888') + test_address('8.8.8.8:1337') + test_address('8.8.8.8:65535') + test_address('8.8.8.8:65536', False) + test_address('8.8.8.8:-1', False) + test_address('00.0.0.0', False) + test_address('010.0.0.0', False) + test_address('001.0.0.0', False) + test_address('0.0.0.0:00', False) + test_address('0.0.0.0:01', False) + test_address('0.0.0.0:001', False) + test_address('0.0.0.0:0001', False) + test_address('0.0.0.0:00001', False) def test_os_check_fails(Pihole): diff --git a/test/tox.fedora_31.ini b/test/tox.fedora_33.ini similarity index 78% rename from test/tox.fedora_31.ini rename to test/tox.fedora_33.ini index 36ab10ad..00ea732a 100644 --- a/test/tox.fedora_31.ini +++ b/test/tox.fedora_33.ini @@ -4,5 +4,5 @@ envlist = py37 [testenv] whitelist_externals = docker deps = -rrequirements.txt -commands = docker build -f _fedora_31.Dockerfile -t pytest_pihole:test_container ../ +commands = docker build -f _fedora_33.Dockerfile -t pytest_pihole:test_container ../ pytest {posargs:-vv -n auto} ./test_automated_install.py ./test_centos_fedora_common_support.py ./test_fedora_support.py