arno/pi-hole
1
0
Fork 0
mirror of https://github.com/pi-hole/pi-hole synced 2024-12-22 23:08:07 +00:00
Code Issues Releases Wiki Activity
6,441 Commits 10 Branches 109 Tags 20 MiB
0e7706ffa3
Commit Graph

560 Commits

Author SHA1 Message Date
DL6ER
534f9a63bf
Copy database tables earlier into the new gravity database to avoid foreign key contraint violations when adding gravity entries refering to an empty adlist table 
Signed-off-by: DL6ER <dl6er@dl6er.de>
 2022-01-30 10:36:20 +01:00
DL6ER
f0f5cc52d9
Use internal SQLite3 engine in more places in gravity.sh 
Signed-off-by: DL6ER <dl6er@dl6er.de>
 2022-01-29 22:39:45 +01:00
DL6ER
ce86157067
Fix gravity in case there are no adlists at all or all are disabled (#4535) 
Signed-off-by: DL6ER <dl6er@dl6er.de>
 2022-01-08 14:15:26 +01:00
DL6ER
3d3bb45a46
Merge pull request #4288 from pi-hole/new/gravity_repair 
Implement fully-automated gravity database recovery method
 2021-12-22 21:08:01 +01:00
DL6ER
d2a98ae954
Document -r recover force case 
Signed-off-by: DL6ER <dl6er@dl6er.de>
 2021-12-22 19:53:52 +01:00
DL6ER
2e1ce7fc87
Apply suggestions from code review 
Co-authored-by: yubiuser <ckoenig@posteo.de>
 2021-12-22 19:52:08 +01:00
DL6ER
54c58327f1
Merge pull request #4450 from pi-hole/unblock_NODATA 
Unblock adlist domain during gravity run in NODATA mode
 2021-12-21 22:08:14 +01:00
yubiuser
db5e94b14a
use +short and omit obsolet awk 
Co-authored-by: DL6ER <DL6ER@users.noreply.github.com>
 2021-12-21 22:01:34 +01:00
DL6ER
7167e6d5e4
Apply suggestions from code review 
Co-authored-by: Dan Schaper <dan.schaper@pi-hole.net>
 2021-12-21 16:20:02 +01:00
DL6ER
b06efb6ab7
Declare variables local 
Signed-off-by: DL6ER <dl6er@dl6er.de>
 2021-12-21 14:00:46 +01:00
DL6ER
ab4bce4787
Allow users to force recovery even when checks are okay using "pihole -g -r recover force" 
Signed-off-by: DL6ER <dl6er@dl6er.de>
 2021-12-21 13:57:03 +01:00
DL6ER
469c179b32
Return early from recovery routine when integrity checks didn't show any database errors. 
Signed-off-by: DL6ER <dl6er@dl6er.de>
 2021-12-21 13:57:03 +01:00
DL6ER
190ab79606
Implement fully-automated gravity database recovery method. 
Signed-off-by: DL6ER <dl6er@dl6er.de>
 2021-12-21 13:57:03 +01:00
Dan Schaper
533a77d6d5
Add database function failure guards. 
Signed-off-by: Dan Schaper <dan.schaper@pi-hole.net>
 2021-12-20 11:36:55 -08:00
Dan Schaper
76ae75689c
Check for DNS before run. 
Signed-off-by: Dan Schaper <dan.schaper@pi-hole.net>
 2021-12-20 11:09:11 -08:00
Dan Schaper
a780fc59e2
Set DBFile permissions on creation. 
Signed-off-by: Dan Schaper <dan.schaper@pi-hole.net>
 2021-12-20 10:56:42 -08:00
Christian König
e243c562c2
Unblock adlist domain during gravity run in NODATA mode 
Signed-off-by: Christian König <ckoenig@posteo.de>
 2021-12-03 09:17:19 +01:00
Christian König
647ba6ec9d
Rename variables to improve comprehensibility 
Signed-off-by: Christian König <ckoenig@posteo.de>
 2021-12-02 23:13:01 +01:00
Christian König
bfda52ed79
Fix number of invalid domains 
Co-authored-by: abesnier <besnier_antoine@yahoo.fr>
Signed-off-by: Christian König <ckoenig@posteo.de>
 2021-12-01 11:07:17 +01:00
jbzdarkid
bc8150adfa Clean up bash script formatting 
Done with the help of beautysh (a python-based bash formatter)

Signed-off-by: jbzdarkid <jbzdarkid@gmail.com>
 2021-11-25 14:12:09 -08:00
yubiuser
38bb4a4908
Remove unused wildcard_regex_converter.sh (#4369) 
* Remove unused wildcard_regex_converter.sh

Signed-off-by: Christian König <ckoenig@posteo.de>

* Remove regexconverter

Signed-off-by: Christian König <ckoenig@posteo.de>
 2021-10-05 13:09:16 +01:00
jpgpi250
3439045228 fix Pi-hole v5.4 update - syntax error #4322 
Signed-off-by: jpgpi250 <jpgpi250@gmail.com>
 2021-09-12 18:24:15 +02:00
Adam Warner
4ac4bc5c18
Merge branch 'development' into fix/gravity_no_locallist 2021-07-04 00:48:09 +01:00
jpgpi250
a2d5b8050a Update gravity.sh 
Signed-off-by: jpgpi250 <jpgpi250@gmail.com>
 2021-06-14 20:27:10 +02:00
DL6ER
f90dffbf1a
Do not put hostname and pi.hole into local.list - this will be handled by FTL after FTL#1111 has been merged 
Signed-off-by: DL6ER <dl6er@dl6er.de>
 2021-04-16 11:41:26 +02:00
DL6ER
f536718aaa
Use possibly custom gravity.db location in the copying SQL script 
Signed-off-by: DL6ER <dl6er@dl6er.de>
 2021-03-18 16:46:59 +01:00
DL6ER
d25240fe9f
pihole -g should respose GRAVITYDB in pihole-FTL.conf 
Signed-off-by: DL6ER <dl6er@dl6er.de>
 2021-03-18 09:00:27 +01:00
DL6ER
126b9ae381
Add missing three new columns to pihole -g -r 
Signed-off-by: DL6ER <dl6er@dl6er.de>
 2021-03-02 20:08:09 +01:00
DL6ER
57fe3b6621
Merge pull request #3951 from pi-hole/new/gravity_adlist_infos 
Store gravity details in gravity.db adlist table
 2021-02-28 17:01:19 +01:00
Adam Warner
9ec012289f
Merge pull request #4017 from mwoolweaver/patch-2 
fix suspected typo in comment
 2021-01-26 08:48:05 +00:00
Michael Woolweaver
c42b97ddb2
fix typo in comment 
Signed-off-by: Michael Woolweaver <m.woolweaver@icloud.com>
 2021-01-26 01:04:37 -06:00
freddii
7c0c30fb0b fixed typos 2021-01-19 19:33:38 +01:00
DL6ER
1791fe22f6
Merge pull request #3912 from mdujava/basic_auth 
Fix validation of adlist url
 2020-12-31 12:19:32 +01:00
DL6ER
a2625df5e2
Remove "The list is new" comment because it is superfluous 
Signed-off-by: DL6ER <dl6er@dl6er.de>
 2020-12-30 11:27:34 +01:00
DL6ER
0944807491
Actually store correct number of domains for the individual lists (and not the sum of the so far collected number of domains) 
Signed-off-by: DL6ER <dl6er@dl6er.de>
 2020-12-29 20:36:49 +01:00
DL6ER
a216848c1d
Only update time of last list change when we see a list for the first time or when it really changed content 
Signed-off-by: DL6ER <dl6er@dl6er.de>
 2020-12-29 20:28:09 +01:00
DL6ER
8c56f54a1e
Compare checksum of downloaded list against older checksums to see if the list content changed since the last download 
Signed-off-by: DL6ER <dl6er@dl6er.de>
 2020-12-29 09:54:25 +01:00
DL6ER
6b33b8b4c0
Store status of downloaded list (downloaded, using cache, some error, ...) and number of (in-)valid domains on this list in the gravity database. This updates the gravity databaes to version 14. 
Signed-off-by: DL6ER <dl6er@dl6er.de>
 2020-12-28 08:08:49 +01:00
Matej Dujava
ef0bdf6470 Fix validation of adlist url 
Already existing regex validation will be used on url after removing @ (in case
its in separating userinfo and host).

Signed-off-by: Matej Dujava <mdujava@kocurkovo.cz>
Fixes: https://github.com/pi-hole/pi-hole/issues/3911
Fixes: 7d19ee1b: validate blocklist URL before adding to the database (#3237)
 2020-12-21 12:50:02 +01:00
user
922f795fe6 Adding comma to the adlist table format 
To account for the Date Updated column in the adlist table adding a comma to the adlist to be imported into the database. This was causing intermittent failures on updates and therefore no domains would be blocked.

Signed-off-by: user <user@fedbrew.home>

 Date:      Sun Dec 6 06:03:30 2020 -0500
 Committer: user <user@fedbrew.home>

 On branch fix/gravity_date_updated
 Your branch and 'origin/fix/gravity_date_updated' have diverged,
 and have 1 and 1 different commits each, respectively.
   (use "git pull" to merge the remote branch into yours)

 Changes to be committed:
	modified:   gravity.sh

Signed-off-by: user <user@fedbrew.home>
 2020-12-06 06:08:30 -05:00
Adam Warner
948337dd8e
Merge pull request #3740 from pi-hole/new/adlist_date_updated 
Add date_updated field to adlist table
 2020-10-09 14:04:13 +01:00
DL6ER
4fd412d7c3
Add date_updated field in adlist table set when a list changes. 
Signed-off-by: DL6ER <dl6er@dl6er.de>
 2020-09-07 20:59:23 +02:00
DL6ER
aa7c3b6852
Remove gravity optimization. Further investigations have shown that it is useless because the standard gravity run already produces an optimal database. 
Signed-off-by: DL6ER <dl6er@dl6er.de>
 2020-09-07 20:50:11 +02:00
Adam Warner
bb1a6243d6
Merge pull request #3669 from pi-hole/release/v5.1.2 
Release/v5.1.2 to development
 2020-08-14 16:28:53 +01:00
Christian König
27399a762a Check for compression onyl once and print result 
Signed-off-by: Christian König <ckoenig@posteo.de>
 2020-08-03 22:46:14 +02:00
DL6ER
87da9084e6
Use compression (if available) when downloading the ad lists. 
Signed-off-by: DL6ER <dl6er@dl6er.de>
 2020-08-02 16:21:58 +01:00
DL6ER
527fc3c5cf
Also pidof -> pgrep in gravity.sh 
Signed-off-by: DL6ER <dl6er@dl6er.de>
 2020-07-21 22:03:54 +02:00
Christian König
73963fecda Use gravity's adlist_id in filename when saving downloaded adlist locally 
Signed-off-by: Christian König <ckoenig@posteo.de>
 2020-07-05 14:32:33 +02:00
Peter Lowe
dd205c55c1 Updated gravity.sh to allow semicolons 
Signed-off-by: Peter Lowe <pgl@yoyo.org>
 2020-05-26 15:53:10 +02:00
Adam Warner
9e96ff37de
Fix merge conflicts 
Signed-off-by: Adam Warner <me@adamwarner.co.uk>
 2020-05-12 22:16:43 +01:00
DL6ER
288d487fc0
Allow ( and ) in adlist URLs. 
Signed-off-by: DL6ER <dl6er@dl6er.de>
 2020-04-26 09:33:09 +02:00
DL6ER
fa57c457f3
Boldify number of unique domains as this is the actually interesting number 
Signed-off-by: DL6ER <dl6er@dl6er.de>
 2020-04-21 09:10:21 +02:00
Dan Schaper
0f20470a38
Merge pull request #3269 from pi-hole/tweak/hosts-comments 
Add support for comments in HOSTS-like files
 2020-04-15 11:11:10 -07:00
DL6ER
982c1b0059
Merge branch 'release/v5.0' into development 
Signed-off-by: DL6ER <dl6er@dl6er.de>
 2020-04-08 12:02:53 +02:00
Adam Warner
7d19ee1b25
validate blocklist URL before adding to the database (#3237) 
Signed-off-by: Adam Warner <me@adamwarner.co.uk>

Co-authored-by: DL6ER <dl6er@dl6er.de>
 2020-03-31 21:48:10 +01:00
DL6ER
7b15a88dc4
Strip comments from downloaded lists instead of discarding lines with comments altogether 
Signed-off-by: DL6ER <dl6er@dl6er.de>
 2020-03-31 18:36:40 +00:00
XhmikosR
7b8611ced0
Assorted typo fixes. (#3126) 
* Assorted typo fixes.

Signed-off-by: XhmikosR <xhmikosr@gmail.com>

Co-authored-by: Dan Schaper <dan.schaper@pi-hole.net>
 2020-03-08 16:53:14 -07:00
DL6ER
3dd05606ca
Call it the received number of domains instead of the imported number as importing does only happen a bit later. Only show the number of invalid domains if there are invalid domains. 
Signed-off-by: DL6ER <dl6er@dl6er.de>
 2020-02-24 07:06:15 +01:00
DL6ER
1e8bfd33f5
Improve output 
Signed-off-by: DL6ER <dl6er@dl6er.de>
 2020-02-23 22:50:06 +01:00
DL6ER
8131b5961c
Add comments to the code describing the changes. 
Signed-off-by: DL6ER <dl6er@dl6er.de>
 2020-02-22 15:22:29 +01:00
DL6ER
81d4531e10
Implement performant list checking routine. 
Signed-off-by: DL6ER <dl6er@dl6er.de>
 2020-02-22 13:01:55 +01:00
DL6ER
050e2963c7
Remove redundant code. 
Signed-off-by: DL6ER <dl6er@dl6er.de>
 2020-02-21 22:28:53 +01:00
DL6ER
3c09cd4a3a
Experimental output of matching line from shown warnings. 
Signed-off-by: DL6ER <dl6er@dl6er.de>
 2020-02-21 21:47:56 +01:00
DL6ER
839fe32042
Fix issue with missing newline at the end of adlists (#3144) 
* Also display non-fatal warnings during the database importing. Previously, we have only show warnings when there were also errors (errors are always fatal).

Signed-off-by: DL6ER <dl6er@dl6er.de>

* Ensure there is always a newline on the last line.

Signed-off-by: DL6ER <dl6er@dl6er.de>

* Stickler linting

Signed-off-by: Dan Schaper <dan.schaper@pi-hole.net>

* Move sed command into subroutine to avoid code duplication.

Signed-off-by: DL6ER <dl6er@dl6er.de>

* Also unify comments.

Signed-off-by: DL6ER <dl6er@dl6er.de>

* Also unify comments.

Signed-off-by: DL6ER <dl6er@dl6er.de>

Co-authored-by: Dan Schaper <dan@glacialmagma.com>
 2020-02-21 18:56:48 +00:00
DL6ER
d14ee26d6a
Merge pull request #3139 from pi-hole/fix/count_before_calling_FTL 
Fix wrong number of blocking domains shown on the dashboard
 2020-02-17 21:32:45 +01:00
DL6ER
52398052e9
Compute number of domains (and store it in the database) BEFORE calling FTL to re-read said value. 
Signed-off-by: DL6ER <dl6er@dl6er.de>
 2020-02-17 21:07:48 +01:00
DL6ER
601f9048cd
Merge pull request #3130 from pi-hole/fix/gravity_updated_timestamp 
Store gravity update timestamp only after database swapping
 2020-02-17 06:07:05 +01:00
DL6ER
dc2fce8e1d
Store gravity update timestamp only after database swapping. 
Signed-off-by: DL6ER <dl6er@dl6er.de>
 2020-02-12 19:26:25 +01:00
DL6ER
2a5cf221fa
Store number of distinct gravity domains in database after counting. 
Signed-off-by: DL6ER <dl6er@dl6er.de>
 2020-02-02 23:46:33 +01:00
DL6ER
10c2dad48a Improve gravity performance (#3100) 
* Gravity performance improvements.

Signed-off-by: DL6ER <dl6er@dl6er.de>

* Do not move downloaded lists into migration_backup directory.

Signed-off-by: DL6ER <dl6er@dl6er.de>

* Do not (strictly) sort domains. Random-leaf access is faster than always-last-leaf access (on average).

Signed-off-by: DL6ER <dl6er@dl6er.de>

* Append instead of overwrite gravity_new collection list.

Signed-off-by: DL6ER <dl6er@dl6er.de>

* Rename table gravity_new to gravity_temp to clarify that this is only an intermediate table.

Signed-off-by: DL6ER <dl6er@dl6er.de>

* Add timers for each of the calls to compute intense parts. They are to be removed before this finally hits the release/v5.0 branch.

Signed-off-by: DL6ER <dl6er@dl6er.de>

* Fix legacy list files import. It currently doesn't work when the gravity database has already been updated to using the single domainlist table.

Signed-off-by: DL6ER <dl6er@dl6er.de>

* Simplify database_table_from_file(), remove all to this function for gravity lost downloads.

Signed-off-by: DL6ER <dl6er@dl6er.de>

* Update gravity.db.sql to version 10 to have newle created databases already reflect the most recent state.

Signed-off-by: DL6ER <dl6er@dl6er.de>

* Create second gravity database and swap them on success. This has a number of advantages such as instantaneous gravity updates (as seen from FTL) and always available gravity blocking. Furthermore, this saves disk space as the old database is removed on completion.

* Add timing output for the database swapping SQLite3 call.

Signed-off-by: DL6ER <dl6er@dl6er.de>

* Explicitly generate index as a separate process.

Signed-off-by: DL6ER <dl6er@dl6er.de>

* Remove time measurements.

Signed-off-by: DL6ER <dl6er@dl6er.de>
 2020-01-24 09:39:13 -08:00
DL6ER
8f22203d24
Wait 30 seconds for obtaining a database lock instead of immediately failing if the database is busy. 
Signed-off-by: DL6ER <dl6er@dl6er.de>
 2020-01-14 20:02:00 +01:00
DL6ER
948f4a8827
Ensure permissions and ownership of gravity.db are correctly set on each run of pihole -g. This would have prevented https://github.com/pi-hole/AdminLTE/issues/1077 
Signed-off-by: DL6ER <dl6er@dl6er.de>
 2019-12-16 09:55:46 +00:00
DL6ER
2cec9eaf65
Merge pull request #3033 from pi-hole/fix/duplicates_in_adlists 
Remove duplicates from adlists before importing
 2019-12-12 21:37:26 +01:00
DL6ER
779fe670f7
Show full URL during gravity download instead of only domain and file 
Signed-off-by: DL6ER <dl6er@dl6er.de>
 2019-12-12 10:29:44 +00:00
DL6ER
570a7a5c11
Use sort -u instead of uniq as it is guaranteed to be safe when doing inline file operations. 
Signed-off-by: DL6ER <dl6er@dl6er.de>
 2019-12-12 10:17:59 +00:00
DL6ER
bd1b004d94
Remove possible duplicates found in lower-quality adlists 
Signed-off-by: DL6ER <dl6er@dl6er.de>
 2019-12-12 10:13:51 +00:00
Adam Warner
d29947ba32
optimise gravity list inserts 
Signed-off-by: Adam Warner <me@adamwarner.co.uk>
 2019-12-09 22:30:41 +00:00
Adam Warner
1f03faddef
shell check recomends 
Signed-off-by: Adam Warner <me@adamwarner.co.uk>
 2019-12-09 21:35:54 +00:00
DL6ER
3231e5c3ba
Address stickler requests. 
Signed-off-by: DL6ER <dl6er@dl6er.de>
 2019-12-09 16:52:03 +00:00
DL6ER
0c5185f8ba
Also display how many unique domains we have caught in the event horizon. 
Signed-off-by: DL6ER <dl6er@dl6er.de>
 2019-12-04 21:02:46 +00:00
DL6ER
037d52104a
New command "pihole -g -r" recreates gravity.db based on files backed up in /etc/pihole/migration_update. This is useful to restore a working version of the database when the user destroyed the original database. Also, update gravity.db to version 5 because of a fix we needed to implement. 
Signed-off-by: DL6ER <dl6er@dl6er.de>
 2019-11-26 10:58:39 +01:00
DL6ER
756c99653e
Merge branch 'development' into new/internal-blocking 2019-10-03 12:01:27 +02:00
Adam Warner
61a40c1b43
merge devel into 4.3.2 And Resolve merge conflicts 
Signed-off-by: Adam Warner <me@adamwarner.co.uk>
 2019-09-22 01:16:44 +01:00
DL6ER
a8af2e1837
Store domains without sorting and unifying them first. This allows us to preserve the relationship of the individual domains to the lists they came from. 
Signed-off-by: DL6ER <dl6er@dl6er.de>
 2019-09-04 23:14:29 +02:00
Adam Warner
8bef5dc805
remove n from -ne 
Signed-off-by: Adam Warner <me@adamwarner.co.uk>
 2019-09-03 23:56:23 +01:00
Adam Warner
ad41bcca5a
Remove support for adblock style lists to prevent false positives 
Signed-off-by: Adam Warner <me@adamwarner.co.uk>
 2019-09-03 23:43:11 +01:00
DL6ER
ca8982494b
Store timestamp when the gravity table was last updated successfully. This fixes https://github.com/pi-hole/AdminLTE/issues/989 
Signed-off-by: DL6ER <dl6er@dl6er.de>
 2019-09-01 14:42:07 +02:00
DL6ER
1820c2c598
Merge branch 'development' into new/whitelist-regex-support 
Signed-off-by: DL6ER <dl6er@dl6er.de>
 2019-08-22 14:19:51 +02:00
Andreas
c53be459c6
quick fix for when dig also returns a CNAME 
Signed-off-by: Andreas <ryrun@online.de>
 2019-08-21 04:37:06 -07:00
DL6ER
4371c9ba03
Ensure proper permissions are set for gravity.db after creation. 
Signed-off-by: DL6ER <dl6er@dl6er.de>
 2019-08-05 21:20:07 +02:00
Andreas
63230cb72d
quick fix for when dig also returns a CNAME 
Signed-off-by: ryrun <ryrun@online.de>
 2019-08-04 21:21:08 +02:00
DL6ER
5ff9052200
Review comments 
Signed-off-by: DL6ER <dl6er@dl6er.de>
 2019-07-09 11:41:44 +02:00
DL6ER
f5121c64be
We should still add the regex lines (initially) to the regex table as the renaming will happen only after the importing. 
Signed-off-by: DL6ER <dl6er@dl6er.de>
 2019-07-08 21:39:30 +02:00
DL6ER
054c7a2c05
Create new table + view regex_whitelist + rename old regex table to regex_blacklist. This updates the gravity.db version to 3. 
Signed-off-by: DL6ER <dl6er@dl6er.de>
 2019-07-08 21:35:31 +02:00
DL6ER
be3e198f9a
Address linting errors. 
Signed-off-by: DL6ER <dl6er@dl6er.de>
 2019-07-07 10:46:20 +02:00
DL6ER
acc50b709e
Only migrate files once (domain and adlist lists druing initial creation of gravity.db auditlog.list on database upgrade from version 1 to 2. 
Signed-off-by: DL6ER <dl6er@dl6er.de>
 2019-07-07 10:33:08 +02:00
DL6ER
0405aaa3da
Review comments and fixing stickler complaints. 
Signed-off-by: DL6ER <dl6er@dl6er.de>
 2019-07-06 09:32:41 +02:00
DL6ER
2fb4256f84
Rename table to "auditlist" 
Signed-off-by: DL6ER <dl6er@dl6er.de>
 2019-07-05 16:28:36 +02:00
DL6ER
4f4a12bb40
Upgrade database if necessary and store audit domains therein. 
Signed-off-by: DL6ER <dl6er@dl6er.de>
 2019-07-05 14:03:57 +02:00
DL6ER
7a1b6fd205
Merge branch 'development' into new/gravity.db_domain_groups 2019-07-03 19:21:55 +02:00
DL6ER
f91421418a
Rename adlists -> adlist table. 
Signed-off-by: DL6ER <dl6er@dl6er.de>
 2019-06-30 23:21:10 +02:00
DL6ER
87c115fc86
Change implementation to use linking tables (domain_groups, adlists_groups). This allows adding domains to multiple lists conveniently. 
Signed-off-by: DL6ER <dl6er@dl6er.de>
 2019-06-30 19:04:38 +02:00
DL6ER
d5e8f1a781
Add 10 seconds timeout to the sqlite3 command that writes tables to the gravity database. This prevents gravity from failing when the database file is locked for a short time. 
Signed-off-by: DL6ER <dl6er@dl6er.de>
 2019-06-26 18:20:36 +02:00
DL6ER
cbc210b014
Add adlist_groups and mark group_id columns as NOT NULL. 
Signed-off-by: DL6ER <dl6er@dl6er.de>
 2019-06-16 17:05:33 +02:00
Chris Crocker-White
69dba022c4 Handle an empty local.list 
Handle the case of an empty local.list file which would otherwise prevent the system from starting

Change-type: patch
Signed-off-by: Chris Crocker-White <chriscw@balena.io>
 2019-05-27 10:27:28 -07:00
Dan Schaper
7de6bbca61
Merge branch 'development' into release/v4.3 2019-05-18 13:18:14 -07:00
Mcat12
8e1bf6e3d4
Merge branch 'development' into new/gravitydb 
# Conflicts:
#	advanced/Scripts/list.sh
#	gravity.sh
#	pihole
 2019-05-12 15:20:52 -07:00
pvogt09
3f90261520 Moves chmod after every creation of ${destination} instead of end of function 
Signed-off-by: pvogt09 <50047961+pvogt09@users.noreply.github.com>
 2019-05-12 11:25:48 +02:00
pvogt09
225285cb97 Removes chmod for ${source} and superfluous chmod for ${destination} 
Signed-off-by: pvogt09 <50047961+pvogt09@users.noreply.github.com>
 2019-05-11 11:14:07 +02:00
pvogt09
a3d2a1062f Makes blocklists world readable 
Signed-off-by: pvogt09 <50047961+pvogt09@users.noreply.github.com>
 2019-05-10 11:13:19 +02:00
Mcat12
ec06fd0ad4
Fix incorrect adlist query when an adlist is blocked during gravity 
hosts-file.net was hard-coded as the domain to check instead of the
actual domain.

Signed-off-by: Mcat12 <newtoncat12@yahoo.com>
 2019-05-04 11:35:21 -07:00
DL6ER
0cfecd31d3
Invert logic to reduce nesting 
Signed-off-by: DL6ER <dl6er@dl6er.de>
 2019-05-01 22:07:14 +02:00
DL6ER
008e88b84b
Only migrate legacy list files when we create gravity.db in the same gravity run 
Signed-off-by: DL6ER <dl6er@dl6er.de>
 2019-05-01 18:04:31 +02:00
DL6ER
540c57f755
Add comment for automigrated list entries 
Signed-off-by: DL6ER <dl6er@dl6er.de>
 2019-05-01 17:51:25 +02:00
DL6ER
983117d788
Add id (primary key, autoincrement) field to lists tables 
Signed-off-by: DL6ER <dl6er@dl6er.de>
 2019-05-01 17:47:15 +02:00
DL6ER
b372f808dd
Fix lint errors 
Signed-off-by: DL6ER <dl6er@dl6er.de>
 2019-05-01 17:06:14 +02:00
pvogt09
8a92fb24c4 Incorporates changes requested by @Mcat12 
Signed-off-by: pvogt09 <50047961+pvogt09@users.noreply.github.com>
 2019-05-01 11:20:26 +02:00
pvogt09
5c575e73c7 Adds world readable attribute to files created by Pi-hole to circumvent #2724 
Signed-off-by: pvogt09 <50047961+pvogt09@users.noreply.github.com>
 2019-04-30 22:41:12 +02:00
DL6ER
a932209143
Set default for date_modified to be the current time 
Signed-off-by: DL6ER <dl6er@dl6er.de>
 2019-04-29 17:21:59 +02:00
DL6ER
4946b5907b
Move migrated files to backup directory instead of deleting them. This has the benefit that gravity.db can be recreated at any time by: 1. deleting gravity.db, 2. copying all files from the migration_backup directory back to /etc/pihole, 3. running pihole -g again. 
Signed-off-by: DL6ER <dl6er@dl6er.de>
 2019-04-28 22:15:47 +02:00
Mark Drobnak
4596d9a577
Merge branch 'development' into new/gravitydb 2019-04-26 23:40:28 -04:00
DL6ER
8524aecfed
Don't set gravity database permissions 
Signed-off-by: DL6ER <dl6er@dl6er.de>
 2019-04-26 17:56:11 +02:00
DL6ER
3dcb9722d3
Remove global haveSourceUrls variable and just skip downloading and processing adlists if there aren't any to be downloaded 
Signed-off-by: DL6ER <dl6er@dl6er.de>
 2019-04-26 17:39:35 +02:00
DL6ER
bd0215524b
Add vw_adlists view which returns only enabled adlists 
Signed-off-by: DL6ER <dl6er@dl6er.de>
 2019-04-26 17:36:00 +02:00
DL6ER
53e3ff2b24
Only add non-empty lines when moving file content into database tables 
Signed-off-by: DL6ER <dl6er@dl6er.de>
 2019-04-26 17:32:05 +02:00
DL6ER
0efc46260f
Use both old and new strings for detecting whether this is a Pi-hole dnsmasq config file 
Signed-off-by: DL6ER <dl6er@dl6er.de>
 2019-04-26 16:16:30 +02:00
DL6ER
d94bbfca7f
Finish migrating gravity.db to using our new gravity database. This removes the --skip-download, --blacklist-only, --whitelist-only, and --wildcard-only options as it is not necessary to run gravity when modifying them from now on (gravity really only builds the gravity database and the local.list file). 
Signed-off-by: DL6ER <dl6er@dl6er.de>
 2019-04-25 11:23:06 +02:00
DL6ER
465a39427b
Source adlists from gravity database 
Signed-off-by: DL6ER <dl6er@dl6er.de>
 2019-04-25 10:46:18 +02:00
DL6ER
2664ac0efc
Improve error reporting 
Signed-off-by: DL6ER <dl6er@dl6er.de>
 2019-04-25 10:18:37 +02:00
DL6ER
c1277705d8
Copy existing whitelist.txt, blacklist.txt, regex.list, and adlists.list to the database. We remove the files afterwards as the content lives in the database now 
Signed-off-by: DL6ER <dl6er@dl6er.de>
 2019-04-25 10:14:45 +02:00
DL6ER
e57d0fb93c
Remove the empty templates gravity.list, white.list, black.list. The old source files whitelist.txt, blacklist.txt, and regex.list still exist and need to be removed in a follow-up commit 
Signed-off-by: DL6ER <dl6er@dl6er.de>
 2019-04-24 19:05:08 +02:00
DL6ER
ee20164207
Remove detect_pihole_blocking_status() as we can modify the database at any time 
Signed-off-by: DL6ER <dl6er@dl6er.de>
 2019-04-24 19:01:31 +02:00
Dan Schaper
54ae9d83b0 Stickler-lints 
Signed-off-by: Dan Schaper <dan.schaper@pi-hole.net>
 2019-02-23 13:42:15 -08:00
DL6ER
bbc392caeb
More review comments 
Signed-off-by: DL6ER <dl6er@dl6er.de>
 2019-02-22 22:49:02 +01:00
DL6ER
61840edb30
Fill comment field with NULL (default) instead of empty string and use a variable for the timestamp instead of periodically calling date 
Signed-off-by: DL6ER <dl6er@dl6er.de>
 2019-02-22 22:46:19 +01:00
DL6ER
f9250d91a5
Explictly escape the escape. 
Signed-off-by: DL6ER <dl6er@dl6er.de>
 2019-02-06 19:17:13 +01:00
DL6ER
491c828661
Improved code, ensure that gravity.sh passes shellcheck as a whole 
Signed-off-by: DL6ER <dl6er@dl6er.de>
 2019-02-06 19:09:09 +01:00
DL6ER
644ec36e64
Review comments 
Signed-off-by: DL6ER <dl6er@dl6er.de>
 2019-02-06 18:57:48 +01:00
DL6ER
ec5b16ef85
Fix incomplete comment 
Signed-off-by: DL6ER <dl6er@dl6er.de>
 2019-02-05 19:08:08 +01:00
DL6ER
dbbf21071b
Invert logic to use ENABLED instead of DISABLED. We make sure to set "enabled = 1 WHERE enabled IS NULL" in gravity_store_in_database() 
Signed-off-by: DL6ER <dl6er@dl6er.de>
 2019-02-05 19:05:11 +01:00
DL6ER
b2f4385232
Avoid infinity loop if gravity optimization fails 
Signed-off-by: DL6ER <dl6er@dl6er.de>
 2019-02-03 17:05:00 +01:00
DL6ER
1f9a7d0b0c
Only run VACUUM on "pihole -g -o" 
Signed-off-by: DL6ER <dl6er@dl6er.de>
 2019-02-03 17:01:38 +01:00
DL6ER
d5feffa117
Do not backup the database 
Signed-off-by: DL6ER <dl6er@dl6er.de>
 2019-02-03 16:45:19 +01:00
DL6ER
fc62cf7e2f
Only run VACUUM on "full" gravity runs 
Signed-off-by: DL6ER <dl6er@dl6er.de>
 2019-02-03 16:45:06 +01:00
DL6ER
93f1859bab
Store regex in gravity.db as well 
Signed-off-by: DL6ER <dl6er@dl6er.de>
 2019-02-03 15:13:18 +01:00
DL6ER
dcf0a605cf
Use views for all tables and set disabled column to false for those tables that support it 
Signed-off-by: DL6ER <dl6er@dl6er.de>
 2019-02-03 14:14:14 +01:00
DL6ER
8a91fe6992
Store domains in gravit tables: gravity, whitelist, blacklist 
Signed-off-by: DL6ER <dl6er@dl6er.de>
 2019-02-03 13:21:26 +01:00
DL6ER
09c4c88a6d
Create and fill gravity.db during pihole -g 
Signed-off-by: DL6ER <dl6er@dl6er.de>
 2019-02-03 13:04:31 +01:00
xch12i5
cd8a423b32 Solves RegEx issue and updates documentation. 
Signed-off-by: xch12i5 <40517505+xCh12i5@users.noreply.github.com>
 2019-01-20 16:09:25 +01:00
xch12i5
e02da90faa Replaces double quotes by single quotes and updates documentation. 
Signed-off-by: xch12i5 <40517505+xCh12i5@users.noreply.github.com>
 2019-01-19 17:52:28 +01:00