DL6ER
8f22203d24
Wait 30 seconds for obtaining a database lock instead of immediately failing if the database is busy.
...
Signed-off-by: DL6ER <dl6er@dl6er.de>
5 years ago
DL6ER
948f4a8827
Ensure permissions and ownership of gravity.db are correctly set on each run of pihole -g. This would have prevented https://github.com/pi-hole/AdminLTE/issues/1077
...
Signed-off-by: DL6ER <dl6er@dl6er.de>
5 years ago
DL6ER
2cec9eaf65
Merge pull request #3033 from pi-hole/fix/duplicates_in_adlists
...
Remove duplicates from adlists before importing
5 years ago
DL6ER
779fe670f7
Show full URL during gravity download instead of only domain and file
...
Signed-off-by: DL6ER <dl6er@dl6er.de>
5 years ago
DL6ER
570a7a5c11
Use sort -u instead of uniq as it is guaranteed to be safe when doing inline file operations.
...
Signed-off-by: DL6ER <dl6er@dl6er.de>
5 years ago
DL6ER
bd1b004d94
Remove possible duplicates found in lower-quality adlists
...
Signed-off-by: DL6ER <dl6er@dl6er.de>
5 years ago
Adam Warner
d29947ba32
optimise gravity list inserts
...
Signed-off-by: Adam Warner <me@adamwarner.co.uk>
5 years ago
Adam Warner
1f03faddef
shell check recomends
...
Signed-off-by: Adam Warner <me@adamwarner.co.uk>
5 years ago
DL6ER
3231e5c3ba
Address stickler requests.
...
Signed-off-by: DL6ER <dl6er@dl6er.de>
5 years ago
DL6ER
0c5185f8ba
Also display how many unique domains we have caught in the event horizon.
...
Signed-off-by: DL6ER <dl6er@dl6er.de>
5 years ago
DL6ER
037d52104a
New command "pihole -g -r" recreates gravity.db based on files backed up in /etc/pihole/migration_update. This is useful to restore a working version of the database when the user destroyed the original database. Also, update gravity.db to version 5 because of a fix we needed to implement.
...
Signed-off-by: DL6ER <dl6er@dl6er.de>
5 years ago
DL6ER
756c99653e
Merge branch 'development' into new/internal-blocking
5 years ago
Adam Warner
61a40c1b43
merge devel into 4.3.2 And Resolve merge conflicts
...
Signed-off-by: Adam Warner <me@adamwarner.co.uk>
5 years ago
DL6ER
a8af2e1837
Store domains without sorting and unifying them first. This allows us to preserve the relationship of the individual domains to the lists they came from.
...
Signed-off-by: DL6ER <dl6er@dl6er.de>
5 years ago
Adam Warner
8bef5dc805
remove n from -ne
...
Signed-off-by: Adam Warner <me@adamwarner.co.uk>
5 years ago
Adam Warner
ad41bcca5a
Remove support for adblock style lists to prevent false positives
...
Signed-off-by: Adam Warner <me@adamwarner.co.uk>
5 years ago
DL6ER
ca8982494b
Store timestamp when the gravity table was last updated successfully. This fixes https://github.com/pi-hole/AdminLTE/issues/989
...
Signed-off-by: DL6ER <dl6er@dl6er.de>
5 years ago
DL6ER
1820c2c598
Merge branch 'development' into new/whitelist-regex-support
...
Signed-off-by: DL6ER <dl6er@dl6er.de>
5 years ago
Andreas
c53be459c6
quick fix for when dig also returns a CNAME
...
Signed-off-by: Andreas <ryrun@online.de>
5 years ago
DL6ER
4371c9ba03
Ensure proper permissions are set for gravity.db after creation.
...
Signed-off-by: DL6ER <dl6er@dl6er.de>
5 years ago
Andreas
63230cb72d
quick fix for when dig also returns a CNAME
...
Signed-off-by: ryrun <ryrun@online.de>
5 years ago
DL6ER
5ff9052200
Review comments
...
Signed-off-by: DL6ER <dl6er@dl6er.de>
5 years ago
DL6ER
f5121c64be
We should still add the regex lines (initially) to the regex table as the renaming will happen only after the importing.
...
Signed-off-by: DL6ER <dl6er@dl6er.de>
5 years ago
DL6ER
054c7a2c05
Create new table + view regex_whitelist + rename old regex table to regex_blacklist. This updates the gravity.db version to 3.
...
Signed-off-by: DL6ER <dl6er@dl6er.de>
5 years ago
DL6ER
be3e198f9a
Address linting errors.
...
Signed-off-by: DL6ER <dl6er@dl6er.de>
5 years ago
DL6ER
acc50b709e
Only migrate files once (domain and adlist lists druing initial creation of gravity.db auditlog.list on database upgrade from version 1 to 2.
...
Signed-off-by: DL6ER <dl6er@dl6er.de>
5 years ago
DL6ER
0405aaa3da
Review comments and fixing stickler complaints.
...
Signed-off-by: DL6ER <dl6er@dl6er.de>
5 years ago
DL6ER
2fb4256f84
Rename table to "auditlist"
...
Signed-off-by: DL6ER <dl6er@dl6er.de>
5 years ago
DL6ER
4f4a12bb40
Upgrade database if necessary and store audit domains therein.
...
Signed-off-by: DL6ER <dl6er@dl6er.de>
5 years ago
DL6ER
7a1b6fd205
Merge branch 'development' into new/gravity.db_domain_groups
5 years ago
DL6ER
f91421418a
Rename adlists -> adlist table.
...
Signed-off-by: DL6ER <dl6er@dl6er.de>
5 years ago
DL6ER
87c115fc86
Change implementation to use linking tables (domain_groups, adlists_groups). This allows adding domains to multiple lists conveniently.
...
Signed-off-by: DL6ER <dl6er@dl6er.de>
5 years ago
DL6ER
d5e8f1a781
Add 10 seconds timeout to the sqlite3 command that writes tables to the gravity database. This prevents gravity from failing when the database file is locked for a short time.
...
Signed-off-by: DL6ER <dl6er@dl6er.de>
5 years ago
DL6ER
cbc210b014
Add adlist_groups and mark group_id columns as NOT NULL.
...
Signed-off-by: DL6ER <dl6er@dl6er.de>
5 years ago
Chris Crocker-White
69dba022c4
Handle an empty local.list
...
Handle the case of an empty local.list file which would otherwise prevent the system from starting
Change-type: patch
Signed-off-by: Chris Crocker-White <chriscw@balena.io>
5 years ago
Dan Schaper
7de6bbca61
Merge branch 'development' into release/v4.3
5 years ago
Mcat12
8e1bf6e3d4
Merge branch 'development' into new/gravitydb
...
# Conflicts:
# advanced/Scripts/list.sh
# gravity.sh
# pihole
5 years ago
pvogt09
3f90261520
Moves chmod after every creation of ${destination} instead of end of function
...
Signed-off-by: pvogt09 <50047961+pvogt09@users.noreply.github.com>
5 years ago
pvogt09
225285cb97
Removes chmod for ${source} and superfluous chmod for ${destination}
...
Signed-off-by: pvogt09 <50047961+pvogt09@users.noreply.github.com>
5 years ago
pvogt09
a3d2a1062f
Makes blocklists world readable
...
Signed-off-by: pvogt09 <50047961+pvogt09@users.noreply.github.com>
5 years ago
Mcat12
ec06fd0ad4
Fix incorrect adlist query when an adlist is blocked during gravity
...
hosts-file.net was hard-coded as the domain to check instead of the
actual domain.
Signed-off-by: Mcat12 <newtoncat12@yahoo.com>
5 years ago
DL6ER
0cfecd31d3
Invert logic to reduce nesting
...
Signed-off-by: DL6ER <dl6er@dl6er.de>
5 years ago
DL6ER
008e88b84b
Only migrate legacy list files when we create gravity.db in the same gravity run
...
Signed-off-by: DL6ER <dl6er@dl6er.de>
5 years ago
DL6ER
540c57f755
Add comment for automigrated list entries
...
Signed-off-by: DL6ER <dl6er@dl6er.de>
5 years ago
DL6ER
983117d788
Add id (primary key, autoincrement) field to lists tables
...
Signed-off-by: DL6ER <dl6er@dl6er.de>
5 years ago
DL6ER
b372f808dd
Fix lint errors
...
Signed-off-by: DL6ER <dl6er@dl6er.de>
5 years ago
pvogt09
8a92fb24c4
Incorporates changes requested by @Mcat12
...
Signed-off-by: pvogt09 <50047961+pvogt09@users.noreply.github.com>
5 years ago
pvogt09
5c575e73c7
Adds world readable attribute to files created by Pi-hole to circumvent #2724
...
Signed-off-by: pvogt09 <50047961+pvogt09@users.noreply.github.com>
5 years ago
DL6ER
a932209143
Set default for date_modified to be the current time
...
Signed-off-by: DL6ER <dl6er@dl6er.de>
5 years ago
DL6ER
4946b5907b
Move migrated files to backup directory instead of deleting them. This has the benefit that gravity.db can be recreated at any time by: 1. deleting gravity.db, 2. copying all files from the migration_backup directory back to /etc/pihole, 3. running pihole -g again.
...
Signed-off-by: DL6ER <dl6er@dl6er.de>
5 years ago
Mark Drobnak
4596d9a577
Merge branch 'development' into new/gravitydb
6 years ago
DL6ER
8524aecfed
Don't set gravity database permissions
...
Signed-off-by: DL6ER <dl6er@dl6er.de>
6 years ago
DL6ER
3dcb9722d3
Remove global haveSourceUrls variable and just skip downloading and processing adlists if there aren't any to be downloaded
...
Signed-off-by: DL6ER <dl6er@dl6er.de>
6 years ago
DL6ER
bd0215524b
Add vw_adlists view which returns only enabled adlists
...
Signed-off-by: DL6ER <dl6er@dl6er.de>
6 years ago
DL6ER
53e3ff2b24
Only add non-empty lines when moving file content into database tables
...
Signed-off-by: DL6ER <dl6er@dl6er.de>
6 years ago
DL6ER
0efc46260f
Use both old and new strings for detecting whether this is a Pi-hole dnsmasq config file
...
Signed-off-by: DL6ER <dl6er@dl6er.de>
6 years ago
DL6ER
d94bbfca7f
Finish migrating gravity.db to using our new gravity database. This removes the --skip-download, --blacklist-only, --whitelist-only, and --wildcard-only options as it is not necessary to run gravity when modifying them from now on (gravity really only builds the gravity database and the local.list file).
...
Signed-off-by: DL6ER <dl6er@dl6er.de>
6 years ago
DL6ER
465a39427b
Source adlists from gravity database
...
Signed-off-by: DL6ER <dl6er@dl6er.de>
6 years ago
DL6ER
2664ac0efc
Improve error reporting
...
Signed-off-by: DL6ER <dl6er@dl6er.de>
6 years ago
DL6ER
c1277705d8
Copy existing whitelist.txt, blacklist.txt, regex.list, and adlists.list to the database. We remove the files afterwards as the content lives in the database now
...
Signed-off-by: DL6ER <dl6er@dl6er.de>
6 years ago
DL6ER
e57d0fb93c
Remove the empty templates gravity.list, white.list, black.list. The old source files whitelist.txt, blacklist.txt, and regex.list still exist and need to be removed in a follow-up commit
...
Signed-off-by: DL6ER <dl6er@dl6er.de>
6 years ago
DL6ER
ee20164207
Remove detect_pihole_blocking_status() as we can modify the database at any time
...
Signed-off-by: DL6ER <dl6er@dl6er.de>
6 years ago
Dan Schaper
54ae9d83b0
Stickler-lints
...
Signed-off-by: Dan Schaper <dan.schaper@pi-hole.net>
6 years ago
DL6ER
bbc392caeb
More review comments
...
Signed-off-by: DL6ER <dl6er@dl6er.de>
6 years ago
DL6ER
61840edb30
Fill comment field with NULL (default) instead of empty string and use a variable for the timestamp instead of periodically calling date
...
Signed-off-by: DL6ER <dl6er@dl6er.de>
6 years ago
DL6ER
f9250d91a5
Explictly escape the escape.
...
Signed-off-by: DL6ER <dl6er@dl6er.de>
6 years ago
DL6ER
491c828661
Improved code, ensure that gravity.sh passes shellcheck as a whole
...
Signed-off-by: DL6ER <dl6er@dl6er.de>
6 years ago
DL6ER
644ec36e64
Review comments
...
Signed-off-by: DL6ER <dl6er@dl6er.de>
6 years ago
DL6ER
ec5b16ef85
Fix incomplete comment
...
Signed-off-by: DL6ER <dl6er@dl6er.de>
6 years ago
DL6ER
dbbf21071b
Invert logic to use ENABLED instead of DISABLED. We make sure to set "enabled = 1 WHERE enabled IS NULL" in gravity_store_in_database()
...
Signed-off-by: DL6ER <dl6er@dl6er.de>
6 years ago
DL6ER
b2f4385232
Avoid infinity loop if gravity optimization fails
...
Signed-off-by: DL6ER <dl6er@dl6er.de>
6 years ago
DL6ER
1f9a7d0b0c
Only run VACUUM on "pihole -g -o"
...
Signed-off-by: DL6ER <dl6er@dl6er.de>
6 years ago
DL6ER
d5feffa117
Do not backup the database
...
Signed-off-by: DL6ER <dl6er@dl6er.de>
6 years ago
DL6ER
fc62cf7e2f
Only run VACUUM on "full" gravity runs
...
Signed-off-by: DL6ER <dl6er@dl6er.de>
6 years ago
DL6ER
93f1859bab
Store regex in gravity.db as well
...
Signed-off-by: DL6ER <dl6er@dl6er.de>
6 years ago
DL6ER
dcf0a605cf
Use views for all tables and set disabled column to false for those tables that support it
...
Signed-off-by: DL6ER <dl6er@dl6er.de>
6 years ago
DL6ER
8a91fe6992
Store domains in gravit tables: gravity, whitelist, blacklist
...
Signed-off-by: DL6ER <dl6er@dl6er.de>
6 years ago
DL6ER
09c4c88a6d
Create and fill gravity.db during pihole -g
...
Signed-off-by: DL6ER <dl6er@dl6er.de>
6 years ago
xch12i5
cd8a423b32
Solves RegEx issue and updates documentation.
...
Signed-off-by: xch12i5 <40517505+xCh12i5@users.noreply.github.com>
6 years ago
xch12i5
e02da90faa
Replaces double quotes by single quotes and updates documentation.
...
Signed-off-by: xch12i5 <40517505+xCh12i5@users.noreply.github.com>
6 years ago
xch12i5
0de5132e2b
Solves RegEx issue.
...
Signed-off-by: xch12i5 <40517505+xCh12i5@users.noreply.github.com>
6 years ago
DL6ER
05bd88b4d8
Update agent string: the selected string is the most abundant one on 06 Dec 2018 with a percentage of almost 10%
...
Signed-off-by: DL6ER <dl6er@dl6er.de>
6 years ago
Adam Warner
83f7bd66e6
Don't `mv` `preEventHorizon`, copy it.
...
Signed-off-by: Adam Warner <adamw@rner.email>
6 years ago
Dan Schaper
5c621fa253
Lints
...
Signed-off-by: Dan Schaper <dan.schaper@pi-hole.net>
6 years ago
Dan Schaper
f9e71bee0a
Account for port, or lack of, in PIHOLE_DNS_1
...
Signed-off-by: Dan Schaper <dan.schaper@pi-hole.net>
6 years ago
Mark Drobnak
33939a30ea
Merge pull request #2347 from Gusher123/patch-1
...
Don't let one blocklist block an other blocklist
6 years ago
Adam Warner
0131055ec9
Submission stale, addressing @mcat12's comments
...
Signed-off-by: Adam Warner <adamw@rner.email>
6 years ago
DL6ER
337cc5ca18
BLOCKING -> BLOCKING_ENABLED
...
Signed-off-by: DL6ER <dl6er@dl6er.de>
6 years ago
DL6ER
b011adc453
Use new setupVars BLOCKING variable to determine which file to update via gravity
...
Signed-off-by: DL6ER <dl6er@dl6er.de>
6 years ago
DL6ER
6cde066edd
Have gravity detect the presence of files like gravity.list.bck and update this file if present (assuming blocking is disabled)
...
Signed-off-by: DL6ER <dl6er@dl6er.de>
6 years ago
Gusher123
2feb5f22cc
Signed-off-by
...
This is my commit message
Signed-off-by: Gusher123 <t.siebeling@gmail.com>
6 years ago
Gusher123
03a9d3bc5b
Not hardcoding dns and covering all blockingmodes
...
Using double brackets
Mode double brackets
6 years ago
Gusher123
e58142b4ac
Added comments
6 years ago
Gusher123
3e5c6afaee
Don't let one blocklist, block an other
...
If a blocklist is blocked by an other and 'pihole -g' is run, update the blocked blocklist by useing an alternate dns server (in this case 1.1.1.1) to acquire the IP adres and download the blocklist using the 'curl --resolve' (https://curl.haxx.se/docs/manpage.html#--resolve )
6 years ago
DL6ER
760dd01b50
Use grep invert-match
...
Signed-off-by: DL6ER <dl6er@dl6er.de>
6 years ago
DL6ER
5bf90cb77d
Use alternative logic for counting regex file entries, skipping commented out lines
...
Signed-off-by: DL6ER <dl6er@dl6er.de>
6 years ago
Mcat12
1a8d5fb4a9
Allow PHP to edit the regex file if installed
...
Fixes permissions error on the web interface when modifying the regex.
Signed-off-by: Mcat12 <newtoncat12@yahoo.com>
6 years ago
Dan Schaper
dbc82cfb6a
Merge pull request #2236 from pi-hole/new/regex-lists
...
Use regex lists instead of wildcards for blocking
6 years ago
Mcat12
72be7ad5f0
Don't count commented out regex lines
...
Signed-off-by: Mcat12 <newtoncat12@yahoo.com>
6 years ago
WaLLy3K
7ddae8f2eb
Treat URLs without a protocol as HTTP
...
Signed off by WaLLy3K <wally3k@pi-hole.net>
6 years ago