DL6ER
d14ee26d6a
Merge pull request #3139 from pi-hole/fix/count_before_calling_FTL
...
Fix wrong number of blocking domains shown on the dashboard
4 years ago
DL6ER
52398052e9
Compute number of domains (and store it in the database) BEFORE calling FTL to re-read said value.
...
Signed-off-by: DL6ER <dl6er@dl6er.de>
4 years ago
DL6ER
601f9048cd
Merge pull request #3130 from pi-hole/fix/gravity_updated_timestamp
...
Store gravity update timestamp only after database swapping
4 years ago
DL6ER
dc2fce8e1d
Store gravity update timestamp only after database swapping.
...
Signed-off-by: DL6ER <dl6er@dl6er.de>
4 years ago
DL6ER
2a5cf221fa
Store number of distinct gravity domains in database after counting.
...
Signed-off-by: DL6ER <dl6er@dl6er.de>
4 years ago
DL6ER
10c2dad48a
Improve gravity performance ( #3100 )
...
* Gravity performance improvements.
Signed-off-by: DL6ER <dl6er@dl6er.de>
* Do not move downloaded lists into migration_backup directory.
Signed-off-by: DL6ER <dl6er@dl6er.de>
* Do not (strictly) sort domains. Random-leaf access is faster than always-last-leaf access (on average).
Signed-off-by: DL6ER <dl6er@dl6er.de>
* Append instead of overwrite gravity_new collection list.
Signed-off-by: DL6ER <dl6er@dl6er.de>
* Rename table gravity_new to gravity_temp to clarify that this is only an intermediate table.
Signed-off-by: DL6ER <dl6er@dl6er.de>
* Add timers for each of the calls to compute intense parts. They are to be removed before this finally hits the release/v5.0 branch.
Signed-off-by: DL6ER <dl6er@dl6er.de>
* Fix legacy list files import. It currently doesn't work when the gravity database has already been updated to using the single domainlist table.
Signed-off-by: DL6ER <dl6er@dl6er.de>
* Simplify database_table_from_file(), remove all to this function for gravity lost downloads.
Signed-off-by: DL6ER <dl6er@dl6er.de>
* Update gravity.db.sql to version 10 to have newle created databases already reflect the most recent state.
Signed-off-by: DL6ER <dl6er@dl6er.de>
* Create second gravity database and swap them on success. This has a number of advantages such as instantaneous gravity updates (as seen from FTL) and always available gravity blocking. Furthermore, this saves disk space as the old database is removed on completion.
* Add timing output for the database swapping SQLite3 call.
Signed-off-by: DL6ER <dl6er@dl6er.de>
* Explicitly generate index as a separate process.
Signed-off-by: DL6ER <dl6er@dl6er.de>
* Remove time measurements.
Signed-off-by: DL6ER <dl6er@dl6er.de>
4 years ago
DL6ER
8f22203d24
Wait 30 seconds for obtaining a database lock instead of immediately failing if the database is busy.
...
Signed-off-by: DL6ER <dl6er@dl6er.de>
4 years ago
DL6ER
948f4a8827
Ensure permissions and ownership of gravity.db are correctly set on each run of pihole -g. This would have prevented https://github.com/pi-hole/AdminLTE/issues/1077
...
Signed-off-by: DL6ER <dl6er@dl6er.de>
5 years ago
DL6ER
2cec9eaf65
Merge pull request #3033 from pi-hole/fix/duplicates_in_adlists
...
Remove duplicates from adlists before importing
5 years ago
DL6ER
779fe670f7
Show full URL during gravity download instead of only domain and file
...
Signed-off-by: DL6ER <dl6er@dl6er.de>
5 years ago
DL6ER
570a7a5c11
Use sort -u instead of uniq as it is guaranteed to be safe when doing inline file operations.
...
Signed-off-by: DL6ER <dl6er@dl6er.de>
5 years ago
DL6ER
bd1b004d94
Remove possible duplicates found in lower-quality adlists
...
Signed-off-by: DL6ER <dl6er@dl6er.de>
5 years ago
Adam Warner
d29947ba32
optimise gravity list inserts
...
Signed-off-by: Adam Warner <me@adamwarner.co.uk>
5 years ago
Adam Warner
1f03faddef
shell check recomends
...
Signed-off-by: Adam Warner <me@adamwarner.co.uk>
5 years ago
DL6ER
3231e5c3ba
Address stickler requests.
...
Signed-off-by: DL6ER <dl6er@dl6er.de>
5 years ago
DL6ER
0c5185f8ba
Also display how many unique domains we have caught in the event horizon.
...
Signed-off-by: DL6ER <dl6er@dl6er.de>
5 years ago
DL6ER
037d52104a
New command "pihole -g -r" recreates gravity.db based on files backed up in /etc/pihole/migration_update. This is useful to restore a working version of the database when the user destroyed the original database. Also, update gravity.db to version 5 because of a fix we needed to implement.
...
Signed-off-by: DL6ER <dl6er@dl6er.de>
5 years ago
DL6ER
756c99653e
Merge branch 'development' into new/internal-blocking
5 years ago
Adam Warner
61a40c1b43
merge devel into 4.3.2 And Resolve merge conflicts
...
Signed-off-by: Adam Warner <me@adamwarner.co.uk>
5 years ago
DL6ER
a8af2e1837
Store domains without sorting and unifying them first. This allows us to preserve the relationship of the individual domains to the lists they came from.
...
Signed-off-by: DL6ER <dl6er@dl6er.de>
5 years ago
Adam Warner
8bef5dc805
remove n from -ne
...
Signed-off-by: Adam Warner <me@adamwarner.co.uk>
5 years ago
Adam Warner
ad41bcca5a
Remove support for adblock style lists to prevent false positives
...
Signed-off-by: Adam Warner <me@adamwarner.co.uk>
5 years ago
DL6ER
ca8982494b
Store timestamp when the gravity table was last updated successfully. This fixes https://github.com/pi-hole/AdminLTE/issues/989
...
Signed-off-by: DL6ER <dl6er@dl6er.de>
5 years ago
DL6ER
1820c2c598
Merge branch 'development' into new/whitelist-regex-support
...
Signed-off-by: DL6ER <dl6er@dl6er.de>
5 years ago
Andreas
c53be459c6
quick fix for when dig also returns a CNAME
...
Signed-off-by: Andreas <ryrun@online.de>
5 years ago
DL6ER
4371c9ba03
Ensure proper permissions are set for gravity.db after creation.
...
Signed-off-by: DL6ER <dl6er@dl6er.de>
5 years ago
Andreas
63230cb72d
quick fix for when dig also returns a CNAME
...
Signed-off-by: ryrun <ryrun@online.de>
5 years ago
DL6ER
5ff9052200
Review comments
...
Signed-off-by: DL6ER <dl6er@dl6er.de>
5 years ago
DL6ER
f5121c64be
We should still add the regex lines (initially) to the regex table as the renaming will happen only after the importing.
...
Signed-off-by: DL6ER <dl6er@dl6er.de>
5 years ago
DL6ER
054c7a2c05
Create new table + view regex_whitelist + rename old regex table to regex_blacklist. This updates the gravity.db version to 3.
...
Signed-off-by: DL6ER <dl6er@dl6er.de>
5 years ago
DL6ER
be3e198f9a
Address linting errors.
...
Signed-off-by: DL6ER <dl6er@dl6er.de>
5 years ago
DL6ER
acc50b709e
Only migrate files once (domain and adlist lists druing initial creation of gravity.db auditlog.list on database upgrade from version 1 to 2.
...
Signed-off-by: DL6ER <dl6er@dl6er.de>
5 years ago
DL6ER
0405aaa3da
Review comments and fixing stickler complaints.
...
Signed-off-by: DL6ER <dl6er@dl6er.de>
5 years ago
DL6ER
2fb4256f84
Rename table to "auditlist"
...
Signed-off-by: DL6ER <dl6er@dl6er.de>
5 years ago
DL6ER
4f4a12bb40
Upgrade database if necessary and store audit domains therein.
...
Signed-off-by: DL6ER <dl6er@dl6er.de>
5 years ago
DL6ER
7a1b6fd205
Merge branch 'development' into new/gravity.db_domain_groups
5 years ago
DL6ER
f91421418a
Rename adlists -> adlist table.
...
Signed-off-by: DL6ER <dl6er@dl6er.de>
5 years ago
DL6ER
87c115fc86
Change implementation to use linking tables (domain_groups, adlists_groups). This allows adding domains to multiple lists conveniently.
...
Signed-off-by: DL6ER <dl6er@dl6er.de>
5 years ago
DL6ER
d5e8f1a781
Add 10 seconds timeout to the sqlite3 command that writes tables to the gravity database. This prevents gravity from failing when the database file is locked for a short time.
...
Signed-off-by: DL6ER <dl6er@dl6er.de>
5 years ago
DL6ER
cbc210b014
Add adlist_groups and mark group_id columns as NOT NULL.
...
Signed-off-by: DL6ER <dl6er@dl6er.de>
5 years ago
Chris Crocker-White
69dba022c4
Handle an empty local.list
...
Handle the case of an empty local.list file which would otherwise prevent the system from starting
Change-type: patch
Signed-off-by: Chris Crocker-White <chriscw@balena.io>
5 years ago
Dan Schaper
7de6bbca61
Merge branch 'development' into release/v4.3
5 years ago
Mcat12
8e1bf6e3d4
Merge branch 'development' into new/gravitydb
...
# Conflicts:
# advanced/Scripts/list.sh
# gravity.sh
# pihole
5 years ago
pvogt09
3f90261520
Moves chmod after every creation of ${destination} instead of end of function
...
Signed-off-by: pvogt09 <50047961+pvogt09@users.noreply.github.com>
5 years ago
pvogt09
225285cb97
Removes chmod for ${source} and superfluous chmod for ${destination}
...
Signed-off-by: pvogt09 <50047961+pvogt09@users.noreply.github.com>
5 years ago
pvogt09
a3d2a1062f
Makes blocklists world readable
...
Signed-off-by: pvogt09 <50047961+pvogt09@users.noreply.github.com>
5 years ago
Mcat12
ec06fd0ad4
Fix incorrect adlist query when an adlist is blocked during gravity
...
hosts-file.net was hard-coded as the domain to check instead of the
actual domain.
Signed-off-by: Mcat12 <newtoncat12@yahoo.com>
5 years ago
DL6ER
0cfecd31d3
Invert logic to reduce nesting
...
Signed-off-by: DL6ER <dl6er@dl6er.de>
5 years ago
DL6ER
008e88b84b
Only migrate legacy list files when we create gravity.db in the same gravity run
...
Signed-off-by: DL6ER <dl6er@dl6er.de>
5 years ago
DL6ER
540c57f755
Add comment for automigrated list entries
...
Signed-off-by: DL6ER <dl6er@dl6er.de>
5 years ago