Commit Graph

509 Commits (master)

Author SHA1 Message Date
DL6ER f0f5cc52d9
Use internal SQLite3 engine in more places in gravity.sh
2 years ago
DL6ER ce86157067
Fix gravity in case there are no adlists at all or all are disabled (#4535)
2 years ago
DL6ER 3d3bb45a46
Merge pull request #4288 from pi-hole/new/gravity_repair
2 years ago
DL6ER d2a98ae954
Document -r recover force case
2 years ago
DL6ER 2e1ce7fc87
Apply suggestions from code review
2 years ago
DL6ER 54c58327f1
Merge pull request #4450 from pi-hole/unblock_NODATA
2 years ago
yubiuser db5e94b14a
use +short and omit obsolet awk
2 years ago
DL6ER 7167e6d5e4
Apply suggestions from code review
2 years ago
DL6ER b06efb6ab7
Declare variables local
2 years ago
DL6ER ab4bce4787
Allow users to force recovery even when checks are okay using "pihole -g -r recover force"
2 years ago
DL6ER 469c179b32
Return early from recovery routine when integrity checks didn't show any database errors.
2 years ago
DL6ER 190ab79606
Implement fully-automated gravity database recovery method.
2 years ago
Dan Schaper 533a77d6d5
Add database function failure guards.
2 years ago
Dan Schaper 76ae75689c
Check for DNS before run.
2 years ago
Dan Schaper a780fc59e2
Set DBFile permissions on creation.
2 years ago
Christian König e243c562c2
Unblock adlist domain during gravity run in NODATA mode
3 years ago
Christian König 647ba6ec9d
Rename variables to improve comprehensibility
3 years ago
Christian König bfda52ed79
Fix number of invalid domains
3 years ago
jbzdarkid bc8150adfa Clean up bash script formatting
3 years ago
yubiuser 38bb4a4908
Remove unused wildcard_regex_converter.sh (#4369)
3 years ago
jpgpi250 3439045228 fix Pi-hole v5.4 update - syntax error #4322
3 years ago
Adam Warner 4ac4bc5c18
Merge branch 'development' into fix/gravity_no_locallist
3 years ago
jpgpi250 a2d5b8050a Update gravity.sh
3 years ago
DL6ER f90dffbf1a
Do not put hostname and pi.hole into local.list - this will be handled by FTL after FTL#1111 has been merged
3 years ago
DL6ER f536718aaa
Use possibly custom gravity.db location in the copying SQL script
3 years ago
DL6ER d25240fe9f
pihole -g should respose GRAVITYDB in pihole-FTL.conf
3 years ago
DL6ER 126b9ae381
Add missing three new columns to pihole -g -r
3 years ago
DL6ER 57fe3b6621
Merge pull request #3951 from pi-hole/new/gravity_adlist_infos
3 years ago
Adam Warner 9ec012289f
Merge pull request #4017 from mwoolweaver/patch-2
3 years ago
Michael Woolweaver c42b97ddb2
fix typo in comment
3 years ago
freddii 7c0c30fb0b fixed typos
3 years ago
DL6ER 1791fe22f6
Merge pull request #3912 from mdujava/basic_auth
3 years ago
DL6ER a2625df5e2
Remove "The list is new" comment because it is superfluous
3 years ago
DL6ER 0944807491
Actually store correct number of domains for the individual lists (and not the sum of the so far collected number of domains)
3 years ago
DL6ER a216848c1d
Only update time of last list change when we see a list for the first time or when it really changed content
3 years ago
DL6ER 8c56f54a1e
Compare checksum of downloaded list against older checksums to see if the list content changed since the last download
3 years ago
DL6ER 6b33b8b4c0
Store status of downloaded list (downloaded, using cache, some error, ...) and number of (in-)valid domains on this list in the gravity database. This updates the gravity databaes to version 14.
3 years ago
Matej Dujava ef0bdf6470 Fix validation of adlist url
3 years ago
user 922f795fe6 Adding comma to the adlist table format
4 years ago
Adam Warner 948337dd8e
Merge pull request #3740 from pi-hole/new/adlist_date_updated
4 years ago
DL6ER 4fd412d7c3
Add date_updated field in adlist table set when a list changes.
4 years ago
DL6ER aa7c3b6852
Remove gravity optimization. Further investigations have shown that it is useless because the standard gravity run already produces an optimal database.
4 years ago
Adam Warner bb1a6243d6
Merge pull request #3669 from pi-hole/release/v5.1.2
4 years ago
Christian König 27399a762a Check for compression onyl once and print result
4 years ago
DL6ER 87da9084e6
Use compression (if available) when downloading the ad lists.
4 years ago
DL6ER 527fc3c5cf
Also pidof -> pgrep in gravity.sh
4 years ago
Christian König 73963fecda Use gravity's adlist_id in filename when saving downloaded adlist locally
4 years ago
Peter Lowe dd205c55c1 Updated gravity.sh to allow semicolons
4 years ago
Adam Warner 9e96ff37de
Fix merge conflicts
4 years ago
DL6ER 288d487fc0
Allow ( and ) in adlist URLs.
4 years ago
DL6ER fa57c457f3
Boldify number of unique domains as this is the actually interesting number
4 years ago
Dan Schaper 0f20470a38
Merge pull request #3269 from pi-hole/tweak/hosts-comments
4 years ago
DL6ER 982c1b0059
Merge branch 'release/v5.0' into development
4 years ago
Adam Warner 7d19ee1b25
validate blocklist URL before adding to the database (#3237)
4 years ago
DL6ER 7b15a88dc4
Strip comments from downloaded lists instead of discarding lines with comments altogether
4 years ago
XhmikosR 7b8611ced0
Assorted typo fixes. (#3126)
4 years ago
DL6ER 3dd05606ca
Call it the received number of domains instead of the imported number as importing does only happen a bit later. Only show the number of invalid domains if there are invalid domains.
4 years ago
DL6ER 1e8bfd33f5
Improve output
4 years ago
DL6ER 8131b5961c
Add comments to the code describing the changes.
4 years ago
DL6ER 81d4531e10
Implement performant list checking routine.
4 years ago
DL6ER 050e2963c7
Remove redundant code.
4 years ago
DL6ER 3c09cd4a3a
Experimental output of matching line from shown warnings.
4 years ago
DL6ER 839fe32042
Fix issue with missing newline at the end of adlists (#3144)
4 years ago
DL6ER d14ee26d6a
Merge pull request #3139 from pi-hole/fix/count_before_calling_FTL
4 years ago
DL6ER 52398052e9
Compute number of domains (and store it in the database) BEFORE calling FTL to re-read said value.
4 years ago
DL6ER 601f9048cd
Merge pull request #3130 from pi-hole/fix/gravity_updated_timestamp
4 years ago
DL6ER dc2fce8e1d
Store gravity update timestamp only after database swapping.
4 years ago
DL6ER 2a5cf221fa
Store number of distinct gravity domains in database after counting.
4 years ago
DL6ER 10c2dad48a Improve gravity performance (#3100)
4 years ago
DL6ER 8f22203d24
Wait 30 seconds for obtaining a database lock instead of immediately failing if the database is busy.
4 years ago
DL6ER 948f4a8827
Ensure permissions and ownership of gravity.db are correctly set on each run of pihole -g. This would have prevented https://github.com/pi-hole/AdminLTE/issues/1077
4 years ago
DL6ER 2cec9eaf65
Merge pull request #3033 from pi-hole/fix/duplicates_in_adlists
5 years ago
DL6ER 779fe670f7
Show full URL during gravity download instead of only domain and file
5 years ago
DL6ER 570a7a5c11
Use sort -u instead of uniq as it is guaranteed to be safe when doing inline file operations.
5 years ago
DL6ER bd1b004d94
Remove possible duplicates found in lower-quality adlists
5 years ago
Adam Warner d29947ba32
optimise gravity list inserts
5 years ago
Adam Warner 1f03faddef
shell check recomends
5 years ago
DL6ER 3231e5c3ba
Address stickler requests.
5 years ago
DL6ER 0c5185f8ba
Also display how many unique domains we have caught in the event horizon.
5 years ago
DL6ER 037d52104a
New command "pihole -g -r" recreates gravity.db based on files backed up in /etc/pihole/migration_update. This is useful to restore a working version of the database when the user destroyed the original database. Also, update gravity.db to version 5 because of a fix we needed to implement.
5 years ago
DL6ER 756c99653e
Merge branch 'development' into new/internal-blocking
5 years ago
Adam Warner 61a40c1b43
merge devel into 4.3.2 And Resolve merge conflicts
5 years ago
DL6ER a8af2e1837
Store domains without sorting and unifying them first. This allows us to preserve the relationship of the individual domains to the lists they came from.
5 years ago
Adam Warner 8bef5dc805
remove n from -ne
5 years ago
Adam Warner ad41bcca5a
Remove support for adblock style lists to prevent false positives
5 years ago
DL6ER ca8982494b
Store timestamp when the gravity table was last updated successfully. This fixes https://github.com/pi-hole/AdminLTE/issues/989
5 years ago
DL6ER 1820c2c598
Merge branch 'development' into new/whitelist-regex-support
5 years ago
Andreas c53be459c6
quick fix for when dig also returns a CNAME
5 years ago
DL6ER 4371c9ba03
Ensure proper permissions are set for gravity.db after creation.
5 years ago
Andreas 63230cb72d
quick fix for when dig also returns a CNAME
5 years ago
DL6ER 5ff9052200
Review comments
5 years ago
DL6ER f5121c64be
We should still add the regex lines (initially) to the regex table as the renaming will happen only after the importing.
5 years ago
DL6ER 054c7a2c05
Create new table + view regex_whitelist + rename old regex table to regex_blacklist. This updates the gravity.db version to 3.
5 years ago
DL6ER be3e198f9a
Address linting errors.
5 years ago
DL6ER acc50b709e
Only migrate files once (domain and adlist lists druing initial creation of gravity.db auditlog.list on database upgrade from version 1 to 2.
5 years ago
DL6ER 0405aaa3da
Review comments and fixing stickler complaints.
5 years ago
DL6ER 2fb4256f84
Rename table to "auditlist"
5 years ago
DL6ER 4f4a12bb40
Upgrade database if necessary and store audit domains therein.
5 years ago
DL6ER 7a1b6fd205
Merge branch 'development' into new/gravity.db_domain_groups
5 years ago
DL6ER f91421418a
Rename adlists -> adlist table.
5 years ago