Commit Graph

524 Commits

Author SHA1 Message Date
Jimmy Zelinskie
6864a8efea versionfmt: init rpm versionfmt 2016-12-30 12:51:25 -05:00
Jimmy Zelinskie
033709eaea add registerable version formats
Since we only ever used dpkg, this change shims everything into using
dpkg.
2016-12-30 12:51:24 -05:00
Jimmy Zelinskie
7106f1c524 Merge pull request #299 from alexei-led/master
fix error when scanning folders for feteched update for Alpine vulnerabilities
2016-12-30 12:48:29 -05:00
Alexei Ledenev
7ec9225bdc fix error scanning folders for feteched Alpine vulnerabilities 2016-12-30 15:36:09 +02:00
Jimmy Zelinskie
3897fb6706 Merge pull request #295 from jzelinskie/fixmigrationorder
psql/migrations: fix ordering
2016-12-25 19:31:49 -05:00
Jimmy Zelinskie
9338f28e82 psql/migrations: fix ordering 2016-12-25 19:25:57 -05:00
Jimmy Zelinskie
7d3d1861d0 Merge pull request #290 from Djelibeybi/oraclelinux-support
Oracle Linux support
2016-12-19 20:58:17 -05:00
Avi Miller
2643d22aaa Updated fetcher and tests to close the file handles and HTTP response.
Signed-off-by: Avi Miller <avi.miller@oracle.com>
2016-12-20 12:14:10 +11:00
Avi Miller
a3c2dae790 Updated README with Oracle Linux security information.
Signed-off-by: Avi Miller <avi.miller@oracle.com>
2016-12-20 11:48:37 +11:00
Avi Miller
9d885f680c Add Oracle Linux fetcher to grab and parse OVAL data.
Signed-off-by: Avi Miller <avi.miller@oracle.com>
2016-12-20 11:25:07 +11:00
Jimmy Zelinskie
9532c03f95 Merge pull request #288 from jzelinskie/200mb
worker: clarify maxFileSize purpose
2016-12-19 19:19:11 -05:00
Jimmy Zelinskie
de1f09e8b3 worker: clarify maxFileSize purpose
Fixes #237.
2016-12-19 19:02:13 -05:00
Avi Miller
5eb57fee37 Update osrelease and redhatrelease detectors to detect Oracle Linux as well. 2016-12-20 10:53:30 +11:00
Jimmy Zelinskie
8e1fe0d01f Merge pull request #289 from jzelinskie/revert-suse
Revert OpenSUSE
2016-12-19 18:43:04 -05:00
Jimmy Zelinskie
740262c055 Revert "Merge pull request #199 from openSUSE/feature/opensuse"
This reverts commit 97347ec44d, reversing
changes made to 051564facd.
2016-12-19 17:03:39 -05:00
Jimmy Zelinskie
7d0f29b28f Merge pull request #287 from jzelinskie/enginebump
worker: bump engine version
2016-12-19 15:42:23 -05:00
Jimmy Zelinskie
2cb23ced02 worker: bump engine version
Now that we support OpenSUSE and Alpine Linux the engine version should
be increased.
2016-12-19 15:35:25 -05:00
Jimmy Zelinskie
d62bddd6e3 Merge pull request #272 from jzelinskie/alpine
[WIP] Alpine support via Alpine-SecDB
2016-12-19 11:39:15 -05:00
Jimmy Zelinskie
f74cd35243 fetchers/alpine: add notes for untracked namespaces 2016-12-19 11:32:46 -05:00
Jimmy Zelinskie
3be8dfcf99 fetchers/alpine: auto detect namespaces 2016-12-19 11:32:46 -05:00
Jimmy Zelinskie
59e6c628dc alpine: refactor fetcher & git pull on update 2016-12-19 11:32:46 -05:00
Jimmy Zelinskie
9be305d19f alpine: truncate namespace to "vMAJOR.MINOR" 2016-12-19 11:32:46 -05:00
Jimmy Zelinskie
f8457b98e7 alpine: compile alpine into clair binary 2016-12-19 11:32:46 -05:00
Jimmy Zelinskie
3d90cac427 alpine: add support for v3.4 YAML schema 2016-12-19 11:32:46 -05:00
Jimmy Zelinskie
805f620b4b README: add alpine data sources 2016-12-19 11:32:46 -05:00
Jimmy Zelinskie
c1e0f618ca dockerfile: add git dependency 2016-12-19 11:32:45 -05:00
Jimmy Zelinskie
0cb8fc9455 updater/fetchers: add alpine secdb fetcher 2016-12-19 11:32:45 -05:00
Jimmy Zelinskie
fc908e65ba detectors/feature: add apk feature detector 2016-12-19 11:32:45 -05:00
Jimmy Zelinskie
e4b5930f77 detectors/feature: consistent naming and godoc 2016-12-19 11:32:45 -05:00
Jimmy Zelinskie
1d5a9ddd3c detectors/namespace: add alpine-release detector 2016-12-19 11:32:45 -05:00
Jimmy Zelinskie
0b2a9ab12b detectors/namespace: support pointers in tests
This change adjusts some names of types being exported and adds some
documentation.
2016-12-19 11:32:45 -05:00
Jimmy Zelinskie
5396396ff7 Merge pull request #282 from jzelinskie/layer-sort-id
api/v1: indexed layers for notifications
2016-12-06 19:37:49 -05:00
Jimmy Zelinskie
12c47e4066 docs: split http and json code blocks 2016-12-06 19:23:33 -05:00
Jimmy Zelinskie
d4522e9c6e api/v1: indexed layers for notifications
This change deprecates the old LayersIntroducingVulnerability for a new
one that orders output and contains an Index. This index is not
guaranteed to be consistent across multiple notifications, despite the
current Postgres implementation using the primary key of Layer table.
2016-12-06 19:23:33 -05:00
Quentin Machu
1fcae6abb8 Merge pull request #280 from coreos/add_idx_deleted_at
pgsql/migrations: add index on Vulnerability_Notification.deleted_at
2016-12-06 19:48:40 +01:00
Quentin Machu
83b5538c65 Merge pull request #281 from coreos/dis_hashjoins_introducing
pgsql: Disable hashjoins to get introducing layers for notifications
2016-12-06 19:48:33 +01:00
Quentin Machu
7a3dd5c817 pgsql: Disable hashjoins to get introducing layers for notifications 2016-12-06 16:19:10 +01:00
Quentin Machu
eeb13a02ba pgsql/migrations: add index on Vulnerability_Notification.deleted_at
`searchNotificationAvailable` never effectively use any indexes because:
- `notified_at < $1`, where $1 is a recent timestamp, returns the
  majority of the table and therefore it is cheaper for PostgreSQL
  to use a sequential scan on the table.
- there is no index for `deleted_at IS NULL`.
However, when Clair has been running for long enough, the grand majority
of rows (99%+) are expected to have a non-NULL `deleted_at` field. This
commit adds a new index on this very field in order to fetch the
remaining 1% in the blink of an eye.

In other words, instead of realizing a full table scan for each
`searchNotificationAvailable` query, we'll use the small branch of a new
index, reducing the total cost from over 30k to a mere 150 on a Clair
database that has already managed more than 1 000 000 notifications.
2016-12-06 14:39:52 +01:00
Quentin Machu
18e0018f80 Merge pull request #277 from jzelinskie/travispg
travis: add matrix for postgres
2016-12-04 19:00:52 +01:00
Quentin Machu
f5af78ed45 Merge branch 'master' into travispg 2016-12-04 19:00:47 +01:00
Jimmy Zelinskie
dab6e492b8 Merge pull request #279 from coreos/searchintro_optimize
pgsql: Reduce cost of GetNotification by 2.5x
2016-12-04 12:08:50 -05:00
Jimmy Zelinskie
2fe4a464e1 Merge pull request #278 from jzelinskie/layerdiffindex
pgsql/migrations: add ldfv compound index
2016-12-04 12:06:27 -05:00
Quentin Machu
dc8f71024f pgsql: Reduce cost of GetNotification by 2.5
By delaying the Layer join to the very end, we can cut the query costs from 540,836 to 219,477.

See Pull Request for details.
2016-12-04 13:21:47 +01:00
Jimmy Zelinskie
7cff31a058 pgsql/migrations: add ldfv compound index
This speeds up the SearchNotificationLayerIntroducingVulnerability query
by an order magnitude.
2016-12-04 05:02:15 -05:00
Jimmy Zelinskie
4fab327397 travis: add matrix for postgres 2016-12-03 21:00:21 -05:00
Jimmy Zelinskie
026f64aa82 Merge pull request #276 from jzelinskie/index
psql: add useful indexes
2016-12-02 16:07:48 -05:00
Jimmy Zelinskie
9dc002621a psql: add useful indexes
This adds some missing UNIQUE constraints and indexes for the
vulnerability table that should improve query performance.
2016-12-02 15:48:12 -05:00
Quentin Machu
39854ac64d Merge pull request #274 from JensPiegsa/patch-1
Restart clair until postgres is up
2016-11-22 13:26:11 +01:00
Jens Piegsa
9d75a90a8a Restart clair until postgres is up 2016-11-22 12:52:25 +01:00
Quentin Machu
639d305cf8 Merge pull request #271 from Quentin-M/nvd_severity
updater: Set vulns' Severity from NVD metadata fetcher if unknown
2016-11-18 21:31:47 +01:00