Commit Graph

559 Commits

Author SHA1 Message Date
Jimmy Zelinskie
7d0f29b28f Merge pull request #287 from jzelinskie/enginebump
worker: bump engine version
2016-12-19 15:42:23 -05:00
Jimmy Zelinskie
2cb23ced02 worker: bump engine version
Now that we support OpenSUSE and Alpine Linux the engine version should
be increased.
2016-12-19 15:35:25 -05:00
Jimmy Zelinskie
d62bddd6e3 Merge pull request #272 from jzelinskie/alpine
[WIP] Alpine support via Alpine-SecDB
2016-12-19 11:39:15 -05:00
Jimmy Zelinskie
f74cd35243 fetchers/alpine: add notes for untracked namespaces 2016-12-19 11:32:46 -05:00
Jimmy Zelinskie
3be8dfcf99 fetchers/alpine: auto detect namespaces 2016-12-19 11:32:46 -05:00
Jimmy Zelinskie
59e6c628dc alpine: refactor fetcher & git pull on update 2016-12-19 11:32:46 -05:00
Jimmy Zelinskie
9be305d19f alpine: truncate namespace to "vMAJOR.MINOR" 2016-12-19 11:32:46 -05:00
Jimmy Zelinskie
f8457b98e7 alpine: compile alpine into clair binary 2016-12-19 11:32:46 -05:00
Jimmy Zelinskie
3d90cac427 alpine: add support for v3.4 YAML schema 2016-12-19 11:32:46 -05:00
Jimmy Zelinskie
805f620b4b README: add alpine data sources 2016-12-19 11:32:46 -05:00
Jimmy Zelinskie
c1e0f618ca dockerfile: add git dependency 2016-12-19 11:32:45 -05:00
Jimmy Zelinskie
0cb8fc9455 updater/fetchers: add alpine secdb fetcher 2016-12-19 11:32:45 -05:00
Jimmy Zelinskie
fc908e65ba detectors/feature: add apk feature detector 2016-12-19 11:32:45 -05:00
Jimmy Zelinskie
e4b5930f77 detectors/feature: consistent naming and godoc 2016-12-19 11:32:45 -05:00
Jimmy Zelinskie
1d5a9ddd3c detectors/namespace: add alpine-release detector 2016-12-19 11:32:45 -05:00
Jimmy Zelinskie
0b2a9ab12b detectors/namespace: support pointers in tests
This change adjusts some names of types being exported and adds some
documentation.
2016-12-19 11:32:45 -05:00
Jimmy Zelinskie
5396396ff7 Merge pull request #282 from jzelinskie/layer-sort-id
api/v1: indexed layers for notifications
2016-12-06 19:37:49 -05:00
Jimmy Zelinskie
12c47e4066 docs: split http and json code blocks 2016-12-06 19:23:33 -05:00
Jimmy Zelinskie
d4522e9c6e api/v1: indexed layers for notifications
This change deprecates the old LayersIntroducingVulnerability for a new
one that orders output and contains an Index. This index is not
guaranteed to be consistent across multiple notifications, despite the
current Postgres implementation using the primary key of Layer table.
2016-12-06 19:23:33 -05:00
Quentin Machu
1fcae6abb8 Merge pull request #280 from coreos/add_idx_deleted_at
pgsql/migrations: add index on Vulnerability_Notification.deleted_at
2016-12-06 19:48:40 +01:00
Quentin Machu
83b5538c65 Merge pull request #281 from coreos/dis_hashjoins_introducing
pgsql: Disable hashjoins to get introducing layers for notifications
2016-12-06 19:48:33 +01:00
Quentin Machu
7a3dd5c817 pgsql: Disable hashjoins to get introducing layers for notifications 2016-12-06 16:19:10 +01:00
Quentin Machu
eeb13a02ba pgsql/migrations: add index on Vulnerability_Notification.deleted_at
`searchNotificationAvailable` never effectively use any indexes because:
- `notified_at < $1`, where $1 is a recent timestamp, returns the
  majority of the table and therefore it is cheaper for PostgreSQL
  to use a sequential scan on the table.
- there is no index for `deleted_at IS NULL`.
However, when Clair has been running for long enough, the grand majority
of rows (99%+) are expected to have a non-NULL `deleted_at` field. This
commit adds a new index on this very field in order to fetch the
remaining 1% in the blink of an eye.

In other words, instead of realizing a full table scan for each
`searchNotificationAvailable` query, we'll use the small branch of a new
index, reducing the total cost from over 30k to a mere 150 on a Clair
database that has already managed more than 1 000 000 notifications.
2016-12-06 14:39:52 +01:00
Quentin Machu
18e0018f80 Merge pull request #277 from jzelinskie/travispg
travis: add matrix for postgres
2016-12-04 19:00:52 +01:00
Quentin Machu
f5af78ed45 Merge branch 'master' into travispg 2016-12-04 19:00:47 +01:00
Jimmy Zelinskie
dab6e492b8 Merge pull request #279 from coreos/searchintro_optimize
pgsql: Reduce cost of GetNotification by 2.5x
2016-12-04 12:08:50 -05:00
Jimmy Zelinskie
2fe4a464e1 Merge pull request #278 from jzelinskie/layerdiffindex
pgsql/migrations: add ldfv compound index
2016-12-04 12:06:27 -05:00
Quentin Machu
dc8f71024f pgsql: Reduce cost of GetNotification by 2.5
By delaying the Layer join to the very end, we can cut the query costs from 540,836 to 219,477.

See Pull Request for details.
2016-12-04 13:21:47 +01:00
Jimmy Zelinskie
7cff31a058 pgsql/migrations: add ldfv compound index
This speeds up the SearchNotificationLayerIntroducingVulnerability query
by an order magnitude.
2016-12-04 05:02:15 -05:00
Jimmy Zelinskie
4fab327397 travis: add matrix for postgres 2016-12-03 21:00:21 -05:00
Jimmy Zelinskie
026f64aa82 Merge pull request #276 from jzelinskie/index
psql: add useful indexes
2016-12-02 16:07:48 -05:00
Jimmy Zelinskie
9dc002621a psql: add useful indexes
This adds some missing UNIQUE constraints and indexes for the
vulnerability table that should improve query performance.
2016-12-02 15:48:12 -05:00
Quentin Machu
39854ac64d Merge pull request #274 from JensPiegsa/patch-1
Restart clair until postgres is up
2016-11-22 13:26:11 +01:00
Jens Piegsa
9d75a90a8a Restart clair until postgres is up 2016-11-22 12:52:25 +01:00
Quentin Machu
639d305cf8 Merge pull request #271 from Quentin-M/nvd_severity
updater: Set vulns' Severity from NVD metadata fetcher if unknown
2016-11-18 21:31:47 +01:00
Quentin Machu
1faf27ba18 utils: Fix OVAL's log statements 2016-11-18 19:00:13 +01:00
Quentin Machu
96398465de updater: Set vulns' Severity from NVD metadata fetcher if unknown
If a Vulnerability that goes through the NVD metadata fetcher has an
empty or Unknown Severity, then use the CVSS score to set one. This
will help to get a more consistent database when a vulnerability source
does not provide this information.
2016-11-18 19:00:13 +01:00
Quentin Machu
1c3daa23b9 updater: minimize vulns' lock duration in the NVD metadata fetcher 2016-11-18 18:09:59 +01:00
Quentin Machu
e38b67a513 Merge pull request #270 from Quentin-M/imp_docs
docs: improve GET/POST /v1/layers documentation
2016-11-18 01:03:16 +01:00
Quentin Machu
37a58260db docs: improve GET/POST /v1/layers documentation 2016-11-17 17:54:23 +01:00
Quentin Machu
7fddd68f4f Merge pull request #263 from Quentin-M/rhel_unique_fixedin
pgsql: Do not insert entry in Vulnerability_FixedIn_Feature if existing
2016-11-13 13:25:08 +01:00
Quentin Machu
ec0aad9b7a pgsql: Use booleans instead of varchar to return creation status 2016-11-12 15:42:59 +01:00
Quentin Machu
43cbc04356 Merge pull request #261 from Quentin-M/replace_goose
Replace liamstask/goose by remind101/migrate
2016-11-11 19:26:04 +01:00
Quentin Machu
cd23262e41 pgsql: Do not insert entry in Vulnerability_FixedIn_Feature if existing
Fixes #238
2016-11-11 19:17:32 +01:00
Jimmy Zelinskie
eed4a9a7dd Merge pull request #262 from jzelinskie/travis
travis: update to use Go 1.7, glide
2016-11-11 12:23:32 -05:00
Quentin Machu
b8865b2106 pgsql: Replace liamstask/goose by remind101/migrate
Fixes #93
2016-11-11 18:11:48 +01:00
Jimmy Zelinskie
2d0be7ccf4 travis: update to use Go 1.7, glide 2016-11-11 12:07:59 -05:00
Quentin Machu
43969afa60 Merge pull request #257 from mattmoor/yakkety
Add Ubuntu yakkety to the namespace mapping.
2016-10-25 09:50:18 +02:00
Matt Moore
90cc8243ba Add Ubuntu yakkety to the namespace mapping. 2016-10-24 21:08:02 -07:00
Jimmy Zelinskie
861cba0f49 README: s/1.2.2/1.2.4 2016-10-24 12:42:22 -04:00