Commit Graph

28 Commits

Author SHA1 Message Date
Sida Chen
0c1b80b2ed pgsql: Implement database queries for detector relationship
* Refactor layer and ancestry
* Add tests
* Fix bugs introduced when the queries were moved
2018-10-08 11:27:15 -04:00
Sida Chen
dca2d4e597 pgsql: Add detector to database schema
'detector' table is added to store the metadata of detectors.
'layer_feature', 'layer_namespace', and 'ancestry_feature' tables are
modified to store the detection relationship between the
feature/namespace with the detector.
2018-10-08 10:42:40 -04:00
Sida Chen
53433090a3 pgsql: update the query format 2018-09-19 14:33:08 -04:00
Sida Chen
2827b9342b Update Database and Worker implementation for layer-wise feature
Feature extraction algorithm is changed to associate features with
ancestry layer. Database is updated to keep the relationship.
2018-09-06 13:56:03 -04:00
Sida Chen
a5c6400065 database: postgres implementation with tests. 2017-08-10 11:25:29 -04:00
Jimmy Zelinskie
abd7d2e013 Merge pull request #394 from KeyboardNerd/multiplens
added support for detecting multiple namespaces in a layer
2017-05-24 17:22:08 -04:00
Sida Chen
bffa6499b7 added support for detect multiple namespaces in a layer
created table layer_namespace to store the many to many unique mapping of layers and namespaces
changed v1 api to provide a list of namespaces for each layer
changed namespace detector to use all registered detectors to detect namespaces
updated tests for multiple namespaces

Fixes #150
2017-05-24 17:01:51 -04:00
Nick Platt
aea74550e1 pgsql: Expand layer, namespace column widths
Presently the layer and namespace tables use type `varchar(128)` for
their respective name columns. For layer, this width works fine enough
using the sha256 digests provided by docker. However, if one wishes to
encode the image name into the layer  name (eg, to avoid collisions like
in [0]), the limit of 128 bytes starts to feel a bit cramped. Bump to
256 bytes, since that "ought to be enough for anybody." (TM)

[0]: https://github.com/coreos/clair/issues/319
2017-05-19 18:22:04 -04:00
Jimmy Zelinskie
224ff82543 pgsql/migrations: fix dpkg default versionfmt 2017-01-20 14:42:33 -05:00
Jimmy Zelinskie
8df8170ba5 db/pgsql/migration: convert to pure SQL 2017-01-03 15:59:22 -05:00
Jimmy Zelinskie
9e39a26f26 backfill version_format column 2017-01-03 13:15:14 -05:00
Jimmy Zelinskie
033709eaea add registerable version formats
Since we only ever used dpkg, this change shims everything into using
dpkg.
2016-12-30 12:51:24 -05:00
Jimmy Zelinskie
9338f28e82 psql/migrations: fix ordering 2016-12-25 19:25:57 -05:00
Quentin Machu
eeb13a02ba pgsql/migrations: add index on Vulnerability_Notification.deleted_at
`searchNotificationAvailable` never effectively use any indexes because:
- `notified_at < $1`, where $1 is a recent timestamp, returns the
  majority of the table and therefore it is cheaper for PostgreSQL
  to use a sequential scan on the table.
- there is no index for `deleted_at IS NULL`.
However, when Clair has been running for long enough, the grand majority
of rows (99%+) are expected to have a non-NULL `deleted_at` field. This
commit adds a new index on this very field in order to fetch the
remaining 1% in the blink of an eye.

In other words, instead of realizing a full table scan for each
`searchNotificationAvailable` query, we'll use the small branch of a new
index, reducing the total cost from over 30k to a mere 150 on a Clair
database that has already managed more than 1 000 000 notifications.
2016-12-06 14:39:52 +01:00
Jimmy Zelinskie
7cff31a058 pgsql/migrations: add ldfv compound index
This speeds up the SearchNotificationLayerIntroducingVulnerability query
by an order magnitude.
2016-12-04 05:02:15 -05:00
Jimmy Zelinskie
9dc002621a psql: add useful indexes
This adds some missing UNIQUE constraints and indexes for the
vulnerability table that should improve query performance.
2016-12-02 15:48:12 -05:00
Quentin Machu
b8865b2106 pgsql: Replace liamstask/goose by remind101/migrate
Fixes #93
2016-11-11 18:11:48 +01:00
Quentin Machu
ccaaff000e database: add created_at field for layers and vulnerabilities 2016-02-24 16:36:45 -05:00
Quentin Machu
94ece7bf2b database: fix notification design and add vulnerability history 2016-02-24 16:36:45 -05:00
Quentin Machu
5fdd9d1a07 *: add metadata support along with NVD CVSS 2016-02-24 16:36:45 -05:00
Quentin Machu
8be18a0a01 database: write more of the notification system 2016-02-24 16:34:54 -05:00
Quentin Machu
ad0531acc7 notifier/database: refactor notification system and add initial Prometheus support 2016-02-24 16:34:54 -05:00
Quentin Machu
63ebddfd36 database: add vulnerability deletion support 2016-02-24 16:34:54 -05:00
Quentin Machu
c60d0054fa notifier/database: draft new notification system 2016-02-24 16:34:54 -05:00
Quentin Machu
82175dcfe9 *: add missing copyright headers 2016-02-24 16:34:54 -05:00
Quentin Machu
7c70fc1c20 database: add initial vulnerability support 2016-02-24 16:32:21 -05:00
Quentin Machu
3a786ae020 database: add lock support 2016-02-24 16:32:21 -05:00
Quentin Machu
2c150b015e *: refactor & do initial work towards PostgreSQL implementation 2016-02-24 16:32:21 -05:00