arno/clair
1
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity
874 Commits 5 Branches 24 Tags 158 MiB
60b0bd27fa
Commit Graph

48 Commits

Author SHA1 Message Date
Ales Raszka
60b0bd27fa Add namespace_id as constraints to layer_feature 
If layer contains more than one potential namespace, features will be
created for each namespace. Layer_feature table now has to contains one
more constrains (namespace_id).
 2019-03-07 11:22:54 +01:00
Ales Raszka
44c4a6f3ce Store PotentialNamespace in database 
PotentialNamespace is part of layer_feature table and it is also stored
in namespace table.
 2019-03-07 11:22:32 +01:00
Sida Chen
79af05e67d pgsql: Fix postgres queries for feature_type 2019-02-19 16:48:42 -05:00
Sida Chen
0c1b80b2ed pgsql: Implement database queries for detector relationship 
* Refactor layer and ancestry
* Add tests
* Fix bugs introduced when the queries were moved
 2018-10-08 11:27:15 -04:00
Sida Chen
9c49d9dc55 pgsql: Move queries to corresponding files 
Aggregate queries in their corresponding files instead of having the
single file for every queries because the database is more complicated.
 2018-10-08 10:42:40 -04:00
Sida Chen
e160616723 database: Use LayerWithContent as Layer 2018-09-13 13:21:39 -04:00
Sida Chen
ff9303905b database: changed Notification interface name 2018-09-11 14:24:09 -04:00
Sida Chen
5d725e67b0 Replace Ancestry with AncestryWithContent struct in database models 
As one of the steps to simplifies the codebase, the AncestryWithContent
struct is renamed to Ancestry, and Ancestry is removed. It will cause
the PostAncestry request to be slower.
 2018-09-10 12:48:23 -04:00
Sida Chen
2827b9342b Update Database and Worker implementation for layer-wise feature 
Feature extraction algorithm is changed to associate features with
ancestry layer. Database is updated to keep the relationship.
 2018-09-06 13:56:03 -04:00
Sida Chen
57a4f97780 pgSQL: fixed invalidating vulnerability cache query. 2017-08-14 16:14:55 -04:00
Sida Chen
a5c6400065 database: postgres implementation with tests. 2017-08-10 11:25:29 -04:00
Sida Chen
bffa6499b7 added support for detect multiple namespaces in a layer 
created table layer_namespace to store the many to many unique mapping of layers and namespaces
changed v1 api to provide a list of namespaces for each layer
changed namespace detector to use all registered detectors to detect namespaces
updated tests for multiple namespaces

Fixes #150
 2017-05-24 17:01:51 -04:00
Jimmy Zelinskie
ca9f340a91 pgsql: only select distinct layers 2017-01-19 17:09:57 -05:00
Jimmy Zelinskie
ea73aa153d pgsql: searchNotificationLayerIntroducingVulnerability order by layer ID 
This fixes a bug where the API was returning Notification pages ordered
by LDFV.ID instead of by Layer ID.
 2017-01-19 13:45:07 -05:00
Jimmy Zelinskie
033709eaea add registerable version formats 
Since we only ever used dpkg, this change shims everything into using
dpkg.
 2016-12-30 12:51:24 -05:00
Jimmy Zelinskie
d4522e9c6e api/v1: indexed layers for notifications 
This change deprecates the old LayersIntroducingVulnerability for a new
one that orders output and contains an Index. This index is not
guaranteed to be consistent across multiple notifications, despite the
current Postgres implementation using the primary key of Layer table.
 2016-12-06 19:23:33 -05:00
Quentin Machu
dc8f71024f pgsql: Reduce cost of GetNotification by 2.5 
By delaying the Layer join to the very end, we can cut the query costs from 540,836 to 219,477.

See Pull Request for details.
 2016-12-04 13:21:47 +01:00
Quentin Machu
ec0aad9b7a pgsql: Use booleans instead of varchar to return creation status 2016-11-12 15:42:59 +01:00
Quentin Machu
cd23262e41 pgsql: Do not insert entry in Vulnerability_FixedIn_Feature if existing 
Fixes #238
 2016-11-11 19:17:32 +01:00
Jimmy Zelinskie
5d8336acb3 pgsql: use subquery to plan GetNotification query (#182) 
This change enables the query planner to wait and sort the result set of
our query rather than attempting to re-use the layer table's index for
the ORDER BY clause. Because the result set is always small, this makes
queries that were previous tens of seconds, now tens of milliseconds.
 2016-05-20 13:10:00 -04:00
Jimmy Zelinskie
51f9c5dcb4 pgsql: remove unnecessary join used in GetNotification (#179) 2016-05-19 11:45:38 -04:00
liangchenye
1a863a06cf remove the useless pointer of NextPage field; check namespace notfound error 
Signed-off-by: liangchenye <liangchenye@huawei.com>
 2016-03-08 11:48:52 +08:00
liangchenye
48ffb2687a use encrypt page in listVuln api 
Signed-off-by: liangchenye <liangchenye@huawei.com>
 2016-03-07 14:07:07 +08:00
liangchenye
a541e964e0 New API: list vulnerabilities by namespace 
Signed-off-by: liangchenye <liangchenye@huawei.com>
 2016-03-07 14:07:07 +08:00
Quentin Machu
9b191fb598 database: Find the FeatureVersion we try to insert before doing any lock 
This commit is issued in order to limit the bottleneck that the
exclusive database lock on Vulnerability_Affects_FeautreVersion
introduces, when we inserting FeatureVersions. This slowdowns a bit
the FeatureVersion insertion on a mostly empty database but should
increase a lot the throughput and parallelism on a populated database.
 2016-03-03 14:15:06 -05:00
Quentin Machu
84319507df database: use constants to store queries 2016-02-24 16:40:40 -05:00
Quentin Machu
06531e01c5 database: disable hash/merge joins in FindLayer 
Our experiments have shown that PostgreSQL 9.4 makes bad
planning decisions about:
- joining the layer tree to feature versions and feature
- joining the feature versions to affected/fixed feature version and vulnerabilities
It would for instance do a merge join between affected feature versions (300 rows, estimated
3000 rows) and fixed in feature version (100k rows). In this case, it is much more
preferred to use a nested loop.
 2016-02-24 16:40:40 -05:00
Quentin Machu
18f2d7e672 database: modify join table in FindLayer to reduce cost by 3.5x 2016-02-24 16:40:40 -05:00
Quentin Machu
6d2eedf121 api/database: add the layer name that add each feature in getLayer 2016-02-24 16:39:25 -05:00
Quentin Machu
ccaaff000e database: add created_at field for layers and vulnerabilities 2016-02-24 16:36:45 -05:00
Quentin Machu
94ece7bf2b database: fix notification design and add vulnerability history 2016-02-24 16:36:45 -05:00
Jimmy Zelinskie
99f3552470 database: add Insert/DeleteVulnerabilityFix 2016-02-24 16:36:45 -05:00
Quentin Machu
5fdd9d1a07 *: add metadata support along with NVD CVSS 2016-02-24 16:36:45 -05:00
Quentin Machu
1e4ded6f2b database: add ability to list namespaces 2016-02-24 16:34:54 -05:00
Quentin Machu
8be18a0a01 database: write more of the notification system 2016-02-24 16:34:54 -05:00
Quentin Machu
ad0531acc7 notifier/database: refactor notification system and add initial Prometheus support 2016-02-24 16:34:54 -05:00
Quentin Machu
63ebddfd36 database: add vulnerability deletion support 2016-02-24 16:34:54 -05:00
Quentin Machu
c60d0054fa notifier/database: draft new notification system 2016-02-24 16:34:54 -05:00
Quentin Machu
5759af5bcf database: test and fix layer updates 2016-02-24 16:34:54 -05:00
Quentin Machu
726bd3c0c6 database/api: add layer deletion support 2016-02-24 16:34:54 -05:00
Quentin Machu
82175dcfe9 *: add missing copyright headers 2016-02-24 16:34:54 -05:00
Quentin Machu
92b734d0a4 database: remove an useless query in FindLayer 2016-02-24 16:34:54 -05:00
Quentin Machu
bd17dfb5e1 database: ensure that concurrent vulnerability/feature versions insertions work fine 2016-02-24 16:34:54 -05:00
Quentin Machu
1b53142e38 database: allow removing fixed packages in vulnerabilities 2016-02-24 16:32:21 -05:00
Quentin Machu
7c70fc1c20 database: add initial vulnerability support 2016-02-24 16:32:21 -05:00
Quentin Machu
3a786ae020 database: add lock support 2016-02-24 16:32:21 -05:00
Quentin Machu
970756cd5a database: do insert/find layers (with their features and vulnerabilities) 2016-02-24 16:32:21 -05:00
Quentin Machu
2c150b015e *: refactor & do initial work towards PostgreSQL implementation 2016-02-24 16:32:21 -05:00