arno/clair
1
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity
861 Commits 5 Branches 24 Tags 158 MiB
4fa03d1c78
Commit Graph

165 Commits

Author SHA1 Message Date
liangchenye
a541e964e0 New API: list vulnerabilities by namespace 
Signed-off-by: liangchenye <liangchenye@huawei.com>
 2016-03-07 14:07:07 +08:00
Quentin Machu
9b191fb598 database: Find the FeatureVersion we try to insert before doing any lock 
This commit is issued in order to limit the bottleneck that the
exclusive database lock on Vulnerability_Affects_FeautreVersion
introduces, when we inserting FeatureVersions. This slowdowns a bit
the FeatureVersion insertion on a mostly empty database but should
increase a lot the throughput and parallelism on a populated database.
 2016-03-03 14:15:06 -05:00
Jimmy Zelinskie
500fc4e407 various: gofmt -s 2016-02-24 19:29:36 -05:00
Jimmy Zelinskie
8fd0aa162b various: spelling corrections 2016-02-24 18:00:36 -05:00
Quentin Machu
84319507df database: use constants to store queries 2016-02-24 16:40:40 -05:00
Quentin Machu
06531e01c5 database: disable hash/merge joins in FindLayer 
Our experiments have shown that PostgreSQL 9.4 makes bad
planning decisions about:
- joining the layer tree to feature versions and feature
- joining the feature versions to affected/fixed feature version and vulnerabilities
It would for instance do a merge join between affected feature versions (300 rows, estimated
3000 rows) and fixed in feature version (100k rows). In this case, it is much more
preferred to use a nested loop.
 2016-02-24 16:40:40 -05:00
Quentin Machu
18f2d7e672 database: modify join table in FindLayer to reduce cost by 3.5x 2016-02-24 16:40:40 -05:00
Quentin Machu
6d2eedf121 api/database: add the layer name that add each feature in getLayer 2016-02-24 16:39:25 -05:00
Quentin Machu
b5d8f9952e database: fix notification test (wrong signature) 2016-02-24 16:36:45 -05:00
Quentin Machu
f0816d2c4d database: add docs about the interface 2016-02-24 16:36:45 -05:00
Quentin Machu
d3b14106a9 database: ignore insertLayer collisions to make it truly idempotent 2016-02-24 16:36:45 -05:00
Quentin Machu
e3a25e5368 database: ignore min versions during new vulnerability insertions 2016-02-24 16:36:45 -05:00
Quentin Machu
7c11e4eb5d updater/database: do not create notifications during the initial update 2016-02-24 16:36:45 -05:00
Quentin Machu
883be8769f database: fix Ping() method in PostgreSQL's implementation 2016-02-24 16:36:45 -05:00
Quentin Machu
f8b4a52f8a database: make notification tests more robust (old/new, update/delete vulnerabilities) 2016-02-24 16:36:45 -05:00
Quentin Machu
ccaaff000e database: add created_at field for layers and vulnerabilities 2016-02-24 16:36:45 -05:00
Quentin Machu
94ece7bf2b database: fix notification design and add vulnerability history 2016-02-24 16:36:45 -05:00
Jimmy Zelinskie
99f3552470 database: add Insert/DeleteVulnerabilityFix 2016-02-24 16:36:45 -05:00
Quentin Machu
03d904c620 database: improve PostgreSQL test inits and cleanups 2016-02-24 16:36:45 -05:00
Quentin Machu
5fdd9d1a07 *: add metadata support along with NVD CVSS 2016-02-24 16:36:45 -05:00
Quentin Machu
8f9779e232 database: cache feature version upon lookup 2016-02-24 16:34:54 -05:00
Quentin Machu
1e4ded6f2b database: add ability to list namespaces 2016-02-24 16:34:54 -05:00
Quentin Machu
35df7ca0eb database: fix feature version cache 2016-02-24 16:34:54 -05:00
Quentin Machu
8be18a0a01 database: write more of the notification system 2016-02-24 16:34:54 -05:00
Quentin Machu
3ecb8b69cb updater: ignore "ubuntu-core" in the Ubuntu fetcher 2016-02-24 16:34:54 -05:00
Quentin Machu
d3d689a26a database: don't prune locks when we renew one 2016-02-24 16:34:54 -05:00
Quentin Machu
2690800331 database: create notification during vulnerability insertion 2016-02-24 16:34:54 -05:00
Quentin Machu
baed60e19b prometheus: add initial Prometheus support 2016-02-24 16:34:54 -05:00
Quentin Machu
ad0531acc7 notifier/database: refactor notification system and add initial Prometheus support 2016-02-24 16:34:54 -05:00
Quentin Machu
b8b7be3f81 *: remove health checker 2016-02-24 16:34:54 -05:00
Quentin Machu
63ebddfd36 database: add vulnerability deletion support 2016-02-24 16:34:54 -05:00
Quentin Machu
21f152c03e database: fix keyvalue/notification tests 2016-02-24 16:34:54 -05:00
Quentin Machu
563b3825d8 database: let handleErrors deal with the not found case 2016-02-24 16:34:54 -05:00
Quentin Machu
c60d0054fa notifier/database: draft new notification system 2016-02-24 16:34:54 -05:00
Quentin Machu
5759af5bcf database: test and fix layer updates 2016-02-24 16:34:54 -05:00
Quentin Machu
726bd3c0c6 database/api: add layer deletion support 2016-02-24 16:34:54 -05:00
Quentin Machu
248fc7df72 database: fix cache collision (feature & feature versions) 2016-02-24 16:34:54 -05:00
Quentin Machu
82175dcfe9 *: add missing copyright headers 2016-02-24 16:34:54 -05:00
Quentin Machu
6e20993bac api: simplify getLayer route and JSON output 2016-02-24 16:34:54 -05:00
Quentin Machu
92b734d0a4 database: remove an useless query in FindLayer 2016-02-24 16:34:54 -05:00
Quentin Machu
bd17dfb5e1 database: ensure that concurrent vulnerability/feature versions insertions work fine 2016-02-24 16:34:54 -05:00
Quentin Machu
74fc5b3e66 database: add missing transaction commits and close opened statement before inserting feature versions. 2016-02-24 16:34:54 -05:00
Quentin Machu
c5d1a8e5f7 database: update vulnerabilities only when necessary 2016-02-24 16:34:54 -05:00
Quentin Machu
77387af2ac updater: port updater and its fetchers 2016-02-24 16:34:54 -05:00
Quentin Machu
1b53142e38 database: allow removing fixed packages in vulnerabilities 2016-02-24 16:32:21 -05:00
Quentin Machu
7c70fc1c20 database: add initial vulnerability support 2016-02-24 16:32:21 -05:00
Quentin Machu
3a786ae020 database: add lock support 2016-02-24 16:32:21 -05:00
Quentin Machu
6a9cf21fd4 database: log and mask SQL errors 2016-02-24 16:32:21 -05:00
Quentin Machu
970756cd5a database: do insert/find layers (with their features and vulnerabilities) 2016-02-24 16:32:21 -05:00
Quentin Machu
2c150b015e *: refactor & do initial work towards PostgreSQL implementation 2016-02-24 16:32:21 -05:00
Quentin Machu
712aa11b8b updater: Add support for Ubuntu Vivid Core and ignore Vivid PhoneOverlay 
Reacts to https://bazaar.launchpad.net/~ubuntu-security/ubuntu-cve-tracker/master/revision/10488
 2016-01-25 13:04:39 -05:00
Quentin Machu
32747a5f25 database: Don't ignore empty results in toValue(s)() 
There is apparently no reason to ignore empty results - it was probably the case in the past (`null` value).

["", "v"] should be considered invalid by toValue() because it represents two values.
["", "v"] should be returned as it by toValues(), not trimming "".

Tests passes, it will hopefully not cause any issue in prod.
 2015-12-15 15:20:38 -05:00
Quentin Machu
eb7e5d5c74 main: Use configuration file instead of flags and simplify app extension. 
Clair will now use a YAML configuration file instead of command line
arguments as the number of parameters grows.

Also, Clair now exposes a Boot() func that allows everyone to easily
create their own project and load dynamically their own fetchers/updaters.
 2015-12-08 11:50:52 -05:00
Quentin Machu
3fe3f3a4c7 database: Update cayley and use Triple instead of Quad 2015-12-06 20:15:40 -05:00
Quentin Machu
9fc29e291c database: put missing predicates in consts and un-expose some of them 
Fixes #16
 2015-12-04 16:50:18 -05:00
Quentin Machu
46fffdfc81 Merge pull request #33 from Quentin-M/insertvulns 
database: Improve InsertVulnerabilities.
 2015-12-04 15:49:56 -05:00
Quentin Machu
8285c567c8 database: Improve InsertVulnerabilities. 2015-12-04 14:42:58 -05:00
Quentin Machu
e444e93c97 api/database: Add the ability to delete layers 2015-12-04 14:42:21 -05:00
Quentin Machu
9db0e63401 api: Specify what packages cause the layer to have vulnerabilities. 2015-12-01 17:02:49 -05:00
Quentin Machu
cfa960d619 database: Update Cayley to fix slow deletions 
subject, predicate, object and labels are not indexed, thus, using a where clause on these fields for the DELETE statement does a full-table scan. Using *_hash columns instead will use the indexes.
 2015-11-18 18:53:00 -05:00
Quentin Machu
3a1d0602fb database: Use an estimator in Cayley's Size() w/ PostgreSQL 2015-11-16 16:22:16 -05:00
Quentin Machu
f229083e1e database/worker: Remove useless log message 2015-11-16 13:21:06 -05:00
Quentin Machu
b0142e1982 database: reduce pruneLocks/Unlock transaction. 
pruneLocks could create deadlocked transactions on PostgreSQL if multiple locks expired and pruneLocks is called by multiple instances. Also adds some logging.
 2015-11-16 12:06:42 -05:00
Quentin Machu
7f1ff8f979 database: reduce InsertPackages transaction 
Inserting packages in a single transaction does not actually buy us anything as we often delete quads during an insertion and thus, Cayley could not use COPY and do a single round-trip. Inserting multiple packages in a single transaction actually creates deadlocks when a transaction tries to insert (A,B) and another one tries to insert (B,A).
 2015-11-13 18:06:01 -05:00
Quentin Machu
3ec262dd51 Initial commit 2015-11-13 14:11:28 -05:00