Commit Graph

121 Commits

Author SHA1 Message Date
Jimmy Zelinskie
a4e7873d14 database: make locks SOI & add Extend method 2019-02-28 15:46:31 -05:00
Sida Chen
73bc2bc36b
Merge pull request #672 from KeyboardNerd/source_package/feature_type
Implement Feature types
2019-02-20 15:58:50 -05:00
Sida Chen
5a94499fdb update according to comments 2019-02-19 17:57:38 -05:00
Sida Chen
79af05e67d pgsql: Fix postgres queries for feature_type 2019-02-19 16:48:42 -05:00
Sida Chen
073c685c5b pgsql: Add proper tests for database migration 2019-02-19 16:48:42 -05:00
Sida Chen
00eed77b45 database: Add feature_type database model 2019-02-19 16:48:42 -05:00
Sida Chen
c6c8fce39a pgsql: Add feature_type to initial schema
feature_type is for differentiating the binary packages and source
packages.
2019-02-19 16:43:35 -05:00
Jimmy Zelinskie
cafe0976a4
Merge pull request #685 from jzelinskie/updater-cleanup
updater: remove FindLock(), use errgroup to avoid races
2019-02-14 14:57:59 -05:00
Ales Raszka
015a79fd5a Layer: replace arrays with slices
The current code is much cleaner because it doesn't use indexes + it
should be more memory efficient.
2019-01-16 09:12:34 +01:00
Ales Raszka
90f5592095 Feature: replace arrays with slices
The current code is much cleaner because it doesn't use indexes + it
should be more memory efficient.
2019-01-16 09:12:33 +01:00
Jimmy Zelinskie
399deab100 database: remove FindLock() 2019-01-10 13:50:46 -05:00
Sida Chen
f759dd54c0 database: Replace Parent Feature with source metadata
Feature's source feature string is directly stored in the database
instead of having the parent pointer to simplify the database.
2018-10-15 16:26:24 -04:00
Sida Chen
3fe894c5ad database: Add parent feature pointer to Feature struct
Feature now has a pointer to parent feature. If a vulnerability affects
a parent feature, this child feature will be affected.
2018-10-09 19:52:10 -04:00
Sida Chen
e657d26313 database: move dbutil and testutil to database from pkg
Move dbutil and testutil to database from pkg
Rename all "result"
2018-10-08 12:10:35 -04:00
Sida Chen
0c1b80b2ed pgsql: Implement database queries for detector relationship
* Refactor layer and ancestry
* Add tests
* Fix bugs introduced when the queries were moved
2018-10-08 11:27:15 -04:00
Sida Chen
9c49d9dc55 pgsql: Move queries to corresponding files
Aggregate queries in their corresponding files instead of having the
single file for every queries because the database is more complicated.
2018-10-08 10:42:40 -04:00
Sida Chen
dca2d4e597 pgsql: Add detector to database schema
'detector' table is added to store the metadata of detectors.
'layer_feature', 'layer_namespace', and 'ancestry_feature' tables are
modified to store the detection relationship between the
feature/namespace with the detector.
2018-10-08 10:42:40 -04:00
Sida Chen
53433090a3 pgsql: update the query format 2018-09-19 14:33:08 -04:00
Sida Chen
e160616723 database: Use LayerWithContent as Layer 2018-09-13 13:21:39 -04:00
Sida Chen
ff9303905b database: changed Notification interface name 2018-09-11 14:24:09 -04:00
Sida Chen
6c69377343
Merge pull request #614 from KeyboardNerd/sidac/simplify
Replace Ancestry with AncestryWithContent struct in database models
2018-09-11 10:50:53 -04:00
Sida Chen
5d725e67b0 Replace Ancestry with AncestryWithContent struct in database models
As one of the steps to simplifies the codebase, the AncestryWithContent
struct is renamed to Ancestry, and Ancestry is removed. It will cause
the PostAncestry request to be slower.
2018-09-10 12:48:23 -04:00
Jimmy Zelinskie
0565938956 pkg/pagination: add token type
This change pulls as much pagination logic out of the database
implementation as possible. Database implementations should now be able
to marshal whatever state they need into opaque tokens with the
utilities in the pagination package.
2018-09-07 16:26:56 -04:00
Jimmy Zelinskie
d193b46449 pkg/pagination: init
This change refactors a lot of the code dealing with pagination so that
fernet implementation details do not leak.

- Deletes database/pgsql/token
- Introduces a pagination package
- Renames idPageNumber to Page and add a constructor and method.
2018-09-07 16:13:51 -04:00
Sida Chen
2827b9342b Update Database and Worker implementation for layer-wise feature
Feature extraction algorithm is changed to associate features with
ancestry layer. Database is updated to keep the relationship.
2018-09-06 13:56:03 -04:00
Jimmy Zelinskie
4491bedf2e database/pgsql: move token lib 2017-08-21 16:25:13 -04:00
Sida Chen
57a4f97780 pgSQL: fixed invalidating vulnerability cache query. 2017-08-14 16:14:55 -04:00
Sida Chen
a5c6400065 database: postgres implementation with tests. 2017-08-10 11:25:29 -04:00
Jimmy Zelinskie
abd7d2e013 Merge pull request #394 from KeyboardNerd/multiplens
added support for detecting multiple namespaces in a layer
2017-05-24 17:22:08 -04:00
Sida Chen
75d5d40d79 featurens: added multiple namespace testing for namespace detector 2017-05-24 17:18:11 -04:00
Sida Chen
bffa6499b7 added support for detect multiple namespaces in a layer
created table layer_namespace to store the many to many unique mapping of layers and namespaces
changed v1 api to provide a list of namespaces for each layer
changed namespace detector to use all registered detectors to detect namespaces
updated tests for multiple namespaces

Fixes #150
2017-05-24 17:01:51 -04:00
Jimmy Zelinskie
c2d8aec157 Merge pull request #382 from caipre/patch-1
pgsql: Change layer name column data type
2017-05-19 20:07:53 -04:00
Nick Platt
aea74550e1 pgsql: Expand layer, namespace column widths
Presently the layer and namespace tables use type `varchar(128)` for
their respective name columns. For layer, this width works fine enough
using the sha256 digests provided by docker. However, if one wishes to
encode the image name into the layer  name (eg, to avoid collisions like
in [0]), the limit of 128 bytes starts to feel a bit cramped. Bump to
256 bytes, since that "ought to be enough for anybody." (TM)

[0]: https://github.com/coreos/clair/issues/319
2017-05-19 18:22:04 -04:00
Sida Chen
9306e99368 converted to structured logging by using logrus
changed from capnslog to logrus for logging JSON structured message.

finished issue #383
2017-05-04 13:59:57 -04:00
Jimmy Zelinskie
9e875f748d database/pgsql: copy whole namespace 2017-02-23 02:05:33 -05:00
Jimmy Zelinskie
6a569fd945 move config to main / decentralize config
This puts config in its relevant location and moves functions around
loading config files into the main package.

As a side effect of removing cyclic imports for the API config, the
context library is no longer used.
2017-01-27 00:36:13 -05:00
Jimmy Zelinskie
9c63a63944 clair: mv updater clair and mv severity to db 2017-01-22 23:20:56 -05:00
Jimmy Zelinskie
3e4dc3834f utils: remove string.go 2017-01-22 23:02:51 -05:00
Jimmy Zelinskie
e7f72ef5ad utils: rm prometheus.go 2017-01-22 23:02:51 -05:00
Jimmy Zelinskie
343e24eb7e clair: remove types package
This removes the `types` package instead moving the contents to the
top-level clair package.
This change also renames the `Priority` type to `Severity` in order to
reduce confusion.
This change also removes the IsValid method and replaces it with a safe
constructor to avoid the creation of invalid values.
Many docstrings were tweaked in the making of this commit.
2017-01-22 23:02:51 -05:00
Jimmy Zelinskie
78cef02fda pkg: cerrors -> commonerr 2017-01-22 23:02:50 -05:00
Jimmy Zelinskie
224ff82543 pgsql/migrations: fix dpkg default versionfmt 2017-01-20 14:42:33 -05:00
Jimmy Zelinskie
ca9f340a91 pgsql: only select distinct layers 2017-01-19 17:09:57 -05:00
Jimmy Zelinskie
ea73aa153d pgsql: searchNotificationLayerIntroducingVulnerability order by layer ID
This fixes a bug where the API was returning Notification pages ordered
by LDFV.ID instead of by Layer ID.
2017-01-19 13:45:07 -05:00
Jimmy Zelinskie
627b98ef31 db/pgsql/feature: fix SQL error reporting 2017-01-03 17:06:08 -05:00
Jimmy Zelinskie
8d29bf860d versionfmt: convert to using constant over literal 2017-01-03 16:00:20 -05:00
Jimmy Zelinskie
8df8170ba5 db/pgsql/migration: convert to pure SQL 2017-01-03 15:59:22 -05:00
Jimmy Zelinskie
9e39a26f26 backfill version_format column 2017-01-03 13:15:14 -05:00
Jimmy Zelinskie
033709eaea add registerable version formats
Since we only ever used dpkg, this change shims everything into using
dpkg.
2016-12-30 12:51:24 -05:00
Jimmy Zelinskie
9338f28e82 psql/migrations: fix ordering 2016-12-25 19:25:57 -05:00