'detector' table is added to store the metadata of detectors.
'layer_feature', 'layer_namespace', and 'ancestry_feature' tables are
modified to store the detection relationship between the
feature/namespace with the detector.
All detected features and namespaces under the context of Layer and
Ancestry will now have the detectors associated, so that the API can
provide the detection information to the Client.
As one of the steps to simplifies the codebase, the AncestryWithContent
struct is renamed to Ancestry, and Ancestry is removed. It will cause
the PostAncestry request to be slower.
This change pulls as much pagination logic out of the database
implementation as possible. Database implementations should now be able
to marshal whatever state they need into opaque tokens with the
utilities in the pagination package.
This change refactors a lot of the code dealing with pagination so that
fernet implementation details do not leak.
- Deletes database/pgsql/token
- Introduces a pagination package
- Renames idPageNumber to Page and add a constructor and method.
Version format documentation is updated in the example of feature
struct and created in clair.proto along with accordingly generated
swagger and protobuf files.
Fixes#514
Main Clair logic is changed in worker, updater, notifier for better adapting
ancestry schema. Extensions are updated with the new model and feature lister
and namespace detector drivers are able to specify the specific listers and
detectors used to process layer's content. InRange and GetFixedIn interfaces
are added to Version format for adapting ranged affected features and next
available fixed in in the future. Tests for worker, updater and extensions
are fixed.
created table layer_namespace to store the many to many unique mapping of layers and namespaces
changed v1 api to provide a list of namespaces for each layer
changed namespace detector to use all registered detectors to detect namespaces
updated tests for multiple namespaces
Fixes#150
Presently the layer and namespace tables use type `varchar(128)` for
their respective name columns. For layer, this width works fine enough
using the sha256 digests provided by docker. However, if one wishes to
encode the image name into the layer name (eg, to avoid collisions like
in [0]), the limit of 128 bytes starts to feel a bit cramped. Bump to
256 bytes, since that "ought to be enough for anybody." (TM)
[0]: https://github.com/coreos/clair/issues/319
This puts config in its relevant location and moves functions around
loading config files into the main package.
As a side effect of removing cyclic imports for the API config, the
context library is no longer used.
This removes the `types` package instead moving the contents to the
top-level clair package.
This change also renames the `Priority` type to `Severity` in order to
reduce confusion.
This change also removes the IsValid method and replaces it with a safe
constructor to avoid the creation of invalid values.
Many docstrings were tweaked in the making of this commit.
This change deprecates the old LayersIntroducingVulnerability for a new
one that orders output and contains an Index. This index is not
guaranteed to be consistent across multiple notifications, despite the
current Postgres implementation using the primary key of Layer table.
`searchNotificationAvailable` never effectively use any indexes because:
- `notified_at < $1`, where $1 is a recent timestamp, returns the
majority of the table and therefore it is cheaper for PostgreSQL
to use a sequential scan on the table.
- there is no index for `deleted_at IS NULL`.
However, when Clair has been running for long enough, the grand majority
of rows (99%+) are expected to have a non-NULL `deleted_at` field. This
commit adds a new index on this very field in order to fetch the
remaining 1% in the blink of an eye.
In other words, instead of realizing a full table scan for each
`searchNotificationAvailable` query, we'll use the small branch of a new
index, reducing the total cost from over 30k to a mere 150 on a Clair
database that has already managed more than 1 000 000 notifications.
This change enables the query planner to wait and sort the result set of
our query rather than attempting to re-use the layer table's index for
the ORDER BY clause. Because the result set is always small, this makes
queries that were previous tens of seconds, now tens of milliseconds.
This commit is issued in order to limit the bottleneck that the
exclusive database lock on Vulnerability_Affects_FeautreVersion
introduces, when we inserting FeatureVersions. This slowdowns a bit
the FeatureVersion insertion on a mostly empty database but should
increase a lot the throughput and parallelism on a populated database.
Our experiments have shown that PostgreSQL 9.4 makes bad
planning decisions about:
- joining the layer tree to feature versions and feature
- joining the feature versions to affected/fixed feature version and vulnerabilities
It would for instance do a merge join between affected feature versions (300 rows, estimated
3000 rows) and fixed in feature version (100k rows). In this case, it is much more
preferred to use a nested loop.
Ales Raszka