Commit Graph

107 Commits

Author SHA1 Message Date
Grégoire Unbekandt
4ab98cfe54 vulnsrc_rhel: one vulnerability by CVE
Get one vulnerability by CVE_ID for RHEL instead of one by RHSA_ID so we can have NVD metadata added to the vulnerabilities.

Fixes #495
2018-09-14 23:54:33 +02:00
Jean Michel MacKay
30848d9eb7 Fixed extra newline 2018-09-11 15:28:40 -04:00
Jean Michel MacKay
56b4f23ae2 Move downloadFeed out to a seperate function 2018-09-11 15:28:39 -04:00
Jean Michel MacKay
f34f94320a Embed nvd's downloading and storing of meta data into a function to help with resource management 2018-09-11 15:28:39 -04:00
Jean Michel MacKay
3959f416fa Fix up error and changing close to defer close 2018-09-11 15:28:39 -04:00
Jean Michel MacKay
49cbdd7a7c Using httputil for NVD
nvd was missed when moving to httputil, this fixes it
2018-09-11 15:28:39 -04:00
Jimmy Zelinskie
06b257cc97
Merge pull request #606 from MackJM/wip/master_httputil
Adding httputil and version packages to master
2018-09-06 11:27:35 -04:00
Jimmy Zelinskie
ce15f73501 *: gofmt -s 2018-09-05 19:20:35 -04:00
Jean Michel MacKay
9df4f5bd70 Adding httputil and version packages
- Debian/RHEL/Oracle vulnsrc now use httputil to download files
- httputil sets the User-Agent to the requests as Clair/<version> (https://github.com/coreos/clair/)
- httputil holds Status2xx() which returns if the response is a http success (2xx)
- GetClientAddr moved from api/httputil to pkg/httputil
- the version packge holds a Version string which is set at build time from the git tag and sha
- the .git directory was removed from .dockerignore so that we can use the git tag to set the version
2018-09-05 14:56:39 -04:00
Jimmy Zelinskie
ce6b00887b vulnmdsrc: update NVD URLs
This connects us via a domain hosted on AWS which should provide
performance benefits for users running Clair on AWS and alleviate load
from the NIST campus network.

Fixes #575.
2018-09-04 11:55:19 -04:00
Daniel Jiang
9e4a347ecd Quickfix to the URL for fetching alpine's vuln data.
Fixes #593

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2018-08-23 13:39:26 +08:00
Jimmy Zelinskie
f32f438a98
Merge branch 'master' into nvdupdates 2018-07-17 10:36:08 -04:00
honglichang(常红立)
0d5f300c5b fix nvd path
1. stop clair, not del nvd xml
2018-07-17 20:11:24 +08:00
ErikThoreson
df1dd5c149 adding publisher datetime and updating nvd feed download 2018-07-12 16:40:05 -05:00
Jimmy Zelinskie
456af5f48c vulnsrc/ubuntu: use new git-based ubuntu tracker 2018-07-10 16:46:46 -04:00
Jimmy Zelinskie
c031f8ea0c vulnsrc/alpine: s/pull/clone 2018-07-05 19:11:30 -04:00
Jimmy Zelinskie
4c2be5285e vulnsrc/alpine: avoid shadowing vars 2018-07-05 19:09:45 -04:00
Joe Ray
947a8aa00c featurens: Ensure RHEL is correctly identified
When trying to identify various RedHat releases, RHEL was not being
picked up as a centos release because the Oracle Linux regex was too
permissive: it would match any release name with '<something> Linux
Server release' in the name. By being more restrictive with the Oracle
regex, RHEL is now properly identified.

I don't know why the Oracle regex used such a permissive matcher for the
name but it still passes all the tests by replacing it with the word
'Oracle'.

Fixes #436
2018-01-12 12:01:25 +00:00
Grégoire Unbekandt
5c5857548d driver: Add proxy support
Enable the use of HTTP_PROXY and HTTPS_PROXY variables when downloading
layers
2017-11-07 12:17:13 +01:00
Grégoire Unbekandt
e953a259b0 nvd: fix the name of a field
The xml's field "availability-impact" wasn't collected due to a typo.
2017-10-23 15:45:23 +02:00
Sida Chen
fb32dcfa58 Clair Logic, Extensions: updated mock tests, extensions, basic logic
Main Clair logic is changed in worker, updater, notifier for better adapting
ancestry schema. Extensions are updated with the new model and feature lister
 and namespace detector drivers are able to specify the specific listers and
detectors used to process layer's content. InRange and GetFixedIn interfaces
are added to Version format for adapting ranged affected features and next
available fixed in in the future. Tests for worker, updater and extensions
are fixed.
2017-08-10 11:24:40 -04:00
Jimmy Zelinskie
04847a016d Merge pull request #418 from KeyboardNerd/multiplens
use namespace's versionfmt to specify listers scanning features
2017-06-28 13:53:21 -04:00
Sida Chen
9561d623c2 featurefmt: use namespace's versionfmt to specify listers
use namespace's versionfmt to specify listers used to scan features
the content detection functions are changed accordingly in worker
2017-06-22 15:36:07 -04:00
Sida Chen
50437f32a1 featurens: fix detecting duplicated namespaces problem 2017-06-22 11:41:18 -04:00
alinar
d4a967e6e6 Fixing always revision 0 for ubuntu 2017-06-07 12:37:24 +01:00
Jimmy Zelinskie
abd7d2e013 Merge pull request #394 from KeyboardNerd/multiplens
added support for detecting multiple namespaces in a layer
2017-05-24 17:22:08 -04:00
Sida Chen
75d5d40d79 featurens: added multiple namespace testing for namespace detector 2017-05-24 17:18:11 -04:00
Sida Chen
bffa6499b7 added support for detect multiple namespaces in a layer
created table layer_namespace to store the many to many unique mapping of layers and namespaces
changed v1 api to provide a list of namespaces for each layer
changed namespace detector to use all registered detectors to detect namespaces
updated tests for multiple namespaces

Fixes #150
2017-05-24 17:01:51 -04:00
Jimmy Zelinskie
f2f213470b Merge pull request #395 from knqyf263/handle_tilde
versionfmt/rpm: handle a tilde correctly
2017-05-15 14:52:31 -04:00
knqyf263
db8a133d21 versionfmt/rpm: handle a tilde correctly 2017-05-14 19:18:57 +09:00
Jimmy Zelinskie
0891bbac00 ext/vulnsrc/alpine: use HTTPS 2017-05-11 15:18:37 -04:00
Sida Chen
9306e99368 converted to structured logging by using logrus
changed from capnslog to logrus for logging JSON structured message.

finished issue #383
2017-05-04 13:59:57 -04:00
Jimmy Zelinskie
09cbfe325b ext/vulnsrc/oracle: ensure flag is largest elsa
If the Oracle Linux directory is ever in the wrong order, this should
ensure that the updaterFlag is always set the latest ELSA value.
2017-04-27 18:57:19 -04:00
Jimmy Zelinskie
34bc722794 ext/featurens: add empty filesmap tests for all 2017-04-21 15:08:47 -04:00
Jimmy Zelinskie
bcf47f53ee ext/vulnsrc/oracle: fix ELSA version comparison
Previously we naively compared integers. However, not all versions have
the same length.
2017-04-19 15:15:41 -04:00
Jimmy Zelinskie
b2f2b2c854 ext/featurefmt/apk: handle malformed packages 2017-04-11 15:48:27 -04:00
Jimmy Zelinskie
590e7e2602 ext/featurefmt/dpkg: handle malformed packages 2017-04-11 15:48:27 -04:00
Jimmy Zelinskie
300fe980ef ext/vulnsrc/ubuntu: add missing version format 2017-03-01 01:12:27 -05:00
supereagle
3f51191d23 configurable for TLS server's certificate chain and hostname verification when pulling layers 2017-02-26 07:30:23 +08:00
Quentin Machu
d606d85afe
ext/vulnsrc/rhel: fix logging namespace 2017-02-22 10:50:42 -08:00
Jimmy Zelinskie
1e9f14ae33 versionfmt/dpkg: remove leading digit requirement
This is not strictly a requirement and affects some tracked Alpine Linux
packages.
2017-02-07 13:31:28 -08:00
Jimmy Zelinskie
c8622d5f34 vulnsrc/alpine: unify schema and parse v3.5
HEAD of Alpine SecDB now uses one consistent schema for all of their
vulnerabilities, so the logic around parsing different versions can now
be removed. This change also crawls the directory structure to parse all
files due to the addition of community.yaml tracking community Alpine
Linux packages.
2017-02-07 13:31:28 -08:00
Jimmy Zelinskie
6a569fd945 move config to main / decentralize config
This puts config in its relevant location and moves functions around
loading config files into the main package.

As a side effect of removing cyclic imports for the API config, the
context library is no longer used.
2017-01-27 00:36:13 -05:00
Jimmy Zelinskie
9c63a63944 clair: mv updater clair and mv severity to db 2017-01-22 23:20:56 -05:00
Jimmy Zelinskie
c2f4a44068 utils: rm exec.go
This change also adds a dependency check at startup, rather than
runtime.
2017-01-22 23:02:51 -05:00
Jimmy Zelinskie
343e24eb7e clair: remove types package
This removes the `types` package instead moving the contents to the
top-level clair package.
This change also renames the `Priority` type to `Severity` in order to
reduce confusion.
This change also removes the IsValid method and replaces it with a safe
constructor to avoid the creation of invalid values.
Many docstrings were tweaked in the making of this commit.
2017-01-22 23:02:51 -05:00
Jimmy Zelinskie
03b8cd9a45 ext/featurens: add missing lock 2017-01-22 23:02:51 -05:00
Jimmy Zelinskie
cda3d4819c ext: feature detector -> featurefmt 2017-01-22 23:02:51 -05:00
Jimmy Zelinskie
71a8b542f9 ext: misc doc comment fixes 2017-01-22 23:02:50 -05:00
Jimmy Zelinskie
fb193e1fde ext: namespace detector -> featurens 2017-01-22 23:02:50 -05:00
Jimmy Zelinskie
d9be34c3c4 ext: data detector -> imagefmt 2017-01-22 23:02:50 -05:00
Jimmy Zelinskie
f9b319089d ext: lock all drivers 2017-01-22 23:02:50 -05:00
Jimmy Zelinskie
78cef02fda pkg: cerrors -> commonerr 2017-01-22 23:02:50 -05:00
Jimmy Zelinskie
4a990372ff refactor: move updaters and notifier into ext 2017-01-22 23:02:50 -05:00
Jimmy Zelinskie
8d29bf860d versionfmt: convert to using constant over literal 2017-01-03 16:00:20 -05:00
Jimmy Zelinskie
6864a8efea versionfmt: init rpm versionfmt 2016-12-30 12:51:25 -05:00
Jimmy Zelinskie
033709eaea add registerable version formats
Since we only ever used dpkg, this change shims everything into using
dpkg.
2016-12-30 12:51:24 -05:00