Commit Graph

94 Commits

Author SHA1 Message Date
Sida Chen
73bc2bc36b
Merge pull request #672 from KeyboardNerd/source_package/feature_type
Implement Feature types
2019-02-20 15:58:50 -05:00
Sida Chen
0e0d8b38bb featurefmt: Extract source packages and binary packages
The featurefmt now extracts both binary packages and source packages
from the package manager infos.
2019-02-19 16:48:42 -05:00
Sida Chen
7dd989c0f2 database: Rename affected Type to feature type 2019-02-19 16:48:42 -05:00
Jimmy Zelinskie
cafe0976a4
Merge pull request #685 from jzelinskie/updater-cleanup
updater: remove FindLock(), use errgroup to avoid races
2019-02-14 14:57:59 -05:00
Jimmy Zelinskie
25078ac838 ext: add CleanAll() utility functions 2019-01-10 13:50:46 -05:00
Flavio Castelli
5a4d4913c1
Reintroduce image scanning for openSUSE and SLE
Handle scanning of openSUSE and SUSE Linux Enterprise images.

Signed-off-by: Flavio Castelli <fcastelli@suse.com>
2019-01-07 18:48:55 +01:00
Ales Raszka
bd7102d963 Vulnsrc rhel: handle "none" CVE impact
Some RHEL CVEs [1] contains "none" string in impact field. This is throwing
warning message when fetching vulnerabilities. The new code handles this
case and it uses advisory severity instead.

[1] https://www.redhat.com/security/data/oval/com.redhat.rhsa-20080038.xml
2019-01-02 14:27:08 +01:00
Geoff Baskwill
3503ddb96f vulnsrc_oracle: one vulnerability per CVE
Get one vulnerability per CVE for Oracle instead of one per ELSA so we
can have NVD metadata added to the vulnerabilities.

Related: #495, #499.
2018-11-02 19:36:43 -04:00
Sida Chen
05cbf328aa
Merge pull request #647 from KeyboardNerd/spkg/cvrf
vulnsrc: Refactor debian and alpine sources
2018-10-23 09:30:01 -04:00
Sida Chen
72674ca871 vulnsrc: Refactor vulnerability sources to use utility functions 2018-10-22 23:00:58 -04:00
Jimmy Zelinskie
0c2e5e73c2
Merge pull request #645 from Katee/include-cvssv3
Switch to NVD JSON feed and include CVSSv3
2018-10-22 13:03:42 -04:00
Kate Murphy
081ae34af1
ext: remove duplicate vectorValuesToLetters definition 2018-10-19 15:00:00 -04:00
Kate Murphy
4f0da12b12
ext: pass through CVSSv3 impact and exploitability score 2018-10-19 10:44:23 -04:00
Jimmy Zelinskie
8efc3e4038 ext: remove unneeded use of init() 2018-10-18 18:48:07 -04:00
Jimmy Zelinskie
699d1143e5 ext: fixup incorrect copyright year 2018-10-18 18:47:37 -04:00
Sida Chen
2236b0a5c9 updater: Add vulnsrc affected feature type
Each vulnerability source has a specific type of feature that it affects

We assume the following:
* Alpine: Binary Package
* Debian: Source Package
* Ubuntu: Source Package
* Oracle OVAL: Binary Package
* RHEL OVAL: Binary Package
2018-10-18 15:06:41 -04:00
Kate Murphy
b81e4454fb
ext: Parse CVSSv3 data from JSON NVD feed 2018-10-16 19:08:17 -04:00
Kate Murphy
14277a8f5d
ext: Add JSON NVD parsing tests 2018-10-16 18:53:32 -04:00
Kate Murphy
aab46f5658
ext: Parse NVD JSON feed instead of XML
The JSON feed provides some values that are not available in the XML
feed such as CVSSv3.
2018-10-16 18:53:32 -04:00
Sida Chen
f759dd54c0 database: Replace Parent Feature with source metadata
Feature's source feature string is directly stored in the database
instead of having the parent pointer to simplify the database.
2018-10-15 16:26:24 -04:00
Jimmy Zelinskie
2ac088dd0f
Merge pull request #639 from Katee/update-sha1-to-sha256
Use SHA256 instead of SHA1 for fingerprinting
2018-10-15 11:43:56 -04:00
Kate Murphy
8d5a0131c4
ext: Use SHA256 instead of SHA1 for fingerprinting
To make static analysis tools happy.

The current use of SHA1 for fingerprinting is safe. However, there is very
little downside to switching to SHA256.
2018-10-12 16:09:14 -04:00
Sida Chen
2cc61f9fc0 ext/featurefmt/apk: Extract origin package information from database
"o" field is used to extract the Package Origin from the APK database.
2018-10-11 18:02:58 -04:00
Sida Chen
a057e4a943 ext/featurefmt/rpm: Extract source package from rpm database
Source package is now extracted from the RPM database by using
${SourceRPM} option in the rpm --qf argument.
2018-10-11 18:02:58 -04:00
Sida Chen
4ac046642f ext/featurefmt/dpkg: Extract source package metadata
The source package metadata is extracted from the source line instead
of forcing the binary package to have source package information.
2018-10-11 18:02:58 -04:00
Sida Chen
1c40e7d016 ext/featurefmt: Refactor featurefmt testing code
1. Featurefmt testing code is moved to featurefmttest package.
2. Featurefmt now can be tested against a csv file, which contains the
expected package information result.
2018-10-11 18:02:58 -04:00
Sida Chen
3c72fa29a6
Merge pull request #620 from KeyboardNerd/feature/detector
Internally version all detected content by extension
2018-10-08 15:16:04 -04:00
Sida Chen
e657d26313 database: move dbutil and testutil to database from pkg
Move dbutil and testutil to database from pkg
Rename all "result"
2018-10-08 12:10:35 -04:00
Sida Chen
53bf19aecf ext: Lister and Detector returns detector info with detected content
1. Every Lister and Detector are versioned
2. detected content, are returned in a map with detector info as the key
2018-10-08 10:42:40 -04:00
Jimmy Zelinskie
0ca9431235
Merge pull request #621 from jzelinskie/gitutil
pkg/gitutil: init
2018-09-26 11:42:35 -04:00
Jimmy Zelinskie
44ae4bc959
Merge pull request #610 from MackJM/wip/master_nvd_httputil
Using httputil for NVD
2018-09-19 14:25:44 -04:00
Jimmy Zelinskie
c2d887f9e9 pkg/gitutil: init
This refactors the code we're using to manage temporary git repositories
into a utility package.
2018-09-19 13:50:54 -04:00
Grégoire Unbekandt
c4ffa0c370 vulnsrc_rhel: cve impact
use the specific CVE's impact field instead of the RHSA's one
2018-09-15 00:00:09 +02:00
Grégoire Unbekandt
a90db713a2 vulnsrc_rhel: add test
Add test for multiple CVE
2018-09-14 23:54:33 +02:00
Grégoire Unbekandt
8b3338ef56 vulnsrc_rhel: minor changes
delete a useless line
2018-09-14 23:54:33 +02:00
Grégoire Unbekandt
4e4e98f328 vulnsrc_rhel: minor changes
Code reorganisation
2018-09-14 23:54:33 +02:00
Grégoire Unbekandt
ac86a36740 vulnsrc_rhel: rhsa_ID by default
If no CVE is present, create a vulnerability with rhsa ID
2018-09-14 23:54:33 +02:00
Grégoire Unbekandt
4ab98cfe54 vulnsrc_rhel: one vulnerability by CVE
Get one vulnerability by CVE_ID for RHEL instead of one by RHSA_ID so we can have NVD metadata added to the vulnerabilities.

Fixes #495
2018-09-14 23:54:33 +02:00
Jean Michel MacKay
30848d9eb7 Fixed extra newline 2018-09-11 15:28:40 -04:00
Jean Michel MacKay
56b4f23ae2 Move downloadFeed out to a seperate function 2018-09-11 15:28:39 -04:00
Jean Michel MacKay
f34f94320a Embed nvd's downloading and storing of meta data into a function to help with resource management 2018-09-11 15:28:39 -04:00
Jean Michel MacKay
3959f416fa Fix up error and changing close to defer close 2018-09-11 15:28:39 -04:00
Jean Michel MacKay
49cbdd7a7c Using httputil for NVD
nvd was missed when moving to httputil, this fixes it
2018-09-11 15:28:39 -04:00
Jimmy Zelinskie
06b257cc97
Merge pull request #606 from MackJM/wip/master_httputil
Adding httputil and version packages to master
2018-09-06 11:27:35 -04:00
Jimmy Zelinskie
ce15f73501 *: gofmt -s 2018-09-05 19:20:35 -04:00
Jean Michel MacKay
9df4f5bd70 Adding httputil and version packages
- Debian/RHEL/Oracle vulnsrc now use httputil to download files
- httputil sets the User-Agent to the requests as Clair/<version> (https://github.com/coreos/clair/)
- httputil holds Status2xx() which returns if the response is a http success (2xx)
- GetClientAddr moved from api/httputil to pkg/httputil
- the version packge holds a Version string which is set at build time from the git tag and sha
- the .git directory was removed from .dockerignore so that we can use the git tag to set the version
2018-09-05 14:56:39 -04:00
Jimmy Zelinskie
ce6b00887b vulnmdsrc: update NVD URLs
This connects us via a domain hosted on AWS which should provide
performance benefits for users running Clair on AWS and alleviate load
from the NIST campus network.

Fixes #575.
2018-09-04 11:55:19 -04:00
Daniel Jiang
9e4a347ecd Quickfix to the URL for fetching alpine's vuln data.
Fixes #593

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2018-08-23 13:39:26 +08:00
Jimmy Zelinskie
f32f438a98
Merge branch 'master' into nvdupdates 2018-07-17 10:36:08 -04:00
honglichang(常红立)
0d5f300c5b fix nvd path
1. stop clair, not del nvd xml
2018-07-17 20:11:24 +08:00