Merge branch 'master' into integrate-clairctl

Dockerfile

@@ -14,8 +14,6 @@
 
 FROM golang:1.8-alpine
 
-MAINTAINER Quentin Machu <quentin.machu@coreos.com>
-
 VOLUME /config
 EXPOSE 6060 6061
 

Documentation/api_v1.md

@@ -26,7 +26,7 @@
 #### Description
 
 Every route can optionally provide an `Error` property on the response object.
-The HTTP status code of the response should indicate what type of failure occurred and how the client should reaction.
+The HTTP status code of the response should indicate what type of failure occurred and how the client should react.
 
 #### Client Retry Behavior
 

Documentation/integrations.md

@@ -0,0 +1,19 @@
+# Integrations
+This document tracks projects that integrate with Clair. [Join the community](https://github.com/coreos/clair/), and help us keep the list up-to-date.
+
+## Projects
+
+[Quay.io](https://quay.io/) and [Quay Enterprise](https://quay.io/plans/?tab=enterprise): Quay was the first container registry to integrate with Clair.
+
+[Dockyard](https://github.com/Huawei/dockyard): an open source container registry with Clair integration.
+
+[Clairctl](https://github.com/jgsqware/clairctl): a lightweight command-line tool for working locally with Clair and generate HTML report.
+
+[Clair-SQS](https://github.com/zalando-incubator/clair-sqs): a container containing Clair and additional processes that integrate Clair with [Amazon SQS][sqs].
+
+[Klar](https://github.com/optiopay/klar): a simple command-line integration of Clair and Docker registry, designed to be used in scripts and CI.
+
+[reg](https://github.com/jessfraz/reg#vulnerability-reports): a docker registry CLI, which also runs Clair.
+
+
+[sqs]: https://aws.amazon.com/sqs/ 

Documentation/production-users.md

@@ -0,0 +1,8 @@
+# Production users
+
+This document tracks people and use cases for Clair in production. [Join the community](https://github.com/coreos/Clair/), and help us keep the list up-to-date.
+
+## [Quay.io](https://quay.io/)
+
+Quay is CoreOS' enterprise-ready container registry. It displays the results of a Clair security scan on each hosted image's page.
+

README.md

@@ -11,51 +11,78 @@ Please use [releases] instead of the `master` branch in order to get stable bina
 
 ![Clair Logo](https://cloud.githubusercontent.com/assets/343539/21630811/c5081e5c-d202-11e6-92eb-919d5999c77a.png)
 
-Clair is an open source project for the static analysis of vulnerabilities in [appc] and [docker] containers.
+Clair is an open source project for the static analysis of vulnerabilities in application containers (currently including [appc] and [docker]).
 
-Vulnerability data is continuously imported from a known set of sources and correlated with the indexed contents of container images in order to produce lists of vulnerabilities that threaten a container.
+1. In regular intervals, Clair ingests vulnerability metadata from a configured set of sources and stores it in the database.
-When vulnerability data changes upstream, a notification can be delivered, and the API queried to provide the previous state and new state of the vulnerability along with the images affected by both.
+2. Clients use the Clair API to index their container images; this parses a list of installed _source packages_ and stores them in the database.
-All major components can be [extended programmatically] at compile-time without forking the project.
+3. Clients use the Clair API to query the database; correlating data is done in real time, rather than a cached result that needs re-scanning.
+4. When updates to vulnerability metadata occur, a webhook containg the affected images can be configured to page or block deployments.
 
 Our goal is to enable a more transparent view of the security of container-based infrastructure.
 Thus, the project was named `Clair` after the French term which translates to *clear*, *bright*, *transparent*.
 
 [appc]: https://github.com/appc/spec
-[docker]: https://github.com/docker/docker/blob/master/image/spec/v1.md
+[docker]: https://github.com/docker/docker/blob/master/image/spec/v1.2.md
-[extended programmatically]: #customization
 [releases]: https://github.com/coreos/clair/releases
 
-## Common Use Cases
+## When would I use Clair?
 
-### Manual Auditing
+* You've found an image by searching the internet and want to determine if it's safe enough for you to use in production.
+* You're regularly deploying into a containerized production environment and want operations to alert or block deployments on insecure software.
 
-You're building an application and want to depend on a third-party container image that you found by searching the internet.
+## Documentation
-To make sure that you do not knowingly introduce a new vulnerability into your production service, you decide to scan the container for vulnerabilities.
-You `docker pull` the container to your development machine and start an instance of Clair.
-Once it finishes updating, you use the [local image analysis tool] to analyze the container.
-You realize this container is vulnerable to many critical CVEs, so you decide to use another one.
 
-[local image analysis tool]: https://github.com/coreos/clair/tree/master/contrib/analyze-local-images
+* [The CoreOS website] has a rendered version of the latest stable documentation
+* [Inside the Documentation directory] is the source markdown files for documentation
 
-### Container Registry Integration
+[The CoreOS website]: https://coreos.com/clair/docs/latest/
+[Inside the Documentation directory]: /Documentation
 
-Your company has a continuous-integration pipeline and you want to stop deployments if they introduce a dangerous vulnerability.
+## Deploying Clair
-A developer merges some code into the master branch of your codebase.
-The first step of your continuous-integration pipeline automates the testing and building of your container and pushes a new container to your container registry.
-Your container registry notifies Clair which causes the download and indexing of the images for the new container.
-Clair detects some vulnerabilities and sends a webhook to your continuous deployment tool to prevent this vulnerable build from seeing the light of day.
 
-## Hello Heartbleed
+### Container Repositories
 
-During the first run, Clair will bootstrap its database with vulnerability data from its data sources.
+Clair is officially packaged and released as a container.
-It can take several minutes before the database has been fully populated.
 
-**NOTE:** These setups are not meant for production workloads, but as a quick way to get started.
+* [quay.io/coreos/clair] - Stable releases
+* [quay.io/coreos/clair-jwt] - Stable releases with an embedded instance of [jwtproxy]
+* [quay.io/coreos/clair-git] - Development releases
 
-### Kubernetes
+[quay.io/coreos/clair]: https://quay.io/repository/coreos/clair
+[jwtproxy]: https://github.com/coreos/jwtproxy
+[quay.io/coreos/clair-jwt]: https://quay.io/repository/coreos/clair-jwt
+[quay.io/coreos/clair-git]: https://quay.io/repository/coreos/clair-git
 
-An easy way to run Clair is with Kubernetes 1.2+.
+### Commercially Supported
-If you are using the [CoreOS Kubernetes single-node instructions][single-node] for Vagrant you will be able to access the Clair's API at http://172.17.4.99:30060/ after following these instructions.
+
+Clair is professionally supported as a data source for the [Quay] Security Scanning feature.
+The setup documentation for using Clair for this environment can be found on the [Quay documentation] on the [CoreOS] website.
+Be sure to adjust the version of the documentation to the version of Quay being used in your deployment.
+
+[Quay]: https://quay.io
+[Quay documentation]: https://coreos.com/quay-enterprise/docs/latest/clair.html
+[CoreOS]: https://coreos.com
+
+### Community Supported
+
+**NOTE:** These instructions demonstrate running HEAD and not stable versions.
+
+The following are community supported instructions to run Clair in a variety of ways.
+A database instance is required for all instructions.
+
+Clair currently supports and tests against:
+
+* [Postgres] 9.4
+* [Postgres] 9.5
+* [Postgres] 9.6
+
+[Postgres]: https://www.postgresql.org
+
+#### Kubernetes
+
+If you don't have a local Kubernetes cluster already, check out [minikube].
+
+[minikube]: https://github.com/kubernetes/minikube
 
 ```
 git clone https://github.com/coreos/clair
@@ -64,18 +91,12 @@ kubectl create secret generic clairsecret --from-file=./config.yaml
 kubectl create -f clair-kubernetes.yaml
 ```
 
-[single-node]: https://coreos.com/kubernetes/docs/latest/kubernetes-on-vagrant-single.html
+#### Docker Compose
-
-### Docker Compose
-
-Another easy way to get an instance of Clair running is to use Docker Compose to run everything locally.
-This runs a PostgreSQL database insecurely and locally in a container.
-This method should only be used for testing.
 
 ```sh
-$ curl -L https://raw.githubusercontent.com/coreos/clair/v1.2.4/docker-compose.yml -o $HOME/docker-compose.yml
+$ curl -L https://raw.githubusercontent.com/coreos/clair/master/docker-compose.yml -o $HOME/docker-compose.yml
 $ mkdir $HOME/clair_config
-$ curl -L https://raw.githubusercontent.com/coreos/clair/v1.2.4/config.example.yaml -o $HOME/clair_config/config.yaml
+$ curl -L https://raw.githubusercontent.com/coreos/clair/master/config.example.yaml -o $HOME/clair_config/config.yaml
 $ $EDITOR $HOME/clair_config/config.yaml # Edit database source to be postgresql://postgres:password@postgres:5432?sslmode=disable
 $ docker-compose -f $HOME/docker-compose.yml up -d
 ```
@@ -83,25 +104,24 @@ $ docker-compose -f $HOME/docker-compose.yml up -d
 Docker Compose may start Clair before Postgres which will raise an error.
 If this error is raised, manually execute `docker-compose start clair`.
 
-
+#### Docker
-### Docker
-
-This method assumes you already have a [PostgreSQL 9.4+] database running.
-This is the recommended method for production deployments.
-
-[PostgreSQL 9.4+]: http://postgresql.org
 
 ```sh
-$ mkdir $HOME/clair_config
+$ mkdir $PWD/clair_config
-$ curl -L https://raw.githubusercontent.com/coreos/clair/v1.2.4/config.example.yaml -o $HOME/clair_config/config.yaml
+$ curl -L https://raw.githubusercontent.com/coreos/clair/master/config.example.yaml -o $PWD/clair_config/config.yaml
-$ $EDITOR $HOME/clair_config/config.yaml # Add the URI for your postgres database
+$ docker run -d -e POSTGRES_PASSWORD="" -p 5432:5432 postgres:9.6
-$ docker run -d -p 6060-6061:6060-6061 -v $HOME/clair_config:/config quay.io/coreos/clair:v1.2. -config=/config/config.yaml
+$ docker run -d -p 6060-6061:6060-6061 -v $PWD/clair_config:/config quay.io/coreos/clair-git:latest -config=/config/config.yaml
 ```
 
-### Source
+#### Source
 
 To build Clair, you need to latest stable version of [Go] and a working [Go environment].
-In addition, Clair requires that [git], [bzr], [rpm], and [xz] be available on the system [$PATH].
+In addition, Clair requires some additional binaries be installed on the system [$PATH] as runtime dependencies:
+
+* [git]
+* [bzr]
+* [rpm]
+* [xz]
 
 [Go]: https://github.com/golang/go/releases
 [Go environment]: https://golang.org/doc/code.html
@@ -115,38 +135,19 @@ In addition, Clair requires that [git], [bzr], [rpm], and [xz] be available on t
 $ go get github.com/coreos/clair
 $ go install github.com/coreos/clair/cmd/clair
 $ $EDITOR config.yaml # Add the URI for your postgres database
-$ ./$GOBIN/clair -config=config.yaml
+$ ./$GOPATH/bin/clair -config=config.yaml
 ```
 
-### Container images
+## Frequently Asked Questions
 
-While container images for every releases are available at [quay.io/repository/coreos/clair], container images built on the latest available source code are available at [quay.io/repository/coreos/clair-git].
+### Who's using Clair?
 
-[quay.io/repository/coreos/clair]: https://quay.io/repository/coreos/clair
+You can find [production users] and third party [integrations] documented in their respective pages of the local documentation.
-[quay.io/repository/coreos/clair-git]: https://quay.io/repository/coreos/clair-git
 
-## Documentation
+[production users]: https://github.com/coreos/clair/blob/master/Documentation/production-users.md
+[integrations]: https://github.com/coreos/clair/blob/master/Documentation/integrations.md
 
-The latest stable documentation can be found [on the CoreOS website].
+### What do you mean by static analysis?
-Documentation for the current branch can be found [inside the Documentation directory][docs-dir] at the root of the project's source code.
-
-[on the CoreOS website]: https://coreos.com/clair/docs/latest/
-[docs-dir]: /Documentation
-
-### Architecture at a Glance
-
-![Simple Clair Diagram](https://cloud.githubusercontent.com/assets/343539/21630809/c1adfbd2-d202-11e6-9dfe-9024139d0a28.png)
-
-### Terminology
-
-- *Image* - a tarball of the contents of a container
-- *Layer* - an *appc* or *Docker* image that may or maybe not be dependent on another image
-- *Feature* - anything that when present could be an indication of a *vulnerability* (e.g. the presence of a file or an installed software package)
-- *Feature Namespace* - a context around *features* and *vulnerabilities* (e.g. an operating system)
-- *Vulnerability Updater* - a Go package that tracks upstream vulnerability data and imports them into Clair
-- *Vulnerability Metadata Appender* - a Go package that tracks upstream vulnerability metadata and appends them into vulnerabilities managed by Clair
-
-### Vulnerability Analysis
 
 There are two major ways to perform analysis of programs: [Static Analysis] and [Dynamic Analysis].
 Clair has been designed to perform *static analysis*; containers never need to be executed.
@@ -156,7 +157,7 @@ By indexing the features of an image into the database, images only need to be r
 [Static Analysis]: https://en.wikipedia.org/wiki/Static_program_analysis
 [Dynamic Analysis]: https://en.wikipedia.org/wiki/Dynamic_program_analysis
 
-### Default Data Sources
+### What data sources does Clair currently support?
 
 | Data Source                   | Data Collected                                                           | Format | License         |
 |-------------------------------|--------------------------------------------------------------------------|--------|-----------------|
@@ -164,7 +165,7 @@ By indexing the features of an image into the database, images only need to be r
 | [Ubuntu CVE Tracker]          | Ubuntu 12.04, 12.10, 13.04, 14.04, 14.10, 15.04, 15.10, 16.04 namespaces | [dpkg] | [GPLv2]         |
 | [Red Hat Security Data]       | CentOS 5, 6, 7 namespaces                                                | [rpm]  | [CVRF]          |
 | [Oracle Linux Security Data]  | Oracle Linux 5, 6, 7 namespaces                                          | [rpm]  | [CVRF]          |
-| [Alpine SecDB]                | Alpine 3.3, Alpine 3.4 namespaces                                        | [apk]  | [MIT]           |
+| [Alpine SecDB]                | Alpine 3.3, Alpine 3.4, Alpine 3.5 namespaces                            | [apk]  | [MIT]           |
 | [NIST NVD]                    | Generic Vulnerability Metadata                                           | N/A    | [Public Domain] |
 
 [Debian Security Bug Tracker]: https://security-tracker.debian.org/tracker
@@ -182,11 +183,31 @@ By indexing the features of an image into the database, images only need to be r
 [apk]: http://git.alpinelinux.org/cgit/apk-tools/
 [MIT]: https://gist.github.com/jzelinskie/6da1e2da728424d88518be2adbd76979
 
+### What do most deployments look like?
 
-### Customization
+From a high-level, most deployments integrate with the registry workflow rather than manual API usage by a human.
+They typically take up a form similar to the following diagram:
+
+![Simple Clair Diagram](https://cloud.githubusercontent.com/assets/343539/21630809/c1adfbd2-d202-11e6-9dfe-9024139d0a28.png)
+
+### I just started up Clair and nothing appears to be working, what's the deal?
+
+During the first run, Clair will bootstrap its database with vulnerability data from the configured data sources.
+It can take several minutes before the database has been fully populated, but once this data is stored in the database, subsequent updates will take far less time.
+
+### What terminology do I need to understand to work with Clair internals?
+
+- *Image* - a tarball of the contents of a container
+- *Layer* - an *appc* or *Docker* image that may or may not be dependent on another image
+- *Feature* - anything that when present could be an indication of a *vulnerability* (e.g. the presence of a file or an installed software package)
+- *Feature Namespace* - a context around *features* and *vulnerabilities* (e.g. an operating system)
+- *Vulnerability Updater* - a Go package that tracks upstream vulnerability data and imports them into Clair
+- *Vulnerability Metadata Appender* - a Go package that tracks upstream vulnerability metadata and appends them into vulnerabilities managed by Clair
+
+### How can I customize Clair?
 
 The major components of Clair are all programmatically extensible in the same way Go's standard [database/sql] package is extensible.
-Everything extendable is located in the `ext` directory.
+Everything extensible is located in the `ext` directory.
 
 Custom behavior can be accomplished by creating a package that contains a type that implements an interface declared in Clair and registering that interface in [init()].
 To expose the new behavior, unqualified imports to the package must be added in your own custom [main.go], which should then start Clair using `Boot(*config.Config)`.
@@ -195,19 +216,9 @@ To expose the new behavior, unqualified imports to the package must be added in
 [init()]: https://golang.org/doc/effective_go.html#init
 [main.go]: https://github.com/coreos/clair/blob/master/cmd/clair/main.go
 
-## Related Links
+### Are there any public presentations on Clair?
-
-### Talks & Slides
 
 - _Clair: The Container Image Security Analyzer @ ContainerDays Boston 2016_ - [Event](http://dynamicinfradays.org/events/2016-boston/) [Video](https://www.youtube.com/watch?v=Kri67PtPv6s) [Slides](https://docs.google.com/presentation/d/1ExQGZs-pQ56TpW_ifcUl2l_ml87fpCMY6-wdug87OFU)
 - _Identifying Common Vulnerabilities and Exposures in Containers with Clair @ CoreOS Fest 2016_ - [Event](https://coreos.com/fest/) [Video](https://www.youtube.com/watch?v=YDCa51BK2q0) [Slides](https://docs.google.com/presentation/d/1pHSI_5LcjnZzZBPiL1cFTZ4LvhzKtzh86eE010XWNLY)
 - _Clair: A Container Image Security Analyzer @  Microservices NYC_ - [Event](https://www.meetup.com/Microservices-NYC/events/230023492/) [Video](https://www.youtube.com/watch?v=ynwKi2yhIX4) [Slides](https://docs.google.com/presentation/d/1ly9wQKQIlI7rlb0JNU1_P-rPDHU4xdRCCM3rxOdjcgc)
 - _Clair: A Container Image Security Analyzer @ Container Orchestration NYC_ - [Event](https://www.meetup.com/Container-Orchestration-NYC/events/229779466/) [Video](https://www.youtube.com/watch?v=wTfCOUDNV_M) [Slides](https://docs.google.com/presentation/d/1ly9wQKQIlI7rlb0JNU1_P-rPDHU4xdRCCM3rxOdjcgc)
-
-### Projects Integrating with Clair
-
-- [Quay](https://quay.io): the first container registry to integrate with Clair
-- [Dockyard](https://github.com/containerops/dockyard): an open source container registry with Clair integration
-- [Clairctl](https://github.com/jgsqware/clairctl): a lightweight command-line tool for working locally with Clair and generate HTML report
-- [Clair w/ SQS](https://github.com/zalando/clair-sqs): a container containing Clair and additional processes that integrate Clair with [Amazon SQS](https://aws.amazon.com/sqs)
-- [Klar](https://github.com/optiopay/klar): a simple command-line integration of Clair and Docker registry, designed to be used in scripts and CI

ROADMAP.md

@@ -7,10 +7,9 @@ The [milestones defined in GitHub](https://github.com/coreos/clair/milestones) r
 
 The roadmap below outlines new features that will be added to Clair, and while subject to change, define what future stable will look like.
 
-### Clair 2.0 (July)
+- Support multiple namespaces per image
-
+  - This enables language-level package managers (e.g. npm, pip)
-- Standardize component registration
+- Improve coverage and readability of documentation
-- Revisit database implementation
+- Decouple the project from Postgres
-- Improve release distribution
+- gRPC API supporting direct uploads of images
-- Address client UX
+- Support operating Clair without internet access
-- Expand detection capabilities

api/router.go

@@ -52,7 +52,7 @@ func (rtr router) ServeHTTP(w http.ResponseWriter, r *http.Request) {
 		return
 	}
 
-	log.Infof("%s %d %s %s", http.StatusNotFound, r.Method, r.RequestURI, r.RemoteAddr)
+	log.Infof("%d %s %s %s", http.StatusNotFound, r.Method, r.RequestURI, r.RemoteAddr)
 	http.NotFound(w, r)
 }
 

cmd/clair/main.go

@@ -30,6 +30,7 @@ import (
 	"github.com/coreos/clair"
 	"github.com/coreos/clair/api"
 	"github.com/coreos/clair/database"
+	"github.com/coreos/clair/ext/imagefmt"
 	"github.com/coreos/clair/pkg/stopper"
 
 	// Register database driver.
@@ -123,6 +124,7 @@ func main() {
 	flagConfigPath := flag.String("config", "/etc/clair/config.yaml", "Load configuration from the specified file.")
 	flagCPUProfilePath := flag.String("cpu-profile", "", "Write a CPU profile to the specified file before exiting.")
 	flagLogLevel := flag.String("log-level", "info", "Define the logging level.")
+	flagInsecureTLS := flag.Bool("insecure-tls", false, "Disable TLS server's certificate chain and hostname verification when pulling layers.")
 	flag.Parse()
 
 	// Check for dependencies.
@@ -149,5 +151,11 @@ func main() {
 		defer stopCPUProfiling(startCPUProfiling(*flagCPUProfilePath))
 	}
 
+	// Enable TLS server's certificate chain and hostname verification
+	// when pulling layers if specified
+	if *flagInsecureTLS {
+		imagefmt.SetInsecureTLS(*flagInsecureTLS)
+	}
+
 	Boot(config)
 }

contrib/analyze-local-images/main.go

@@ -154,10 +154,7 @@ func AnalyzeLocalImage(imageName string, minSeverity types.Priority, endpoint, m
 
 	// Retrieve history.
 	log.Println("Retrieving image history")
-	layerIDs, err := historyFromManifest(tmpPath)
+	layerIDs, err = historyFromCommand(imageName)
-	if err != nil {
-		layerIDs, err = historyFromCommand(imageName)
-	}
 	if err != nil || len(layerIDs) == 0 {
 		return fmt.Errorf("Could not get image's history: %s", err)
 	}
@@ -270,6 +267,8 @@ func AnalyzeLocalImage(imageName string, minSeverity types.Priority, endpoint, m
 		fmt.Printf("%s No vulnerabilities were detected in your image\n", color.GreenString("Success!"))
 	} else if !hasVisibleVulnerabilities {
 		fmt.Printf("%s No vulnerabilities matching the minimum severity level were detected in your image\n", color.YellowString("NOTE:"))
+	} else {
+		return fmt.Errorf("A total of %d vulnerabilities have been detected in your image", len(vulnerabilities))
 	}
 
 	return nil
@@ -307,33 +306,6 @@ func save(imageName, path string) error {
 	return nil
 }
 
-func historyFromManifest(path string) ([]string, error) {
-	mf, err := os.Open(path + "/manifest.json")
-	if err != nil {
-		return nil, err
-	}
-	defer mf.Close()
-
-	// https://github.com/docker/docker/blob/master/image/tarexport/tarexport.go#L17
-	type manifestItem struct {
-		Config   string
-		RepoTags []string
-		Layers   []string
-	}
-
-	var manifest []manifestItem
-	if err = json.NewDecoder(mf).Decode(&manifest); err != nil {
-		return nil, err
-	} else if len(manifest) != 1 {
-		return nil, err
-	}
-	var layers []string
-	for _, layer := range manifest[0].Layers {
-		layers = append(layers, strings.TrimSuffix(layer, "/layer.tar"))
-	}
-	return layers, nil
-}
-
 func historyFromCommand(imageName string) ([]string, error) {
 	var stderr bytes.Buffer
 	cmd := exec.Command("docker", "history", "-q", "--no-trunc", imageName)

docker-compose.yml

@@ -9,7 +9,7 @@ services:
 
   clair:
     container_name: clair_clair
-    image: quay.io/coreos/clair:v1.2.2
+    image: quay.io/coreos/clair-git:latest
     restart: unless-stopped
     depends_on:
       - postgres

ext/featurefmt/apk/apk.go

@@ -66,6 +66,10 @@ func (l lister) ListFeatures(files tarutil.FilesMap) ([]database.FeatureVersion,
 			} else {
 				ipkg.Version = version
 			}
+		case line == "":
+			// Restart if the parser reaches another package definition before
+			// creating a valid package.
+			ipkg = database.FeatureVersion{}
 		}
 
 		// If we have a whole feature, store it in the set and try to parse a new

ext/featurefmt/apk/testdata/installed

@@ -20,6 +20,8 @@ R:ld-musl-x86_64.so.1
 a:0:0:755
 Z:Q1KUwsFGLHn/enpN9+QIpK/FmixtQ=
 
+P:invalidPackageWithoutAVersion
+
 C:Q1yhJHGSZ80L7cL0y4UKKGrBPwrUQ=
 P:busybox
 V:1.24.2-r9

ext/featurefmt/dpkg/dpkg.go

@@ -97,6 +97,9 @@ func (l lister) ListFeatures(files tarutil.FilesMap) ([]database.FeatureVersion,
 			} else {
 				pkg.Version = version
 			}
+		} else if line == "" {
+			pkg.Feature.Name = ""
+			pkg.Version = ""
 		}
 
 		// Add the package to the result array if we have all the informations

ext/featurefmt/dpkg/testdata/status

@@ -60,6 +60,8 @@ This package is not necessary for most modern Linux systems, where the udev
 subsystem provides a more dynamic mechanism for device file management.
 Original-Maintainer: Debian QA Group <packages@qa.debian.org>
 
+Package: brokenPackageWithNoVersionThatShouldGetThrownOut
+
 Package: libgcc1
 Status: install ok installed
 Priority: required

ext/imagefmt/driver.go

@@ -21,6 +21,7 @@
 package imagefmt
 
 import (
+	"crypto/tls"
 	"fmt"
 	"io"
 	"math"
@@ -38,6 +39,10 @@ var (
 	// ErrCouldNotFindLayer is returned when we could not download or open the layer file.
 	ErrCouldNotFindLayer = commonerr.NewBadRequestError("could not find layer")
 
+	// insecureTLS controls whether TLS server's certificate chain and hostname are verified
+	// when pulling layers, verified in default.
+	insecureTLS = false
+
 	log = capnslog.NewPackageLogger("github.com/coreos/clair", "ext/imagefmt")
 
 	extractorsM sync.RWMutex
@@ -116,7 +121,11 @@ func Extract(format, path string, headers map[string]string, toExtract []string)
 		}
 
 		// Send the request and handle the response.
-		r, err := http.DefaultClient.Do(request)
+		tr := &http.Transport{
+			TLSClientConfig: &tls.Config{InsecureSkipVerify: insecureTLS},
+		}
+		client := &http.Client{Transport: tr}
+		r, err := client.Do(request)
 		if err != nil {
 			log.Warningf("could not download layer: %s", err)
 			return nil, ErrCouldNotFindLayer
@@ -148,3 +157,9 @@ func Extract(format, path string, headers map[string]string, toExtract []string)
 
 	return nil, commonerr.NewBadRequestError(fmt.Sprintf("unsupported image format '%s'", format))
 }
+
+// SetInsecureTLS sets the insecureTLS to control whether TLS server's certificate chain
+// and hostname are verified when pulling layers.
+func SetInsecureTLS(insecure bool) {
+	insecureTLS = insecure
+}

ext/vulnsrc/oracle/oracle.go

@@ -85,6 +85,36 @@ func init() {
 	vulnsrc.RegisterUpdater("oracle", &updater{})
 }
 
+func compareELSA(left, right int) int {
+	// Fast path equals.
+	if right == left {
+		return 0
+	}
+
+	lstr := strconv.Itoa(left)
+	rstr := strconv.Itoa(right)
+
+	for i := range lstr {
+		// If right is too short to be indexed, left is greater.
+		if i >= len(rstr) {
+			return 1
+		}
+
+		ldigit, _ := strconv.Atoi(string(lstr[i]))
+		rdigit, _ := strconv.Atoi(string(rstr[i]))
+
+		if ldigit > rdigit {
+			return 1
+		} else if ldigit < rdigit {
+			return -1
+		}
+		continue
+	}
+
+	// Everything the length of left is the same.
+	return len(lstr) - len(rstr)
+}
+
 func (u *updater) Update(datastore database.Datastore) (resp vulnsrc.UpdateResponse, err error) {
 	log.Info("fetching Oracle Linux vulnerabilities")
 
@@ -115,7 +145,7 @@ func (u *updater) Update(datastore database.Datastore) (resp vulnsrc.UpdateRespo
 		r := elsaRegexp.FindStringSubmatch(line)
 		if len(r) == 2 {
 			elsaNo, _ := strconv.Atoi(r[1])
-			if elsaNo > firstELSA {
+			if compareELSA(elsaNo, firstELSA) > 0 {
 				elsaList = append(elsaList, elsaNo)
 			}
 		}

ext/vulnsrc/oracle/oracle_test.go

@@ -115,3 +115,23 @@ func TestOracleParser(t *testing.T) {
 		}
 	}
 }
+
+func TestELSAComparison(t *testing.T) {
+	var table = []struct {
+		left     int
+		right    int
+		expected int
+	}{
+		{20170935, 20170935, 0},
+		{20170934, 20170935, -1},
+		{20170936, 20170935, 1},
+
+		{20170935, 201709331, 1},
+		{201709351, 20170935, 1},
+		{201709331, 20170935, -1},
+	}
+
+	for _, tt := range table {
+		assert.Equal(t, tt.expected, compareELSA(tt.left, tt.right))
+	}
+}

ext/vulnsrc/ubuntu/ubuntu.go

@@ -377,8 +377,11 @@ func parseUbuntuCVE(fileContent io.Reader) (vulnerability database.Vulnerability
 				// Create and add the new package.
 				featureVersion := database.FeatureVersion{
 					Feature: database.Feature{
-						Namespace: database.Namespace{Name: "ubuntu:" + database.UbuntuReleasesMapping[md["release"]]},
+						Namespace: database.Namespace{
-						Name:      md["package"],
+							Name:          "ubuntu:" + database.UbuntuReleasesMapping[md["release"]],
+							VersionFormat: dpkg.ParserName,
+						},
+						Name: md["package"],
 					},
 					Version: version,
 				}

ext/vulnsrc/ubuntu/ubuntu_test.go

@@ -24,6 +24,7 @@ import (
 
 	"github.com/coreos/clair/database"
 	"github.com/coreos/clair/ext/versionfmt"
+	"github.com/coreos/clair/ext/versionfmt/dpkg"
 )
 
 func TestUbuntuParser(t *testing.T) {
@@ -46,22 +47,31 @@ func TestUbuntuParser(t *testing.T) {
 		expectedFeatureVersions := []database.FeatureVersion{
 			{
 				Feature: database.Feature{
-					Namespace: database.Namespace{Name: "ubuntu:14.04"},
+					Namespace: database.Namespace{
-					Name:      "libmspack",
+						Name:          "ubuntu:14.04",
+						VersionFormat: dpkg.ParserName,
+					},
+					Name: "libmspack",
 				},
 				Version: versionfmt.MaxVersion,
 			},
 			{
 				Feature: database.Feature{
-					Namespace: database.Namespace{Name: "ubuntu:15.04"},
+					Namespace: database.Namespace{
-					Name:      "libmspack",
+						Name:          "ubuntu:15.04",
+						VersionFormat: dpkg.ParserName,
+					},
+					Name: "libmspack",
 				},
 				Version: "0.4-3",
 			},
 			{
 				Feature: database.Feature{
-					Namespace: database.Namespace{Name: "ubuntu:15.10"},
+					Namespace: database.Namespace{
-					Name:      "libmspack-anotherpkg",
+						Name:          "ubuntu:15.10",
+						VersionFormat: dpkg.ParserName,
+					},
+					Name: "libmspack-anotherpkg",
 				},
 				Version: "0.1",
 			},