arno/kube-bench
1
0
Fork 0
mirror of https://github.com/aquasecurity/kube-bench.git synced 2024-11-22 08:08:07 +00:00
Code Issues Releases Wiki Activity

Improve docs (#437)

Browse Source
This commit is contained in:
Alexey Pyltsyn 2019-10-24 11:15:29 +03:00 committed by Liz Rice
parent bf383ec1f7
commit 7a2cc3f554
2 changed files with 93 additions and 63 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

78
README.md
View File

@ -34,7 +34,7 @@ Table of Contents
* [Omitting checks](#omitting-checks) * [Omitting checks](#omitting-checks)
* [Roadmap](#roadmap) * [Roadmap](#roadmap)
* [Testing locally with kind](#testing-locally-with-kind) * [Testing locally with kind](#testing-locally-with-kind)
* [GitHub Issues](#github-issues) * [Contributing](#contributing)
* [Bugs](#bugs) * [Bugs](#bugs)
* [Features](#features) * [Features](#features)
* [Pull Requests](#pull-requests) * [Pull Requests](#pull-requests)
@ -50,7 +50,7 @@ kube-bench supports the tests for Kubernetes as defined in the CIS Benchmarks 1.
By default kube-bench will determine the test set to run based on the Kubernetes version running on the machine. By default kube-bench will determine the test set to run based on the Kubernetes version running on the machine.
There is also preliminary support for Red Hat's Openshift Hardening Guide for 3.10 and 3.11. Please note that kube-bench does not automatically detect Openshift - see below. There is also preliminary support for Red Hat's OpenShift Hardening Guide for 3.10 and 3.11. Please note that kube-bench does not automatically detect OpenShift - see below.
## Installation ## Installation
@ -62,31 +62,30 @@ You can choose to
## Running kube-bench ## Running kube-bench
If you run kube-bench directly from the command line you may need to be root / sudo in order to have access to all the config files. If you run kube-bench directly from the command line you may need to be root/sudo in order to have access to all the config files.
kube-bench automatically selects which `controls` to use based on the detected kube-bench automatically selects which `controls` to use based on the detected
node type and the version of kubernetes a cluster is running. This behaviour node type and the version of Kubernetes a cluster is running. This behavior
can be overridden by specifying the `master` or `node` subcommand and the can be overridden by specifying the `master` or `node` subcommand and the
`--version` flag on the command line. `--version` flag on the command line.
The kubernetes version can also be set with the KUBE_BENCH_VERSION environment variable. The Kubernetes version can also be set with the `KUBE_BENCH_VERSION` environment variable.
The value of `--version` takes precedence over the value of KUBE_BENCH_VERSION. The value of `--version` takes precedence over the value of `KUBE_BENCH_VERSION`.
For example: For example, run kube-bench against a master with version auto-detection:
run kube-bench against a master with version auto-detection:
``` ```
kube-bench master kube-bench master
``` ```
or run kube-bench against a node with the node `controls` for kubernetes Or run kube-bench against a node with the node `controls` for Kubernetes version 1.13:
version 1.13:
``` ```
kube-bench node --version 1.13 kube-bench node --version 1.13
``` ```
`controls` for the various versions of kubernetes can be found in directories `controls` for the various versions of Kubernetes can be found in directories
with same name as the kubernetes versions under `cfg/`, for example `cfg/1.13`. with same name as the Kubernetes versions under `cfg/`, for example `cfg/1.13`.
`controls` are also organized by distribution under the `cfg` directory for `controls` are also organized by distribution under the `cfg` directory for
example `cfg/ocp-3.10`. example `cfg/ocp-3.10`.
@ -110,7 +109,7 @@ You can use your own configs by mounting them over the default ones in `/opt/kub
docker run --pid=host -v /etc:/etc:ro -v /var:/var:ro -t -v path/to/my-config.yaml:/opt/kube-bench/cfg/config.yam -v $(which kubectl):/usr/bin/kubectl -v ~/.kube:/.kube -e KUBECONFIG=/.kube/config aquasec/kube-bench:latest [master|node] docker run --pid=host -v /etc:/etc:ro -v /var:/var:ro -t -v path/to/my-config.yaml:/opt/kube-bench/cfg/config.yam -v $(which kubectl):/usr/bin/kubectl -v ~/.kube:/.kube -e KUBECONFIG=/.kube/config aquasec/kube-bench:latest [master|node]
``` ```
### Running in a kubernetes cluster ### Running in a Kubernetes cluster
You can run kube-bench inside a pod, but it will need access to the host's PID namespace in order to check the running processes, as well as access to some directories on the host where config files and other files are stored. You can run kube-bench inside a pod, but it will need access to the host's PID namespace in order to check the running processes, as well as access to some directories on the host where config files and other files are stored.
@ -157,7 +156,7 @@ There are two significant differences on EKS:
### Installing from a container ### Installing from a container
This command copies the kube-bench binary and configuration files to your host from the Docker container: This command copies the kube-bench binary and configuration files to your host from the Docker container:
** binaries compiled for linux-x86-64 only (so they won't run on OSX or Windows) ** ** binaries compiled for linux-x86-64 only (so they won't run on macOS or Windows) **
``` ```
docker run --rm -v `pwd`:/host aquasec/kube-bench:latest install docker run --rm -v `pwd`:/host aquasec/kube-bench:latest install
``` ```
@ -166,7 +165,7 @@ You can then run `./kube-bench [master|node]`.
### Installing from sources ### Installing from sources
If Go is installed on the target machines, you can simply clone this repository and run as follows (assuming your [$GOPATH is set](https://github.com/golang/go/wiki/GOPATH)): If Go is installed on the target machines, you can simply clone this repository and run as follows (assuming your [`GOPATH` is set](https://github.com/golang/go/wiki/GOPATH)):
```shell ```shell
go get github.com/aquasecurity/kube-bench go get github.com/aquasecurity/kube-bench
@ -180,17 +179,17 @@ go build -o kube-bench .
# Run the all checks # Run the all checks
./kube-bench ./kube-bench
``` ```
## Running on OpenShift ## Running on OpenShift
kube-bench includes a set of test files for Red Hat's OpenShift hardening guide for OCP 3.10 and 3.11. To run this you will need to specify `--version ocp-3.10` when you run the `kube-bench` command (either directly or through YAML). This config version is valid for OCP 3.10 and 3.11. kube-bench includes a set of test files for Red Hat's OpenShift hardening guide for OCP 3.10 and 3.11. To run this you will need to specify `--version ocp-3.10` when you run the `kube-bench` command (either directly or through YAML). This config version is valid for OCP 3.10 and 3.11.
## Output ## Output
There are three output states There are three output states:
- [PASS] and [FAIL] indicate that a test was run successfully, and it either passed or failed - [PASS] and [FAIL] indicate that a test was run successfully, and it either passed or failed.
- [WARN] means this test needs further attention, for example it is a test that needs to be run manually - [WARN] means this test needs further attention, for example it is a test that needs to be run manually.
- [INFO] is informational output that needs no further action. - [INFO] is informational output that needs no further action.
Note: Note:
@ -209,7 +208,7 @@ You can read more about `kube-bench` configuration in our [documentation](docs/R
## Test config YAML representation ## Test config YAML representation
The tests (or "controls") are represented as YAML documents (installed by default into ./cfg). There are different versions of these test YAML files reflecting different versions of the CIS Kubernetes Benchmark. You will find more information about the test file YAML definitions in our [documentation](docs/README.md). The tests (or "controls") are represented as YAML documents (installed by default into `./cfg`). There are different versions of these test YAML files reflecting different versions of the CIS Kubernetes Benchmark. You will find more information about the test file YAML definitions in our [documentation](docs/README.md).
### Omitting checks ### Omitting checks
@ -225,14 +224,15 @@ If you decide that a recommendation is not appropriate for your environment, you
No tests will be run for this check and the output will be marked [INFO]. No tests will be run for this check and the output will be marked [INFO].
# Roadmap ## Roadmap
Going forward we plan to release updates to kube-bench to add support for new releases of the Benchmark, which in turn we can anticipate being made for each new Kubernetes release. Going forward we plan to release updates to kube-bench to add support for new releases of the Benchmark, which in turn we can anticipate being made for each new Kubernetes release.
We welcome PRs and issue reports. We welcome PRs and issue reports.
# Testing locally with kind ## Testing locally with kind
Our makefile contains targets to test your current version of kube-bench inside a [Kind](https://kind.sigs.k8s.io/) cluster. This can be very handy if you don't want to run a real kubernetes cluster for development purpose. Our makefile contains targets to test your current version of kube-bench inside a [Kind](https://kind.sigs.k8s.io/) cluster. This can be very handy if you don't want to run a real Kubernetes cluster for development purpose.
First you'll need to create the cluster using `make kind-test-cluster` this will create a new cluster if it cannot be found on your machine. By default the cluster is named `kube-bench` but you can change the name by using the environment variable `KIND_PROFILE`. First you'll need to create the cluster using `make kind-test-cluster` this will create a new cluster if it cannot be found on your machine. By default the cluster is named `kube-bench` but you can change the name by using the environment variable `KIND_PROFILE`.
@ -243,3 +243,35 @@ Next you'll have to build the kube-bench docker image using `make build-docker`,
Finally we can use the `make kind-run` target to run the current version of kube-bench in the cluster and follow the logs of pods created. (Ctrl+C to exit) Finally we can use the `make kind-run` target to run the current version of kube-bench in the cluster and follow the logs of pods created. (Ctrl+C to exit)
Everytime you want to test a change, you'll need to rebuild the docker image and push it to cluster before running it again. ( `make build-docker kind-push kind-run` ) Everytime you want to test a change, you'll need to rebuild the docker image and push it to cluster before running it again. ( `make build-docker kind-push kind-run` )
## Contributing
### Bugs
If you think you have found a bug please follow the instructions below.
- Please spend a small amount of time giving due diligence to the issue tracker. Your issue might be a duplicate.
- Open a [new issue](https://github.com/aquasecurity/kube-bench/issues/new) if a duplicate doesn't already exist.
- Note the version of kube-bench you are running (from `kube-bench version`) and the command line options you are using.
- Note the version of Kubernetes you are running (from `kubectl version` or `oc version` for OpenShift).
- Set `-v 10` command line option and save the log output. Please paste this into your issue.
- Remember users might be searching for your issue in the future, so please give it a meaningful title to help others.
### Features
We also use the GitHub issue tracker to track feature requests. If you have an idea to make kube-bench even more awesome follow the steps below.
- Open a [new issue](https://github.com/aquasecurity/kube-bench/issues/new).
- Remember users might be searching for your issue in the future, so please give it a meaningful title to helps others.
- Clearly define the use case, using concrete examples. For example: I type `this` and kube-bench does `that`.
- If you would like to include a technical design for your feature please feel free to do so.
### Pull Requests
We welcome pull requests!
- Your PR is more likely to be accepted if it focuses on just one change.
- Please include a comment with the results before and after your change.
- Your PR is more likely to be accepted if it includes tests. (We have not historically been very strict about tests, but we would like to improve this!).
- You're welcome to submit a draft PR if you would like early feedback on an idea or an approach.
- Happy coding!

78
docs/README.md
View File

@ -2,22 +2,22 @@
`kube-bench` runs checks specified in `controls` files that are a YAML `kube-bench` runs checks specified in `controls` files that are a YAML
representation of the CIS Kubernetes Benchmark checks. There is a representation of the CIS Kubernetes Benchmark checks. There is a
`controls` file per kubernetes version and node type. `controls` file per Kubernetes version and node type.
`controls` for the various versions of kubernetes can be found in directories `controls` for the various versions of Kubernetes can be found in directories
with same name as the kubernetes versions under `cfg/`, for example `cfg/1.12`. with same name as the Kubernetes versions under `cfg/`, for example `cfg/1.12`.
`controls` are also organized by distribution under the `cfg` directory for `controls` are also organized by distribution under the `cfg` directory for
example `cfg/ocp-3.10`. example `cfg/ocp-3.10`.
## Controls ## Controls
`controls` is a YAML document that contains checks that must be run against a `controls` is a YAML document that contains checks that must be run against a
specific kubernetes node type, master or node and version. specific Kubernetes node type, master or node and version.
`controls` is the fundamental input to `kube-bench`. The following is an example `controls` is the fundamental input to `kube-bench`. The following is an example
of a basic `controls`: of a basic `controls`:
``` ```yml
--- ---
controls: controls:
id: 1 id: 1
@ -58,29 +58,29 @@ groups:
scored: true scored: true
``` ```
`controls` is composed of a hierachy of groups, sub-groups and checks. Each of `controls` is composed of a hierarchy of groups, sub-groups and checks. Each of
the `controls` components have an id and a text description which are displayed the `controls` components have an id and a text description which are displayed
in the `kube-bench` output. in the `kube-bench` output.
`type` specifies what kubernetes node type a `controls` is for. Possible values `type` specifies what Kubernetes node type a `controls` is for. Possible values
for `type` are `master` and `node`. for `type` are `master` and `node`.
## Groups ## Groups
`groups` is list of subgroups which test the various kubernetes components `groups` is list of subgroups which test the various Kubernetes components
that run on the node type specified in the `controls`. that run on the node type specified in the `controls`.
For example one subgroup checks parameters passed to the apiserver binary, while For example one subgroup checks parameters passed to the apiserver binary, while
another subgroup checks parameters passed to the controller-manager binary. another subgroup checks parameters passed to the controller-manager binary.
``` ```yml
groups: groups:
- id: 1.1 - id: 1.1
text: API Server text: API Server
... # ...
- id: 1.2 - id: 1.2
text: Scheduler text: Scheduler
... # ...
``` ```
These subgroups have `id`, `text` fields which serve the same purposes described These subgroups have `id`, `text` fields which serve the same purposes described
@ -89,7 +89,7 @@ in the previous paragraphs. The most important part of the subgroup is the
This is an example of a subgroup and checks in the subgroup. This is an example of a subgroup and checks in the subgroup.
``` ```yml
id: 1.1 id: 1.1
text: API Server text: API Server
checks: checks:
@ -97,12 +97,12 @@ checks:
text: "Ensure that the --allow-privileged argument is set (Scored)" text: "Ensure that the --allow-privileged argument is set (Scored)"
audit: "ps -ef | grep kube-apiserver | grep -v grep" audit: "ps -ef | grep kube-apiserver | grep -v grep"
tests: tests:
... # ...
- id: 1.1.2 - id: 1.1.2
text: "Ensure that the --anonymous-auth argument is set to false (Not Scored)" text: "Ensure that the --anonymous-auth argument is set to false (Not Scored)"
audit: "ps -ef | grep kube-apiserver | grep -v grep" audit: "ps -ef | grep kube-apiserver | grep -v grep"
tests: tests:
... # ...
``` ```
`kube-bench` supports running a subgroup by specifying the subgroup `id` on the `kube-bench` supports running a subgroup by specifying the subgroup `id` on the
@ -110,18 +110,18 @@ command line, with the flag `--group` or `-g`.
## Check ## Check
The CIS Kubernetes Benchmark recommends configurations to harden kubernetes The CIS Kubernetes Benchmark recommends configurations to harden Kubernetes
components. These recommendations are usually configuration options, and can be components. These recommendations are usually configuration options, and can be
specified by flags to kubernetes binaries, or in configuration files. specified by flags to Kubernetes binaries, or in configuration files.
The Benchmark also provides commands to audit a kubernetes installation, identify The Benchmark also provides commands to audit a Kubernetes installation, identify
places where the cluster security can be improved, and steps to remediate these places where the cluster security can be improved, and steps to remediate these
identified problems. identified problems.
In `kube-bench`, `check` objects embody these recommendations. This an example In `kube-bench`, `check` objects embody these recommendations. This an example
`check` object: `check` object:
``` ```yml
id: 1.1.1 id: 1.1.1
text: "Ensure that the --anonymous-auth argument is set to false (Not Scored)" text: "Ensure that the --anonymous-auth argument is set to false (Not Scored)"
audit: "ps -ef | grep kube-apiserver | grep -v grep" audit: "ps -ef | grep kube-apiserver | grep -v grep"
@ -139,7 +139,7 @@ remediation: |
scored: false scored: false
``` ```
A `check` object has an `id`, a `text`, an `audit` , a `tests`,`remediation` A `check` object has an `id`, a `text`, an `audit`, a `tests`, `remediation`
and `scored` fields. and `scored` fields.
`kube-bench` supports running individual checks by specifying the check's `id` `kube-bench` supports running individual checks by specifying the check's `id`
@ -164,40 +164,39 @@ The are two ways to extract keywords from the output of the `audit` command,
command is usually a `ps` command and a `grep` for the binary whose flag we are command is usually a `ps` command and a `grep` for the binary whose flag we are
checking: checking:
``` ```sh
ps -ef | grep somebinary | grep -v grep ps -ef | grep somebinary | grep -v grep
``` ```
Here is an example usage of the `flag` option: Here is an example usage of the `flag` option:
``` ```yml
... # ...
audit: "ps -ef | grep kube-apiserver | grep -v grep" audit: "ps -ef | grep kube-apiserver | grep -v grep"
tests: tests:
test_items: test_items:
- flag: "--anonymous-auth" - flag: "--anonymous-auth"
... # ...
``` ```
`path` is used when the keyword is an option set in a JSON or YAML config file. `path` is used when the keyword is an option set in a JSON or YAML config file.
The associated `audit` command is usually `cat /path/to/config-yaml-or-json`. The associated `audit` command is usually `cat /path/to/config-yaml-or-json`.
For example: For example:
``` ```yml
... # ...
text: "Ensure that the --anonymous-auth argument is set to false (Not Scored)" text: "Ensure that the --anonymous-auth argument is set to false (Not Scored)"
audit: "cat /path/to/some/config" audit: "cat /path/to/some/config"
tests: tests:
test_items: test_items:
- path: "{.someoption.value}" - path: "{.someoption.value}"
... # ...
``` ```
`test_item` compares the output of the audit command and keywords using the `test_item` compares the output of the audit command and keywords using the
`set` and `compare` fields. `set` and `compare` fields.
``` ```yml
test_items: test_items:
- flag: "--anonymous-auth" - flag: "--anonymous-auth"
compare: compare:
@ -236,7 +235,7 @@ The `op` (operations) currently supported in `kube-bench` are:
## Configuration and Variables ## Configuration and Variables
Kubernetes component configuration and binary file locations and names Kubernetes component configuration and binary file locations and names
vary based on cluster deployment methods and kubernetes distribution used. vary based on cluster deployment methods and Kubernetes distribution used.
For this reason, the locations of these binaries and config files are configurable For this reason, the locations of these binaries and config files are configurable
by editing the `cfg/config.yaml` file and these binaries and files can be by editing the `cfg/config.yaml` file and these binaries and files can be
referenced in a `controls` file via variables. referenced in a `controls` file via variables.
@ -245,7 +244,7 @@ The `cfg/config.yaml` file is a global configuration file. Configuration files
can be created for specific Kubernetes versions (distributions). Values in the can be created for specific Kubernetes versions (distributions). Values in the
version specific config overwrite similar values in `cfg/config.yaml`. version specific config overwrite similar values in `cfg/config.yaml`.
For example, the kube-apiserver in Redhat OCP distribution is run as For example, the kube-apiserver in Red Hat OCP distribution is run as
`hypershift openshift-kube-apiserver` instead of the default `kube-apiserver`. `hypershift openshift-kube-apiserver` instead of the default `kube-apiserver`.
This difference can be specified by editing the `master.apiserver.defaultbin` This difference can be specified by editing the `master.apiserver.defaultbin`
entry `cfg/ocp-3.10/config.yaml`. entry `cfg/ocp-3.10/config.yaml`.
@ -277,7 +276,7 @@ Every node type has a subsection that specifies the main configurations items.
- `bins`: A list of candidate binaries for a component. `kube-bench` checks this - `bins`: A list of candidate binaries for a component. `kube-bench` checks this
list and selects the first binary that is running on the node. list and selects the first binary that is running on the node.
if none of the binaries in `bins` list is running, `kube-bench` checks if the If none of the binaries in `bins` list is running, `kube-bench` checks if the
binary specified by `defaultbin` is running and terminates if none of the binary specified by `defaultbin` is running and terminates if none of the
binaries in both `bins` and `defaultbin` is running. binaries in both `bins` and `defaultbin` is running.
@ -286,11 +285,11 @@ Every node type has a subsection that specifies the main configurations items.
the selected API server binary with the variable `$apiserverbin` in an `audit` the selected API server binary with the variable `$apiserverbin` in an `audit`
command. command.
``` ```yml
id: 1.1.1 id: 1.1.1
text: "Ensure that the --anonymous-auth argument is set to false (Scored)" text: "Ensure that the --anonymous-auth argument is set to false (Scored)"
audit: "ps -ef | grep $apiserverbin | grep -v grep" audit: "ps -ef | grep $apiserverbin | grep -v grep"
... # ...
``` ```
- `confs`: A list of candidate configuration files for a component. `kube-bench` - `confs`: A list of candidate configuration files for a component. `kube-bench`
@ -303,12 +302,11 @@ Every node type has a subsection that specifies the main configurations items.
selected API server config file with the variable `$apiserverconf` in an `audit` selected API server config file with the variable `$apiserverconf` in an `audit`
command. command.
``` ```yml
id: 1.4.1 id: 1.4.1
text: "Ensure that the API server pod specification file permissions are text: "Ensure that the API server pod specification file permissions are
set to 644 or more restrictive (Scored)" set to 644 or more restrictive (Scored)"
audit: "/bin/sh -c 'if test -e $apiserverconf; then stat -c %a $apiserverconf; fi'" audit: "/bin/sh -c 'if test -e $apiserverconf; then stat -c %a $apiserverconf; fi'"
``` ```
- `svcs`: A list of candidates unitfiles for a component. `kube-bench` checks this - `svcs`: A list of candidates unitfiles for a component. `kube-bench` checks this
@ -320,9 +318,9 @@ Every node type has a subsection that specifies the main configurations items.
kubelet unitfile is referenced with `$kubeletsvc` in the `remediation` of the kubelet unitfile is referenced with `$kubeletsvc` in the `remediation` of the
`check`. `check`.
``` ```yml
id: 2.1.1 id: 2.1.1
... # ...
remediation: | remediation: |
Edit the kubelet service file $kubeletsvc Edit the kubelet service file $kubeletsvc
on each worker node and set the below parameter in KUBELET_SYSTEM_PODS_ARGS variable. on each worker node and set the below parameter in KUBELET_SYSTEM_PODS_ARGS variable.
@ -330,7 +328,7 @@ Every node type has a subsection that specifies the main configurations items.
Based on your system, restart the kubelet service. For example: Based on your system, restart the kubelet service. For example:
systemctl daemon-reload systemctl daemon-reload
systemctl restart kubelet.service systemctl restart kubelet.service
... # ...
``` ```
- `kubeconfig`: A list of candidate kubeconfig files for a component. `kube-bench` - `kubeconfig`: A list of candidate kubeconfig files for a component. `kube-bench`
@ -343,10 +341,10 @@ Every node type has a subsection that specifies the main configurations items.
selected kubelet kubeconfig is referenced with `$kubeletkubeconfig` in the selected kubelet kubeconfig is referenced with `$kubeletkubeconfig` in the
`audit` command. `audit` command.
``` ```yml
id: 2.2.1 id: 2.2.1
text: "Ensure that the kubelet.conf file permissions are set to 644 or text: "Ensure that the kubelet.conf file permissions are set to 644 or
more restrictive (Scored)" more restrictive (Scored)"
audit: "/bin/sh -c 'if test -e $kubeletkubeconfig; then stat -c %a $kubeletkubeconfig; fi'" audit: "/bin/sh -c 'if test -e $kubeletkubeconfig; then stat -c %a $kubeletkubeconfig; fi'"
... # ...
``` ```