1
0
mirror of https://github.com/aquasecurity/kube-bench.git synced 2024-12-04 22:08:22 +00:00
kube-bench/docs/controls.md
Yoav Rotem 0d1bd2bbd9
mkdocs support and update docs (#884)
* Delete README.md

* Edit readme and separate into different files

* Update README.md

* Update Running.md

* Update CONTRIBUTING.md

* Create Contributing.md

* Add files via upload

* Update Index.md

* Rename Flags and Commands.md to Flags_and_commands.md

* Rename Index.md to index.md

* Create mkdocs.yml

* Delete images directory

* Update README.md

* Update README.md

* Update README.md

* Update README.md

* Update README.md

* Update README.md

* Create mkdocs-dev.yaml

* Create mkdocs-latest.yaml

* Update mkdocs.yml

* Update mkdocs.yml

* Update mkdocs.yml

Add yamllint ---

* Make it yamllint comply

* Make Yamllint comply

* Make Yamllint comply

* Change description

Co-authored-by: Itay Shakury <itay@itaysk.com>

* Fix syntax

Co-authored-by: Itay Shakury <itay@itaysk.com>

* Update docs/Architecture.md

Co-authored-by: Itay Shakury <itay@itaysk.com>

* Update docs/Architecture.md

Co-authored-by: Itay Shakury <itay@itaysk.com>

* Update example for test files

* Update contributing

* Delete Contributing.md

* Update Flags_and_commands.md

* Change syntax and add source

* Update Platforms.md

* lower case file names

* lower case file names

* Lower case file names

* Lower case file names

* Lower case file names

* Lower case file names

* Add note about inspect master in some platforms

* Add quick start

* Lower case files names

* Lower case files names

* Fixing typo

* Remove section about old ocp

* Fix typos

Co-authored-by: Itay Shakury <itay@itaysk.com>
2021-06-09 11:17:16 +03:00

381 lines
14 KiB
Markdown

# Test and config files
`kube-bench` runs checks specified in `controls` files that are a YAML
representation of the CIS Kubernetes Benchmark checks (or other distribution-specific hardening guides).
## Controls
`controls` is a YAML document that contains checks that must be run against a
specific Kubernetes node type, master or node and version.
`controls` is the fundamental input to `kube-bench`. The following is an example
of a basic `controls`:
```yml
---
controls:
id: 1
text: "Master Node Security Configuration"
type: "master"
groups:
- id: 1.1
text: API Server
checks:
- id: 1.1.1
text: "Ensure that the --allow-privileged argument is set (Scored)"
audit: "ps -ef | grep kube-apiserver | grep -v grep"
tests:
bin_op: or
test_items:
- flag: "--allow-privileged"
set: true
- flag: "--some-other-flag"
set: false
remediation: "Edit the /etc/kubernetes/config file on the master node and
set the KUBE_ALLOW_PRIV parameter to '--allow-privileged=false'"
scored: true
- id: 1.2
text: Scheduler
checks:
- id: 1.2.1
text: "Ensure that the --profiling argument is set to false (Scored)"
audit: "ps -ef | grep kube-scheduler | grep -v grep"
tests:
bin_op: and
test_items:
- flag: "--profiling"
set: true
- flag: "--some-other-flag"
set: false
remediation: "Edit the /etc/kubernetes/config file on the master node and
set the KUBE_ALLOW_PRIV parameter to '--allow-privileged=false'"
scored: true
```
`controls` is composed of a hierarchy of groups, sub-groups and checks. Each of
the `controls` components have an id and a text description which are displayed
in the `kube-bench` output.
`type` specifies what kubernetes node type a `controls` is for. Possible values
for `type` are `master` and `node`.
## Groups
`groups` is a list of subgroups that test the various Kubernetes components
that run on the node type specified in the `controls`.
For example, one subgroup checks parameters passed to the API server binary, while
another subgroup checks parameters passed to the controller-manager binary.
```yml
groups:
- id: 1.1
text: API Server
# ...
- id: 1.2
text: Scheduler
# ...
```
These subgroups have `id`, `text` fields which serve the same purposes described
in the previous paragraphs. The most important part of the subgroup is the
`checks` field which is the collection of actual `check`s that form the subgroup.
This is an example of a subgroup and checks in the subgroup.
```yml
id: 1.1
text: API Server
checks:
- id: 1.1.1
text: "Ensure that the --allow-privileged argument is set (Scored)"
audit: "ps -ef | grep kube-apiserver | grep -v grep"
tests:
# ...
- id: 1.1.2
text: "Ensure that the --anonymous-auth argument is set to false (Not Scored)"
audit: "ps -ef | grep kube-apiserver | grep -v grep"
tests:
# ...
```
`kube-bench` supports running a subgroup by specifying the subgroup `id` on the
command line, with the flag `--group` or `-g`.
## Check
The CIS Kubernetes Benchmark recommends configurations to harden Kubernetes components. These recommendations are usually configuration options and can be
specified by flags to Kubernetes binaries, or in configuration files.
The Benchmark also provides commands to audit a Kubernetes installation, identify
places where the cluster security can be improved, and steps to remediate these
identified problems.
In `kube-bench`, `check` objects embody these recommendations. This an example
`check` object:
```yml
id: 1.1.1
text: "Ensure that the --anonymous-auth argument is set to false (Not Scored)"
audit: "ps -ef | grep kube-apiserver | grep -v grep"
tests:
test_items:
- flag: "--anonymous-auth"
compare:
op: eq
value: false
set: true
remediation: |
Edit the API server pod specification file kube-apiserver
on the master node and set the below parameter.
--anonymous-auth=false
scored: false
```
A `check` object has an `id`, a `text`, an `audit`, a `tests`, `remediation`
and `scored` fields.
`kube-bench` supports running individual checks by specifying the check's `id`
as a comma-delimited list on the command line with the `--check` flag.
The `audit` field specifies the command to run for a check. The output of this
command is then evaluated for conformance with the CIS Kubernetes Benchmark
recommendation.
The audit is evaluated against criteria specified by the `tests`
object. `tests` contain `bin_op` and `test_items`.
`test_items` specify the criteria(s) the `audit` command's output should meet to
pass a check. This criteria is made up of keywords extracted from the output of
the `audit` command and operations that compare these keywords against
values expected by the CIS Kubernetes Benchmark.
There are three ways to run and extract keywords from the output of the command used,
| Command | Output var |
|---|---|
| `audit` | `flag` |
| `audit_config` | `path` |
| `audit_env` | `env` |
`flag` is used when the keyword is a command-line flag. The associated `audit` command could
be any binaries available on the system like `ps` command and a `grep` for the binary whose flag we are
checking:
```sh
ps -ef | grep somebinary | grep -v grep
```
Here is an example usage of the `flag` option:
```yml
# ...
audit: "ps -ef | grep kube-apiserver | grep -v grep"
tests:
test_items:
- flag: "--anonymous-auth"
# ...
```
`path` is used when the keyword is an option set in a JSON or YAML config file.
The associated `audit_command` command is usually `cat /path/to/config-yaml-or-json`.
For example:
```yml
# ...
text: "Ensure that the --anonymous-auth argument is set to false (Not Scored)"
audit: "cat /path/to/some/config"
tests:
test_items:
- path: "{.someoption.value}"
# ...
```
`env` is used to check if the value is present within a specified environment variable. The presence of `env` is treated as an OR operation, if both `flag` and `env` are supplied it will use either to attempt pass the check.
The command used for checking the environment variables of a process **is generated by default**.
If the command being generated is causing errors, you can override the command used by setting `audit_env` on the check.
Similarly, if you don't want the environment checking command to be generated or run at all, specify `disableEnvTesting` as true on the check.
The example below will check if the flag `--auto-tls` is equal to false *OR* `ETCD_AUTO_TLS` is equal to false
```yml
test_items:
- flag: "--auto-tls"
env: "ETCD_AUTO_TLS"
compare:
op: eq
value: false
```
**Note:** flag, path and env will act as OR if more then one present.
`test_item` compares the output of the audit command and keywords using the
`set` and `compare` fields.
```yml
test_items:
- flag: "--anonymous-auth"
compare:
op: eq
value: false
set: true
```
`set` checks if a keyword is present in the output of the audit command or a config file. The possible values for `set` are true and false.
If `set` is true, the check passes only if the keyword is present in the output
of the audit command, or config file. If `set` is false, the check passes only
if the keyword is not present in the output of the audit command, or config file.
`set` is true by default.
`compare` has two fields `op` and `value` to compare keywords with expected
value. `op` specifies which operation is used for the comparison, and `value`
specifies the value to compare against.
> To use `compare`, `set` must true. The comparison will be ignored if `set` is
> false
The `op` (operations) currently supported in `kube-bench` are:
- `eq`: tests if the keyword is equal to the compared value.
- `noteq`: tests if the keyword is unequal to the compared value.
- `gt`: tests if the keyword is greater than the compared value.
- `gte`: tests if the keyword is greater than or equal to the compared value.
- `lt`: tests if the keyword is less than the compared value.
- `lte`: tests if the keyword is less than or equal to the compared value.
- `has`: tests if the keyword contains the compared value.
- `nothave`: tests if the keyword does not contain the compared value.
- `regex`: tests if the flag value matches the compared value regular expression.
When defining regular expressions in YAML it is generally easier to wrap them in
single quotes, for example `'^[abc]$'`, to avoid issues with string escaping.
- `bitmask` : tests if keyward is bitmasked with the compared value, common usege is for
comparing file permissions in linux.
## Omitting checks
If you decide that a recommendation is not appropriate for your environment, you can choose to omit it by editing the test YAML file to give it the check type `skip` as in this example:
```yaml
checks:
- id: 2.1.1
text: "Ensure that the --allow-privileged argument is set to false (Scored)"
type: "skip"
scored: true
```
No tests will be run for this check and the output will be marked [INFO].
## Configuration and Variables
Kubernetes component configuration and binary file locations and names
vary based on cluster deployment methods and Kubernetes distribution used.
For this reason, the locations of these binaries and config files are configurable
by editing the `cfg/config.yaml` file and these binaries and files can be
referenced in a `controls` file via variables.
The `cfg/config.yaml` file is a global configuration file. Configuration files
can be created for specific Kubernetes versions (distributions). Values in the
version-specific config overwrite similar values in `cfg/config.yaml`.
For example, the kube-apiserver in Red Hat OCP distribution is run as
`hypershift openshift-kube-apiserver` instead of the default `kube-apiserver`.
This difference can be specified by editing the `master.apiserver.defaultbin`
entry `cfg/rh-0.7/config.yaml`.
Below is the structure of `cfg/config.yaml`:
```
nodetype
|-- components
|-- component1
|-- component1
|-- bins
|-- defaultbin (optional)
|-- confs
|-- defaultconf (optional)
|-- svcs
|-- defaultsvc (optional)
|-- kubeconfig
|-- defaultkubeconfig (optional)
```
Every node type has a subsection that specifies the main configuration items.
- `components`: A list of components for the node type. For example master
will have an entry for **apiserver**, **scheduler** and **controllermanager**.
Each component has the following entries:
- `bins`: A list of candidate binaries for a component. `kube-bench` checks this
list and selects the **first** binary that is running on the node.
If none of the binaries in `bins` list is running, `kube-bench` checks if the
binary specified by `defaultbin` is running and terminates if none of the
binaries in both `bins` and `defaultbin` is running.
The selected binary for a component can be referenced in `controls` using a
variable in the form `$<component>bin`. In the example below, we reference
the selected API server binary with the variable `$apiserverbin` in an `audit`
command.
```yml
id: 1.1.1
text: "Ensure that the --anonymous-auth argument is set to false (Scored)"
audit: "ps -ef | grep $apiserverbin | grep -v grep"
# ...
```
- `confs`: A list of candidate configuration files for a component. `kube-bench`
checks this list and selects the **first** config file that is found on the node.
If none of the config files exists, `kube-bench` defaults conf to the value
of `defaultconf`.
The selected config for a component can be referenced in `controls` using a
variable in the form `$<component>conf`. In the example below, we reference the
selected API server config file with the variable `$apiserverconf` in an `audit`
command.
```yml
id: 1.4.1
text: "Ensure that the API server pod specification file permissions are
set to 644 or more restrictive (Scored)"
audit: "/bin/sh -c 'if test -e $apiserverconf; then stat -c %a $apiserverconf; fi'"
```
- `svcs`: A list of candidate unitfiles for a component. `kube-bench` checks this
list and selects the **first** unitfile that is found on the node. If none of the
unitfiles exists, `kube-bench` defaults unitfile to the value of `defaultsvc`.
The selected unitfile for a component can be referenced in `controls` via a
variable in the form `$<component>svc`. In the example below, the selected
kubelet unitfile is referenced with `$kubeletsvc` in the `remediation` of the
`check`.
```yml
id: 2.1.1
# ...
remediation: |
Edit the kubelet service file $kubeletsvc
on each worker node and set the below parameter in KUBELET_SYSTEM_PODS_ARGS variable.
--allow-privileged=false
Based on your system, restart the kubelet service. For example:
systemctl daemon-reload
systemctl restart kubelet.service
# ...
```
- `kubeconfig`: A list of candidate kubeconfig files for a component. `kube-bench`
checks this list and selects the **first** file that is found on the node. If none
of the files exists, `kube-bench` defaults kubeconfig to the value of
`defaultkubeconfig`.
The selected kubeconfig for a component can be referenced in `controls` with a variable in the form `$<component>kubeconfig`. In the example below, the
selected kubelet kubeconfig is referenced with `$kubeletkubeconfig` in the
`audit` command.
```yml
id: 2.2.1
text: "Ensure that the kubelet.conf file permissions are set to 644 or
more restrictive (Scored)"
audit: "/bin/sh -c 'if test -e $kubeletkubeconfig; then stat -c %a $kubeletkubeconfig; fi'"
# ...
```