Merge pull request #163 from noqcks/master

Update tests for Kubernetes 1.11 - thank you @noqcks!
pull/167/head v0.0.22
Liz Rice 6 years ago committed by GitHub
commit 277ec9c823
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23

@ -7,20 +7,20 @@
kube-bench is a Go application that checks whether Kubernetes is deployed securely by running the checks documented in the [CIS Kubernetes Benchmark](https://www.cisecurity.org/benchmark/kubernetes/).
Tests are configured with YAML files, making this tool easy to update as test specifications evolve.
Tests are configured with YAML files, making this tool easy to update as test specifications evolve.
![Kubernetes Bench for Security](https://raw.githubusercontent.com/aquasecurity/kube-bench/master/images/output.png "Kubernetes Bench for Security")
## CIS Kubernetes Benchmark support
kube-bench supports the tests for multiple versions of Kubernetes (1.6, 1.7 and 1.8) as defined in the CIS Benchmarks 1.0.0, 1.1.0 and 1.2.0 respectively. It will determine the test set to run based on the Kubernetes version running on the machine.
kube-bench supports the tests for multiple versions of Kubernetes (1.6, 1.7, 1.8, and 1.11) as defined in the CIS Benchmarks 1.0.0, 1.1.0, 1.2.0, and 1.30 respectively. It will determine the test set to run based on the Kubernetes version running on the machine.
## Installation
You can choose to
You can choose to
* run kube-bench from inside a container (sharing PID namespace with the host)
* run a container that installs kube-bench on the host, and then run kube-bench directly on the host
* install the latest binaries from the [Releases page](https://github.com/aquasecurity/kube-bench/releases),
* install the latest binaries from the [Releases page](https://github.com/aquasecurity/kube-bench/releases),
* compile it from source.
### Running inside a container
@ -43,13 +43,13 @@ docker run --pid=host -t -v path/to/my-config.yaml:/opt/kube-bench/cfg/config.ya
Run the master check
```
kubectl run --rm -i -t kube-bench-master --image=aquasec/kube-bench:latest --restart=Never --overrides="{ \"apiVersion\": \"v1\", \"spec\": { \"hostPID\": true, \"nodeSelector\": { \"kubernetes.io/role\": \"master\" }, \"tolerations\": [ { \"key\": \"node-role.kubernetes.io/master\", \"operator\": \"Exists\", \"effect\": \"NoSchedule\" } ] } }" -- master --version 1.8
kubectl run --rm -i -t kube-bench-master --image=aquasec/kube-bench:latest --restart=Never --overrides="{ \"apiVersion\": \"v1\", \"spec\": { \"hostPID\": true, \"nodeSelector\": { \"kubernetes.io/role\": \"master\" }, \"tolerations\": [ { \"key\": \"node-role.kubernetes.io/master\", \"operator\": \"Exists\", \"effect\": \"NoSchedule\" } ] } }" -- master --version 1.11
```
Run the node check
```
kubectl run --rm -i -t kube-bench-node --image=aquasec/kube-bench:latest --restart=Never --overrides="{ \"apiVersion\": \"v1\", \"spec\": { \"hostPID\": true } }" -- node --version 1.8
kubectl run --rm -i -t kube-bench-node --image=aquasec/kube-bench:latest --restart=Never --overrides="{ \"apiVersion\": \"v1\", \"spec\": { \"hostPID\": true } }" -- node --version 1.11
```
### Installing from a container
@ -59,7 +59,7 @@ This command copies the kube-bench binary and configuration files to your host f
docker run --rm -v `pwd`:/host aquasec/kube-bench:latest install
```
You can then run `./kube-bench <master|node>`.
You can then run `./kube-bench <master|node>`.
### Installing from sources
@ -81,13 +81,13 @@ go build -o kube-bench .
```
## Configuration
Kubernetes config and binary file locations and names can vary from installation to installation, so these are configurable in the `cfg/config.yaml` file.
Kubernetes config and binary file locations and names can vary from installation to installation, so these are configurable in the `cfg/config.yaml` file.
For each type of node (*master*, *node* or *federated*) there is a list of components, and for each component there is a set of binaries (*bins*) and config files (*confs*) that kube-bench will look for (in the order they are listed). If your installation uses a different binary name or config file location for a Kubernetes component, you can add it to `cfg/config.yaml`.
For each type of node (*master*, *node* or *federated*) there is a list of components, and for each component there is a set of binaries (*bins*) and config files (*confs*) that kube-bench will look for (in the order they are listed). If your installation uses a different binary name or config file location for a Kubernetes component, you can add it to `cfg/config.yaml`.
* **bins** - If there is a *bins* list for a component, at least one of these binaries must be running. The tests will consider the parameters for the first binary in the list found to be running.
* **podspecs** - From version 1.2.0 of the benchmark (tests for Kubernetes 1.8), the remediation instructions were updated to assume that the configuration for several kubernetes components is defined in a pod YAML file, and podspec settings define where to look for that configuration.
* **confs** - If one of the listed config files is found, this will be considered for the test. Tests can continue even if no config file is found. If no file is found at any of the listed locations, and a *defaultconf* location is given for the component, the test will give remediation advice using the *defaultconf* location.
* **bins** - If there is a *bins* list for a component, at least one of these binaries must be running. The tests will consider the parameters for the first binary in the list found to be running.
* **podspecs** - From version 1.2.0 of the benchmark (tests for Kubernetes 1.8), the remediation instructions were updated to assume that the configuration for several kubernetes components is defined in a pod YAML file, and podspec settings define where to look for that configuration.
* **confs** - If one of the listed config files is found, this will be considered for the test. Tests can continue even if no config file is found. If no file is found at any of the listed locations, and a *defaultconf* location is given for the component, the test will give remediation advice using the *defaultconf* location.
* **unitfiles** - From version 1.2.0 of the benchmark (tests for Kubernetes 1.8), the remediation instructions were updated to assume that kubelet configuration is defined in a service file, and this setting defines where to look for that configuration.
## Test config YAML representation
@ -147,7 +147,7 @@ These operations are:
- `has`: tests if the flag value contains the compared value.
- `nothave`: tests if the flag value does not contain the compared value.
# Roadmap
Going forward we plan to release updates to kube-bench to add support for new releases of the Benchmark, which in turn we can anticipate being made for each new Kubernetes release.
# Roadmap
Going forward we plan to release updates to kube-bench to add support for new releases of the Benchmark, which in turn we can anticipate being made for each new Kubernetes release.
We welcome PRs and issue reports.
We welcome PRs and issue reports.

@ -0,0 +1,47 @@
---
## Controls Files.
# These are YAML files that hold all the details for running checks.
#
## Uncomment to use different control file paths.
# masterControls: ./cfg/master.yaml
# nodeControls: ./cfg/node.yaml
# federatedControls: ./cfg/federated.yaml
master:
apiserver:
confs:
- /etc/kubernetes/manifests/kube-apiserver.yaml
- /etc/kubernetes/manifests/kube-apiserver.manifest
defaultconf: /etc/kubernetes/manifests/kube-apiserver.yaml
scheduler:
confs:
- /etc/kubernetes/manifests/kube-scheduler.yaml
- /etc/kubernetes/manifests/kube-scheduler.manifest
defaultconf: /etc/kubernetes/manifests/kube-scheduler.yaml
controllermanager:
confs:
- /etc/kubernetes/manifests/kube-controller-manager.yaml
- /etc/kubernetes/manifests/kube-controller-manager.manifest
defaultconf: /etc/kubernetes/manifests/kube-controller-manager.yaml
etcd:
confs:
- /etc/kubernetes/manifests/etcd.yaml
- /etc/kubernetes/manifests/etcd.manifest
defaultconf: /etc/kubernetes/manifests/etcd.yaml
node:
kubelet:
confs:
- /etc/systemd/system/kubelet.service.d/10-kubeadm.conf
- /etc/kubernetes/kubelet.conf
defaultconf: /etc/systemd/system/kubelet.service.d/10-kubeadm.conf
proxy:
confs:
- /etc/kubernetes/addons/kube-proxy-daemonset.yaml
defaultconf: /etc/kubernetes/addons/kube-proxy-daemonset.yaml

File diff suppressed because it is too large Load Diff

@ -0,0 +1,497 @@
---
controls:
version: 1.11
id: 2
text: "Worker Node Security Configuration"
type: "node"
groups:
- id: 2.1
text: "Kubelet"
checks:
- id: 2.1.1
text: "Ensure that the --allow-privileged argument is set to false (Scored)"
audit: "ps -ef | grep $kubeletbin | grep -v grep"
tests:
test_items:
- flag: "--allow-privileged"
compare:
op: eq
value: false
set: true
remediation: |
Edit the kubelet service file $kubeletconf
on each worker node and set the below parameter in KUBELET_SYSTEM_PODS_ARGS variable.
--allow-privileged=false
Based on your system, restart the kubelet service. For example:
systemctl daemon-reload
systemctl restart kubelet.service
scored: true
- id: 2.1.2
text: "Ensure that the --anonymous-auth argument is set to false (Scored)"
audit: "ps -ef | grep $kubeletbin | grep -v grep"
tests:
test_items:
- flag: "--anonymous-auth"
compare:
op: eq
value: false
set: true
remediation: |
If using a Kubelet config file, edit the file to set authentication: anonymous: enabled to
false .
If using executable arguments, edit the kubelet service file
$kubeletconf on each worker node and
set the below parameter in KUBELET_SYSTEM_PODS_ARGS variable.
--anonymous-auth=false
Based on your system, restart the kubelet service. For example:
systemctl daemon-reload
systemctl restart kubelet.service
scored: true
- id: 2.1.3
text: "Ensure that the --authorization-mode argument is not set to AlwaysAllow (Scored)"
audit: "ps -ef | grep $kubeletbin | grep -v grep"
tests:
test_items:
- flag: "--authorization-mode"
compare:
op: nothave
value: "AlwaysAllow"
set: true
remediation: |
If using a Kubelet config file, edit the file to set authorization: mode to Webhook.
If using executable arguments, edit the kubelet service file
$kubeletconf on each worker node and
set the below parameter in KUBELET_AUTHZ_ARGS variable.
--authorization-mode=Webhook
Based on your system, restart the kubelet service. For example:
systemctl daemon-reload
systemctl restart kubelet.service
scored: true
- id: 2.1.4
text: "Ensure that the --client-ca-file argument is set as appropriate (Scored)"
audit: "ps -ef | grep $kubeletbin | grep -v grep"
tests:
test_items:
- flag: "--client-ca-file"
set: true
remediation: |
If using a Kubelet config file, edit the file to set authentication: x509: clientCAFile to
the location of the client CA file.
If using command line arguments, edit the kubelet service file
$kubeletconf on each worker node and
set the below parameter in KUBELET_AUTHZ_ARGS variable.
--client-ca-file=<path/to/client-ca-file>
Based on your system, restart the kubelet service. For example:
systemctl daemon-reload
systemctl restart kubelet.service
scored: true
- id: 2.1.5
text: "Ensure that the --read-only-port argument is set to 0 (Scored)"
audit: "ps -ef | grep $kubeletbin | grep -v grep"
tests:
test_items:
- flag: "--read-only-port"
compare:
op: eq
value: 0
set: true
remediation: |
If using a Kubelet config file, edit the file to set readOnlyPort to 0 .
If using command line arguments, edit the kubelet service file
$kubeletconf on each worker node and
set the below parameter in KUBELET_SYSTEM_PODS_ARGS variable.
--read-only-port=0
Based on your system, restart the kubelet service. For example:
systemctl daemon-reload
systemctl restart kubelet.service
scored: true
- id: 2.1.6
text: "Ensure that the --streaming-connection-idle-timeout argument is not set to 0 (Scored)"
audit: "ps -ef | grep $kubeletbin | grep -v grep"
tests:
test_items:
- flag: "--streaming-connection-idle-timeout"
compare:
op: noteq
value: 0
set: true
remediation: |
If using a Kubelet config file, edit the file to set streamingConnectionIdleTimeout to a
value other than 0.
If using command line arguments, edit the kubelet service file
$kubeletconf on each worker node and
set the below parameter in KUBELET_SYSTEM_PODS_ARGS variable.
--streaming-connection-idle-timeout=5m
Based on your system, restart the kubelet service. For example:
systemctl daemon-reload
systemctl restart kubelet.service
scored: true
- id: 2.1.7
text: "Ensure that the --protect-kernel-defaults argument is set to true (Scored)"
audit: "ps -ef | grep $kubeletbin | grep -v grep"
tests:
test_items:
- flag: "--protect-kernel-defaults"
compare:
op: eq
value: true
set: true
remediation: |
If using a Kubelet config file, edit the file to set protectKernelDefaults: true .
If using command line arguments, edit the kubelet service file
$kubeletconf on each worker node and
set the below parameter in KUBELET_SYSTEM_PODS_ARGS variable.
--protect-kernel-defaults=true
Based on your system, restart the kubelet service. For example:
systemctl daemon-reload
systemctl restart kubelet.service
scored: true
- id: 2.1.8
text: "Ensure that the --make-iptables-util-chains argument is set to true (Scored)"
audit: "ps -ef | grep $kubeletbin | grep -v grep"
tests:
bin_op: or
test_items:
- flag: "--make-iptables-util-chains"
compare:
op: eq
value: true
set: true
remediation: |
If using a Kubelet config file, edit the file to set makeIPTablesUtilChains: true .
If using command line arguments, edit the kubelet service file
$kubeletconf on each worker node and
remove the --make-iptables-util-chains argument from the
KUBELET_SYSTEM_PODS_ARGS variable.
Based on your system, restart the kubelet service. For example:
systemctl daemon-reload
systemctl restart kubelet.service
scored: true
- id: 2.1.9
text: "Ensure that the --hostname-override argument is not set (Scored)"
audit: "ps -ef | grep $kubeletbin | grep -v grep"
tests:
test_items:
- flag: "--hostname-override"
set: false
remediation: |
Edit the kubelet service file $kubeletconf
on each worker node and remove the --hostname-override argument from the
KUBELET_SYSTEM_PODS_ARGS variable.
Based on your system, restart the kubelet service. For example:
systemctl daemon-reload
systemctl restart kubelet.service
scored: true
- id: 2.1.10
text: "Ensure that the --event-qps argument is set to 0 (Scored)"
audit: "ps -ef | grep $kubeletbin | grep -v grep"
tests:
test_items:
- flag: "--event-qps"
compare:
op: eq
value: 0
set: true
remediation: |
If using a Kubelet config file, edit the file to set eventRecordQPS: 0 .
If using command line arguments, edit the kubelet service file
$kubeletconf on each worker node and
set the below parameter in KUBELET_SYSTEM_PODS_ARGS variable.
--event-qps=0
Based on your system, restart the kubelet service. For example:
systemctl daemon-reload
systemctl restart kubelet.service
scored: true
- id: 2.1.11
text: "Ensure that the --tls-cert-file and --tls-private-key-file arguments are set as appropriate (Scored)"
audit: "ps -ef | grep $kubeletbin | grep -v grep"
tests:
bin_op: and
test_items:
- flag: "--tls-cert-file"
set: true
- flag: "--tls-private-key-file"
set: true
remediation: |
If using a Kubelet config file, edit the file to set tlsCertFile to the location of the certificate
file to use to identify this Kubelet, and tlsPrivateKeyFile to the location of the
corresponding private key file.
If using command line arguments, edit the kubelet service file
$kubeletconf on each worker node and
set the below parameters in KUBELET_CERTIFICATE_ARGS variable.
--tls-cert-file=<path/to/tls-certificate-file>
file=<path/to/tls-key-file>
Based on your system, restart the kubelet service. For example:
systemctl daemon-reload
systemctl restart kubelet.service
scored: true
- id: 2.1.12
text: "Ensure that the --cadvisor-port argument is set to 0 (Scored)"
audit: "ps -ef | grep $kubeletbin | grep -v grep"
tests:
bin_op: or
test_items:
- flag: "--cadvisor-port"
compare:
op: eq
value: 0
set: true
- flag: "--cadvisor-port"
set: false
remediation: |
Edit the kubelet service file $kubeletconf
on each worker node and set the below parameter in KUBELET_CADVISOR_ARGS variable.
--cadvisor-port=0
Based on your system, restart the kubelet service. For example:
systemctl daemon-reload
systemctl restart kubelet.service
scored: true
- id: 2.1.13
text: "Ensure that the --rotate-certificates argument is not set to false (Scored)"
audit: "ps -ef | grep $kubeletbin | grep -v grep"
tests:
test_items:
- flag: "--rotate-certificates"
compare:
op: eq
value: true
set: true
remediation: |
If using a Kubelet config file, edit the file to add the line rotateCertificates: true.
If using command line arguments, edit the kubelet service file $kubeletconf
on each worker node and add --rotate-certificates=true argument to the KUBELET_CERTIFICATE_ARGS variable.
Based on your system, restart the kubelet service. For example:
systemctl daemon-reload
systemctl restart kubelet.service
scored: true
- id: 2.1.14
text: "Ensure that the RotateKubeletServerCertificate argument is set to true (Scored)"
audit: "ps -ef | grep $kubeletbin | grep -v grep"
tests:
test_items:
- flag: "RotateKubeletServerCertificate"
compare:
op: eq
value: true
set: true
remediation: |
Edit the kubelet service file $kubeletconf
on each worker node and set the below parameter in KUBELET_CERTIFICATE_ARGS variable.
--feature-gates=RotateKubeletServerCertificate=true
Based on your system, restart the kubelet service. For example:
systemctl daemon-reload
systemctl restart kubelet.service
scored: true
- id: 2.1.15
text: "Ensure that the Kubelet only makes use of Strong Cryptographic Ciphers (Not Scored)"
audit: "ps -ef | grep $kubeletbin | grep -v grep"
tests:
test_items:
- flag: "--tls-cipher-suites"
compare:
op: eq
value: "TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,TLS_RSA_WITH_AES_256_GCM_SHA384,TLS_RSA_WITH_AES_128_GCM_SHA256"
set: true
remediation: |
If using a Kubelet config file, edit the file to set TLSCipherSuites: to TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 ,TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 ,TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384 ,TLS_RSA_WITH_AES_256_GCM_SHA384,TLS_RSA_WITH_AES_128_GCM_SHA256
If using executable arguments, edit the kubelet service file $kubeletconf on each worker node and set the below parameter.
--tls-cipher- suites=TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_128_GCM _SHA256,TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_RSA_WITH_AES_256_GCM _SHA384,TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_ECDSA_WITH_AES_256_GCM _SHA384,TLS_RSA_WITH_AES_256_GCM_SHA384,TLS_RSA_WITH_AES_128_GCM_SHA256
scored: false
- id: 2.2
text: "Configuration Files"
checks:
- id: 2.2.1
text: "Ensure that the kubelet.conf file permissions are set to 644 or
more restrictive (Scored)"
audit: "/bin/sh -c 'if test -e $kubeletconf; then stat -c %a $kubeletconf; fi'"
tests:
bin_op: or
test_items:
- flag: "644"
compare:
op: eq
value: "644"
set: true
- flag: "640"
compare:
op: eq
value: "640"
set: true
- flag: "600"
compare:
op: eq
value: "600"
set: true
remediation: |
Run the below command (based on the file location on your system) on the each worker
node. For example,
chmod 644 $kubeletconf
scored: true
- id: 2.2.2
text: "Ensure that the kubelet.conf file ownership is set to root:root (Scored)"
audit: "/bin/sh -c 'if test -e $kubeletconf; then stat -c %U:%G $kubeletconf; fi'"
tests:
test_items:
- flag: "root:root"
compare:
op: eq
value: root:root
set: true
remediation: |
Run the below command (based on the file location on your system) on the each worker
node. For example,
chown root:root $kubeletconf
scored: true
- id: 2.2.3
text: "Ensure that the kubelet service file permissions are set to 644 or
more restrictive (Scored)"
audit: "/bin/sh -c 'if test -e $kubeletconf; then stat -c %a $kubeletconf; fi'"
tests:
bin_op: or
test_items:
- flag: "644"
compare:
op: eq
value: 644
set: true
- flag: "640"
compare:
op: eq
value: "640"
set: true
- flag: "600"
compare:
op: eq
value: "600"
set: true
remediation: |
Run the below command (based on the file location on your system) on the each worker
node. For example,
chmod 755 $kubeletconf
scored: true
- id: 2.2.4
text: "Ensure that the kubelet service file ownership is set to root:root (Scored)"
audit: "/bin/sh -c 'if test -e $kubeletconf; then stat -c %U:%G $kubeletconf; fi'"
tests:
test_items:
- flag: "root:root"
set: true
remediation: |
Run the below command (based on the file location on your system) on the each worker
node. For example,
chown root:root $kubeletconf
scored: true
- id: 2.2.5
text: "Ensure that the proxy kubeconfig file permissions are set to 644 or more restrictive (Scored)"
audit: "/bin/sh -c 'if test -e $proxyconf; then stat -c %a $proxyconf; fi'"
tests:
bin_op: or
test_items:
- flag: "644"
compare:
op: eq
value: "644"
set: true
- flag: "640"
compare:
op: eq
value: "640"
set: true
- flag: "600"
compare:
op: eq
value: "600"
set: true
remediation: |
Run the below command (based on the file location on your system) on the each worker
node. For example,
chmod 644 $proxyconf
scored: true
- id: 2.2.6
text: "Ensure that the proxy kubeconfig file ownership is set to root:root (Scored)"
audit: "/bin/sh -c 'if test -e $proxyconf; then stat -c %U:%G $proxyconf; fi'"
tests:
test_items:
- flag: "root:root"
set: true
remediation: |
Run the below command (based on the file location on your system) on the each worker
node. For example,
chown root:root $proxyconf
scored: true
- id: 2.2.7
text: "Ensure that the certificate authorities file permissions are set to
644 or more restrictive (Scored)"
type: manual
remediation: |
Run the following command to modify the file permissions of the --client-ca-file
chmod 644 <filename>
scored: true
- id: 2.2.8
text: "Ensure that the client certificate authorities file ownership is set to root:root (Scored)"
audit: "/bin/sh -c 'if test -e $ca-file; then stat -c %U:%G $ca-file; fi'"
type: manual
remediation: |
Run the following command to modify the ownership of the --client-ca-file .
chown root:root <filename>
scored: true
- id: 2.2.9
text: "Ensure that the kubelet configuration file ownership is set to root:root (Scored)"
audit: "/bin/sh -c 'if test -e $/var/lib/kubelet/config.yaml; then stat -c %U:%G $/var/lib/kubelet/config.yaml; fi'"
tests:
test_items:
- flag: "root:root"
set: true
remediation: |
Run the following command (using the config file location identied in the Audit step)
chown root:root /etc/kubernetes/kubelet.conf
scored: true
- id: 2.2.10
text: "Ensure that the kubelet configuration file has permissions set to 644 or more restrictive (Scored)"
audit: "/bin/sh -c 'if test -e $/var/lib/kubelet/config.yaml; then stat -c %a $/var/lib/kubelet/config.yaml; fi'"
tests:
bin_op: or
test_items:
- flag: "644"
compare:
op: eq
value: "644"
set: true
- flag: "640"
compare:
op: eq
value: "640"
set: true
- flag: "600"
compare:
op: eq
value: "600"
set: true
remediation: |
Run the following command (using the config file location identied in the Audit step)
chmod 644 /var/lib/kubelet/config.yaml
scored: true
Loading…
Cancel
Save