Improve Proxykubeconfig tests (#708)

* Changes for 1.5

* Update cis-1.3 through 1.6 to also work with configmaps.

* Switch on if proxykubeconfig is set, instead of setting a variable in the script.

* permissons -> proxykubeconfig for 2.2.5/4.1.3 to keep these tests locked with 2.2.6/4.1.4

* Updating test output? Maybe?

* Copy integration test output files into docker image?

* Make entrypoint move integration folder to host, print 1.5 node info.

* Change the order of tests in travis to load files before testing.

* Return tests to place

Those tests comes first since there is more likely to fail with them and then the test will fail "faster" which will save time

* Remove copy integration 

When running in a container we don't need to test, only when build and running in Travis to make sure everything is working fine.

* Add $ mark before proxykubeconfig

If not having $ before the parameter then it won't get substituted

* Add $ mark before proxykubeconfig

If not having $ before the parameter then it won't get substituted

* Remove test relate lines

We don't test while running, only integration testing when building and unit testing

* Add spaces

* Change 4.1.3 4.1.4

Those tests now should pass.

* Change tests 4.1.3 and 4.1.4

Those tests now should PASS

* Update job.data with more accurate counts. Thanks to @yoavrotems for getting the project this far!

* Thanks for linting, yamllint!

Co-authored-by: Yoav Rotem <yoavrotems97@gmail.com>

cfg/cis-1.5/node.yaml

@@ -41,12 +41,15 @@ groups:
         text: "Ensure that the proxy kubeconfig file permissions are set to 644 or more restrictive (Scored)"
         audit: '/bin/sh -c ''if test -e $proxykubeconfig; then stat -c permissions=%a $proxykubeconfig; fi'' '
         tests:
+          bin_op: or
           test_items:
             - flag: "permissions"
               set: true
               compare:
                 op: bitmask
                 value: "644"
+            - flag: "$proxykubeconfig"
+              set: false
         remediation: |
           Run the below command (based on the file location on your system) on the each worker node.
           For example,
@@ -57,9 +60,12 @@ groups:
         text: "Ensure that the proxy kubeconfig file ownership is set to root:root (Scored)"
         audit: '/bin/sh -c ''if test -e $proxykubeconfig; then stat -c %U:%G $proxykubeconfig; fi'' '
         tests:
+          bin_op: or
           test_items:
             - flag: root:root
               set: true
+            - flag: "$proxykubeconfig"
+              set: false
         remediation: |
           Run the below command (based on the file location on your system) on the each worker node.
           For example, chown root:root $proxykubeconfig

cfg/cis-1.6/node.yaml

@@ -39,11 +39,15 @@ groups:
         text: "If proxy kubeconfig file exists ensure permissions are set to 644 or more restrictive (Manual)"
         audit: '/bin/sh -c ''if test -e $proxykubeconfig; then stat -c permissions=%a $proxykubeconfig; fi'' '
         tests:
+          bin_op: or
           test_items:
             - flag: "permissions"
+              set: true
               compare:
                 op: bitmask
                 value: "644"
+            - flag: "$proxykubeconfig"
+              set: false
         remediation: |
           Run the below command (based on the file location on your system) on the each worker node.
           For example,
@@ -54,8 +58,11 @@ groups:
         text: "Ensure that the proxy kubeconfig file ownership is set to root:root (Manual)"
         audit: '/bin/sh -c ''if test -e $proxykubeconfig; then stat -c %U:%G $proxykubeconfig; fi'' '
         tests:
+          bin_op: or
           test_items:
             - flag: root:root
+            - flag: "$proxykubeconfig"
+              set: false
         remediation: |
           Run the below command (based on the file location on your system) on the each worker node.
           For example, chown root:root $proxykubeconfig

codecov.yml

@@ -1,3 +1,4 @@
+---
 coverage:
   status:
     project:

integration/testdata/cis-1.5/job-node.data

@@ -2,8 +2,8 @@
 [INFO] 4.1 Worker Node Configuration Files
 [PASS] 4.1.1 Ensure that the kubelet service file permissions are set to 644 or more restrictive (Scored)
 [PASS] 4.1.2 Ensure that the kubelet service file ownership is set to root:root (Scored)
-[FAIL] 4.1.3 Ensure that the proxy kubeconfig file permissions are set to 644 or more restrictive (Scored)
+[PASS] 4.1.3 Ensure that the proxy kubeconfig file permissions are set to 644 or more restrictive (Scored)
-[FAIL] 4.1.4 Ensure that the proxy kubeconfig file ownership is set to root:root (Scored)
+[PASS] 4.1.4 Ensure that the proxy kubeconfig file ownership is set to root:root (Scored)
 [PASS] 4.1.5 Ensure that the kubelet.conf file permissions are set to 644 or more restrictive (Scored)
 [PASS] 4.1.6 Ensure that the kubelet.conf file ownership is set to root:root (Scored)
 [PASS] 4.1.7 Ensure that the certificate authorities file permissions are set to 644 or more restrictive (Scored)
@@ -26,13 +26,6 @@
 [PASS] 4.2.13 Ensure that the Kubelet only makes use of Strong Cryptographic Ciphers (Not Scored)
 
 == Remediations ==
-4.1.3 Run the below command (based on the file location on your system) on the each worker node.
-For example,
-chmod 644 /etc/kubernetes/proxy.conf
-
-4.1.4 Run the below command (based on the file location on your system) on the each worker node.
-For example, chown root:root /etc/kubernetes/proxy.conf
-
 4.2.6 If using a Kubelet config file, edit the file to set protectKernelDefaults: true.
 If using command line arguments, edit the kubelet service file
 /etc/systemd/system/kubelet.service.d/10-kubeadm.conf on each worker node and
@@ -71,7 +64,7 @@ systemctl restart kubelet.service
 
 
 == Summary ==
-17 checks PASS
+19 checks PASS
-5 checks FAIL
+3 checks FAIL
 1 checks WARN
 0 checks INFO

integration/testdata/cis-1.5/job.data

@@ -215,8 +215,8 @@ minimum.
 [INFO] 4.1 Worker Node Configuration Files
 [PASS] 4.1.1 Ensure that the kubelet service file permissions are set to 644 or more restrictive (Scored)
 [PASS] 4.1.2 Ensure that the kubelet service file ownership is set to root:root (Scored)
-[FAIL] 4.1.3 Ensure that the proxy kubeconfig file permissions are set to 644 or more restrictive (Scored)
+[PASS] 4.1.3 Ensure that the proxy kubeconfig file permissions are set to 644 or more restrictive (Scored)
-[FAIL] 4.1.4 Ensure that the proxy kubeconfig file ownership is set to root:root (Scored)
+[PASS] 4.1.4 Ensure that the proxy kubeconfig file ownership is set to root:root (Scored)
 [PASS] 4.1.5 Ensure that the kubelet.conf file permissions are set to 644 or more restrictive (Scored)
 [PASS] 4.1.6 Ensure that the kubelet.conf file ownership is set to root:root (Scored)
 [PASS] 4.1.7 Ensure that the certificate authorities file permissions are set to 644 or more restrictive (Scored)
@@ -239,13 +239,6 @@ minimum.
 [PASS] 4.2.13 Ensure that the Kubelet only makes use of Strong Cryptographic Ciphers (Not Scored)
 
 == Remediations ==
-4.1.3 Run the below command (based on the file location on your system) on the each worker node.
-For example,
-chmod 644 /etc/kubernetes/proxy.conf
-
-4.1.4 Run the below command (based on the file location on your system) on the each worker node.
-For example, chown root:root /etc/kubernetes/proxy.conf
-
 4.2.6 If using a Kubelet config file, edit the file to set protectKernelDefaults: true.
 If using command line arguments, edit the kubelet service file
 /etc/systemd/system/kubelet.service.d/10-kubeadm.conf on each worker node and
@@ -284,8 +277,8 @@ systemctl restart kubelet.service
 
 
 == Summary ==
-17 checks PASS
+19 checks PASS
-5 checks FAIL
+3 checks FAIL
 1 checks WARN
 0 checks INFO
 [INFO] 5 Kubernetes Policies

integration/testdata/cis-1.6/job-node.data

@@ -2,8 +2,8 @@
 [INFO] 4.1 Worker Node Configuration Files
 [PASS] 4.1.1 Ensure that the kubelet service file permissions are set to 644 or more restrictive (Automated)
 [PASS] 4.1.2 Ensure that the kubelet service file ownership is set to root:root (Automated)
-[WARN] 4.1.3 If proxy kubeconfig file exists ensure permissions are set to 644 or more restrictive (Manual)
+[PASS] 4.1.3 If proxy kubeconfig file exists ensure permissions are set to 644 or more restrictive (Manual)
-[WARN] 4.1.4 Ensure that the proxy kubeconfig file ownership is set to root:root (Manual)
+[PASS] 4.1.4 Ensure that the proxy kubeconfig file ownership is set to root:root (Manual)
 [PASS] 4.1.5 Ensure that the --kubeconfig kubelet.conf file permissions are set to 644 or more restrictive (Automated)
 [PASS] 4.1.6 Ensure that the --kubeconfig kubelet.conf file ownership is set to root:root (Manual)
 [PASS] 4.1.7 Ensure that the certificate authorities file permissions are set to 644 or more restrictive (Manual)
@@ -26,13 +26,6 @@
 [PASS] 4.2.13 Ensure that the Kubelet only makes use of Strong Cryptographic Ciphers (Manual)
 
 == Remediations ==
-4.1.3 Run the below command (based on the file location on your system) on the each worker node.
-For example,
-chmod 644 /etc/kubernetes/proxy.conf
-
-4.1.4 Run the below command (based on the file location on your system) on the each worker node.
-For example, chown root:root /etc/kubernetes/proxy.conf
-
 4.2.6 If using a Kubelet config file, edit the file to set protectKernelDefaults: true.
 If using command line arguments, edit the kubelet service file
 /etc/systemd/system/kubelet.service.d/10-kubeadm.conf on each worker node and
@@ -71,7 +64,7 @@ systemctl restart kubelet.service
 
 
 == Summary ==
-17 checks PASS
+19 checks PASS
 1 checks FAIL
-5 checks WARN
+3 checks WARN
 0 checks INFO

integration/testdata/cis-1.6/job.data

@@ -222,8 +222,8 @@ minimum.
 [INFO] 4.1 Worker Node Configuration Files
 [PASS] 4.1.1 Ensure that the kubelet service file permissions are set to 644 or more restrictive (Automated)
 [PASS] 4.1.2 Ensure that the kubelet service file ownership is set to root:root (Automated)
-[WARN] 4.1.3 If proxy kubeconfig file exists ensure permissions are set to 644 or more restrictive (Manual)
+[PASS] 4.1.3 If proxy kubeconfig file exists ensure permissions are set to 644 or more restrictive (Manual)
-[WARN] 4.1.4 Ensure that the proxy kubeconfig file ownership is set to root:root (Manual)
+[PASS] 4.1.4 Ensure that the proxy kubeconfig file ownership is set to root:root (Manual)
 [PASS] 4.1.5 Ensure that the --kubeconfig kubelet.conf file permissions are set to 644 or more restrictive (Automated)
 [PASS] 4.1.6 Ensure that the --kubeconfig kubelet.conf file ownership is set to root:root (Manual)
 [PASS] 4.1.7 Ensure that the certificate authorities file permissions are set to 644 or more restrictive (Manual)
@@ -246,13 +246,6 @@ minimum.
 [PASS] 4.2.13 Ensure that the Kubelet only makes use of Strong Cryptographic Ciphers (Manual)
 
 == Remediations ==
-4.1.3 Run the below command (based on the file location on your system) on the each worker node.
-For example,
-chmod 644 /etc/kubernetes/proxy.conf
-
-4.1.4 Run the below command (based on the file location on your system) on the each worker node.
-For example, chown root:root /etc/kubernetes/proxy.conf
-
 4.2.6 If using a Kubelet config file, edit the file to set protectKernelDefaults: true.
 If using command line arguments, edit the kubelet service file
 /etc/systemd/system/kubelet.service.d/10-kubeadm.conf on each worker node and
@@ -291,9 +284,9 @@ systemctl restart kubelet.service
 
 
 == Summary ==
-17 checks PASS
+19 checks PASS
 1 checks FAIL
-5 checks WARN
+3 checks WARN
 0 checks INFO
 [INFO] 5 Kubernetes Policies
 [INFO] 5.1 RBAC and Service Accounts