Refactor group skip (#783)

* Add example IAM policy

* Pass RotateKubeletServerCertificate related checks if it's not found (#767)

* Allow for environment variables to be checked in tests (#755)

* Initial commit for checking environment variables for etcd

* Revert config changes

* Remove redundant struct data

* Fix issues with failing tests

* Initial changes based on code review

* Add option to disable envTesting + Update docs

* Initial tests

* Finished testing

* Fix broken tests

* Add a total summary and always show all tests. (#759)

Whether the total summary is shown can be specified with an option.

Fixes #528

Signed-off-by: Christian Zunker <christian.zunker@codecentric.cloud>

* Update Readme.md file with link to Contribution guide (#754)

* Update License with the year and the owner name

Please add this to make your license agreement strong

* Updated Readme.md file with license and proper documentation links

I have added a proper license agreement to the documentation. Also shortened the links to the issues so that it does not break in any on the forks.

* Update LICENSE

* Update README.md

* Update README.md

* Remove erroneous license info

Co-authored-by: Liz Rice <liz@lizrice.com>

* Support auto-detect platform when running on EKS or GKE (#683)

* Support auto-detect platform when running on EKS or GKE

* Change to get platform name from `kubectl version`

* fix regexp and add test

* Update Server Version match for EKS

* try to get version info from api sever at first

* Refactor group skip

changed group 'skip' from being a bool to be 'type' string as done in check

* Change skip: true -> type: skip

Co-authored-by: Huang Huang <mozillazg101@gmail.com>
Co-authored-by: Wicked <jason_attwood@hotmail.co.uk>
Co-authored-by: Christian Zunker <827818+czunker@users.noreply.github.com>
Co-authored-by: Kaiwalya Koparkar <kaiwalyakoparkar@gmail.com>
Co-authored-by: Yoav Rotem <yoavrotems97@gmail.com>

README.md

@@ -411,6 +411,7 @@ Finally, we can use the `make kind-run` target to run the current version of kub
 Every time you want to test a change, you'll need to rebuild the docker image and push it to cluster before running it again. ( `make build-docker kind-push kind-run` )
 
 ## Contributing
+Kindly read [Contributing.md](CONTRIBUTING.md) before contributing. Some instructions for the common contributions are stated below.
 
 ### Bugs
 

cfg/cis-1.5/master.yaml

@@ -997,12 +997,15 @@ groups:
         text: "Ensure that the RotateKubeletServerCertificate argument is set to true (Scored)"
         audit: "/bin/ps -ef | grep $controllermanagerbin | grep -v grep"
         tests:
+          bin_op: or
           test_items:
             - flag: "--feature-gates"
               compare:
-                op: eq
+                op: nothave
-                value: "RotateKubeletServerCertificate=true"
+                value: "RotateKubeletServerCertificate=false"
               set: true
+            - flag: "--feature-gates"
+              set: false
         remediation: |
           Edit the Controller Manager pod specification file $controllermanagerconf
           on the master node and set the --feature-gates parameter to include RotateKubeletServerCertificate=true.

cfg/cis-1.5/node.yaml

@@ -442,13 +442,17 @@ groups:
         audit: "/bin/ps -fC $kubeletbin"
         audit_config: "/bin/cat $kubeletconf"
         tests:
+          bin_op: or
           test_items:
             - flag: RotateKubeletServerCertificate
               path: '{.featureGates.RotateKubeletServerCertificate}'
               set: true
               compare:
-                op: eq
+                op: nothave
-                value: true
+                value: false
+            - flag: RotateKubeletServerCertificate
+              path: '{.featureGates.RotateKubeletServerCertificate}'
+              set: false
         remediation: |
           Edit the kubelet service file $kubeletsvc
           on each worker node and set the below parameter in KUBELET_CERTIFICATE_ARGS variable.

cfg/cis-1.6/etcd.yaml

@@ -15,7 +15,9 @@ groups:
           bin_op: and
           test_items:
             - flag: "--cert-file"
+              env: "ETCD_CERT_FILE"
             - flag: "--key-file"
+              env: "ETCD_KEY_FILE"
         remediation: |
           Follow the etcd service documentation and configure TLS encryption.
           Then, edit the etcd pod specification file /etc/kubernetes/manifests/etcd.yaml
@@ -30,6 +32,7 @@ groups:
         tests:
           test_items:
             - flag: "--client-cert-auth"
+              env: "ETCD_CLIENT_CERT_AUTH"
               compare:
                 op: eq
                 value: true
@@ -46,8 +49,10 @@ groups:
           bin_op: or
           test_items:
             - flag: "--auto-tls"
+              env: "ETCD_AUTO_TLS"
               set: false
             - flag: "--auto-tls"
+              env: "ETCD_AUTO_TLS"
               compare:
                 op: eq
                 value: false
@@ -65,7 +70,9 @@ groups:
           bin_op: and
           test_items:
             - flag: "--peer-cert-file"
+              env: "ETCD_PEER_CERT_FILE"
             - flag: "--peer-key-file"
+              env: "ETCD_PEER_KEY_FILE"
         remediation: |
           Follow the etcd service documentation and configure peer TLS encryption as appropriate
           for your etcd cluster.
@@ -81,6 +88,7 @@ groups:
         tests:
           test_items:
             - flag: "--peer-client-cert-auth"
+              env: "ETCD_PEER_CLIENT_CERT_AUTH"
               compare:
                 op: eq
                 value: true
@@ -97,8 +105,10 @@ groups:
           bin_op: or
           test_items:
             - flag: "--peer-auto-tls"
+              env: "ETCD_PEER_AUTO_TLS"
               set: false
             - flag: "--peer-auto-tls"
+              env: "ETCD_PEER_AUTO_TLS"
               compare:
                 op: eq
                 value: false
@@ -114,6 +124,7 @@ groups:
         tests:
           test_items:
             - flag: "--trusted-ca-file"
+              env: "ETCD_TRUSTED_CA_FILE"
         remediation: |
           [Manual test]
           Follow the etcd documentation and create a dedicated certificate authority setup for the

cfg/cis-1.6/master.yaml

@@ -918,11 +918,15 @@ groups:
         text: "Ensure that the RotateKubeletServerCertificate argument is set to true (Automated)"
         audit: "/bin/ps -ef | grep $controllermanagerbin | grep -v grep"
         tests:
+          bin_op: or
           test_items:
             - flag: "--feature-gates"
               compare:
-                op: eq
+                op: nothave
-                value: "RotateKubeletServerCertificate=true"
+                value: "RotateKubeletServerCertificate=false"
+              set: true
+            - flag: "--feature-gates"
+              set: false
         remediation: |
           Edit the Controller Manager pod specification file $controllermanagerconf
           on the master node and set the --feature-gates parameter to include RotateKubeletServerCertificate=true.

cfg/cis-1.6/node.yaml

@@ -419,12 +419,16 @@ groups:
         audit: "/bin/ps -fC $kubeletbin"
         audit_config: "/bin/cat $kubeletconf"
         tests:
+          bin_op: or
           test_items:
             - flag: RotateKubeletServerCertificate
               path: '{.featureGates.RotateKubeletServerCertificate}'
               compare:
-                op: eq
+                op: nothave
-                value: true
+                value: false
+            - flag: RotateKubeletServerCertificate
+              path: '{.featureGates.RotateKubeletServerCertificate}'
+              set: false
         remediation: |
           Edit the kubelet service file $kubeletsvc
           on each worker node and set the below parameter in KUBELET_CERTIFICATE_ARGS variable.

check/check.go

@@ -68,6 +68,7 @@ type Check struct {
 	ID                string   `yaml:"id" json:"test_number"`
 	Text              string   `json:"test_desc"`
 	Audit             string   `json:"audit"`
+	AuditEnv          string   `yaml:"audit_env"`
 	AuditConfig       string   `yaml:"audit_config"`
 	Type              string   `json:"type"`
 	Tests             *tests   `json:"-"`
@@ -81,7 +82,9 @@ type Check struct {
 	ExpectedResult    string `json:"expected_result"`
 	Reason            string `json:"reason,omitempty"`
 	AuditOutput       string `json:"-"`
+	AuditEnvOutput    string `json:"-"`
 	AuditConfigOutput string `json:"-"`
+	DisableEnvTesting bool `json:"-"`
 }
 
 // Runner wraps the basic Run method.
@@ -184,6 +187,14 @@ func (c *Check) run() State {
 }
 
 func (c *Check) runAuditCommands() (lastCommand string, err error) {
+	// Always run auditEnvOutput if needed
+	if c.AuditEnv != "" {
+		c.AuditEnvOutput, err = runAudit(c.AuditEnv)
+		if err != nil {
+			return c.AuditEnv, err
+		}
+	}
+
 	// Run the audit command and auditConfig commands, if present
 	c.AuditOutput, err = runAudit(c.Audit)
 	if err != nil {
@@ -207,11 +218,15 @@ func (c *Check) execute() (finalOutput *testOutput, err error) {
 		t.isMultipleOutput = c.IsMultiple
 
 		// Try with the auditOutput first, and if that's not found, try the auditConfigOutput
-		t.isConfigSetting = false
+		t.auditUsed = AuditCommand
 		result := *(t.execute(c.AuditOutput))
-		if !result.flagFound {
+		if !result.found {
-			t.isConfigSetting = true
+			t.auditUsed = AuditConfig
 			result = *(t.execute(c.AuditConfigOutput))
+			if !result.found && t.Env != "" {
+				t.auditUsed = AuditEnv
+				result = *(t.execute(c.AuditEnvOutput))
+			}
 		}
 		res[i] = result
 		expectedResultArr[i] = res[i].ExpectedResult

check/check_test.go

@@ -81,6 +81,39 @@ func TestCheck_Run(t *testing.T) {
 	}
 }
 
+func TestCheckAuditEnv(t *testing.T){
+	passingCases := []*Check{
+		controls.Groups[2].Checks[0],
+		controls.Groups[2].Checks[2],
+		controls.Groups[2].Checks[3],
+		controls.Groups[2].Checks[4],
+	}
+
+	failingCases := []*Check{
+		controls.Groups[2].Checks[1],
+		controls.Groups[2].Checks[5],
+		controls.Groups[2].Checks[6],
+	}
+
+	for _, c := range passingCases {
+		t.Run(c.Text, func(t *testing.T) {
+			c.run()
+			if c.State != "PASS" {
+				t.Errorf("Should PASS, got: %v", c.State)
+			}
+		})
+	}
+
+	for _, c := range failingCases {
+		t.Run(c.Text, func(t *testing.T) {
+			c.run()
+			if c.State != "FAIL" {
+				t.Errorf("Should FAIL, got: %v", c.State)
+			}
+		})
+	}
+}
+
 func TestCheckAuditConfig(t *testing.T) {
 
 	passingCases := []*Check{

check/controls.go

@@ -40,6 +40,11 @@ const (
 	TYPE = "Software and Configuration Checks/Industry and Regulatory Standards/CIS Kubernetes Benchmark"
 )
 
+type OverallControls struct {
+	Controls []*Controls
+	Totals   Summary
+}
+
 // Controls holds all controls to check for master nodes.
 type Controls struct {
 	ID      string   `yaml:"id" json:"id"`
@@ -53,7 +58,7 @@ type Controls struct {
 // Group is a collection of similar checks.
 type Group struct {
 	ID     string   `yaml:"id" json:"section"`
-	Skip   bool     `yaml:"skip" json:"skip"`
+	Type   string   `yaml:"type" json:"type"`
 	Pass   int      `json:"pass"`
 	Fail   int      `json:"fail"`
 	Warn   int      `json:"warn"`
@@ -105,7 +110,7 @@ func (controls *Controls) RunChecks(runner Runner, filter Predicate, skipIDMap m
 			_, groupSkippedViaCmd := skipIDMap[group.ID]
 			_, checkSkippedViaCmd := skipIDMap[check.ID]
 
-			if group.Skip || groupSkippedViaCmd || checkSkippedViaCmd {
+			if group.Type == SKIP || groupSkippedViaCmd || checkSkippedViaCmd {
 				check.Type = SKIP
 			}
 
@@ -119,7 +124,6 @@ func (controls *Controls) RunChecks(runner Runner, filter Predicate, skipIDMap m
 				w := &Group{
 					ID:     group.ID,
 					Text:   group.Text,
-					Skip:   group.Skip,
 					Checks: []*Check{},
 				}
 

check/controls_test.go

@@ -149,7 +149,7 @@ func TestControls_RunChecks_Skipped(t *testing.T) {
 type: "master"
 groups:
 - id: G1
-  skip: true
+  type: skip
   checks:
   - id: G1/C1
 `)
@@ -256,7 +256,7 @@ func TestControls_JUnitIncludesJSON(t *testing.T) {
 			},
 			expect: []byte(`<testsuite name="" tests="0" failures="0" errors="0" time="0">
     <testcase name="check1id check1text" classname="" time="0">
-        <system-out>{&#34;test_number&#34;:&#34;check1id&#34;,&#34;test_desc&#34;:&#34;check1text&#34;,&#34;audit&#34;:&#34;&#34;,&#34;AuditConfig&#34;:&#34;&#34;,&#34;type&#34;:&#34;&#34;,&#34;remediation&#34;:&#34;&#34;,&#34;test_info&#34;:null,&#34;status&#34;:&#34;PASS&#34;,&#34;actual_value&#34;:&#34;&#34;,&#34;scored&#34;:false,&#34;IsMultiple&#34;:false,&#34;expected_result&#34;:&#34;&#34;}</system-out>
+        <system-out>{&#34;test_number&#34;:&#34;check1id&#34;,&#34;test_desc&#34;:&#34;check1text&#34;,&#34;audit&#34;:&#34;&#34;,&#34;AuditEnv&#34;:&#34;&#34;,&#34;AuditConfig&#34;:&#34;&#34;,&#34;type&#34;:&#34;&#34;,&#34;remediation&#34;:&#34;&#34;,&#34;test_info&#34;:null,&#34;status&#34;:&#34;PASS&#34;,&#34;actual_value&#34;:&#34;&#34;,&#34;scored&#34;:false,&#34;IsMultiple&#34;:false,&#34;expected_result&#34;:&#34;&#34;}</system-out>
     </testcase>
 </testsuite>`),
 		}, {
@@ -279,7 +279,7 @@ func TestControls_JUnitIncludesJSON(t *testing.T) {
 			},
 			expect: []byte(`<testsuite name="" tests="402" failures="99" errors="0" time="0">
     <testcase name="check1id check1text" classname="" time="0">
-        <system-out>{&#34;test_number&#34;:&#34;check1id&#34;,&#34;test_desc&#34;:&#34;check1text&#34;,&#34;audit&#34;:&#34;&#34;,&#34;AuditConfig&#34;:&#34;&#34;,&#34;type&#34;:&#34;&#34;,&#34;remediation&#34;:&#34;&#34;,&#34;test_info&#34;:null,&#34;status&#34;:&#34;PASS&#34;,&#34;actual_value&#34;:&#34;&#34;,&#34;scored&#34;:false,&#34;IsMultiple&#34;:false,&#34;expected_result&#34;:&#34;&#34;}</system-out>
+        <system-out>{&#34;test_number&#34;:&#34;check1id&#34;,&#34;test_desc&#34;:&#34;check1text&#34;,&#34;audit&#34;:&#34;&#34;,&#34;AuditEnv&#34;:&#34;&#34;,&#34;AuditConfig&#34;:&#34;&#34;,&#34;type&#34;:&#34;&#34;,&#34;remediation&#34;:&#34;&#34;,&#34;test_info&#34;:null,&#34;status&#34;:&#34;PASS&#34;,&#34;actual_value&#34;:&#34;&#34;,&#34;scored&#34;:false,&#34;IsMultiple&#34;:false,&#34;expected_result&#34;:&#34;&#34;}</system-out>
     </testcase>
 </testsuite>`),
 		}, {
@@ -299,19 +299,19 @@ func TestControls_JUnitIncludesJSON(t *testing.T) {
 			},
 			expect: []byte(`<testsuite name="" tests="0" failures="0" errors="0" time="0">
     <testcase name="check1id check1text" classname="" time="0">
-        <system-out>{&#34;test_number&#34;:&#34;check1id&#34;,&#34;test_desc&#34;:&#34;check1text&#34;,&#34;audit&#34;:&#34;&#34;,&#34;AuditConfig&#34;:&#34;&#34;,&#34;type&#34;:&#34;&#34;,&#34;remediation&#34;:&#34;&#34;,&#34;test_info&#34;:null,&#34;status&#34;:&#34;PASS&#34;,&#34;actual_value&#34;:&#34;&#34;,&#34;scored&#34;:false,&#34;IsMultiple&#34;:false,&#34;expected_result&#34;:&#34;&#34;}</system-out>
+        <system-out>{&#34;test_number&#34;:&#34;check1id&#34;,&#34;test_desc&#34;:&#34;check1text&#34;,&#34;audit&#34;:&#34;&#34;,&#34;AuditEnv&#34;:&#34;&#34;,&#34;AuditConfig&#34;:&#34;&#34;,&#34;type&#34;:&#34;&#34;,&#34;remediation&#34;:&#34;&#34;,&#34;test_info&#34;:null,&#34;status&#34;:&#34;PASS&#34;,&#34;actual_value&#34;:&#34;&#34;,&#34;scored&#34;:false,&#34;IsMultiple&#34;:false,&#34;expected_result&#34;:&#34;&#34;}</system-out>
     </testcase>
     <testcase name="check2id check2text" classname="" time="0">
         <skipped></skipped>
-        <system-out>{&#34;test_number&#34;:&#34;check2id&#34;,&#34;test_desc&#34;:&#34;check2text&#34;,&#34;audit&#34;:&#34;&#34;,&#34;AuditConfig&#34;:&#34;&#34;,&#34;type&#34;:&#34;&#34;,&#34;remediation&#34;:&#34;&#34;,&#34;test_info&#34;:null,&#34;status&#34;:&#34;INFO&#34;,&#34;actual_value&#34;:&#34;&#34;,&#34;scored&#34;:false,&#34;IsMultiple&#34;:false,&#34;expected_result&#34;:&#34;&#34;}</system-out>
+        <system-out>{&#34;test_number&#34;:&#34;check2id&#34;,&#34;test_desc&#34;:&#34;check2text&#34;,&#34;audit&#34;:&#34;&#34;,&#34;AuditEnv&#34;:&#34;&#34;,&#34;AuditConfig&#34;:&#34;&#34;,&#34;type&#34;:&#34;&#34;,&#34;remediation&#34;:&#34;&#34;,&#34;test_info&#34;:null,&#34;status&#34;:&#34;INFO&#34;,&#34;actual_value&#34;:&#34;&#34;,&#34;scored&#34;:false,&#34;IsMultiple&#34;:false,&#34;expected_result&#34;:&#34;&#34;}</system-out>
     </testcase>
     <testcase name="check3id check3text" classname="" time="0">
         <skipped></skipped>
-        <system-out>{&#34;test_number&#34;:&#34;check3id&#34;,&#34;test_desc&#34;:&#34;check3text&#34;,&#34;audit&#34;:&#34;&#34;,&#34;AuditConfig&#34;:&#34;&#34;,&#34;type&#34;:&#34;&#34;,&#34;remediation&#34;:&#34;&#34;,&#34;test_info&#34;:null,&#34;status&#34;:&#34;WARN&#34;,&#34;actual_value&#34;:&#34;&#34;,&#34;scored&#34;:false,&#34;IsMultiple&#34;:false,&#34;expected_result&#34;:&#34;&#34;}</system-out>
+        <system-out>{&#34;test_number&#34;:&#34;check3id&#34;,&#34;test_desc&#34;:&#34;check3text&#34;,&#34;audit&#34;:&#34;&#34;,&#34;AuditEnv&#34;:&#34;&#34;,&#34;AuditConfig&#34;:&#34;&#34;,&#34;type&#34;:&#34;&#34;,&#34;remediation&#34;:&#34;&#34;,&#34;test_info&#34;:null,&#34;status&#34;:&#34;WARN&#34;,&#34;actual_value&#34;:&#34;&#34;,&#34;scored&#34;:false,&#34;IsMultiple&#34;:false,&#34;expected_result&#34;:&#34;&#34;}</system-out>
     </testcase>
     <testcase name="check4id check4text" classname="" time="0">
         <failure type=""></failure>
-        <system-out>{&#34;test_number&#34;:&#34;check4id&#34;,&#34;test_desc&#34;:&#34;check4text&#34;,&#34;audit&#34;:&#34;&#34;,&#34;AuditConfig&#34;:&#34;&#34;,&#34;type&#34;:&#34;&#34;,&#34;remediation&#34;:&#34;&#34;,&#34;test_info&#34;:null,&#34;status&#34;:&#34;FAIL&#34;,&#34;actual_value&#34;:&#34;&#34;,&#34;scored&#34;:false,&#34;IsMultiple&#34;:false,&#34;expected_result&#34;:&#34;&#34;}</system-out>
+        <system-out>{&#34;test_number&#34;:&#34;check4id&#34;,&#34;test_desc&#34;:&#34;check4text&#34;,&#34;audit&#34;:&#34;&#34;,&#34;AuditEnv&#34;:&#34;&#34;,&#34;AuditConfig&#34;:&#34;&#34;,&#34;type&#34;:&#34;&#34;,&#34;remediation&#34;:&#34;&#34;,&#34;test_info&#34;:null,&#34;status&#34;:&#34;FAIL&#34;,&#34;actual_value&#34;:&#34;&#34;,&#34;scored&#34;:false,&#34;IsMultiple&#34;:false,&#34;expected_result&#34;:&#34;&#34;}</system-out>
     </testcase>
 </testsuite>`),
 		},

check/data

@@ -327,6 +327,53 @@ groups:
               op: eq
               value: false
             set: true
+    - id: 29
+      text: "flag is set (via env)"
+      tests:
+        test_items:
+          - flag: "--allow-privileged"
+            env: "ALLOW_PRIVILEGED"
+            set: true
+
+    - id: 30
+      text: "flag is not set (via env)"
+      tests:
+        test_items:
+          - flag: "--basic-auth"
+            env: "BASIC_AUTH"
+            set: false
+
+    - id: 31
+      text: "flag value is set to some value (via env)"
+      tests:
+        test_items:
+          - flag: "--insecure-port"
+            env: "INSECURE_PORT"
+            compare:
+              op: eq
+              value: 0
+            set: true
+
+    - id: 32
+      text: "flag value is greater than or equal some number (via env)"
+      tests:
+        test_items:
+          - flag: "--audit-log-maxage"
+            env: "AUDIT_LOG_MAXAGE"
+            compare:
+              op: gte
+              value: 30
+            set: true
+
+    - id: 33
+      text: "flag value is less than some number (via env)"
+      tests:
+        test_items:
+          - env: "MAX_BACKLOG"
+            compare:
+              op: lt
+              value: 30
+            set: true
 
 - id: 2.1
   text: "audit and audit_config commands"
@@ -557,3 +604,104 @@ groups:
             path: '{.readOnlyPort}'
             set: false
       scored: true
+
+- id: 3.1
+  text: "audit_env commands"
+  checks:
+    - id: 0
+      text: "audit fails to find flag, audit_env finds flag -> pass"
+      audit: "echo in=incorrect"
+      audit_env: "echo flag=correct"
+      tests:
+        test_items:
+          - flag: "flag"
+            env: "flag"
+            compare:
+              op: eq
+              value: "correct"
+            set: true
+      scored: true
+    - id: 1
+      text: "audit fails to find flag, audit_env finds flag and fails -> fail"
+      audit: "echo in=wrong"
+      audit_env: "echo flag=wrong"
+      tests:
+        test_items:
+          - flag: "flag"
+            env: "flag"
+            compare:
+              op: eq
+              value: "correct"
+            set: true
+      scored: true
+    - id: 2
+      text: "audit finds correct flag, audit_env is incorrect -> pass"
+      audit: "echo flag=correct"
+      audit_env: "echo flag=incorrect"
+      tests:
+        test_items:
+          - flag: "flag"
+            env: "flag"
+            compare:
+              op: eq
+              value: "correct"
+            set: true
+      scored: true
+    - id: 3
+      text: "audit doesn't flag flag, audit_config finds it and passes, audit_env is not present -> pass"
+      audit: "echo in=correct"
+      audit_config: "echo 'flag: correct'"
+      tests:
+        test_items:
+          - flag: "flag"
+            path: "{.flag}"
+            compare:
+              op: eq
+              value: "correct"
+            set: true
+      scored: true
+    - id: 4
+      text: "audit doesn't flag flag, audit_config doesn't find flag, audit_env finds and passes -> pass"
+      audit: "echo in=correct"
+      audit_config: "echo 'in: correct'"
+      audit_env: "echo flag=correct"
+      tests:
+        test_items:
+          - flag: "flag"
+            path: "{.flag}"
+            env: "flag"
+            compare:
+              op: eq
+              value: "correct"
+            set: true
+      scored: true
+    - id: 5
+      text: "audit doesn't find flag, audit_config doesn't find flag, audit_env finds and fails -> fails"
+      audit: "echo in=correct"
+      audit_config: "echo 'in: correct'"
+      audit_env: "echo flag=incorrect"
+      tests:
+        test_items:
+          - flag: "flag"
+            path: "{.flag}"
+            env: "flag"
+            compare:
+              op: eq
+              value: "correct"
+            set: true
+      scored: true
+    - id: 6
+      text: "audit finds flag and fails, audit_config finds flag and fails, audit_env finds and passes -> fails"
+      audit: "echo flag=incorrect"
+      audit_config: "echo 'flag: incorrect'"
+      audit_env: "echo flag=correct"
+      tests:
+        test_items:
+          - flag: "flag"
+            path: "{.flag}"
+            env: "flag"
+            compare:
+              op: eq
+              value: "correct"
+            set: true
+      scored: true

check/test.go

@@ -48,17 +48,27 @@ type tests struct {
 	BinOp     binOp       `yaml:"bin_op"`
 }
 
+type AuditUsed string
+
+const (
+	AuditCommand AuditUsed = "auditCommand"
+	AuditConfig AuditUsed = "auditConfig"
+	AuditEnv AuditUsed = "auditEnv"
+)
+
 type testItem struct {
 	Flag             string
+	Env              string
 	Path             string
 	Output           string
 	Value            string
 	Set              bool
 	Compare          compare
 	isMultipleOutput bool
-	isConfigSetting  bool
+	auditUsed  AuditUsed
 }
 
+type envTestItem testItem
 type pathTestItem testItem
 type flagTestItem testItem
 
@@ -69,7 +79,7 @@ type compare struct {
 
 type testOutput struct {
 	testResult     bool
-	flagFound      bool
+	found      bool
 	actualResult   string
 	ExpectedResult string
 }
@@ -78,16 +88,25 @@ func failTestItem(s string) *testOutput {
 	return &testOutput{testResult: false, actualResult: s}
 }
 
-func (t testItem) flagValue() string {
+func (t testItem) value() string {
-	if t.isConfigSetting {
+	if t.auditUsed == AuditConfig {
 		return t.Path
 	}
 
+	if t.auditUsed == AuditEnv {
+		return t.Env
+	}
+
 	return t.Flag
 }
 
 func (t testItem) findValue(s string) (match bool, value string, err error) {
-	if t.isConfigSetting {
+	if t.auditUsed == AuditEnv {
+		et := envTestItem(t)
+		return et.findValue(s)
+	}
+
+	if t.auditUsed == AuditConfig {
 		pt := pathTestItem(t)
 		return pt.findValue(s)
 	}
@@ -145,10 +164,28 @@ func (t pathTestItem) findValue(s string) (match bool, value string, err error)
 	}
 
 	glog.V(3).Infof("In pathTestItem.findValue %s", value)
-	match = (value != "")
+	match = value != ""
 	return match, value, err
 }
 
+func (t envTestItem) findValue(s string) (match bool, value string, err error) {
+	if s != "" && t.Env != "" {
+		r, _ := regexp.Compile(fmt.Sprintf("%s=.*(?:$|\\n)", t.Env))
+		out := r.FindString(s)
+		out = strings.Replace(out, "\n", "", 1)
+		out = strings.Replace(out, fmt.Sprintf("%s=", t.Env), "", 1)
+
+		if len(out) > 0 {
+			match = true
+			value = out
+		}else{
+			match = false
+			value = ""
+		}
+	}
+	return match, value, nil
+}
+
 func (t testItem) execute(s string) *testOutput {
 	result := &testOutput{}
 	s = strings.TrimRight(s, " \n")
@@ -186,16 +223,16 @@ func (t testItem) evaluate(s string) *testOutput {
 		if match && t.Compare.Op != "" {
 			result.ExpectedResult, result.testResult = compareOp(t.Compare.Op, value, t.Compare.Value)
 		} else {
-			result.ExpectedResult = fmt.Sprintf("'%s' is present", t.flagValue())
+			result.ExpectedResult = fmt.Sprintf("'%s' is present", t.value())
 			result.testResult = match
 		}
 	} else {
-		result.ExpectedResult = fmt.Sprintf("'%s' is not present", t.flagValue())
+		result.ExpectedResult = fmt.Sprintf("'%s' is not present", t.value())
 		result.testResult = !match
 	}
 
-	result.flagFound = match
+	result.found = match
-	glog.V(3).Info(fmt.Sprintf("flagFound %v", result.flagFound))
+	glog.V(3).Info(fmt.Sprintf("found %v", result.found))
 
 	return result
 }

check/test_test.go

@@ -51,168 +51,231 @@ func TestTestExecute(t *testing.T) {
 		*Check
 		str       string
 		strConfig string
+		strEnv    string
 	}{
 		{
 			controls.Groups[0].Checks[0],
 			"2:45 ../kubernetes/kube-apiserver --allow-privileged=false --option1=20,30,40",
 			"",
+			"",
 		},
 		{
 			controls.Groups[0].Checks[1],
 			"2:45 ../kubernetes/kube-apiserver --allow-privileged=false",
 			"",
+			"",
 		},
 		{
 			controls.Groups[0].Checks[2],
 			"niinai   13617  2635 99 19:26 pts/20   00:03:08 ./kube-apiserver --insecure-port=0 --anonymous-auth",
 			"",
+			"",
 		},
 		{
 			controls.Groups[0].Checks[3],
 			"2:45 ../kubernetes/kube-apiserver --secure-port=0 --audit-log-maxage=40 --option",
 			"",
+			"",
 		},
 		{
 			controls.Groups[0].Checks[4],
 			"2:45 ../kubernetes/kube-apiserver --max-backlog=20 --secure-port=0 --audit-log-maxage=40 --option",
 			"",
+			"",
 		},
 		{
 			controls.Groups[0].Checks[5],
 			"2:45 ../kubernetes/kube-apiserver --option --admission-control=WebHook,RBAC ---audit-log-maxage=40",
 			"",
+			"",
 		},
 		{
 			controls.Groups[0].Checks[6],
 			"2:45 .. --kubelet-clientkey=foo --kubelet-client-certificate=bar --admission-control=Webhook,RBAC",
 			"",
+			"",
 		},
 		{
 			controls.Groups[0].Checks[7],
 			"2:45 ..  --secure-port=0 --kubelet-client-certificate=bar --admission-control=Webhook,RBAC",
 			"",
+			"",
 		},
 		{
 			controls.Groups[0].Checks[8],
 			"644",
 			"",
+			"",
 		},
 		{
 			controls.Groups[0].Checks[9],
 			"640",
 			"",
+			"",
 		},
 		{
 			controls.Groups[0].Checks[9],
 			"600",
 			"",
+			"",
 		},
 		{
 			controls.Groups[0].Checks[10],
 			"2:45 ../kubernetes/kube-apiserver --option --admission-control=WebHook,RBAC ---audit-log-maxage=40",
 			"",
+			"",
 		},
 		{
 			controls.Groups[0].Checks[11],
 			"2:45 ../kubernetes/kube-apiserver --option --admission-control=WebHook,RBAC ---audit-log-maxage=40",
 			"",
+			"",
 		},
 		{
 			controls.Groups[0].Checks[12],
 			"2:45 ../kubernetes/kube-apiserver --option --admission-control=WebHook,Something,RBAC ---audit-log-maxage=40",
 			"",
+			"",
 		},
 		{
 			controls.Groups[0].Checks[13],
 			"2:45 ../kubernetes/kube-apiserver --option --admission-control=Something ---audit-log-maxage=40",
 			"",
+			"",
 		},
 		{
 			// check for ':' as argument-value separator, with space between arg and val
 			controls.Groups[0].Checks[14],
 			"2:45 kube-apiserver some-arg: some-val --admission-control=Something ---audit-log-maxage=40",
 			"",
+			"",
 		},
 		{
 			// check for ':' as argument-value separator, with no space between arg and val
 			controls.Groups[0].Checks[14],
 			"2:45 kube-apiserver some-arg:some-val --admission-control=Something ---audit-log-maxage=40",
 			"",
+			"",
 		},
 		{
 			controls.Groups[0].Checks[15],
 			"",
 			"{\"readOnlyPort\": 15000}",
+			"",
 		},
 		{
 			controls.Groups[0].Checks[16],
 			"",
 			"{\"stringValue\": \"WebHook,Something,RBAC\"}",
+			"",
 		},
 		{
 			controls.Groups[0].Checks[17],
 			"",
 			"{\"trueValue\": true}",
+			"",
 		},
 		{
 			controls.Groups[0].Checks[18],
 			"",
 			"{\"readOnlyPort\": 15000}",
+			"",
 		},
 		{
 			controls.Groups[0].Checks[19],
 			"",
 			"{\"authentication\": { \"anonymous\": {\"enabled\": false}}}",
+			"",
 		},
 		{
 			controls.Groups[0].Checks[20],
 			"",
 			"readOnlyPort: 15000",
+			"",
 		},
 		{
 			controls.Groups[0].Checks[21],
 			"",
 			"readOnlyPort: 15000",
+			"",
 		},
 		{
 			controls.Groups[0].Checks[22],
 			"",
 			"authentication:\n  anonymous:\n    enabled: false",
+			"",
 		},
 		{
 			controls.Groups[0].Checks[26],
 			"",
 			"currentMasterVersion: 1.12.7",
+			"",
 		},
 		{
 			controls.Groups[0].Checks[27],
 			"--peer-client-cert-auth",
 			"",
+			"",
 		},
 		{
 			controls.Groups[0].Checks[27],
 			"--abc=true --peer-client-cert-auth --efg=false",
 			"",
+			"",
 		},
 		{
 			controls.Groups[0].Checks[27],
 			"--abc --peer-client-cert-auth --efg",
 			"",
+			"",
 		},
 		{
 			controls.Groups[0].Checks[27],
 			"--peer-client-cert-auth=true",
 			"",
+			"",
 		},
 		{
 			controls.Groups[0].Checks[27],
 			"--abc --peer-client-cert-auth=true --efg",
 			"",
+			"",
 		},
 		{
 			controls.Groups[0].Checks[28],
 			"--abc --peer-client-cert-auth=false --efg",
 			"",
+			"",
+		},
+		{
+			controls.Groups[0].Checks[29],
+			"2:45 ../kubernetes/kube-apiserver --option1=20,30,40",
+			"",
+			"SOME_OTHER_ENV=true\nALLOW_PRIVILEGED=false",
+		},
+		{
+			controls.Groups[0].Checks[30],
+			"2:45 ../kubernetes/kube-apiserver --option1=20,30,40",
+			"",
+			"",
+		},
+		{
+			controls.Groups[0].Checks[31],
+			"2:45 ../kubernetes/kube-apiserver --option1=20,30,40",
+			"",
+			"INSECURE_PORT=0",
+		},
+		{
+			controls.Groups[0].Checks[32],
+			"2:45 ../kubernetes/kube-apiserver --option1=20,30,40",
+			"",
+			"AUDIT_LOG_MAXAGE=40",
+		},
+		{
+			controls.Groups[0].Checks[33],
+			"2:45 ../kubernetes/kube-apiserver --option1=20,30,40",
+			"",
+			"MAX_BACKLOG=20",
 		},
 	}
 
@@ -220,6 +283,7 @@ func TestTestExecute(t *testing.T) {
 		t.Run(c.Text, func(t *testing.T) {
 			c.Check.AuditOutput = c.str
 			c.Check.AuditConfigOutput = c.strConfig
+			c.Check.AuditEnvOutput = c.strEnv
 			res, err := c.Check.execute()
 			if err != nil {
 				t.Errorf(err.Error())

cmd/common.go

@@ -100,11 +100,11 @@ func runChecks(nodetype check.NodeType, testYamlFile string) {
 
 	// Variable substitutions. Replace all occurrences of variables in controls files.
 	s := string(in)
-	s = makeSubstitutions(s, "bin", binmap)
+	s, binSubs := makeSubstitutions(s, "bin", binmap)
-	s = makeSubstitutions(s, "conf", confmap)
+	s, _ = makeSubstitutions(s, "conf", confmap)
-	s = makeSubstitutions(s, "svc", svcmap)
+	s, _ = makeSubstitutions(s, "svc", svcmap)
-	s = makeSubstitutions(s, "kubeconfig", kubeconfmap)
+	s, _ = makeSubstitutions(s, "kubeconfig", kubeconfmap)
-	s = makeSubstitutions(s, "cafile", cafilemap)
+	s, _ = makeSubstitutions(s, "cafile", cafilemap)
 
 	controls, err := check.NewControls(nodetype, []byte(s))
 	if err != nil {
@@ -117,10 +117,36 @@ func runChecks(nodetype check.NodeType, testYamlFile string) {
 		exitWithError(fmt.Errorf("error setting up run filter: %v", err))
 	}
 
+	generateDefaultEnvAudit(controls, binSubs)
+
 	controls.RunChecks(runner, filter, parseSkipIds(skipIds))
 	controlsCollection = append(controlsCollection, controls)
 }
 
+func generateDefaultEnvAudit(controls *check.Controls, binSubs []string){
+	for _, group := range controls.Groups {
+		for _, checkItem := range group.Checks {
+			if checkItem.Tests != nil && !checkItem.DisableEnvTesting {
+				for _, test := range checkItem.Tests.TestItems {
+					if test.Env != "" && checkItem.AuditEnv == "" {
+						binPath := ""
+
+						if len(binSubs) == 1 {
+							binPath = binSubs[0]
+						} else {
+							fmt.Printf("AuditEnv not explicit for check (%s), where bin path cannot be determined\n", checkItem.ID)
+						}
+
+						if test.Env != "" && checkItem.AuditEnv == "" {
+							checkItem.AuditEnv = fmt.Sprintf("cat \"/proc/$(/bin/ps -C %s -o pid= | tr -d ' ')/environ\" | tr '\\0' '\\n'", binPath)
+						}
+					}
+				}
+			}
+		}
+	}
+}
+
 func parseSkipIds(skipIds string) map[string]bool {
 	var skipIdMap = make(map[string]bool, 0)
 	if skipIds != "" {
@@ -159,7 +185,7 @@ func prettyPrint(r *check.Controls, summary check.Summary) {
 	// Print remediations.
 	if !noRemediations {
 		if summary.Fail > 0 || summary.Warn > 0 {
-			colors[check.WARN].Printf("== Remediations ==\n")
+			colors[check.WARN].Printf("== Remediations %s ==\n", r.Type)
 			for _, g := range r.Groups {
 				for _, c := range g.Checks {
 					if c.State == check.FAIL {
@@ -181,6 +207,11 @@ func prettyPrint(r *check.Controls, summary check.Summary) {
 
 	// Print summary setting output color to highest severity.
 	if !noSummary {
+		printSummary(summary, string(r.Type))
+	}
+}
+
+func printSummary(summary check.Summary, sectionName string) {
 	var res check.State
 	if summary.Fail > 0 {
 		res = check.FAIL
@@ -190,11 +221,10 @@ func prettyPrint(r *check.Controls, summary check.Summary) {
 		res = check.PASS
 	}
 
-		colors[res].Printf("== Summary ==\n")
+	colors[res].Printf("== Summary %s ==\n", sectionName)
-		fmt.Printf("%d checks PASS\n%d checks FAIL\n%d checks WARN\n%d checks INFO\n",
+	fmt.Printf("%d checks PASS\n%d checks FAIL\n%d checks WARN\n%d checks INFO\n\n",
 		summary.Pass, summary.Fail, summary.Warn, summary.Info,
 	)
-	}
 }
 
 // loadConfig finds the correct config dir based on the kubernetes version,
@@ -287,13 +317,17 @@ func getBenchmarkVersion(kubeVersion, benchmarkVersion string, v *viper.Viper) (
 	if !isEmpty(kubeVersion) && !isEmpty(benchmarkVersion) {
 		return "", fmt.Errorf("It is an error to specify both --version and --benchmark flags")
 	}
+	if isEmpty(benchmarkVersion) && isEmpty(kubeVersion) {
+		benchmarkVersion = getPlatformBenchmarkVersion(getPlatformName())
+	}
 
 	if isEmpty(benchmarkVersion) {
 		if isEmpty(kubeVersion) {
-			kubeVersion, err = getKubeVersion()
+			kv, err := getKubeVersion()
 			if err != nil {
 				return "", fmt.Errorf("Version check failed: %s\nAlternatively, you can specify the version with --version", err)
 			}
+			kubeVersion = kv.BaseVersion()
 		}
 
 		kubeToBenchmarkMap, err := loadVersionMapping(v)
@@ -381,7 +415,16 @@ func writeOutput(controlsCollection []*check.Controls) {
 }
 
 func writeJSONOutput(controlsCollection []*check.Controls) {
-	out, err := json.Marshal(controlsCollection)
+	var out []byte
+	var err error
+	if !noTotals {
+		var totals check.OverallControls
+		totals.Controls = controlsCollection
+		totals.Totals = getSummaryTotals(controlsCollection)
+		out, err = json.Marshal(totals)
+	} else {
+		out, err = json.Marshal(controlsCollection)
+	}
 	if err != nil {
 		exitWithError(fmt.Errorf("failed to output in JSON format: %v", err))
 	}
@@ -425,6 +468,21 @@ func writeStdoutOutput(controlsCollection []*check.Controls) {
 		summary := controls.Summary
 		prettyPrint(controls, summary)
 	}
+	if !noTotals {
+		printSummary(getSummaryTotals(controlsCollection), "total")
+	}
+}
+
+func getSummaryTotals(controlsCollection []*check.Controls) check.Summary {
+	var totalSummary check.Summary
+	for _, controls := range controlsCollection {
+		summary := controls.Summary
+		totalSummary.Fail = totalSummary.Fail + summary.Fail
+		totalSummary.Warn = totalSummary.Warn + summary.Warn
+		totalSummary.Pass = totalSummary.Pass + summary.Pass
+		totalSummary.Info = totalSummary.Info + summary.Info
+	}
+	return totalSummary
 }
 
 func printRawOutput(output string) {

cmd/common_test.go

@@ -30,6 +30,15 @@ import (
 	"github.com/stretchr/testify/assert"
 )
 
+type JsonOutputFormat struct {
+	Controls     []*check.Controls `json:"Controls"`
+	TotalSummary map[string]int    `json:"Totals"`
+}
+
+type JsonOutputFormatNoTotals struct {
+	Controls []*check.Controls `json:"Controls"`
+}
+
 func TestParseSkipIds(t *testing.T) {
 	skipMap := parseSkipIds("4.12,4.13,5")
 	_, fourTwelveExists := skipMap["4.12"]
@@ -315,7 +324,7 @@ func TestGetBenchmarkVersion(t *testing.T) {
 
 	withFakeKubectl := func(kubeVersion, benchmarkVersion string, v *viper.Viper, fn getBenchmarkVersionFnToTest) (string, error) {
 		execCode := `#!/bin/sh
-		echo "Server Version: v1.15.10"
+		echo '{"serverVersion": {"major": "1", "minor": "15", "gitVersion": "v1.15.10"}}'
 		`
 		restore, err := fakeExecutableInPath("kubectl", execCode)
 		if err != nil {
@@ -527,13 +536,13 @@ func TestWriteResultToJsonFile(t *testing.T) {
 	}
 	writeOutput(controlsCollection)
 
-	var expect []*check.Controls
+	var expect JsonOutputFormat
-	var result []*check.Controls
+	var result JsonOutputFormat
-	result, err = parseControlsJsonFile(outputFile)
+	result, err = parseResultJsonFile(outputFile)
 	if err != nil {
 		t.Error(err)
 	}
-	expect, err = parseControlsJsonFile("./testdata/result.json")
+	expect, err = parseResultJsonFile("./testdata/result.json")
 	if err != nil {
 		t.Error(err)
 	}
@@ -541,7 +550,39 @@ func TestWriteResultToJsonFile(t *testing.T) {
 	assert.Equal(t, expect, result)
 }
 
-func TestExitCodeSelection(t *testing.T){
+func TestWriteResultNoTotalsToJsonFile(t *testing.T) {
+	defer func() {
+		controlsCollection = []*check.Controls{}
+		jsonFmt = false
+		outputFile = ""
+	}()
+	var err error
+	jsonFmt = true
+	outputFile = path.Join(os.TempDir(), fmt.Sprintf("%d", time.Now().UnixNano()))
+
+	noTotals = true
+
+	controlsCollection, err = parseControlsJsonFile("./testdata/controlsCollection.json")
+	if err != nil {
+		t.Error(err)
+	}
+	writeOutput(controlsCollection)
+
+	var expect []*check.Controls
+	var result []*check.Controls
+	result, err = parseResultNoTotalsJsonFile(outputFile)
+	if err != nil {
+		t.Error(err)
+	}
+	expect, err = parseResultNoTotalsJsonFile("./testdata/result_no_totals.json")
+	if err != nil {
+		t.Error(err)
+	}
+
+	assert.Equal(t, expect, result)
+}
+
+func TestExitCodeSelection(t *testing.T) {
 	exitCode = 10
 	controlsCollectionAllPassed, errPassed := parseControlsJsonFile("./testdata/passedControlsCollection.json")
 	if errPassed != nil {
@@ -559,6 +600,149 @@ func TestExitCodeSelection(t *testing.T){
 	assert.Equal(t, 10, exitCodeFailure)
 }
 
+func TestGenerationDefaultEnvAudit(t *testing.T) {
+	input := []byte(`
+---
+type: "master"
+groups:
+- id: G1
+  checks:
+  - id: G1/C1
+- id: G2
+  checks:
+  - id: G2/C1
+    text: "Verify that the SomeSampleFlag argument is set to true"
+    audit: "grep -B1 SomeSampleFlag=true /this/is/a/file/path"
+    tests:
+      test_items:
+      - flag: "SomeSampleFlag=true"
+        env: "SOME_SAMPLE_FLAG"
+        compare:
+          op: has
+          value: "true"
+        set: true
+    remediation: |
+      Edit the config file /this/is/a/file/path and set SomeSampleFlag to true.
+    scored: true
+`)
+	controls, err := check.NewControls(check.MASTER, input)
+	assert.NoError(t, err)
+
+	binSubs := []string{"TestBinPath"}
+	generateDefaultEnvAudit(controls, binSubs)
+
+	expectedAuditEnv := fmt.Sprintf("cat \"/proc/$(/bin/ps -C %s -o pid= | tr -d ' ')/environ\" | tr '\\0' '\\n'", binSubs[0])
+	assert.Equal(t, expectedAuditEnv, controls.Groups[1].Checks[0].AuditEnv)
+}
+
+func TestGetSummaryTotals(t *testing.T) {
+	controlsCollection, err := parseControlsJsonFile("./testdata/controlsCollection.json")
+	if err != nil {
+		t.Error(err)
+	}
+
+	resultTotals := getSummaryTotals(controlsCollection)
+	assert.Equal(t, 12, resultTotals.Fail)
+	assert.Equal(t, 14, resultTotals.Warn)
+	assert.Equal(t, 0, resultTotals.Info)
+	assert.Equal(t, 49, resultTotals.Pass)
+}
+
+func TestPrintSummary(t *testing.T) {
+	controlsCollection, err := parseControlsJsonFile("./testdata/controlsCollection.json")
+	if err != nil {
+		t.Error(err)
+	}
+
+	resultTotals := getSummaryTotals(controlsCollection)
+	rescueStdout := os.Stdout
+	r, w, _ := os.Pipe()
+	os.Stdout = w
+	printSummary(resultTotals, "totals")
+	w.Close()
+	out, _ := ioutil.ReadAll(r)
+	os.Stdout = rescueStdout
+
+	assert.Contains(t, string(out), "49 checks PASS\n12 checks FAIL\n14 checks WARN\n0 checks INFO\n\n")
+}
+
+func TestPrettyPrintNoSummary(t *testing.T) {
+	controlsCollection, err := parseControlsJsonFile("./testdata/controlsCollection.json")
+	if err != nil {
+		t.Error(err)
+	}
+
+	resultTotals := getSummaryTotals(controlsCollection)
+	rescueStdout := os.Stdout
+	r, w, _ := os.Pipe()
+	os.Stdout = w
+	noSummary = true
+	prettyPrint(controlsCollection[0], resultTotals)
+	w.Close()
+	out, _ := ioutil.ReadAll(r)
+	os.Stdout = rescueStdout
+
+	assert.NotContains(t, string(out), "49 checks PASS")
+}
+
+func TestPrettyPrintSummary(t *testing.T) {
+	controlsCollection, err := parseControlsJsonFile("./testdata/controlsCollection.json")
+	if err != nil {
+		t.Error(err)
+	}
+
+	resultTotals := getSummaryTotals(controlsCollection)
+	rescueStdout := os.Stdout
+	r, w, _ := os.Pipe()
+	os.Stdout = w
+	noSummary = false
+	prettyPrint(controlsCollection[0], resultTotals)
+	w.Close()
+	out, _ := ioutil.ReadAll(r)
+	os.Stdout = rescueStdout
+
+	assert.Contains(t, string(out), "49 checks PASS")
+}
+
+func TestWriteStdoutOutputNoTotal(t *testing.T) {
+	controlsCollection, err := parseControlsJsonFile("./testdata/controlsCollection.json")
+	if err != nil {
+		t.Error(err)
+	}
+
+	rescueStdout := os.Stdout
+	r, w, _ := os.Pipe()
+	os.Stdout = w
+	noTotals = true
+	writeStdoutOutput(controlsCollection)
+	w.Close()
+	out, _ := ioutil.ReadAll(r)
+	os.Stdout = rescueStdout
+
+	assert.NotContains(t, string(out), "49 checks PASS")
+}
+
+func TestWriteStdoutOutputTotal(t *testing.T) {
+	controlsCollection, err := parseControlsJsonFile("./testdata/controlsCollection.json")
+	if err != nil {
+		t.Error(err)
+	}
+
+	rescueStdout := os.Stdout
+
+	r, w, _ := os.Pipe()
+
+	os.Stdout = w
+	noTotals = false
+	writeStdoutOutput(controlsCollection)
+	w.Close()
+	out, _ := ioutil.ReadAll(r)
+
+	os.Stdout = rescueStdout
+
+	assert.Contains(t, string(out), "49 checks PASS")
+}
+
 func parseControlsJsonFile(filepath string) ([]*check.Controls, error) {
 	var result []*check.Controls
 
@@ -574,6 +758,36 @@ func parseControlsJsonFile(filepath string) ([]*check.Controls, error) {
 	return result, nil
 }
 
+func parseResultJsonFile(filepath string) (JsonOutputFormat, error) {
+	var result JsonOutputFormat
+
+	d, err := ioutil.ReadFile(filepath)
+	if err != nil {
+		return result, err
+	}
+	err = json.Unmarshal(d, &result)
+	if err != nil {
+		return result, err
+	}
+
+	return result, nil
+}
+
+func parseResultNoTotalsJsonFile(filepath string) ([]*check.Controls, error) {
+	var result []*check.Controls
+
+	d, err := ioutil.ReadFile(filepath)
+	if err != nil {
+		return nil, err
+	}
+	err = json.Unmarshal(d, &result)
+	if err != nil {
+		return nil, err
+	}
+
+	return result, nil
+}
+
 func loadConfigForTest() (*viper.Viper, error) {
 	viperWithData := viper.New()
 	viperWithData.SetConfigFile("../cfg/config.yaml")

cmd/kubernetes_version.go

@@ -14,7 +14,25 @@ import (
 	"github.com/golang/glog"
 )
 
-func getKubeVersionFromRESTAPI() (string, error) {
+type KubeVersion struct {
+	Major       string
+	Minor       string
+	baseVersion string
+	GitVersion  string
+}
+
+func (k *KubeVersion) BaseVersion() string {
+	if k.baseVersion != "" {
+		return k.baseVersion
+	}
+	// Some provides return the minor version like "15+"
+	minor := strings.Replace(k.Minor, "+", "", -1)
+	ver := fmt.Sprintf("%s.%s", k.Major, minor)
+	k.baseVersion = ver
+	return ver
+}
+
+func getKubeVersionFromRESTAPI() (*KubeVersion, error) {
 	k8sVersionURL := getKubernetesURL()
 	serviceaccount := "/var/run/secrets/kubernetes.io/serviceaccount"
 	cacertfile := fmt.Sprintf("%s/ca.crt", serviceaccount)
@@ -22,23 +40,23 @@ func getKubeVersionFromRESTAPI() (string, error) {
 
 	tlsCert, err := loadCertficate(cacertfile)
 	if err != nil {
-		return "", err
+		return nil, err
 	}
 
 	tb, err := ioutil.ReadFile(tokenfile)
 	if err != nil {
-		return "", err
+		return nil, err
 	}
 	token := strings.TrimSpace(string(tb))
 
 	data, err := getWebDataWithRetry(k8sVersionURL, token, tlsCert)
 	if err != nil {
-		return "", err
+		return nil, err
 	}
 
 	k8sVersion, err := extractVersion(data)
 	if err != nil {
-		return "", err
+		return nil, err
 	}
 	return k8sVersion, nil
 }
@@ -61,8 +79,7 @@ func getWebDataWithRetry(k8sVersionURL, token string, cacert *tls.Certificate) (
 	return
 }
 
-func extractVersion(data []byte) (string, error) {
+type VersionResponse struct {
-	type versionResponse struct {
 	Major        string
 	Minor        string
 	GitVersion   string
@@ -72,20 +89,22 @@ func extractVersion(data []byte) (string, error) {
 	GoVersion    string
 	Compiler     string
 	Platform     string
-	}
+}
 
-	vrObj := &versionResponse{}
+func extractVersion(data []byte) (*KubeVersion, error) {
+	vrObj := &VersionResponse{}
 	glog.V(2).Info(fmt.Sprintf("vd: %s\n", string(data)))
 	err := json.Unmarshal(data, vrObj)
 	if err != nil {
-		return "", err
+		return nil, err
 	}
 	glog.V(2).Info(fmt.Sprintf("vrObj: %#v\n", vrObj))
 
-	// Some provides return the minor version like "15+"
+	return &KubeVersion{
-	minor := strings.Replace(vrObj.Minor, "+", "", -1)
+		Major:      vrObj.Major,
-	ver := fmt.Sprintf("%s.%s", vrObj.Major, minor)
+		Minor:      vrObj.Minor,
-	return ver, nil
+		GitVersion: vrObj.GitVersion,
+	}, nil
 }
 
 func getWebData(srvURL, token string, cacert *tls.Certificate) ([]byte, error) {

cmd/kubernetes_version_test.go

@@ -218,7 +218,7 @@ func TestExtractVersion(t *testing.T) {
 				if err != nil {
 					t.Errorf("unexpected error: %v", err)
 				}
-				if c.expectedVer != ver {
+				if c.expectedVer != ver.BaseVersion() {
 					t.Errorf("Expected %q but Got %q", c.expectedVer, ver)
 				}
 			} else {

cmd/root.go

@@ -54,6 +54,7 @@ var (
 	noSummary           bool
 	noRemediations      bool
 	skipIds             string
+	noTotals            bool
 	filterOpts          FilterOpts
 	includeTestOutput   bool
 	outputFile          string
@@ -87,6 +88,8 @@ var RootCmd = &cobra.Command{
 				glog.V(1).Info("== Running control plane checks ==")
 				runChecks(check.CONTROLPLANE, loadConfig(check.CONTROLPLANE, bv))
 			}
+		} else {
+			glog.V(1).Info("== Skipping master checks ==")
 		}
 
 		// Etcd is only valid for CIS 1.5 and later,
@@ -98,6 +101,8 @@ var RootCmd = &cobra.Command{
 		if valid && isEtcd() {
 			glog.V(1).Info("== Running etcd checks ==")
 			runChecks(check.ETCD, loadConfig(check.ETCD, bv))
+		} else {
+			glog.V(1).Info("== Skipping etcd checks ==")
 		}
 
 		glog.V(1).Info("== Running node checks ==")
@@ -112,6 +117,8 @@ var RootCmd = &cobra.Command{
 		if valid {
 			glog.V(1).Info("== Running policies checks ==")
 			runChecks(check.POLICIES, loadConfig(check.POLICIES, bv))
+		} else {
+			glog.V(1).Info("== Skipping policies checks ==")
 		}
 
 		// Managedservices is only valid for GKE 1.0 and later,
@@ -123,6 +130,8 @@ var RootCmd = &cobra.Command{
 		if valid {
 			glog.V(1).Info("== Running managed services checks ==")
 			runChecks(check.MANAGEDSERVICES, loadConfig(check.MANAGEDSERVICES, bv))
+		} else {
+			glog.V(1).Info("== Skipping managed services checks ==")
 		}
 
 		writeOutput(controlsCollection)
@@ -154,6 +163,7 @@ func init() {
 	RootCmd.PersistentFlags().BoolVar(&noResults, "noresults", false, "Disable printing of results section")
 	RootCmd.PersistentFlags().BoolVar(&noSummary, "nosummary", false, "Disable printing of summary section")
 	RootCmd.PersistentFlags().BoolVar(&noRemediations, "noremediations", false, "Disable printing of remediations section")
+	RootCmd.PersistentFlags().BoolVar(&noTotals, "nototals", false, "Disable printing of totals for failed, passed, ... checks across all sections")
 	RootCmd.PersistentFlags().BoolVar(&jsonFmt, "json", false, "Prints the results as JSON")
 	RootCmd.PersistentFlags().BoolVar(&junitFmt, "junit", false, "Prints the results as JUnit")
 	RootCmd.PersistentFlags().BoolVar(&pgSQL, "pgsql", false, "Save the results to PostgreSQL")

cmd/testdata/result.json

@@ -1,4 +1,5 @@
-[
+{
+  "Controls": [
     {
       "id": "1",
       "version": "1.5",
@@ -111,4 +112,11 @@
       "total_warn": 3,
       "total_info": 0
     }
-]
+  ],
+  "Totals": {
+    "total_pass": 49,
+    "total_fail": 12,
+    "total_warn": 14,
+    "total_info": 0
+  }
+}

cmd/testdata/result_no_totals.json

@@ -0,0 +1,114 @@
+[
+  {
+    "id": "1",
+    "version": "1.5",
+    "text": "Master Node Security Configuration",
+    "node_type": "master",
+    "tests": [
+      {
+        "section": "1.1",
+        "pass": 15,
+        "fail": 1,
+        "warn": 5,
+        "info": 0,
+        "desc": "Master Node Configuration Files",
+        "results": [
+          {
+            "test_number": "1.1.1",
+            "test_desc": "Ensure that the API server pod specification file permissions are set to 644 or more restrictive (Scored)",
+            "audit": "/bin/sh -c 'if test -e /etc/kubernetes/manifests/kube-apiserver.yaml; then stat -c permissions=%a /etc/kubernetes/manifests/kube-apiserver.yaml; fi'",
+            "AuditConfig": "",
+            "type": "",
+            "remediation": "Run the below command (based on the file location on your system) on the\nmaster node.\nFor example, chmod 644 /etc/kubernetes/manifests/kube-apiserver.yaml\n",
+            "test_info": [
+              "Run the below command (based on the file location on your system) on the\nmaster node.\nFor example, chmod 644 /etc/kubernetes/manifests/kube-apiserver.yaml\n"
+            ],
+            "status": "PASS",
+            "actual_value": "permissions=600\n",
+            "scored": true,
+            "expected_result": "bitmask '600' AND '644'"
+          }
+        ]
+      }
+    ],
+    "total_pass": 42,
+    "total_fail": 12,
+    "total_warn": 11,
+    "total_info": 0
+  },
+  {
+    "id": "2",
+    "version": "1.15",
+    "text": "Etcd Node Configuration",
+    "node_type": "etcd",
+    "tests": [
+      {
+        "section": "2",
+        "pass": 7,
+        "fail": 0,
+        "warn": 0,
+        "info": 0,
+        "desc": "Etcd Node Configuration Files",
+        "results": [
+          {
+            "test_number": "2.1",
+            "test_desc": "Ensure that the --cert-file and --key-file arguments are set as appropriate (Scored)",
+            "audit": "/bin/ps -ef | /bin/grep etcd | /bin/grep -v grep",
+            "AuditConfig": "",
+            "type": "",
+            "remediation": "Follow the etcd service documentation and configure TLS encryption.\nThen, edit the etcd pod specification file /etc/kubernetes/manifests/etcd.yaml\non the master node and set the below parameters.\n--cert-file=</path/to/ca-file>\n--key-file=</path/to/key-file>\n",
+            "test_info": [
+              "Follow the etcd service documentation and configure TLS encryption.\nThen, edit the etcd pod specification file /etc/kubernetes/manifests/etcd.yaml\non the master node and set the below parameters.\n--cert-file=</path/to/ca-file>\n--key-file=</path/to/key-file>\n"
+            ],
+            "status": "PASS",
+            "actual_value": "root      3277  3218  3 Apr19 ?        03:57:52 etcd --advertise-client-urls=https://192.168.64.4:2379 --cert-file=/var/lib/minikube/certs/etcd/server.crt --client-cert-auth=true --data-dir=/var/lib/minikube/etcd --initial-advertise-peer-urls=https://192.168.64.4:2380 --initial-cluster=minikube=https://192.168.64.4:2380 --key-file=/var/lib/minikube/certs/etcd/server.key --listen-client-urls=https://127.0.0.1:2379,https://192.168.64.4:2379 --listen-metrics-urls=http://127.0.0.1:2381 --listen-peer-urls=https://192.168.64.4:2380 --name=minikube --peer-cert-file=/var/lib/minikube/certs/etcd/peer.crt --peer-client-cert-auth=true --peer-key-file=/var/lib/minikube/certs/etcd/peer.key --peer-trusted-ca-file=/var/lib/minikube/certs/etcd/ca.crt --snapshot-count=10000 --trusted-ca-file=/var/lib/minikube/certs/etcd/ca.crt\nroot      4624  4605  8 Apr21 ?        04:55:10 kube-apiserver --advertise-address=192.168.64.4 --allow-privileged=true --authorization-mode=Node,RBAC --client-ca-file=/var/lib/minikube/certs/ca.crt --enable-admission-plugins=NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,NodeRestriction,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,ResourceQuota,PodSecurityPolicy --enable-bootstrap-token-auth=true --etcd-cafile=/var/lib/minikube/certs/etcd/ca.crt --etcd-certfile=/var/lib/minikube/certs/apiserver-etcd-client.crt --etcd-keyfile=/var/lib/minikube/certs/apiserver-etcd-client.key --etcd-servers=https://127.0.0.1:2379 --insecure-port=0 --kubelet-client-certificate=/var/lib/minikube/certs/apiserver-kubelet-client.crt --kubelet-client-key=/var/lib/minikube/certs/apiserver-kubelet-client.key --kubelet-preferred-address-types=InternalIP,ExternalIP,Hostname --proxy-client-cert-file=/var/lib/minikube/certs/front-proxy-client.crt --proxy-client-key-file=/var/lib/minikube/certs/front-proxy-client.key --requestheader-allowed-names=front-proxy-client --requestheader-client-ca-file=/var/lib/minikube/certs/front-proxy-ca.crt --requestheader-extra-headers-prefix=X-Remote-Extra- --requestheader-group-headers=X-Remote-Group --requestheader-username-headers=X-Remote-User --secure-port=8443 --service-account-key-file=/var/lib/minikube/certs/sa.pub --service-cluster-ip-range=10.96.0.0/12 --tls-cert-file=/var/lib/minikube/certs/apiserver.crt --tls-private-key-file=/var/lib/minikube/certs/apiserver.key\n",
+            "scored": true,
+            "expected_result": "'--cert-file' is present AND '--key-file' is present"
+          }
+        ]
+      }
+    ],
+    "total_pass": 7,
+    "total_fail": 0,
+    "total_warn": 0,
+    "total_info": 0
+  },
+  {
+    "id": "3",
+    "version": "1.5",
+    "text": "Control Plane Configuration",
+    "node_type": "controlplane",
+    "tests": [
+      {
+        "section": "3.1",
+        "pass": 0,
+        "fail": 0,
+        "warn": 1,
+        "info": 0,
+        "desc": "Authentication and Authorization",
+        "results": [
+          {
+            "test_number": "3.1.1",
+            "test_desc": "Client certificate authentication should not be used for users (Not Scored)",
+            "audit": "",
+            "AuditConfig": "",
+            "type": "manual",
+            "remediation": "Alternative mechanisms provided by Kubernetes such as the use of OIDC should be\nimplemented in place of client certificates.\n",
+            "test_info": [
+              "Alternative mechanisms provided by Kubernetes such as the use of OIDC should be\nimplemented in place of client certificates.\n"
+            ],
+            "status": "WARN",
+            "actual_value": "",
+            "scored": false,
+            "expected_result": "",
+            "reason": "Test marked as a manual test"
+          }
+        ]
+      }
+    ],
+    "total_pass": 0,
+    "total_fail": 0,
+    "total_warn": 3,
+    "total_info": 0
+  }
+]

cmd/util.go

@@ -1,6 +1,7 @@
 package cmd
 
 import (
+	"encoding/json"
 	"fmt"
 	"os"
 	"os/exec"
@@ -93,7 +94,7 @@ func getBinaries(v *viper.Viper, nodetype check.NodeType) (map[string]string, er
 		if len(bins) > 0 {
 			bin, err := findExecutable(bins)
 			if err != nil && !optional {
-				glog.Warning(buildComponentMissingErrorMessage(nodetype, component, bins))
+				glog.V(1).Info(buildComponentMissingErrorMessage(nodetype, component, bins))
 				return nil, fmt.Errorf("unable to detect running programs for component %q", component)
 			}
 
@@ -279,7 +280,7 @@ Alternatively, you can specify the version with --version
    kube-bench --version <VERSION> ...
 `
 
-func getKubeVersion() (string, error) {
+func getKubeVersion() (*KubeVersion, error) {
 
 	if k8sVer, err := getKubeVersionFromRESTAPI(); err == nil {
 		glog.V(2).Info(fmt.Sprintf("Kubernetes REST API Reported version: %s", k8sVer))
@@ -300,7 +301,7 @@ func getKubeVersion() (string, error) {
 			}
 
 			glog.Warning(missingKubectlKubeletMessage)
-			return "", fmt.Errorf("unable to find the programs kubectl or kubelet in the PATH")
+			return nil, fmt.Errorf("unable to find the programs kubectl or kubelet in the PATH")
 		}
 		return getKubeVersionFromKubelet(), nil
 	}
@@ -308,8 +309,8 @@ func getKubeVersion() (string, error) {
 	return getKubeVersionFromKubectl(), nil
 }
 
-func getKubeVersionFromKubectl() string {
+func getKubeVersionFromKubectl() *KubeVersion {
-	cmd := exec.Command("kubectl", "version", "--short")
+	cmd := exec.Command("kubectl", "version", "-o", "json")
 	out, err := cmd.CombinedOutput()
 	if err != nil {
 		glog.V(2).Info(err)
@@ -318,7 +319,7 @@ func getKubeVersionFromKubectl() string {
 	return getVersionFromKubectlOutput(string(out))
 }
 
-func getKubeVersionFromKubelet() string {
+func getKubeVersionFromKubelet() *KubeVersion {
 	cmd := exec.Command("kubelet", "--version")
 	out, err := cmd.CombinedOutput()
 
@@ -329,31 +330,42 @@ func getKubeVersionFromKubelet() string {
 	return getVersionFromKubeletOutput(string(out))
 }
 
-func getVersionFromKubectlOutput(s string) string {
+func getVersionFromKubectlOutput(s string) *KubeVersion {
-	serverVersionRe := regexp.MustCompile(`Server Version: v(\d+.\d+)`)
+	glog.V(2).Info(s)
-	subs := serverVersionRe.FindStringSubmatch(s)
+	type versionResult struct {
-	if len(subs) < 2 {
+		ServerVersion VersionResponse
+	}
+	vrObj := &versionResult{}
+	if err := json.Unmarshal([]byte(s), vrObj); err != nil {
+		glog.V(2).Info(err)
 		if strings.Contains(s, "The connection to the server") {
 			msg := fmt.Sprintf(`Warning: Kubernetes version was not auto-detected because kubectl could not connect to the Kubernetes server. This may be because the kubeconfig information is missing or has credentials that do not match the server. Assuming default version %s`, defaultKubeVersion)
 			fmt.Fprintln(os.Stderr, msg)
 		}
 		glog.V(1).Info(fmt.Sprintf("Unable to get Kubernetes version from kubectl, using default version: %s", defaultKubeVersion))
-		return defaultKubeVersion
+		return &KubeVersion{baseVersion: defaultKubeVersion}
+	}
+	sv := vrObj.ServerVersion
+	return &KubeVersion{
+		Major:      sv.Major,
+		Minor:      sv.Minor,
+		GitVersion: sv.GitVersion,
 	}
-	return subs[1]
 }
 
-func getVersionFromKubeletOutput(s string) string {
+func getVersionFromKubeletOutput(s string) *KubeVersion {
+	glog.V(2).Info(s)
 	serverVersionRe := regexp.MustCompile(`Kubernetes v(\d+.\d+)`)
 	subs := serverVersionRe.FindStringSubmatch(s)
 	if len(subs) < 2 {
 		glog.V(1).Info(fmt.Sprintf("Unable to get Kubernetes version from kubelet, using default version: %s", defaultKubeVersion))
-		return defaultKubeVersion
+		return &KubeVersion{baseVersion: defaultKubeVersion}
 	}
-	return subs[1]
+	return &KubeVersion{baseVersion: subs[1]}
 }
 
-func makeSubstitutions(s string, ext string, m map[string]string) string {
+func makeSubstitutions(s string, ext string, m map[string]string) (string, []string) {
+	substitutions := make([]string, 0)
 	for k, v := range m {
 		subst := "$" + k + ext
 		if v == "" {
@@ -361,10 +373,14 @@ func makeSubstitutions(s string, ext string, m map[string]string) string {
 			continue
 		}
 		glog.V(2).Info(fmt.Sprintf("Substituting %s with '%s'\n", subst, v))
+		beforeS := s
 		s = multiWordReplace(s, subst, v)
+		if beforeS != s {
+			substitutions = append(substitutions, v)
+		}
 	}
 
-	return s
+	return s, substitutions
 }
 
 func isEmpty(str string) bool {
@@ -403,3 +419,31 @@ These program names are provided in the config.yaml, section '%s.%s.bins'
 
 	return fmt.Sprintf(errMessageTemplate, component, componentRoleName, binList, componentType, component)
 }
+
+func getPlatformName() string {
+	kv, err := getKubeVersion()
+	if err != nil {
+		glog.V(2).Info(err)
+		return ""
+	}
+	return getPlatformNameFromVersion(kv.GitVersion)
+}
+
+func getPlatformNameFromVersion(s string) string {
+	versionRe := regexp.MustCompile(`v\d+\.\d+\.\d+-(\w+)(?:[.\-])\w+`)
+	subs := versionRe.FindStringSubmatch(s)
+	if len(subs) < 2 {
+		return ""
+	}
+	return subs[1]
+}
+
+func getPlatformBenchmarkVersion(platform string) string {
+	switch platform {
+	case "eks":
+		return "eks-1.0"
+	case "gke":
+		return "gke-1.0"
+	}
+	return ""
+}

cmd/util_test.go

@@ -15,6 +15,7 @@
 package cmd
 
 import (
+	"github.com/magiconair/properties/assert"
 	"io/ioutil"
 	"os"
 	"path/filepath"
@@ -201,17 +202,21 @@ func TestMultiWordReplace(t *testing.T) {
 	}
 }
 
-func TestKubeVersionRegex(t *testing.T) {
+func Test_getVersionFromKubectlOutput(t *testing.T) {
-	ver := getVersionFromKubectlOutput(`Client Version: v1.8.0
+	ver := getVersionFromKubectlOutput(`{
-		Server Version: v1.8.12
+  "serverVersion": {
-		`)
+    "major": "1",
-	if ver != "1.8" {
+    "minor": "8",
-		t.Fatalf("Expected 1.8 got %s", ver)
+    "gitVersion": "v1.8.0"
+  }
+}`)
+	if ver.BaseVersion() != "1.8" {
+		t.Fatalf("Expected 1.8 got %s", ver.BaseVersion())
 	}
 
 	ver = getVersionFromKubectlOutput("Something completely different")
-	if ver != defaultKubeVersion {
+	if ver.BaseVersion() != defaultKubeVersion {
-		t.Fatalf("Expected %s got %s", defaultKubeVersion, ver)
+		t.Fatalf("Expected %s got %s", defaultKubeVersion, ver.BaseVersion())
 	}
 }
 
@@ -387,17 +392,19 @@ func TestMakeSubsitutions(t *testing.T) {
 		input string
 		subst map[string]string
 		exp   string
+		expectedSubs []string
 	}{
-		{input: "Replace $thisbin", subst: map[string]string{"this": "that"}, exp: "Replace that"},
+		{input: "Replace $thisbin", subst: map[string]string{"this": "that"}, exp: "Replace that", expectedSubs: []string{"that"}},
-		{input: "Replace $thisbin", subst: map[string]string{"this": "that", "here": "there"}, exp: "Replace that"},
+		{input: "Replace $thisbin", subst: map[string]string{"this": "that", "here": "there"}, exp: "Replace that", expectedSubs: []string{"that"}},
-		{input: "Replace $thisbin and $herebin", subst: map[string]string{"this": "that", "here": "there"}, exp: "Replace that and there"},
+		{input: "Replace $thisbin and $herebin", subst: map[string]string{"this": "that", "here": "there"}, exp: "Replace that and there", expectedSubs: []string{"that", "there"}},
 	}
 	for _, c := range cases {
 		t.Run(c.input, func(t *testing.T) {
-			s := makeSubstitutions(c.input, "bin", c.subst)
+			s, subs := makeSubstitutions(c.input, "bin", c.subst)
 			if s != c.exp {
 				t.Fatalf("Got %s expected %s", s, c.exp)
 			}
+			assert.Equal(t, c.expectedSubs, subs)
 		})
 	}
 }
@@ -509,3 +516,89 @@ func TestGetYamlFilesFromDir(t *testing.T) {
 		t.Fatalf("Expected to find something.yaml, found %s", files[0])
 	}
 }
+
+func Test_getPlatformNameFromKubectlOutput(t *testing.T) {
+	type args struct {
+		s string
+	}
+	tests := []struct {
+		name string
+		args args
+		want string
+	}{
+		{
+			name: "eks",
+			args: args{s: "v1.17.9-eks-4c6976"},
+			want: "eks",
+		},
+		{
+			name: "gke",
+			args: args{s: "v1.17.6-gke.1"},
+			want: "gke",
+		},
+		{
+			name: "unknown",
+			args: args{s: "v1.17.6"},
+			want: "",
+		},
+		{
+			name: "empty string",
+			args: args{s: ""},
+			want: "",
+		},
+	}
+	for _, tt := range tests {
+		t.Run(tt.name, func(t *testing.T) {
+			if got := getPlatformNameFromVersion(tt.args.s); got != tt.want {
+				t.Errorf("getPlatformNameFromKubectlOutput() = %v, want %v", got, tt.want)
+			}
+		})
+	}
+}
+
+func Test_getPlatformBenchmarkVersion(t *testing.T) {
+	type args struct {
+		platform string
+	}
+	tests := []struct {
+		name string
+		args args
+		want string
+	}{
+		{
+			name: "eks",
+			args: args{
+				platform: "eks",
+			},
+			want: "eks-1.0",
+		},
+		{
+			name: "gke",
+			args: args{
+				platform: "gke",
+			},
+			want: "gke-1.0",
+		},
+		{
+			name: "unknown",
+			args: args{
+				platform: "rh",
+			},
+			want: "",
+		},
+		{
+			name: "empty",
+			args: args{
+				platform: "",
+			},
+			want: "",
+		},
+	}
+	for _, tt := range tests {
+		t.Run(tt.name, func(t *testing.T) {
+			if got := getPlatformBenchmarkVersion(tt.args.platform); got != tt.want {
+				t.Errorf("getPlatformBenchmarkVersion() = %v, want %v", got, tt.want)
+			}
+		})
+	}
+}

docs/README.md

@@ -150,8 +150,8 @@ pass a check. This criteria is made up of keywords extracted from the output of
 the `audit` command and operations that compare these keywords against
 values expected by the CIS Kubernetes Benchmark. 
 
-There are two ways to extract keywords from the output of the `audit` command,
+There are three ways to extract keywords from the output of the `audit` command,
-`flag` and `path`.
+`flag`, `path`, `env`.
 
 `flag` is used when the keyword is a command-line flag. The associated `audit`
 command is usually a `ps` command and a `grep` for the binary whose flag we are
@@ -186,6 +186,23 @@ tests:
     # ...
 ```
 
+`env` is used to check if the value is present within a specified environment variable. The presence of `env` is treated as an OR operation, if both `flag` and `env` are supplied it will use either to attempt pass the check.
+The command used for checking the environment variables of a process **is generated by default**.
+
+If the command being generated is causing errors, you can override the command used by setting `auditEnv` on the check.
+Similarly, if you don't want the environment checking command to be generated or run at all, specify `disableEnvTesting` as true on the check.
+
+The example below will check if the flag `--auto-tls` is equal to false *OR* `ETCD_AUTO_TLS` is equal to false
+
+```yml
+  test_items:
+  - flag: "--auto-tls"
+    env: "ETCD_AUTO_TLS"
+    compare:
+      op: eq
+      value: false
+```
+
 `test_item` compares the output of the audit command and keywords using the
 `set` and `compare` fields.
 

docs/asff.md

@@ -19,6 +19,23 @@ You can configure kube-bench with the `--asff` to send findings to AWS Security
   * You can run the kube-bench pod under a specific [service account associated with an IAM role](https://docs.aws.amazon.com/eks/latest/userguide/iam-roles-for-service-accounts.html) that has these permissions to write Security Hub findings.
   * Alternatively the pod can be granted permissions specified by the Role that your [EKS node group uses](https://docs.aws.amazon.com/eks/latest/userguide/managed-node-groups.html).
   
+Here is an example IAM Policy that you can attach to your EKS node group's IAM Role: 
+
+```json
+{
+    "Version": "2012-10-17",
+    "Statement": [
+        {
+            "Effect": "Allow",
+            "Action": "securityhub:BatchImportFindings",
+            "Resource": [
+                "arn:aws:securityhub:us-east-1::product/aqua-security/kube-bench"
+            ]
+        }
+    ]
+}
+```
+
 ## Configure and rebuild kube-bench
 
 You will need to download, build and push the kube-bench container image to your ECR repo as described in Step 3 of the [EKS instructions][eks-instructions], except that before you build the container image, you need to edit `cfg/eks-1.0/config.yaml` to specify the AWS account, AWS region, and the EKS Cluster ARN.

go.mod

@@ -15,6 +15,7 @@ require (
 	github.com/jinzhu/inflection v0.0.0-20170102125226-1c35d901db3d // indirect
 	github.com/jinzhu/now v1.0.1 // indirect
 	github.com/lib/pq v0.0.0-20171126050459-83612a56d3dd // indirect
+	github.com/magiconair/properties v1.8.0
 	github.com/mattn/go-colorable v0.0.0-20170210172801-5411d3eea597 // indirect
 	github.com/mattn/go-isatty v0.0.0-20170307163044-57fdcb988a5c // indirect
 	github.com/mattn/go-sqlite3 v1.10.0 // indirect

integration/testdata/cis-1.5/job-master.data

@@ -63,13 +63,13 @@
 [PASS] 1.3.3 Ensure that the --use-service-account-credentials argument is set to true (Scored)
 [PASS] 1.3.4 Ensure that the --service-account-private-key-file argument is set as appropriate (Scored)
 [PASS] 1.3.5 Ensure that the --root-ca-file argument is set as appropriate (Scored)
-[FAIL] 1.3.6 Ensure that the RotateKubeletServerCertificate argument is set to true (Scored)
+[PASS] 1.3.6 Ensure that the RotateKubeletServerCertificate argument is set to true (Scored)
 [PASS] 1.3.7 Ensure that the --bind-address argument is set to 127.0.0.1 (Scored)
 [INFO] 1.4 Scheduler
 [FAIL] 1.4.1 Ensure that the --profiling argument is set to false (Scored)
 [PASS] 1.4.2 Ensure that the --bind-address argument is set to 127.0.0.1 (Scored)
 
-== Remediations ==
+== Remediations master ==
 1.1.9 Run the below command (based on the file location on your system) on the master node.
 For example,
 chmod 644 <path/to/cni/files>
@@ -160,17 +160,19 @@ for example:
 on the master node and set the below parameter.
 --profiling=false
 
-1.3.6 Edit the Controller Manager pod specification file /etc/kubernetes/manifests/kube-controller-manager.yaml
-on the master node and set the --feature-gates parameter to include RotateKubeletServerCertificate=true.
---feature-gates=RotateKubeletServerCertificate=true
-
 1.4.1 Edit the Scheduler pod specification file /etc/kubernetes/manifests/kube-scheduler.yaml file
 on the master node and set the below parameter.
 --profiling=false
 
 
-== Summary ==
+== Summary master ==
-44 checks PASS
+45 checks PASS
-11 checks FAIL
+10 checks FAIL
+10 checks WARN
+0 checks INFO
+
+== Summary total ==
+45 checks PASS
+10 checks FAIL
 10 checks WARN
 0 checks INFO

integration/testdata/cis-1.5/job-node.data

@@ -22,10 +22,10 @@
 [WARN] 4.2.9 Ensure that the --event-qps argument is set to 0 or a level which ensures appropriate event capture (Not Scored)
 [FAIL] 4.2.10 Ensure that the --tls-cert-file and --tls-private-key-file arguments are set as appropriate (Scored)
 [PASS] 4.2.11 Ensure that the --rotate-certificates argument is not set to false (Scored)
-[FAIL] 4.2.12 Ensure that the RotateKubeletServerCertificate argument is set to true (Scored)
+[PASS] 4.2.12 Ensure that the RotateKubeletServerCertificate argument is set to true (Scored)
 [PASS] 4.2.13 Ensure that the Kubelet only makes use of Strong Cryptographic Ciphers (Not Scored)
 
-== Remediations ==
+== Remediations node ==
 4.2.6 If using a Kubelet config file, edit the file to set protectKernelDefaults: true.
 If using command line arguments, edit the kubelet service file
 /etc/systemd/system/kubelet.service.d/10-kubeadm.conf on each worker node and
@@ -55,16 +55,15 @@ Based on your system, restart the kubelet service. For example:
 systemctl daemon-reload
 systemctl restart kubelet.service
 
-4.2.12 Edit the kubelet service file /etc/systemd/system/kubelet.service.d/10-kubeadm.conf
-on each worker node and set the below parameter in KUBELET_CERTIFICATE_ARGS variable.
---feature-gates=RotateKubeletServerCertificate=true
-Based on your system, restart the kubelet service. For example:
-systemctl daemon-reload
-systemctl restart kubelet.service
 
-
+== Summary node ==
-== Summary ==
+20 checks PASS
-19 checks PASS
+2 checks FAIL
-3 checks FAIL
+1 checks WARN
+0 checks INFO
+
+== Summary total ==
+20 checks PASS
+2 checks FAIL
 1 checks WARN
 0 checks INFO

integration/testdata/cis-1.5/job.data

@@ -63,13 +63,13 @@
 [PASS] 1.3.3 Ensure that the --use-service-account-credentials argument is set to true (Scored)
 [PASS] 1.3.4 Ensure that the --service-account-private-key-file argument is set as appropriate (Scored)
 [PASS] 1.3.5 Ensure that the --root-ca-file argument is set as appropriate (Scored)
-[FAIL] 1.3.6 Ensure that the RotateKubeletServerCertificate argument is set to true (Scored)
+[PASS] 1.3.6 Ensure that the RotateKubeletServerCertificate argument is set to true (Scored)
 [PASS] 1.3.7 Ensure that the --bind-address argument is set to 127.0.0.1 (Scored)
 [INFO] 1.4 Scheduler
 [FAIL] 1.4.1 Ensure that the --profiling argument is set to false (Scored)
 [PASS] 1.4.2 Ensure that the --bind-address argument is set to 127.0.0.1 (Scored)
 
-== Remediations ==
+== Remediations master ==
 1.1.9 Run the below command (based on the file location on your system) on the master node.
 For example,
 chmod 644 <path/to/cni/files>
@@ -160,20 +160,17 @@ for example:
 on the master node and set the below parameter.
 --profiling=false
 
-1.3.6 Edit the Controller Manager pod specification file /etc/kubernetes/manifests/kube-controller-manager.yaml
-on the master node and set the --feature-gates parameter to include RotateKubeletServerCertificate=true.
---feature-gates=RotateKubeletServerCertificate=true
-
 1.4.1 Edit the Scheduler pod specification file /etc/kubernetes/manifests/kube-scheduler.yaml file
 on the master node and set the below parameter.
 --profiling=false
 
 
-== Summary ==
+== Summary master ==
-44 checks PASS
+45 checks PASS
-11 checks FAIL
+10 checks FAIL
 10 checks WARN
 0 checks INFO
+
 [INFO] 2 Etcd Node Configuration
 [INFO] 2 Etcd Node Configuration Files
 [PASS] 2.1 Ensure that the --cert-file and --key-file arguments are set as appropriate (Scored)
@@ -184,11 +181,12 @@ on the master node and set the below parameter.
 [PASS] 2.6 Ensure that the --peer-auto-tls argument is not set to true (Scored)
 [PASS] 2.7 Ensure that a unique Certificate Authority is used for etcd (Not Scored)
 
-== Summary ==
+== Summary etcd ==
 7 checks PASS
 0 checks FAIL
 0 checks WARN
 0 checks INFO
+
 [INFO] 3 Control Plane Configuration
 [INFO] 3.1 Authentication and Authorization
 [WARN] 3.1.1 Client certificate authentication should not be used for users (Not Scored)
@@ -196,7 +194,7 @@ on the master node and set the below parameter.
 [FAIL] 3.2.1 Ensure that a minimal audit policy is created (Scored)
 [WARN] 3.2.2 Ensure that the audit policy covers key security concerns (Not Scored)
 
-== Remediations ==
+== Remediations controlplane ==
 3.1.1 Alternative mechanisms provided by Kubernetes such as the use of OIDC should be
 implemented in place of client certificates.
 
@@ -206,11 +204,12 @@ implemented in place of client certificates.
 minimum.
 
 
-== Summary ==
+== Summary controlplane ==
 0 checks PASS
 1 checks FAIL
 2 checks WARN
 0 checks INFO
+
 [INFO] 4 Worker Node Security Configuration
 [INFO] 4.1 Worker Node Configuration Files
 [PASS] 4.1.1 Ensure that the kubelet service file permissions are set to 644 or more restrictive (Scored)
@@ -235,10 +234,10 @@ minimum.
 [WARN] 4.2.9 Ensure that the --event-qps argument is set to 0 or a level which ensures appropriate event capture (Not Scored)
 [FAIL] 4.2.10 Ensure that the --tls-cert-file and --tls-private-key-file arguments are set as appropriate (Scored)
 [PASS] 4.2.11 Ensure that the --rotate-certificates argument is not set to false (Scored)
-[FAIL] 4.2.12 Ensure that the RotateKubeletServerCertificate argument is set to true (Scored)
+[PASS] 4.2.12 Ensure that the RotateKubeletServerCertificate argument is set to true (Scored)
 [PASS] 4.2.13 Ensure that the Kubelet only makes use of Strong Cryptographic Ciphers (Not Scored)
 
-== Remediations ==
+== Remediations node ==
 4.2.6 If using a Kubelet config file, edit the file to set protectKernelDefaults: true.
 If using command line arguments, edit the kubelet service file
 /etc/systemd/system/kubelet.service.d/10-kubeadm.conf on each worker node and
@@ -268,19 +267,13 @@ Based on your system, restart the kubelet service. For example:
 systemctl daemon-reload
 systemctl restart kubelet.service
 
-4.2.12 Edit the kubelet service file /etc/systemd/system/kubelet.service.d/10-kubeadm.conf
-on each worker node and set the below parameter in KUBELET_CERTIFICATE_ARGS variable.
---feature-gates=RotateKubeletServerCertificate=true
-Based on your system, restart the kubelet service. For example:
-systemctl daemon-reload
-systemctl restart kubelet.service
 
-
+== Summary node ==
-== Summary ==
+20 checks PASS
-19 checks PASS
+2 checks FAIL
-3 checks FAIL
 1 checks WARN
 0 checks INFO
+
 [INFO] 5 Kubernetes Policies
 [INFO] 5.1 RBAC and Service Accounts
 [WARN] 5.1.1 Ensure that the cluster-admin role is only used where required (Not Scored)
@@ -313,7 +306,7 @@ systemctl restart kubelet.service
 [WARN] 5.7.3 Apply Security Context to Your Pods and Containers (Not Scored)
 [WARN] 5.7.4 The default namespace should not be used (Scored)
 
-== Remediations ==
+== Remediations policies ==
 5.1.1 Identify all clusterrolebindings to the cluster-admin role. Check if they are used and
 if they need this role or if they could use a role with fewer privileges.
 Where possible, first bind users to a lower privileged role and then remove the
@@ -410,8 +403,14 @@ Containers.
 resources and that all new resources are created in a specific namespace.
 
 
-== Summary ==
+== Summary policies ==
 0 checks PASS
 0 checks FAIL
 24 checks WARN
 0 checks INFO
+
+== Summary total ==
+72 checks PASS
+13 checks FAIL
+37 checks WARN
+0 checks INFO

integration/testdata/cis-1.6/job-master.data

@@ -63,13 +63,13 @@
 [PASS] 1.3.3 Ensure that the --use-service-account-credentials argument is set to true (Automated)
 [PASS] 1.3.4 Ensure that the --service-account-private-key-file argument is set as appropriate (Automated)
 [PASS] 1.3.5 Ensure that the --root-ca-file argument is set as appropriate (Automated)
-[FAIL] 1.3.6 Ensure that the RotateKubeletServerCertificate argument is set to true (Automated)
+[PASS] 1.3.6 Ensure that the RotateKubeletServerCertificate argument is set to true (Automated)
 [PASS] 1.3.7 Ensure that the --bind-address argument is set to 127.0.0.1 (Automated)
 [INFO] 1.4 Scheduler
 [FAIL] 1.4.1 Ensure that the --profiling argument is set to false (Automated)
 [PASS] 1.4.2 Ensure that the --bind-address argument is set to 127.0.0.1 (Automated)
 
-== Remediations ==
+== Remediations master ==
 1.1.9 Run the below command (based on the file location on your system) on the master node.
 For example,
 chmod 644 <path/to/cni/files>
@@ -163,17 +163,19 @@ for example:
 on the master node and set the below parameter.
 --profiling=false
 
-1.3.6 Edit the Controller Manager pod specification file /etc/kubernetes/manifests/kube-controller-manager.yaml
-on the master node and set the --feature-gates parameter to include RotateKubeletServerCertificate=true.
---feature-gates=RotateKubeletServerCertificate=true
-
 1.4.1 Edit the Scheduler pod specification file /etc/kubernetes/manifests/kube-scheduler.yaml file
 on the master node and set the below parameter.
 --profiling=false
 
 
-== Summary ==
+== Summary master ==
-44 checks PASS
+45 checks PASS
-11 checks FAIL
+10 checks FAIL
+10 checks WARN
+0 checks INFO
+
+== Summary total ==
+45 checks PASS
+10 checks FAIL
 10 checks WARN
 0 checks INFO

integration/testdata/cis-1.6/job-node.data

@@ -22,10 +22,10 @@
 [WARN] 4.2.9 Ensure that the --event-qps argument is set to 0 or a level which ensures appropriate event capture (Manual)
 [WARN] 4.2.10 Ensure that the --tls-cert-file and --tls-private-key-file arguments are set as appropriate (Manual)
 [PASS] 4.2.11 Ensure that the --rotate-certificates argument is not set to false (Manual)
-[WARN] 4.2.12 Verify that the RotateKubeletServerCertificate argument is set to true (Manual)
+[PASS] 4.2.12 Verify that the RotateKubeletServerCertificate argument is set to true (Manual)
 [PASS] 4.2.13 Ensure that the Kubelet only makes use of Strong Cryptographic Ciphers (Manual)
 
-== Remediations ==
+== Remediations node ==
 4.2.6 If using a Kubelet config file, edit the file to set protectKernelDefaults: true.
 If using command line arguments, edit the kubelet service file
 /etc/systemd/system/kubelet.service.d/10-kubeadm.conf on each worker node and
@@ -55,16 +55,15 @@ Based on your system, restart the kubelet service. For example:
 systemctl daemon-reload
 systemctl restart kubelet.service
 
-4.2.12 Edit the kubelet service file /etc/systemd/system/kubelet.service.d/10-kubeadm.conf
-on each worker node and set the below parameter in KUBELET_CERTIFICATE_ARGS variable.
---feature-gates=RotateKubeletServerCertificate=true
-Based on your system, restart the kubelet service. For example:
-systemctl daemon-reload
-systemctl restart kubelet.service
 
-
+== Summary node ==
-== Summary ==
+20 checks PASS
-19 checks PASS
 1 checks FAIL
-3 checks WARN
+2 checks WARN
+0 checks INFO
+
+== Summary total ==
+20 checks PASS
+1 checks FAIL
+2 checks WARN
 0 checks INFO

integration/testdata/cis-1.6/job.data

@@ -63,13 +63,13 @@
 [PASS] 1.3.3 Ensure that the --use-service-account-credentials argument is set to true (Automated)
 [PASS] 1.3.4 Ensure that the --service-account-private-key-file argument is set as appropriate (Automated)
 [PASS] 1.3.5 Ensure that the --root-ca-file argument is set as appropriate (Automated)
-[FAIL] 1.3.6 Ensure that the RotateKubeletServerCertificate argument is set to true (Automated)
+[PASS] 1.3.6 Ensure that the RotateKubeletServerCertificate argument is set to true (Automated)
 [PASS] 1.3.7 Ensure that the --bind-address argument is set to 127.0.0.1 (Automated)
 [INFO] 1.4 Scheduler
 [FAIL] 1.4.1 Ensure that the --profiling argument is set to false (Automated)
 [PASS] 1.4.2 Ensure that the --bind-address argument is set to 127.0.0.1 (Automated)
 
-== Remediations ==
+== Remediations master ==
 1.1.9 Run the below command (based on the file location on your system) on the master node.
 For example,
 chmod 644 <path/to/cni/files>
@@ -163,20 +163,17 @@ for example:
 on the master node and set the below parameter.
 --profiling=false
 
-1.3.6 Edit the Controller Manager pod specification file /etc/kubernetes/manifests/kube-controller-manager.yaml
-on the master node and set the --feature-gates parameter to include RotateKubeletServerCertificate=true.
---feature-gates=RotateKubeletServerCertificate=true
-
 1.4.1 Edit the Scheduler pod specification file /etc/kubernetes/manifests/kube-scheduler.yaml file
 on the master node and set the below parameter.
 --profiling=false
 
 
-== Summary ==
+== Summary master ==
-44 checks PASS
+45 checks PASS
-11 checks FAIL
+10 checks FAIL
 10 checks WARN
 0 checks INFO
+
 [INFO] 2 Etcd Node Configuration
 [INFO] 2 Etcd Node Configuration Files
 [PASS] 2.1 Ensure that the --cert-file and --key-file arguments are set as appropriate (Automated)
@@ -187,11 +184,12 @@ on the master node and set the below parameter.
 [PASS] 2.6 Ensure that the --peer-auto-tls argument is not set to true (Automated)
 [PASS] 2.7 Ensure that a unique Certificate Authority is used for etcd (Manual)
 
-== Summary ==
+== Summary etcd ==
 7 checks PASS
 0 checks FAIL
 0 checks WARN
 0 checks INFO
+
 [INFO] 3 Control Plane Configuration
 [INFO] 3.1 Authentication and Authorization
 [WARN] 3.1.1 Client certificate authentication should not be used for users (Manual)
@@ -199,7 +197,7 @@ on the master node and set the below parameter.
 [WARN] 3.2.1 Ensure that a minimal audit policy is created (Manual)
 [WARN] 3.2.2 Ensure that the audit policy covers key security concerns (Manual)
 
-== Remediations ==
+== Remediations controlplane ==
 3.1.1 Alternative mechanisms provided by Kubernetes such as the use of OIDC should be
 implemented in place of client certificates.
 
@@ -209,11 +207,12 @@ implemented in place of client certificates.
 minimum.
 
 
-== Summary ==
+== Summary controlplane ==
 0 checks PASS
 0 checks FAIL
 3 checks WARN
 0 checks INFO
+
 [INFO] 4 Worker Node Security Configuration
 [INFO] 4.1 Worker Node Configuration Files
 [PASS] 4.1.1 Ensure that the kubelet service file permissions are set to 644 or more restrictive (Automated)
@@ -238,10 +237,10 @@ minimum.
 [WARN] 4.2.9 Ensure that the --event-qps argument is set to 0 or a level which ensures appropriate event capture (Manual)
 [WARN] 4.2.10 Ensure that the --tls-cert-file and --tls-private-key-file arguments are set as appropriate (Manual)
 [PASS] 4.2.11 Ensure that the --rotate-certificates argument is not set to false (Manual)
-[WARN] 4.2.12 Verify that the RotateKubeletServerCertificate argument is set to true (Manual)
+[PASS] 4.2.12 Verify that the RotateKubeletServerCertificate argument is set to true (Manual)
 [PASS] 4.2.13 Ensure that the Kubelet only makes use of Strong Cryptographic Ciphers (Manual)
 
-== Remediations ==
+== Remediations node ==
 4.2.6 If using a Kubelet config file, edit the file to set protectKernelDefaults: true.
 If using command line arguments, edit the kubelet service file
 /etc/systemd/system/kubelet.service.d/10-kubeadm.conf on each worker node and
@@ -271,19 +270,13 @@ Based on your system, restart the kubelet service. For example:
 systemctl daemon-reload
 systemctl restart kubelet.service
 
-4.2.12 Edit the kubelet service file /etc/systemd/system/kubelet.service.d/10-kubeadm.conf
-on each worker node and set the below parameter in KUBELET_CERTIFICATE_ARGS variable.
---feature-gates=RotateKubeletServerCertificate=true
-Based on your system, restart the kubelet service. For example:
-systemctl daemon-reload
-systemctl restart kubelet.service
 
-
+== Summary node ==
-== Summary ==
+20 checks PASS
-19 checks PASS
 1 checks FAIL
-3 checks WARN
+2 checks WARN
 0 checks INFO
+
 [INFO] 5 Kubernetes Policies
 [INFO] 5.1 RBAC and Service Accounts
 [WARN] 5.1.1 Ensure that the cluster-admin role is only used where required (Manual)
@@ -316,7 +309,7 @@ systemctl restart kubelet.service
 [WARN] 5.7.3 Apply Security Context to Your Pods and Containers (Manual)
 [WARN] 5.7.4 The default namespace should not be used (Manual)
 
-== Remediations ==
+== Remediations policies ==
 5.1.1 Identify all clusterrolebindings to the cluster-admin role. Check if they are used and
 if they need this role or if they could use a role with fewer privileges.
 Where possible, first bind users to a lower privileged role and then remove the
@@ -413,8 +406,14 @@ Containers.
 resources and that all new resources are created in a specific namespace.
 
 
-== Summary ==
+== Summary policies ==
 0 checks PASS
 0 checks FAIL
 24 checks WARN
 0 checks INFO
+
+== Summary total ==
+72 checks PASS
+11 checks FAIL
+39 checks WARN
+0 checks INFO