Skip to content

Commit 59f059d

Browse files
shissamraghavkaul
authored andcommitted
✨ Improved Security Policy Check (ossf#2195)
* ✨ Improved Security Policy Check (ossf#2137) * Examines and awards points for linked content (URLs / Emails) * Examines and awards points for hints of disclosure and vulnerability practices * Examines and awards points for hints of elaboration of timelines Signed-off-by: Scott Hissam <[email protected]> * Repaired Security Policy to correctly use linked content length for evaluation Signed-off-by: Scott Hissam <[email protected]> * gofmt'ed changes Signed-off-by: Scott Hissam <[email protected]> * Repaired the case in the evaluation which was too sensitive to content length over the length of the linked content for urls and emails Signed-off-by: Scott Hissam <[email protected]> * added unit test cases for the new content-based Security Policy checks Signed-off-by: Scott Hissam <[email protected]> * reverted the direct (mistaken) change to checks.md and updated the checks.yaml for generate-docs Signed-off-by: Scott Hissam <[email protected]> * ✨ Improved Security Policy Check (ossf#2137) (revisted based on comments) * replaced reason strings with log.Info & log.Warn (as seen in --show-details) * internal assertion check for nil (*pinfo) and empty pfile * internal switched to FileTypeText over FileTypeSource * internal implement type SecurityPolicyInformationType/SecurityPolicyInformation revised SecurityPolicyData to support only one file * revised expected unit-test results and revised unit-test to reflect the new SecurityPolicyData type Signed-off-by: Scott Hissam <[email protected]> * revised the score value based on observation of one *or more* url(s) or one email(s) found; unit tests update accordingly Signed-off-by: Scott Hissam <[email protected]> * revised the score value based on observation of one *or more* url(s) or one email(s) found; unit tests update accordingly Signed-off-by: Scott Hissam <[email protected]> * revised the score value based on observation of one *or more* url(s) or one email(s) found; e2e tests update accordingly Signed-off-by: Scott Hissam <[email protected]> * Addressed PR comments; added telemetry for policy hits in security policy file to track hits by line number Signed-off-by: Scott Hissam <[email protected]> * Resolved merge conflict with checks.yaml Signed-off-by: Scott Hissam <[email protected]> * updated raw results to emit all the raw information for the new security policy check Signed-off-by: Scott Hissam <[email protected]> * Resolved merge conflicts and lint errors with json_raw_results.go Signed-off-by: Scott Hissam <[email protected]> * Addressed review comments to reorganize security policy data struct to support the potential for multiple security policy files. Signed-off-by: Scott Hissam <[email protected]> * Added logic to the security policy to process multiple security policy files only after future improvements to aggregating scoring across such files are designed. For now the security policy behaves as originally designed to stop once one of the expected policy files are found in the repo Signed-off-by: Scott Hissam <[email protected]> * added comments regarding the capacity to support multiple policy files and removed unneeded break statements in the code Signed-off-by: Scott Hissam <[email protected]> * Addressed review comments to remove the dependency on the path in the filename from the code and introduced FileSize to checker.File type and removed the SecurityContentLength which was used to hold that information for the new security policy assessment Signed-off-by: Scott Hissam <[email protected]> * restored reporting full security policy path and filename for policies found in the org level repos Signed-off-by: Scott Hissam <[email protected]> * Resolved conflicts in checks.yaml for documentation Signed-off-by: Scott Hissam <[email protected]> * ✨ CLI for scorecard-attestor (ossf#2309) * Reorganize Signed-off-by: Raghav Kaul <[email protected]> * Working commit Signed-off-by: Raghav Kaul <[email protected]> * Compile with local scorecard; go mod tidy Signed-off-by: Raghav Kaul <[email protected]> * Add signing code Heavily borrowed from https://github.com/grafeas/kritis/blob/master/cmd/kritis/signer/main.go Signed-off-by: Raghav Kaul <[email protected]> * Update deps * Naming * Makefile Signed-off-by: Raghav Kaul <[email protected]> * Edit license, add lint.yml Signed-off-by: Raghav Kaul <[email protected]> * checks: go mod tidy, license Signed-off-by: Raghav Kaul <[email protected]> * Address PR comments * Split into checker/signer files * Naming convention Signed-off-by: Raghav Kaul <[email protected]> * License, remove golangci.yml Signed-off-by: Raghav Kaul <[email protected]> * Address PR comments * Use cobra Signed-off-by: Raghav Kaul <[email protected]> * Add tests for root command Signed-off-by: Raghav Kaul <[email protected]> * Filter out checks that aren't needed for policy evaluation Signed-off-by: Raghav Kaul <[email protected]> * Add `make` targets for attestor; submit coverage stats Signed-off-by: Raghav Kaul <[email protected]> * Improvements * Use sclog instead of glog * Remove unneeded subcommands * Formatting Signed-off-by: Raghav Kaul <[email protected]> * Flags: Make note-name constant and fix messaging Signed-off-by: Raghav Kaul <[email protected]> * Remove SupportedRequestTypes Signed-off-by: Raghav Kaul <[email protected]> * go mod tidy Signed-off-by: Raghav Kaul <[email protected]> * go mod tidy, makefile Signed-off-by: Raghav Kaul <[email protected]> * Fix GH actions run Signed-off-by: Raghav Kaul <[email protected]> Signed-off-by: Raghav Kaul <[email protected]> Signed-off-by: Scott Hissam <[email protected]> * removed whitespace before stanza for Run attestor e2e Signed-off-by: Scott Hissam <[email protected]> * resolved code review and doc review comments Signed-off-by: Scott Hissam <[email protected]> * repaired the link for the maintainer's guide for supporting the coordinated vulnerability disclosure guidelines Signed-off-by: Scott Hissam <[email protected]> Signed-off-by: Scott Hissam <[email protected]>
1 parent 9799e6c commit 59f059d

24 files changed

Lines changed: 517 additions & 64 deletions

.github/workflows/integration.yml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -76,7 +76,7 @@ jobs:
7676
retry_on: error
7777
timeout_minutes: 30
7878
command: make e2e-pat
79-
79+
8080
- name: Run attestor e2e #using retry because the GitHub token is being throttled.
8181
uses: nick-invision/retry@3e91a01664abd3c5cd539100d10d33b9c5b68482
8282
env:

checker/raw_result.go

Lines changed: 29 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -163,11 +163,37 @@ type VulnerabilitiesData struct {
163163
Vulnerabilities []clients.Vulnerability
164164
}
165165

166+
type SecurityPolicyInformationType string
167+
168+
const (
169+
// forms of security policy hints being evaluated.
170+
SecurityPolicyInformationTypeEmail SecurityPolicyInformationType = "emailAddress"
171+
SecurityPolicyInformationTypeLink SecurityPolicyInformationType = "httpLink"
172+
SecurityPolicyInformationTypeText SecurityPolicyInformationType = "vulnDisclosureText"
173+
)
174+
175+
type SecurityPolicyValueType struct {
176+
Match string // Snippet of match
177+
LineNumber uint // Line number in policy file of match
178+
Offset uint // Offset in the line of the match
179+
}
180+
181+
type SecurityPolicyInformation struct {
182+
InformationType SecurityPolicyInformationType
183+
InformationValue SecurityPolicyValueType
184+
}
185+
186+
type SecurityPolicyFile struct {
187+
// security policy information found in repo or org
188+
Information []SecurityPolicyInformation
189+
// file that contains the security policy information
190+
File File
191+
}
192+
166193
// SecurityPolicyData contains the raw results
167194
// for the Security-Policy check.
168195
type SecurityPolicyData struct {
169-
// Files contains a list of files.
170-
Files []File
196+
PolicyFiles []SecurityPolicyFile
171197
}
172198

173199
// BinaryArtifactData contains the raw results
@@ -236,6 +262,7 @@ type File struct {
236262
Snippet string // Snippet of code
237263
Offset uint // Offset in the file of Path (line for source/text files).
238264
EndOffset uint // End of offset in the file, e.g. if the command spans multiple lines.
265+
FileSize uint // Total size of file.
239266
Type FileType // Type of file.
240267
// TODO: add hash.
241268
}

checks/evaluation/security_policy.go

Lines changed: 113 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -19,6 +19,104 @@ import (
1919
sce "github.com/ossf/scorecard/v4/errors"
2020
)
2121

22+
func scoreSecurityCriteria(f checker.File,
23+
info []checker.SecurityPolicyInformation,
24+
dl checker.DetailLogger,
25+
) int {
26+
var urls, emails, discvuls, linkedContentLen, score int
27+
28+
emails = countSecInfo(info, checker.SecurityPolicyInformationTypeEmail, true)
29+
urls = countSecInfo(info, checker.SecurityPolicyInformationTypeLink, true)
30+
discvuls = countSecInfo(info, checker.SecurityPolicyInformationTypeText, false)
31+
32+
for _, i := range findSecInfo(info, checker.SecurityPolicyInformationTypeEmail, true) {
33+
linkedContentLen += len(i.InformationValue.Match)
34+
}
35+
for _, i := range findSecInfo(info, checker.SecurityPolicyInformationTypeLink, true) {
36+
linkedContentLen += len(i.InformationValue.Match)
37+
}
38+
39+
msg := checker.LogMessage{
40+
Path: f.Path,
41+
Type: f.Type,
42+
Text: "",
43+
}
44+
45+
// #1: linked content found (email/http): score += 6
46+
if (urls + emails) > 0 {
47+
score += 6
48+
msg.Text = "Found linked content in security policy"
49+
dl.Info(&msg)
50+
} else {
51+
msg.Text = "no email or URL found in security policy"
52+
dl.Warn(&msg)
53+
}
54+
55+
// #2: more bytes than the sum of the length of all the linked content found: score += 3
56+
// rationale: there appears to be information and context around those links
57+
// no credit if there is just a link to a site or an email address (those given above)
58+
// the test here is that each piece of linked content will likely contain a space
59+
// before and after the content (hence the two multiplier)
60+
if f.FileSize > 1 && (f.FileSize > uint(linkedContentLen+((urls+emails)*2))) {
61+
score += 3
62+
msg.Text = "Found text in security policy"
63+
dl.Info(&msg)
64+
} else {
65+
msg.Text = "No text (beyond any linked content) found in security policy"
66+
dl.Warn(&msg)
67+
}
68+
69+
// #3: found whole number(s) and or match(es) to "Disclos" and or "Vuln": score += 1
70+
// rationale: works towards the intent of the security policy file
71+
// regarding whom to contact about vuls and disclosures and timing
72+
// e.g., we'll disclose, report a vulnerabily, 30 days, etc.
73+
// looking for at least 2 hits
74+
if discvuls > 1 {
75+
score += 1
76+
msg.Text = "Found disclosure, vulnerability, and/or timelines in security policy"
77+
dl.Info(&msg)
78+
} else {
79+
msg.Text = "One or no descriptive hints of disclosure, vulnerability, and/or timelines in security policy"
80+
dl.Warn(&msg)
81+
}
82+
83+
return score
84+
}
85+
86+
func countSecInfo(secInfo []checker.SecurityPolicyInformation,
87+
infoType checker.SecurityPolicyInformationType,
88+
unique bool,
89+
) int {
90+
keys := make(map[string]bool)
91+
count := 0
92+
for _, entry := range secInfo {
93+
if _, present := keys[entry.InformationValue.Match]; !present && entry.InformationType == infoType {
94+
keys[entry.InformationValue.Match] = true
95+
count += 1
96+
} else if !unique && entry.InformationType == infoType {
97+
count += 1
98+
}
99+
}
100+
return count
101+
}
102+
103+
func findSecInfo(secInfo []checker.SecurityPolicyInformation,
104+
infoType checker.SecurityPolicyInformationType,
105+
unique bool,
106+
) []checker.SecurityPolicyInformation {
107+
keys := make(map[string]bool)
108+
var secList []checker.SecurityPolicyInformation
109+
for _, entry := range secInfo {
110+
if _, present := keys[entry.InformationValue.Match]; !present && entry.InformationType == infoType {
111+
keys[entry.InformationValue.Match] = true
112+
secList = append(secList, entry)
113+
} else if !unique && entry.InformationType == infoType {
114+
secList = append(secList, entry)
115+
}
116+
}
117+
return secList
118+
}
119+
22120
// SecurityPolicy applies the score policy for the Security-Policy check.
23121
func SecurityPolicy(name string, dl checker.DetailLogger, r *checker.SecurityPolicyData) checker.CheckResult {
24122
if r == nil {
@@ -27,23 +125,31 @@ func SecurityPolicy(name string, dl checker.DetailLogger, r *checker.SecurityPol
27125
}
28126

29127
// Apply the policy evaluation.
30-
if r.Files == nil || len(r.Files) == 0 {
31-
// If the file is null or has zero lengths, directly return as not detected.
128+
if len(r.PolicyFiles) == 0 {
129+
// If the file is unset, directly return as not detected.
32130
return checker.CreateMinScoreResult(name, "security policy file not detected")
33131
}
34132

35-
for _, f := range r.Files {
133+
// TODO: although this a loop, the raw checks will only return one security policy
134+
// when more than one security policy file can be aggregated into a composite
135+
// score, that logic can be comprehended here.
136+
score := 0
137+
for _, spd := range r.PolicyFiles {
138+
score = scoreSecurityCriteria(spd.File,
139+
spd.Information, dl)
140+
36141
msg := checker.LogMessage{
37-
Path: f.Path,
38-
Type: f.Type,
39-
Offset: f.Offset,
142+
Path: spd.File.Path,
143+
Type: spd.File.Type,
40144
}
41145
if msg.Type == checker.FileTypeURL {
42146
msg.Text = "security policy detected in org repo"
43147
} else {
44148
msg.Text = "security policy detected in current repo"
45149
}
150+
46151
dl.Info(&msg)
47152
}
48-
return checker.CreateMaxScoreResult(name, "security policy file detected")
153+
154+
return checker.CreateResultWithScore(name, "security policy file detected", score)
49155
}

checks/evaluation/security_policy_test.go

Lines changed: 10 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -59,32 +59,34 @@ func TestSecurityPolicy(t *testing.T) {
5959
args: args{
6060
name: "test_security_policy_3",
6161
r: &checker.SecurityPolicyData{
62-
Files: []checker.File{
63-
{
62+
PolicyFiles: []checker.SecurityPolicyFile{{
63+
File: checker.File{
6464
Path: "/etc/security/pam_env.conf",
6565
Type: checker.FileTypeURL,
6666
},
67+
Information: make([]checker.SecurityPolicyInformation, 0),
6768
},
68-
},
69+
}},
6970
},
7071
want: checker.CheckResult{
71-
Score: 10,
72+
Score: 0,
7273
},
7374
},
7475
{
7576
name: "test_security_policy_4",
7677
args: args{
7778
name: "test_security_policy_4",
7879
r: &checker.SecurityPolicyData{
79-
Files: []checker.File{
80-
{
80+
PolicyFiles: []checker.SecurityPolicyFile{{
81+
File: checker.File{
8182
Path: "/etc/security/pam_env.conf",
8283
},
84+
Information: make([]checker.SecurityPolicyInformation, 0),
8385
},
84-
},
86+
}},
8587
},
8688
want: checker.CheckResult{
87-
Score: 10,
89+
Score: 0,
8890
},
8991
},
9092
}

0 commit comments

Comments
 (0)