Configuring Git secrets scan

[David Keaveny]

I'm trying to integrate the AWS Git secrets scan pipe in my pipeline (https://bitbucket.org/atlassian/git-secrets-scan/src/master/); however, I can't get the `FILES` filter to work, and the documentation doesn't give much help.

My simple pipeline builds a .NET Core 2.2 application, and checks for hardcoded AWS secrets in the source code:

image: mcr.microsoft.com/dotnet/core/sdk:2.2

pipelines: 
  default: 
    - step: 
        name: Build 
        caches:
          - dotnetcore
        script:
          - dotnet build 
    - step:
        name: Analyse
        script:
          - pipe: atlassian/git-secrets-scan:0.4.1
            variables:
              FILES: '*.cs'

However, when the pipeline runs, I see the following:

INFO: Executing the pipe...Traceback (most recent call last):  File "/pipe.py", line 112, in <module>    main()  File "/pipe.py", line 104, in main    raise Exception(result.stderr)Exception: grep: *.cs: No such file or directory

If I remove the FILES setting (so it scans all files) then the pipeline runs correctly, although the Git secrets pipe fails with false positives on a project file (which is why I wanted the file filter in the first place).

Have I set up the pipe correctly? And how can I add other file types to the filter expression?

[Halyna Berezovska · Accepted solution]

@David Keaveny thanks for using the pipe!

I checked git secrets scan and such error is appearing when you do not have files with this extension.

For example, at the snippet below  command scanning txt files gives nothing, so it is all right and searching for *.cs files does not work, because such do not exist

$ ls test.txt 
test.txt

$ git secrets --scan *.txt
$ git secrets --scan *.cs
grep: *.cs: No such file or directory
$ ls *.cs
ls: *.cs: No such file or directory

If you propose to catch this , to ensure that nothing sensitive does not appear during the build (when such files could be created), we can discuss it as a proposition for the future improvement.

For now I just would like to understand your case

[David Keaveny]

Hi Galyna,

Thanks for your response!

The *.cs files will be in subfolders of the project root, so I'm guessing that it's not searching recursively?

As a side question, if I wanted to search *.cs and *.json files, would I configure it as:

FILES: '*.cs;*.json'

?

[Halyna Berezovska]

@David Keaveny hello, we released new version of git-secrets-scan 0.4.3, there is a fix of wildcard issue, indeed , subprocess tool did not work with wildcard *.

Now it should work in 0.4.3 and your example

FILES: '*.cs;*.json'

also will work, but you need specify this variable like this:

FILES: '*.cs *.json'

 without semicolon.

Regards, Galyna

[David Keaveny]

Hi @Halyna Berezovska ,

I gave the 0.4.3 image a go, and it seems to be ignoring the `FILES` parameter, as it is giving a false positive on a .DotSettings file. My configuration is now:

- step:
    name: Analyse
    script:
      - pipe: atlassian/git-secrets-scan:0.4.3
        variables:
          FILES: '*.cs *.json'

[Halyna Berezovska]

@David Keaveny yes, this is because you are trying to find files in no-root directories.

In python glob that we use there is such specific to search recursively:

script:
  - pipe: atlassian/git-secrets-scan:0.4.3
    variables:
      FILES: "**/*.txt **/*.json"

 This is not obvious, see how it should work in our doc: Git Secrets Scan, Advanced examples .

Also, check your dot settings extension. If you suspect sensitive string to be found in such files as *.DotSettings , include this, too.

[David Keaveny]

Ah yes, that fixed it, many thanks!