Download of bitbucket-dependency-scanner:0.1.4 terminates with unexpected EOF

[Paul S]

Specific error message:

Status: Downloaded newer image for bitbucketpipelines/bitbucket-dependency-scanner:0.1.4

time="2024-11-19T16:21:25Z" level=error msg="error waiting for container: unexpected EOF"

I tried multiple times. Think these would fail with the memory limit error as well.

Then tried 0.1.3 and it worked (sort of - it then fails of course.):



Status: Downloaded newer image for bitbucketpipelines/bitbucket-dependency-scanner:0.1.3

WARNING: New version available: atlassian/bitbucket-dependency-scanner '0.1.3' to '0.1.4'

Tied 0.1.4 again and this time it seemed to get further, yet still didn't complete. Tried adding 'DEBUG: "true"' and then every run would fail "Container 'docker' exceeded memory limit."
And no log file to be had.

Trying with 0.1.2 which eventually also failed with an EOF error following a docker exceeded memory limit error.

[ERROR] NodeAuditAnalyzer failed on /opt/atlassian/pipelines/agent/build/package-lock.json

[WARN] An error occurred while analyzing '/opt/atlassian/pipelines/agent/build/package-lock.json' (Node Audit Analyzer).

[INFO] Finished Node Audit Analyzer (0 seconds)

[WARN] The Yarn Audit Analyzer has been disabled. Yarn executable was not found.

time="2024-11-19T21:41:41Z" level=error msg="error waiting for container: unexpected EOF"

Finally got a log file with the following error at the end:

2024-11-20 16:19:26,129 org.owasp.dependencycheck.App:216 ERROR - Could not perform Node Audit analysis. Invalid payload submitted to Node Audit API. 2024-11-20 16:19:26,129 org.owasp.dependencycheck.App:217 DEBUG - unexpected error org.owasp.dependencycheck.analyzer.exception.SearchException: Could not perform Node Audit analysis. Invalid payload submitted to Node Audit API. at org.owasp.dependencycheck.data.nodeaudit.NodeAuditSearch.submitPackage(NodeAuditSearch.java:209) at org.owasp.dependencycheck.data.nodeaudit.NodeAuditSearch.submitPackage(NodeAuditSearch.java:133) at org.owasp.dependencycheck.analyzer.NodeAuditAnalyzer.analyzePackage(NodeAuditAnalyzer.java:189) at org.owasp.dependencycheck.analyzer.NodeAuditAnalyzer.analyzeDependency(NodeAuditAnalyzer.java:146) at org.owasp.dependencycheck.analyzer.AbstractAnalyzer.analyze(AbstractAnalyzer.java:131) at org.owasp.dependencycheck.AnalysisTask.call(AnalysisTask.java:88) at org.owasp.dependencycheck.AnalysisTask.call(AnalysisTask.java:37) at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:317) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642) at java.base/java.lang.Thread.run(Thread.java:1570) 2024-11-20 16:19:26,130 org.owasp.dependencycheck.utils.Settings:890 DEBUG - Deleting ALL temporary files from `/tmp/dctemp0ba92bc5-01f2-458d-a6f0-bbb48e95f856` 2024-11-20 16:19:26,134 org.owasp.dependencycheck.App:90 DEBUG - Exit code: 14



Today trying again with 1.0.2 it keeps failing (exceeded memory limit):

WARNING: New version available: atlassian/bitbucket-dependency-scanner '0.1.2' to '0.1.4' INFO: Writing Dependency Scanner verbose logging information to ./dependency-check-report-73.log DEBUG: dependency-check generated command: /usr/share/dependency-check/bin/dependency-check.sh --enableExperimental --format=JSON --prettyPrint --scan=. --nvdApiKey=$NVDAPIKEY --out=. --format=HTML --log=./dependency-check-report-73.log [INFO] Checking for updates [INFO] NVD API has 10,756 records in this update [INFO] Downloaded 10,000/10,756 (93%)

[Oleksandr Kyrdan]

Hi @Paul S 

Happy to know that the issue was solved.

The new version of the pipe is released:

- pipe: atlassian/bitbucket-dependency-scanner:0.2.0
  variables:
    NVD_API_KEY: $NVD_API_KEY
    UPDATE_NVD_CACHE: 'true'

Best regards,
Oleksandr Kyrdan

[Oleksandr Kyrdan]

Hi @Paul S 

Thank you for your question!

Could you check and provide a screenshot of the Metrics tab for the build step, please?

You could try to increase a build step memory to solve the memory limit issue, please, check this good answer.

Also, you could follow the guide with details on how memory is allocated in Pipelines Databases and service containers documentation.

Best regards,
Oleksandr Kyrdan

[Paul S]

There is no Metrics tab :( 

I've already tried bumping memory up, although I haven't configured the docker service 'per step' atm

[Paul S]

So tried some more options:

options:
  services:
    docker:
    memory: 40960
  clone:
    depth: full
  caches:
    - docker
    - node

- step: &bb-audit
    name: Dependency Scanner
    services:
      - docker
    size: 8x
    caches:
      - docker
      - node
    script:
      - npm install --legacy-peer-deps
      - pipe: atlassian/bitbucket-dependency-scanner:0.1.2
        variables:
          NVD_API_KEY: $NVDAPIKEY
          EXTRA_ARGS:
            - "--format=HTML"
            - "--nvdApiDelay=20000"
            - "--nvdMaxRetryCount=10"
    artifacts:
      - dependency-check-report-*.log
      - dependency-check-report.json
      - dependency-check-report.html

Now it timed out after 2 hours!

[Oleksandr Kyrdan]

Hi @Paul S 

Thanks for sharing this.

Looks like the issue on the NVD API side.
Dependency checker tries to update NVD databases before the scanner execution.

We are investigating this unexpected behaviour and will try to find a solution to prevent this.

Best regards,
Oleksandr Kyrdan

[Oleksandr Kyrdan]

Hi @Paul S 

Could you try the dev version with upgraded base docker image to the latest released? 

The new version already contains nvd database in the new upgraded H2 database format that is not compatible with an old one:

- pipe: docker://bitbucketpipelines/bitbucket-dependency-scanner:0.1.4.52-dev

Best regards,
Oleksandr Kyrdan

[Andrej Ograjenšek]

@Oleksandr Kyrdan I had the same issue as @Paul S and the new version resolved it. One thing I get is a warning:

WARNING: Skip to create annotations because of HTTP error.400 Client Error: Bad Request for url...

Since I'm setting up this pipe for the first time I'm not sure if this is a misconfiguration on my part or it's something in the new version.

[Paul S]

For me it is failing with "Container 'docker' exceeded memory limit." with the config as I had it yesterday.

An earlier run where I had halved nvdApiDelay and nvdMaxRetryCount also managed to output "time="2024-11-26T13:59:38Z" level=error msg="error waiting for container: unexpected EOF""

[Oleksandr Kyrdan]

Hi @Paul S 

Could you try dev version with force NVD cache update?

- pipe: docker://bitbucketpipelines/bitbucket-dependency-scanner:0.1.4.59-dev
  variables:
    NVD_API_KEY: $NVD_API_KEY
    UPDATE_NVD_CACHE: 'true'

as for memory issue try to set up the next:

definitions:
  services:
    docker:
      memory: 2048

options:
  size: 4x

According to investigations, NVD DB update via API takes ~35 min and docker build container memory usage grows up to 3550MB.

Update the cache takes ~1-3 min.

[Paul S]

A bit confusing when it fails with "✖ Dependency scan failed. The result status code: 14", but I now get log file and report.

From the log file:

ERROR - Could not perform Node Audit analysis. Invalid payload submitted to Node Audit API.
2024-11-28 13:53:51,838 org.owasp.dependencycheck.App:221
DEBUG - unexpected error
org.owasp.dependencycheck.analyzer.exception.SearchException: Could not perform Node Audit analysis. Invalid payload submitted to Node Audit API.
at org.owasp.dependencycheck.data.nodeaudit.NodeAuditSearch.submitPackage(NodeAuditSearch.java:199)
at org.owasp.dependencycheck.data.nodeaudit.NodeAuditSearch.submitPackage(NodeAuditSearch.java:135)
at org.owasp.dependencycheck.analyzer.NodeAuditAnalyzer.analyzePackage(NodeAuditAnalyzer.java:189)
at org.owasp.dependencycheck.analyzer.NodeAuditAnalyzer.analyzeDependency(NodeAuditAnalyzer.java:146)
at org.owasp.dependencycheck.analyzer.AbstractAnalyzer.analyze(AbstractAnalyzer.java:131)
at org.owasp.dependencycheck.AnalysisTask.call(AnalysisTask.java:88)
at org.owasp.dependencycheck.AnalysisTask.call(AnalysisTask.java:37)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:317)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
at java.base/java.lang.Thread.run(Thread.java:1583)
Caused by: org.owasp.dependencycheck.utils.DownloadFailedException: https://registry.npmjs.org/-/npm/v1/security/audits - Server status: 400 - Server reason: Bad Request
at org.owasp.dependencycheck.utils.Downloader.wrapAndThrowHttpResponseException(Downloader.java:346)
at org.owasp.dependencycheck.utils.Downloader.postBasedFetchContent(Downloader.java:440)
at org.owasp.dependencycheck.data.nodeaudit.NodeAuditSearch.submitPackage(NodeAuditSearch.java:165)
... 10 common frames omitted
Caused by: org.apache.hc.client5.http.HttpResponseException: status code: 400, reason phrase: Bad Request
at org.apache.hc.client5.http.impl.classic.AbstractHttpClientResponseHandler.handleResponse(AbstractHttpClientResponseHandler.java:69)
at org.apache.hc.client5.http.impl.classic.BasicHttpClientResponseHandler.handleResponse(BasicHttpClientResponseHandler.java:72)
at org.apache.hc.client5.http.impl.classic.BasicHttpClientResponseHandler.handleResponse(BasicHttpClientResponseHandler.java:55)
at org.apache.hc.client5.http.impl.classic.CloseableHttpClient.execute(CloseableHttpClient.java:247)
at org.apache.hc.client5.http.impl.classic.CloseableHttpClient.execute(CloseableHttpClient.java:188)
at org.apache.hc.client5.http.impl.classic.CloseableHttpClient.execute(CloseableHttpClient.java:162)
at org.owasp.dependencycheck.utils.Downloader.postBasedFetchContent(Downloader.java:435)
... 11 common frames omitted
2024-11-28 13:53:51,838 org.owasp.dependencycheck.utils.Settings:920
DEBUG - Deleting ALL temporary files from `/tmp/dctemp21832184-f640-4572-9907-86a3fcdf6da2`
2024-11-28 13:53:51,848 org.owasp.dependencycheck.App:91
DEBUG - Exit code: 14

[Oleksandr Kyrdan]

@Paul S 

It would be nice if you could explain without sensitive details what the type of project and dependencies you try to scan?

[Paul S]

To be clear it DOES complete a scan and produces report files (and a log file in debug.)

Library of typescript (*.ts/*.tsx source files) React components, project incorporating a .storybook and a bunch of config files - e.g. .node-version, .npm-ignore, .npmrc, .jest.config.js, .jest.setup.js, package.json, package-lock.json .postinstall.js, rollup.config.ts, tsconfig.jest.json, tsconfig.json, tslint.json

Folders with json resources, and there should be a node_modules folder.

I'm caching the docker and node images.

It's a project that was on Bamboo and migrated to BitBucket

[Oleksandr Kyrdan]

@Paul S 

Thanks for sharing this details, trying to understand and find a way to improve the solution.

Best regards,
Oleksandr Kyrdan

[Paul S]

Still failing with "Container 'docker' exceeded memory limit." - I'm sure I didn't change anything significant. :( 

[Paul S]

And then, just now it ran almost fully and I have at least got a log file. (And then further runs failed again.)

BTW it's no longer an issue with NVD cache:

INFO: Updating NVD Cache...

INFO: Writing Dependency Scanner verbose logging information to ./dependency-check-report-164.log

The build tab last logged:

[INFO] Writing HTML report to: /opt/atlassian/pipelines/agent/build/./dependency-check-report.html

[ERROR] Could not perform Node Audit analysis. Invalid payload submitted to Node Audit API.

@Oleksandr Kyrdan if you can contact me by email I can send you the log file.

[Oleksandr Kyrdan]

@Paul S 

[ERROR] Could not perform Node Audit analysis. Invalid payload submitted to Node Audit API.

This issue relates to the dependency-check tool and NodeJS setup.
Please, follow the next question or issues 2641, 3716, 3717  for dependency-check devs.

[Paul S]

If I nuke node_modules and package-lock.json it doesn't blow up. But then it doesn't actually check anything nor report any CVEs.

Otherwise, when it fails it terminates abruptly and I don't even get a log file so can't debug what's happening. Like there's an uncaught exception.

Trying to re-create them with 'npm install' the dependency check fails - but no log file to debug why.