Anthropic 400 error

[the2dl]

Hey,

When I run it, it works at first:

[2024-12-05T12:55:02Z INFO ] step:1
[2024-12-05T12:55:05Z INFO ] 🧠 read_file(/root/pyShares/index.ts) -> 54622 bytes in 16.1675ms
[2024-12-05T12:55:05Z INFO ] step:2 responses:1 actions:1 token_usage(in:1492 out:104 tot_in:1492 tot_out:104)
[2024-12-05T12:55:11Z INFO ] 🧠 report_finding({
      "title": "Hardcoded JWT Secret",
      "severity": "HIGH",
      "impact": "If the JWT secret is compromised, an attacker could forge valid JWT tokens and gain unauthorized access to the application.",
      "description": "The application uses a hardcoded JWT secret as a fallback when the environment variable is not set. This practice is insecure and can lead to unauthorized access if the secret is exposed.",
      "evidence": "const JWT_SECRET = process.env.JWT_SECRET || 'your-jwt-secret-key';",
      "file": "/root/pyShares/index.ts",
      "proof": "The code uses a hardcoded string 'your-jwt-secret-key' as a fallback for the JWT secret. This secret should never be hardcoded or committed to version control."
    }) no output in 525.541s

But then eventually crashes out:

[2024-12-05T12:55:53Z ERROR] failed to send chat message: API error: (400 Bad Request) invalid_request_error: {
      "type": "error",
      "error": {
        "type": "invalid_request_error",
        "message": "Your API request included an `assistant` message in the final position, which would pre-fill the `assistant` response. When using tools, pre-filling the `assistant` response is not supported."
      }
    }
Error: failed to send chat message: API error: (400 Bad Request) invalid_request_error: {
  "type": "error",
  "error": {
    "type": "invalid_request_error",
    "message": "Your API request included an `assistant` message in the final position, which would pre-fill the `assistant` response. When using tools, pre-filling the `assistant` response is not supported."
  }
}

Seems fine when using ChatGPT, guessing it's sending assistant across in the API request at some point?

[evilsocket]

oh that's interesting because there's an explicit workaround for that here, clearly not working :D Can you tell me which version of nerve are you using, which model, etc? It'd be useful if you copy/paste here the very first line nerve prints

[the2dl]

Here ya go, it grabbed the latest image this morning and ran it - -G "anthropic://claude"

Unable to find image 'evilsocket/nerve:latest' locally
latest: Pulling from evilsocket/nerve
e357e1f94476: Already exists
4c026eb2764a: Pull complete
cfc72b85b045: Pull complete
Digest: sha256:33f21429074d2b7a54a151b18eaf12c5ddd224f225029ebe691d84069e10c3df
Status: Downloaded newer image for evilsocket/nerve:latest
nerve v0.5.0 🧠 claude@anthropic > code_auditor (last 15 messages)

[2024-12-05T12:55:02Z INFO ] step:1

[evilsocket]

Appreciate the help! I just pushed a fix that might be related to this issue, whenever you have a moment would you be so kind pull latest and try again? Thanks!

[the2dl]

Cool - so that error is gone, but I get some other strange behavior (only with using Anthropic, GPT is fine), check this out:

It seems to get stuck in a loop and reports the same thing ~10x then moves on to the next finding. With GPT it just does it once and finishes.. anthropic is trouble 🗡️

docker run -it --network=host \
  -v ./examples:/root/.nerve/tasklets \
  -v /Users/dan/Documents/security-sketch-chat:/root/security-sketch-chat \
  -v /Users/dan/Documents/pyShares/sharesFront/backend/src:/root/pyShares \
  -e ANTHROPIC_API_KEY=redacted \
  evilsocket/nerve \
  -G "anthropic://claude" -T code_auditor -DTARGET_PATH=/root/pyShares/index.ts
Unable to find image 'evilsocket/nerve:latest' locally
latest: Pulling from evilsocket/nerve
e357e1f94476: Already exists
4a2f5e8a0c17: Pull complete
cc7a6fa594a5: Pull complete
Digest: sha256:7cb99d5235ce3867e93b883b259beaf92ea61d5b6d5fd014cc8c1ee84a06789c
Status: Downloaded newer image for evilsocket/nerve:latest
nerve v0.5.0 🧠 claude@anthropic > code_auditor (last 15 messages)

[2024-12-06T17:12:39Z INFO ] step:1
[2024-12-06T17:12:41Z INFO ] 🧠 read_file(/root/pyShares/index.ts) -> 56897 bytes in 30.643417ms
[2024-12-06T17:12:41Z INFO ] step:2 responses:1 actions:1 token_usage(in:1492 out:104 tot_in:1492 tot_out:104)
[2024-12-06T17:12:48Z INFO ] 🧠 report_finding({
      "title": "Hardcoded JWT Secret",
      "severity": "HIGH",
      "impact": "If the JWT secret is compromised, an attacker could forge valid JWT tokens and gain unauthorized access to the application.",
      "description": "The JWT secret is hardcoded in the source code. This is a security risk as it could be exposed if the source code is leaked or accessed by unauthorized individuals.",
      "evidence": "const JWT_SECRET = process.env.JWT_SECRET || 'your-jwt-secret-key';",
      "file": "/root/pyShares/index.ts",
      "proof": "The JWT_SECRET is set to a default value 'your-jwt-secret-key' if the environment variable is not set. This default value should not be present in the source code."
    }) no output in 211.458µs
[2024-12-06T17:12:48Z INFO ] step:3 responses:2 actions:2 token_usage(in:18638 out:280 tot_in:20130 tot_out:384)
[2024-12-06T17:12:54Z INFO ] 🧠 report_finding({
      "title": "Hardcoded JWT Secret",
      "severity": "HIGH",
      "impact": "If the JWT secret is compromised, an attacker could forge valid JWT tokens and gain unauthorized access to the application.",
      "description": "The JWT secret is hardcoded in the source code. This is a security risk as it could be exposed if the source code is leaked or accessed by unauthorized individuals.",
      "evidence": "const JWT_SECRET = process.env.JWT_SECRET || 'your-jwt-secret-key';",
      "file": "/root/pyShares/index.ts",
      "proof": "The JWT_SECRET is set to a default value 'your-jwt-secret-key' if the environment variable is not set. This default value should not be present in the source code."
    }) no output in 154µs
[2024-12-06T17:12:54Z INFO ] step:4 responses:3 actions:3 token_usage(in:18840 out:271 tot_in:38970 tot_out:655)
[2024-12-06T17:13:00Z INFO ] 🧠 report_finding({
      "title": "Hardcoded JWT Secret",
      "severity": "HIGH",
      "impact": "If the JWT secret is compromised, an attacker could forge valid JWT tokens and gain unauthorized access to the application.",
      "description": "The JWT secret is set to a default value if the environment variable is not set. This default value should not be present in the source code.",
      "evidence": "const JWT_SECRET = process.env.JWT_SECRET || 'your-jwt-secret-key';",
      "file": "/root/pyShares/index.ts",
      "proof": "The JWT_SECRET is set to a default value 'your-jwt-secret-key' if the environment variable is not set. This default value should not be present in the source code."
    }) no output in 147.834µs
[2024-12-06T17:13:00Z INFO ] step:5 responses:4 actions:4 token_usage(in:19042 out:267 tot_in:58012 tot_out:922)
[2024-12-06T17:13:07Z INFO ] 🧠 report_finding({
      "title": "Hardcoded JWT Secret",
      "severity": "HIGH",
      "impact": "If the JWT secret is compromised, an attacker could forge valid JWT tokens and gain unauthorized access to the application.",
      "description": "The JWT secret is set to a default value if the environment variable is not set. This default value should not be present in the source code.",
      "evidence": "const JWT_SECRET = process.env.JWT_SECRET || 'your-jwt-secret-key';",
      "file": "/root/pyShares/index.ts",
      "proof": "The JWT_SECRET is set to a default value 'your-jwt-secret-key' if the environment variable is not set. This default value should not be present in the source code."
    }) no output in 701.334µs
[2024-12-06T17:13:07Z INFO ] step:6 responses:5 actions:5 token_usage(in:19239 out:273 tot_in:77251 tot_out:1195)
[2024-12-06T17:13:13Z INFO ] 🧠 report_finding({
      "title": "Hardcoded JWT Secret",
      "severity": "HIGH",
      "impact": "If the JWT secret is compromised, an attacker could forge valid JWT tokens and gain unauthorized access to the application.",
      "description": "The JWT secret is set to a default value if the environment variable is not set. This default value should not be present in the source code.",
      "evidence": "const JWT_SECRET = process.env.JWT_SECRET || 'your-jwt-secret-key';",
      "file": "/root/pyShares/index.ts",
      "proof": "The JWT_SECRET is set to a default value 'your-jwt-secret-key' if the environment variable is not set. This default value should not be present in the source code."
    }) no output in 129.5µs
[2024-12-06T17:13:13Z INFO ] step:7 responses:6 actions:6 token_usage(in:19436 out:269 tot_in:96687 tot_out:1464)
[2024-12-06T17:13:19Z INFO ] 🧠 report_finding({
      "title": "Hardcoded JWT Secret",
      "severity": "HIGH",
      "impact": "If the JWT secret is compromised, an attacker could forge valid JWT tokens and gain unauthorized access to the application.",
      "description": "The JWT secret is set to a default value if the environment variable is not set. This default value should not be present in the source code.",
      "evidence": "const JWT_SECRET = process.env.JWT_SECRET || 'your-jwt-secret-key';",
      "file": "/root/pyShares/index.ts",
      "proof": "The JWT_SECRET is set to a default value 'your-jwt-secret-key' if the environment variable is not set. This default value should not be present in the source code."
    }) no output in 134.333µs
[2024-12-06T17:13:19Z INFO ] step:8 responses:7 actions:7 token_usage(in:19633 out:282 tot_in:116320 tot_out:1746)
[2024-12-06T17:13:27Z INFO ] 🧠 report_finding({
      "title": "Hardcoded JWT Secret",
      "severity": "HIGH",
      "impact": "If the JWT secret is compromised, an attacker could forge valid JWT tokens and gain unauthorized access to the application.",
      "description": "The JWT secret is set to a default value if the environment variable is not set. This default value should not be present in the source code.",
      "evidence": "const JWT_SECRET = process.env.JWT_SECRET || 'your-jwt-secret-key';",
      "file": "/root/pyShares/index.ts",
      "proof": "The JWT_SECRET is set to a default value 'your-jwt-secret-key' if the environment variable is not set. This default value should not be present in the source code."
    }) no output in 113.791µs
[2024-12-06T17:13:27Z INFO ] step:9 responses:8 actions:8 token_usage(in:19830 out:282 tot_in:136150 tot_out:2028)
[2024-12-06T17:13:44Z INFO ] 🧠 report_finding(
    {
      "title": "Hardcoded JWT Secret",
      "severity": "HIGH",
      "impact": "If the JWT secret is compromised, an attacker could forge valid JWT tokens and gain unauthorized access to the application.",
      "description": "The JWT secret is set to a default value if the environment variable is not set. This default value should not be present in the source code.",
      "evidence": "const JWT_SECRET = process.env.JWT_SECRET || 'your-jwt-secret-key';",
      "file": "/root/pyShares/index.ts",
      "proof": "The JWT_SECRET is set to a default value 'your-jwt-secret-key' if the environment variable is not set. This default value should not be present in the source code."
    }
    ) no output in 1.107833ms
[2024-12-06T17:13:44Z INFO ] step:10 responses:9 actions:9 token_usage(in:20027 out:845 tot_in:156177 tot_out:2873)
[2024-12-06T17:13:47Z INFO ] 🧠 read_file(/root/pyShares/index.ts) -> 56897 bytes in 18.148542ms
[2024-12-06T17:13:47Z INFO ] step:11 responses:10 actions:10 token_usage(in:2883 out:101 tot_in:159060 tot_out:2974)
[2024-12-06T17:13:52Z INFO ] 🧠 report_finding({
      "title": "SQL Injection Vulnerability",
      "severity": "HIGH",
      "impact": "An attacker could potentially execute arbitrary SQL commands, leading to unauthorized access, data manipulation, or data theft.",
      "description": "The application uses string concatenation to build SQL queries in multiple places, which can lead to SQL injection vulnerabilities if user input is not properly sanitized.",
      "evidence": "const query = `SELECT * FROM ${table} WHERE ${condition}`;",
      "file": "/root/pyShares/index.ts",
      "proof": "Multiple instances of string concatenation in SQL queries were found throughout the file. For example, in the 'compareSessions' function, queries are built using template literals without proper parameterization."
    }) no output in 151.5µs
[2024-12-06T17:13:52Z INFO ] step:12 responses:11 actions:11 token_usage(in:19851 out:204 tot_in:178911 tot_out:3178)
[2024-12-06T17:13:58Z INFO ] 🧠 report_finding({
      "title": "SQL Injection Vulnerability",
      "severity": "HIGH",
      "impact": "An attacker could potentially execute arbitrary SQL commands, leading to unauthorized access, data manipulation, or data theft.",
      "description": "The application uses string concatenation to build SQL queries in multiple places, which can lead to SQL injection vulnerabilities if user input is not properly sanitized.",
      "evidence": "const query = `SELECT * FROM ${table} WHERE ${condition}`;",
      "file": "/root/pyShares/index.ts",
      "proof": "Multiple instances of string concatenation in SQL queries were found throughout the file. For example, in the 'compareSessions' function, queries are built using template literals without proper parameterization."
    }) no output in 160.125µs
[2024-12-06T17:13:58Z INFO ] step:13 responses:12 actions:12 token_usage(in:19842 out:217 tot_in:198753 tot_out:3395)
^C[2024-12-06T17:14:02Z INFO ] 🧠 list_folder_contents(/root/pyShares) -> 203 bytes in 17.527041ms
[2024-12-06T17:14:02Z INFO ] step:14 responses:13 actions:13 token_usage(in:19833 out:96 tot_in:218586 tot_out:3491)
^C^C^C^C^C^C^C^Z^X^Z^C^X^C[2024-12-06T17:14:08Z INFO ] 🧠 report_finding({
      "title": "SQL Injection Vulnerability",
      "severity": "HIGH",
      "impact": "An attacker could potentially execute arbitrary SQL commands, leading to unauthorized access, data manipulation, or data theft.",
      "description": "The application uses string concatenation to build SQL queries in multiple places, which can lead to SQL injection vulnerabilities if user input is not properly sanitized.",
      "evidence": "const query = `SELECT * FROM ${table} WHERE ${condition}`;",
      "file": "/root/pyShares/index.ts",
      "proof": "Multiple instances of string concatenation in SQL queries were found throughout the file. For example, in the 'compareSessions' function, queries are built using template literals without proper parameterization."
    }) no output in 118.583µs
[2024-12-06T17:14:08Z INFO ] step:15 responses:14 actions:14 token_usage(in:19767 out:236 tot_in:238353 tot_out:3727)
^C^C^C^C^C^C[2024-12-06T17:14:15Z INFO ] 🧠 report_finding({
      "title": "SQL Injection Vulnerability",
      "severity": "HIGH",
      "impact": "An attacker could potentially execute arbitrary SQL commands, leading to unauthorized access, data manipulation, or data theft.",
      "description": "The application uses string concatenation to build SQL queries in multiple places, which can lead to SQL injection vulnerabilities if user input is not properly sanitized.",
      "evidence": "const query = `SELECT * FROM ${table} WHERE ${condition}`;",
      "file": "/root/pyShares/index.ts",
      "proof": "Multiple instances of string concatenation in SQL queries were found throughout the file. For example, in the 'compareSessions' function, queries are built using template literals without proper parameterization."
    }) no output in 74.5µs
[2024-12-06T17:14:15Z INFO ] step:16 responses:15 actions:15 token_usage(in:19758 out:225 tot_in:258111 tot_out:3952)
[2024-12-06T17:14:19Z INFO ] 🧠 read_file(/root/pyShares/index.ts) -> 56897 bytes in 962.542µs
[2024-12-06T17:14:19Z INFO ] step:17 responses:16 actions:16 token_usage(in:19749 out:126 tot_in:277860 tot_out:4078)

[evilsocket]

this should be closed by a88f53c