More strict SQL validation

[amoffat]

Hey there, cool project. This prompt constraint caught my eye. It seems you are soft validating the resulting SQL query, which can be risky.

I have written a library to do hard validation, HeimdaLLM. It uses a grammar to parse, validate, and potentially edit the query created from an LLM. It gives you a frontend for rigorously constraining the output query so that it can only perform safe actions. You can read more about the attack surface that it addresses here.

Is there any interest in collaborating? HeimdaLLM provides the rigorous query validation, and dataherald could provide the LLM integration? Thoughts?

[aazo11]

Hi @amoffat -- thanks for reaching out. We just merged this PR to do some more stringent blocking of DML statements, but are definitely interested to learn more and always open to collaborating. We will take a look at HeimdaLLM in the meanwhile.

[amoffat]

Some constructive criticism.. the regex in the PR appears to have false positives:

select field from table where field="hugh grant";

Postgres also has the concept of "non-reserved" keywords, which are keywords that can be used unquoted as column names:

select update from table;

HeimdaLLM uses full SQL grammars+parsers for each SQL dialect that can handle these cases, as well as restrict what columns can be selected and joined on. I don't want to shill my project on your project's issues, so I'll just say I'm passionate about this problem and if you'd like to chat more, reach out on the email in my profile.