Severe pink pixel flickering on macOS

[forthrin]

llama.cpp exhibits severe, albeit temporal, graphical flickering in the macOS UI under certain circumstances, probably when maxing out -c 4000. It feels like the OS is going to crash, but it recovers. Possibly related to incorrect use of GPU?

Couldn't find any reports on this using keywords, "flicker", "pink", etc. Anyone know what this is, solutions, etc.?

[ggerganov]

Most likely your system runs out of RAM. Use smaller models.

[forthrin]

Thanks! Could a check be added to anticipate this overrun and downgrade resource use, or at the very least panic and exit, rather than rendering temporal havoc on the OS?

[ggerganov]

Setting GGML_METAL_NO_RESIDENCY=1 environment might help with that, but can degrade the performance in some use cases.

[forthrin]

Will try. Reopening if still an issue. Thanks for your prompt (pun intended) and helpful response.