httpclient -d:ssl and db_postgres incompatible and cause SIGSEGV! (MacOSX) #9419
Comments
|
I added a: if newCTX.isNil:
raiseSSLError()Just above net.nim |
|
Is there a way to check if openSSL was already initialized? I think it happens inside libpq.dylib code not mine. |
|
Try commenting out the init lines in the openssl (or wherever they are)
module.
…On Wed, 17 Oct 2018, 23:42 treeform, ***@***.***> wrote:
Is there a way to check if openSSL was already initialized? I think it
happens inside libpq.dylib code not mine.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
<#9419 (comment)>, or mute
the thread
<https://github.com/notifications/unsubscribe-auth/AAPDe0cgciecgIgPzCwG-1MCyYY66ugpks5ul7JigaJpZM4Xph8c>
.
|
|
I already tried that commenting "SSL_library_init" does nothing. Same error. |
|
You may be interested in this. |
proc pqinitOpenSSL*(do_ssl: int32, do_crypto: int32) {.cdecl, dynlib: dllName, importc: "PQinitOpenSSL".}
echo "calling pqinitOpenSSL"
pqinitOpenSSL(0, 0)Added this in, compiles and runs but still fails in the same way... I could try compiling libpg without SSL. |
|
I figured it out! It looks like -d:ssl and libpq grab different libssl.dylibs. I used otool to figure out what dylib libpq wants: Its strange but otool does not work on nim programs (does nim use some non-standard system for dylib includes?) I did this to openssl.nim and it works: I think the problem here is that I am not sure how to fix it in the source code though? Should libssl and libcrypot be a -d:ssl_version to be passed? |
|
huh damn. That's really tricky. One thing I can think of is:
Problem: is it possible to query the DLLs that are loaded by a process? Can a handle be grabbed to such a process? |
|
This 1 line patch fixes it: #10230 |
|
reopening see #10281 (comment) for context; we need to find a proper fix that doens't introduce a regression |
|
somewhat relevant: https://github.com/nim-lang/nimble#troubleshooting regarding SSL issues on OSX; but we need proper fix |
|
Using choosenim with devel#head I'm getting a similar crash. Something I learned, I have two versions of OpenSSL installed on MacOSX: 1.x and 1.1.x.
|
|
Okay, I have postgres and httpclient working with when defined(windows):
const
dllName = "libpq.dll"
elif defined(macosx):
const
dllName = "libpq.dylib"
else:
const
dllName = "libpq.so(.5|)"
proc pqinitOpenSSL*(do_ssl: int32, do_crypto: int32) {.cdecl, dynlib: dllName, importc: "PQinitOpenSSL".}
proc main() =
echo "Initting OpenSSL: " & $SSL_library_init()
echo "Setting up db connection ..."
pqinitOpenSSL(0, 0) # Tell postgres not to init OpenSSL
# ... opening database code
# ... using httpclient to make http requestsI had both My env: |
|
I found that |
|
@treeform that's strange. Because I was getting the same crash as you in my code before I managed this workaround. And my libpq has similar deps as your version. Okay, I tried your code and got the same crash as you. I modified to be this and it worked fine for me. from openssl import SSL_library_init
import db_postgres, httpclient
when defined(windows):
const
dllName = "libpq.dll"
elif defined(macosx):
const
dllName = "libpq.dylib"
else:
const
dllName = "libpq.so(.5|)"
proc pqinitOpenSSL*(do_ssl: int32, do_crypto: int32) {.cdecl, dynlib: dllName, importc: "PQinitOpenSSL".}
proc main() =
echo "Initting OpenSSL: " & $SSL_library_init()
echo "Setting up db connection ..."
pqinitOpenSSL(0, 0) # Tell postgres not to init OpenSSL
var client = newHttpClient()
discard client.getContent("https://google.com")
var db = open("", "", "", "") # fails because it can't connect, but no crash
when isMainModule:
main() |
|
How about we enforce --dynlibOverride for our SSL wrapper and always link "statically" to the lib*.so or whatever the correct terminology is for this clusterfuck. |
|
Mixed feelings. The security community seems to oscillate between static and dynamically linking OpenSSL. But with how often vulnerabilities are being found and the frequency that OpenSSL is being updated, I'd lean towards keeping dynamic linking and making it more foolproof. |
|
I think the real problem is that Nim just does not support all of the openSSL it claims it supports on all operating systems. So you have to find ssl that both pg and you will support. This mostly fixes: #10230 - it tells nim to use the newest openSSL you can find. But that fix runs into this issue: #10281 (comment) - here nim finds some newest openSSL that breaks it. Reverting my fix forces it to start looking for an old openSSL that works. I think the reason for the revert |
|
I created a spread sheet to track support and versions of different libssl. Turns it its too confusing! https://docs.google.com/spreadsheets/d/1JgbQS5e2I5bEzFaVYbYrt714CoWGHj-gRIovd7MUfMw/edit?usp=sharing I think we should drop support for openSSL 0.9.9 as it never shipped. LibreSSL which is also compatible with openSSL goes by the same name but their version are really confusing. Its really hard to find mapping between libssl.##.so to LibreSSL version. Its really unclear when any of these shipped, but they shipped before 4/11/2017: We almost need a test case that can test all versions of openSSL/LibreSSL. Or drop older versions and prevent nim from compiling with them. |
|
This was already reported earlier in this tread and is the same root cause for issue 122 on the choosenim issue tracker. choosenim loads libcurl and libssl/libcrypto on OSX using Nim's dynlib loading. libcurl itself is dynamically linked to libssl/libcrypto. If versions don't match, it crashes. |
|
IRC discussion: https://irclogs.nim-lang.org/12-02-2020.html#17:04:09 |
|
Consensus of the conversation was to link ssl/crypto at compile time instead of loading it at runtime. I tried this for choosenim though and it didn't work: when defined(macosx):
--define:curl
--dynlibOverride:ssl
--dynlibOverride:crypto
switch("passL", "-lssl -lcrypto")
--define:sslCompiling with this results in a link time error - We got lucky in that choosenim doesn't really need when defined(macosx):
--define:curl
else:
--define:sslThis fixes the crash for choosenim but does not really provide a path forward for the general issue. Need help from others to figure out how to link against openssl in a traditional fashion on osx. |
|
My thoughts:
I fixed the order but it was reverted because it caused issues for @timotheecour at #10282. Because it was reverted that made me feel unwanted so I did not continue down this path. But I still believe the real solution lives there. If you apply my patch again does it fix the issues you are having? I made a sheet detailing every version: https://docs.google.com/spreadsheets/d/1JgbQS5e2I5bEzFaVYbYrt714CoWGHj-gRIovd7MUfMw/edit#gid=0 You can see on the sheet Nim claims to load version that never existed. Nim missed some. Nim loads openSSL out of order. It's not surprising we are having these OpenSSL issues because of this. Some other library is going i'll load the most recent openSSL, while nim goes I'll load a random openSSL user might have laying around. |
very sorry to hear the revert made you feel unwanted; I greatly appreciate all your contributions; as I wrote here #10281 (comment) :
|
|
On Ubuntu 20.04, I have the same issue with nake Without nake, |
|
I have written a HTTP library to solve this problem: https://github.com/treeform/puppy |
nim c -r -d:ssl server/crazyssl2
nim -v
The text was updated successfully, but these errors were encountered: