BoundsError Issue when converting to DataFrames regression #306

[pankgeorg]

#306

ODBC v1.0.4, julia 1.7
The violating character is \xce\0

Branch jq/fixdec works normally.
"Select @@version" gives
Microsoft SQL Server 2014 - 12.0.2269.0 (X64) \n\tJun 10 2015 03:35:45 \n\tCopyright (c) Microsoft Corporation\n\tExpress Edition (64-bit) on Windows NT 6.1 <X64> (Build 7601: Service Pack 1)\n"

Update: The violating character is not \xce\0 that's what the branch jq/fixdec shows. The correct string is much weirder, it's

"Α ΕΛΕΥΘΕΡΩΝ ΕΠΑΓΓΕΛΜΑΤΩΝ ΑΘΗΝΩΝ" vs "A ΕΛΕΥΘΕΡΩΝ ΕΠΑΓΓΕΛΜΑΤΩΝ Α\xce\0". Collation is Greek_CI_AS and max_size = 50.
Most likely this is an encoding issue. The error mentions something about accessing positions 1-59 of string with 50 characters.

[quinnj]

I've opened the PR again; but you're saying it doesn't fix the issue?

[pankgeorg]

I've opened the PR again; but you're saying it doesn't fix the issue?

It does work with the PR, meaning it doesn't explode, but it returns a truncated version of the string. I think there is an underlying encoding issue.

[jakobnissen]

I've just hit this issue. The string that triggers it is "Lokalt prøvenr.: IKKE ANGIVET Uoplyst prøvetagningsdato Overføres til konto 65", and the index error is ERROR: BoundsError: attempt to access 81-element Vector{UInt8} at index [1:82]. Indeed, there are 82 codeunits in the string. So the issue must be that somehow, not enough space is allocated to the string.

[jakobnissen]

Bump - is there any MWE or similar I can provide to have this resolved?

It seems to be a matter of characters vs. codeunits. The aforementioned field in my database has VARCHAR(80), i.e. a cap of 80 characters. For some reason ODBC.jl treats this as a cap of 80 bytes instead. Since SQL uses UTF16 by default, shouldn't it be 160 bytes max?

Supporting this hypothesis, changing line 355 in dbinterface.jl from bytes = data[1:b.totallen] to bytes = data[1:min(length(data), b.totallen)] means it no longer errors. Instead, however, the string listed above is truncated to "Lokalt prøvenr.: IKKE ANGIVET Uoplyst prøvetagningsdato Overføres til konto \0". This is the first 80 bytes of the original string plus an errant null byte terminator.

[pankgeorg]

I've been able to work around this by casting the result into a TEXT type on the SQL side:

SELECT CAST(name AS TEXT), .... FROM t_cust instead of SELECT name, .... FROM t_cust

This is not a solution (As you can't use sql's *) but it got me going for now!