OutOfMemory Erro with Oracle 11g

[ffmorais]

I can't perform a query over Oracle Database via ODBC with Oracle 11g (64 bit) driver.
I got where following error:
julia> oRec=ODBC.query(oConn,"Select managedobject from table.foo t where rownum < 10;");
ERROR: OutOfMemoryError()
Stacktrace:
[1] Type at .\boot.jl:394 [inlined]
[2] Type at .\boot.jl:403 [inlined]
[3] Type at .\sysimg.jl:155 [inlined]
[4] zeros at .\array.jl:467 [inlined]
[5] zeros at .\array.jl:464 [inlined]
[6] internal_allocate(::Type{UInt8}, ::Int64, ::UInt64) at C:\Users\me.julia\packages\ODBC\KFiOK\src\Query.jl:143
[7] ODBC.Query(::ODBC.DSN, ::String) at C:\Users\me.julia\packages\ODBC\KFiOK\src\Query.jl:85
[8] #query#14(::Bool, ::Bool, ::Dict{Int64,Function}, ::Function, ::ODBC.DSN, ::String, ::Type{DataFrame}) at C:\Users\me.julia\packages\ODBC\KFiOK\src\Query.jl:278
[9] query at C:\Users\me.julia\packages\ODBC\KFiOK\src\Query.jl:273 [inlined]
[10] query(::ODBC.DSN, ::String) at C:\Users\me.julia\packages\ODBC\KFiOK\src\Query.jl:264
[11] top-level scope at none:0

[GeraldKaeferle]

Same here Oracle 11g (64 bit) driver.

df = ODBC.query(dsn, "SELECT * FROM DUAL")

OutOfMemoryError()
in top-level scope at base\none
in query at ODBC\KFiOK\src\Query.jl:264
in query at ODBC\KFiOK\src\Query.jl:273 
in #query#14 at ODBC\KFiOK\src\Query.jl:278
in ODBC.Query at ODBC\KFiOK\src\Query.jl:85
in internal_allocate at ODBC\KFiOK\src\Query.jl:143
in zeros at base\array.jl:464 
in zeros at base\array.jl:467 
in Type at base\sysimg.jl:155 
in Type at base\boot.jl:403 
in Type at base\boot.jl:394 

[ffmorais]

For Oracle Instant Client 18.3 ODBC works fine
https://www.oracle.com/technetwork/topics/winx64soft-089540.html

[quinnj]

Thanks for the reports here @ffmorais and @GeraldKaeferle; would either of you be willing/available to do a coding session with me to help figure this out? These kinds of issues tend to be tricky to debug since I don't have access to the same kind of system to reproduce, but if we can jump on a google hangouts or just connect via chat (like the public julialang slack) to live code/debug things, we can usually figure out what the issue is pretty quickly.

Let me know if either of you could help out like that and I can make myself available when you are to figure things out.

[pnorth1]

Wondering if this OOM behavior is in anyway related to this issue

[metanoid]

The problem seems to be here - the API.SQLRowCount function is returning very large numbers (e.g. 4294967295) even for queries from very small tables with small numbers of rows.

I don't know enough to know why this is happening, but this is the source of the Out Of Memory errors on my side - this causes the internal_allocate function to error downstream.

[quinnj]

Is there anyone who is able to reproduce this and could try latest master to see if the issue persists? To try master, you'd do ] add ODBC#master. If the issue is, as @metanoid suggested, SQLRowCount returning bogus numbers, then I'll need to dig in and see if there's a way to avoid that or work-around, but it'd be great to have someone who can reproduce in order to test a fix.

[metanoid]

Can't test, sorry, we've since moved to Oracle12c, where ODBC.jl is working correctly

[quinnj]

Ok, I'll close this for now.

[nielsls]

Hi @quinnj

Here you go - using ODBC#master and "Driver={Oracle in OraClient11g_home1}"

julia> DBInterface.execute(conn, query)
ERROR: OutOfMemoryError()
Stacktrace:
 [1] Array at .\boot.jl:406 [inlined]
 [2] Array at .\baseext.jl:12 [inlined]
 [3] ODBC.Buffer(::Int16, ::UInt64, ::Int64, ::Int16) at C:\Users\adm-nls\.julia\packages\ODBC\qhwMX\src\utils.jl:142
 [4] Binding at C:\Users\adm-nls\.julia\packages\ODBC\qhwMX\src\utils.jl:262 [inlined]
 [5] (::ODBC.var"#7#8"{ODBC.API.Handle,Bool,Array{Int16,1},Array{Int16,1},Array{UInt64,1},Array{Int16,1},Array{Bool,1},Int64})(::Int64) at .\none:0
 [6] iterate at .\generator.jl:47 [inlined]
 [7] collect(::Base.Generator{UnitRange{Int64},ODBC.var"#7#8"{ODBC.API.Handle,Bool,Array{Int16,1},Array{Int16,1},Array{UInt64,1},Array{Int16,1},Array{Bool,1},Int64}}) at .\array.jl:686
 [8] getbindings at C:\Users\adm-nls\.julia\packages\ODBC\qhwMX\src\utils.jl:301 [inlined]
 [9] ODBC.Cursor(::ODBC.API.Handle; iterate_rows::Bool, ignore_driver_row_count::Bool, normalizenames::Bool, debug::Bool) at C:\Users\adm-nls\.julia\packages\ODBC\qhwMX\src\dbinterface.jl:260
 [10] execute(::ODBC.Connection, ::String, ::Tuple{}; debug::Bool, kw::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}) at C:\Users\adm-nls\.julia\packages\ODBC\qhwMX\src\dbinterface.jl:197
 [11] execute at C:\Users\adm-nls\.julia\packages\ODBC\qhwMX\src\dbinterface.jl:189 [inlined] (repeats 2 times)
 [12] top-level scope at REPL[13]:1

[nielsls]

Hi @quinnj
Any chance we could reopen this?
We could also do a live coding session if required?
Appreciate your work - thx!

[cmhh]

Same thing happens for me with the Sybase / ASE driver:

julia> DBInterface.execute(con, "select top 10 * from classfn") |> DataFrame
ERROR: OutOfMemoryError()
Stacktrace:
  [1] Array
    @ ./boot.jl:448 [inlined]
  [2] Array
    @ ./baseext.jl:12 [inlined]
  [3] ODBC.Buffer(ctype::Int16, columnsize::UInt64, rows::Int64, nullable::Int16)
    @ ODBC ~/.julia/packages/ODBC/qhwMX/src/utils.jl:142
  [4] Binding
    @ ~/.julia/packages/ODBC/qhwMX/src/utils.jl:262 [inlined]
  [5] (::ODBC.var"#7#8"{ODBC.API.Handle, Bool, Vector{Int16}, Vector{Int16}, Vector{UInt64}, Vector{Int16}, Vector{Bool}, Int64})(i::Int64)
    @ ODBC ./none:0
  [6] iterate
    @ ./generator.jl:47 [inlined]
  [7] collect(itr::Base.Generator{UnitRange{Int64}, ODBC.var"#7#8"{ODBC.API.Handle, Bool, Vector{Int16}, Vector{Int16}, Vector{UInt64}, Vector{Int16}, Vector{Bool}, Int64}})
    @ Base ./array.jl:678
  [8] getbindings
    @ ~/.julia/packages/ODBC/qhwMX/src/utils.jl:301 [inlined]
  [9] ODBC.Cursor(stmt::ODBC.API.Handle; iterate_rows::Bool, ignore_driver_row_count::Bool, normalizenames::Bool, debug::Bool)
    @ ODBC ~/.julia/packages/ODBC/qhwMX/src/dbinterface.jl:260
 [10] execute(conn::ODBC.Connection, sql::String, params::Tuple{}; debug::Bool, kw::Base.Iterators.Pairs{Union{}, Union{}, Tuple{}, NamedTuple{(), Tuple{}}})
    @ ODBC ~/.julia/packages/ODBC/qhwMX/src/dbinterface.jl:197
 [11] execute (repeats 2 times)
    @ ~/.julia/packages/ODBC/qhwMX/src/dbinterface.jl:189 [inlined]
 [12] top-level scope
    @ REPL[8]:1

The schema of the underlying table is (varchar, int, int, int), though it fails with a similar message even if I run select getdate() as today. That said, I accept that Sybase is probably getting almost esoteric at this point, and I'll try and install the TDS driver and see if that works anyway. We mostly use SQL Server and that works just fine, but we have some older systems running on Sybase.