[query] Add query_matrix_table an analogue to query_table #14806
Conversation
| private val zippedReader = PartitionZippedNativeReader(rowsReader, entriesReader) | ||
|
|
||
| def contextType = rowsReader.contextType | ||
| def fullRowType = zippedReader.fullRowType |
There was a problem hiding this comment.
@patrick-schultz I'm getting an NPE in initialization, I think it's because this assertion in PartitionReader is running before zippedReader is initialized. Thoughts on resolving that?
hail/hail/hail/src/is/hail/expr/ir/IR.scala
Lines 1300 to 1301 in ecaa96b
There was a problem hiding this comment.
That's annoying. My first thought is to make the sub-readers lazy vals. Maybe try that for now, and we can think more if there's any more satisfying fix.
There was a problem hiding this comment.
Thanks for the suggestion. It worked. I guess lazy val can force initialization, but val is only truly valid after full initialization of the class.
There was a problem hiding this comment.
Right, lazy val foo = init is basically private var _foo = null and def foo = { if (_foo == null) _foo = init; _foo }. So it will be initialized the first time it's accessed, rather than when the class initializer is run. Usually that means it's initialized later than the class, but in this case it happens earlier.
chrisvittal
force-pushed
the
query/query-matrix-table
branch
4 times, most recently
from
e065c6d to
4e0dbe5
Compare
|
I think this is ready for review. |
chrisvittal
force-pushed
the
query/query-matrix-table
branch
2 times, most recently
from
b04b3f5 to
cd1d3a0
Compare
| private[this] class PartitionEntriesNativeIntervalReader( | ||
| sm: HailStateManager, | ||
| entriesPath: String, | ||
| entriesSpec: AbstractTableSpec, | ||
| uidFieldName: String, | ||
| rowsTableSpec: AbstractTableSpec, | ||
| ) extends PartitionNativeIntervalReader(sm, entriesPath, entriesSpec, uidFieldName) { | ||
| override lazy val partitioner = rowsTableSpec.rowsSpec.partitioner(sm) | ||
| } | ||
|
|
||
| case class PartitionZippedNativeIntervalReader( | ||
| sm: HailStateManager, | ||
| mtPath: String, | ||
| mtSpec: AbstractMatrixTableSpec, | ||
| uidFieldName: String, | ||
| ) extends PartitionReader { | ||
| require(mtSpec.indexed) | ||
|
|
||
| // XXX: rows and entries paths are hardcoded, see MatrixTableSpec | ||
| private lazy val rowsReader = | ||
| PartitionNativeIntervalReader(sm, mtPath + "/rows", mtSpec.rowsSpec, "__dummy") | ||
|
|
||
| private lazy val entriesReader = | ||
| new PartitionEntriesNativeIntervalReader( | ||
| sm, | ||
| mtPath + "/entries", | ||
| mtSpec.entriesSpec, | ||
| uidFieldName, | ||
| rowsReader.tableSpec, | ||
| ) | ||
|
|
||
| private lazy val zippedReader = PartitionZippedNativeReader(rowsReader, entriesReader) | ||
|
|
||
| def contextType = rowsReader.contextType | ||
| def fullRowType = zippedReader.fullRowType |
There was a problem hiding this comment.
Forgive me Chris, I'm not really following what's mandating all these private lazy vals.
Since you're not using PartitionZippedNativeIntervalReader for pattern matching, can you make this a smart constructor that returns an anonymous instance of PartitionReader? Perhaps that might solve the initialisation order issue?
chrisvittal
force-pushed
the
query/query-matrix-table
branch
2 times, most recently
from
08bfa1a to
4999e0f
Compare
CHANGELOG: Add query_matrix_table an analogue to query_table
chrisvittal
force-pushed
the
query/query-matrix-table
branch
from
6bc01af to
c1840f8
Compare
CHANGELOG: Add query_matrix_table an analogue to query_table
Part of #14499.
Security Assessment
Delete all except the correct answer:
Impact Description
Increases the query python API surface, but is only a query change.