feat(arrow): GeoArrow utilities #2744
Conversation
| ] | ||
| }; | ||
|
|
||
| test.skip('ArrowUtils#parseGeometryFromArrow', async (t) => { |
There was a problem hiding this comment.
@lixun910 This test case crashes, haven't had time to debug yet.
There was a problem hiding this comment.
Update the serializeArrowType() and deserializeArrowType() functions to fix the issues.
Signed-off-by: Xun Li <[email protected]>
| @@ -71,7 +71,7 @@ export function deserializeArrowMetadata(metadata?: SchemaMetadata): Map<string, | |||
| export function serializeArrowField(field: ArrowField): Field { | |||
| return { | |||
| name: field.name, | |||
| type: serializeArrowType(field.type), | |||
| type: field.type.toString(), | |||
There was a problem hiding this comment.
The test case failed because serializeArrowType(field.type) can't handle the class 'Int', which could be Int8/16/32/64 based on the bitwise and isSigned value.
It seems that the Type class in apache-arrow has toString() to get the stringified arrow field type. I am checking if it compatible with the deserializeArrowType() function.
Update: the DataType toString() is NOT compatible with the deserializeArrowType() function. New cases e.g. Int, Float, List are added in serializeArrowType(field.type).
There was a problem hiding this comment.
It might be easier in some places to use arrow.Type.isUint8() etc instead of manually checking the type yourself
Signed-off-by: Xun Li <[email protected]>
Signed-off-by: Xun Li <[email protected]>
Signed-off-by: Xun Li <[email protected]>
| case List: | ||
| const listType = arrowType as List; | ||
| const listField = listType.valueField; | ||
| return { | ||
| type: 'list', | ||
| children: [serializeArrowField(listField)] | ||
| }; |
There was a problem hiding this comment.
Need to add List class type when serialization, e.g. for a polygon field, the field type could be like List<List<FixedSizeList[2]<Float64>>>
This is a repackaging for #2738 so that those utilities fit into the right places/sub-modules in loaders.gl