Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use new Enso Hash Codes and Comparable #6060

Merged
Show file tree
Hide file tree
Changes from 11 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -117,6 +117,13 @@ type Comparable
hash_callback : Atom -> Integer
hash_callback atom = (Comparable.from atom).hash atom

## PRIVATE
A callback allowing to compare two atoms with a custom comparator.
compare_callback : Atom -> Atom -> Integer | Nothing
compare_callback atom that =
ordering = Ordering.compare atom that
if ordering.is_error then Nothing else ordering.to_sign

## PRIVATE
A custom comparator is any comparator that is different than the
default ones.
Expand Down

This file was deleted.

Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,6 @@ import project.Data.Array.Array
import project.Data.Numbers.Decimal
import project.Data.Numbers.Integer
import project.Data.Numbers.Number
import project.Data.Ordering.Comparator
import project.Data.Ordering.Ordering
import project.Data.Ordering.Comparable
import project.Data.Range.Extensions
Expand Down Expand Up @@ -66,7 +65,7 @@ type Rank_Method
Error.throw (Incomparable_Values.Error exc.payload.getLeftOperand exc.payload.getRightOperand)

handle_cmp_exc <| handle_nullpointer <|
java_ranks = Rank.rank input.to_array Comparator.new java_method
java_ranks = Rank.rank input.to_array java_method
Vector.from_polyglot_array java_ranks

type Statistic
Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,9 @@
import project.Data.Numbers.Integer

from project.Data.Ordering import all
from project.Data.Boolean import Boolean, True, False


polyglot java import java.time.DayOfWeek

type Day_Of_Week
Expand Down Expand Up @@ -34,8 +37,9 @@ type Day_Of_Week
Day_Of_Week.Friday -> 5
Day_Of_Week.Saturday -> 6

shifted = if first_day == Day_Of_Week.Sunday then day_number else
(day_number + 7 - (first_day.to_integer start_at_zero=True)) % 7
shifted = case first_day of
Day_Of_Week.Sunday -> day_number
_ -> (day_number + 7 - (first_day.to_integer start_at_zero=True)) % 7
Comment on lines +40 to +42
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
shifted = case first_day of
Day_Of_Week.Sunday -> day_number
_ -> (day_number + 7 - (first_day.to_integer start_at_zero=True)) % 7
shifted = (day_number + 7 - (first_day.to_integer start_at_zero=True)) % 7

btw. won't just that work? I'm not sure if we have to special-case the Sunday given that we do % later anyway.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

stack overflow - to_integer would be called each time,

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe the culprit is #6065.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh right! Okay then let's keep it.

I guess it would have worked if we did:

        day_number day = case day of
            Day_Of_Week.Sunday -> 0
            Day_Of_Week.Monday -> 1
            Day_Of_Week.Tuesday -> 2
            Day_Of_Week.Wednesday -> 3
            Day_Of_Week.Thursday -> 4
            Day_Of_Week.Friday -> 5
            Day_Of_Week.Saturday -> 6
        shifted = ((day_number self) + 7 - (day_number first_day)) % 7

right?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes. This was only changed to avoid having == in it.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, I was just wondering on a side if we could further simplify it, removing the possibility of the infinite loop altogether.


shifted + if start_at_zero then 0 else 1

Expand All @@ -49,3 +53,14 @@ type Day_Of_Week
Day_Of_Week.Thursday -> DayOfWeek.THURSDAY
Day_Of_Week.Friday -> DayOfWeek.FRIDAY
Day_Of_Week.Saturday -> DayOfWeek.SATURDAY

## PRIVATE
type Day_Of_Week_Comparator
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks exactly as I have envisioned that when dreaming about Comparator design.

compare x y =
x_int = x.to_integer
y_int = y.to_integer
Comparable.from x_int . compare x_int y_int

hash x = x.to_integer

Comparable.from (_:Day_Of_Week) = Day_Of_Week_Comparator
27 changes: 17 additions & 10 deletions distribution/lib/Standard/Table/0.0.0-dev/src/Data/Column.enso
Original file line number Diff line number Diff line change
@@ -1,6 +1,5 @@
from Standard.Base import all
import Standard.Base.Data.Array_Proxy.Array_Proxy
import Standard.Base.Data.Ordering.Comparator
import Standard.Base.Errors.Common.Index_Out_Of_Bounds
import Standard.Base.Errors.Illegal_Argument.Illegal_Argument
import Standard.Base.Errors.Illegal_State.Illegal_State
Expand Down Expand Up @@ -1258,15 +1257,23 @@ type Column
my_compare a b = Ordering.compare a.abs b.abs
Examples.decimal_column.sort by=my_compare
sort : Sort_Direction -> Boolean -> (Any -> Any -> Ordering) | Nothing -> Column
sort self order=Sort_Direction.Ascending missing_last=True by=Nothing =
order_bool = case order of
Sort_Direction.Ascending -> True
Sort_Direction.Descending -> False
java_cmp = Comparator.new by
rule = OrderBuilder.OrderRule.new self.java_column java_cmp order_bool missing_last
mask = OrderBuilder.buildOrderMask [rule].to_array
new_col = self.java_column.applyMask mask
Column.Value new_col
sort self order=Sort_Direction.Ascending missing_last=True by=Nothing = case by of
Nothing ->
order_bool = case order of
Sort_Direction.Ascending -> True
Sort_Direction.Descending -> False
rule = OrderBuilder.OrderRule.new self.java_column order_bool missing_last
mask = OrderBuilder.buildOrderMask [rule].to_array
new_col = self.java_column.applyMask mask
Column.Value new_col
_ ->
wrapped a b = case a of
Nothing -> if b.is_nothing then Ordering.Equal else if missing_last then Ordering.Greater else Ordering.Less
_ -> case b of
Nothing -> if missing_last then Ordering.Less else Ordering.Greater
_ -> by a b
Comment on lines +1270 to +1274
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This feels like it could be a separate method that will be useful in other places:
make_comparator_nullsafe missing_last by = a-> b-> ...

sorted = self.to_vector.sort order by=wrapped
Column.from_vector self.name sorted

## Creates a new Column with the specified range of rows from the input
Column.
Expand Down
18 changes: 12 additions & 6 deletions distribution/lib/Standard/Table/0.0.0-dev/src/Data/Table.enso
Original file line number Diff line number Diff line change
@@ -1,7 +1,6 @@
from Standard.Base import all
import Standard.Base.Data.Array_Proxy.Array_Proxy
import Standard.Base.Data.Index_Sub_Range as Index_Sub_Range_Module
import Standard.Base.Data.Ordering.Comparator
import Standard.Base.Errors.Common.Incomparable_Values
import Standard.Base.Errors.Common.Index_Out_Of_Bounds
import Standard.Base.Errors.Common.No_Such_Method
Expand Down Expand Up @@ -47,6 +46,7 @@ from project.Errors import all
from project.Data.Column import get_item_string, normalize_string_for_display
from project.Internal.Filter_Condition_Helpers import make_filter_column

polyglot java import org.enso.base.ObjectComparator
polyglot java import org.enso.table.data.column.builder.object.StorageTypeMismatch
polyglot java import org.enso.table.data.table.Table as Java_Table
polyglot java import org.enso.table.data.table.Column as Java_Column
Expand Down Expand Up @@ -684,7 +684,16 @@ type Table
c.column.java_column
directions = columns_for_ordering.map c->
c.associated_selector.direction.to_sign
comparator = Comparator.for_text_ordering text_ordering

comparator = case text_ordering.sort_digits_as_numbers of
True ->
txt_cmp a b = Natural_Order.compare a b text_ordering.case_sensitivity . to_sign
ObjectComparator.new txt_cmp
False -> case text_ordering.case_sensitivity of
Case_Sensitivity.Default -> ObjectComparator.DEFAULT
Case_Sensitivity.Sensitive -> ObjectComparator.DEFAULT
Case_Sensitivity.Insensitive locale -> ObjectComparator.new False locale.java_locale

java_table = Illegal_Argument.handle_java_exception <| Incomparable_Values.handle_errors <|
self.java_table.orderBy java_columns.to_array directions.to_array comparator
Table.Value java_table
Expand Down Expand Up @@ -1280,11 +1289,8 @@ type Table
join_resolution = make_join_helpers self right . resolve on on_problems
right_columns_to_drop = join_resolution.redundant_column_names

object_comparator = Comparator.new
equality_fallback = .==

java_conditions = join_resolution.conditions
new_java_table = self.java_table.join right.java_table java_conditions (rows_to_keep.at 0) (rows_to_keep.at 1) (rows_to_keep.at 2) (columns_to_keep.at 0) (columns_to_keep.at 1) right_columns_to_drop right_prefix object_comparator equality_fallback
new_java_table = self.java_table.join right.java_table java_conditions (rows_to_keep.at 0) (rows_to_keep.at 1) (rows_to_keep.at 2) (columns_to_keep.at 0) (columns_to_keep.at 1) right_columns_to_drop right_prefix

on_problems.attach_problems_after (Table.Value new_java_table) <|
problems = new_java_table.getProblems
Expand Down
Original file line number Diff line number Diff line change
@@ -1,5 +1,4 @@
from Standard.Base import all hiding First, Last
import Standard.Base.Data.Ordering.Comparator

import project.Data.Column.Column
import project.Data.Column_Selector.Column_Selector
Expand Down Expand Up @@ -222,14 +221,14 @@ java_aggregator name column =
if ordering.is_nothing then FirstAggregator.new name c.java_column ignore_nothing else
order_columns = ordering.map c->c.column.java_column
order_directions = ordering.map c->c.direction.to_sign
FirstAggregator.new name c.java_column ignore_nothing order_columns.to_array order_directions.to_array Comparator.new
FirstAggregator.new name c.java_column ignore_nothing order_columns.to_array order_directions.to_array
Last c _ ignore_nothing ordering ->
if ordering.is_nothing then LastAggregator.new name c.java_column ignore_nothing else
order_columns = ordering.map c->c.column.java_column
order_direction = ordering.map c->c.direction.to_sign
LastAggregator.new name c.java_column ignore_nothing order_columns.to_array order_direction.to_array Comparator.new
Maximum c _ -> MinOrMaxAggregator.new name c.java_column 1 Comparator.new
Minimum c _ -> MinOrMaxAggregator.new name c.java_column -1 Comparator.new
LastAggregator.new name c.java_column ignore_nothing order_columns.to_array order_direction.to_array
Maximum c _ -> MinOrMaxAggregator.new name c.java_column 1
Minimum c _ -> MinOrMaxAggregator.new name c.java_column -1
Shortest c _ -> ShortestOrLongestAggregator.new name c.java_column -1
Longest c _ -> ShortestOrLongestAggregator.new name c.java_column 1
Concatenate c _ join prefix suffix quote -> ConcatenateAggregator.new name c.java_column join prefix suffix quote
Expand Down
98 changes: 43 additions & 55 deletions std-bits/base/src/main/java/org/enso/base/ObjectComparator.java
Original file line number Diff line number Diff line change
@@ -1,73 +1,69 @@
package org.enso.base;

import org.graalvm.polyglot.Context;
import org.graalvm.polyglot.Value;

import java.time.LocalDate;
import java.time.LocalTime;
import java.time.ZonedDateTime;
import java.util.Comparator;
import java.util.Locale;
import java.util.function.BiFunction;
import java.util.function.Function;
import java.util.function.BiFunction;

public class ObjectComparator implements Comparator<Object> {
private static ObjectComparator INSTANCE;

/**
* A singleton instance of an ObjectComparator.
*
* @param fallbackComparator this MUST be the default .compare_to function for Enso. Needs to be
* passed to allow calling back from Java.
* @return Comparator object.
*/
public static ObjectComparator getInstance(Function<Object, Function<Object, Value>> fallbackComparator) {
if (INSTANCE == null) {
INSTANCE = new ObjectComparator(fallbackComparator);
public static final ObjectComparator DEFAULT = new ObjectComparator();

private static BiFunction<Object, Object, Integer> EnsoCompareCallback = null;

private static void initCallbacks() {
if (EnsoCompareCallback == null) {
var module = Context.getCurrent().getBindings("enso").invokeMember("get_module", "Standard.Base.Data.Ordering");
var type = module.invokeMember("get_type", "Comparable");
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This kind of poking around in Enso doesn't make me particularly happy.

However the only alternative I can offer is to always make sure each call from Enso to Java (that needs this callback) passes the compare_callback function to Java.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I also don't like such a poking in Enso. But I guess it is OK for this PR, as a temporary solution.


var are_equal = module.invokeMember("get_method", type, "compare_callback");
EnsoCompareCallback = (v, u) -> {
var result = are_equal.execute(null, v, u);
if (result.isNull()) {
throw new CompareException(u, v);
} else {
return result.asInt();
}
};
}

return INSTANCE;
}

private final Function<Object, Function<Object, Value>> fallbackComparator;
private final Function<String, Function<String, Value>> textComparator;


public ObjectComparator() {
this(
(a) -> (b) -> {
throw new CompareException(a, b);
});
public static int ensoCompare(Object value, Object other) throws ClassCastException {
initCallbacks();
return EnsoCompareCallback.apply(value, other);
}

public ObjectComparator(Function<Object, Function<Object, Value>> fallbackComparator) {
this(fallbackComparator, (a) -> (b) -> Value.asValue(Text_Utils.compare_normalized(a, b)));
}
private final BiFunction<String, String, Integer> textComparator;

private ObjectComparator(Function<Object, Function<Object, Value>> fallbackComparator, Function<String, Function<String, Value>> textComparator) {
this.fallbackComparator = fallbackComparator;
this.textComparator = textComparator;
public ObjectComparator() {
this(true, Locale.ROOT);
}

/**
* Create a copy of the ObjectComparator with case-insensitive text comparisons.
* @param locale to use for case folding.
* @return Comparator object.
*/
public ObjectComparator withCaseInsensitivity(Locale locale) {
return new ObjectComparator(this.fallbackComparator, (a) -> (b) -> Value.asValue(Text_Utils.compare_normalized_ignoring_case(a, b, locale)));
public ObjectComparator(boolean caseSensitive, Locale locale) {
if (caseSensitive) {
textComparator = Text_Utils::compare_normalized;
} else {
textComparator = (a, b) -> Text_Utils.compare_normalized_ignoring_case(a, b, locale);
}
}

/**
* Create a copy of the ObjectComparator with case-insensitive text comparisons.
* @param textComparator custom comparator for Text.
* @return Comparator object.
*/
public ObjectComparator withCustomTextComparator(Function<String, Function<String, Value>> textComparator) {
return new ObjectComparator(this.fallbackComparator, textComparator);
public ObjectComparator(Function<Object, Function<Object, Value>> textComparator) {
this.textComparator = (a, b) -> {
var result = textComparator.apply(a).apply(b);
if (result.isNull()) {
throw new CompareException(a, b);
}
return result.asInt();
};
}

@Override
public int compare(Object thisValue, Object thatValue) throws ClassCastException {
public int compare(Object thisValue, Object thatValue) {
// NULLs
if (thisValue == null) {
if (thatValue != null) {
Comment on lines -70 to 94
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ideally, this should also be removed and delegate to some common method in common-polyglot-core-utils. But ofc ok as a workaround, just - I assume this will be implemented by @Akirathan in #5259 (although I see that this is not scheduled nor assigned to him - do we plan to do this anytime soon @jdunkerley @JaroslavTulach?).

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The code delegates to EnsoObjectWrapper and that class is using org.graalvm.polyglot.Context. We cannot/shouldn't use org.graalvm.polyglot.Context in the engine - e.g. I am afraid that isn't much to share in current implementation.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't understand?

This is just a workaround that should be replaced with a proper solution. The workaround uses a Context, because it is its way to 'access' the current implementation and delegate to it (through a slow Java-to-Enso call).

The whole idea of extracting the shared code is so that the logic for computing the hash for particular primitive types can be exposed in pure Java, so that our library could call into it directly in Java, without the overhead of an Java-to-Enso call. That is what is described in #5259, although maybe I did not describe this well enough. If it's not clear - let's discuss.

Expand Down Expand Up @@ -121,7 +117,7 @@ public int compare(Object thisValue, Object thatValue) throws ClassCastException

// Text
if (thisValue instanceof String thisString && thatValue instanceof String thatString) {
return convertComparatorResult(textComparator.apply(thisString).apply(thatString), thisString, thatString);
return this.textComparator.apply(thisString, thatString);
}

// DateTimes
Expand All @@ -145,14 +141,6 @@ public int compare(Object thisValue, Object thatValue) throws ClassCastException
}

// Fallback to Enso
return convertComparatorResult(fallbackComparator.apply(thisValue).apply(thatValue), thisValue, thatValue);
}

private static int convertComparatorResult(Value comparatorResult, Object leftOperand, Object rightOperand) {
if (comparatorResult.isNumber() && comparatorResult.fitsInInt()) {
return comparatorResult.asInt();
} else {
throw new CompareException(leftOperand, rightOperand);
}
return ensoCompare(thisValue, thatValue);
}
}
Loading