Infer generics for assignments #1131

haewiful · 2025-01-15T23:04:59Z

Infer Nullability for assignments of instances with type parameters. ex) Foo<@Nullable Object> f = Foo.make(null);
void genericInferenceOnAssignments() in GenericMethodTests.java

…bility at call

msridhar · 2025-01-26T21:18:52Z

@haewiful thanks for this! It looks like CI fails on the :nullaway:buildWithNullAway task:

https://github.com/uber/NullAway/actions/runs/12978381939/job/36192901792?pr=1131#step:5:606

I suggest looking at the code it is crashing on and trying to write a unit test that causes the same failure, which will make it easier to debug.

# Conflicts: # nullaway/src/main/java/com/uber/nullaway/NullAway.java

codecov · 2025-02-03T21:09:01Z

Codecov Report

Attention: Patch coverage is 92.20779% with 6 lines in your changes missing coverage. Please review.

Project coverage is 88.16%. Comparing base (15c817a) to head (b45bbad).

Files with missing lines	Patch %	Lines
...ava/com/uber/nullaway/generics/GenericsChecks.java	92.68%	0 Missing and 3 partials ⚠️
...a/com/uber/nullaway/generics/InferTypeVisitor.java	90.32%	1 Missing and 2 partials ⚠️

Additional details and impacted files

@@             Coverage Diff              @@
##             master    #1131      +/-   ##
============================================
+ Coverage     88.12%   88.16%   +0.03%     
- Complexity     2268     2286      +18     
============================================
  Files            87       88       +1     
  Lines          7438     7512      +74     
  Branches       1484     1500      +16     
============================================
+ Hits           6555     6623      +68     
- Misses          445      446       +1     
- Partials        438      443       +5

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

msridhar

Looking good! Some feedback below

msridhar · 2025-02-13T18:24:12Z

nullaway/src/main/java/com/uber/nullaway/NullAway.java

@@ -277,6 +277,8 @@ public Config getConfig() {
   */
  private final Map<ExpressionTree, Nullness> computedNullnessMap = new LinkedHashMap<>();

+  private GenericsChecks genericsChecks = new GenericsChecks();


In a follow up (not for this PR), we should probably make a Config field inside GenericsChecks and then stop passing it as a parameter to a whole bunch of methods. Again not for this PR

msridhar · 2025-02-13T18:26:15Z

nullaway/src/main/java/com/uber/nullaway/generics/GenericsChecks.java

@@ -46,8 +46,7 @@
 /** Methods for performing checks related to generic types and nullability. */
 public final class GenericsChecks {

-  /** Do not instantiate; all methods should be static */
-  private GenericsChecks() {}
+  private final Map<Tree, Map<Type, Type>> inferredTypes = new HashMap<>();


Add Javadoc on what this is. For now, can the key type for the outer map be MethodInvocationTree? And are the keys of the nested maps always TypeVariables?

msridhar · 2025-02-13T18:28:09Z

nullaway/src/main/java/com/uber/nullaway/generics/GenericsChecks.java

+        // update inferredTypes cache for assignments
+        if (methodSymbol.type instanceof Type.ForAll // generic method call
+            && methodInvocationTree.getTypeArguments().isEmpty() // no explicit generic arguments
+            && lhsTypeTree instanceof ParameterizedTypeTree) { // lhs type has generic


Rather than checking the tree, can we check if lhsType is generic? That might be more general.

msridhar · 2025-02-13T18:34:14Z

nullaway/src/main/java/com/uber/nullaway/generics/GenericsChecks.java

@@ -443,6 +487,58 @@ public static void checkTypeParameterNullnessForAssignability(
    }
  }

+  private @Nullable Type inferMethodTypeArgument(
+      Type typeParam, List<Type> lhsTypeArg, List<Type> typeArg, VisitorState state) {
+    Type inferType = null;


Declare this variable inside the else if case in the loop. Declaring it up here obscures the logic

msridhar · 2025-02-13T18:34:23Z

nullaway/src/main/java/com/uber/nullaway/generics/GenericsChecks.java

+      }
+    }
+    // base case: typeArg doesn't contain typeParam
+    return inferType;


Here just do return null

msridhar · 2025-02-13T18:35:02Z

nullaway/src/main/java/com/uber/nullaway/generics/GenericsChecks.java

+              if (getTypeNullness(upperBound, analysis.getConfig()) == Nullness.NULLABLE) {
+                Type lhsInferredType =
+                    inferMethodTypeArgument(
+                        typeParam.get(i).type,


Store this expression in a local variable with a readable name, like curTypeParam, and re-use it

msridhar · 2025-02-13T18:35:59Z

nullaway/src/main/java/com/uber/nullaway/generics/GenericsChecks.java

@@ -443,6 +487,58 @@ public static void checkTypeParameterNullnessForAssignability(
    }
  }

+  private @Nullable Type inferMethodTypeArgument(


add javadoc for this method

msridhar · 2025-02-13T18:39:17Z

nullaway/src/main/java/com/uber/nullaway/generics/GenericsChecks.java

@@ -443,6 +487,58 @@ public static void checkTypeParameterNullnessForAssignability(
    }
  }

+  private @Nullable Type inferMethodTypeArgument(


For both this method and the next one (replaceGenerics), I think you need a proper visitor rather than just a recursive function. E.g., what if you have a generic method with return type Foo<U>[], or Foo<Foo<U>[]>? I don't think the current code will handle it.

msridhar · 2025-02-13T18:40:17Z

nullaway/src/main/java/com/uber/nullaway/generics/GenericsChecks.java

+    return inferType;
+  }
+
+  private Type replaceGenerics(


Could this method be rewritten as a call to com.uber.nullaway.generics.TypeSubstitutionUtils#subst?

msridhar · 2025-02-22T17:23:59Z

nullaway/src/main/java/com/uber/nullaway/generics/GenericsChecks.java

    Tree rhsTree;
    if (tree instanceof VariableTree) {
      VariableTree varTree = (VariableTree) tree;
      rhsTree = varTree.getInitializer();
+
+      if (rhsTree instanceof MethodInvocationTree) {


I think this logic is not in the right place, as it won't work for regular assignments; it only runs for variable declarations. I added a failing test to illustrate.

msridhar

Getting there! Few more comments and another failing test

msridhar · 2025-02-22T21:59:58Z

nullaway/src/main/java/com/uber/nullaway/generics/InferTypeVisitor.java

+import org.jspecify.annotations.Nullable;
+
+/** Visitor that uses two types to infer the type of type variables. */
+public class InferTypeVisitor


I think that returning Maps from each visit method can work, but it's a bit convoluted and also inefficient (as many short-lived maps may be allocated). Instead, I suggest making the genericNullness map an instance field of InferTypeVisitor, initialized to an empty map. The visit methods could mutate this map. And then, at the end, the caller could call a method like getGenericNullessMap() to get the result. With this approach, this class could extend Types.DefaultTypeVisitor<Void, Type>, and all the visit methods would just return null.

msridhar · 2025-02-22T22:03:31Z

nullaway/src/main/java/com/uber/nullaway/generics/GenericsChecks.java

    // rhsTree can be null for a VariableTree.  Also, we don't need to do a check
    // if rhsTree is the null literal
    if (rhsTree == null || rhsTree.getKind().equals(Tree.Kind.NULL_LITERAL)) {
      return;
    }
    Type rhsType = getTreeType(rhsTree, config);

-    if (lhsType != null && rhsType != null) {
+    if (rhsTree instanceof MethodInvocationTree) {


Couldn't we move the code under the if at line 440 to also be under this if? We'd still need to store genericNullness inside inferredTypes but then we could just use it immediately here to reconstruct the inferred type.

msridhar · 2025-02-22T22:08:59Z

nullaway/src/main/java/com/uber/nullaway/generics/GenericsChecks.java

+        List<Type> keyTypeList =
+            genericNullness.keySet().stream()
+                .map(typeVar -> (Type) typeVar)
+                .collect(Collectors.toList());
+        com.sun.tools.javac.util.List<Type> from = com.sun.tools.javac.util.List.from(keyTypeList);
+        com.sun.tools.javac.util.List<Type> to =
+            com.sun.tools.javac.util.List.from(genericNullness.values());


From this code, it's unclear that the from and to lists will end up in the appropriate order, where the right keys are lined up with the right values. (It might work with LinkedHashMaps but the code doesn't use those or enforce that right now.) Instead, I suggest looping over the entries of genericNullness, and building the lists one key-value pair at a time. You can build up the right list type using a ListBuffer and calling append on it, like here:

NullAway/nullaway/src/main/java/com/uber/nullaway/generics/TypeSubstitutionUtils.java

Line 181 in 15c817a

ListBuffer<Type> buf = new ListBuffer<>();

msridhar · 2025-02-22T22:26:20Z

nullaway/src/test/java/com/uber/nullaway/jspecify/GenericMethodTests.java

+            "            return new Foo<>(u);",
+            "        }",
+            "        static void test(Foo<@Nullable Object> f1, Foo<Object> f2) {",
+            "            // no error expected",


This test currently does not pass. I would expect no error for the first call to create, but an error for the second call to create (since the second argument is Foo<Object>). Do you agree? If so we need to fix this.

msridhar and others added 23 commits November 5, 2024 13:08

fix and test

b3eba82

another fix and test

bc9e5fb

add TODO

174b262

check explicit type arguments when comparing generic type param nulla…

c94dc28

…bility at call

formatting

32b87ab

add test case for explicit type arguments

e1dfa8a

Merge branch 'master' into check-type-param

be9c657

apply substitution to method type

24ca5a1

Merge branch 'master' into check-type-param

cc45e43

make function that substitutes explicit type arguments

3f3ebf2

test case

7004926

add test for multiple generics

aaba869

temp structure of inferred_types cache - need to change map key type

37ef4d1

infer on assignments draft

c6a6244

new test case for inference on assignments

277e2b6

update test case

e7346d3

simple cleanup

bca1d7d

formatting

453623a

Merge remote-tracking branch 'upstream/master' into infer-generics

9375170

minor changes

db0b6ce

move cache into GenericsChecks and make related methods nonstatic

35b18cf

Merge branch 'master' into infer-generics

0308a5b

formatting

f5ece94

haewiful added 5 commits January 28, 2025 13:28

fix CI Job failure

f89fef0

# Conflicts: # nullaway/src/main/java/com/uber/nullaway/NullAway.java

remove unused code

0c577ae

debugged for testJdk23

61e4f3e

fetch and merge

9f6c91b

Merge branch 'master' into infer-generics

8a30816

haewiful added 5 commits February 9, 2025 23:33

version passing all tests

e15754a

change according to comments

d5c4e5f

mid status

6c4430e

make code more simple

e4051cc

make pass ./gradlew :nullaway:buildWithNullAway

82628ca

haewiful marked this pull request as ready for review February 10, 2025 08:44

haewiful and others added 4 commits February 10, 2025 12:32

Merge branch 'master' into infer-generics

430e855

remove always true condition

f8c8d07

make method for clearing inferredTypes cache

6bc1d0c

Merge branch 'master' into infer-generics

099fae7

msridhar reviewed Feb 13, 2025

View reviewed changes

haewiful and others added 14 commits February 21, 2025 10:20

change based on comments

29255bb

add visitor for infering types on assignments using method calls

575006e

javadoc

2ab7a31

update according to changed method definition

61d6f95

Merge branch 'master' into infer-generics

d984de2

add test case for return types with arrays

f7c4a3f

add array type inference to visitor

461c309

clean up

48bb542

change key of inferredTypes to MethodInvokationTree

5573800

change inferredTypes value type to Map<TypeVariable, Type>

bbae811

format

3c77d70

remove filter call

ea785ba

simplify

2aec29b

add failing test

014fc93

msridhar reviewed Feb 22, 2025

View reviewed changes

haewiful and others added 3 commits February 22, 2025 13:32

make it work for assignments that are not declarations

b45bbad

simplify

50f73de

another test

cd2ec93

msridhar reviewed Feb 22, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Infer generics for assignments #1131

Infer generics for assignments #1131

haewiful commented Jan 15, 2025

msridhar commented Jan 26, 2025

codecov bot commented Feb 3, 2025 •

edited

Loading

msridhar left a comment

msridhar Feb 13, 2025

msridhar Feb 13, 2025

msridhar Feb 13, 2025

msridhar Feb 13, 2025

msridhar Feb 13, 2025

msridhar Feb 13, 2025

msridhar Feb 13, 2025

msridhar Feb 13, 2025

msridhar Feb 13, 2025

msridhar Feb 22, 2025

msridhar left a comment

msridhar Feb 22, 2025

msridhar Feb 22, 2025

msridhar Feb 22, 2025

msridhar Feb 22, 2025 •

edited

Loading

Infer generics for assignments #1131

Are you sure you want to change the base?

Infer generics for assignments #1131

Conversation

haewiful commented Jan 15, 2025

msridhar commented Jan 26, 2025

codecov bot commented Feb 3, 2025 • edited Loading

Codecov Report

msridhar left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

msridhar left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

msridhar Feb 22, 2025 • edited Loading

Choose a reason for hiding this comment

codecov bot commented Feb 3, 2025 •

edited

Loading

msridhar Feb 22, 2025 •

edited

Loading