Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

binding partition: Start enforcing minimum world age for constant bin… #57102

Merged
merged 5 commits into from
Jan 22, 2025

Conversation

Keno
Copy link
Member

@Keno Keno commented Jan 20, 2025

…dings

Currently, even though the binding partition system is implemented, it is largely enabled. New const definitions get magically "backdated" to the first world age in which the binding was undefined.

Additionally, they do not get their own world age and there is currently no latestworld marker after const definitions. This PR changes this situation to give const markers their own world age with appropriate latestworld increments. Both of these are mandatory for const replacement to work.

The other thing this PR does is to remove the automatic "backdating". To see the difference, consider:

function foo($i)
    Core.eval(:(const x = $i))
    x
end

Without an intervening world age increment, this will throw an UndefVarError on this PR. I believe this is the best option for two reasons:

  1. It will allow us infer these to Union{} in the future (thus letting inference prune dead code faster).
  2. I think it is less confusing in terms of the world age semantics for const definitions to become active only after they are defined.

To illustrate the second point, suppose we did keep the automatic backdating. Then we would have:

foo(1) => 1
foo(2) => 1
foo(3) => 2

as opposed to on this PR:

foo(1) => UndefVarError
foo(2) => 1
foo(3) => 2

The semantics are consistent, of course, but I am concerned that an automatic backdating will give users the wrong mental model about world age, since it "fixes itself" the first time, but if you Revise it, it will give an unexpected answer. I think this would encourage accidentally bad code patterns that only break under Revise (where they are hard to debug).

The counterpoint of course is that that not backdating is a more breaking choice. As with the rest of the 1.12-era world age semantics changes, I think taking a look at PkgEval will be helpful.

…dings

Currently, even though the binding partition system is implemented, it is
largely enabled. New `const` definitions get magically "backdated" to the
first world age in which the binding was undefined.

Additionally, they do not get their own world age and there is currently no `latestworld` marker
after `const` definitions. This PR changes this situation to give
const markers their own world age with appropriate `latestworld` increments.
Both of these are mandatory for `const` replacement to work.

The other thing this PR does is to remove  the automatic "backdating".
To see the difference, consider:

```
function foo($i)
    Core.eval(:(const x = $i))
    x
end
```

Without an intervening world age increment, this will throw an
UndefVarError on this PR. I believe this is the best option for
two reasons:

1. It will allow us infer these to `Union{}` in the future (thus
   letting inference prune dead code faster).
2. I think it is less confusing in terms of the world age semantics
   for `const` definitions to become active only after they are defined.

To illustrate the second point, suppose we did keep the automatic backdating.
Then we would have:
```
foo(1) => 1
foo(2) => 1
foo(3) => 2
```
as opposed to on this PR:
```
foo(1) => UndefVarError
foo(2) => 1
foo(3) => 2
```

The semantics are consistent, of course, but I am concerned that
an automatic backdating will give users the wrong mental model
about world age, since it "fixes itself" the first time, but
if you Revise it, it will give an unexpected answer. I think
this would encourage accidentally bad code patterns that only
break under Revise (where they are hard to debug).

The counterpoint of course is that that not backdating is a
more breaking choice. As with the rest of the 1.12-era world
age semantics changes, I think taking a look at PkgEval will
be helpful.
@Keno Keno added this to the 1.12 milestone Jan 20, 2025
@Keno
Copy link
Member Author

Keno commented Jan 21, 2025

Alright, this is probably too breaking as-is. Don't even need to look at pkgeval, since Documenter is unhappy also. That said, I still think the automatic silent backdating is too confusing. As a compromise, I think we can do the backdating, but emit a warning. When and if we turn off that warning (i.e. whether it's a warning or an error in the full 1.12 release or we wait until 1.13 or later), we can decide later. I will update this PR to re-instate the automatic backdating and then I will add the warning in a separate PR.

Keno added 4 commits January 21, 2025 19:19
This doctest is bad, because it tests the internal representation of
the macro, not its behavior. With the additional features added to
the macro, the expansion is no longer as simple, so remove the test.
@Keno Keno merged commit 7f99e95 into master Jan 22, 2025
7 checks passed
@Keno Keno deleted the kf/constminworld branch January 22, 2025 12:40
Keno added a commit that referenced this pull request Jan 22, 2025
This implements the strategy proposed in #57102 (comment).
Example:
```
julia> function foo(i)
           eval(:(const x = $i))
           x
       end
foo (generic function with 1 method)

julia> foo(1)
WARNING: Detected access to binding Main.x in a world prior to its definition world.
  Julia 1.12 has introduced more strict world age semantics for global bindings.
  !!! This code may malfunction under Revise.
  !!! This code will error in future versions of Julia.
Hint: Add an appropriate `invokelatest` around the access to this binding.
1
```

The warning is triggered once per binding to avoid spamming for repeated access.
Keno added a commit that referenced this pull request Jan 22, 2025
This implements the strategy proposed in #57102 (comment).
Example:
```
julia> function foo(i)
           eval(:(const x = $i))
           x
       end
foo (generic function with 1 method)

julia> foo(1)
WARNING: Detected access to binding Main.x in a world prior to its definition world.
  Julia 1.12 has introduced more strict world age semantics for global bindings.
  !!! This code may malfunction under Revise.
  !!! This code will error in future versions of Julia.
Hint: Add an appropriate `invokelatest` around the access to this binding.
1
```

The warning is triggered once per binding to avoid spamming for repeated access.
Keno added a commit that referenced this pull request Jan 22, 2025
This implements the strategy proposed in #57102 (comment).
Example:
```
julia> function foo(i)
           eval(:(const x = $i))
           x
       end
foo (generic function with 1 method)

julia> foo(1)
WARNING: Detected access to binding Main.x in a world prior to its definition world.
  Julia 1.12 has introduced more strict world age semantics for global bindings.
  !!! This code may malfunction under Revise.
  !!! This code will error in future versions of Julia.
Hint: Add an appropriate `invokelatest` around the access to this binding.
1
```

The warning is triggered once per binding to avoid spamming for repeated access.
Keno added a commit that referenced this pull request Jan 23, 2025
This implements the strategy proposed in #57102 (comment).
Example:
```
julia> function foo(i)
           eval(:(const x = $i))
           x
       end
foo (generic function with 1 method)

julia> foo(1)
WARNING: Detected access to binding Main.x in a world prior to its definition world.
  Julia 1.12 has introduced more strict world age semantics for global bindings.
  !!! This code may malfunction under Revise.
  !!! This code will error in future versions of Julia.
Hint: Add an appropriate `invokelatest` around the access to this binding.
1
```

The warning is triggered once per binding to avoid spamming for repeated access.
Keno added a commit that referenced this pull request Jan 23, 2025
This implements the strategy proposed in #57102 (comment).
Example:
```
julia> function foo(i)
           eval(:(const x = $i))
           x
       end
foo (generic function with 1 method)

julia> foo(1)
WARNING: Detected access to binding Main.x in a world prior to its definition world.
  Julia 1.12 has introduced more strict world age semantics for global bindings.
  !!! This code may malfunction under Revise.
  !!! This code will error in future versions of Julia.
Hint: Add an appropriate `invokelatest` around the access to this binding.
1
```

The warning is triggered once per binding to avoid spamming for repeated access.
Keno added a commit that referenced this pull request Jan 23, 2025
This implements the strategy proposed in #57102 (comment).
Example:
```
julia> function foo(i)
           eval(:(const x = $i))
           x
       end
foo (generic function with 1 method)

julia> foo(1)
WARNING: Detected access to binding Main.x in a world prior to its definition world.
  Julia 1.12 has introduced more strict world age semantics for global bindings.
  !!! This code may malfunction under Revise.
  !!! This code will error in future versions of Julia.
Hint: Add an appropriate `invokelatest` around the access to this binding.
1
```

The warning is triggered once per binding to avoid spamming for repeated access.
Keno added a commit that referenced this pull request Jan 24, 2025
This implements the strategy proposed in #57102 (comment).
Example:
```
julia> function foo(i)
           eval(:(const x = $i))
           x
       end
foo (generic function with 1 method)

julia> foo(1)
WARNING: Detected access to binding Main.x in a world prior to its definition world.
  Julia 1.12 has introduced more strict world age semantics for global bindings.
  !!! This code may malfunction under Revise.
  !!! This code will error in future versions of Julia.
Hint: Add an appropriate `invokelatest` around the access to this binding.
1
```

The warning is triggered once per binding to avoid spamming for repeated access.
Keno added a commit that referenced this pull request Jan 24, 2025
This implements the strategy proposed in #57102 (comment).
Example:
```
julia> function foo(i)
           eval(:(const x = $i))
           x
       end
foo (generic function with 1 method)

julia> foo(1)
WARNING: Detected access to binding Main.x in a world prior to its definition world.
  Julia 1.12 has introduced more strict world age semantics for global bindings.
  !!! This code may malfunction under Revise.
  !!! This code will error in future versions of Julia.
Hint: Add an appropriate `invokelatest` around the access to this binding.
1
```

The warning is triggered once per binding to avoid spamming for repeated access.
Keno added a commit that referenced this pull request Jan 24, 2025
This is the analog of #57102 for global variables. Unlike for consants,
there is no automatic global backdate mechanism. The reasoning for this
is that global variables can be declared at any time, unlike constants
which can only be decalared once their value is available. As a result
code patterns using `Core.eval` to declare globals are rarer and likely
incorrect.
Keno added a commit that referenced this pull request Jan 24, 2025
This is the analog of #57102 for global variables. Unlike for consants,
there is no automatic global backdate mechanism. The reasoning for this
is that global variables can be declared at any time, unlike constants
which can only be decalared once their value is available. As a result
code patterns using `Core.eval` to declare globals are rarer and likely
incorrect.
xal-0 pushed a commit to xal-0/julia that referenced this pull request Jan 24, 2025
…57102)

Currently, even though the binding partition system is implemented, it
is largely enabled. New `const` definitions get magically "backdated" to
the first world age in which the binding was undefined.

Additionally, they do not get their own world age and there is currently
no `latestworld` marker after `const` definitions. This PR changes this
situation to give const markers their own world age with appropriate
`latestworld` increments. Both of these are mandatory for `const`
replacement to work.

The other thing this PR does is prepare to remove the automatic "backdating". To
see the difference, consider:

```
function foo($i)
    Core.eval(:(const x = $i))
    x
end
```

Without an intervening world age increment, this will throw an
UndefVarError on this PR. I believe this is the best option for two
reasons:

1. It will allow us infer these to `Union{}` in the future (thus letting
inference prune dead code faster).
2. I think it is less confusing in terms of the world age semantics for
`const` definitions to become active only after they are defined.

To illustrate the second point, suppose we did keep the automatic
backdating. Then we would have:
```
foo(1) => 1
foo(2) => 1
foo(3) => 2
```
as opposed to on this PR:
```
foo(1) => UndefVarError
foo(2) => 1
foo(3) => 2
```

The semantics are consistent, of course, but I am concerned that an
automatic backdating will give users the wrong mental model about world
age, since it "fixes itself" the first time, but if you Revise it, it
will give an unexpected answer. I think this would encourage
accidentally bad code patterns that only break under Revise (where they
are hard to debug).

The counterpoint of course is that that not backdating is a more
breaking choice. As with the rest of the 1.12-era world age semantics
changes, I think taking a look at PkgEval will be helpful.
Keno added a commit that referenced this pull request Jan 25, 2025
This implements the strategy proposed in
#57102 (comment).
Example:
```
julia> function foo(i)
           eval(:(const x = $i))
           x
       end
foo (generic function with 1 method)

julia> foo(1)
WARNING: Detected access to binding Main.x in a world prior to its definition world.
  Julia 1.12 has introduced more strict world age semantics for global bindings.
  !!! This code may malfunction under Revise.
  !!! This code will error in future versions of Julia.
Hint: Add an appropriate `invokelatest` around the access to this binding.
1
```

The warning is triggered once per binding to avoid spamming for repeated
access.
Keno added a commit that referenced this pull request Jan 25, 2025
This is the analog of #57102 for global variables. Unlike for consants,
there is no automatic global backdate mechanism. The reasoning for this
is that global variables can be declared at any time, unlike constants
which can only be decalared once their value is available. As a result
code patterns using `Core.eval` to declare globals are rarer and likely
incorrect.
Keno added a commit that referenced this pull request Jan 28, 2025
This is the analog of #57102 for global variables. Unlike for consants,
there is no automatic global backdate mechanism. The reasoning for this
is that global variables can be declared at any time, unlike constants
which can only be decalared once their value is available. As a result
code patterns using `Core.eval` to declare globals are rarer and likely
incorrect.
Comment on lines +1390 to +1394
!!! note
If `f` is a global, it will be resolved consistently
in the (latest) world as the call target. However, all other arguments
(as well as `f` itself if it is not a literal global) will be evaluated
in the current world age.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this mean that in invokelatest(f, args...), f itself is not evaluated in the latest world, whereas using @invokelatest f(args...) ensures that both the function binding and the function call are evaluated in the latest world?

I’ve written the following snippet to find the difference, but it seems that invokelatest behaves as expected (i.e. always returns latest result). I’m curious to know in what situations the difference between these two would become significant?

julia> function foo()
           global fg3 = sin
           x = invokelatest(fg3, 42)
           Main.fg3 = cos
           y = invokelatest(fg3, 42)
           x, y
       end
foo (generic function with 1 method)

julia> foo()
(-0.9165215479156338, -0.39998531498835127)

Keno added a commit that referenced this pull request Jan 28, 2025
This is the analog of #57102 for global variables. Unlike for consants,
there is no automatic global backdate mechanism. The reasoning for this
is that global variables can be declared at any time, unlike constants
which can only be decalared once their value is available. As a result
code patterns using `Core.eval` to declare globals are rarer and likely
incorrect.
Keno added a commit that referenced this pull request Jan 29, 2025
)

This is the analog of #57102 for global variables. Unlike for consants,
there is no automatic global backdate mechanism. The reasoning for this
is that global variables can be declared at any time, unlike constants
which can only be decalared once their value is available. As a result
code patterns using `Core.eval` to declare globals are rarer and likely
incorrect.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants