Engine should send notification about node status

[wdanilo]

This task is automatically imported from the old Task Issue Board and it was originally created by Dmitry Bushev.
Original issue is here.

---

Why

As an IDE
I want to know the status of the node
So that to display it to the user

Acceptance Criteria

Scenario: 
GIVEN the enso program
WHEN executed in interactive mode
THEN the runtime sends status notifications for each instrumented node

Notes

From the original issue:

Information about node status (computed, to be computed, in progress + info about progress)

The info about progress part is scheduled as a separate #182781062 issue

The to be computed part of the update can be calculated as weights - cached

Plan

• IDE wants to show the progress of the program graph execution. We're not talking of the progress of a single node (expression), but of the overall program (consisting of multiple nodes). To do that IDE needs to know the status of every node (computed or to be computed).
• Engine can use IdExecutionInstrument that intercepts the nodes, and gives us ability to send those notifications.
• Now, I can imagine two approaches.
• First, we can just add the callback to the instrument that sends ToBeComputed {nodeId} or Computed {nodeId} messages. But it's not very helpful because then IDE won't be able to show what part of the graph is already executed, and what part is pending.
• Second approach would be to send all the nodes that will be computed beforehand, and then send Computed {nodeid} messages from the instrument after. The first part (all nodes that will be computed) is a bit tricky because we kinda only know that as (after) the program executes. But we can do some approximation. During the compilation in CachePreferenceAnalysis pass we gather expressions that should be cached during the runtime (and assign weight > 0 to them). So, when the program is about to run, we can send those expressions minus the expressions that are already in the cache (weights - cached in the notes above).

Execution

1. Design a JSONRPC API for the updates (protocol-language-server.md), create a PR, and discuss with Adam and/or Michael if it suits the IDE needs
2. Implement the protocol in a subsequent PR

Comments:

Let's create a test case to demonstrate the problem. Here is a sample, long computation:

from Standard.Base import all
import Standard.Base.Data.Statistics
import Standard.Base.Data.Text.Extensions
import Standard.Visualization

polyglot java import java.lang.Thread


sleep n = Thread.sleep n

compute ~x time =
    sleep time
    x

main =
    wait = 1111
    wait2 = 2 * wait
    
    value1 = 10
    value2 = value1 * 2
    value3 = value2 + value1
    value4 = value2 * 2
    value5 = value3 + value2

    number1 = compute value1 wait
    number2 = compute value2 wait2
    number3 = compute value3 wait
    number4 = compute value4 wait
    number5 = compute value5 wait2


    sum12 = compute (number1 + number2) wait
    sum34 = compute (number3 + number4) wait2
    all = compute (sum12 + sum34 + number5) wait2

When you open this program in the IDE it is nicely visible how the computation progresses. Once computed:

• open visualization for all - shows 150
• change value1 = 11. Nothing happens for 10s
• after long time the visualization shows 165

The IDE needs to provide feedback (at least as good as when computing the nodes for the first time) when value of value1 variable is changed. (jaroslavtulach - Sep 21, 2022)

---

**Jaroslav Tulach** reports a new **STANDUP** for yesterday (2022-09-21):

Progress: Sample program with slowly computing nodes: https://www.pivotaltracker.com/story/show/182818735/comments/233402238; discussions with Dmitry about RuntimeCache.weight & co.; meetings; Rust parser - Merging #3723 into #3611: 0b4f017 It should be finished by 2022-09-26.

Next Day: Send some notification to the IDE (Enso Bot - Sep 22, 2022)

---

**Jaroslav Tulach** reports a new **STANDUP** for yesterday (2022-09-22):

Progress: Adding Pending message to the protocol; Deliver Pending messages to the IDE: 7a7c11d throughout the whole messaging stack; lots of debugging of inconsistencies between engine, LS, IDE - no luck; Review of parser work: #3716 (review) It should be finished by 2022-09-26.

Next Day: Update tests to conver the new notification (Enso Bot - Sep 23, 2022)

---

**Jaroslav Tulach** reports a new **STANDUP** for yesterday (2022-09-23):

Progress: Demo of engine should send notifications is working: ee488ae; prototype of ServiceLoader for File.read working: https://www.pivotaltracker.com/story/show/183349691/comments/233449089; reviews of type_of, Vector.slice, breefing Maric about recent GraalVM happenings. It should be finished by 2022-09-26.

Next Day: Move pending notification to ProgramExecutionSupport; update tests; prepare for review (Enso Bot - Sep 24, 2022)

---

**Jaroslav Tulach** reports a new **STANDUP** for yesterday (2022-09-26):

Progress: Moved pending notification to ProgramExecutionSupport: 2c7652d; update & fixing tests; requested review; Review & bugfix for VectorSlice: #3737 It should be finished by 2022-09-26.

Next Day: Fix remaining tests; get reviewers approvals (Enso Bot - Sep 27, 2022)

---

The IDE still needs to provide better visualization (for example this Picaso-like picture provided by @gregociepka), but otherwise the engine seems to be sending the right information to the IDE.

(jaroslavtulach - Sep 27, 2022)

---

@jtulach 👍 (Greg Ociepka - Sep 27, 2022)

---

**Jaroslav Tulach** reports a new **🔴 DELAY** for yesterday (2022-09-27):

Summary: There is 4 days delay in implementation of the Engine should send notification about node status (#182818735) task.
It will cause 4 days delay for the delivery of this weekly plan.

Currently engine gate is closed for Marcin's statics. Will integrate then.

Delay Cause: Demo was ready on time, but fixing tests and integration requires additional time.

Possible solutions: Less aggressive/optimistic schedules. (Enso Bot - Sep 28, 2022)

---

**Jaroslav Tulach** reports a new **STANDUP** for yesterday (2022-09-27):

Progress: Fixed all Runtime*Tests: https://github.com/enso-org/enso/pull/3729/files#r981034044; got necessary approvals; ready to integrate once the "engine gate" is open. It should be finished by 2022-09-30.

Next Day: Focus on Rust based parser. (Enso Bot - Sep 28, 2022)

---

**Jaroslav Tulach** reports a new **STANDUP** for yesterday (2022-09-28):

Progress: Bank Holiday. Merged #3729. Discussing GraalVM EE licensing. It should be finished by 2022-09-30.

Next Day: Focus on Rust based parser. (Enso Bot - Sep 29, 2022)

---