Intrinsic for making HTTP requests #8347
Comments
|
As discussed briefly in slack: I think that we should be very, very cautious as we choose to add more non-deterministic/side-effecting APIs. The ones that currently exist are:
This is not to say that we definitely shouldn't do this: just that we should be conscious of the surface area of code that will be complex. One example: |
|
We can probably ignore side-effecting (i.e., POST requests) for now. But in general rules need to be able to access APIs. |
|
@gshuflin had a design for this here: https://docs.google.com/document/d/1CdlGMXWxdwpGqet7-k6DFIbg90IqR75cFg8SkRu06H8/edit |
### Problem The rust level `Node::cacheable` flag is currently only used to mark `@goal_rule`s as uncacheable (because they are allowed to operate on `@sideeffecting` types, such as the `Console` and the `Workspace`). But since the implementation of `cacheable` did not allow it to operate deeply in the Graph, we additionally needed to mark their parent `Select` nodes uncacheable, and could not use the flag in more positions. Via #7350, #8495, #8347, and #8974, it has become clear that we would like to safely allow nodes deeper in the graph to be uncacheable, as this allows for the re-execution of non-deterministic processes, or re-consumption of un-trackable state, such as: 1. a process receiving stdin from a user 2. an intrinsic rule that pokes an un-watched file on the filesystem 3. interacting with a stateful process like git Note that these would all be intrinsic Nodes: it's not clear that we want to expose this facility to `@rule`s directly. ### Solution Finish adding support for uncacheable nodes. Fixes #6598. When an uncacheable node completes, it will now keep the value it completed with (in order to correctly compute a `Generation` value), but it will re-compute the value once per `Session`. The accurate `Generation` value for the uncacheable node allows its dependents to "clean" themselves and not re-run unless the uncacheable node produced a different value than it had before. ### Result The `Node::cacheable` flag may be safely used deeper in the graph, with the semantics that requests for any of an uncacheable node's dependents will cause it to re-run once per `Session`. The dependents will not re-run unless the value of the uncacheable node changes (regardless of the `Session`).
|
Building upon the lack of side-effects, the I think that this ticket is also stale, so will close. |
We'd like to implement an engine intrinsic that allows pants
@rulesto make arbitrary HTTP requests in a non-blocking way.Open questions:
-what should the API for this look like? (I'm currently thinking two types
HttpRequestandHttpResponse, both of which have reasonable-looking fields on them).-how exactly should this be implemented in the engine? Is
UrlToDownloada good guide?The text was updated successfully, but these errors were encountered: