Implementing an external resource/module reader

[z-jxy]

I was trying to get a baseline implementation of an external reader and was having trouble receiving the initial InitializeResourceReaderRequest message. Debug messages from the server show that the message is being sent, but in the external reader nothing gets received over stdin

Version: Pkl 0.27.1 (macOS 14.3.1, native)

PKL_DEBUG=1 pkl eval tests/pkl/external-reader.pkl --external-resource-reader ldap=target/debug/examples/external_resource_reader
[pkl-core][external-process][target/debug/examples/external_resource_reader] Starting transport: org.pkl.core.messaging.MessageTransports$EncodingMessageTransport@6d00699b
[pkl-core][external-process][target/debug/examples/external_resource_reader] Sending message: InitializeResourceReaderRequest[requestId=-143843479944626018, scheme=ldap]

external_reader.rs:

pub fn main() {
    // ... logging setup
    
    let mut stdin = std::io::stdin();
    let mut buf = Vec::new();
    
    let sz = stdin.read(&mut buf).unwrap();

    info!("Received data size: {:?}", sz); 
    // logged output: `external_resource_reader: Received data size: 0`
}

[HT154]

It's a little tough to tell from the snippet you posted (and without any real Rust knowledge), but my initial suspicion is that the external reader process is attempting to read stdin before the message has been sent, as the message is sent after the process starts. External reader processes are expected to block (until a Close External Process message is received) and continually read data from stdin.

[z-jxy]

Thanks for the help, the issue turned out to be rust related. I had this read method confused with another; this function will only read up to the capacity/length of the vector, so passing in an empty vector was causing 0 bytes to be read. After increasing the buffer size I can receive the message