typst watch doesn't work when editing the watched file with (neo)vim

[tretre91]

Description

The watch command does not seem to work with (neo)vim anymore.

The issue seems to come from the fact that when saving a file, vim will rename the old file and write a new file with the old name. However, for notify-rs this causes a Remove event to be triggered on that file, and removed files are implicitly unwatched on some platforms according to their docs:

On some platforms, if the path is renamed or removed while being watched, behaviour may be unexpected. See discussions in #165 and #166. If less surprising behaviour is wanted one may non-recursively watch the parent directory as well and manage related events.

When running the code through gdb, after saving a first time, the program just waits for events there, and saving the file again does not trigger events.

Note: the oldest commit where this saving behavior caused an issue was e1d7696, trying to unwatch the dependencies, some of which were already unwatched, caused a failed to unwatch something.typ error to happen.

steps to reproduce

• create a typst file, let's say test.typ
• watch the file with typst watch test.typ
• edit the file in vim or neovim
• save the file (with :w), this one should work
• try saving again, and the modifications are not picked up by typst

Vim's saving method can be changed to copy the new file into the old file by executing the command :set backupcopy=yes, in this case the typst watch command works as expected.

Reproduction URL

No response

Operating system

Linux

Typst version

• I am using the latest version of Typst

[alerque]

Workaround:

$ echo main.typ | entr -c typst compile /_

Incidentally the availability and general usefulness of entr is also why I never bothered to build an equivalent of Typst's watch into SILE.

[korken89]

I am also having this issue with typst and neovim.

[laurmaedje]

I guess for Typst it's a bit different than for SILE because built-in watch can profit from incremental compilation. If I understand the entr command correctly, it will also only watch main.typ and not images and imports.

Unfortunate that the new approach has this issue because watching individual files instead of the root directory is much nicer in principle (less unnecessary recompilations for files that don't affect the document even though they are in the working dir and it correctly picks up imports from packages).

[alerque]

Yes, this level of workaround does loose the benefit of incremental compilation.

Entr can watch anything you tell it to. For example one approach is to tell it to watch everything tracked by git: git ls-files | entr .... Of course you could also just pass the output of find or more specialized inputs. For example SILE has a feature that cat output a list of every file used while rendering a document: font files, libraries, images, includes, everything. That feature can was originally added to pass to make to know when a document needs re-rendering but can be adapted for use by entr as a watch list.

[alkeryn]

this issue stems from the fact that vim create a new file and move it to replace the old one.
the library used to watch is notify, which afaik doesn't get when that happens (maybe there is a workaround).
watching the parent directory instead of the file would work but maybe there is a cleaner fix.

[laurmaedje]

Unfortunately I cannot currently test any potential fix as I'm on macOS and don't have a Linux machine available. If anybody on Linux wants to give fixing this a shot, I'd appreciate it.

(It works for me in vim on macOS.)

[ayoubelmhamdi]

I have some issue using version:0.6

typst watch main.typ

My solution

find .  -type f -iname "*.typ" | entr make

all: compile
compile:
	@typst  compile main.typ

[alerque]

@ayoubeimhamdi By rights your Makefile should either by a Justfile (since you're just using it as a task runner) or maybe try including dependencies ;-)

.PHONY: all
all: main.pdf

DEPS != find . -type f -iname "*.typ"

main.pdf: main.typ $(DEPS)
	@typst compile $<

Now make will understand when you PDF does not need rebuilding. It still won't get you incremental rendering updates.

Back to topic:

@laurmaedje Given that many other tools are able to catch changes to files when Neovim and other apps that do write→rename sequences (including entr as discussed in this issue) there is probably a way to do it. You probably just need to add more filesystem events to your notify loop including creation of files by the given name.

[raphCode]

I want to quickly chime in to point out the rust project watchexec.
Basically, the command line app behaves similar to entr.

However, they also provide libraries for custom file watches and various other subfeatures of the main app.
Maybe this is a suitable building block for typst watch?

[tretre91]

Removing the deleted files from the previous compilation's dependencies in the watch function seems to work. That way, when calling watch_dependencies, if the files are in the current compilation's dependencies and still exist, they will be considered as not watched and be watched again.

(I am not very familiar with rust, especially the borrow checker part, so there might be a better way to do this)

loop {
    let mut removed = HashSet::new(); // will contain paths to the removed files
    let mut recompile = false;
    for event in rx
        .recv()
        .into_iter()
        .chain(std::iter::from_fn(|| rx.recv_timeout(timeout).ok()))
    {
        let event = event.map_err(|_| "failed to watch directory")?;

        if matches!(event.kind, notify::EventKind::Remove(notify::event::RemoveKind::File)) {
            let path = &event.paths[0];
            removed.insert(path.clone());

            // Tries to remove the watch if it still exists
            // I don't know if watching the same file multiple times could be problematic on some
            // platforms, so this may not be necessary
            match watcher.unwatch(&path) {
                Err(notify::Error { kind, .. }) => match kind {
                    notify::ErrorKind::WatchNotFound => {},
                    _ => return Err("failed to unwatch file".into()),
                }
                _ => {}
            }
        }

        recompile |= is_event_relevant(&event, &output);
    }

    if recompile {
        // Retrieve the dependencies of the last compilation.
        let previous: HashSet<PathBuf> = world
            .dependencies()
            .filter(|path| !removed.contains(*path)) // remove the deleted files from the previous deps
            .map(ToOwned::to_owned)
            .collect();

        // Recompile.
        compile_once(&mut world, &mut command, true)?;
        comemo::evict(10);

        // Adjust the watching.
        watch_dependencies(&mut world, &mut watcher, previous)?;
    }
}

[laurmaedje]

@tretre91 Nice, that seems like a good solution!