Adding a watch for a large directory on a slow drive is slow.

[markus-bauer]

I know the title sounds stupid, but I couldn't figure out why it should be slow, or how to make it not slow.

Here's a basic example of what I'm doing:

System:

ubuntu linux 22.04
rustc 1.62.0-nightly
notify verion `5.0.0-pre.15`

use notify::{
    poll::{PollWatcher, PollWatcherConfig},
    RecursiveMode::NonRecursive,
    Watcher,
};
use std::{path::Path, time::Duration};

fn main() {
    let path = Path::new("/path/to/dir");
    let mut watcher = PollWatcher::with_config(
        |_| {
            // nothing
        },
        PollWatcherConfig {
            poll_interval: Duration::from_millis(1000),
            compare_contents: false,
        },
    )
    .unwrap();

    watcher.watch(path, NonRecursive);
}

I was adding a watch for a directory with ca. 6000 images on a slow drive.
And it took more than a second just to set the watch.
Watching another directory with less files on the same drive was faster.

If I just use inotify-tools on the same dir it is immediate.

My question is:

• It looks like the time it takes to set a watch scales with the number of files in a directory (and perhaps file read speed). But why? Are the files read or stated?
• I'm realy just interested in adding a watch for a directory, not all the files inside it. Note that I'm using NonRecursive. Is that possible?

[0xpr03]

We have to subscribe to each folder individually, so it's listing 6000 files, looking for folders. So yes, it's linear on linux. And on linux you'll have to do that, otherwise you won't get events for nested folders. source

Edit: earlier version stated we're listening on 6000 files, instead we have to look through all files

[markus-bauer]

Thanks for the reply.

The source code explains why it's slow. I assume it's because it's calling metadata/is_dir on all files.

Is this documented? I didn't know that setting a watch needs to stat all files in that dir. That's a lot of work (or it can be in some cases).

And why do you need that?
I looked through https://www.man7.org/linux/man-pages/man7/inotify.7.html
and https://docs.rs/inotify/latest/inotify/struct.EventMask.html
but I still don't know. Superficially, it sound like watching a directory sends events for objects inside that directory.
What would happen if you didn't subscribe to subfolders like you do, and only subscribed to the main dir? What events wouldn't I get?

[0xpr03]

I looked through https://www.man7.org/linux/man-pages/man7/inotify.7.html

       Inotify monitoring of directories is not recursive: to monitor
       subdirectories under a directory, additional watches must be
       created.  This can take a significant amount time for large
       directory trees.

[0xpr03]

That's essentially the problem: We have to check for subdirectories and place watches on them. This is what requires going through the whole directory.

[markus-bauer]

Yes, I obviously did read what you quoted.

It makes sense that you need to do this, if you want to watch a tree or sub-tree. But that not when watching a single directory.

But, I'm not asking you to explain inotify to me. I'm going to trust that you know more about this than me.

---

We have to check for subdirectories and place watches on them. This is what requires going through the whole directory.

What is the NoRecursive option for then? If I set NonRecursive I assume that you set one watch only.

From the documentation:
https://docs.rs/notify/5.0.0-pre.15/notify/enum.RecursiveMode.html#variant.NonRecursive

Indicates whether only the provided directory or its sub-directories as well should be watched
...
Recursive
 Watch all sub-directories as well, including directories created after installing the watch

NonRecursive
 Watch only the provided directory

Is there no way to subscribe to just the directory? I assumed that this is what NonRecursive means.

But you closed this already, so that's that...

[0xpr03]

What is the NoRecursive option for then? If I set NonRecursive I assume that you set one watch only.

Well then yes, then it's not going through the recursive lookup. I would recommend running strace and watching how long the actual linux call takes. My guess is that the linux kernel itself has to do a lot of book keeping for this.

[markus-bauer]

Okay, I did run my example code from above with strace. I also cloned the repo and put in some print statements.

Please not again, that my example is using PollWatcher and NonRecursive.

• Running with strace shows, that there are statx calls for every file in that folder. strace says 98% of the time is spent on statx calls.
• It looks like the source code you linked (

  notify/src/inotify.rs

  Lines 458 to 465 in 1cfcbbb

  	for entry in WalkDir::new(path)
  	.follow_links(true)
  	.into_iter()
  	.filter_map(filter_dir)
  	{
  	self.add_single_watch(entry.path().to_path_buf(), is_recursive, watch_self)?;
  	watch_self = false;
  	}

  ) is not actually run. The metadata calls happen here:
  

  notify/src/poll.rs

  Lines 313 to 327 in 1cfcbbb

  	let depth = if recursive_mode.is_recursive() {
  	usize::max_value()
  	} else {
  	1
  	};
  	for entry in WalkDir::new(watch.clone())
  	.follow_links(true)
  	.max_depth(depth)
  	.into_iter()
  	.filter_map(|e| e.ok())
  	{
  	let path = entry.path();
  	match entry.metadata() {
  	Err(e) => {

  
  And the stat calls happen because walkdir with max_depth(1) is looping over all the files in that dir.
  I don't know if that's on purpose.
• This brings up another question. Is the PollWatcher using inotify?

---

Update:
Here's the answer to my last question:
https://docs.rs/notify/5.0.0-pre.15/notify/poll/index.html

This implementation only uses Rust stdlib APIs and should work on all of the platforms it supports.

Good that we talked about (and you linked to) inotify the whole time.

[0xpr03]

So ok, sorry for missing that initially:
You're using the pollwatcher which, as described, is "Polling based Watcher implementation", so no, it just checks all files regularly.

If you want inotify, use the INotifyWatcher, or as I would recommend: The RecommendedWatcher

[0xpr03]

Good that we talked about (and you linked to) inotify the whole time.

What I don't get is, why did you choose something polling based when you wanted inotify and looked up the man page ?

[markus-bauer]

I missed that part of the documentation and just assumed it's also using inotify. Sorry. I'm going to use RecommendedWatcher. Thanks for your patience.

I didn't want inotify. I wanted a cross platform solution. I only looked up the man page after you linked to inotify.rs.

I used PollWatcher because it sends events only at a certain interval. But now I realize that the delay is a polling delay. It was my fault for not reading the documentation. Sorry again.

[0xpr03]

Ah no worries, I just glanced over the code and assumed you're using the inotify watcher when specifically talking about inotify. The polling approach is only for 3rd party platforms or since #396 to watch syfs/procfs (listed specifically as unsupported by the manpage).

[markus-bauer]

Thanks again and I think we're done here.