From eebe44b9fe1a2a0b2abcefeb6c8147ddbe462ac6 Mon Sep 17 00:00:00 2001 From: Todd Derr Date: Fri, 17 Jun 2022 18:25:55 -0400 Subject: [PATCH 1/4] perf: use watchman's `watch-project` if available If watchman is version 3.3 or later, use `watch-project` rather than the deprecated `watch`. The main advantage is eliminating redundant nested watches (i.e. where one is a subtree of another), which reduces RAM and CPU usage in the `watchman` daemon and eliminates the indexing delay when invoking Command-T after moving down the directory hierarchy. Although `watch-project` was introduced in 3.1 it is awkward/inefficient to use without the `relative_root` query which was introduced in 3.3. 3.3 was released in June 2015, so most users should have a newer version but backward compatibility with older versions is maintained. Fixes: https://github.com/wincent/command-t/issues/389 Tested: * Tested the new code under various scenarios by invoking Command-T in my homedir and various subdirectories, as well as other paths (/tmp). * Manually tested the fallback path to `watch`, which is unchanged from existing code. * Tested various version numbers by manually simulating the output of `--version` * If the version number is missing or malformed, it will end up as version 0.0 * test cases: nil, "", "foo.bar", "-5.2", "1", "3.2", 3.3.449", "4", "2022.06.13.00" See also: - https://facebook.github.io/watchman/docs/cmd/watch.html - https://facebook.github.io/watchman/docs/cmd/watch-project.html Signed-off-by: Greg Hurrell --- .../file_scanner/watchman_file_scanner.rb | 36 ++++++++++++++----- 1 file changed, 27 insertions(+), 9 deletions(-) diff --git a/ruby/command-t/lib/command-t/scanner/file_scanner/watchman_file_scanner.rb b/ruby/command-t/lib/command-t/scanner/file_scanner/watchman_file_scanner.rb index 60f412a7..330bc3c9 100644 --- a/ruby/command-t/lib/command-t/scanner/file_scanner/watchman_file_scanner.rb +++ b/ruby/command-t/lib/command-t/scanner/file_scanner/watchman_file_scanner.rb @@ -25,20 +25,29 @@ def paths! UNIXSocket.open(sockname) do |socket| root = Pathname.new(@path).realpath.to_s - roots = Watchman::Utils.query(['watch-list'], socket)['roots'] - if !roots.include?(root) - # this path isn't being watched yet; try to set up watch - result = Watchman::Utils.query(['watch', root], socket) + # use `watch-project` for efficiency if it's available + if use_watch_project? + result = Watchman::Utils.query(['watch-project', root], socket) + root = extract_value(result, 'watch') + relative_root = extract_value(result, 'relative_path') if result.has_key?('relative_path') + else + roots = Watchman::Utils.query(['watch-list'], socket)['roots'] + if !roots.include?(root) + # this path isn't being watched yet; try to set up watch + result = Watchman::Utils.query(['watch', root], socket) - # root_restrict_files setting may prevent Watchman from working - # or enforce_root_files/root_files (>= version 3.1) - extract_value(result) + # root_restrict_files setting may prevent Watchman from working + # or enforce_root_files/root_files (>= version 3.1) + extract_value(result) + end end - query = ['query', root, { + query_params = { 'expression' => ['type', 'f'], 'fields' => ['name'], - }] + } + query_params['relative_root'] = relative_root if relative_root; + query = ['query', root, query_params] paths = Watchman::Utils.query(query, socket) # could return error if watch is removed @@ -68,6 +77,15 @@ def get_raw_sockname end raw_sockname end + + # watch_project is available in 3.1+ but it's awkward to use without + # relative_root (3.3+), so use the latter as our minimum version. + def use_watch_project? + return @use_watch_project if defined?(@use_watch_project) + version = %x{watchman --version 2>/dev/null} + major, minor = version.split('.')[0..1] if !$?.exitstatus.nil? && $?.exitstatus.zero? && version + @use_watch_project = major.to_i > 3 || (major.to_i == 3 && minor.to_i >= 3) + end end end end From 6792fdf68b97d88066ffe21731452b69f23476ec Mon Sep 17 00:00:00 2001 From: Greg Hurrell Date: Sat, 18 Jun 2022 23:28:39 +0200 Subject: [PATCH 2/4] style: apply comment edits and remove an unnecessary semi-colon --- .../scanner/file_scanner/watchman_file_scanner.rb | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/ruby/command-t/lib/command-t/scanner/file_scanner/watchman_file_scanner.rb b/ruby/command-t/lib/command-t/scanner/file_scanner/watchman_file_scanner.rb index 330bc3c9..5616d8f7 100644 --- a/ruby/command-t/lib/command-t/scanner/file_scanner/watchman_file_scanner.rb +++ b/ruby/command-t/lib/command-t/scanner/file_scanner/watchman_file_scanner.rb @@ -25,7 +25,7 @@ def paths! UNIXSocket.open(sockname) do |socket| root = Pathname.new(@path).realpath.to_s - # use `watch-project` for efficiency if it's available + # Use `watch-project` for efficiency if available. if use_watch_project? result = Watchman::Utils.query(['watch-project', root], socket) root = extract_value(result, 'watch') @@ -33,11 +33,11 @@ def paths! else roots = Watchman::Utils.query(['watch-list'], socket)['roots'] if !roots.include?(root) - # this path isn't being watched yet; try to set up watch + # This path isn't being watched yet; try to set up watch. result = Watchman::Utils.query(['watch', root], socket) - # root_restrict_files setting may prevent Watchman from working - # or enforce_root_files/root_files (>= version 3.1) + # `root_restrict_files` setting may prevent Watchman from + # working or enforce_root_files/root_files (>= version 3.1). extract_value(result) end end @@ -46,7 +46,7 @@ def paths! 'expression' => ['type', 'f'], 'fields' => ['name'], } - query_params['relative_root'] = relative_root if relative_root; + query_params['relative_root'] = relative_root if relative_root query = ['query', root, query_params] paths = Watchman::Utils.query(query, socket) @@ -78,8 +78,8 @@ def get_raw_sockname raw_sockname end - # watch_project is available in 3.1+ but it's awkward to use without - # relative_root (3.3+), so use the latter as our minimum version. + # `watch_project` is available in 3.1+ but it's awkward to use without + # `relative_root` (3.3+), so use the latter as our minimum version. def use_watch_project? return @use_watch_project if defined?(@use_watch_project) version = %x{watchman --version 2>/dev/null} From 4d2569a815e43340d57b733b220d7821d3684c7e Mon Sep 17 00:00:00 2001 From: Greg Hurrell Date: Sat, 18 Jun 2022 23:31:46 +0200 Subject: [PATCH 3/4] refactor: make .watchmanconfig a valid JSON file Added this a long time ago just to help Watchman identify the project root, but current versions don't like it being a zero-byte file: watchman::RootResolveError: failed to resolve root: unable to resolve root /Users/wincent/code/wincent/aspects/nvim/files/.config/nvim/pack/bundle/opt/command-t: failed to parse json from /Users/wincent/code/wincent/aspects/nvim/files/.config/nvim/pack/bundle/opt/command-t/.watchmanconfig: '[' or '{' expected near end of file" --- .watchmanconfig | 1 + 1 file changed, 1 insertion(+) diff --git a/.watchmanconfig b/.watchmanconfig index e69de29b..0967ef42 100644 --- a/.watchmanconfig +++ b/.watchmanconfig @@ -0,0 +1 @@ +{} From 24b717509cf5e5e4e3cf1fc7733e773527ab3f46 Mon Sep 17 00:00:00 2001 From: Greg Hurrell Date: Sat, 18 Jun 2022 23:40:40 +0200 Subject: [PATCH 4/4] docs: update AUTHORS and HISTORY section --- doc/command-t.txt | 58 ++++++++++++++++++++++++++--------------------- 1 file changed, 32 insertions(+), 26 deletions(-) diff --git a/doc/command-t.txt b/doc/command-t.txt index d8dd1f4c..f77047bd 100644 --- a/doc/command-t.txt +++ b/doc/command-t.txt @@ -1413,32 +1413,32 @@ AUTHORS *command-t-authors* Command-T is written and maintained by Greg Hurrell . Other contributors that have submitted patches include, in alphabetical order: - Abhinav Gupta Nate Kane - Adrian Keet Nicholas T. - Aleksandrs Ļedovskis Nicolas Alpi - Alexey Terekhov Nikolai Aleksandrovich Pavlov - Andrius Grabauskas Nilo César Teixeira - Andy Waite Noon Silk - Anthony Panozzo Ole Petter Bang - Artem Nezvigin Patrick Hayes - Ben Boeckel Paul Jolly - Brendan Mulholland Pavel Sergeev - Daniel Burgess Rainux Luo - Daniel Hahler Richard Feldman - David Emett Roland Puntaier - David Szotten Ross Lagerwall - Douglas Drumond Sam Morris - Emily Strickland Scott Bronson - Felix Tjandrawibawa Seth Fowler - Gary Bernhardt Sherzod Gapirov - Henric Trotzig Shlomi Fish - Ivan Ukhov Stefan Schmidt - Jakob Pfender Stephen Gelman - Jeff Kreeftmeijer Steve Herrell - Jerome Castaneda Steven Moazami - Joe Lencioni Steven Stallion - KJ Tsanaktsidis Sung Pae - Kevin Webster Thomas Pelletier + Abhinav Gupta Nicholas T. + Adrian Keet Nicolas Alpi + Aleksandrs Ļedovskis Nikolai Aleksandrovich Pavlov + Alexey Terekhov Nilo César Teixeira + Andrius Grabauskas Noon Silk + Andy Waite Ole Petter Bang + Anthony Panozzo Patrick Hayes + Artem Nezvigin Paul Jolly + Ben Boeckel Pavel Sergeev + Brendan Mulholland Rainux Luo + Daniel Burgess Richard Feldman + Daniel Hahler Roland Puntaier + David Emett Ross Lagerwall + David Szotten Sam Morris + Douglas Drumond Scott Bronson + Emily Strickland Seth Fowler + Felix Tjandrawibawa Sherzod Gapirov + Gary Bernhardt Shlomi Fish + Henric Trotzig Stefan Schmidt + Ivan Ukhov Stephen Gelman + Jakob Pfender Steve Herrell + Jeff Kreeftmeijer Steven Moazami + Jerome Castaneda Steven Stallion + Joe Lencioni Sung Pae + KJ Tsanaktsidis Thomas Pelletier + Kevin Webster Todd Derr Kien Nguyen Duc Ton van den Heuvel Lucas de Vries Victor Hugo Borja Marcus Brito Vlad Seghete @@ -1447,6 +1447,7 @@ Other contributors that have submitted patches include, in alphabetical order: Max Timkovich Yan Pritzker Mike Lundy Zak Johnson Nadav Samet xiaodezhang + Nate Kane This list produced with: @@ -1572,6 +1573,11 @@ POSSIBILITY OF SUCH DAMAGE. HISTORY *command-t-history* +main (not yet released) ~ + +- Teach watchman scanner to favor `watch-project` over `watch` when + available (#390, patch from Todd Derr). + 5.0.4 (28 May 2022) ~ - Support opening files which contain newlines (#365).