UNC - Unable to access sub-directories using dokany mirror sample

[velislav87]

Feature request can skip this form. Bug report must complete it. Check List must be 100% match or it will be automatically closed without further discussion. Please remove this line.

Environment

• Windows version: Windows 10
• Processor architecture: x86
• Dokany version: 1.4.0
• Library type (Dokany/FUSE): Dokany

Check List

• I checked my issue doesn't exist yet
• My issue is valid with mirror default sample and not specific to my user-mode driver implementation
• I can always reproduce the issue with the provided description below.
• I have updated Dokany to the latest version and have reboot my computer after.
• I tested one of the last snapshot from appveyor CI

Description

Hi dokany team,

Using the mirror example with a UNC path e.g. \\Host\Path does not work with sub-directories. Example if we mount a folder as NetworkDrive using the sample mirror and that folder has a folder "test", accessing directly \\Host\Path\test would not work. To my investigation no call for this path comes to the mirror or my implementation. See attached logs for more info.

Best Regards,
Velislav

Logs

Please attach in separate files: mirror output, library logs and kernel logs.
In case of BSOD, please attach minidump or dump analyze output.

Log file: unc_path_logs.zip
Mounted with the command: "mirror.exe /r c:\test /l K /t 5 /d /s /n /u \localhost\myfs"
In it I tried the following:

1. Open the \\localhost\myfs directly from the explorer
2. Tried to open \\localhost\myfs\test from explorer - error Directory not found
3. Tried to open the same trough the browser(by clicking on the folder and not typing the name)
4. Tried again step 2 to check if it might be a cache issue - same error Directory not found

[Liryna]

Hi @velislav87 ,

When you open explorer to \\localhost\myfs do you see the test folder ?
Have you tried \\localhost\myfs\test ? (with double \ at start)
You need to have c:\test\test folder to exist to have \\localhost\myfs\test. Is that the case ?

[velislav87]

Hi @Liryna

Yes, when I am navigating trough the explorer i see see the folder test in \\localhost\myfs. The only issue is when I try directly to open the subfolder with the explorer(\\localhost\myfs\test).
The folder c:\test\test exists, yes - I have created a test textfile into it also.
I am trying with double \ at the start(\\localhost\myfs\test). I will edit the ticket description to represent the same to avoid confusion.

One interesting thing I noticed is that today the issue is not reproducible with \\localhost\myfs\test - explorer does open the folder directly. However when I tried to change the host in the mount command to something new(\\newhost\myfs\test) the issue is reproducible again - into that host I cannot open the sub-folder directly. Quite strange behaviour.

[Liryna]

@velislav87 Thanks for the feedback I can reproduce the issue indeed.

I would suspect the issue to be in the network provider. The dokan_np code. Would need to look at the logs of a debug version with DbgView and see if we are not getting a request for such path which we do not correctly handle.

Is that something you would be able to look at ? I will be able to guide you but I am lacking time to handle that.

[velislav87]

Hi @Liryna , yes I can try to collect the logs with DbgView.
I think I remember a guide on how to do so in the dokany documentation. I will go trough it and get back into the issue.

[velislav87]

Hi @Liryna ,

I am following the guide https://github.com/dokan-dev/dokany/wiki/How-to-Debug-Dokan on how to collect the DBG traces. You may find the same here.

I tried several times accessing folders both using the explorer view and entering the hostname directly. The command for mount is mirror.exe /r c:\test /l K /t 5 /d /n /u \dbghost\myfs.

Hopefully there is something useful in the logs, because I didn't see any logs streaming in when reproducing the issue(they were streaming only when browsing normally).

Best Regards,
Velislav

[Liryna]

It is the dokannp library logs we need to look at and not the mirror or dokan library.
For this you need to build a debug version and register it using dokanctl /I n and be sure it does not use the one install by default.
And then with DbgView you can get the logs.

[velislav87]

Hi @Liryna , thanks I will try to do the same and get back :)

[Liryna]

Looks like this code is called with the subfolder provided. https://github.com/dokan-dev/dokany/blob/master/dokan_np/dokannp.c#L954-L988
Maybe we incorrectly fill the information.

[velislav87]

Hi @Liryna ,

I am following the documentation https://github.com/dokan-dev/dokany/wiki/Build on how to build dokany, however I ran into an issue when compiling the "sys"(I assume the driver). I get the following error:
A WDK corresponding to target version '10.0.18362.0' was not found. Please install this WDK version before building.
I installed the WDK from https://docs.microsoft.com/en-us/windows-hardware/drivers/download-the-wdk. However I see that the latest version is "WDK for Windows 10, version 2004"(I assume newer than the required one by dokany driver).
In the page to download older versions https://docs.microsoft.com/en-us/windows-hardware/drivers/other-wdk-downloads I don't see WDK with the version required by the dokany driver(10.0.18362.0).

Have you encountered such a build issue?

Thanks,
Velislav

[Liryna]

@velislav87 Hi,

The WDK is not needed to build the dokan_np, you can simply use the driver available in the installer but replace the dokan library and dokan_np from system32 with the one you build in debug.
Just remove/disable the sys project from the solution and you will not need the WDK.

[velislav87]

Hi again @Liryna ,

I've successfully did a manual install of the latest driver dokan.sys from the latest release build(debug version of binary) and the dokannp1.dll and dokan1.dll that I've built. I registered them with dokanctl.exe, set the debug to on "dokanctl /d 7", however I don't see any logs streaming in WinDbg program. The mount is successful, however no debug is printed at all. I tried removing all the dokan drivers and dlls, restarting and then re-installing manually, but to no avail.

Best Regards,
Velislav

[Liryna]

Hey,

Have you tried to use DebugView and not WinDbg ?
You do not need to run dokanctl /d 7, this is only to enable the driver logs.
Here dokannp logs are directly enabled when the debug version of the dll is used by explorer.

[velislav87]

I tried using DebugView - again no logs are streamed.
I am using the dll built in the directory dokany-1.4.0.1000\x64\Debug - dokan1.dll and dokannp1.dll. I've downloaded the latest source code zip file and built the binaries from there. Is this correct or am I missing something?

[Liryna]

@velislav87 This should be good yes. Unless this dokannp is not used by explorer or DbgView configuration is not good. I have not other idea.

[velislav87]

@Liryna Thanks for the tips. Might be something I messed up or missed in the process. I will try to spin-up a clean windows 10 VM and see if this helps.

[velislav87]

Hi @Liryna , I think after re-trying it on a new VM that I managed to collect some logs with debug output. I did a quick search in them and there seems to be dokannp logs there also. Hopefully they are useful :)

One message I see that might be connected to the issue is :

00003979	63.32384109	[4776] NPAddConnection3	
00003980	63.35470200	[4776]   LocalName: (null)	
00003981	63.38486481	[4776]   RemoteName: \\newhost\myfs	
00003982	63.50526047	[4776]   WN_BAD_NETNAME	

The function output is reminiscent of the error I get in windows also.
Link to logs: DESKTOP-CBNDDGJ.LOG

Edit:
Adding a DESKTOP-CBNDDGJ_new.LOG(new set of logs) to a new set of logs where I only execute the following:

1. Open \\testhost\myfs\test -> Error
2. Open \\testhost\myfs -> Success

In the first set I browsed more in the directory structure and with the second one reviewing might be more easy.

[velislav87]

Hi @Liryna , did you manage to see the new sets of debug logs? Do they contain the necessary output? Please let me know if I can help out with anything else!

[Liryna]

Thanks for pinging me @velislav87 I kind of forgot that issue
I am surprised to see in the driver logs request for \testhost\myfs we should not have the driver requested for such files. Feels like it is really the dokannp that badly convert the UNC path to local path.

I do not have much time to dedicate to dokannp, do you feel you are able to review the dokannp code and see what should be fixed / changed ?

[the-Arioch]

@Liryna what do you think about some crazy idea that you (dokan team) set some ready to be used machine, so to buid both release and debug versions of drivers would be as easy as RDP/VNC/AnyDesk/TeamViewer in and gitting changes in and pressing few buttons? You can not provide VMs for copyright reasons, but, you now, learning curve of setting up new totally unknown environments might be a beast in itself.

For example. in the "new log" above there are rather damning lines.

00000342	19.89424133	[5040] NPAddConnection3	
00000343	19.89431381	[5040] NPGetResourceInformation: NetResource 000000000746DCD0, Buffer 0000000013437920, BufferSize 000000000746DB70, System 000000000746DB68	
00000344	19.89434433	[5040]   LocalName: (null)	
00000345	19.89438248	[5040] NPGetResourceInformation: lpRemoteName \\testhost, *BufferSize 0x830	
00000346	19.89440346	[5040]   RemoteName: \\testhost\myfs	
00000347	19.89443016	[5040] parseServerName: cLeadingBackslashes 2	
00000348	19.89446259	[5040]   WN_BAD_NETNAME	
00000349	19.89448738	[5040] NPGetResourceInformation: lpServerName testhost - 8	
00000350	19.89451981	[5040] NPGetResourceInformation: lpAfterName 	
00000351	19.89454651	[5040] NPGetResourceInformation: type 1	
00000352	19.89456749	[5040] NPGetResourceInformation: lpRemoteName: \\testhost, strings 0000000013438150/0000000013438150	

What is do daming? Well, if you look into line 202

dokany/dokan_np/dokannp.c

Line 202 in b5edb0b

DbgPrintW(L"NPAddConnection3\n");

You see that is just impossible output. And then you understand those are different threads writing at the same time with no distinction!!! Basically such a log is very confusing and less useful than it has to be.

Fix seem be trivial. Replace DbgPrintW with an inline function or macro that would call GetCurrentThreadId() and prepend debug line with a marker like "<th:1234> ".
Any code monkey can do it in Notepad. But code monkey would have troubles climbing up that learning curve of setting up unknown environment properly and even finding a correct button to click... Dead end.

I am surprised to see in the driver logs request for \\testhost\myfs we should not have the driver requested for such files.

And i do not see any at all.
To my uninformed guess, driver's output is what marked with [DokanFS] and there is no any line having both that and \\
All the \\ lines happens with [5040] which i believe to be userland ProcessID ?

[Liryna]

@the-Arioch The project is configured with the appveyor CI. You can fork the project, open a pull request and it will build an installer that you can download.

You could also just fork and subscribe your project to appveyor on your side to not have to open a pull request.

[the-Arioch]

Hmmm.... Sounds like all those mainframes of 1960's, when programmers were giving punch cards to one window at morning, and got rolls of printed paper for another at dinner (either output or errors) :-)
Frankly, i was about 8 y.o. when did something like that, without an interactive IDE :-)))

Would have to google about look into such a workflow, thanks for the hint.

Is there a good crashcourse on it ?

[the-Arioch]

I made a fork and will see if it would work out.

There are lines in the log like

00007729	53.44504166	[DokanFS] Try acquire SpinLock...

I found DbgPrintW and friends in sources, but nothing like [DokanFS].
Am i right that this ID is not in sources, but somehow set into registry in the process you described at https://github.com/dokan-dev/dokany/wiki/How-to-Debug-Dokan ?
And there is just nothing for me to find :-)

[the-Arioch]

@Liryna did you try https://pvs-studio.com/en/order/open-source-license/ ?
I am a bit annoyed by warnings like potential data loss on type conversion.
Usually it is just a noise given expected data, but occasionally real problem alerts are lost in that noise...

Also, would it be easy to make a non-WiX zip artifact?
Skipping installer and reboot would make little sense if the driver itself is to be recompiled.
But if only dokan1.dll or mirror.exe got changed, then creating and then manually unpacking setup .exe and then manually unpacking .msi - that all seems quite redundant.

[the-Arioch]

i installed Win7AndW2K8R2-KB3191566-x64.msu

still

powershell.exe  $PSVersionTable.PSVersion

Major  Minor  Build  Revision
-----  -----  -----  --------
5      1      14409  1018

Powershell.exe -executionpolicy remotesigned -File .\cert\dokan-import.ps1
Restarting in an Admin prompt

Windows PowerShell
Copyright (C) 2016 Microsoft Corporation. All rights reserved.

Looking for CertMgr
Import Dokan certificate
Import-Certificate : The term 'Import-Certificate' is not recognized as the nam
e of a cmdlet, function, script file, or operable program. Check the spelling o
f the name, or if a path was included, verify that the path is correct and try
again.

ans same on Import-PfxCertificate

funny that then script said it executed succesfully

C:\>powershell Get-Command -Module PKI

C:\>

P.S. https://www.powershelladmin.com/wiki/Import_PFX_certificates_on_remote_servers_(PSv2-compatible)

[the-Arioch]

i hopefully managed to install them certs maunally, but thing is, the window7/2008 how-to does not work

[the-Arioch]

Plus, CI-built installer can not install upon itself, there is not forced update / repair mode, only de-install and re-install :-(

[the-Arioch]

Something cursed indeed.

C:\Program Files\Dokan\Dokan Library-1.5.0\sample\mirror>mirror /l d:\SSH_xws\mi
nefield\tests\dok_GH_945 /d /n /u \\dokan\myfs
Can't assign a drive letter

C:\Program Files\Dokan\Dokan Library-1.5.0\sample\mirror>mirror /l d:\SSH_xws\mi
nefield\tests\dok_GH_945 /d /n /u \\dokan\myfs /l n

C:\Program Files\Dokan\Dokan Library-1.5.0\sample\mirror>
C:\Program Files\Dokan\Dokan Library-1.5.0\sample\mirror>mirror /l d:\SSH_xws\mi
nefield\tests\dok_GH_945 /d /n /u \\dok-test\myfs /l n

C:\Program Files\Dokan\Dokan Library-1.5.0>dokanctl.exe /l
Driver path: 'C:\Windows\system32\drivers\dokan1.sys'
  Mount points: 1
  0# MountPoint: \DosDevices\N: - UNC: \\dok-test\myfs - DeviceName: \Device\Dok
anRedirector1{d6cc17c5-1736-4085-bce7-964f1e9f5de9}

P.S. i wonder if UNC "network path" can be made without mounting drive letter.

I noticed before, that Total Commander (probably because it is old program gradually evolving since Windows 16-bit times) uses network drives in some manner dissimilar to modern Explorer, it manifests for example in logging into network shares with saved credentials, ones saved by TC are not seen by Explorer...

Truly something unexpected comes w.r.t. pretend server name

[the-Arioch]

C:\Program Files\Dokan\Dokan Library-1.5.0\sample\mirror>mirror /l d:\SSH_xws\mi
nefield\tests\dok_GH_945 /d /n /u dok-test\myfs /l n

C:\Program Files\Dokan\Dokan Library-1.5.0>dokanctl.exe /l
Driver path: 'C:\Windows\system32\drivers\dokan1.sys'
  Mount points: 1
  0# MountPoint: \DosDevices\N: - UNC: dok-test\myfs - DeviceName: \Device\Dokan
Redirector1{d6cc17c5-1737-4085-bce7-964f1e9f5de9}

Drive N: was "mounted" to C:\ instead of the given path! And is accessible.
UNC share is not.

UPDATE. Disregard parts about drive C:\ - it is my blunder with command line.

Look at how cursed names are. No any sanytization done and effects looks quite dangerous to me, at least security/repeatability wise. Driver should not act like that whatever trash a userland app feeds to it...

---

Does anyone know what an intention was ???
A c ommand line having only single backslash for UNC name seems counter-intuitive to me, if it specifies server name. Then it better has two bslashes, or optionally non at all. But single one defies conventions and common sense.
.....unless the name was meant to be prepend to the local computer, so /u \test\subfolder actually becoming \\.\test\subfolder and \\localhost\test\subfolder but dokanctl built-in doc seems to disapprove it

[the-Arioch]

C:\Program Files\Dokan\Dokan Library-1.5.0\sample\mirror>mirror /l d:\SSH_xws\mi
nefield\tests\dok_GH_945 /d /n /u \dok-test\myfs /l n /t 1

C:\Program Files\Dokan\Dokan Library-1.5.0>dokanctl.exe /l
Driver path: 'C:\Windows\system32\drivers\dokan1.sys'
  Mount points: 1
  0# MountPoint: \DosDevices\N: - UNC: \dok-test\myfs - DeviceName: \Device\Doka
nRedirector1{d6cc17c5-1730-4085-bce7-964f1e9f5de9}

C:\Program Files\Dokan\Dokan Library-1.5.0\sample\mirror>dir /l d:\SSH_xws\minef
ield\tests\dok_GH_945
 Том в устройстве D имеет метку Data
 Серийный номер тома: 05FE-F8ED

 Содержимое папки d:\SSH_xws\minefield\tests\dok_GH_945

26.09.2021  15:06    <DIR>          .
26.09.2021  15:06    <DIR>          ..
26.09.2021  15:06    <DIR>          subfol
               0 файлов              0 байт
               3 папок  907 674 238 976 байт свободно

This way i can access the UNC "share".

Log tagging is partially succesfull though.
Expected as i only did it with Unicode output, have to lurk more for mor outputs.

[5788] ###Close 0030
[5788] Close: C:\MSOCache
[5788] 
[5788] index from 1
[5788]  <th:5680> FileMatch? : . (*,1,0)
[5788]  <th:5680> FileMatch? : .. (*,1,1)
[5788]   =>return
[5788]  <th:5680> FileMatch? : $Recycle.Bin (*,1,2)
[5788]   =>return
[5788]  <th:5680> FileMatch? : .rnd (*,1,3)
[5788]   =>return
[5788]  <th:5680> FileMatch? : Android (*,1,4)
[5788]   =>return
[5788]  <th:5680> FileMatch? : codetyphon (*,1,5)

What about trailing slaahes?

C:\Program Files\Dokan\Dokan Library-1.5.0\sample\mirror>mirror /l d:\SSH_xws\mi
nefield\tests\dok_GH_945\ /d /n /u \dok-test\myfs /l n /t 1

Seems no change.

C:\Program Files\Dokan\Dokan Library-1.5.0\sample\mirror>mirror /l d:\SSH_xws\mi
nefield\tests\dok_GH_945 /d /n /u \dok-test\myfs\ /l n /t 1

Oooops!

UNC again inaccessible and names are damned.

---

@Liryna why you still using ASCIIZ strings in userland code? maybe in drivers it would be risky to use templates, dunno. ATL, STL, boost, whatever? But in userland??? All those wstr-this, wscp-that... Ain't that stuff crazy and badly time consuming???

[the-Arioch]

OK, concept proved: using CI as a rented compiler is possible, albeit has a number of anti-features.

...and the first thing i found in AppVeyor the first hour we met was a bug. As usual with new software...

Maybe @Liryna can do somethign about pints above:

• generating plain old zip snapshot along WiX automagical installer
• letting WiX install be reinstalled upon itself without de-installing first

Hopefully i'll give some more time to it in October.

Last test notice:

Two last mirror commands only differ in trailing slash in UNC.
And in behavior.

The latter make the share accessible as seen on the screenshot. However the "server name" it totally ruined!
The former suddenly asks for login and password, then gives no access.

Unmounting does not work either.

C:\Program Files\Dokan\Dokan Library-1.5.0>net use
Новые подключения будут запомнены.

Состояние   Локальный  Удаленный                 Сеть
-------------------------------------------------------------------------------
   --- skip ---
             N:        \\dok-test\myfs\          Dokan1
   --- skip ---
Команда выполнена успешно.

C:\Program Files\Dokan\Dokan Library-1.5.0>net use n: /del
n: успешно удален.   --- SUCCESFULLY DELETED

C:\Program Files\Dokan\Dokan Library-1.5.0>dokanctl.exe /l
Driver path: 'C:\Windows\system32\drivers\dokan1.sys'
  Mount points: 1
  0# MountPoint: \DosDevices\N: - UNC: \dok-test\myfs\ - DeviceName: \Device\Dok
anRedirector1{d6cc17c5-1739-4085-bce7-964f1e9f5de9}

C:\Program Files\Dokan\Dokan Library-1.5.0>net use
Новые подключения будут запомнены.

Состояние   Локальный  Удаленный                 Сеть
-------------------------------------------------------------------------------
   --- skip ---
             N:        \\dok-test\myfs\          Dokan1
   --- skip ---
Команда выполнена успешно.

I tried breaking the "network connection" to N: to trigger Windows asking me for network password again, to no avail.

[Liryna]

1 -

00007729 53.44504166 [DokanFS] Try acquire SpinLock...

This feels like a log from a previous version. Any idea why ?

2 -

did you try https://pvs-studio.com/en/order/open-source-license/ ?

I do not know this one but our CI is already attached to similar tools Sonar and Codacy
They nice for userland code but for the drive they are just fine. Only the Visual studio analyzer actually know the magic behind the Kernel API. If you build on Debug you will see the output of the VS analyzer.

3 -

Also, would it be easy to make a non-WiX zip artifact?

That's a good idea to have the zip part of the artifacts! Please see c4e62f7 and especially e1cef6e if you want to build in Debug, just change the config in your fork

4 - For the Win7 cert install issue, you should normally get an error 🤔 See #342

5 -
But single one defies conventions and common sense. That's the format the Kernel API is requesting. We could ask another format and then convert it to a single backslash but that work would only be for the good of the user.

6 -

why you still using ASCIIZ strings in userland code?

We should not 😮 Could you point that code ? Please open an issue if there are too much

7 -

letting WiX install be reinstalled upon itself without de-installing first

This is due to how the installer is used in your case. Normally have one installer per driver release so you would need a reboot anyway. (I am adding the zip artifact to solve this use case)

8 - Arf....the trailing backslash issue should have its own issue :( it is pretty bad. Nice catch!

[the-Arioch]

Hello. I fell we better go out. To a maillist or seomhing, because issues started intertwining and broading :-)

4. i'll see if i can make sense of it. I saw certmgr mentioned in the source.
   At very least i saw red error lines in console and PKI modle clearly was not installed.

I have PowershellPlus IDE installed but never actually made use of it.
Maybe it is a chance ot expand my toolbox.
OTOH i have grim feeling about my current employment so equally possible i would soon have neither win7, not a need for SSHFS

5. okay, i would if i meet it. Just to clarify, by ASCIIZ i meant all zero-ended ones including WHAR* (those technically are UTF16-Z), sorry if i confused you.

Am i right that everything that is not in sys\ is userland ?

That said, i believe all *.c files would use them by necessity, as any template requires use of C++ not C and i do not think plain C has anything else easy and safe to use to replace ???char*

DbgPrint family however clearly are userland, see the-Arioch@48021a0 and prior commits.
You can see Google, MSDN, StackOverflow and me ganged to implemented thread-tagging in DbgPrintW - but there also is a non-Unicode macro, and the log shows there are many non-changed lines (frankly, there is no warranty ALL Dokan's debug output going through those macros.

So it a proof of concept that my codemonkeying works with CI, it is yet to be extended though:

• implement same for non-Unicode macro at least
• revieweing code to see other routes toward OutpuDebugString
• i came to think Dokan has to have yet one more marker, so that system-global or even app-global log could be grepped to clear out all non-Dokan messages, but would be merely changing two constants.

2. PVS Studio actively advertizes themselves printing in Russian and English their reviews of well-known opensource projects, i think you can find any and read and see if they are any better than ones you know.

3. [DokanFS]

This feels like a log from a previous version. Any idea why ?

But it is! It is not my but from that zq-something guy's log in another issue. I think that is how DmgView marked kernel driver log, afterall you probably don't have ProcessIDs insise the kernel :-)

7. Normally have one installer per driver release so you would need a reboot anyway

Mmmm... no.
Or am i missing something?

See, the reboot requirement is about drivers that patch the kernel (install hooks) and thus can not be cleanly unloaded. Those drivers are load-only. Today they are considered obsolete and replaced with WDM and mini-filters and stuff that can be loaded/unloaded cleanly on demand. And i believe that is exactly what happens with dokan driver, when it is unloaded the kernel becomes "clean" and you can replace dokan1.sys file with another version and load that new one. Am i wrong? is there some remain of the unloaded driver in the kernel that requires reboot on upgrade still ?

As to my [ab]using the installer i can only say that there are packages like Microsoft Office or Thales HASP drivers that have "repair" mode, which iu believe usualyl means just reinstalling upon itself, fixing corruptions as a byproduct. I don't see such a function required for Dokan, but don't think it harmful either.

3. thank you, i would use it next time i am diving into it :-)

[Liryna]

I am now able to directly access a subfolder like \\unc\test\sub_folder in explorer with
7f5afc7