Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make sure temporary file handles are not inherited by child processes #755

Merged
merged 2 commits into from
May 26, 2016

Conversation

bwijen
Copy link

@bwijen bwijen commented May 12, 2016

When testing a merge driver which spawns a merge server (for future merges)
I got the following error:

Rename from 'xxx/.git/index.lock' to 'xxx/.git/index' failed. Should I try again? (y/n)

Only after I stop the merge server the lock is released.
This is caused by windows handle inheritance

Starting childs with bInheritHandles==FALSE does not work.
(For some reason the merge commit window did not pop up.)

However creating the handle with bInheritHandle==FALSE does seem to work.
Although I think this should not break anything, it would be great to get some feedback on this.

mcve:

git init

cat > mergetest.sh << EOF
#!/bin/sh
echo mergetest.sh

sleep 60 &
EOF

git config --local merge.mergetest.driver "./mergetest.sh %O %A %B %P"

echo "*.test merge=mergetest" > .gitattributes
echo first line > fake.test

git add .
git commit -m "initial"

git checkout -b other master
echo other change >> fake.test
git add fake.test
git commit -m "other"

git checkout -b this master
echo this change >> fake.test
git add fake.test
git commit -m "this"

#now try: git merge other

@dscho
Copy link
Member

dscho commented May 13, 2016

When testing a merge driver which spawns a merge server (for future merges)
I got the following error:

Rename from 'xxx/.git/index.lock' to 'xxx/.git/index' failed. Should I try again? (y/n)

Only after I stop the merge server the lock is released.
This is caused by windows handle inheritance

Starting childs with bInheritHandles==FALSE does not work.
(For some reason the merge commit window did not pop up.)

However creating the handle with bInheritHandle==FALSE does seem to work.

Two comments:

  • this information really needs to go into the commit message.
  • we need to find out why the latter method does not work, and document our findings in the commit message.

I also have to admit that I am reluctant to change all open() calls wholesale. In other words, the commit message does not convince me that it is a safe change.

mcve:

Do you think you can turn that into a patch to, say, t7606-merge-custom.sh (without the sleep, of course)?

@bwijen bwijen changed the title Make sure file handles are not inherited by child processes Make sure temporary file handles are not inherited by child processes May 19, 2016
@bwijen
Copy link
Author

bwijen commented May 19, 2016

Did some more testing... Using O_NOINHERIT for all files was indeed to much.
Since git-upload-pack for example, expects inherited handles.

Added a test scenario and a fix for the failed test, now only for temporary files.

@bwijen
Copy link
Author

bwijen commented May 24, 2016

@dscho: Any comments on my previous attempt?

@dscho
Copy link
Member

dscho commented May 25, 2016

I was a bit busy trying to set up some automatic testing. It's still too slow (3.5h for any single run) but at least it showed that your branch passes the test suite: https://gitlab.com/git-for-windows1/git/commit/9251e5b7bad7db1f6541d4c2ac87894546ee4867/builds

chmod +x ./custom-merge

test_expect_success 'custom merge backend with delay' '
git reset --hard anchor &&

This comment was marked as off-topic.

This comment was marked as off-topic.

@dscho
Copy link
Member

dscho commented May 25, 2016

Interesting approach. I am not really certain that this is safe, but I guess it is correct code. It certainly passes the test suite now which is necessary to give me a little peace of mind.

Could you address my concerns with the test script? After that I will gratefully merge this PR.

bwijen added 2 commits May 26, 2016 09:21
This test was added to verify the behaviour of merge
when a merge server is spawned.

Because file handles are inherited by child processes,
the index.lock is also inherited and - on win32 - can't
be released until the child has stopped.

Signed-off-by: Ben Wijen <[email protected]>
When testing a merge driver which spawns a merge server (for future merges)
I got the following error:

    Rename from 'xxx/.git/index.lock' to 'xxx/.git/index' failed. Should I try again? (y/n)

Only after I stop the merge server the lock is released.
This is caused by windows handle inheritance.

Starting childs with bInheritHandles==FALSE does not work,
because no file handles would be inherited,
not even the hStdXxx handles in STARTUPINFO.

Opening every file with O_NOINHERIT does not work,
Since it is used by git-upload-pack for example,
which expects inherited handles.

This leaves us with only creating temp files with the O_NOINHERIT flag.
Which (currently) only used by lock_file which is exactly what we want.


Signed-off-by: Ben Wijen <[email protected]>
@bwijen
Copy link
Author

bwijen commented May 26, 2016

I changed the test-script as requested.
Also rebased to the latest version.

Furthermore I also tested on a (virtual) linux box, which shows no failed tests...

@dscho
Copy link
Member

dscho commented May 26, 2016

Thank you!

@dscho dscho merged commit 2db0641 into git-for-windows:master May 26, 2016
dscho added a commit to git-for-windows/build-extra that referenced this pull request May 26, 2016
Child processes [no longer inherit handles to temporary
files](git-for-windows/git#755), which
previously could prevent `index.lock` from being deleted.

Signed-off-by: Johannes Schindelin <[email protected]>
dscho added a commit that referenced this pull request Jun 7, 2016
Prevent child processes from inheriting a handle to index.lock
dscho added a commit that referenced this pull request Jun 9, 2016
Prevent child processes from inheriting a handle to index.lock
dscho added a commit that referenced this pull request Jun 9, 2016
Prevent child processes from inheriting a handle to index.lock
dscho added a commit that referenced this pull request Jun 9, 2016
Prevent child processes from inheriting a handle to index.lock
dscho added a commit that referenced this pull request Jun 9, 2016
Prevent child processes from inheriting a handle to index.lock
dscho added a commit that referenced this pull request Jun 14, 2016
Prevent child processes from inheriting a handle to index.lock
dscho added a commit that referenced this pull request Jul 12, 2016
Prevent child processes from inheriting a handle to index.lock
dscho added a commit that referenced this pull request Jul 16, 2016
Prevent child processes from inheriting a handle to index.lock
dscho added a commit that referenced this pull request Aug 10, 2016
Prevent child processes from inheriting a handle to index.lock
dscho added a commit that referenced this pull request Aug 10, 2016
Prevent child processes from inheriting a handle to index.lock
dscho added a commit that referenced this pull request Aug 11, 2016
Prevent child processes from inheriting a handle to index.lock
dscho added a commit that referenced this pull request Aug 11, 2016
Prevent child processes from inheriting a handle to index.lock
dscho added a commit that referenced this pull request Aug 12, 2016
Prevent child processes from inheriting a handle to index.lock
dscho added a commit that referenced this pull request Aug 12, 2016
Prevent child processes from inheriting a handle to index.lock
dscho added a commit that referenced this pull request Aug 13, 2016
Prevent child processes from inheriting a handle to index.lock
dscho pushed a commit that referenced this pull request Aug 13, 2016
Prevent child processes from inheriting a handle to index.lock
dscho pushed a commit that referenced this pull request Aug 13, 2016
Prevent child processes from inheriting a handle to index.lock
dscho pushed a commit that referenced this pull request Aug 13, 2016
Prevent child processes from inheriting a handle to index.lock
dscho pushed a commit that referenced this pull request Aug 15, 2016
Prevent child processes from inheriting a handle to index.lock
dscho pushed a commit that referenced this pull request Aug 15, 2016
Prevent child processes from inheriting a handle to index.lock
dscho added a commit that referenced this pull request Aug 15, 2016
Prevent child processes from inheriting a handle to index.lock
dscho added a commit that referenced this pull request Aug 15, 2016
Prevent child processes from inheriting a handle to index.lock
dscho pushed a commit that referenced this pull request Aug 20, 2016
Prevent child processes from inheriting a handle to index.lock
dscho pushed a commit that referenced this pull request Aug 20, 2016
Prevent child processes from inheriting a handle to index.lock
dscho added a commit that referenced this pull request Aug 22, 2016
Prevent child processes from inheriting a handle to index.lock
dscho added a commit that referenced this pull request Aug 22, 2016
Prevent child processes from inheriting a handle to index.lock
dscho pushed a commit that referenced this pull request Aug 25, 2016
Prevent child processes from inheriting a handle to index.lock
dscho pushed a commit that referenced this pull request Aug 25, 2016
Prevent child processes from inheriting a handle to index.lock
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants