Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix for module import issue #360

Merged
merged 1 commit into from
Dec 23, 2019
Merged

Conversation

pbaughman
Copy link
Contributor

  • Try a 'real' import of the test file name before we load the module from source.
    This prevents the source file loader from hiding the real module (if it exists)
    in the sys.modules cache

Signed-off-by: Pete Baughman [email protected]

Closes #357

@pbaughman pbaughman force-pushed the 357_fix_module_import branch from adec9be to a30f4ad Compare December 9, 2019 15:51
@pbaughman
Copy link
Contributor Author

@hidmic Here's the fix. It's more explanation than code

Copy link
Member

@wjwwood wjwwood left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess this is ok? Seems like a hack, but maybe it will help someone avoid a rough edge, though I wouldn't expect launch to do this for you, so it may be surprising either way.

@wjwwood wjwwood requested a review from hidmic December 13, 2019 00:33
@pbaughman
Copy link
Contributor Author

pbaughman commented Dec 13, 2019

@wjwwood launch gives the file a fixed module name "python_launch_file" so unless someone makes a module with the same name, launch will not encounter the same problem.

I don't remember why 'launch' and 'launch_testing' don't use the same load_python_launch_file_as_module method. It's probably the result of launch_testing being developed against a frozen version of 'launch' back when it was apex_launchtest

An alternative fix would be to have launch_testing use the same method as launch to import the test file - maybe import it as python_launch_test instead of python_launch_file, but as mentioned in the issue that will change the names of the tests that get generated.

@hidmic
Copy link
Contributor

hidmic commented Dec 16, 2019

Seems like a hack, but maybe it will help someone avoid a rough edge, though I wouldn't expect launch to do this for you, so it may be surprising either way.

It is a bit of a hack, and I agree that it might silently incur in unwanted side effects. I didn't know of a superior alternative though, for the reasons that @pbaughman already mentioned in #357.

Having said that, after digging a bit further into importlib documentation, there might be a way to import a module while preventing the interpreter from caching it in sys.modules. @pbaughman mind to try this (without actually caching anything)?

@pbaughman
Copy link
Contributor Author

@hidmic I tried doing this:

def _load_python_file_as_module(test_module_name, python_file_path):                 
    """Load a given Python launch file (by path) as a Python module."""              

    spec = importlib.util.spec_from_file_location(test_module_name, python_file_path)
    return importlib.util.module_from_spec(spec)                                     

But now things are blowing up here: https://github.com/ros2/launch/blob/master/launch_testing/launch_testing/loader.py#L154

with an error "module lidar_integration has no attribute 'generate_test_description'

I think I've just reversed the problem. Before, it was looking in the test file for things that should be found in the real module. Now I think it's looking in the real module for things that should be found in the test file.

@pbaughman
Copy link
Contributor Author

Maybe the correct fix here is "don't name your test files the same thing as a real module" and a better warning/error message when you do. . .

@pbaughman
Copy link
Contributor Author

@hidmic Ah, no I was wrong. . . This does it:

def _load_python_file_as_module(test_module_name, python_file_path):                 
    """Load a given Python launch file (by path) as a Python module."""              
    # Taken from launch_testing to not introduce a weird dependency thing            
    spec = importlib.util.spec_from_file_location(test_module_name, python_file_path)
    module = importlib.util.module_from_spec(spec)                                   
    spec.loader.exec_module(module)                                                  
    return module                                                                    

@pbaughman pbaughman force-pushed the 357_fix_module_import branch from a30f4ad to 43f883d Compare December 18, 2019 01:17
Copy link
Contributor

@hidmic hidmic left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM pending green CI

@hidmic
Copy link
Contributor

hidmic commented Dec 20, 2019

CI (up to launch_testing, launch_testing_ros, test_communication)

  • Linux Build Status
  • Linux-aarch64 Build Status
  • macOS Build Status
  • Windows Build Status

@hidmic
Copy link
Contributor

hidmic commented Dec 23, 2019

Alright, all green. Going in!

@hidmic hidmic merged commit e4323a6 into ros2:master Dec 23, 2019
@pbaughman pbaughman deleted the 357_fix_module_import branch December 23, 2019 15:52
ivanpauno pushed a commit that referenced this pull request Jan 17, 2020
ivanpauno pushed a commit that referenced this pull request Jan 20, 2020
Signed-off-by: Pete Baughman <[email protected]>
Signed-off-by: Ivan Santiago Paunovic <[email protected]>
ivanpauno added a commit that referenced this pull request Jan 21, 2020
* Handle case where output buffer is closed during shutdown (#365)

* Handle case where output buffer is closed during shutdown

  - Prevent crash during launch shutdown when a process IO event happens
    after the buffers have been closed
  - Use unbuffered output in that case so IO still has a chance of being seen

Signed-off-by: Pete Baughman <[email protected]>

* Address MR feedback

Signed-off-by: Pete Baughman <[email protected]>
Signed-off-by: Ivan Santiago Paunovic <[email protected]>

* Import test file without contaminating sys.modules (#360)

Signed-off-by: Pete Baughman <[email protected]>
Signed-off-by: Ivan Santiago Paunovic <[email protected]>

* Release loop lock before waiting for it to do work (#369)

Main thread can be blocked trying to acquire
__loop_from_run_thread_lock while emit_event() in another thread is
holding that lock and waiting for the main thread to emit the event.
This change releases the lock before blocking.

Signed-off-by: Shane Loretz <[email protected]>
Signed-off-by: Ivan Santiago Paunovic <[email protected]>

Co-authored-by: Peter Baughman <[email protected]>
Co-authored-by: Shane Loretz <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

launch_testing test file import can cause confusing module import issues
3 participants