-
-
Notifications
You must be signed in to change notification settings - Fork 1.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
PR: Fix Windows tunneling error when connecting to kernels (Remote client) #22223
PR: Fix Windows tunneling error when connecting to kernels (Remote client) #22223
Conversation
Gave this a quick check on Windows and I'm being able to connect to the server on first try 👍 However, I'm getting an error related with the Probably the |
Exactly. I've just updated |
9ab05c6
to
9eadbc9
Compare
51f29d5
to
0995ed5
Compare
…hen restarting kernels
451a604
to
c89a473
Compare
- That's to be more consistent with the naming used in Jupyter. - Also, since this is a supporting class, move it to be before the SpyderKernelClient one.
- That signal doesn't require anymore two signatures, so we can remove the unused one. - Also, use functools.partial to simplify the code for a slot.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I tested this on my Windows VM and it's working as expected. Great work @hlouzada!
Gave this another check and seems like launching a console, restarting the kernel and removing variables works 👍 However, I was able to trigger an error by stopping the server connection while having multiple consoles open. I got the following traceback: exception calling callback for <Future at 0x2121550c3d0 state=finished raised ClientConnectorError>
Traceback (most recent call last):
File "C:\Users\dalth\anaconda3\envs\spyder-dev\lib\site-packages\aiohttp\connector.py", line 1025, in _wrap_create_connection
return await self._loop.create_connection(*args, **kwargs)
File "C:\Users\dalth\anaconda3\envs\spyder-dev\lib\asyncio\base_events.py", line 1065, in create_connection
raise exceptions[0]
File "C:\Users\dalth\anaconda3\envs\spyder-dev\lib\asyncio\base_events.py", line 1050, in create_connection
sock = await self._connect_sock(
File "C:\Users\dalth\anaconda3\envs\spyder-dev\lib\asyncio\base_events.py", line 961, in _connect_sock
await self.sock_connect(sock, address)
File "C:\Users\dalth\anaconda3\envs\spyder-dev\lib\asyncio\proactor_events.py", line 703, in sock_connect
return await self._proactor.connect(sock, address)
File "C:\Users\dalth\anaconda3\envs\spyder-dev\lib\asyncio\windows_events.py", line 817, in _poll
value = callback(transferred, key, ov)
File "C:\Users\dalth\anaconda3\envs\spyder-dev\lib\asyncio\windows_events.py", line 604, in finish_connect
ov.getresult()
ConnectionRefusedError: [WinError 1225] El equipo remoto rechazó la conexión de red
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "C:\Users\dalth\anaconda3\envs\spyder-dev\lib\concurrent\futures\_base.py", line 330, in _invoke_callbacks
callback(self)
File "E:\Acer\Documentos\Spyder\Spyder otros\hlouzada\spyder\spyder\api\asyncdispatcher.py", line 128, in _callback_task_done
raise exception
File "E:\Acer\Documentos\Spyder\Spyder otros\hlouzada\spyder\spyder\plugins\remoteclient\plugin.py", line 316, in _shutdown_kernel
await client.terminate_kernel(kernel_id)
File "E:\Acer\Documentos\Spyder\Spyder otros\hlouzada\spyder\spyder\plugins\remoteclient\api\client.py", line 719, in terminate_kernel
response = await jupyter.delete_kernel(kernel_id=kernel_id)
File "E:\Acer\Documentos\Spyder\Spyder otros\hlouzada\spyder\spyder\plugins\remoteclient\api\jupyterhub\__init__.py", line 343, in delete_kernel
async with self.session.delete(
File "C:\Users\dalth\anaconda3\envs\spyder-dev\lib\site-packages\aiohttp\client.py", line 1197, in __aenter__
self._resp = await self._coro
File "C:\Users\dalth\anaconda3\envs\spyder-dev\lib\site-packages\aiohttp\client.py", line 581, in _request
conn = await self._connector.connect(
File "C:\Users\dalth\anaconda3\envs\spyder-dev\lib\site-packages\aiohttp\connector.py", line 544, in connect
proto = await self._create_connection(req, traces, timeout)
File "C:\Users\dalth\anaconda3\envs\spyder-dev\lib\site-packages\aiohttp\connector.py", line 944, in _create_connection
_, proto = await self._create_direct_connection(req, traces, timeout)
File "C:\Users\dalth\anaconda3\envs\spyder-dev\lib\site-packages\aiohttp\connector.py", line 1257, in _create_direct_connection
raise last_exc
File "C:\Users\dalth\anaconda3\envs\spyder-dev\lib\site-packages\aiohttp\connector.py", line 1226, in _create_direct_connection
transp, proto = await self._wrap_create_connection(
File "C:\Users\dalth\anaconda3\envs\spyder-dev\lib\site-packages\aiohttp\connector.py", line 1033, in _wrap_create_connection
raise client_error(req.connection_key, exc) from exc
aiohttp.client_exceptions.ClientConnectorError: Cannot connect to host 127.0.0.1:52429 ssl:default [El equipo remoto rechazó la conexión de red]
exception calling callback for <Future at 0x212154f7250 state=finished raised ClientConnectorError>
Traceback (most recent call last):
File "C:\Users\dalth\anaconda3\envs\spyder-dev\lib\site-packages\aiohttp\connector.py", line 1025, in _wrap_create_connection
return await self._loop.create_connection(*args, **kwargs)
File "C:\Users\dalth\anaconda3\envs\spyder-dev\lib\asyncio\base_events.py", line 1065, in create_connection
raise exceptions[0]
File "C:\Users\dalth\anaconda3\envs\spyder-dev\lib\asyncio\base_events.py", line 1050, in create_connection
sock = await self._connect_sock(
File "C:\Users\dalth\anaconda3\envs\spyder-dev\lib\asyncio\base_events.py", line 961, in _connect_sock
await self.sock_connect(sock, address)
File "C:\Users\dalth\anaconda3\envs\spyder-dev\lib\asyncio\proactor_events.py", line 703, in sock_connect
return await self._proactor.connect(sock, address)
File "C:\Users\dalth\anaconda3\envs\spyder-dev\lib\asyncio\windows_events.py", line 817, in _poll
value = callback(transferred, key, ov)
File "C:\Users\dalth\anaconda3\envs\spyder-dev\lib\asyncio\windows_events.py", line 604, in finish_connect
ov.getresult()
ConnectionRefusedError: [WinError 1225] El equipo remoto rechazó la conexión de red
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "C:\Users\dalth\anaconda3\envs\spyder-dev\lib\concurrent\futures\_base.py", line 330, in _invoke_callbacks
callback(self)
File "E:\Acer\Documentos\Spyder\Spyder otros\hlouzada\spyder\spyder\api\asyncdispatcher.py", line 128, in _callback_task_done
raise exception
File "E:\Acer\Documentos\Spyder\Spyder otros\hlouzada\spyder\spyder\plugins\remoteclient\plugin.py", line 316, in _shutdown_kernel
await client.terminate_kernel(kernel_id)
File "E:\Acer\Documentos\Spyder\Spyder otros\hlouzada\spyder\spyder\plugins\remoteclient\api\client.py", line 719, in terminate_kernel
response = await jupyter.delete_kernel(kernel_id=kernel_id)
File "E:\Acer\Documentos\Spyder\Spyder otros\hlouzada\spyder\spyder\plugins\remoteclient\api\jupyterhub\__init__.py", line 343, in delete_kernel
async with self.session.delete(
File "C:\Users\dalth\anaconda3\envs\spyder-dev\lib\site-packages\aiohttp\client.py", line 1197, in __aenter__
self._resp = await self._coro
File "C:\Users\dalth\anaconda3\envs\spyder-dev\lib\site-packages\aiohttp\client.py", line 581, in _request
conn = await self._connector.connect(
File "C:\Users\dalth\anaconda3\envs\spyder-dev\lib\site-packages\aiohttp\connector.py", line 544, in connect
proto = await self._create_connection(req, traces, timeout)
File "C:\Users\dalth\anaconda3\envs\spyder-dev\lib\site-packages\aiohttp\connector.py", line 944, in _create_connection
_, proto = await self._create_direct_connection(req, traces, timeout)
File "C:\Users\dalth\anaconda3\envs\spyder-dev\lib\site-packages\aiohttp\connector.py", line 1257, in _create_direct_connection
raise last_exc
File "C:\Users\dalth\anaconda3\envs\spyder-dev\lib\site-packages\aiohttp\connector.py", line 1226, in _create_direct_connection
transp, proto = await self._wrap_create_connection(
File "C:\Users\dalth\anaconda3\envs\spyder-dev\lib\site-packages\aiohttp\connector.py", line 1033, in _wrap_create_connection
raise client_error(req.connection_key, exc) from exc
aiohttp.client_exceptions.ClientConnectorError: Cannot connect to host 127.0.0.1:52429 ssl:default [El equipo remoto rechazó la conexión de red] Not sure if that error is expected or if it is even actually related with the work here though 🤔 |
This is triggering an error because it tries to delete the kernel after the connection is closed. We should first close any created kernels before terminating the connection in the widget disconnect routine. However, I believe that we should address this issue in another PR as this is unrelated to this PR scope. |
We can't do that because the scenario @dalthviz simulated (he told me about that internally) was breaking the connection to the server by shutting down the Docker machine. In that case, I decided to close all consoles automatically because it makes little sense to leave them open when there's no possibility of reconnection. And closing a console requests to shutdown its associated kernel.
Well, I think the fix is not that hard: we can either catch the What do you think? |
I agree, that this is an easy fix but the plugin should also handle the any disconnection error mid session properly without raising any errors. I'll push this quick fix with the exception handling, but we should further address this matter in another PR. |
So, just in case and to give some clarity over the testing I have been doing, the steps I used to raise the error I posted above were launching multiple console and then clicking the However, the case @ccordoba12 describes (for some reason the connection to the server gets interrupted from outside Spyder, e.g the docker instance you are connected is stopped from outside Spyder) could also be a possible one and checking it, even with the latest changes, an error is raised: exception calling callback for <Future at 0x25e59fb1400 state=finished raised ClientConnectorError>
Traceback (most recent call last):
File "C:\Users\dalth\anaconda3\envs\spyder-dev\lib\site-packages\aiohttp\connector.py", line 1025, in _wrap_create_connection
return await self._loop.create_connection(*args, **kwargs)
File "C:\Users\dalth\anaconda3\envs\spyder-dev\lib\asyncio\base_events.py", line 1065, in create_connection
raise exceptions[0]
File "C:\Users\dalth\anaconda3\envs\spyder-dev\lib\asyncio\base_events.py", line 1050, in create_connection
sock = await self._connect_sock(
File "C:\Users\dalth\anaconda3\envs\spyder-dev\lib\asyncio\base_events.py", line 961, in _connect_sock
await self.sock_connect(sock, address)
File "C:\Users\dalth\anaconda3\envs\spyder-dev\lib\asyncio\proactor_events.py", line 703, in sock_connect
return await self._proactor.connect(sock, address)
File "C:\Users\dalth\anaconda3\envs\spyder-dev\lib\asyncio\windows_events.py", line 817, in _poll
value = callback(transferred, key, ov)
File "C:\Users\dalth\anaconda3\envs\spyder-dev\lib\asyncio\windows_events.py", line 604, in finish_connect
ov.getresult()
ConnectionRefusedError: [WinError 1225] El equipo remoto rechazó la conexión de red
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "C:\Users\dalth\anaconda3\envs\spyder-dev\lib\concurrent\futures\_base.py", line 330, in _invoke_callbacks
callback(self)
File "E:\Acer\Documentos\Spyder\Spyder otros\hlouzada\spyder\spyder\api\asyncdispatcher.py", line 128, in _callback_task_done
raise exception
File "E:\Acer\Documentos\Spyder\Spyder otros\hlouzada\spyder\spyder\plugins\remoteclient\plugin.py", line 338, in _restart_kernel
return await client.restart_kernel(kernel_id)
File "E:\Acer\Documentos\Spyder\Spyder otros\hlouzada\spyder\spyder\plugins\remoteclient\api\client.py", line 739, in restart_kernel
response = await jupyter.restart_kernel(kernel_id=kernel_id)
File "E:\Acer\Documentos\Spyder\Spyder otros\hlouzada\spyder\spyder\plugins\remoteclient\api\jupyterhub\__init__.py", line 367, in restart_kernel
async with self.session.post(
File "C:\Users\dalth\anaconda3\envs\spyder-dev\lib\site-packages\aiohttp\client.py", line 1197, in __aenter__
self._resp = await self._coro
File "C:\Users\dalth\anaconda3\envs\spyder-dev\lib\site-packages\aiohttp\client.py", line 581, in _request
conn = await self._connector.connect(
File "C:\Users\dalth\anaconda3\envs\spyder-dev\lib\site-packages\aiohttp\connector.py", line 544, in connect
proto = await self._create_connection(req, traces, timeout)
File "C:\Users\dalth\anaconda3\envs\spyder-dev\lib\site-packages\aiohttp\connector.py", line 944, in _create_connection
_, proto = await self._create_direct_connection(req, traces, timeout)
File "C:\Users\dalth\anaconda3\envs\spyder-dev\lib\site-packages\aiohttp\connector.py", line 1257, in _create_direct_connection
raise last_exc
File "C:\Users\dalth\anaconda3\envs\spyder-dev\lib\site-packages\aiohttp\connector.py", line 1226, in _create_direct_connection
transp, proto = await self._wrap_create_connection(
File "C:\Users\dalth\anaconda3\envs\spyder-dev\lib\site-packages\aiohttp\connector.py", line 1033, in _wrap_create_connection
raise client_error(req.connection_key, exc) from exc
aiohttp.client_exceptions.ClientConnectorError: Cannot connect to host 127.0.0.1:58777 ssl:default [El equipo remoto rechazó la conexión de red]
exception calling callback for <Future at 0x25e59fb1400 state=finished raised ClientConnectorError>
Traceback (most recent call last):
File "C:\Users\dalth\anaconda3\envs\spyder-dev\lib\site-packages\aiohttp\connector.py", line 1025, in _wrap_create_connection
return await self._loop.create_connection(*args, **kwargs)
File "C:\Users\dalth\anaconda3\envs\spyder-dev\lib\asyncio\base_events.py", line 1065, in create_connection
raise exceptions[0]
File "C:\Users\dalth\anaconda3\envs\spyder-dev\lib\asyncio\base_events.py", line 1050, in create_connection
sock = await self._connect_sock(
File "C:\Users\dalth\anaconda3\envs\spyder-dev\lib\asyncio\base_events.py", line 961, in _connect_sock
await self.sock_connect(sock, address)
File "C:\Users\dalth\anaconda3\envs\spyder-dev\lib\asyncio\proactor_events.py", line 703, in sock_connect
return await self._proactor.connect(sock, address)
File "C:\Users\dalth\anaconda3\envs\spyder-dev\lib\asyncio\windows_events.py", line 817, in _poll
value = callback(transferred, key, ov)
File "C:\Users\dalth\anaconda3\envs\spyder-dev\lib\asyncio\windows_events.py", line 604, in finish_connect
ov.getresult()
ConnectionRefusedError: [WinError 1225] El equipo remoto rechazó la conexión de red
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "C:\Users\dalth\anaconda3\envs\spyder-dev\lib\concurrent\futures\_base.py", line 330, in _invoke_callbacks
callback(self)
File "E:\Acer\Documentos\Spyder\Spyder otros\hlouzada\spyder\spyder\plugins\remoteclient\widgets\container.py", line 282, in <lambda>
ipyclient, future.result()
File "C:\Users\dalth\anaconda3\envs\spyder-dev\lib\concurrent\futures\_base.py", line 439, in result
return self.__get_result()
File "C:\Users\dalth\anaconda3\envs\spyder-dev\lib\concurrent\futures\_base.py", line 391, in __get_result
raise self._exception
File "C:\Users\dalth\anaconda3\envs\spyder-dev\lib\concurrent\futures\_base.py", line 330, in _invoke_callbacks
callback(self)
File "E:\Acer\Documentos\Spyder\Spyder otros\hlouzada\spyder\spyder\api\asyncdispatcher.py", line 128, in _callback_task_done
raise exception
File "E:\Acer\Documentos\Spyder\Spyder otros\hlouzada\spyder\spyder\plugins\remoteclient\plugin.py", line 338, in _restart_kernel
return await client.restart_kernel(kernel_id)
File "E:\Acer\Documentos\Spyder\Spyder otros\hlouzada\spyder\spyder\plugins\remoteclient\api\client.py", line 739, in restart_kernel
response = await jupyter.restart_kernel(kernel_id=kernel_id)
File "E:\Acer\Documentos\Spyder\Spyder otros\hlouzada\spyder\spyder\plugins\remoteclient\api\jupyterhub\__init__.py", line 367, in restart_kernel
async with self.session.post(
File "C:\Users\dalth\anaconda3\envs\spyder-dev\lib\site-packages\aiohttp\client.py", line 1197, in __aenter__
self._resp = await self._coro
File "C:\Users\dalth\anaconda3\envs\spyder-dev\lib\site-packages\aiohttp\client.py", line 581, in _request
conn = await self._connector.connect(
File "C:\Users\dalth\anaconda3\envs\spyder-dev\lib\site-packages\aiohttp\connector.py", line 544, in connect
proto = await self._create_connection(req, traces, timeout)
File "C:\Users\dalth\anaconda3\envs\spyder-dev\lib\site-packages\aiohttp\connector.py", line 944, in _create_connection
_, proto = await self._create_direct_connection(req, traces, timeout)
File "C:\Users\dalth\anaconda3\envs\spyder-dev\lib\site-packages\aiohttp\connector.py", line 1257, in _create_direct_connection
raise last_exc
File "C:\Users\dalth\anaconda3\envs\spyder-dev\lib\site-packages\aiohttp\connector.py", line 1226, in _create_direct_connection
transp, proto = await self._wrap_create_connection(
File "C:\Users\dalth\anaconda3\envs\spyder-dev\lib\site-packages\aiohttp\connector.py", line 1033, in _wrap_create_connection
raise client_error(req.connection_key, exc) from exc
aiohttp.client_exceptions.ClientConnectorError: Cannot connect to host 127.0.0.1:58777 ssl:default [El equipo remoto rechazó la conexión de red] Which I think is related with the console trying to restart the kernel. Nevertheless, as @hlouzada says, there should be a proper disconnection handling logic that takes into account the multiple scenarios where a connection to a server is closed/terminated (from within Spyder or from outside Spyder) |
Also, following the above, should this be merged as it is or a similar band-aid/temporary approach for the kernel restart traceback should be done first here @ccordoba12 @hlouzada ? I will leave this approved but let me know if further changes are done and testing needs to be done 👍 |
4bef7a6
to
d5f8088
Compare
Ok, thanks for the clarification @dalthviz.
This is the scenario I thought you checked, but thanks for doing it and posting the results here!
Good idea. I amended @hlouzada's last commit to include another temporary fix for that problem. After CIs are green, I'll merge. |
When trying to connect to a remote Linux target, from a local Windows source, using Spyder 6.0.3, I get the following error: Not sure whether it's a problem on your part, or on the asyncssh package (file connection.py, line 3485) |
Hi @ialvata thank you for the feedback! I would suggest you to open a new issue about that. From a quick check, a way that occurs to me to workaround the issue is to do an initial manual connection to your Linux remote machine outside Spyder (for example using a powershell instance) to add the remote machine to the Let us know if the info above helps! |
Description of Changes
Paramiko in Windows with/or Windows native SSH Client (Win32-OpenSSH) doesn't tunnel successfully, probably due to PowerShell/Win32-OpenSSH#999.
So it was used asyncssh that reimplements the Open SSH protocol in Python.
Affirmation
By submitting this Pull Request or typing my (user)name below,
I affirm the Developer Certificate of Origin
with respect to all commits and content included in this PR,
and understand I am releasing the same under Spyder's MIT (Expat) license.
I certify the above statement is true and correct: @hlouzada