Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement GUI protocol 1.4 #105

Merged
merged 4 commits into from
Jul 20, 2022
Merged

Implement GUI protocol 1.4 #105

merged 4 commits into from
Jul 20, 2022

Conversation

DemiMarie
Copy link
Contributor

This allows bidirectional protocol negotiation, which means that the
agent can be newer than the daemon.

The daemon must be upgraded to support protocol 1.4 first. Afterwards,
agents and daemons can be independently upgraded.

@DemiMarie
Copy link
Contributor Author

Build is broken because this has a hard dependency on QubesOS/qubes-gui-common#16.

@qubesos-bot
Copy link

qubesos-bot commented Jul 8, 2022

OpenQA test summary

Complete test suite and dependencies: https://openqa.qubes-os.org/tests/overview?distri=qubesos&version=4.1&build=2022071409-4.1&flavor=pull-requests

New failures, excluding unstable

Compared to: https://openqa.qubes-os.org/tests/overview?distri=qubesos&version=4.1&build=2022062106-4.1&flavor=update

  • system_tests_basic_vm_qrexec_gui_xfs

  • system_tests_suspend@hw1

  • system_tests_basic_vm_qrexec_gui

  • system_tests_extra

    • TC_00_InputProxy_debian-11: test_050_mouse_late_attach (failure)
      AssertionError: unexpectedly None : Device 'test-inst-input: Test i...

    • TC_00_InputProxy_fedora-36: test_050_mouse_late_attach (failure)
      AssertionError: unexpectedly None : Device 'test-inst-input: Test i...

  • system_tests_pvgrub_salt_storage

    • TC_10_VMSalt_debian-11: test_000_simple_sls (error)
      libvirt.libvirtError: internal error: libxenlight failed to create ...
  • system_tests_manager

  • system_tests_network_updates

    • TC_10_QvmTemplate_debian-11: test_000_template_list (failure)
      AssertionError: libvirt event impl drain timeout

    • TC_10_QvmTemplate_fedora-36: test_000_template_list (failure)
      AssertionError: libvirt event impl drain timeout

    • TC_11_QvmTemplateMgmtVM_fedora-36: test_010_template_install (failure)
      AssertionError: qvm-template failed: Downloading 'qubes-template-de...

  • system_tests_qrexec

    • TC_00_Qrexec_debian-11: test_050_qrexec_simple_eof (failure + cleanup)
      AssertionError: Timeout, probably EOF wasn't transferred to the VM ...

    • TC_00_Qrexec_whonix-gw-16: test_050_qrexec_simple_eof (failure + cleanup)
      AssertionError: Timeout, probably EOF wasn't transferred to the VM ...

    • TC_00_Qrexec_whonix-ws-16: test_050_qrexec_simple_eof (failure + cleanup)
      AssertionError: Timeout, probably EOF wasn't transferred to the VM ...

  • system_tests_basic_vm_qrexec_gui_ext4

  • system_tests_network_ipv6

    • VmIPv6Networking_debian-11: test_040_inter_vm (error)
      qubes.exc.QubesVMError: Cannot connect to qrexec agent for 90 secon...

    • VmIPv6Networking_fedora-36: test_040_inter_vm (error)
      qubes.exc.QubesVMError: Cannot connect to qrexec agent for 90 secon...

    • VmIPv6Networking_fedora-36: test_540_ipv6_inter_vm (error)
      qubes.exc.QubesVMError: Cannot connect to qrexec agent for 90 secon...

  • system_tests_basic_vm_qrexec_gui_btrfs

  • system_tests_guivm_gui_interactive

Failed tests

31 failures
  • system_tests_basic_vm_qrexec_gui_xfs

  • system_tests_suspend@hw1

    • suspend: unnamed test (unknown)
    • [unstable] suspend: Failed (test died)
      # Test died: no candidate needle with tag(s) 'xscreensaver-prompt' ...
  • system_tests_basic_vm_qrexec_gui

  • system_tests_extra

    • TC_00_InputProxy_debian-11: test_050_mouse_late_attach (failure)
      AssertionError: unexpectedly None : Device 'test-inst-input: Test i...

    • TC_00_InputProxy_fedora-36: test_050_mouse_late_attach (failure)
      AssertionError: unexpectedly None : Device 'test-inst-input: Test i...

  • system_tests_devices

    • [unstable] TC_10_Attach_debian-11: test_000_attach_reattach (error)
      subprocess.CalledProcessError: Command 'ls /dev/xvdi' returned non-...

    • [unstable] TC_10_Attach_fedora-36: test_000_attach_reattach (error)
      subprocess.CalledProcessError: Command 'ls /dev/xvdi' returned non-...

    • [unstable] TC_10_Attach_whonix-ws-16: test_000_attach_reattach (error)
      subprocess.CalledProcessError: Command 'ls /dev/xvdi' returned non-...

  • system_tests_pvgrub_salt_storage

    • TC_10_VMSalt_debian-11: test_000_simple_sls (error)
      libvirt.libvirtError: internal error: libxenlight failed to create ...
  • system_tests_manager

  • system_tests_network_updates

    • TC_10_QvmTemplate_debian-11: test_000_template_list (failure)
      AssertionError: libvirt event impl drain timeout

    • TC_10_QvmTemplate_fedora-36: test_000_template_list (failure)
      AssertionError: libvirt event impl drain timeout

    • TC_11_QvmTemplateMgmtVM_fedora-36: test_010_template_install (failure)
      AssertionError: qvm-template failed: Downloading 'qubes-template-de...

  • system_tests_qrexec

    • TC_00_Qrexec_debian-11: test_050_qrexec_simple_eof (failure + cleanup)
      AssertionError: Timeout, probably EOF wasn't transferred to the VM ...

    • TC_00_Qrexec_whonix-gw-16: test_050_qrexec_simple_eof (failure + cleanup)
      AssertionError: Timeout, probably EOF wasn't transferred to the VM ...

    • TC_00_Qrexec_whonix-ws-16: test_050_qrexec_simple_eof (failure + cleanup)
      AssertionError: Timeout, probably EOF wasn't transferred to the VM ...

  • system_tests_basic_vm_qrexec_gui_ext4

  • system_tests_network_ipv6

    • VmIPv6Networking_debian-11: test_040_inter_vm (error)
      qubes.exc.QubesVMError: Cannot connect to qrexec agent for 90 secon...

    • [unstable] VmIPv6Networking_debian-11: test_540_ipv6_inter_vm (error)
      qubes.exc.QubesVMError: Cannot connect to qrexec agent for 90 secon...

    • VmIPv6Networking_fedora-36: test_040_inter_vm (error)
      qubes.exc.QubesVMError: Cannot connect to qrexec agent for 90 secon...

    • VmIPv6Networking_fedora-36: test_540_ipv6_inter_vm (error)
      qubes.exc.QubesVMError: Cannot connect to qrexec agent for 90 secon...

  • system_tests_basic_vm_qrexec_gui_btrfs

  • system_tests_dispvm

    • [unstable] TC_20_DispVM_fedora-36: test_030_edit_file (failure)
      AssertionError: b'test1\n' != b'Test test 2\ntest1\n'

    • [unstable] TC_20_DispVM_fedora-36: test_100_open_in_dispvm (failure)
      self.assertEqual(test_txt_content.s... AssertionError: b'' != b'test1'

  • system_tests_guivm_gui_interactive

    • guivm_startup: unnamed test (unknown)
    • guivm_startup: Failed (test died)
      # Test died: no candidate needle with tag(s) 'desktop' matched...

Fixed failures

Compared to: https://openqa.qubes-os.org/tests/42506#dependencies

6 fixed
  • system_tests_suspend@hw1

    • suspend: unnamed test (unknown)
    • suspend: wait_serial (wait serial expected)
      # wait_serial expected: "xl info; echo 8Ye1l-\$?-"...
  • system_tests_network_updates

  • system_tests_basic_vm_qrexec_gui@hw1

    • TC_00_AppVM_whonix-ws-16: test_223_audio_play_hvm (error)
      qubes.exc.QubesVMError: Cannot connect to qrexec agent for 90 secon...
  • system_tests_dispvm

  • system_tests_guivm_gui_interactive

Unstable tests

  • system_tests_basic_vm_qrexec_gui_xfs

    TC_00_AppVM_fedora-35-pool/test_225_audio_rec_unmuted_hvm (1/5 times with errors)
    • job 42209 AssertionError: Timeout waiting for pulseaudio start in test-inst-v...
    TC_00_AppVM_whonix-ws-16-pool/test_225_audio_rec_unmuted_hvm (1/5 times with errors)
    • job 43010 AssertionError: only silence detected, no useful audio data
  • system_tests_suspend@hw1

    suspend/ (1/5 times with errors)
    suspend/Failed (1/5 times with errors)
    • job 42195 # Test died: no candidate needle with tag(s) 'xscreensaver-prompt' ...
  • system_tests_suspend

    suspend/ (1/5 times with errors)
    suspend/Failed (1/5 times with errors)
    • job 42195 # Test died: no candidate needle with tag(s) 'xscreensaver-prompt' ...
  • system_tests_usbproxy

    TC_20_USBProxy_core3_fedora-35/test_061_auto_attach_on_reconnect (1/5 times with errors)
    • job 42232 AssertionError: 1 != 0 : Device reconnection failed
    TC_20_USBProxy_core3_whonix-gw-16/test_070_attach_not_installed_front (1/5 times with errors)
    • job 42965 qubesusbproxy.core3ext.QubesUSBException: Device attach failed: 202...
  • system_tests_basic_vm_qrexec_gui

    TC_00_AppVM_fedora-35/test_223_audio_play_hvm (1/5 times with errors)
    • job 41349 AssertionError: Timeout waiting for pulseaudio start in test-inst-v...
    TC_00_AppVM_whonix-gw-16/test_300_bug_1028_gui_memory_pinning (1/5 times with errors)
    • job 41818 AssertionError: Dom0 window doesn't match VM window content
  • system_tests_devices

    TC_10_Attach_debian-11/test_000_attach_reattach (1/5 times with errors)
    • job 42968 subprocess.CalledProcessError: Command 'ls /dev/xvdi' returned non-...
    TC_10_Attach_fedora-35/test_000_attach_reattach (1/5 times with errors)
    • job 42612 subprocess.CalledProcessError: Command 'ls /dev/xvdi' returned non-...
    TC_10_Attach_fedora-36/test_000_attach_reattach (1/5 times with errors)
    • job 42968 subprocess.CalledProcessError: Command 'ls /dev/xvdi' returned non-...
    TC_10_Attach_whonix-gw-16/test_000_attach_reattach (2/5 times with errors)
    • job 42612 subprocess.CalledProcessError: Command 'ls /dev/xvdi' returned non-...
    • job 42968 subprocess.CalledProcessError: Command 'ls /dev/xvdi' returned non-...
    TC_10_Attach_whonix-ws-16/test_000_attach_reattach (1/5 times with errors)
    • job 42968 subprocess.CalledProcessError: Command 'ls /dev/xvdi' returned non-...
  • system_tests_pvgrub_salt_storage

    TC_41_HVMGrub_debian-11/test_000_standalone_vm (2/5 times with errors)
    • job 42193 qubes.exc.QubesVMError: Cannot connect to qrexec agent for 90 secon...
    • job 43833 qubes.exc.QubesVMError: Cannot connect to qrexec agent for 90 secon...
    TC_42_PVHGrub_debian-11/test_000_standalone_vm (2/5 times with errors)
    • job 42193 qubes.exc.QubesVMError: Cannot connect to qrexec agent for 90 secon...
    • job 43833 qubes.exc.QubesVMError: Cannot connect to qrexec agent for 90 secon...
    TC_41_HVMGrub_debian-11/test_010_template_based_vm (2/5 times with errors)
    • job 42193 qubes.exc.QubesVMError: Cannot connect to qrexec agent for 90 secon...
    • job 43833 qubes.exc.QubesVMError: Cannot connect to qrexec agent for 90 secon...
    TC_41_HVMGrub_fedora-35/test_010_template_based_vm (1/5 times with errors)
    • job 42193 qubes.exc.QubesVMError: Cannot connect to qrexec agent for 90 secon...
    TC_41_HVMGrub_fedora-36/test_010_template_based_vm (1/5 times with errors)
    • job 43833 qubes.exc.QubesVMError: Cannot connect to qrexec agent for 90 secon...
    TC_42_PVHGrub_debian-11/test_010_template_based_vm (2/5 times with errors)
    • job 42193 qubes.exc.QubesVMError: Cannot connect to qrexec agent for 90 secon...
    • job 43833 qubes.exc.QubesVMError: Cannot connect to qrexec agent for 90 secon...
  • system_tests_network

    VmNetworking_debian-11/test_020_simple_proxyvm_nm (2/5 times with errors)
    • job 42235 AssertionError: 1 != 0 : nm-applet window not found
    • job 42306 AssertionError: 1 != 0 : nm-applet window not found
  • system_tests_network_updates

    TC_10_QvmTemplate_whonix-gw-16/test_000_template_list (1/5 times with errors)
    • job 41859 subprocess.CalledProcessError: Command 'systemcheck --cli' returned...
    TC_11_QvmTemplateMgmtVM_debian-11/test_010_template_install (1/5 times with errors)
    • job 42602 AssertionError: libvirt event impl drain timeout
    TC_11_QvmTemplateMgmtVM_whonix-gw-16/test_010_template_install (1/5 times with errors)
    • job 41859 subprocess.CalledProcessError: Command 'systemcheck --cli' returned...
  • system_tests_qrexec

    TC_00_Qrexec_debian-11/test_090_qrexec_service_socket_dom0 (1/5 times with errors)
    • job 42973 AssertionError: service timeout, probably EOF wasn't transferred to...
  • system_tests_splitgpg

    TC_10_Thunderbird_debian-11/test_000_send_receive_default (1/5 times with errors)
    • job 43834 Exception: Failed to send message with error 'unknown'
    TC_10_Thunderbird_fedora-36/test_000_send_receive_default (1/5 times with errors)
    • job 43834 dogtail.tree.SearchError: descendent of [menu bar | Application]: c...
    TC_10_Thunderbird_debian-11/test_010_send_receive_inline_signed_only (1/5 times with errors)
    • job 42194 Exception: Failed to send message with error 'unknown'
    TC_10_Thunderbird_fedora-35/test_010_send_receive_inline_signed_only (1/5 times with errors)
    • job 42194 dogtail.tree.SearchError: descendent of [panel | ]: child with name...
    TC_10_Thunderbird_fedora-36/test_010_send_receive_inline_signed_only (1/5 times with errors)
    • job 43834 dogtail.tree.SearchError: descendent of [menu bar | Application]: c...
    TC_10_Thunderbird_debian-11/test_020_send_receive_inline_with_attachment (1/5 times with errors)
    • job 43834 dogtail.tree.SearchError: descendent of [application | Thunderbird]...
    TC_10_Thunderbird_fedora-35/test_020_send_receive_inline_with_attachment (1/5 times with errors)
    • job 42194 dogtail.tree.SearchError: descendent of [application | Thunderbird]...
    TC_10_Thunderbird_fedora-36/test_020_send_receive_inline_with_attachment (1/5 times with errors)
    • job 43834 dogtail.tree.SearchError: descendent of [menu bar | Application]: c...
    TC_10_Thunderbird_whonix-ws-16/test_020_send_receive_inline_with_attachment (1/5 times with errors)
    • job 42194 assert received_attachment == orig_attachment... AssertionError
  • system_tests_basic_vm_qrexec_gui_ext4

    TC_00_AppVM_whonix-gw-16-pool/test_100_qrexec_filecopy (1/5 times with errors)
    • job 42178 qubes.exc.QubesVMError: Cannot connect to qrexec agent for 90 secon...
  • system_tests_network_ipv6

    VmIPv6Networking_debian-11/test_540_ipv6_inter_vm (1/5 times with errors)
    • job 43248 raise exceptions.TimeoutError()... asyncio.exceptions.TimeoutError
  • system_tests_basic_vm_qrexec_gui@hw1

    TC_00_AppVM_fedora-35/test_223_audio_play_hvm (1/5 times with errors)
    • job 41349 AssertionError: Timeout waiting for pulseaudio start in test-inst-v...
    TC_00_AppVM_whonix-gw-16/test_300_bug_1028_gui_memory_pinning (1/5 times with errors)
    • job 41818 AssertionError: Dom0 window doesn't match VM window content
  • system_tests_basic_vm_qrexec_gui_btrfs

    TC_00_AppVM_whonix-gw-16-pool/test_000_start_shutdown (1/5 times with errors)
    • job 43019 qubes.exc.QubesVMShutdownTimeoutError: Domain shutdown timed out: '...
    TC_00_AppVM_whonix-ws-16-pool/test_223_audio_play_hvm (1/5 times with errors)
    • job 42970 AssertionError: Timeout waiting for pulseaudio start in test-inst-v...
  • system_tests_dispvm

    TC_04_DispVM/test_003_cleanup_destroyed (1/5 times with errors)
    • job 42585 raise exceptions.TimeoutError()... asyncio.exceptions.TimeoutError
    TC_20_DispVM_debian-11/test_010_simple_dvm_run (1/5 times with errors)
    • job 43841 qubes.exc.QubesVMError: Cannot connect to qrexec agent for 90 secon...
    TC_20_DispVM_fedora-35/test_010_simple_dvm_run (1/5 times with errors)
    • job 42585 assert len(self.loop._selector.get_map()) \... AssertionError
    TC_20_DispVM_fedora-36/test_010_simple_dvm_run (1/5 times with errors)
    • job 43841 assert len(self.loop._selector.get_map()) \... AssertionError
    TC_20_DispVM_whonix-gw-16/test_010_simple_dvm_run (2/5 times with errors)
    TC_20_DispVM_whonix-ws-16/test_010_simple_dvm_run (2/5 times with errors)
    • job 42585 assert len(self.loop._selector.get_map()) \... AssertionError
    • job 43841 qubes.exc.QubesVMError: Cannot connect to qrexec agent for 90 secon...
    TC_20_DispVM_debian-11/test_020_gui_app (1/5 times with errors)
    • job 43841 raise exceptions.TimeoutError()... asyncio.exceptions.TimeoutError
    TC_20_DispVM_whonix-gw-16/test_020_gui_app (2/5 times with errors)
    TC_20_DispVM_debian-11/test_030_edit_file (2/5 times with errors)
    • job 42585 AssertionError: Timeout while waiting for disp[0-9]* window to show
    • job 43841 AssertionError: Timeout while waiting for disp[0-9]* window to show
    TC_20_DispVM_fedora-35/test_030_edit_file (1/5 times with errors)
    • job 42585 AssertionError: Timeout while waiting for disp[0-9]* window to show
    TC_20_DispVM_fedora-36/test_030_edit_file (3/5 times with errors)
    • job 42967 AssertionError: Timeout waiting for editor window
    • job 43255 AssertionError: Timeout waiting for editor window
    • job 43841 AssertionError: Timeout while waiting for disp[0-9]* window to show
    TC_20_DispVM_whonix-gw-16/test_030_edit_file (2/5 times with errors)
    TC_20_DispVM_whonix-ws-16/test_030_edit_file (2/5 times with errors)
    • job 42585 AssertionError: Timeout while waiting for disp[0-9]* window to show
    • job 43841 AssertionError: Timeout while waiting for disp[0-9]* window to show
    TC_20_DispVM_debian-11/test_100_open_in_dispvm (2/5 times with errors)
    • job 42585 AssertionError: './open-file test.txt' failed with ./open-file test...
    • job 43841 AssertionError: './open-file test.txt' failed with ./open-file test...
    TC_20_DispVM_fedora-35/test_100_open_in_dispvm (1/5 times with errors)
    • job 42585 AssertionError: './open-file test.txt' failed with ./open-file test...
    TC_20_DispVM_fedora-36/test_100_open_in_dispvm (3/5 times with errors)
    • job 42967 AssertionError: Timeout waiting for editor window
    • job 43255 AssertionError: Timeout waiting for editor window
    • job 43841 AssertionError: './open-file test.txt' failed with ./open-file test...
    TC_20_DispVM_whonix-gw-16/test_100_open_in_dispvm (2/5 times with errors)
    TC_20_DispVM_whonix-ws-16/test_100_open_in_dispvm (5/5 times with errors)
    • job 42585 AssertionError: Timeout while waiting for disp[0-9]* window to show
    • job 42626 AssertionError: libvirt event impl drain timeout
    • job 42967 AssertionError: libvirt event impl drain timeout
    • job 43255 AssertionError: libvirt event impl drain timeout
    • job 43841 AssertionError: './open-file test.txt' failed with ./open-file test...

It can at least theoretically fail.
It may fail due to out of memory or other reasons.
@iacore
Copy link

iacore commented Jul 18, 2022

Instead of using protocol version, I suggest we use feature detection and never change the semantic of message types.

For example, we rename current message MSG_BUTTON to const int MSG_BUTTON_DISTINCT_NAME = xxxx;

Then we can have MSG_BUTTON_ANOTHER_NAME.

If we change the semantic of API, the endpoint name should change.

Feature detection is better to understand than mysterious code like

only using version:

if (ctx.version <= "1.4") use_legacy_api() else use_new_api();

Feature detection

bool supported = query_feature("legacy");
if (supported) use_legacy_api() else use_new_api();

@DemiMarie
Copy link
Contributor Author

@locriacyber The problem with this is that it requires support from both daemon and agent, and right now neither have such support. Therefore, there needs to be some way to announce support for this. Also, having to deal with all possible feature bits is a testing nightmare. Just having to deal with all the different version combinations will be hard enough.

This implements GUI protocol v1.4, which includes bidirectional protocol
negotiation.  This means that not only does the agent tell the daemon
what protocol version it supports, the daemon also tells the agent what
protocol version it decided to actually use.  This may be less than the
version supported by the agent.  As a result, a GUI daemon including
this patch may be used with any version of the GUI agent supporting
protocol major version 1, even ones that support higher minor versions
than the daemon does.
This avoids having to bump build dependency versions and allows CI to
run.
@marmarek marmarek merged commit 0cfa15d into QubesOS:master Jul 20, 2022
@DemiMarie DemiMarie deleted the version-1.4 branch July 20, 2022 05:11
@iacore
Copy link

iacore commented Jul 20, 2022

@DemiMarie This is more like 2.0 rather than 1.4? Since newer clients will not work with older server.

I'll try to rebase my patch for the server on this.

@marmarek
Copy link
Member

marmarek commented Jul 20, 2022

Since newer clients will not work with older server.

Yes, but older clients will work with newer server. And it was always required that daemon proto version >= agent proto version.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants