Test runner monopolizes stdin, stdout, stderr

[rubi-rastko]

Issue Type: Bug

Behaviour

ERROR: test_xyz (xyz.xyz_tests.xyz_test.Test_XYZ)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "[REDACTED]_test.py", line 57, in test_[REDACTED]
    sys.stderr.buffer.write(struct.pack("L", 20))
  File "[REDACTED]/.vscode/extensions/ms-python.python-2021.12.1559732655/pythonFiles/visualstudio_py_testlauncher.py", line 88, in write
    _channel.send_event("stdout" if self.is_stdout else "stderr", content=data)
  File "[REDACTED]/.vscode/extensions/ms-python.python-2021.12.1559732655/pythonFiles/visualstudio_py_testlauncher.py", line 132, in send_event
    content = json.dumps(body).encode("utf8")
  File "/usr/lib64/python3.7/json/__init__.py", line 231, in dumps
    return _default_encoder.encode(obj)
  File "/usr/lib64/python3.7/json/encoder.py", line 199, in encode
    chunks = self.iterencode(o, _one_shot=True)
  File "/usr/lib64/python3.7/json/encoder.py", line 257, in iterencode
    return _iterencode(o, 0)
  File "/usr/lib64/python3.7/json/encoder.py", line 179, in default
    raise TypeError(f'Object of type {o.__class__.__name__} '
TypeError: Object of type bytes is not JSON serializable

Expected vs. Actual Behaviour

>>> 8

Steps to reproduce:

[NOTE: Self-contained, minimal reproducing code samples are extremely helpful and will expedite addressing your issue]

1. Write a test class
2. Write a test function
3. Call sys.stdout.buffer.write()

import unittest
import struct
import sys

class Test_XYZ(unittest.TestCase):
    def setUp(self):
        pass
    def test_xyz(self):
        sys.stderr.buffer.write(struct.pack("I", 65))

4. Run test (from anywhere)

Diagnostic data

• Python version (& distribution if applicable, e.g. Anaconda): 3.7.12
• Type of virtual environment used (e.g. conda, venv, virtualenv, etc.): VirtualEnvWrapper
• Value of the python.languageServer setting: Default

User Settings

defaultLS: {"defaultLSType":"Pylance"}

downloadLanguageServer: true

envFile: "<placeholder>"

venvPath: "<placeholder>"

venvFolders: "<placeholder>"

condaPath: "<placeholder>"

pipenvPath: "<placeholder>"

poetryPath: "<placeholder>"

languageServer: "Pylance"

linting
• enabled: true
• cwd: "<placeholder>"
• Flake8Args: "<placeholder>"
• flake8Enabled: false
• flake8Path: "<placeholder>"
• lintOnSave: true
• banditArgs: "<placeholder>"
• banditEnabled: false
• banditPath: "<placeholder>"
• mypyArgs: "<placeholder>"
• mypyEnabled: false
• mypyPath: "<placeholder>"
• pycodestyleArgs: "<placeholder>"
• pycodestyleEnabled: false
• pycodestylePath: "<placeholder>"
• prospectorArgs: "<placeholder>"
• prospectorEnabled: false
• prospectorPath: "<placeholder>"
• pydocstyleArgs: "<placeholder>"
• pydocstyleEnabled: false
• pydocstylePath: "<placeholder>"
• pylamaArgs: "<placeholder>"
• pylamaEnabled: false
• pylamaPath: "<placeholder>"
• pylintArgs: "<placeholder>"
• pylintPath: "<placeholder>"

sortImports
• args: "<placeholder>"
• path: "<placeholder>"

formatting
• autopep8Args: "<placeholder>"
• autopep8Path: "<placeholder>"
• provider: "autopep8"
• blackArgs: "<placeholder>"
• blackPath: "<placeholder>"
• yapfArgs: "<placeholder>"
• yapfPath: "<placeholder>"

testing
• cwd: "<placeholder>"
• debugPort: 3000
• nosetestArgs: "<placeholder>"
• nosetestsEnabled: undefined
• nosetestPath: "<placeholder>"
• promptToConfigure: true
• pytestArgs: "<placeholder>"
• pytestEnabled: false
• pytestPath: "<placeholder>"
• unittestArgs: "<placeholder>"
• unittestEnabled: true
• autoTestDiscoverOnSaveEnabled: true

terminal
• activateEnvironment: true
• executeInFileDir: "<placeholder>"
• launchArgs: "<placeholder>"

experiments
• enabled: false
• optInto: []
• optOutFrom: []

insidersChannel: "off"

tensorBoard
• logDirectory: "<placeholder>"

Extension version: 2021.12.1559732655
VS Code version: Code 1.63.2 (899d46d82c4c95423fb7e10e68eba52050e30ba3, 2021-12-15T09:39:46.686Z)
OS version: Linux x64 5.15.16-200.fc35.x86_64
Restricted Mode: No

System Info

Item	Value
CPUs	Intel(R) Core(TM) i3-6006U CPU @ 2.00GHz (4 x 2000)
GPU Status	2d_canvas: enabled gpu_compositing: enabled multiple_raster_threads: enabled_on oop_rasterization: disabled_off opengl: enabled_on rasterization: disabled_software skia_renderer: enabled_on video_decode: disabled_software vulkan: disabled_off webgl: enabled webgl2: enabled
Load (avg)	1, 1, 1
Memory (System)	7.67GB (0.20GB free)
Process Argv	--unity-launch --crash-reporter-id a5081874-60a8-4967-8df5-8df8194cc4a1
Screen Reader	no
VM	0%
DESKTOP_SESSION	gnome-xorg
XDG_CURRENT_DESKTOP	GNOME
XDG_SESSION_DESKTOP	gnome-xorg
XDG_SESSION_TYPE	x11

A/B Experiments

vsliv368cf:30146710
vsreu685:30147344
python383cf:30185419
vspor879:30202332
vspor708:30202333
vspor363:30204092
pythonvspyl392:30425749
pythontb:30283811
pythonvspyt551:30345470
pythonptprofiler:30281270
vshan820:30294714
vstes263:30335439
vscoreces:30384385
pythondataviewer:30285071
vscod805:30301674
pythonvspyt200:30340761
binariesv615:30325510
bridge0708:30335490
bridge0723:30353136
vsaa593:30376534
vsc1dst:30424896
pythonvs932:30410667
vscop804:30404766
vs360cf:30404996
vsrem710:30416614
vsbas813:30426126
vsic-wlc-tsjs:30426436

[kimadeline]

Hi @rubi-rastko, thank you for reaching out. We currently rely on stdout and stderr to pass test results around, so if your tests use them you will need to find a different way to pass your data around or you will encounter this bug.

We are currently working on changing this behaviour with #17242.

[rubi-rastko]

Hi @rubi-rastko, thank you for reaching out. We currently rely on stdout and stderr to pass test results around, so if your tests use them you will need to find a different way to pass your data around or you will encounter this bug.

We are currently working on changing this behaviour with #17242.

Thank you for replying.
Given what you've said there, I wouldn't consider this a bug, but a user error on my part.
Using stdout, stdin in tests is probably an anti-pattern.
I retract my issue report. This bug report is a bug.
Feel free to close it as such.

[kimadeline]

I don't think it's an anti-pattern in general, but rather that when we built the testing part of the extension we didn't consider the fact that stdout and stderr might be used by tests 🙂

I'm going to keep this issue open so we can verify that stdout and stderr are not monopolized by the extension once we're done with #17242.

Thank you!

[eleanorjboyd]

See all the following issues as related. With the old test architecture, all these problems came from how we processed the info. I will need to look through all of these and see which ones are still outstanding and look for a fix.

#18547

#18431

#18346

#16705

#11729

[eleanorjboyd]

This is fixed on insiders when you have the testing experiment turned on. Let me know if it is not working for you!