Debugger doesn't start if LANG is set to non-existent locale

[lorenzo-cavazzi]

Environment data

• VS Code version: 1.43.1
• Extension version: 2020.3.69010
• OS: Linux x64 5.5.11-1-MANJARO
• Python version: 3.8.2 (system), 3.7.5 (pipenv, used in the referenced repo)
• Type of virtual environment used: virtualenv
• Jedi or Language Server? Jedi enabled
• Value of the python.languageServer setting: "python.languageServer": "Microsoft"

Expected behaviour

Being able to debug pytest and hit breakpoints.

Actual behaviour

When I click on "Debug test", the debug interface quickly starts (I end up in the "Run" tab and I can see the "play", "stop", ... buttons on top right) but it stops after less than a second without hitting the breakpoint and without completing the test.
Using "Run Test" works as expected and the test run and is marked as passed.

Steps to reproduce:

• Clone a repo with pytest. I am using https://github.com/SwissDataScienceCenter/renku-python
• Pick a test and put a breakpoint. Let's say I try to debug test_init_repository located in tests/core/management/test_repository.py and I put a breakpoint at line 27
• NOTE: in pytest.ini we need to comment out line 20 with addopts due to known problems with pytest-cov
• click on Debug Test

P.S: this is my settings.json

{
    "python.pythonPath": "/home/lorenzo/.local/share/virtualenvs/renku-python-WcnE3QeY/bin/python",
    "python.testing.unittestEnabled": false,
    "python.testing.nosetestsEnabled": false,
    "python.testing.pytestEnabled": true,
    "python.testing.pytestArgs": ["--no-cov"],
    "python.linting.pylintEnabled": false,
    "python.linting.flake8Enabled": true,
    "python.linting.enabled": true,
	"python.formatting.provider": "yapf"
}

Logs

Output for Python in the Output panel (View→Output, change the drop-down the upper-right of the Output panel to Python)

I don't see any output in "Output --> Python" nor in "Python Test Log" after clicking on "Debug test".
When I start VSCode and I select the file containing the test, I see the following

User belongs to experiment group 'ShowPlayIcon - start'
User belongs to experiment group 'ShowExtensionSurveyPrompt - control'
User belongs to experiment group 'DebugAdapterFactory - experiment'
User belongs to experiment group 'PtvsdWheels37 - experiment'
User belongs to experiment group 'Reload - control'
User belongs to experiment group 'LS - control'
User belongs to experiment group 'UseTerminalToGetActivatedEnvVars - control'
User belongs to experiment group 'WebHostNotebook - control'
User belongs to experiment group 'CollectLSRequestTiming - control'
User belongs to experiment group 'CollectNodeLSRequestTiming - experiment'
> conda --version
> conda info --json
> pyenv root
> python3.7 -c "import sys;print(sys.executable)"
> python3.6 -c "import sys;print(sys.executable)"
> python3 -c "import sys;print(sys.executable)"
> python2 -c "import sys;print(sys.executable)"
> python -c "import sys;print(sys.executable)"
> ~/.local/share/virtualenvs/renku-python-WcnE3QeY/bin/python -c "import sys;print(sys.executable)"
> ~/.local/share/virtualenvs/renku-notebooks-4n1SxYW5/bin/python -c "import jupyter"
> ~/.local/share/virtualenvs/renku-notebooks-4n1SxYW5/bin/python -c "import notebook"
> ~/.local/share/virtualenvs/renku-notebooks-4n1SxYW5/bin/python -m jupyter kernelspec --version
> pipenv --version
cwd: ~/development/sdsc/renku-python
> conda env list
> conda env list
> pipenv --venv
cwd: ~/development/sdsc/renku-python
Starting Jedi Python language engine.
> conda --version
> ~/.local/share/virtualenvs/renku-python-WcnE3QeY/bin/python ~/.vscode-oss/extensions/ms-python.python-2020.3.69010/pythonFiles/testing_tools/run_adapter.py discover pytest -- --rootdir ~/development/sdsc/renku-python -s --cache-clear --no-cov
cwd: ~/development/sdsc/renku-python
> pipenv --py
cwd: ~/development/sdsc/renku-python
> pipenv --version
cwd: ~/development/sdsc/renku-python
> ~/.local/share/virtualenvs/renku-python-WcnE3QeY/bin/python -c "import sys;print(sys.prefix)"
cwd: ~/development/sdsc/renku-python
> ~/.local/share/virtualenvs/renku-python-WcnE3QeY/bin/python -c "import sys;print(sys.prefix)"
cwd: ~/development/sdsc/renku-python
> ~/.local/share/virtualenvs/renku-python-WcnE3QeY/bin/python -c "import sys;print(sys.executable)"
cwd: ~/development/sdsc/renku-python
> ~/.local/share/virtualenvs/renku-python-WcnE3QeY/bin/python -c "import sys;print(sys.executable)"
cwd: ~/development/sdsc/renku-python
> ~/.local/share/virtualenvs/renku-python-WcnE3QeY/bin/python -c "from distutils.sysconfig import get_python_lib; print(get_python_lib())"
cwd: ~/development/sdsc/renku-python
> ~/.local/share/virtualenvs/renku-python-WcnE3QeY/bin/python -c "from distutils.sysconfig import get_python_lib; print(get_python_lib())"
cwd: ~/development/sdsc/renku-python
> ~/.local/share/virtualenvs/renku-python-WcnE3QeY/bin/python -m site --user-site
cwd: ~/development/sdsc/renku-python
> ~/.local/share/virtualenvs/renku-python-WcnE3QeY/bin/python -m site --user-site
cwd: ~/development/sdsc/renku-python
> ~/.local/share/virtualenvs/renku-python-WcnE3QeY/bin/python completion.py
cwd: ~/.vscode-oss/extensions/ms-python.python-2020.3.69010/pythonFiles
> ~/.local/share/virtualenvs/renku-python-WcnE3QeY/bin/python completion.py
cwd: ~/.vscode-oss/extensions/ms-python.python-2020.3.69010/pythonFiles
> pipenv --venv
cwd: ~/development/sdsc/renku-python
> ~/.local/share/virtualenvs/renku-python-WcnE3QeY/bin/python -m flake8 --format=%(row)d,%(col)d,%(code).1s,%(code)s:%(text)s ~/development/sdsc/renku-python/tests/core/management/test_repository.py
cwd: ~/development/sdsc/renku-python
> ~/.local/share/virtualenvs/renku-python-WcnE3QeY/bin/python -m flake8 --format=%(row)d,%(col)d,%(code).1s,%(code)s:%(text)s ~/development/sdsc/renku-python/tests/core/management/test_repository.py
cwd: ~/development/sdsc/renku-python
##########Linting Output - flake8##########
> pipenv --py
cwd: ~/development/sdsc/renku-python

Output from Console under the Developer Tools panel (toggle Developer Tools on under Help; turn on source maps to make any tracebacks be useful by running Enable source map support for extension debugging)

  ERR timeout after 500 ms: Error: timeout after 500 ms
    at t.RawDebugSession.handleErrorResponse (file:///usr/lib/code/out/vs/workbench/workbench.desktop.main.js:2954:952)
    at file:///usr/lib/code/out/vs/workbench/workbench.desktop.main.js:2954:250
    at async t.RawDebugSession.shutdown (file:///usr/lib/code/out/vs/workbench/workbench.desktop.main.js:2952:336)

Not sure if relevant, but I also add the error I see in "Log Window" -- sometimes only the first of the 2 errors is reported

[2020-03-25 15:05:19.148] [renderer1] [error] timeout after 500 ms: Error: timeout after 500 ms
    at t.RawDebugSession.handleErrorResponse (file:///opt/visual-studio-code/resources/app/out/vs/workbench/workbench.desktop.main.js:2954:952)
    at file:///opt/visual-studio-code/resources/app/out/vs/workbench/workbench.desktop.main.js:2954:250
    at async t.RawDebugSession.shutdown (file:///opt/visual-studio-code/resources/app/out/vs/workbench/workbench.desktop.main.js:2952:336)
[2020-03-25 15:05:20.859] [renderer1] [error] TreeError [CustomView] Data tree node not found: [object Object]: Error: TreeError [CustomView] Data tree node not found: [object Object]
    at H.getDataNode (file:///opt/visual-studio-code/resources/app/out/vs/workbench/workbench.desktop.main.js:1097:771)
    at H._updateChildren (file:///opt/visual-studio-code/resources/app/out/vs/workbench/workbench.desktop.main.js:1095:64)
    at H.updateChildren (file:///opt/visual-studio-code/resources/app/out/vs/workbench/workbench.desktop.main.js:1094:798)
    at file:///opt/visual-studio-code/resources/app/out/vs/workbench/workbench.desktop.main.js:5773:30
    at Array.map (<anonymous>)
    at G.doRefresh (file:///opt/visual-studio-code/resources/app/out/vs/workbench/workbench.desktop.main.js:5773:21)
    at G.refresh (file:///opt/visual-studio-code/resources/app/out/vs/workbench/workbench.desktop.main.js:5772:111)
    at p.$refresh (file:///opt/visual-studio-code/resources/app/out/vs/workbench/workbench.desktop.main.js:3671:510)
    at g._doInvokeHandler (file:///opt/visual-studio-code/resources/app/out/vs/workbench/workbench.desktop.main.js:3681:363)
    at g._invokeHandler (file:///opt/visual-studio-code/resources/app/out/vs/workbench/workbench.desktop.main.js:3681:55)
    at g._receiveRequest (file:///opt/visual-studio-code/resources/app/out/vs/workbench/workbench.desktop.main.js:3679:739)
    at g._receiveOneMessage (file:///opt/visual-studio-code/resources/app/out/vs/workbench/workbench.desktop.main.js:3678:518)
    at file:///opt/visual-studio-code/resources/app/out/vs/workbench/workbench.desktop.main.js:3676:754
    at c.fire (file:///opt/visual-studio-code/resources/app/out/vs/workbench/workbench.desktop.main.js:110:24)
    at v.fire (file:///opt/visual-studio-code/resources/app/out/vs/workbench/workbench.desktop.main.js:382:801)
    at t.PersistentProtocol._receiveMessage (file:///opt/visual-studio-code/resources/app/out/vs/workbench/workbench.desktop.main.js:387:276)
    at file:///opt/visual-studio-code/resources/app/out/vs/workbench/workbench.desktop.main.js:384:321
    at c.fire (file:///opt/visual-studio-code/resources/app/out/vs/workbench/workbench.desktop.main.js:110:24)
    at g.acceptChunk (file:///opt/visual-studio-code/resources/app/out/vs/workbench/workbench.desktop.main.js:380:306)
    at file:///opt/visual-studio-code/resources/app/out/vs/workbench/workbench.desktop.main.js:379:657
    at Socket.t (file:///opt/visual-studio-code/resources/app/out/vs/workbench/workbench.desktop.main.js:390:450)
    at Socket.emit (events.js:203:13)
    at addChunk (_stream_readable.js:295:12)
    at readableAddChunk (_stream_readable.js:276:11)
    at Socket.Readable.push (_stream_readable.js:210:10)
    at Pipe.onStreamRead (internal/stream_base_commons.js:166:17)

[karthiknadig]

@lorenzo-cavazzi Can you add this debug configuration and try? this will collect debugger logs to python extension directory. Please share all the *.log files, there should be multiple.

        {
            "name": "Debug Test",
            "type": "python",
            "request": "test",
            "console": "integratedTerminal",
            "redirectOutput": false,
            "logToFile": true
        },

[lorenzo-cavazzi]

I get only 1 log per run.
One thing worth to mention is that "logToFiles" is underlined with a warning Property logToFile is not allowed

debugger.vscode_942eb306-ea49-4510-aaeb-c3943e43f3e2.log

[karthiknadig]

The warning is fine. This is a hidden property. Can we try another thing? Lets start the debug server externally and connect to it from vscode. From a terminal outside of VS Code, activate the environment that you need and start the debugger like this. This should dump logs to /home/lorenzo/development/sdsc/renku-python:

python /home/lorenzo/.vscode-oss/extensions/ms-python.python-2020.3.69010/pythonFiles/lib/python/debugpy/wheels/debugpy/adapter --host localhost --port 5678 --log-dir /home/lorenzo/development/sdsc/renku-python

Remove the previous test configuration with this one:

        {
            "name": "Debug Test",
            "type": "python",
            "request": "test",
            "console": "integratedTerminal",
            "redirectOutput": false,
            "debugServer": 5678
        },

Then start debugging the tests, as you normally do. If things go right you should see logs in your project directory.

[lorenzo-cavazzi]

This helped me finding the error! 🎉 Thanks a ton!

First time I tried to execute it, I got

> python /home/lorenzo/.vscode-oss/extensions/ms-python.python-2020.3.69010/pythonFiles/lib/python/debugpy/wheels/debugpy/adapter --host localhost --port 5678 --log-dir /home/lorenzo/development/sdsc/renku-python

Traceback (most recent call last):
  File "/home/lorenzo/.pyenv/versions/3.7.5/lib/python3.7/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/home/lorenzo/.pyenv/versions/3.7.5/lib/python3.7/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/home/lorenzo/.vscode-oss/extensions/ms-python.python-2020.3.69010/pythonFiles/lib/python/debugpy/wheels/debugpy/adapter/__main__.py", line 213, in <module>
    locale.setlocale(locale.LC_ALL, "")
  File "/home/lorenzo/.local/share/virtualenvs/renku-python-WcnE3QeY/lib/python3.7/locale.py", line 608, in setlocale
    return _setlocale(category, locale)
locale.Error: unsupported locale setting

So I tried first executing export LC_ALL=C. I could start it and I got a similar error in VScode

env DEBUGPY_LAUNCHER_PORT=36307 DEBUGPY_LOG_DIR=/home/lorenzo/development/sdsc/renku-python /home/lorenzo/.local/share/virtualenvs/renku-python-WcnE3QeY/bin/python /home/lorenzo/.vscode-oss/extensions/ms-python.python-2020.3.69010/pythonFiles/lib/python/debugpy/wheels/debugpy/launcher /home/lorenzo/.vscode-oss/extensions/ms-python.python-2020.3.69010/pythonFiles/testlauncher.py /home/lorenzo/development/sdsc/renku-python pytest --rootdir /home/lorenzo/development/sdsc/renku-python --junitxml=/tmp/tmp-79105JpOLVZ5FXLgd.xml --no-cov ./tests/cli/test_init.py::test_template_selection_helpers 
Traceback (most recent call last):
  File "/home/lorenzo/.pyenv/versions/3.7.5/lib/python3.7/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/home/lorenzo/.pyenv/versions/3.7.5/lib/python3.7/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/home/lorenzo/.vscode-oss/extensions/ms-python.python-2020.3.69010/pythonFiles/lib/python/debugpy/wheels/debugpy/launcher/__main__.py", line 78, in <module>
    locale.setlocale(locale.LC_ALL, "")
  File "/home/lorenzo/.local/share/virtualenvs/renku-python-WcnE3QeY/lib/python3.7/locale.py", line 608, in setlocale
    return _setlocale(category, locale)
locale.Error: unsupported locale setting

Trying to start VScode with LC_ALL=C code fixed it 🎉

Is there any better way to fix this in VScode, like a specific setting?

[int19h]

What's your LANG and/or LC_ALL set it if you don't set it explicitly?

[lorenzo-cavazzi]

LANG and LC_ALL were set in .bashrc / .zshrc, so they looked fine to me but it seems they were not applied everywhere in the session 🤷‍♂️

[int19h]

We do setlocale(""), which should make those variables apply. But if they're not set at all, then it should be equivalent to setlocale("C"). Given that this works for you, the variables must be set to something else, and that value is not a supported locale - so I'm curious what that is.

[lorenzo-cavazzi]

LC_ALL was not set, LANG was en_US

[int19h]

Ah, that would explain why it's failing. The correct locale setting should include the encoding, e.g. en_US.UTF-8. I can actually replicate the error now by doing:

env LANG=en_US python3 -c "import locale; locale.setlocale(locale.LC_ALL, '')"

while this works fine:

env LANG=en_US.UTF-8 python3 -c "import locale; locale.setlocale(locale.LC_ALL, '')"

Anyway, the takeaway is that we cannot assume that setlocale("") can't fail - the fix will be to swallow any errors that come out of that, and just keep running with "C" locale in effect.

But you might also want to fix the locale setting anyway, because other apps are likely being affected by it, just not failing so noisily. If it's something that the distro sets automatically, it's a bug on their end - if you file it, the above shell / Python snippets should be enough to demonstrate the problem for a minimal repro without debugpy in the picture.