WARNING: Could not lex literal_block as "python". Highlighting skipped.

[adamjstewart]

I have a cell block containing a shell command:

!nvidia-smi

  {
   "cell_type": "code",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "4QhMOtYzLmVK",
    "outputId": "fa0443da-8b4d-47f7-e713-93e1e4976e87"
   },
   "source": [
    "!nvidia-smi"
   ],
   "execution_count": 8,
   "outputs": [
    {
     "output_type": "stream",
     "name": "stdout",
     "text": [
      "Tue Sep 28 20:52:49 2021       \n",
      "+-----------------------------------------------------------------------------+\n",
      "| NVIDIA-SMI 470.63.01    Driver Version: 460.32.03    CUDA Version: 11.2     |\n",
      "|-------------------------------+----------------------+----------------------+\n",
      "| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |\n",
      "| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |\n",
      "|                               |                      |               MIG M. |\n",
      "|===============================+======================+======================|\n",
      "|   0  Tesla K80           Off  | 00000000:00:04.0 Off |                    0 |\n",
      "| N/A   33C    P8    27W / 149W |      3MiB / 11441MiB |      0%      Default |\n",
      "|                               |                      |                  N/A |\n",
      "+-------------------------------+----------------------+----------------------+\n",
      "                                                                               \n",
      "+-----------------------------------------------------------------------------+\n",
      "| Processes:                                                                  |\n",
      "|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |\n",
      "|        ID   ID                                                   Usage      |\n",
      "|=============================================================================|\n",
      "|  No running processes found                                                 |\n",
      "+-----------------------------------------------------------------------------+\n"
     ]
    }
   ]
  },

This runs and renders fine on Google Colab. However, when I build my docs with nbsphinx, I see the following warning message:

WARNING: Could not lex literal_block as "python". Highlighting skipped.

This seems to work for nbsphinx, but not for Jupyter:

%nvidia-smi

This seems to work for Jupyter, but not for nbsphinx:

!nvidia-smi

See https://readthedocs.org/projects/torchgeo/builds/17978930/ for the failing build and see microsoft/torchgeo#756 for links to the full notebook. This is with nbsphinx 0.8.9.

[mgeier]

The code highlighting is done with pygments, and the information about the language is taken from the cell metadata magics_language. If that's not present (as in your case), the notebook metadata language_info.pygments_lexer is checked. If that's not present (as in your case), language_info.name is used, which in your case is python, so it doesn't know the special IPython syntax.

nbsphinx/src/nbsphinx.py

Lines 117 to 123 in fc79fd1

	.. nbinput:: {% if cell.metadata.magics_language -%}
	{{ cell.metadata.magics_language }}
	{%- elif nb.metadata.language_info -%}
	{{ nb.metadata.language_info.pygments_lexer or nb.metadata.language_info.name }}
	{%- else -%}
	{{ resources.codecell_lexer }}
	{%- endif -%}

In general, you could override it with nbsphinx_codecell_lexer but language_info.name seems to take precedence. Maybe that should be changed?

[mgeier]

BTW, if you just want to ignore the warning, you can use that in your conf.py:

suppress_warnings = ['misc.highlighting_failure']

[adamjstewart]

Yes, I ignored the warning for now. What's the correct value for magics_language, ipython?

[adamjstewart]

Is there any downside to using ipython as the lexer for all .ipynb files? I'm surprised the default is python instead of ipython.

[adamjstewart]

Yep, that seems to work. Thanks for all of your help!

[mgeier]

What's the correct value for magics_language, ipython?

I don't know.
I've never knowingly seen a notebook where this was set.
I don't know if there is any client that writes to that field.
I don't know why I originally implemented it that way (see #59 and #60, which was 6 years ago!), maybe I copied it from nbconvert?

But both ipython and ipython3 should work.

Is there any downside to using ipython as the lexer for all .ipynb files?

I'm not sure.

The idea is that Jupyter notebooks are language-agnostic, so I would prefer not assuming IPython.

I'm surprised the default is python instead of ipython.

Well, none of those are explicitly the default, but nb.metadata.language_info.name seems to be python.
But that may not be a good idea, because, as we saw, this cannot handle IPython-specific features.

So we might just remove this?
This would lead to no highlighting instead of raising an error.

In such a case, a user could override the language with nbsphinx_codecell_lexer.

Does that make sense?

[adamjstewart]

A warning and highlighting is definitely better than no warning and no highlighting.