1.14.4: Tags are not saved in MyST format

[davidvandebunte]

Add example tags to a MyST .md cell with a plain text editor as described here:

• https://github.com/mwouts/jupytext/blob/main/docs/formats.md#myst-markdown

For example:

:tags: [hide-output, show-input]

When I then open the .md file with Jupytext, make changes, and save, the tags are removed. It seems like this is an obvious issue, but can anyone else reproduce? I'm having trouble guessing what I'm doing wrong.

Saving the tags to a cell within Jupyter to start also doesn't get them written (just ignored on save).

[maxbane]

I've encountered the same problem. When I author my notebooks as .ipynb, then convert them to MyST .md, none of the cell tags are preserved.

[mwouts]

Well, my expectation is that tags are supported in all the main formats.

I will soon add a test to make sure this is the case.

Locally the test pass, so maybe this might indicate that you are not using the latest implementation of the MyST format? What versions of markdown-it-py and of mdit_py_plugins do you have?

[maxbane]

Hi @mwouts. I have created a repository with a minimal reproducible example for you:
https://github.com/maxbane/repro-jupytext-myst-tags

You can see that all I do is pip install the latest version of jupytext, and use it convert a minimal ipynb file (which has a tagged cell) to MyST markdown. As I'm sure you can confirm by running the same, the output has no tags.

[maxbane]

To answer your question about dependency versions, you can see from the versions.txt file that I included in the above-linked repository that pip install jupytext in a fresh venv results in the following versions of markdown-it-py and mdit_py_plugins:

markdown-it-py==2.1.0
mdit-py-plugins==0.3.3

If more recent versions are needed for tags to function, perhaps jupytext's declared dependencies need to be updated? Above are what come from a plain pip install jupytext in a fresh environment.

Using Python 3.10, FWIW.

[mwouts]

Oh thank you @maxbane for taking the time to create the example repository!

Well, what happens in your example is that you have "cell_metadata_filter": "-all", in your notebook metadata.

This is the reason why you don't get any cell metadata in the text representation!
Can you remove this cell_metadata_filter and give it another try?

[maxbane]

Aha, thank you @mwouts for spotting that! Removing that does indeed solve the problem.

So now I've been wracking my brain trying to figure out where that "cell_metadata_filter": "-all" came from in the first place; I certainly never added it intentionally.

It turns out, it's from the jupyter-book starter template! Probably like many other people using MyST notebooks, I'm using jupyter-book to build my documentation, together with jupytext to sync between markdown/MyST (which go into source control) and ipynb representations (which can be edited WYSIWYG).

I simply followed the "getting started" directions here: https://jupyterbook.org/en/stable/start/create.html...

Create a new virtual environment, and pip install jupyter-book. Then run jupyter-book create mynewbook/ to generate the starter template. Examine the generated file mynewbook/markdown-notebooks.md:

---
jupytext:
  cell_metadata_filter: -all
  formats: md:myst
  text_representation:
    extension: .md
    format_name: myst
    format_version: 0.13
    jupytext_version: 1.11.5
kernelspec:
  display_name: Python 3
  language: python
  name: python3
---

# Notebooks with MyST Markdown

Jupyter Book also lets you write text-based notebooks using MyST Markdown.
[...]

There it is! That's where it came from. Why in the HECK is the jupyter-book template disabling all cell metadata??? That file became the copy-and-paste basis for all of my documentation notebooks. Surely every new jupyter-book user is starting their project by building on the template, and unknowingly disabling cell metadata in their MyST notebooks! No wonder I could never successfully hide cell inputs or remove cells from HTML output!

Thanks for identifying the problem here. You can probably close this issue.

@davidvandebunte, is it the same for you?

[davidvandebunte]

It is! My workflow was the same as yours, i.e. copying/pasting headings from one file to another starting from the jupyter-book starter template (and my goal was even the same, to hide cell inputs). I agree we can close this issue and instead make changes in jupyter-book.

[mwouts]

Great news! Thank you also for opening the linked PR on the Jupyter book project!