RST writer does not produce grid tables any more

[rpuntaie]

pandoc 2.7.1

#4750 seems to have changed to simple RST tables by default.

I suggest to retain the original default behavior to produce grid tables
and implement #4750 via extension: pandoc foo.org -t rst+simple_tables -o foo.rst.

[jgm]

Simple tables aren't really a syntax extension in RST. RST has a clearly defined syntax, and simple tables are always valid. So I don't think using an extension is the right approach here.

What is wrong with using simple tables when possible?

[rpuntaie]

Yes. But simple tables are no good, if you have a block of text in a cell. Then it becomes unpredictable whether simple table or grid table is generated. Better always generate grid tables.

From the user's perspective, extension is a mechanism to tell Pandoc what the user wants.

An automatic decision to use simple or grid table is OK, if a +grid_tables forces grid tables, or a +simple_tables forces simple tables, or a +list_tables forces list tables. This would also allow to use Pandoc to convert table styles.

[jgm]

Better always generate grid tables.

Better for you, but apparently not for the person whose request prompted this change.

An automatic decision to use simple or grid table is OK, if a `+grid_tables` forces grid tables, or a `+simple_tables` forces simple tables, or a `+list_tables` forces list tables. This would also allow to use Pandoc to convert table styles.

That's not how extensions work. `+foo` means that the syntax `foo` is *allowed*, not that it is forced. And we generally don't have extensions for things that are part of the base specification of a language (as both kinds of tables are for RST).

[jgm]

Then it becomes unpredictable whether simple table or grid table is generated.

It is predictable, actually. A simple table is used when the cell widths are all 0s in the internal representation of the table. The markdown reader generates tables like this for simple tables and narrow-enough pipe tables.

[rpuntaie]

Different users will have different needs. That is OK. Ideally SW should be widely usable. In this case I think it is possible to make it versatile enough to satisfy more needs. It is not yet so. I was talking about a way it could be done, not about how things work now.

It's about the writer here. "Allowed" would fit to a reader. But for the reader the table is given
and just needs to be read in. For the writer, there are more possibilities. "+grid_tables" could be interpreted as a user selection from these possibilities, extension or not. The CLI interface is a context separate from implementation. "Extension" can have (slightly) different meanings in the two contexts.

[jgm]

Extension means the same in both contexts (that's how it has always been, and I don't want to change that). You can force pipe tables in markdown output, for example, but you have to do it by disabling the extensions for other table types, not by enabling pipe tables. (After all, other kinds of tables are enabled too.)

[jgm]

I still don't understand why simple tables aren't adequate for your purposes, in the cases where pandoc produces them. Maybe you could say more.

[rpuntaie]

Actually I want a list table, as that has a line per cell and thus is more easily diffed in a VCS.
rstfromdocx creates it from a grid table. It does not work any more, because now a simple table is produced by pandoc.
In #4750 simple tables can be further processed, in my case grid tables.
It would be great to have some control on which table types are generated.
#4564 (list table option for RST) would help me, too.

[mrdrewkeller]

+1 to this, the change to simple tables for a default has been great but the option to use grid tables or pipe tables would be nice for instances where cell spans are necessary. The option shown here to specify which type of tables to use does not seem to work for the RST writer https://github.com/jgm/pandoc/wiki/Pandoc-Tricks#convert-between-the-4-table-syntaxes-in-pandoc

[jgm]

+1 to this, the change to simple tables for a default has been great but the option to use grid tables or pipe tables would be nice for instances where cell spans are necessary.

Cell spans aren't yet supported in pandoc's table model, even when writing grid tables. See #1024.

[mrdrewkeller]

I see, this may be a pretty specific use case and not worth implementing, but for what it's worth in the past I at least had the option to manually add cell spans after running pandoc when using grid tables. With the simple tables output that doesn't seem to be possible.

[reagle]

+1 for some way to force the output to grid tables since the default changed as my text editor doesn't know how to edit/reformat simple tables.

[pkarlovsek]

Why converting html (pandoc -f html -t rst data-simple.html)

<thead>
<tr>
<th>H1</th>
<th>H2</th>
</tr>
</thead>
<tbody>
<tr>
<td>1</td>
<td>2</td>
</tr>
</tbody>
</table>

to rst produces simple table

H1 H2
== ==
1  2
== ==

while converting this (the diff is <td>2</td> --> <td><ul>2</ul></td>)

<table>
<thead>
<tr>
<th>H1</th>
<th>H2</th>
</tr>
</thead>
<tbody>
<tr>
<td>1</td>
<td><ul>2</ul></td>
</tr>
</tbody>
</table>

produces grid table

+-----------------------------------+-----------------------------------+
| H1                                | H2                                |
+===================================+===================================+
| 1                                 | - 2                               |
+-----------------------------------+-----------------------------------+

But there is no way to force first example (without <ul> tags) to produce grid tables.
Consequently, we can't convert csv tables to rst grid tables.
And the same is for markdown.

pandoc -v
pandoc 3.6.2
Features: +server +lua
Scripting engine: Lua 5.4

[jgm]

The difference is because block-level content (such as lists) can't appear in simple table cells.

There is a way to achieve what you want, using a Lua filter.

The first table above will be converted to the following pandoc AST:

[ Table
    ( "" , [] , [] )
    (Caption Nothing [])
    [ ( AlignDefault , ColWidthDefault )
    , ( AlignDefault , ColWidthDefault )
    ]
    (TableHead
       ( "" , [] , [] )
       [ Row
           ( "" , [] , [] )
           [ Cell
               ( "" , [] , [] )
               AlignDefault
               (RowSpan 1)
               (ColSpan 1)
               [ Plain [ Str "H1" ] ]
           , Cell
               ( "" , [] , [] )
               AlignDefault
               (RowSpan 1)
               (ColSpan 1)
               [ Plain [ Str "H2" ] ]
           ]
       ])
    [ TableBody
        ( "" , [] , [] )
        (RowHeadColumns 0)
        []
        [ Row
            ( "" , [] , [] )
            [ Cell
                ( "" , [] , [] )
                AlignDefault
                (RowSpan 1)
                (ColSpan 1)
                [ Plain [ Str "1" ] ]
            , Cell
                ( "" , [] , [] )
                AlignDefault
                (RowSpan 1)
                (ColSpan 1)
                [ Plain [ Str "2" ] ]
            ]
        ]
    ]
    (TableFoot ( "" , [] , [] ) [])
]

If you change both occurrences of ColWidthDefault to ColWidth 0.5 (specifying a relative width), then converting this -f native -t rst will yield a grid table.

So, if you write a Lua filter that converts ColWidthDefault in tables to specific widths, and use this in your -f csv -t rst pipeline, you'll get a grid table.

Here is a filter that will do this:

function Table(el)
  local specs = el.colspecs
  local numcells = #specs
  local width = 1.0 / numcells
  el.colspecs = {}
  local i
  for i = 1,numcells,1 do
    el.colspecs[i] = {"AlignDefault", width}
  end
  return el
end

Change 1.0 to a smaller value if you don't want it to take the whole width.

An alternative approach would be a filter that encloses every Plain element in a Div.