BUG: to_latex: Problems with MultiIndex / Mixed Index

[wlter]

Code Sample, a copy-pastable example if possible

import pandas as pd
import numpy as np


df = pd.DataFrame()


for val0 in [3,2,1]:
  for val1 in range(val0):
    r0,r1 = np.random.uniform(size = 2)
    df = df.append({"val0" : val0,\
                          "val1":val1, 
                          "r0":r0,\
                          "r1":r1}, ignore_index=True)

print(df.to_latex())
print(df.to_latex(columns = ["r0","r1"],header = ["R0","R1"]))
  
df.set_index(['val0','val1'], inplace=True)

print(df.to_latex())
print(df.to_latex(columns = ["r0","r1"],header = ["R0","R1"]))

produces:

\begin{tabular}{lrrrr}
	\toprule
	{} &        r0 &        r1 &  val0 &  val1 \\
	\midrule
	0 &  0.631918 &  0.445245 &   3.0 &   0.0 \\
	1 &  0.418763 &  0.011685 &   3.0 &   1.0 \\
	2 &  0.742899 &  0.384768 &   3.0 &   2.0 \\
	3 &  0.275884 &  0.849828 &   2.0 &   0.0 \\
	4 &  0.371853 &  0.782338 &   2.0 &   1.0 \\
	5 &  0.763312 &  0.358247 &   1.0 &   0.0 \\
	\bottomrule
\end{tabular}

\begin{tabular}{lrr}
	\toprule
	{} &        R0 &        R1 \\
	\midrule
	0 &  0.631918 &  0.445245 \\
	1 &  0.418763 &  0.011685 \\
	2 &  0.742899 &  0.384768 \\
	3 &  0.275884 &  0.849828 \\
	4 &  0.371853 &  0.782338 \\
	5 &  0.763312 &  0.358247 \\
	\bottomrule
\end{tabular}

\begin{tabular}{llrr}
	\toprule
	&     &        r0 &        r1 \\
	val0 & val1 &           &           \\
	\midrule
	3.0 & 0.0 &  0.631918 &  0.445245 \\
	& 1.0 &  0.418763 &  0.011685 \\
	& 2.0 &  0.742899 &  0.384768 \\
	2.0 & 0.0 &  0.275884 &  0.849828 \\
	& 1.0 &  0.371853 &  0.782338 \\
	1.0 & 0.0 &  0.763312 &  0.358247 \\
	\bottomrule
\end{tabular}

\begin{tabular}{llrr}
	\toprule
	&     &        R0 &        R1 \\
	val0 & val1 &  0.631918 &  0.445245 \\
	\midrule
	3.0 & 0.0 &  0.418763 &  0.011685 \\
	& 1.0 &  0.742899 &  0.384768 \\
	& 2.0 &  0.275884 &  0.849828 \\
	2.0 & 0.0 &  0.371853 &  0.782338 \\
	& 1.0 &  0.763312 &  0.358247 \\
	\bottomrule
\end{tabular}

Problem description

Hi,

just found out about the to_latex-functionality. Thumbs up for that! Unfortunately I seem to have stumbled upon a bug - or I'm not getting multiindices right.

I'd like to use to_latex with a multiindex-array, such that the hierarchical parameters don't repeat in the rows. The example code produces 4 tabular sections of latex code. The first two, without the multi-index seem okay. The 3rd, with "multiindex = True" starts to introduce an unnecessary shift in the header names. The multi-index headers are a level below the others. When using the "header" option with this (4th block), the data gets shifted and the last row of the multiindex is missing completely.

It actually seems to be a problem with index mixing. If the index is created over all columns (df.set_index(['val0','val1',"r0","r1"], inplace=True)), then the table is complete again - there's however on more blank row above all header names.

So, this might be a bug... but I'd also accept that incomplete multi-indices are just not the way to go...

Expected Output

• all header names on the same level
• multiindex and non-multiindex results not shifted
• all parameters of the multiindex visible

Output of pd.show_versions()

INSTALLED VERSIONS ------------------ commit: None python: 3.5.3.final.0 python-bits: 32 OS: Windows OS-release: 7 machine: AMD64 processor: AMD64 Family 16 Model 4 Stepping 3, AuthenticAMD byteorder: little LC_ALL: None LANG: en LOCALE: None.None

pandas: 0.20.2
pytest: 3.0.7
pip: 9.0.1
setuptools: 27.2.0
Cython: 0.25.2
numpy: 1.12.1
scipy: 0.19.0
xarray: None
IPython: 5.3.0
sphinx: 1.5.6
patsy: 0.4.1
dateutil: 2.6.0
pytz: 2017.2
blosc: None
bottleneck: 1.2.1
tables: 3.2.2
numexpr: 2.6.2
feather: None
matplotlib: 2.0.2
openpyxl: 2.4.7
xlrd: 1.0.0
xlwt: 1.2.0
xlsxwriter: 0.9.6
lxml: 3.7.3
bs4: 4.6.0
html5lib: 0.999
sqlalchemy: 1.1.9
pymysql: None
psycopg2: None
jinja2: 2.9.6
s3fs: None
pandas_gbq: None
pandas_datareader: None

[toobaz]

The 3rd, with "multiindex = True" starts to introduce an unnecessary shift in the header names. The multi-index headers are a level below the others.

I think this is a feature, not a bug. Not just because it looks clearer (otherwise you wouldn't distinguish index levels from columns), but also for coherence with the case in which columns also have a name.

When using the "header" option with this (4th block), the data gets shifted and the last row of the multiindex is missing completely.

This clearly looks like a bug.

[sgsaenger]

@toobaz correctly explained the third case.

The 4th case is indeed a bug, but is caused in DataFrameFormatter._to_str_columns which is the source of the initial string representation. Compare the .to_string that also uses _to_str_columns method and shows similar behavior:

print(df.to_string(header=['R0','R1'])

                 R0        R1
val0 val1  0.872907  0.932874
3.0  0.0   0.729201  0.343559
     1.0   0.819604  0.447002
     2.0   0.419192  0.076606
2.0  0.0   0.955415  0.317518
     1.0   0.141231  0.875158
1.0  0.0