Differentiate between footnotes and endnotes

[link2xt]

Some formats, like OpenDocument and Docx have support for footnotes and endnotes. LaTeX has endnotes package. FB3 format will have an option for endnotes: https://github.com/gribuser/FB3/blob/master/fb3_links.xsd

Now converting Docx to ODT or ODT to Docx converts all endnotes to footnotes. (edit: i stated that converting to ODT converted to endnotes, it was a result of my local modifications)

I suggest to add footnote/endnote marker to AST to avoid losing this information for ODT and Docx. For LaTeX read footnotes as footnotes and add support for endnotes package later. For wiki-like formats it has to be footnotes for compatibility so users converting markdown to PDF via LaTeX are not surprised when they see their notes placed at the end of the document.

[link2xt]

Definition even has a comment saying "footnote or endnote":
https://github.com/jgm/pandoc-types/blob/f0731bf2106f0cd38c34a25c59f16ccc9082fff7/Text/Pandoc/Definition.hs#L266

I want to add a NoteType similar to QuoteType and MathType and add it to Note constructor. If there are no objections, I can try to do it myself and submit a PR for pandoc-types.

[jgm]

Any discussion of changes to Note in the AST should also consider these issues: #1603, #2053.
Fixing those issues would seem to require storing the note reference separately from the note contents.

[link2xt]

Only #1603 requires storing note reference separately from the note contents. Great idea, as it makes sense to refer to the same footnote multiple times sometimes, but I rarely see it in real texts. Also, as pointed out in #2053 discussion, referring to previous footnote in LaTeX can only be done manually with \footnotemark. I think #1603 is a completely separate issue which can be postponed.

As for #2053, it is just a problem with Markdown reader. It is correct that trying to convert it to PDF via LaTeX will result in missing footnotes, but it is a problem of LaTeX writer or LaTeX itself.

What I see in real texts is endnotes referenced from footnotes (also discussed in #2053, @Kedrigern even suggested adding endnotes), but I see no problem with them after making AST changes that I suggest. LibreOffice does not support endnotes inside footnotes, manually produced .fodt document with endnote placed inside footnote does not load, but it is the bug of LibreOffice I think. LaTeX document with endnote inside footnote compiles correctly.

[link2xt]

Pull request submitted: jgm/pandoc-types#34

[link2xt]

Just a note: texinfo has @footnotestyle which can be set to end or separate. Need to test if they can be mixed to emulate footnotes and endnotes.

[adunning]

There's also #1425 (migrated from the mailing list), which has some interesting discussion attached to it

[link2xt]

@adunning Looks like I implemented exactly the same thing as described except for indentation (EndNote instead of Endnote) and I have no Marginnote.

[rriemann]

Looking forward to use endnotes with pandoc! Hurray!

[jeremyong]

Found this issue looking for marginnote support directly. Is there any motion and getting this functionality in? Thanks

[brainchild0]

My understanding is that the current structure of the AST, which makes no distinction among notes according to type, most appropriately follows the objective of capturing a semantic representation of a document.

Distinction between footnotes and endnotes (and indeed, further, margin notes) appears to be a choice of styling based on the nature of a publication.

The Chicago Manual of Style explains that notes may be formatted according to either method, but appears to assume that a publication will generally commit to one or the other.

Relevant comments begin in Section 14.39 (16th Edition):

Readers of scholarly printed works usually prefer footnotes for ease of reference.

They continue, in Section 14.40:

Because of [flexibility from simpler page layout], and because pages free of footnotes are less intimidating to many readers, publishers' marketing and sales staff may recommend endnotes in books directed to general as well as scholarly or professional readers.

Finally, a special case appears, in Section 14.44:

In a heavily documented work it is occasionally helpful to separate substantive notes from source citations. In such a case, the citation notes should be numbered and appear as endnotes. The substantive notes, indicated by asterisks and other symbols, appear as footnotes.

Following the last remarks, a means to distinguish among notes in a document may be valuable, but relevant observations point away from a static distinction between two kinds, which all readers and writers would attempt to support, and toward a distinction by user-defined class names, which writers can variously be configured to process as desired for a specific document.

Indeed, all notes appearing in a single sequence best supports the basic case, of only one method of formatting notes being available in the output.

Whereas classical publishing appears to support at least three common styles of notes, the meaning of these styles transfers poorly to digital display targets. In a hypertext document, such as a web page, footnotes and endnotes appear to lack distinction generally, and margin notes are possible in principle, but often only if a margin is created expressly for such purpose. Popup notes become possible in such an interactive environment, but never of course in hard printing.

Division of notes in a document into two kinds, as suggested, appears to be unable to generalize.

I recommend the following:

• Roll out support in writers for producing endnotes instead of footnotes according to user option.
• Open discussion over attaching class names to notes. (The most expedient approach, involving no AST changes, is to wrap inside a span.)
• Add options for writers to support user control over notes style according to class names.

In LaTeX support is possible through very minor changes, which are to use the endnotes package, to abstract the macro for notes into one that can be redefined from \footnote to \endnote, and finally, to add \theendnotes to the document end. Despite objections over using further packages in the output document, I am aware of no compelling reason to avoid it.

Note however that formatting a LaTeX document with footnotes and endnotes combined is less trivial than merely using the basic macros provided by the endnotes package. Through the basic use, both notes are indicated by identically-styled numeric superscripts (see example). Further customization would be required to comport with, for example, the CMS recommendation that footnotes are marked by non-numeric symbols.

[jgclark]

@brainchild0 gives us very helpful analysis. I can't comment on the practicalities of the AST, but I can say that a work around for me at the moment is to use LaTeX classes to use the endnotes package etc. as you describe.

[the-solipsist]

Just wanted to point out that endnotes are sometimes section-level endnotes and sometimes document-level endnotes. The way of addressing this issue would ideally allow for that flexibility. As a somewhat-related aside: currently, the markdown writer has --reference-location = block|section|document which is used to "specify whether footnotes (and references, if reference-links is set) are placed at the end of the current (top-level) block, the current section, or the document. The default is document."

[brainchild0]

Just wanted to point out that endnotes are sometimes section-level endnotes and sometimes document-level endnotes. The way of addressing this issue would ideally allow for that flexibility.

Yes, ultimately it's futile to try to represent these distinctions in the transitional document schema (the AST). Semantic tags that the publisher (i.e. writer and user) can resolve flexibly according to the constraints of the target format, and the intention for the document appearance, are necessary to cover the full breadth of possible cases.

[the-solipsist]

Yes, ultimately it's futile to try to represent these distinctions in the transitional document schema (the AST). Semantic tags that the publisher (i.e. writer and user) can resolve flexibly according to the constraints of the target format, and the intention for the document appearance, are necessary to cover the full breadth of possible cases.

I didn't quite understand what precisely you're suggesting.

The four most common options I've seen for paged media seem to be: (1) bottom-of-the-page, (2) margin-of-the-text, (3) end-of-section, (4) end-of-document. The first two are kinds of footnotes (the note text is shown close to the note reference), while the latter two are kinds of endnotes (the notes are collected together and shown in a location separate from the reference).

But in essence, there are two types: footnotes and endnotes, and their placement may vary.

I did some research on various (paged-media) formats, and here's what I found:

• EPUB3 currently has two options, with placement variations: <aside id="fn01" role="doc-footnote" epub:type="footnote"> for footnote text and <aside id="fn01" role="doc-endnote" epub:type="endnote"> for endnote (formerly "rearnote") text. You can also alter where the endnotes are displayed by instead using <section role="doc-endnotes" epub:type="endnotes">. Marginilia/sidebars are indicated by the aside element, and there is no epub:type for them anymore (they've been deprecated).
• ODT has two options: text:note-class="footnote" and text:note-class="endnote", and four placement variations: the attribute text:footnotes-position can have the values: document, page, section, text.
• DOCX (OOXML) has two options: w:footnotes and w:endnotes. Footnotes having the following 4 positions: beneathText, pageBottom, docEnd and sectEnd. Endnotes have the following 2 positions: docEnd and sectEnd.
• LaTeX natively only has one option: \footnote, and without using additional packages like footmisc or endnotes/enotez/pagenotes, footnote position cannot be specified, nor can endnotes be represented.

For unpaged formats (like HTML), all of this is ultimately handled by CSS. (The default HTML template for pandoc creates footnote references using <a href="#fn1" class="footnote-ref" id="fnref1" role="doc-noteref"> and footnote sections using <section class="footnotes" role="doc-endnotes">.) While there is a proposal to add a note element from folks concerned with accessibility, there isn't a specific wayelement to indicate footnotes in HTML yet. The Digital Publishing WAI-ARIA Module 1.0 provides for role="doc-endnote" (for each endnote), role="doc-endnotes" (for the endnotes section), and role="doc-footnote".

So, it seems to me that information as to whether a note is a footnote or an endnote should at the least be preserved during conversion, given that EPUB3, ODT, and DOCX all support those two. The location of their display is something that can be controlled via templates, I think.

[kysko]

Just adding a detail that might (or might not) help in this matter.

My understanding is that the current structure of the AST, which makes no distinction among notes according to type, most appropriately follows the objective of capturing a semantic representation of a document.

Since the posting of this comment in may 2020, there has been a release (2.10.1 in july 2020) that added the capability to give attributes to all elements (at least for commonmark and gfm for now).

So although Notes themselves make no distinction among types of notes, one could indicate the intent of a different kind of note. In commonmark_x, one can write:

foo[^1]{.endnote}

[^1]: bar

and the result in the AST looks like :

[Para [Str "foo", Span ("",["endnote"],[]) [Note [Para [Str "bar"]]]]]

Note here that the Span affects the Note, not the preceding element (Str "foo").
(From the release notes linked above:

Since the Pandoc AST doesn’t include attributes on each element type, the attributes will sometimes be added by creating a surrounding Div or Span container.

)

This would give a way, as the-solipsist asks, to preserve the information of that intent.
Of course, writers for outputs that would support that kind of differentiation would need to make use of it, as indicated in the preceding comment by the-solipsist.

/+: ... And if the input document supports both footnotes and endnotes, as mentioned in the opening issue message, the appropriate reader could also put that information in the Span, in case a writer supports it.

/++: ... missed that quote from bc in may, sorry:

The most expedient approach, involving no AST changes, is to wrap inside a span

[brainchild0]

I didn't quite understand what precisely you're suggesting.

So, it seems to me that information as to whether a note is a footnote or an endnote should at the least be preserved during conversion, given that EPUB3, ODT, and DOCX all support those two. The location of their display is something that can be controlled via templates, I think.

My comment was not meant as an objection to yours, though the context may have inadvertently implied such an intention.

The futility is to generalize all possible note types and attributes into the AST (the internal, intermediary document format used for conversion between formats), because the set of possibilities is unbounded in general, as illustrated by the numerous permutations outlined in your comments. This observation is relevant because the original request proposes changes to the AST to capture the distinction between footnotes and endnotes. Generalizing this approach to achieve the further functionality you propose amounts to chasing the end of the rainbow, such that the AST would be augmented constantly and endlessly in an effort to capture full generality for notes.

My comment only suggests that an effective solution is likely to be one other than adding the greater generality to the AST. I support the broader objective of more flexible handling of notes.

[mars0i]

Sorry to ask a non-technical question in a github issue, but I gather from the preceding that there is still no general method to cause pandoc to create endnotes rather than footnotes. Is that correct? It appears that --reference-location can produce something like endnotes, in the form of endnotes at the end of a document, but only for Markdown output. (I've spent a lot of time trying to sift through the pandoc documentation and the web, and so far this github issue is actually the most informative page I've found on the topic.) I'll ask in Google groups, too, but I hope you don't mind me asking here.

[jgm]

That's right. There are notes. Whether they render as footnotes or endnotes really depends on the output format. (Some output formats don't separate things into pages anyway, which makes the distinction irrelevant.) But there's no way to have two sets of notes, numbered independently, one rendered as footnotes and the other as endnotes.

If you just need one set of notes but want them to be endnotes, then there may be ways to make this happen, depending on the output format you want.

[mars0i]

Thanks @jgm! Yes, I just want endnotes. When I use the LaTeX \footnote command, the notes render on the page in Word, as expected. When I use \usepackage{endnotes}, \endnote, and \theendnotes, I end up with no notes in Word--no note numbers, no notes. (I suppose it doesn't really matter if they appear in any particular place in the document. I need the note numbers to match the notes, but if all of the notes are in one place, I can copy them to somewhere else.) Oh, right--and I'd like the numbering to restart with each chapter. I forgot about that. Maybe that's more difficult.

[mars0i]

A friend pointed out that I don't need pandoc to create endnotes! One can highlight footnotes in Word and make it convert them to endnotes. I hope that there is or will be a way for pandoc to create endnotes, but for my immediate needs I guess I don't need it. (MS Word is not my native language.)

Sorry for cluttering up the github issue.

[donboyd5]

Just wanted to add my hope that endnotes can be supported. My rationale is here. Many thanks.

[johannesburg]

I also hope it can be supported. Even if one can choose whether notes in the .md file are endnotes or footnotes when converted to .docx would be a big step. Thanks in advance!

[ghost]

It would be interesting to be able to put the footnotes in a custom location, as we can do with references:

# Heading of my choice

<div id="refs"></div>

This would be the easiest solution to anyone that want in a custom place the footnotes and not at the end of their respective page.

Obviously, this would not solve the cases in which you need footnotes and endnotes in the same document.