Pandoc: Duplicate footnotes - multiple files - same name

Question

I have a folder structure like the following:

chapters/
  01-chapter.md
  02-chapter.md
  03-chapter.md
format.sh
parse.sh

parse.sh

pandoc -t html5+smart -so dist/book.html --html-q-tags chapters/*.md

format.sh

for f in chapters/*.md;
  do
    pandoc "$f" -o "$f" -t \
    markdown+smart+footnotes-escaped_line_breaks+example_lists \
    --columns=80;
done

My idea was basically to use Pandoc not only as converter but also as formatter - this worked pretty well until I began to use footnotes:

When I run format.sh every reference name I use for my footnotes get overwritten by a number - which would be fine, but the problem is if I than run parse.sh I get the following message:

[WARNING] Duplicate note reference '1' at line 360 column 1

The problem is 01-chapter.md and 02-chapter.md have both a footnote [^1]. Basically I would like Pandoc to handle the footnotes for each file separately and not allow cross-referencing across my markdown files but I can't see any way to do this.

Anyone any ideas?

Answer 1

No need for lua

I just stumbled on the --file-scope option. Add this to parse.sh and you will be off trouble...

--file-scope

Parse each file individually before combining for multifile documents. This will allow footnotes in different files with the same identifiers to work as expected. If this option is set, footnotes and links will not work across files.

Pandoc User’s Guide

Answer 2

Pandoc is not designed to work as a formatter, but it works reasonably well most of the time.

There is no way to get pandoc to preserve the original footnotes, so you'll have to find a different method. The challenge is to get pandoc to read the files sequentially as individual documents, and to then combine the documents within pandoc into the final document. Just passing all input files (via *.md) causes pandoc to concatenate these files into a single document before parsing the markdown.

My suggestion would be to use an "index" (or master) file. Pandoc Markdown has no built-in syntax for the inclusion of child-files, but we can roll our own with a pandoc Lua filter. E.g., define a file which will tie all your chapters together, where files are listed one-per-line in a code block:

``` include
chapters/1.md
chapters/2.md
chapters/3.md
```

Put the following code into a file include.lua (taken from this gist)

--- Pandoc Lua filter to include other Markdown files
local List = require 'pandoc.List'

function CodeBlock(cb)
  if cb.classes:includes'include' then
    local blocks = List:new()
    for line in cb.text:gmatch('[^\n]+') do
      if line:sub(1,1)~='#' then
        local fh = io.open(line)
        blocks:extend(pandoc.read (fh:read '*a').blocks)
        fh:close()
      end
    end
    return blocks
  end
end

Then call pandoc with the Lua filter and the master/index file as input:

pandoc -t html5+smart -so dist/book.html --html-q-tags --lua-filter=include.lua index.md

This has the disadvantage that you'll have to edit index.md whenever you add a new Markdown file; also, only metadata defined in the index file will make it into the output (although both of these limitations could be avoided with a little coding). Other than that, it should do exactly what you are looking for.