Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Break out doctesting / refactor #774

Merged
merged 23 commits into from
Jul 1, 2019
Merged

Break out doctesting / refactor #774

merged 23 commits into from
Jul 1, 2019

Conversation

mortenpi
Copy link
Member

@mortenpi mortenpi commented Jul 31, 2018

This refactors a few parts of the build pipeline to make it possible to run doctests without having to compile the whole document (close #198, close #535, close #756).

  • Moves doctesting to an earlier stage in the build.
  • Implements doctest = :only to makedocs, which only runs doctests and skips the later build steps.
  • When fixing doctests you probably do not care about the output either, so doctest = :fix also skips compilation and rendering of the docs now.
  • Creates an API for storing module-level documentation metadata (analogue to an at-meta block), via the exported DocMeta module (e.g. DocMeta.setdocmeta!(module, :DocTestSetup, :(using Foo)). Only supports :DocTestSetup at the moment.
  • Adds an exported doctest(::Vector{Module}) function to easily run doctests in a module.

The changes are a bit more extensive than they necessarily should be, but I am also trying to clean up and refactor the internals in the process. There is a bit more info about where I want to go with these changes in the wiki.

This is still very much a WIP and I think I might have to rethink some decisions here. Also needs testing, tests etc. Any drive-by comments will be much appreciated though. This is now ready for review.

  • Update the manual page on doctesting

@mortenpi
Copy link
Member Author

So.. this should work now.. theoretically.

One hurdle I came across as I was trying to test it with the Julia manual, is that it actually breaks nearly all the doctests in docstrings. This is because they usually rely on DocTestSetups in the Markdown files that no longer have any effect on the doctesting of the docstrings.

Now, this is not necessarily a bad thing. The whole idea of this is to be able to run doctest without having to go through all the hoops of figuring out which docstring ends where etc. It is also weird that the validity of a doctest depends on some metadata far away in a manual .md file. But having to put at-meta blocks into each docstring (which is now technically supported) is not optimal either.

So I've prototyped a way to add doctest metadata to modules instead. Essentially, every module gets a special gensymed variable that contains the metadata (it's how docstrings themselves are handled). It does require Documenter -- there is the Documenter.DocTests.doctestsetup! function which declares an Expr object as a DocTestSetup of a module.

The question is: when should you actually add the metadata? Ideally, it would be close to the docstrings.

  • You could make your package depend on Documenter and just call Documenter.DocTests.doctestsetup!(@__MODULE__, :( ... )) in the module. But we don't want packages to depend on Documenter.

  • You can add the metadata in the make.jl file by just making all the doctestsetup! calls after you import your package. In a sense, this is similar to how it's done now, where the metadata lives in the .md files under docs/. This might be a good solution for Base and the standard libraries.

  • But I am also thinking about making it possible to define the metadata using a global constant in your module, e.g. const _DOCUMENTER_DOCTESTSETUP = :( ... ). Documenter can then just walk through all the modules passed with modules keyword (these are the ones that get tested anyway) and check for this variable. The downside is that it introduces a visible variable into the module namespace.

@mortenpi mortenpi added this to the 0.23.0 milestone Jun 24, 2019
@mortenpi mortenpi changed the title WIP: Break out doctesting / refactor Break out doctesting / refactor Jun 27, 2019
@mortenpi
Copy link
Member Author

Ok, this is close to being finished now I think. I've updated the OP with a summary of what this adds and changes (Documenter will also export a doctest function and DocMeta module with this).

Bikeshedding invitation: I can't decide if it should be DocMeta / DocMeta.getdocmeta / DocMeta.setdocmeta! or DocsMeta / DocsMeta.getdocsmeta / DocsMeta.setdocsmeta! though (or maybe something else altogether).

Project.toml Outdated Show resolved Hide resolved
@mortenpi mortenpi merged commit 41484c4 into master Jul 1, 2019
@mortenpi mortenpi deleted the mp/parse-step branch July 1, 2019 10:12
mortenpi added a commit that referenced this pull request Jul 3, 2019
This commits empty dev/ docs to gh-pages, introduced in #774.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
2 participants