[ARROW-13159] [Doc][Python] The use of IPython directive or doctest code blocks in the python user guide

• Type: Improvement
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Component/s: Documentation, Python
• External issue URL: https://github.com/apache/arrow/issues/28859
• Assignee: Unassigned
• Reporter: Joris Van den Bossche
• Created: 24/Jun/21 09:35
• Updated: 11/Jan/23 08:31

Description

From https://github.com/apache/arrow/pull/10266#discussion_r630837422

We are currently using the IPython directive in many places in the Python docs, so that something written as

.. ipython:: python

  x = 1
  x + 2

is converted during the doc build to (by running the code):

.. code-block:: ipython

  In [1]: x = 1 

  In [2]: x + 1
  Out[2]: 2

Running all the code during the doc build can be costly, and the more docs we add, the slower building the docs becomes.

We could convert all those to code-block, but personally I think ideally we still check the code examples for correctness, where applicable. For this, we could also use the doctest format instead of the IPython directive, and verify the docs using pytest doctests support.

This can be run separate as tests, and doesn't need to be part of doc building (at least when you only change wording / rst syntax, and want to verify the resulting html, you don't need to run the doctests).

But maintaining examples as doctests also certainly adds some extra cost (eg when outputs change slightly)

Another option could also be to add an option to the IPython directive to skip the execution of the code examples (I think this should be rather easy to add to the IPython directive, but then it's still a matter of passing this through from the build command invocation).

cc apitrou amol-

Issue Links

• relates to: ARROW-16018 [Doc][Python] Run doctests on Python docstring examples
• relates to: ARROW-17076 [Python][Docs] Enable building documentation with pyarrow nightly builds