Python: Documenting Higher-Order Artefacts


This is something that I would’ve touched on in my Introduction to Functional Programming in Python talk had it been about 4 hours long.

Suppose I make a nice healthy currying function:

def curry(f, l) : return lambda *r : f(l, *r)

and then use it to define a max function like this:

max=curry(reduce, lambda x,y : [x,y][x<y])

(in Python 2.5 instead of defining curry you can go «curry = functools.partial»)

(Aside: I really don’t know what to think of that code on the right hand side of the lambda. It’s more or less forced on me by the fact that that body of a lambda is an expression so I can’t use if. It’s also a compelling reason why Python will probably continue to use Iverson’s Convention for some while.)

I’m not saying that it’s necessarily good idea to define max like this, it’s a bit of a contrived example, but when you get used to higher order functions you end up defining a lot of really useful functions like this instead of using def statements. Sometimes such a function will be sufficiently useful that it will form part of a public interface, in which case it needs documenting.

The documentation for max is rubbish:

>>> help(max)
Help on function <lambda> in module __main__:

<lambda> lambda *r

The way to fix that is to give max an explicit docstring via its secret __doc__ attribute and a proper name via its __name__ attribute:

max.__doc__="""Return largest element of sequence."""

Now the help looks like this:

>>> help(max)
Help on function max in module __main__:

    Return largest element of sequence.

The __name__ attribute means that max’s name correctly appears after «help on function» and in the function prototype. The __doc__ attribute appears as max’s description.

The way the argument list appears is slightly misleading, but I’ll leave it at that for now. Maybe there’s also a sneaky way to fix the argument list?

It occurs to me that curry could produce some sort of docstring automatically:

def okname(x) :
  """Return some okay name for any object."""
  def isnum(x) :
    """Return true if x is a number."""
    try :
      return complex(x) == x
    except Exception :
      return False
  def isstr(x) :
    """Return true if x is a string."""
    try :
      return str(x) == x
    except Exception :
      return False
  if isnum(x) or isstr(x) :
    return repr(x)
  try :
    if x.__name__ == '<lambda>' :
      return 'some anonymous lambda'
    else :
      return x.__name__
  except Exception :
    return 'some nameless object'

def curry(f, l) :
  r = lambda *r : f(l, *r)

  r.__doc__ = ('Curried application of ' +
               okname(f) + ' to ' +
  return r

The documentation is not great, but it is slightly better than nothing:

>>> help(curry(reduce, operator.add))
Help on function <lambda> in module __main__:

<lambda> lambda *r
    Curried application of reduce to add

>>> help(curry(operator.add, 1))
Help on function <lambda> in module __main__:

<lambda> lambda *r
    Curried application of add to 1

>>> map=curry(reduce, lambda x,y : [x,y][x<y])
>>> help(map)
Help on function <lambda> in module __main__:

<lambda> lambda *r
    Curried application of reduce to some anonymous lambda

Along with the compiler module, the possibilities are endless.

5 Responses to “Python: Documenting Higher-Order Artefacts”

  1. njharman Says:

    Just add optional parameter to curry for the docstring:

    def curry(f,l,doc=None):

      if doc:
        r.__doc__ = doc

    Less complex and more flexible (unless I’m missing something)

    It’s bad to think of __doc__ and other __foo__ methods as secret or tricky. They are important parts of the language that any mid-level python programmer should know and use. This use of docstrings is perfect example.

    Also, I’m fairly certain I read about way (or maybe a feature in 3000?) to change the argument list. But, I’ve never used it.

  2. drj11 Says:

    Adding the documentation as an optional paramter to curry is of course way simpler and equally flexible. Equally flexible since I can always overwrite my automatically generated documentation if I want to.

    I was attracted by the prospect that at least some documentation could be produced automatically.

    Your “if” is unnecessary of course.

  3. drj11 Says:

    Hello people from reddit. Hope you enjoy your stay; feel free to scribble anywhere you like.

    Sorry about the archaic spelling. I guess it makes me feel British.

  4. Paul Says:

    You can write ‘[x, y][x < y]’ as ‘y if x < y else x’

  5. drj11 Says:

    Thanks Paul. I had vaguely remembered a new “if expression” thing, but I couldn’t find it when I looked for it. It’s new in 2.5 (which is fine, it just means I won’t be using it in earnest for a while).

    I like the fact that “if” is an expression, but the syntax somehow rather unpleasantly reminds me of perl.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: