Live Code Documentation Maker Experiment Alpha

DETAILS

• This is an initial exploration of creating source code documentation with metadata that identifies what lines they apply to. The goal is to make sure the source code you post is the same source code you ran when verifying it works. No more botched examples because of copy/paste goofs.

EXAMPLE

This is a live example that uses the source code of the pyton script that generated these snippets as it's own target. This is the full file. Snippet examples and more notes are further below.

1  #!/usr/bin/env python3
2  
3  import glob
4  import json 
5  import os
6  
7  class SnippetDocumentor():
8  
9      def __init__(self):
10          self.source = []
11          self.docs = []
12          self.content = []
13  
14      def assemble_content(self, output_path):
15          for doc in self.docs:
16              parts = doc.split('---')
17              meta = json.loads(parts[1])
18              self.content.append(parts[0])
19              self.content.append('
')
20              for line_group in meta['lines']:
21                  for line_index in range(line_group[0] - 1, line_group[1]):
22                      self.content.append(
23                          f"{line_index + 1}  {self.source[line_index]}"
24                      )
25              self.content.append('
')
26  
27          with open(output_path, 'w') as _out:
28              _out.write("\n".join(self.content))
29  
30      def load_source(self, source_path):
31          with open(source_path) as _source:
32              self.source = _source.read().split("\n")
33  
34      def load_docs(self, docs_dir):
35          local_file_list = [
36              file for file in glob.glob(f"{docs_dir}/*")
37              if os.path.isfile(file)
38          ]
39          local_file_list.sort()
40          for local_file in local_file_list:
41              with open(local_file) as _in:
42                  self.docs.append(_in.read())
43  
44  if __name__ == "__main__":
45      working_dir = os.path.dirname(os.path.realpath(__file__))
46  
47      sd = SnippetDocumentor()
48      sd.load_source(__file__)
49      sd.load_docs(f"{working_dir}/docs")
50      sd.assemble_content(f"{working_dir}/DOCSCONTENT.html")
51  

The docs files pull out lines from the original source for direct display instead of having to copy/paste them. Here's a few lines pulled from the same source file as above.

30      def load_source(self, source_path):
31          with open(source_path) as _source:
32              self.source = _source.read().split("\n")

Note that because lines are all coming from the same source it's trivial to show line numbers that match the location of the full file. I'm not sure I'm going to like that in the long term, but my early impression is good.

There's no need for the lines to be consequetive. Here's an example with a split.

9      def __init__(self):
10          self.source = []
11          self.docs = []
12          self.content = []
13  
46  
47      sd = SnippetDocumentor()
48      sd.load_source(__file__)
49      sd.load_docs(f"{working_dir}/docs")
50      sd.assemble_content(f"{working_dir}/DOCSCONTENT.html")

I don't like the way lines 13 and 46 collide. Since this is a prototype, I'm not worrying about it. Going through iterations, that's something I'd address so it had an `...` in the middle or something.

DOCS FILE EXAMPLE

This is an example of the docs file format I'm using. It's plain HTML with a `---` at the end followed by a JSON string that identifies what lines to show.

<p>
  Note that because lines are all coming from the same source it's trivial to
  show line numbers that match the location of the full file. I'm not sure I'm
  going to like that in the long term, but my early impression is good.
</p>
<p>
  There's no need for the lines to be consecutive. Here's an example with a
  split.
</p>

--- { "lines": [[9,13], [46,50]] }

It's an array of arrays. The top level represents a code block. The arrays inside that correspond to sections of code (of which there can be one or more). And the numbers inside those identify the lines.

Having gone through the process it feels like there's potential here but not with this specific approach. Mainly because it's annoying to have to change numbers whenever the source code changes. That's not not surprising, but I wanted to see what it was like.

It's possible that tooling could help with the line shifts, but it feels like that would always be fragile. Another approach that's come to mind is pattern matching. For example, looking for function definitions and using those as the search keys on which place the content from the docs. There are tricks there and things are rarely perfect, but based on this experiment the matching approach has some real potential.