rubylit/README.html

1 <html><body> 2 3 <h1 id="label-RubyLit+-+This+README+is+a+program-21">RubyLit - This README is a program!<a href="#label-RubyLit+-+This+README+is+a+program-21">¶</a> <a href="#top">↑</a></h1> 4 5 The output of this README.md file is a program that turns documents into: 6 <ul><li> 7 <a href=“html/README.rb.html”>README.rb</a> - a Ruby program 8 </li><li> 9 <a href=“html/README.html.html”>README.html</a> - a (hideously) formatted HTML document 10 </li></ul> 11 12 It's a <a href=“<a href="https://en.wikipedia.org/wiki/Literate_programming">literate”>en.wikipedia.org/wiki/Literate_programming“>literate</a> program</a> (wikipedia.org) and the concept comes from Donald Knuth. 13 14 In order to get the process rolling, there's a non-literate <code>stage1.rb</code> script that does the initial “tangling”. You'll see what that means in a moment. 15 16 To make this literate programming stuff work, all source is indented. That not only makes it easy to read in source form, but by chosing that method, I've also made this document a valid Markdown file, so the README will be properly formatted when you view it on the Web as HTML output via tools such as <a href=“<a href="http://ratfactor.com/repos/reporat/">RepoRat</a">ratfactor.com/repos/reporat/”>RepoRat</a</a>>. 17 18 Here's how it starts. The program takes the root filename as a command line argument: 19 20 <pre class="ruby">fname = ARGV[0] 21 </pre> 22 23 I intially wanted to use the file extension “.lit”, but the real magic happened when I realized I could use “.md” and have the README itself be the program: 24 25 <pre class="ruby">fname_input = "#{fname}.md" 26 </pre> 27 28 Next, I “tangle” and “weave” the input document: 29 30 <pre><<Tangle>> 31 <<Weave>></pre> 32 33 “Tangle” creates the program and “Weave” creates the documentation. Those little <code><<bracket things>></code> are literate programming macros that include source code from other sections of the document (identified by Markdown subheadings). 34 35 <h2 id="label-Tangle">Tangle<a href="#label-Tangle">¶</a> <a href="#top">↑</a></h2> 36 37 What's neat about literate programming is that the source can be presented in any order, so you can explain it however you like. The tangling process puts it back into order so it can actually run. 38 39 The full literate programming concept as imagined by Knuth and implemented in his initial 'WEB' system not only allows you to include bits of code, but even lets you define parametric macros, so the literate document is actually a meta-language on top of the underlying programming language! 40 41 I've just implemented a crude “include” macro for this demonstration, but that alone gives me a ton of flexibility! 42 43 Here's how I've made that work. I've got a <code>segments</code> hash that will store all of the lines of a “segment” in the literate program. I have the program begin in a segment identified with the <code>:start</code> symbol: 44 45 <pre class="ruby">myseg = :start 46 segments = {} 47 segments[myseg] = [] 48 </pre> 49 50 Then I loop through all of the lines in the source document and handle just two special cases. (All other lines are treated as the “document” part of the literate program and are completely ignored here!) 51 52 <pre>File.open(fname_input).each do |line| 53 <<Handle segments>> 54 <<Handle code lines>> 55 end</pre> 56 57 And after gathering the lines into segments, I recursively follow the includes to write out the final program to a file: 58 59 <pre><<Write the program>></pre> 60 61 <h2 id="label-Handle+segments">Handle segments<a href="#label-Handle+segments">¶</a> <a href="#top">↑</a></h2> 62 63 When I see a “## ” at the beginning of the line, I know it's a Markdown level 2 heading, which I'm using to indicate new code segments. They don't have to include code and even if they do, they don't have to be used. 64 65 Here you can see that I'm setting the current segment to the name of the heading and initializing a new array to store the code lines: 66 67 <pre class="ruby"> if(line.start_with?('## ')) 68 myseg = line[3..].chomp 69 segments[myseg] = [] 70 end 71 </pre> 72 73 <h2 id="label-Handle+code+lines">Handle code lines<a href="#label-Handle+code+lines">¶</a> <a href="#top">↑</a></h2> 74 75 When I see a space at the beginning of the line, I know it's indented source code. This is extremely strict and not flexible and is just one example of the non-industrial nature of this demonstration program. :-) 76 77 <pre class="ruby"> if(line.start_with?(' ')) 78 segments[myseg].push(line) 79 end 80 </pre> 81 82 <h2 id="label-Write+the+program">Write the program<a href="#label-Write+the+program">¶</a> <a href="#top">↑</a></h2> 83 84 Here's the fun part! I've got this recursive method called <code>put_lines</code> that takes an open destination file, the hash of named code segments (each segment being an array of lines), and a target segment to print. 85 86 I'm looking at each line to see if it's a <code><<macro thingy>></code> (include request). If it is, I recurse into the requested segment. Otherwise, I just output the current line to the file: 87 88 <pre class="ruby">def put_lines(file, segments, sname) 89 segments[sname].each do |line| 90 91 if(m = /^\s*<<([^>]+)>>\s*$/.match(line)) 92 93 put_lines(file, segments, m[1]) 94 95 else 96 97 file.puts line 98 99 end 100 101 end 102 end 103 </pre> 104 105 To start the above recursive process, I open the “.rb” output file and request the <code>:start</code> segment: 106 107 <pre class="ruby">File.open("#{fname}.rb", 'w') do |out| 108 put_lines(out, segments, :start) 109 end 110 </pre> 111 112 That's it! 113 114 <h2 id="label-Weave">Weave<a href="#label-Weave">¶</a> <a href="#top">↑</a></h2> 115 116 The “weave” part of the application is the documentation creation portion. 117 118 Since my scheme for this literate program is to encode it as pure Markdown, I could just rely on an external tool to create the HTML (in fact, that's probably how you're reading this README right now). 119 120 But since Ruby comes with a Markdown parser as part of it's Standard Library, I figured I might as well include it. The parser and generator are part of the RDoc (Ruby documentation) module: 121 122 <pre class="ruby">require 'rdoc' 123 </pre> 124 125 The markdown source is the literate program document (yeah, we read it in the Tangle process and we'll read it again for Weave): 126 127 <pre class="ruby">data = File.read(fname_input) 128 </pre> 129 130 Then some boilerplate. RDoc is like a mini-Pandoc in that it can take input and produce output in a bunch of different formats, and we pay for that flexibility with some complexity: 131 132 <pre class="ruby">formatter = RDoc::Markup::ToHtml.new(RDoc::Options.new, nil) 133 html = RDoc::Markdown.parse(data).accept(formatter) 134 </pre> 135 136 And then I just write that out to a “.html” file, bookended by start and end document tags: 137 138 <pre class="ruby">File.open("#{fname}.html", 'w') do |out| 139 out.puts("<html><body>") 140 out.print(html) 141 out.puts("</body></html>") 142 end 143 </pre> 144 145 And that's it! 146 147 <h2 id="label-Running+it-21">Running it!<a href="#label-Running+it-21">¶</a> <a href="#top">↑</a></h2> 148 149 To turn this README into a program starts with “stage1”, which only includes the “tangle” part of the process (no documentation output): 150 151 <pre>$ ruby stage1.rb README 152 Found segment 'Tangle' 153 Found segment 'Handle segments' 154 Found segment 'Handle code lines' 155 Found segment 'Write the program' 156 Found segment 'Weave' 157 Found segment 'Comparing this document with its output' 158 Fetching segment 'start' 159 Fetching segment 'Tangle' 160 Fetching segment 'Handle segments' 161 Fetching segment 'Handle code lines' 162 Fetching segment 'Write the program' 163 Fetching segment 'Weave'</pre> 164 165 (As you can see, I also gave it some output to help me debug segment names.) 166 167 That produces <code>README.rb</code>, which is now the Ruby program we've described above, which can be used to process itself again: 168 169 <pre class="ruby">ruby README.rb README 170 </pre> 171 172 And running that again proves that we're fully bootstrapped. We're running the output of the README against the README: 173 174 <pre class="ruby">ruby README.rb README 175 </pre> 176 177 (Note that this part of the document you're reading right now has indented “code” blocks to show the command line and output. Those are not valid Ruby, so why is that okay? That's okay because they're never explicitly included in the program! The Ruby interpreter never sees them.) 178 179 <h2 id="label-Bootstrapping">Bootstrapping<a href="#label-Bootstrapping">¶</a> <a href="#top">↑</a></h2> 180 181 This repo includes two simple literate test programs (Markdown files) I used to get the intial “stage1” program working: 182 <ul><li> 183 <code>hello.md</code> 184 </li><li> 185 <code>hello-segments.md</code> 186 </li></ul> 187 188 When stage1 was done, I copied it to use as the basis for the final document/program you're reading now. Then I followed the “Running it!” process exactly as shown above and it worked! :-) 189 </body></html>