| Home | Calls | TPS | Org-Mode

Debugging radtools (in medias res)

Debugging radtools (in medias res)

Like most development projects, you reach a point where things are messy, and you just need to get something out. Cue the cobbling on radtools, recently.

For background, the idea is to build documentation that can adapt to the reader's choices of things like product version, preferred interface (e.g., UI vs. CLI), and packaging method (snap vs. Debian package). It's a little bit of a misnomer to call it a shellscript, since a lot of it's done in Python, but it's driven largely by shellscripts.

Goals

Simply put, we want to be able to publish different versions of the documentation, based on embedded keywords (RAD specifiers) in the documentation. Here's an example of how it currently looks:

The ultimate purpose of MAAS is to deploy and manage machines.  As explained in
[About the machine life-cycle](/t/about-machines/nnnn#heading--about-the-machine-life-cycle),
machines must first be enlisted or commissioned, then acquired, then deployed.
This article will tell you:

 - [How to commission a machine](#heading--how-to-commission-a-machine)
 - [How to test machines](#heading--how-to-test-machines)
 - [How to acquire machines](#heading--acquire-machines)
 - [How to deploy machines](#heading--deploy)

<a href="#heading--how-to-commission-a-machine"><h2 id="heading--how-to-commission-a-machine">How to commission a machine</h2></a>

rad-begin /snap/2.9/ui /deb/2.9/ui /snap/3.0/ui /deb/3.0/ui /snap/3.1/ui /deb/3.1/ui 
To commission a machine:

1. Go to the "Machines" page.

2. Select the machine(s) you want to commission.

3. Choose "Commission" under the "Take action" drop-down menu:

<a href="https://discourse.maas.io/uploads/default/original/1X/5f196ca5e175e3f37d7cffbb2341fb0ee9cee16a.png" target = "_blank"><img src="https://discourse.maas.io/uploads/default/original/1X/5f196ca5e175e3f37d7cffbb2341fb0ee9cee16a.png"></a>

You have the option of selecting some extra parameters (checkboxes) and performing hardware tests.

These options include:

-   **Allow SSH access and prevent machine powering off**: Machines are normally powered off after commissioning. This option keeps the machine on and enables SSH so you can access the machine.

-   **Retain network configuration**: When enabled, preserves any custom network settings previously configured for the machine. See [Networking](/t/about-networking/nnnn) for more information.

-   **Retain storage configuration**: When enabled, preserves any storage settings previously configured for the machine. See [Storage](/t/about-machines/nnnn#heading--about-storage) for more details.

-   **Update firmware**: Runs scripts tagged with `update_firmware`.

-   **Configure HBA**: Runs scripts tagged with `configure_hba`.

<a href="https://discourse.maas.io/uploads/default/original/1X/5f196ca5e175e3f37d7cffbb2341fb0ee9cee16a.png" target = "_blank"><img src="https://discourse.maas.io/uploads/default/original/1X/5f196ca5e175e3f37d7cffbb2341fb0ee9cee16a.png"></a>

4. Click the Hardware tests field to reveal a drop-down list of tests to add and run during commissioning. See [Hardware testing](/t/about-machines/nnnn#heading--about-testing-hardware) for more information on hardware testing scripts.

5. Finalise the directive by hitting "Commission machine".

While commissioning, the machine status will change to reflect this state (Commissioning).  MAAS discovers the machine's network topology.  MAAS then prompts a machine network interface to connect to the fabric, VLAN, and subnet combination for configuration. Usually, MAAS assigns a static IP address out of the reserved IP range for the subnet ("Auto assign" mode). The next section details several assignment modes.

Once commissioned, a machine's status will change to Ready, and an extra tab for the machine called "Commissioning" will become available. This tab contains the results of the scripts executed during the commissioning process.
rad-end

rad-begin   /snap/2.9/cli /deb/2.9/cli /snap/3.0/cli /deb/3.0/cli /snap/3.1/cli /deb/3.1/cli
To commission a machine that's in the "Ready" state, via the CLI, use the following command:

```nohighlight
maas $PROFILE machine commission $SYSTEM_ID
```

RAD specifiers look like this:

rad-begin /snap/2.9/ui /deb/2.9/ui /snap/3.0/ui /deb/3.0/ui /snap/3.1/ui /deb/3.1/ui
...
... some specific text ...
...
rad-end

varying links

And you'll also notice it has links like this:

/t/about-networking/nnnn

These links refer to discourse topics (we use the Discourse open-source forum tool to manage our user forums and generate our documentation). This particular link is open-ended (the "nnnn") because RAD has multiple complete sets of documentation, all hosted in the same set of pages. If a user is looking at the "snap 2.9 CLI" version of a page, and wants to link to the "About networking" page (as in this case), they need to be taken to the correct version (that is, the "snap 2.9 CLI" version of "About networking").

These open-ended links can be processed by one of the radtools to figure out which particular discourse topic represents the "snap 2.9 CLI" version of "About networking", and that topic number can be substituted for the "nnnn" when building the "snap 2.9 CLI" version of the document. If it seems a little complicated, well, yeah, it kinda is that way. But within the constraints of the problem that we had, it's still the simplest way to solve it.

page-top menus

Another requirement is a way for users to get back and forth between different versions of a specific document. Because of constraints on our left-hand navigation, it isn't convenient to switch between versions when you make a selection. Instead, each page in the left-hand navigation links to a preferred, default version of the doc (usually the UI interface for current snap version).

To accommodate users, we've added menus at the top that transition between active versions of the docs, arranged in a tabular format. You can see one of these at this page, and get a feel for how RAD works, as well. Note that, as you switch between UI and CLI, for example, the instructions change from UI screenshots to console outputs. As you change versions, different features may come and go, based on what's available in a specific version. And as you change from Deb to Snap, pathnames and some key operating instructions change.

This kind of "contextual switching" is what RAD is all about.

In Medias Res

Rather than give a huge history here -- especially since I'm trying to get something specific done -- I'll just start where I am and kind of "explain in all directions" as needed. Currently, you'll see that my git repo for radtools is pretty messy:

/home/stormrider/Dropbox/src/git/radtools:
total used in directory 240 available 821.4 GiB
drwxr-xr-x  6 stormrider stormrider  4096 Nov 16 18:14 .
drwxr-xr-x 10 stormrider stormrider  4096 Oct 27 16:25 ..
drwxr-xr-x  2 stormrider stormrider  4096 Oct 12 17:47 beta-1
drwxr-xr-x  2 stormrider stormrider  4096 Oct 12 17:46 beta-2
-rwxr--r--  1 stormrider stormrider  1717 Nov 16 18:14 discdel
-rwxr--r--  1 stormrider stormrider  2017 Oct  8 16:18 discget
-rw-r--r--  1 stormrider stormrider 24478 Sep 11 10:53 disclib.py
-rwxr--r--  1 stormrider stormrider  1951 Oct  8 16:22 discnew
-rwxr--r--  1 stormrider stormrider  2144 Oct  8 16:55 discput
drwxr-xr-x  8 stormrider stormrider  4096 Nov 16 17:47 .git
-rw-rw-r--  1 stormrider stormrider 24478 Nov 16 14:06 maas_discourse.py
-rwxrwxr-x  1 stormrider stormrider  8217 Nov 16 17:47 makehtml
-rw-rw-r--  1 stormrider stormrider  9546 Nov 16 14:06 man.7
drwxr-xr-x  2 stormrider stormrider  4096 Oct 12 17:46 __pycache__
-rwxrwxrwx  1 stormrider stormrider  6042 Nov 16 11:56 rad2htm
-rwxrwxr-x  1 stormrider stormrider  6040 Nov 16 16:07 rad2html
-rwxrwxr-x  1 stormrider stormrider  1563 Nov 16 14:06 raddel
-rwxr--r--  1 stormrider stormrider   273 Oct  8 18:10 raddiff
-rwxrwxr-x  1 stormrider stormrider  1765 Nov 16 14:06 radfilter
-rwxr--r--  1 stormrider stormrider  1770 Nov 16 13:23 radfltr
-rwxrwxr-x  1 stormrider stormrider  2017 Nov 16 14:06 radget
-rwxrwxr-x  1 stormrider stormrider  4273 Nov 16 14:06 radlastmod
-rwxr--r--  1 stormrider stormrider  3732 Oct  8 16:02 radlink
-rwxr--r--  1 stormrider stormrider  4273 Oct  8 17:58 radlmod
-rwxrwxr-x  1 stormrider stormrider   416 Nov 16 14:06 radmake
-rwxrwxr-x  1 stormrider stormrider  1184 Nov 16 16:53 radmakehtml
-rwxr--r--  1 stormrider stormrider  5247 Oct  8 16:06 radmenu
-rwxr--r--  1 stormrider stormrider  2541 Nov 16 14:13 radmock
-rwxrwxr-x  1 stormrider stormrider  1951 Nov 16 14:06 radnew
-rwxr--r--  1 stormrider stormrider   433 Nov 16 13:48 radpubd
-rwxrwxrwx  1 stormrider stormrider  1222 Nov 16 11:59 radpubh
-rwxrwxr-x  1 stormrider stormrider  2144 Nov 16 14:06 radput
-rwxr--r--  1 stormrider stormrider   949 Aug 16 17:50 radsmod
-rwxrwxr-x  1 stormrider stormrider   949 Nov 16 14:06 radsortmods
-rwxrwxr-x  1 stormrider stormrider  2396 Nov 16 14:06 radstage
-rwxr--r--  1 stormrider stormrider  4605 Oct 13 14:55 radupdb
-rw-r--r--  1 stormrider stormrider 13762 Oct  8 16:57 README.md

It's a little munged because I've got three different generations of code and ideas floating around in the same directory, without really good separation. For example, you'll note that there are beta-1 and a beta2 subdirectories. You'll also notice that the main set of utilities seems to have some duplicated names, following two different naming schemes.

That's because I started to do a big revamp in the final push to get the 3.1 version of MAAS HTML doc together, but then changed my mind. I got excited or frustrated or both, and decided to just start from the top and smooth all the wrinkles out. Bad idea. It also violates at least two rules in my credo: "keep it simple" and "do one thing well (at a time)".

So I recanted, but not before pushing duplicate versions of my tools to the github repo. So it's a big mess at the moment. The first order of business is to simplify what I'm looking at by separating the three generations of code more cleanly in this repo.

preserving the past

I started out with some different ideas, as you can see from the beta-* subdirectories. I need to bundle those together, so the first thing I'll do is create a directory called "Beta" and push them off into it. Notice that I'm not trying to go back and create a Beta branch, because that doesn't seem reasonable at this late stage. Then I'll commit those changes to the existing repo, leaving me with a slightly cleaner kit:

stormrider@cloudburst:~/Dropbox/src/git/radtools$ git add .
stormrider@cloudburst:~/Dropbox/src/git/radtools$ git commit -m 'archive RAD beta versions to single subdirectory'
[main 7637ff3] archive RAD beta versions to single subdirectory
 29 files changed, 168 insertions(+), 138 deletions(-)
 delete mode 100755 #discdel#
 delete mode 120000 .#discdel
 rename {beta-1 => Beta/beta-1}/dpull (100%)
 rename {beta-1 => Beta/beta-1}/dpull.7 (100%)
 rename {beta-1 => Beta/beta-1}/dpush (100%)
 rename {beta-1 => Beta/beta-1}/dpush.7 (100%)
 rename {beta-1 => Beta/beta-1}/maas-documentation-25.md (100%)
 rename {beta-1 => Beta/beta-1}/rad (100%)
 rename {beta-1 => Beta/beta-1}/rad.7 (100%)
 rename {beta-1 => Beta/beta-1}/rdbload (100%)
 rename {beta-1 => Beta/beta-1}/rdbload.7 (100%)
 rename {beta-1 => Beta/beta-1}/rlet.7 (100%)
 rename {beta-1 => Beta/beta-1}/rletu (100%)
 rename {beta-1 => Beta/beta-1}/rlink (100%)
 rename {beta-1 => Beta/beta-1}/rlink.7 (100%)
 rename {beta-1 => Beta/beta-1}/rmake (100%)
 rename {beta-1 => Beta/beta-1}/rmenu (100%)
 rename {beta-1 => Beta/beta-1}/rmenu.7 (100%)
 rename {beta-2 => Beta/beta-2}/doscript.sh (100%)
 rename {beta-2 => Beta/beta-2}/getdoc (100%)
 rename {beta-2 => Beta/beta-2}/maas_rad.py (100%)
 rename {beta-2 => Beta/beta-2}/make_rad (100%)
 rename {beta-2 => Beta/beta-2}/putdoc (100%)
 rename {beta-2 => Beta/beta-2}/retitle (100%)
 create mode 100755 radmock
stormrider@cloudburst:~/Dropbox/src/git/radtools$ git push
Warning: Permanently added the RSA host key for IP address '140.82.113.4' to the list of known hosts.
Enumerating objects: 11, done.
Counting objects: 100% (11/11), done.
Delta compression using up to 16 threads
Compressing objects: 100% (7/7), done.
Writing objects: 100% (7/7), 2.06 KiB | 2.06 MiB/s, done.
Total 7 (delta 4), reused 0 (delta 0), pack-reused 0
remote: Resolving deltas: 100% (4/4), completed with 4 local objects.        
To github.com:billwear/radtools.git
   0870f41..7637ff3  main -> main

making room for the future

Currently, I have some ideas about how I want the next version to look. I kinda started doing that by copying and renaming some of the tools. I'm really shooting for a slightly different architecture, but it seemed good enough at the time to start with renaming. This is confusing, but there's nothing wrong with preserving this line of thinking, for now.

Again, I don't want to branch yet, because I don't actually have anything final and stable, either for the current or the next generation. So the best approach is to give myself another subdirectory -- call it "Next" -- and sock away these new ideas until I can get the current one up and running reasonably well.

This idea also has merit because it's unclear which direction we'll take at Canonical for this doc. RAD is accepted, and it seems like it will continue to be used, but it puts a lot of strain on the current system, both in terms of performance and user experience. We're discussing a number of different changes, but haven't decided yet exactly how we want to move forward. So it doesn't make sense to optimise this generation of the code. At least, not as part of my paid open-source gig.

Effecting this change, I'm left with a listing that looks like this:

/home/stormrider/Dropbox/src/git/radtools:
total used in directory 164 available 821.4 GiB
drwxr-xr-x  6 stormrider stormrider  4096 Nov 17 14:23 .
drwxr-xr-x 10 stormrider stormrider  4096 Oct 27 16:25 ..
drwxrwxr-x  4 stormrider stormrider  4096 Nov 17 14:13 Beta
drwxr-xr-x  8 stormrider stormrider  4096 Nov 17 14:15 .git
-rw-rw-r--  1 stormrider stormrider 24478 Nov 16 14:06 maas_discourse.py
-rwxrwxr-x  1 stormrider stormrider  8217 Nov 16 17:47 makehtml
-rw-rw-r--  1 stormrider stormrider  9546 Nov 16 14:06 man.7
drwxrwxr-x  2 stormrider stormrider  4096 Nov 17 14:23 Next
drwxr-xr-x  2 stormrider stormrider  4096 Oct 12 17:46 __pycache__
-rwxrwxr-x  1 stormrider stormrider  6040 Nov 16 16:07 rad2html
-rwxrwxr-x  1 stormrider stormrider  1563 Nov 16 14:06 raddel
-rwxr--r--  1 stormrider stormrider   273 Oct  8 18:10 raddiff
-rwxrwxr-x  1 stormrider stormrider  1765 Nov 16 14:06 radfilter
-rwxrwxr-x  1 stormrider stormrider  2017 Nov 16 14:06 radget
-rwxrwxr-x  1 stormrider stormrider  4273 Nov 16 14:06 radlastmod
-rwxr--r--  1 stormrider stormrider  3732 Oct  8 16:02 radlink
-rwxrwxr-x  1 stormrider stormrider   416 Nov 16 14:06 radmake
-rwxrwxr-x  1 stormrider stormrider  1184 Nov 16 16:53 radmakehtml
-rwxr--r--  1 stormrider stormrider  5247 Oct  8 16:06 radmenu
-rwxrwxr-x  1 stormrider stormrider  1951 Nov 16 14:06 radnew
-rwxrwxr-x  1 stormrider stormrider  2144 Nov 16 14:06 radput
-rwxrwxr-x  1 stormrider stormrider   949 Nov 16 14:06 radsortmods
-rwxrwxr-x  1 stormrider stormrider  2396 Nov 16 14:06 radstage
-rwxr--r--  1 stormrider stormrider  4605 Oct 13 14:55 radupdb
-rw-r--r--  1 stormrider stormrider 13762 Oct  8 16:57 README.md

What was the Point, Again?

Now, having done that, I need to sort of figure out where I'm at, what I'm actually doing, and why I built these tools. Maybe a little block diagram would help?

+------------+    +-----------+    +---------+    +---------+    +--------+
|            |    |           |    |         |    |         |    |        |
| RAD MASTER +--->+ radfilter +--->+ radlink +--->+ radmenu +--->+ radput |
|            |    |           |    |         |    |         |    |        |
+------------+    +-----------+    +---------+    +---------+    +--------+
       |                |               |              |              |
all RAD vsns in    one specific    adjust links    add menus      push to
the same doc       RAD version     to point to     to allow       specific
				   same RAD vsn    selecting      discourse
				   of linked       diff vsns      topic
				   pages           of the page

The remaining tools are used like this:

  • raddel - a utility to delete a discourse topic (not RAD specific)
  • radnew - a utility to create a new discourse topic (not RAD specific)
  • rad2html - a utility to convert a markdown RAD version (output of radmenu, above) to an offline HTML document, including capturing and processing all the images.
  • raddiff - some utility I created to diff RAD versions; don't remember how it works, or if it's even useful at this point.
  • radlastdmod - a utility to figure out who last modified a topic and when.
  • radsortmods - a python tool to sort out discourse topic last mod times; dont' ask me why it's in a separate tool, I don't remember at this point. It turns out to be easier to just check manually, before you edit a file (or push an edit), because discourse has some great tools built in. I can probably access those tools with the API, but I didn't get that far yet....
  • radstage - a workhorse shellscript that's supposed to build discourse and HTML versions of RAD (by running a single RAD master through the flowchart above). The only difference is that it puts the results in a temporary local directory, so they can be checked, rather than publishing them.
  • radmake - same as radstage, except that it loads the resulting RAD version markdown file into the correct topic on discourse, effectively publishing it.
  • radmakehtml - a version of radmake for the HTML, originally intended to put the HTML files in the publication directories in the "MAAS offline docs" github. It's since been butchered so that it can put them wherever you specify, since it's the same operation either way.
  • radupdb - a tool that updates the RAD database that maps URLs to topic numbers. This is an artifact of discourse, one that relieves me of the need to constantly verify which topic number goes with which RAD version of what page. It updates from the MAAS doc index file, which contains URL mapping and redirect tables (and is thus the source of truth, here).
  • makehtml - a psuedo-makfile-shellscript that runs all the commands necessary to get an HTML set built and in-place. It currently takes a bunch of parameters, and it can be confusing to remember how it links to the other tools to do its job. Something I seriously want to fix, on the Next iteration -- there's no time or need to risk breakage at this point.

So there's my messy sandbox as it stands at the moment. Currently, I just need to keep it pinned together so I can get a version out, and then I can go back and straighten things up.

What's Working (and What's Not) at the Moment

So the next step is to walk through what I've generated so far and see what remains to be done. It won't be huge, but it could be a little bit of work. To simplify this work, though, a few ground rules:

  1. Don't re-run files that are okay.
  2. If no images are missing, turn off image processing, so that things will run faster. This will mean adding an option to rad2html to skip over the image processing steps.
  3. Also, if no images are missing, stop copying images at that point in the makehtml script.
  4. Finally, comment out anything in makehtml that doesn't need to run on iterations, so it runs faster.

Given that list, the first thing to do would be to check the current set to see if all the images are there and in the correct form.

the image check pass: what I found

There were two images that failed to get picked up by the tool: one because the image line was broken up into multiple lines, and the other because of an extra space. These are both things I could fix in future versions, if that's necessary, but I got them fixed and re-ran to pages. It now appears that all images are correctly downloaded and replaced. If there's a stray broken image out there, I can find it later.

turning off image manipulation and unnecessary iteration

First, in makehtml, I turned off everything except the lines that remove topic numbers from links. I'll turn on individual lines for specific files if I find issues.

Second, in rad2html, I added a command line option "-n" for "no images", and routed around the image processing routines only if that optional flag is present. That way I don't mess up the routine for normal operation. I also had to alter my radmakehtml to add that option, temporarily, to the routine.

checking broken links

I can figure out what broken links would look like and grep for them, as opposed to a painstaking line-by-line search. First, though, I'll need to find one so I can figure out what they look like.

Using linkchecker locally worked well, including taking the system offline to see if any external links were being checked inadvertently. They shouldn't have been, according to the instructions, but I made sure. About two hours of fixing various gaffs, some of which had to be fixed by sed in the makehtml shellscript. But I think I got a pretty clean set of HTML docs for this pass. I'm sure I missed something (or something broke), but I can wait for another day to find that out.


Updated 2021-11-18 Thu 09:08 by Bill Wear (stormrider)


Copyright (C) 2020 by Bill Wear. All rights reserved, but asking to use is permitted and welcome.