08 - More neat jq things
MAAS: Install ~ Configure ~ DHCP ~ Commission ~ Deploy ~ jq ~ SSH ~ More jq
More neat things to do with jq
I previously covered basic usage of jq with MAAS a couple of posts ago. Now I want to dig in a little bit, and see what else I can do with it. sorting natively with jq
In my previous jq post, I showed how to use Linux command line utilities to sort lines in a multi-record jq output. It's not the only way – nor maybe even the easiest way. The jq tool has a native sort capability. Let's take a look.
Suppose we have a random list of machines that I quickly created on one of my local MAAS installs – haven't even bothered to commission them, since that doesn't matter here. If I run the command:
on my very casually defined set of new machines, I'll get a listing something like this:
These aren't really sorted by anything, in particular, so let's just start at the front and work our way in, shall we? sorting by one parameter
Rather than passing our output through an awk script, we can just use the jq construct for sorting, which is:
For example, if we wanted to sort by HOSTNAME, we could try:
This doesn't give us anything like what we want, just a mysterious error:
Huh? Don't you just link pipes together, and they work?
jq is not UNIX (jqNU?)
No, that's UNIX, but that's not necessarily jq. Too see what's wrong, we have to break down the earlier jq statement a little more – you know, the one that just gives us the unsorted list:
If we read it from left to right, we have the MAAS command that generates the JSON to begin with:
We pipe that through jq, with raw output:
We give jq some series of commands, between the single quotes:
Finally, we pipe that to column to make the columns line up nicely:
So essentially, from a UNIX perspective, we have a data source and two filters:
The options we give to jq most definitely don't work like UNIX. where's the data coming from?
A great question to ask yourself when building jq blocks is, "Where's the data coming from?" Let's take a closer look at the jq options:
Ask yourself where the array data is coming from. Does anything in this line retrieve data?
Nope. Those are literals, which don't retrieve any data at all. How about the next line?
No, that line acts on data it gets, but it doesn't get data. And the next one?
Yep, that's it. The .[] command tells jq to iterate over what it gets. The rest of the line just tells jq which of the fields in the .[] iteration to actually retrieve.
So by process of elimination, there's only one place we could sort data – where it comes into the command, that is, where we currently iterate over the incoming JSON arrays, .[].
So how does "sortby()" work?
The "sortby()" subcommand operates on whatever it gets. In this case, we're sending it a JSON array of machine datasets. So there are two things wrong with our first attempt:
First, it's set up to handle a single JSON dataset, and sort it by a single parameter – sort_by(.hostname) – so there's actually nothing to sort, hence an error. Second, we trying to pipe a formatted entry from an array of JSON datasets through it, which doesn't even make any sense: you can't sort one thing.
To make the command work, we have to make sort_by itself the data source. In other words, we want jq to sort the record stream before it gets our field list, like this:
As we can see, this command works as we expect:
Reading the new command from left to right helps understand it. Basically, it reads: "Take the data for all machines from MAAS, display a heading line and a horizontal rule line; then sort the machine data by hostname, and show the following 5 parameters, in tab-separated rows, lined up in neat columns."
Checking our work
We can check our work by trying different sorts:
This command sorts by system ID:
We can also try pool name:
which also works as we'd expect:
Sorting by more than one parameter
To sort by more than one parameter, simply include a comma-separated list of parameters for the sort, in order. For example, to sort by tag first and pool next, use this incantation:
This will give you:
You can even try three parameters. Note that I set this up with the two hosts "superb-koi" and "amazed-hermit" – so we can use a command like this to see how multiple parameters work:
Note that "superb-koi" and "amazed-hermit" have changed places in the listing below:
Summary
So there's a little more jq for you, but still a lot more to learn. Might as well keep going with this, it seems to be useful to some folks – so more jq to come.