FSL Interceptor Creation

The Fsl package created by our very own Marc Simpson puts a really cool extension wrapper around the core of the great Fossil SCM. Adding new commands and such is easy, but requires both familiarity with a bit of a fringe language (Tcl) and requires knowledge of how the pieces of the Fsl wrapper interact. This tutorial goes through the creation of such an "interceptor" command as an illustration of how to get to there from here.

Fossil is a powerhouse of a distributed SCM system.  It's fast, small, powerful and capable.  It blends a distributed SCM, a distributed Wiki, a distributed versioned document server and a distributed ticket tracking system all into one sweet package.  Like all software intended for a broader audience, however, it doesn't always do things exactly the way that a particular user would want it to match their tastes or workflow.

 

Enter Fsl, Marc Simpson's Tcl/Expect wrapper around FossilTcl is a natural fit for Fossil given that the latter is scripted (should you so desire) by TH1—a minimalist dialect of Tcl—and that the full-blown Tcl environment can be added at build time if so desired.  This means that learning to write interceptors (or filters or aliases) translates directly into making Fossil's use easier as well.

 

This tutorial aims to show both how easy it is to write an interceptor for Fsl and to show some of the pitfalls involved along the way.

Use case

Because I am a dullard, I frequently find myself in the working directory of a repository doing things like rm <list of files> without typing the equivalent in the SCM I happen to be using at the moment.  Even worse I may use mv, rm, etc. a dozen or more times without thinking.  In the case of Fossil, I really should type fossil rm or fossil mv instead, but I don't.  The result is that when it comes time to commit my changes, I have a problem: dozens (or hundreds!) of files may be flagged as MISSING when I type fossil changes.  Trying to commit now will make Fossil very unhappy and it will give up on me in disgust.  So now comes the boring process of removing them from source control:

 

for a in $(fossil changes | awk '/^MISSING/{print $2}') ; do fossil rm $a ; done

 

That doesn't exactly roll off the tongue easily, does it?  Putting stuff like this into shell scripts is an option, true, but an option that starts to leave fragments of rarely-used code lying all over the place; code that doesn't necessarily get updated to reflect reality as things change.  Now of course it's possible to use fossil addremove instead, but ... I only want to remove missing files, not add ones that have as yet to be put under revision control.  (Not everything I have in my working directory is intended to go into revision control!  Sometimes I do this kind of work while I still have compilation artifacts live, after all.)

 

Wouldn't it be nice if we could easily add commands to Fossil instead?

The solution

Fsl is a transparent wrapper around Fossil.  How transparent?  You could just put it in your PATH somewhere and start using it by just replacing typing fossil with fsl.  Everything would look and work just like before (well, except for the output of some commands perhaps being just a little prettier).

 

As you explore using Fsl, however, you'll note that there are some changes.  There's new subcommands, for starters: ., d, ,, log, and heads.  If you use changes, status, timeline, add, rm, addremove, branch, or leaves you'll see the output is a bit more colourful (and hopefully a bit more informative as a result).  The big change, however, is that you can very easily add new ways to format output or aliases for existing commands or even entirely new functionality to the wrapper.  All without touching the Fossil core underneath it.

The first stab

Reading the documentation for Fsl, it seems clear that I have to add what is called an "interceptor" to my .fslrc file.  An "alias" just maps one command to another.  It doesn't change formatting or add any complex behaviour.  It's just a way of providing shortcuts to common commands, perhaps with default options.  A "filter" just filters the output of another command (or alias or interceptor) and allows you to change it on the fly.  These changes can include colourization, rearrangement, elimination, aggregation or whatever else your imagination can come up with.  They're like Unix pipes, only for Fossil commands.  This leaves us with an interceptor, where an interceptor is more like a shell script, if we're going to keep with the Unix analogies.  So let's see what happens when we add this to ~/.fslrc:

 

1
2
3
4
interceptor purge {
    set lines [fossil changes]
    puts $lines
}

 

Now I'm not yet actually trying to manipulate anything.  I'm trying to see what captured lines look like.  Before I try it, though, let me add some files to my Fossil repository, then delete them on disk: for a in 1 2 3 4 5 ; do touch $a ; done ; fsl add . ; rm 1 2 3 4 5

 

Now I'll type fsl purge and look at the output.  You might want to try this yourselves.  This is what we'll see:

 

MISSING    1

MISSING    2

MISSING    3

MISSING    4

MISSING    5

0

That … is a bit weird, that 0 at the end, but otherwise seems to do what we want.  Let's just look for the lines that have 3 intead:

 

1
2
3
4
5
6
7
8
interceptor purge {
    set lines [fossil changes]
    foreach line [split $lines "\n"] {
        if [string match *3* $line] {
            puts $line
        }
    }
}

 

Whoa, wait.  Why is everything still showing?  I mean the 0 is thankfully gone, but we're still getting every line, whether it has a "3" or not.

 

Well, here's the thing.  The fossil … suite of Tcl commands provided inside of Fsl actually launches the fossil program.  It doesn't capture the output.  It executes (and controls, this being an Expect script) the actual Fossil program.  So the set lines [fossil changes] in line 2 does precisely nothing.  The output we saw when it looked like things were working is the actual output from Fossil itself.  We need to do something else it seems.

The second stab

So what can actually capture the output of the fossil command?  Remember that thing above where I described aliases and filters?  Remember how I described filters as something that could make …changes [that] can include colourization, rearrangement, elimination, aggregation or whatever else…That is what you need to use to capture the text of the Fossil program when invoked.

 

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
filter captured_changes {changes} {
    variable captured
    lappend captured $line
    return {}
}

interceptor purge {
    variable captured {}
    fossil changes
    foreach line $captured {
        if [string match *3* $line] {
            puts $line
        }
    }
}

 

So now the purge interceptor sets up a captured variable with an empty array.  It then issues the fossil changes command (the Tcl version in the Fsl script).  This captures the output of the actual Fossil command and runs it through the captured_changes filter.  This in turn captures each line, appending it to the captured variable and then, here's the key, returns nothing, thus effectively swallowing the output.  So now when we issue the fossil purge command from the shell, we get just the MISSING    3 line exactly as we expect.  Hooray!  We're almost done!  We've got the hard part behind us!

 

Well, except for one small problem.  We haven't.  Because we're filtering the fossil changes command.  ANY INVOCATION OF IT!  If we type fsl changes at the command line now we'll always see no changes of any kind.  This is probably not what we're looking for.

Straight in the heart

Now we're at an impasse.  We need to filter the output to stop it from being printed, but when we do that, it stops it from being printed ever, decidedly not what we really want.  Sure we could mess around with setting flags to say "eat the output now" and "don't eat the output now", but that's a clumsy stopgap.  It also misses another problem: other filters may be working on fossil changes' output too.  Indeed by default there's a status filter that colourizes the output of several Fossil commands, including changes.  This, too, could screw us up with capturing and processing any output.  Time to give up.

 

Or is it?

 

Because there's another piece of the puzzle.  Aliases.  One rule of filters is that if you filter on a core command, all of its aliases are filtered as well, but if you filter on an alias, only that alias is filtered.  And with this we have the key to this impasse:

 

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
alias capture_changes changes

filter captured_changes {capture_changes} {
    variable captured
    lappend captured $line
    return {}
}

interceptor purge {
    variable captured {}
    fossil capture_changes
    foreach line $captured {
        if [string match *3* $line] {
            puts $line
        }
    }
}

 

In line 1 we've added an alias for the changes command called capture_changes.  The filter now works on capture_changes instead on line 3, meaning any other filter looking at changes won't be invoked.  The interceptor now invokes capture_changes in line 11.  Issuing fsl purge from the command line now gives us exactly what we want: MISSING    3.  From here it's simple to go on and do our final version of the code, the one that actually purges missing files:

 

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
alias capture_changes changes

filter captured_changes {capture_changes} {
    variable captured
    lappend captured $line
    return {}
}

interceptor purge {
    variable captured {}
    fossil capture_changes
    foreach line $captured {
        if [string match MISSING* $line] {
            regsub ^MISSING $line {} file
            fossil rm [string trim $file]
        }
    }
    return {}
}

 

And here we have the final version.  The only thing we've done here now is to expand the foreach block to look for lines beginning with MISSING, extract from them the file name and then issue the fossil rm command on each of them.  We've also added a pro format return of an empty string just to be absolutely certain.

 

In under 20 lines of code we've added some pretty sophisticated behaviour to Fossil.  Pretty heady stuff!

TL;DR

Despite its use of an oddball scripting language, the Fsl project lends itself well to making custom workflows using Fossil.  All that is required is an understanding of the Fsl architecture and its three major components: aliases, filters and interceptors.  Creative assembly of these three things can lead to sophisticated behaviour in a small amount of code.  (Consult the Fsl cookbook for more ideas.)  It should even be possible to do something as large and as all-encompassing as gitflow for Fossil in a simpler and easier (and better-integrated) way.  Hopefully this tutorial will lead to more ideas being implemented.