Captain's Log Supplemental

It’s been a rather slow day, but somewhat productive (only about 10 or 11 commits :@). My families made sure that I’ve had a throbbing headache most of the day… rat fuckers! But at least there is some work to show for it. tpsh now has a concept of $PATHEXT based on cmd.exe’s %PathExt% variable.

Windows is ruled by file extensions, while UNIX could care less about them; so really one of the few good things about Microsoft’s cmd.exe is you can tell it what file extensions should be “understood”, i.e. so you can type ‘notepad’ instead of ‘notepad.exe’. Since tpsh is modeled after the standard Unix sh, it’s mostly oblivious to file extensions: it cares about names. However it is virtually _impossible_ to use Windows from the CLI level without implementing something like PATHEXT or typing yourself into a nightmare (winxp/cmd.exe is actually good at making you do that, compared to a unix/sh).

For better compatibility with a Windows environment, tpsh now implements it’s (my) own concept of the feature, complete with a ‘pathext’ option (default on) to toggle the functionality. The main reason tpsh does this, is so I can type ‘gvim’ when I mean ‘C:Pathtogvim.bat’, the fact that I’m used to typing ‘vim’ is aside the point lol. (I rarely use gvim off win32 b/c of diffs between nt and unix cli)

About the only time I use file extensions is when forced (Win32), when ideal (.zip, .tar, etc), when saving text with CRLF for new line indicators (notepad friendly .txt), multimedia files (.png, .ogg, etc), or when deploying crap to a Windows machine lol. So it’s not an important thing for me; just a time saver.

Another thing I sorted out is what I call the “hash separator” for environment variables. UNIX shells by convention separate values like $PATH with a ‘:’, e.g. ‘/bin:/usr/bin’, but DOS and related bastards use ‘;’ for things like %PATH%, e.g. ‘C:Windows;C:WindowsSystem32’. Because many older operating systems use a ‘letter:path’ style for paths anyway, there is no universally portable default setting. Since virtually all operations involving a variables like PATH, CDPATH, and ENV (an extension used in tpsh, which I’ve never seen another shell use) involve a hash table, I call the mark the ‘hash-sep’ for short. One of today’s changes was exposing the hash-sep directly to the user.

The default hash-sep is ‘;’ under MSWin32, DOS, and OS/2 and ‘:’ otherwise, I’ve no clue what the hell VMS uses and don’t have access to it, so no worries yet ;-). Whenever the user changes the HASHSEP environment variable, the next time an operation that relies on it (basically hsplit(), short for hash-sep split()) the shell compiles it down to a suitable regular expression to save time on future commands.

setenv HASHSEP '/'
rehash

the rehash will cause ‘/’ to compile via qr for to speed up later splits; and due to the change of hashsep, causes the internal cache of $PATH to switch from the default ‘:’ or ‘;’, to ‘/’, which is probably not something anyone wants to do, but being able to fiddle with it can be useful for scripting reasons ^_^. Not to mention the fact that it makes concatenating things like PATH/CDPATH settings more portable when the environment requires something different.

Another bit of today’s work, was setting it up so that the history built-in now displays the correct line numbers. I want to setup builtins to save/load data from HISTFILE; so that command history is not lost between sessions – I’ve always had a bit of an itch about how most shells deal with it; wonder what tpsh might do hehe. Not sure about the $LINENO yet, I never really use it in scripting or interactive usage that much: will probably take the single unix specification into consideration on $LINENO.

If I didn’t have to so much crap for tomorrow, I could probably have half the manual done tonight and still get to bed before 0600 local. But no…. work early, work long, to be driven nuts after work, and probably end up PTFO instead of coding the night way.

my idea of a great vacation: a Ferrari, laptops, solar panels to power them, hot date, and a tropical island with a sunny beach 😉

3. a sudden, intuitive perception of or insight into the reality or essential meaning of something, usually initiated by some simple, homely, or commonplace occurrence or experience.

I actually thanked my mother for dragging my out on a shopping expedition today, because I figured out sometime a ways “down” my todo list: implementing control flow in tpsh. I guess you can take the programmer away from the code, but you can’t take the code away from programmer ^_^ ^_^ ^_^.

Earlier I recorded that the goal was for tpsh to use a queue of commands to execute rather then going by lines, the idea being:

`cmd1; cmd2; cmd3`

would parse into a list like ‘(cmd1, cmd2, cmd3’) which tpsh would then walk, eval, and execute the result in sequence (i.e. first in, first out). Then it occurred to me, if the shell will go that route for lexical reasons: why not implement the shells control flow operators as keywords that manipulate that queue directly? So that conceptually, a shell snippet like:

cmd0
if [ expr ]; then
  cmd1
  cmd2
else
  cmd3
  cmd4
fi
cmd5

would be inserted into the queue in a suitable manor and passed to the call back, so if the queue looked something like this:

( cmd0, if [ expr ], cmd1, cmd2, else, cmd3, cmd4, fi, cmd5 )

the call back would receive that relevant portion of the queue, (i.e. queue₁ through queue₇), evaluate the test and return the appropriate portion to be spliced into the queue, in this case:

( cmd0, cmd1, cmd2, cmd5 ) or ( cmd0, cmd3, cmd4, cmd5 )

depending on whether `[ expr ]` evaluated as true or false.

Damn, this so makes me want to straighten up the necessary parts of tpsh… now if only I didn’t have to work tomorrow from morning to whenever I drop over undead, or pass out lol. These kinds of things have always fascinated me about programming, but life has never offered much chance to study compilers, virtual machines, and interpretors… let along create something like tpsh, that requires implementing a scripting language :. The one good thing, despite the ups/downs and expressiveness of sh script, it’s really an easy language for a human to understand.

hmm, there was something I wanted to add a `set -o option` for, but I can’t remember what the heck it was lol.

man it’s been a slow day… I’ve only made about 15~16 commits on tpsh, one of which was merging branches lol. The main focuses of today’s tinkering has been minor bugfixes (mostly related to and tweaking the completion function.

Previously my completer would just take a quick approach, but always doing completions for built in commands, stored macros, known programs, and filenames. Now it has some notion of *what* it should complete.

$ b^I -> assume it can be anything
$ echo b^ -> assume it’s a file (formally a full eval() was done)
$ builtin l^I -> complete built-in commands
$ alias x -> complete to names in %Macros.

When I get the time, I want to make the completion function have a more hot-plug-in nature to it. A pre-completion hook, completion hook, and post-completion hook; the pre getting to modify the thing before completion, the completion hook being able to totally replace do_completion() aside from the pre/post hooks; and the post-hook having a change to modify the completions being returned. The current working idea is: a TPSH_WHAT_HOOK environment variable being set to A. perl code to generate a CODE ref from, or B. an external program to delegate to. (e.g. TPSH_COMPLETION_HOOK=”compl.rb” would pass compl.rb the necessary data, and expect the program to generate a n delimited list on its stdout for tpsh to parse). Odds are there will be more variables to tune it with later, as well as the API avail. being documented in the manual page My main reason for desiring this style of dynamically configurable completion is so I could do things like, seting up my tpshrc to “ignore” any ‘lost+found’ entries in filename completions; i.e. /usr/local/l^I expands to /usr/local/local not /usr/local/lo – completion list for local / lost+found. I know the zsh (which I use a lot) is noted for it’s configurable completion and spelling correction, but honestly I’ve never customized it beyond the lines zsh’s setup program added to my rc, let along RTFM’d about it lol.

Making it easy to add command sensitive completions through the hooks would be nice too, e.g. `man 2 foo^I` and have a hook spot the man 2 and go about completing foo to manual pages in section 2, hehe. (the only zsh feature I abuse, actually lol)

Really what needs doing atm in tpsh, is cleaning up the lex/eval subroutines and merging them and a few related changes with resolve_cmd(). Basically the principal adjustment needed is to go from line-based handling of commands, to (properly) breaking them lexically for storage into an execution queue. Generally though I like how stuffs coming along, except for a few odds and ends here and there.

At one point,tpsh relied on a list of “this command marks something to skip macro expansion on”, a quick solution until expand_aliases() could be rewritten. One night I found myself working on a scratch file with commands to modify said list (add, rm, and show the non expanding commands list via expr or array slice/index). That’s when I threw my hands up in disguest, and decided tomorrows job would be fixing expand_aliases (note to self: s/aliases/macro/). Imagine if you tried to type ‘unalias ls’ and the shell expanded ls to it’s aliased value, that kinda stuff. Kind a cool to be able to configure that, but kinda wrong to need a linear look up on each pass through the loop (ugh!).

To add built-in commands to manipulate said list rather then fix the problem, just was not something I was NOT willing to do… because to my eyes, it would be like a pox on my honour as a programmer: to gamma-weld such a cheap trick into place with built in commands to configure and abuse it rather then fix it — thus the macro expansion system got re-implemented, much more correctly this time. (didn’t have time to sort positional params and what nots).

I am also VERY glad that tpsh is being developed with git, rather then cvs! (I’ve also spent part of tonight learning more about gits plumbing)