Terry’s Portable SHell

Hmm, my recent pet project (tpsh) is actually shaping up quite nicely. It is almost to what I would consider a usable interactive state, the only big irk left I guess is the pipe handling and a few odds and ends. Development started like last month (first commit in my master branch is dated 2009-02-28), and has been a very on/off thing in my spare time; a lot remains to be done, but progress is quite nice.

I rather like the korn shells, but I find it annoying to see what implementation of korn from who behaves in what way. OpenBSDs customized pdksh being the best of the lot in my opinion, the only problem with the korn *family* being consistency in the wild… cmd.exe just barely counts as a command shell, better for scripting then for interactive use. I can’t help but think, cmd.exe’s filename completion was either written by a 5th grader or a hold over from command.com (I haven’t used command.com in years). Currently I use zsh most of the day on FreeBSD, ksh on OpenBSD, and cmd.exe on Windows XP. So it is really important to me to have a single and SANE shell.

I used to use tcsh a lot, but after getting into Bourne-style shell scripting (Nota bene: I know nothing about csh scripting) I got pissed and sought out a shell that would allow me to test sh snippets at the prompt. To be honest, I do not have any issues with the GNU Bourne Again SHell (bash); some distros default init files aside. Yet just like tcsh/zsh — I have no big desire to cart bash around with me everywhere (and I do not like cygwin).

The solution? Create a shell that behaves more or less consistently across platforms, does what I want, and is not a pain in the ass to port along for the ride.

Line editing is completely delegated to Term::ReadLine. Someday writing something like readline or curses on an alien OS for terminal XYZ would be one thing, having to implement shell line editing and be portable between the UNIX and Windows NT environments: I wouldn’t do it if you paid me. (and even if drunk, I would still likely use normal libraries!)

On the interactive level, line editing for me is an absolute must. Command completion I can live without for awhile. Unix names are usually short and I like fairly terse and structured ones myself, winsucks is a bit of a different story. The child like way cmd.exe does command completion, it’s almost better to do without completion on windows haha!

Right now basic expansions work, ${VAR} and $VAR, line editing and history is via Term::ReadLine (I’ve been using Term::ReadLine:: Zoid and Perl implementations). There’s only rudimentary handling of quoting for the time being, something that needs labouring with in order to handle nested quotes (e.g. echo “`ls ‘C:/Program Files’`”) but currently does the job well enough for right now. The ‘single’ and “double” quotes mostly work as expected, The `backticks` work as expected in simple cases, but since there is no proper support for shell script or -c “command” yet, things like `find /some/where -name foo -print` will work as expected, but `find /some/where -name foo -print | xargs whatever` will not work. Once things are more evolved on the scripting side, it should be trivial to implement backticks correctly (nested quotes aside). The idea being to pass the backticks onto a child shell. Right now the only big catch 22 with quotes is only one set can be used and they do not nest yet; fixing that will be fun.

I/O Redirection and pipes also work for starters, but not completely. The >, >>, and < operators do work, but don't accept an optional file descriptor number yet (e.g. foobitz 2>/dev/null) or other tricks of the trade. Basically just the I/O redirection I use a lot is implemented for right now; since the focus is on interactive use by yours truly, it will probably stay that way for a while. Command1 | command2 also works well enough right now, but I’ve yet to implement pipe lines properly (e.g. cat f1 f2 f3 | sort | uniq | less; doesn’t work yet); just wasn’t enough time to shift from prototype to finish without sleep lol. Only one use of pipes currently is supported, until I have the time for coding unlimited segments in a pipe line.

Aliases even work ok at the moment, but no (damn!) addictive runtime parameter expansions have been added yet (e.g. alias qux=”xuq -f $1 -x $2″). That will have to be done someday, hehe. In point of fact, $*, $@, and the things related to positional parameters are not even implemented yet.

One benefit of my development environment, is I can use I/O redirection to run a simple test suite. I created a ‘test-sh’ script that strips the environment of all but the interesting stuffs before invoking tpsh. Then I have a test file with commands written out.

 $ ./test-sh < file

and I can watch the results of the test file fly across my screen, as if typed interactively (I freaking love unix shells). It is not quite a unit test, but it is enough to be able to run the test file and monitor the results for unexpected regressions or fuck ups, before pushing the code to vectra. Currently shell globbing is limited to Perls glob(), probably to be implemented directly with File::Glob::bsd_glob() in the future, and someway of letting an environment variable specify the desired GLOB_* flags. (perl 5.8s glob() seems to do what I want). I have yet to decide how to handle tab completion across the Gnu, Perl, and Zoid ReadLine modules but coding the completion functionality itself should be fairly easy; the issue is just how to plug it in properly. Exactly what the globbing behaviour will be like, I dunno yet: because to be honest I have never noticed any big difference between (t)csh, perl, and zsh when it comes to using globs (but I also admit that I rarely do more then abuse * and {x,y,z} on a daily basis). If at all possible, I would like to use File::Glob rather then roll my own (and modern perl glob() does use File::Glob in the background afaik). I believe that as soon as a full pipe line implementation is done, I’ll be ready to start coding it from a tpsh session lol. Goal wise, where am I trying to go? It is implemented in Perl, because perl is fun, perl is less pain to port then C/C++, and I’m not going to use Java or C# period lol (my zsh setup starts up slow enough as is). Python, Ruby, or PHP would also have been good choices, except I’m trying to avoid rubber banding between languages. (Also note: I _hate_ writing C code on Win32.) It should generally behave like a traditional Bourne shell most of the time, but not be a true sh knock off. Whenever in doubt, mimic ash / sh behaviour. It’s not meant to be POSIX compliant, anymore then it is meant to be a real sh: based on, not clone of. It is supposed to behave as close to identical under unix and winnt as possible, with “pains of installing” meant to be a working ReadLine module. A good example is perl -e ‘code’ should work on Windows without having to adapt cmd.exe quoting rules. (really I think cmd.exe has shitty documentation for such an essential program). One reason I like languages like Perl and Python, they mostly behave the same on whatever OS, with a minimal of pain involved (especially Python) It should be easy to use, providing all the expected features — without massive bloat. It behaves like _I_ expect, not what someone else expects (good example: unix style shell, not dos style shell) It is not an interface to Perl. There’s at least 2 different perl shells out there that I know of, but I do not want a shell script that behaves like perl or even does configuration in perl; I want a shell written in perl that behaves like a regular shell lol. It’s also an easy project to take my mind else other matters in life right now… tpsh’s builtin eval command will work more or less like sh, not CORE::eval !!! Maybe I will implement a peval or pleval builtin that works like shells eval, but instead executes it as perl code rather then shell commands. (basically type `peval perl` code instead of `perl -e ‘perl code’`). Accessing perl code through tpsh just is not a priority, when I want to talk to a perl interpretor, I’ll invoke one. Eventually I want it to be able to execute shell script in some form, the goal of which (obviously) will be handling my shells initialization files (~/.${USER}_shrc and ~/.site_shrc) without going belly up. To pull that off, a Bourne-style shell, a which program or built in, and support for functions / aliases is required. (My init files handle sh, bash, zsh, and several flavours of korn; plus several OSes from one main file + a site local settings file). My init file is rather elaborate in design compared to most others I’ve seen, yet very conservative in what it demands of the shell: the most advanced sh features required being function support lol. Basic idea of feature/priorities for tpsh:

 a usable shell for daily use (basically done)
tab completion (filenames, then command; then pluggable, then 'smart' completion)
code the remaining builtins and allow a coloured prompt ;-) (etc)
expand the expression syntax into a more usable (read normal) form
do shell functions
handle job control (basic (i.e. bg, fg, &), then rest)
allow shell scripting in a conventional way, rather then tpsh < cmdfile
grow the control flow and things for shell script (if, while, for, ...)
take POSIX into closer consideration for the minor details

the only part that worries me, is implementing the [ builtin lol. (mm, maybe feed [ stuff ] into a syntax tree that we can build a corresponding perl statement to eval())

note to self: finish the e-paper work on Carters big day

Using git for all my scm/vcs needs, I’m starting to wonder how the hell I ever lived with CVS lol. (The documentation for git isn’t to bad these days either)

Writer’s Block: Deal or No Deal

What’s on your list of dealbreakers when it comes to romantic relationships?

Live Journals Writer’s Block

  • If she’s a bitch, just plain cruel, or has no feeling for other people (Gee, sounds familiar), then I’m more likely to regard her with contempt more then anything else, just like everyone else with that kind of attitude towards others.
  • If she is a Bimbo, or worse a first class / major bimbo… that is an end to almost all romantic interest on my part lol. Bimbo = common name for the type of girl that usually gives me a _H_E_A_D_A_C_H_E_ in the long run. I don’t mean the woman has to be a rocket scientist or something like that, but some common level of intelligence and attention span is a must!
  • If she (or I) is only interested in sex or that’s the primary goal in/nature of the relationship. For me, things have to go much deeper then that to be lasting, let along develop a romantic tone. I’m a guy, so such things obviously cross my mind eventually, but still I would take the effort for her mind, body, and soul: not solely for the sake of a home run.
  • If she’s currently involved with someone or married, then that’s a no go: automatic mental disconnect (my brain generally raises the off limits sign).
  • If she can’t love me for who and what I am, deals off.

Hmm

A hopeless romantic is not the same as a hopeless flirter. A hopeless romantic dreams of who they will spend the rest of their life with and what the two of them will do together. They want to be romanced with sweet simple things and the thoughtful amazing surprises. They dream of being loved but also loving somebody. They don’t just want somebody to hold them. they also want to hold someone. They realize that love isn’t just about one person but both people. they are hopelessly in love with being loved AND loving back.

Writer’s Block: More Island Time

You’re packing your bag for that other desert island—the one with no electricity—what 5 books do you take with you?

Live Journals Writer’s Block

If trillogies count as 3 books:

0. My copy of the Bible (it’s an NIV)
1. The Thrawn Trilogy: Dark Force Rising
2. The Thrawn Trilogy: Heir to the Empire
3. The Thrawn Trilogy: The Last Command
4. Lord of the Rings (it is one insanely huge book!) or Dune.

If trillogies count as 1 book:

0. My copy of the Bible (it’s an NIV)
1. The Thrawn Trilogy
2. Dune
3. Lord of the Rings
4. My old U.S. Army survival field manual

Wouldn’t be much of a list without a copy of the Bible on it, now would it? When I started in Fusion (one of two youth groups at the church), my pastor was polite enough to equip me with a copy free of charge. Haha, I still remember the first time I read it… friends had to teach me the whole chapter _and_ verse notation (yeah, that was an awkard moment lol). One day everyone in the group was asked to read a passage picked at random, I ended up in the book of Acts. Later on my D-Group had a week long pledge to read the bible every night, being me I of course had to inhale all of Genesis. I rarely get to read the bible these days, but it’s still a book I would take. The perfect chance to the entire thing cover to cover.

Timothy Zahns first tale about the imperial grand admerial Thrawn is one of my favorites, including the first appearance of my all time favorite Star Wars charactor: Mara Jade. I remember the first time I read it, I couldn’t help but wonder where Luke and Mara might end up in the future. The Hand of Thrawn duology solved that question lol (the skywalker curse aside, which she seems to be the only women to ever survive reasonably well). For some good reason, all the best Star Wars novels seem to come in trilogies and average at least 950pg or better xD. I’ve always loved Star Wars, especially the expanded universe. Largely gave up reading them after Vector Prime though, but If I had the cash; from things that I’ve heard more recently, I think I would pick up where I left off from and catch up with the years. Out of all the SW books I’ve read, the original Thrawn stories are my top fav, hence a must on the list 🙂

Dune, a movie I loved and a novel I really loved once I got to read it. Dune is just my kind of story, big, long, so much depth and imersion – like an RPG taken to an extreme. I also love Sci-Fi and stories that give one room to think a bit. Anyone who likes science fiction or fantasy must read Dune eventuall!!!!

I’m not a huge LoTR fan, but the Lord of the Rings is a very big, long, and in depth book – just right up my ally (like Dune). That reminds me, one of these days I need to puchase the remaining volumes and inhale them haha. J.R.R. Tolkien and Frank Herbert seem to have a great nack for creating their own little (eh, big) world, and finding a way to suck you into it…. you’ll either have to learn all the history of it, or be bored out of your mind before the end of the book (in which case, you probably wouldn’t like such books anyway)

Some years ago, well many now I guess lol. My brother picked up a few old field manuals from a military antique and surplus store. I have the manual to an M1 .30-06 rifle (the Grand), fm on booby trapping (where most of my minor-knowledge of explosives comes from), and Survival; which covers survival issues on land and sea, both tropical/desert and arctic environments (even for the avator that has fallen from the sky). After ~40 years, I’m sure it is woefully out of date or been replaced at least once or twice by the army, but hey… I wouldn’t dare plan a camping trip without it, let along being stuck on a desert island!

I love to read, although honestly these days any reading time I actually get to myself… usually ends up on something techy. There is just nothing like a good book…. born a couch potato, but books are better then watching the TV. Unlike television, where they have to pick and choose what to show you, a book lets you choose what you see. To explore that authors vision, and gaze upon it with your own eyes; dive into an eventure or visit a strange new world that only exists there in. Plus there is a lot more content then TV can offer lol.

And yes, Star Wars dominates my book shelfs better segments xD.

Writer’s Block: Desert Island Time

You’re packing your bag for that magical desert island that happens to have electricity, a TV, and a DVD player—what five DVDs do you take with you?

Live Journals Writer’s Block

0. Terminator 2: Judgement Day
1. Spaceballs
2. ALIENS (think space marines and xenomorphs)
3. El Dorado (my favorite western)
4. A classic, maybe Now Voyager or something like Arsenic & Old Lace

Spaceballs though, would have to be on the list! Heh, I think I’ve seen that move so many times over the years that I could probably be the dork on set with the script, should they ever have to re-film it ^_^

Yet Another Way To Hang Your Web Browser

Try copy/pasting ~4.5MB of data into a text area, then watch your CPU catch fire and the laptop overheat. If you have a decent browser or a cooling system, you might get as far as the form submission timing out ^_^,

Honestly, I could swear there is almost no complex software on this damn planet that does not f***ing suck!

Complex software that doesn’t suck: FreeBSD, OpenBSD, nvi, vim, GNU Screen, rxvt-unicode, GNU grep, nawk, ed, the better forms of emacs, gcc/g++, perl, fvwm2, but what else….. lol. There is more software that sucks then that doesn’t.

learning to use warnings, the hardway

Trying to write to file names instead of file handles by way of typographical error, yes I am a super schmuck! Trying to figure out for at least 5 minutes why the parant isn’t getting any output from the child process, and sure enough…. Don’t print $filename when you mean print $filehandle, haha! (or better yet, don’t use ($lhs, $rhs) and ($lfd, $rfd) for the file names and handles when tired)

Tried running my scratch-file under use warnings, and sure enough the expected warnings poped up. This is what I get for not enabling warnings when writing a function in a separate file for testing crap. Hmm, what was it she said to me years back? Something like “We need to pump some caffeine into you”, maybe it’s not a bad idea lool.

Rule #1 of Perl, use warnings or die $!
Rule #2 of Perl, use strict or warn $!
Rule #3 of Perl, goto Rule #0

(use what or … is not actually perl syntax, let a long setting $!, and is infact a compiler error that would have to be trapped in an eval, but hey at this time of morning it works well enough in ascii :-P)

Writer’s Block: Taking It Personally

Have you ever taken a personality test like the Myers-Briggs or Enneagram? If so, did you agree with the results? And what was your type?

Live Journals Writer’s Block

Once took something based on the Myers-Briggs style, can’t remember the type though. The result was fairly accurate, the only problem was it was also classified as being among like 2% of the population with that type lol.

I’ve always valued my uniqueness, hehe.