casual fun with the Perl profiler

call: dproffpp -ap shift.pl

there are three sets of data, each meant to represent a small, medium, or large set. Each set is a list of words, 5, 25, and 100 words long respectively. (realistically the elements would average within 2 to 5 words inclusive). For simplicity N is 10.

There are 2 functions, xx and yy; representing different ways of solving the same problem: pretty printing the last N items of a given data set. In xx(), the set is reversed and then $#set -= N’d to clip all but the last N items, then reversed again to put it back into proper order. In yy() we avoid any reversals and just shift off the front of the list one at a time, until we reach N items left in the set. If the set contains less then N elements, no adjustment need be done.

Each function is called 3 times per iteration, once with each data set, over 3000 iterations (that’s 9000 calls to each function, or 3000 times with each set). The test was then executed 10 times.

The things so simple, it’s not important how long it takes to finish, but I’m interested in how big the the difference is between for (expr1; expr2; expr3) { shift @list } and $#list -= expr; and how much those two reversals hurt.

Every time, yy() ran faster by at least a half second. Then ran tests with xx() doing one reversal, then no reversals and yy() still beat it.

Now out of more curiosity, let’s see how larger data sets work. Each data set now contains 3 words instead of 1, and N is now 43; with the data sets being 5, 25, 100, 250, 500, and 1000 elements long.

A new function, zz() which is xx() without the reversals is also executed during the tests. After running the tests a short duration, it seems that the $#set -= N’ing is a bit faster, more so then the cost of the reversals.

here’s the new run down:

$ do_test() {
> local T=10
> while [ $T -gt 0 ]; do
> N=$1 dprofpp -ap shift.pl | tail -n 7 >> dp
> T=$(($T - 1))
> done
> }
$ for NUM in `builtin echo "3n10n43n51n227n"`; do do_test $NUM; done

The above (z)sh code will execute the test on shift.pl 10 times with an N of 3, 10, 43, 51, and then 227; appending the report (3 subs = (4+3) lines) to the file ‘dp’ for post-cpu meltdown review, otherwise we would have to take a look at all the I/O the tests generate before the report is printed by dprofpp.

Yes, I’m to damn lazy to use command history, let along retype the commands each time; why else would they have invented functions and loops 😛

About 15 minutes and 17 degrees Celsius later, some of the arithmetic involved finally caught up with my throbbing head.

Recap:

  • 5 tests, each test has a different value of N
  • 10 runs of each test, meaning 50 runs
  • each run examines the data sets 3000 times, for 150,000 examinations
  • each examination calls 3 functions once with each of 6 data sets, 18 function calls per examination.
  • The six data sets consists of a list of 5, 25, 100, 250, 500, and 1,000 elements; each element is 3 words long. So like 1880 elements in the data set, and 5,640,000 elements processed per examination

So that is what, maybe 2,700,000 function calls to &xx, &yy, and &zz; without counting the calls within those functions… and 846,000,000,000 list elements processed overall? After a little estimation based on the original data set/run time, I stopped counting after the estimated execution time passed 8 hours * X on this old laptop. Hmm, how’s that old saying go, curiosity fried the geeks cooling fan? lol.

I’m beginning to understand why some peoples workstations have like 32 ~ 64 GB of ECC RAM, and Symmetrical Multiple Processor (SMP) configurations to drool $! for, haha!

How the code comes out, when you’re ready to pass out

# message for commit 041a7343eb452b827dbd97a0c82c8538597f86f6:
#
# read built-in command implemented


sub read_bin {

my ($prmpt, $time);
my $line = "";
my @argv = @_;
my %opts = ( 'p=s' => $prmpt,
't=s' => $time,
'e' => sub { "no-op" },
);

do_getopt(@argv, %opts);
unless (@argv) {
warn "I can't read into thin air!";
return 0;
}

if ($prmpt and -t *STDIN) {
print $prmpt;
}

eval {
# remove custom die for ease of error check/report
local %SIG;
$SIG{__DIE__} = sub { die @_ };
$SIG{ALRM} = sub { die "timed-outn" };
if ($time) {
# an s, m, or h suffix causes sleep for sec, min, or hour
#
if ($time =~ /^(d*)([smh])/) {
if ($2 eq 's') {
$time = $1;
} elsif ($2 eq 'm') {
$time = $1 * 60;
} elsif ($2 eq 'h') {
$time = $1 * 3600;
} else {
warn "internal error on ", __LINE__;
# NOTREACHED
}
}
alarm $time;
}
chomp($line = );
alarm 0 if $time;
};
if ($@) {
warn $@ unless $@ eq "timed-outn";
# on time out, init the vars to empty strings
@ENV{@argv} = ('') x scalar @argv;
return 0;
} else {
# set each var to the words
#
# XXX because ifsplit has no notion of a &split 'LIMIT'
# if we used ifsplit here instead of a manual split,
# read x y
# foo bar ham
# would set $y to 'bar' instead of 'bar ham'
#
my $ifs = defined $ENV{IFS} ? qr/[$ENV{IFS}]/ : qr/s/;
@ENV{@argv} = grep !/^$/, split $ifs, $line, scalar(@argv);
return 1;
}
}

from the manual page updated in commit b1317d7e6e7f91b6c3a2650f44cd4f425e381d42 with message:

read built-in command documented

and blockquoted here using the pod2html output

read [-p prompt] [-t timeout] variable …]

Read a line from standard input, split by fields, and assign each field to the indicated variables. If the number of variables is less then the number of fields, the remaining fields will be stored ‘as is’ in the last variable. If there are more variables then fields, the excess variables will be undefined. A prompt may be printed before reading input, by using the -p option. The -t option may be used to specify a timeout in which to abort the operation, should the user take their sweet time about pressing CR. The timeout value can take an optional s, m, or h suffix to denote seconds, minutes, or hours. If no suffix is given, s will be assumed.

It’s not the greatest… but hey, I ain’t had any sleep since this mornings roll out… lol. It’ll do fine for an initial implementation, until I’ve actually got a functioning brain to deal with it 😛

QOTD

Don’t worry about avoiding temptation — as you grow older, it starts
avoiding you.
— The Old Farmer’s Almanac

I wonder, if I’ll ever live to be that old lol.

The wonders of Raven Shield….

Ok, not so surprising I slice ’round the corner, spot the X-Ray on the top of the stairs -> squeeze off a shot from my M16A2 on 3-round burst. Ok, no effect not even a flinch so 1ce more… 2ce more…. POW he shoots me.

The evil thing?

Trigger pulls -> 3×9 rounds should have been fired
Magazine -> 23/31 rounds remaining (on HUD)
Score board -> 8 rounds fired

the guy should have a least 7 holes in him, 9 if I didn’t hit the rail below the POI by some work of magic… but 8 rounds on the score board, with 0 hits to a slow tango rofl!!!!!

Now if I didn’t have to be awake in ~4 and a half hours, I could get the . / source commands implemented, so handling tpsh_profile / .tpshrc files on startup could be done… and have more of the manual page written out :. Hmm, that reminds me another big thing I need to work on is the shell special variables ($0…$9…, $*, $@, $#, $?) and the rest of ${parameter expansion} syntax.

If I could get anything done during the day here, and actually sleep at night… instead of coding odd hours, now that would just be heavenly lol.

$ git log | sed 's/my name/<snip>/g' | sed 's//<sed@removed.it>/g' | vim -

commit 4835c4091e59af282ee98952568f0e9ac91c8d09
Author: Terry <snip> <sed@removed.it>
Date: Sat Mar 21 09:27:41 2009 +0000

do_getop() made generic, `set [options]` now works.

The do_getopt() function now takes an array ref and hash to pass onto
Getopt::Long::GetOptionsFromArray, and twists it in place.

set_bin() now calls do_getopt() with it’s argument vector and a global
hash; startup code now calls do_getopt() with @ARGV and that same hash.
Because of that, `set -x`, `set -o xtrace`, and such work but `set +o
xtrace` and such can’t be used to turn the option off yet lol. (+o atm
is just an alias for -o.)

commit 90bf68e8b5f0aed867ae4a9fd2eca2c8d99dc1fd
Author: Terry <snip> <sed@removed.it>
Date: Sat Mar 21 08:16:39 2009 +0000

tpsh -o longname now works

further more as an extension, ‘-o logname1,longname2’ also works but is
not documented yet.

commit dcffa41bb57d70c82f502b08ea8c76ebeb559b1c
Author: Terry <snip> <sed@removed.it>
Date: Sat Mar 21 07:49:28 2009 +0000

which built-in command added and documents

see Built-in Commands and CEVEATS & BUGS in tpsh.1.pod for details.

It is really time to make do_getopt() a generic wrapper around
Getopt::Long, the current functionality in do_getopt() needs to be moved
into set_bin() anyway…

Thanks Wiz… lol

After a few other commits, I’ve finally buckled down and made do_getopt() a simple helper, previously it did parsing for @ARGV via the Getopt::Long module, and twiddle %Options accordingly; now it takes an array ref and a hash to do the job. The wonderful thing? Now both tpsh [options] and the built in `set [options]` command work with half a line of duplicated code 🙂

In prepping my code for commit, I wanted to take one last look at do_getopt(), so I went for a quick jump over with the vi/vim :tag command. Wiz IM’d me, so he ended up getting the :ta command instead of vim haha! After alt+tabbing back to the editor, I tried something different just for the heck of it, :ta do_ and guess what!? Vims ex mode completion works on tags as well…. this is so going to spoil me lol. ex/vi lacked command completion, history, and editing; but vim added them. One of these days, I need a refresher on the improved tag commands, my muscle memory is pretty vi compatible until we get into tab completion, :split windows, and multiple tabs lol.

So thanks to Wiz, I’ve just found a lovely Vi IMproved feature that I never dreamed existed xD

more recent tpsh commits

commit 6f94f751ac76177a23138a1ad014b4af1af50de7
Author: Terry <snip> <sed@removed.it>
Date: Fri Mar 20 18:29:43 2009 +0000

%Aliases renamed %Macros

because it stores all named macros, whether defined with function, macro, or alias

commit 2a83cbc45daf78cb41cba40c28735252e521f965
Author: Terry <snip> <sed@removed.it>
Date: Fri Mar 20 18:24:55 2009 +0000

is_macro() and is_builtin() subroutines added

commit 1853b61eff14d35ddf1f89282ed26a8f3957d373
Author: Terry <snip> <sed@removed.it>
Date: Fri Mar 20 08:43:50 2009 +0000

type built-in documented

documented the type built-in command, and tweaked type_bin; it now
prints macro instead of alias, and will stop at the first match found.

commit 39a9824ca29d7beea6c5a054f970b86968861568
Author: Terry <snip> <sed@removed.it>
Date: Fri Mar 20 08:30:12 2009 +0000

pwd built-in added and documented

commit 488aa51ed2c430848aa80fb26ad093ccec604237
Author: Terry <snip> <sed@removed.it>
Date: Fri Mar 20 08:08:04 2009 +0000

function and macro built-in commands documented

commit 273b7f8bbbf0b3db8759053969e854f1de963003
Author: Terry <snip> <sed@removed.it>
Date: Fri Mar 20 08:02:41 2009 +0000

pipes and I/O redirection can be used inside a named macro definition created with the macro built-in

commit 7aa70dd6873e147f3bfdec7460312dc54482696b
Author: Terry <snip> <sed@removed.it>
Date: Fri Mar 20 07:53:06 2009 +0000

macro_bin now reports `macro name()` as an error, correctly

commit e52aed3dec5455c6488dc4465aa75d20aca4e696
Author: Terry <snip> <sed@removed.it>
Date: Fri Mar 20 07:22:39 2009 +0000

hash built-in documented

commit 843a0de91b3c2df2b1fed8b9416426bc0e7c01af
Author: Terry <snip> <sed@removed.it>
Date: Fri Mar 20 07:21:22 2009 +0000

hash built-in now has an -r option that nukes the %Path hash table

commit 27526bb013cc1ab7b425ee1f3b685585746a0160
Author: Terry <snip> <sed@removed.it>
Date: Fri Mar 20 07:17:36 2009 +0000

hash_bin now displays an error if the item to be deleted does not exist

commit 45c71f7c54faa81106de5e68290884150823e10c
Author: Terry <snip> <sed@removed.it>
Date: Fri Mar 20 07:12:50 2009 +0000

hash_bin can now remove elements from %Path

When the only damn time I can get any sleep… is when I’m ready to pass out, why can’t anyone let me sleep? Some days it feels like a game: keep spidey01 awake. *sigh*, and to be waken up early tomorrow…. really, why do I even try to sleep anymore?

Home at last for the rest of the day; got off work early enough for a quick sandwich and some kartoffelchips before taking Willow for her checkup.

13.04 lbs (~5.915kg) and strong as an ox, gained 1lb since last year lol. It’s funny though, she’s like a little bully at home but as soon as we get within earshot of the vets office…. petrified chihuahua! I’m standing in the examination room with Willow on my one arm, and her and my mom are nervous wrecks; ma asked me how I could be so calm, my reply? It would take something like a zombie invasion to worry me lol.

Pardoning grocery shopping (dang), I’m off until Tuesday; maybe I might actually get some sleep for a change.

Want a dirty look? Try telling a dog she’s fat >_<.

Oh for korn sake….

Willow saw the mouse cursor move, and tried to lick the screen! I gave her a hug, “Willow, I love you but you’re a silly girl” lol.

May as well hit the hay, before the dog tries anything else with the laptop… lol. Tomorrows gonna be a wreck anyway, need to take the dog to the Victor Echo Tango for a checkup after work, that will be a bark-fest once we arrive, haha!