Qute I/O

Been meandering about with Pythons subprocess and various IPC (inter Process Communication) focuses modules tonight. When I remembered I had noted that QT and KDE had classes for working with processes.

Looked up the class in the documentation and played with it. Then I found a small example program that they included. Here is the QT written example using C++, the primary language.


/****************************************************************************
** $Id: qt/process.cpp 3.3.7 edited Aug 31 2005 $
**
** Copyright (C) 1992-2005 Trolltech AS. All rights reserved.
**
** This file is part of an example program for Qt. This example
** program may be used, distributed and modified without limitation.
**
*****************************************************************************/

#include <qobject.h>
#include <qprocess.h>
#include <qvbox.h>
#include <qtextview.h>
#include <qpushbutton.h>
#include <qapplication.h>
#include <qmessagebox.h>

#include <stdlib.h>

class UicManager : public QVBox
{
Q_OBJECT

public:
UicManager();
~UicManager() {}

public slots:
void readFromStdout();
void scrollToTop();

private:
QProcess *proc;
QTextView *output;
QPushButton *quitButton;
};

UicManager::UicManager()
{
// Layout
output = new QTextView( this );
quitButton = new QPushButton( tr("Quit"), this );
connect( quitButton, SIGNAL(clicked()),
qApp, SLOT(quit()) );
resize( 500, 500 );

// QProcess related code
proc = new QProcess( this );

// Set up the command and arguments.
// On the command line you would do:
// uic -tr i18n "small_dialog.ui"
proc->addArgument( "uic" );
proc->addArgument( "-tr" );
proc->addArgument( "i18n" );
proc->addArgument( "small_dialog.ui" );

connect( proc, SIGNAL(readyReadStdout()),
this, SLOT(readFromStdout()) );
connect( proc, SIGNAL(processExited()),
this, SLOT(scrollToTop()) );

if ( !proc->start() ) {
// error handling
QMessageBox::critical( 0,
tr("Fatal error"),
tr("Could not start the uic command."),
tr("Quit") );
exit( -1 );
}
}

void UicManager::readFromStdout()
{
// Read and process the data.
// Bear in mind that the data might be output in chunks.
output->append( proc->readStdout() );
}

void UicManager::scrollToTop()
{
output->setContentsPos( 0, 0 );
}

int main( int argc, char **argv )
{
QApplication a( argc, argv );
UicManager manager;
a.setMainWidget( &manager );
manager.show();
return a.exec();
}

#include "process.moc"

Needless to say, I am not really a fan of C++ at times ^_^. I translated the example into Python for a simple test and I like it quite a bit.

#!/usr/local/bin/python

# I translated the example: process/process.cpp to python and boy do I love qt!
#
#############################################################################
##
## Copyright (C) 1992-2005 Trolltech AS. All rights reserved.
##
## This file is part of an example program for Qt. This example
## program may be used, distributed and modified without limitation.
##
##############################################################################

import sys
from qt import *

class UicManager(QVBox):
def __init__(self):
QVBox.__init__(self)
self.output = QTextView(self)
self.quitButton = QPushButton(self.tr('Quit'), self)
self.connect(self.quitButton, SIGNAL('clicked()'), qApp,
SLOT('quit()'))
self.resize(500,500)

# QProcess related code
self.proc = QProcess(self)

self.proc.addArgument("uic")
self.proc.addArgument("form1.ui")

self.connect(self.proc, SIGNAL('readyReadStdout()'),
self.readFromStdout)
self.connect(self.proc, SIGNAL('processExited()'),
self.scrollToTop)

if not self.proc.start():
sys.stderr.write('fatal error could not start uicn')
sys.exit(1)

def readFromStdout(self):
'read in proc data which may be in chucks'
self.output.append(str(self.proc.readStdout()))

def scrollToTop(self):
self.output.setContentsPos(0,0)

if __name__ == "__main__":
a = QApplication(sys.argv)
manager = UicManager()
a.setMainWidget(manager)
manager.show()
sys.exit(a.exec_loop())

QProcess is nice but I’m not sure about using enormous amounts of data though yet.

EDIT:

As a test I set up the Python script to pass a ‘ls -R /’ basically causing it to list all files on all mounted file systems. The code to scroll back to the top caused major lock up to the program and some performance loss for the entire laptop: probably screaming at being down to about a 150MB of swap space left.

Removing that and closing programs so only konsole running it and top + amarok playing music was running. At times top reported free memory over < 1MB about 30-40% swap usage. However without the scroll to top thing gone it didn't cause any real performance for the laptop; probably because there was much less swap usage used up. As a test I ran ls -R / direct afterwards and while it did pile on the stats in top, it still kept to about 50MB free and even less swap usage on the displays.
So far I think it would probably work for whatever is needed as long as the computer continues to provide adequate memory and the kernel doesn’t have a hissy fit about the pipes and things in the background.

Ramblings of a drowsy spider

I comptemplate using C++ again as one of my primary languages, it was the first I started to learn with but one I’ve never put to much good use.

I generally consider C and C++ different languages, with C++ being C like in so far as C++ is similar to C. Although I do actually like it if the commonalities allow for using either a standard C compiler or C++ compiler. But I still think one should pick one and stick to it.

The main reason for considering C++ as a primary language again, is that it is more C like then any language I use regularly (besides C) so adapting it would be easy. The differences it has over C, might actually make C++ potentionally more productive for me or just annoying but.

The main reason I think digging back into C++ would be counter-productive is the low-level aspects it shares with C, would likely have the same result. Namely the reason I tend look at Bourne shell scripting before deciding to implement things.

The many utilities available outweigh the sense of writing ones own methods of processing the problem. Why have a specialized set of functions when you can just use cat, sed, grep, and awk? And with languages like Ruby or Python; the ability to test code quickly in an interpreter and a fast library of useful tools (batteries included) can be time savers of the write it or find a library for it problem I often face in C.

The main reason I use C, is I love the language and find it very beautiful. Learning the inner secrets of how computers work is a goal for me, one I also find can be a bit of an anti-problem for just getting stuff done.

Using C++ again might also have the problem of my understanding of C interfering with my use of C++ and vice versa after awhile…

Of all the languages I have encountered, The worst things I can say about them…

C -> All fine and dandy till you shoot yourself in the foot without realizing it.

C++ -> See above

Java -> Well, other then the shere bulk these days… I would prefer it to C++ if I wasn’t more used to gcc/g++ then javac 😉

Perl -> A fine language but I dislike reading other peoples scripts at times; nether beautiful nor elegant but like a sharp knife very effective if occasionally complicated to use it as such.

PHP -> I just hate using PHP, period; I’d rather use Perl or C++ — Assembly might be preferable to PHP in my book !

Python -> A radical departure syntactically from other languages I am used to working with but it is growing on me… The documentation also is quite nice and the occasional irks I have with Ruby are missing.

Ruby -> Like wise very different from the CBF but much more productive for me then either C or Java. A big advantage for me is similarities Ruby has with Perl, while remaining *not* Perl.

Scheme -> very rudermentry knowledge of as I never got to finish reading the standard before work caught me again lol.

Also I have had brief looks into Ada and X86 Assembly but I still consider Bourne Shell, Python, and Ruby my strongest languages; with C and Java as secondary. I don’t honestly consider myself a good programmer but I think I am better at programming then I am at playing Chess b||b

Ada fascinates me but never had the time to really study it the way I want to. Assembly, is more so some thing that interests me for one I could learn through it rather then some thing I would want to work with a lot.

I love to study different programming languages, it allows me to learn a lot and to try and wrap my head around different ways of doing things.

Standard libraries and things aside, I am mostly indifferent to language, to me it is just another way to write. I used my abilities to read English to learn C++ and Perl. I bumped into C one day and that was that.. lol. It is just a matter of my ‘comfort zone’ that I tend to prefer specific languages, namely that I can get stuff done very quickly in Ruby and Python, bourne shell saves me time, and I love C.

From Ada to Vim script, it is all fun to learn.

When I was entering the 7th Grade I tested as having college level reading comprehension and most related things also had high marks aside from my Spelling. It was actually quite strange to get the test results because I was held back one grade as a child because I “couldn’t read”. My little secret?

I failed the 2nd Grade because I did the reading tests systematically my way. Rather then only hunting down and reading what I could read on the exam when I saw it.

I scanned it for the words I could read and there were plenty of them but the instructor didn’t tell me what she wanted me to do other then read it.

So I went through it word by word, left to right, line by line reading what I could: just as I would today now that I could read. And of course I ran out of time; personally I think the instructor thought my scanning time was just struggling on the bigging of the test lol.

Because I was only scored on the words I read allowed, all of the ones I could read down them, well I guess like 80-90% of the test papers; they didn’t count. I might very well have passed that test as a boy and got into the 3rd Grade rather then having to repeat the 2nd Grade.

My mechanical nature made me fail that test horribly as a child but if I had merely sat down and tried to ‘pass’ the test rather then ‘doing’ it my way. I might never have learned to read the way I did, which I must say is the greatest thing I have ever learned.

English is a very screwed up language when it comes to its written form -> and the family dictionary admits this! But I feel if one can read and understand English well enough, you can learn computers easy.

That is why I find it a bit strange when people struggle with simple tasks on the computer. I don’t expect any one to be able to decipher a hexdump but applying a little reading comprehension and common sense yeah I do expect ^_^

I first used a computer when I was still in diapers, I needed to find the command names on a piece of paper in order to run a program. Because I didn’t know how to list the contents of a floppy disk and figure out what was on it. Today I regularly use 3 PC’s and the command line is my friend. And although I have found superiors when it comes to people that know computers I have never found any one that shares my love of learning.

In gaming I have often said, give me a secure firing position, a mark to shoot at, and I’ll hit it given a good sniper rifle for the job. Like wise, give me a book and often I can inhale it, give me a manual and I might love it. I love to learn but I find school boring as shit… teaches nothing beyond the 3 R’s… In way to many years.

Heck, I could learn more visiting the library daily then I have from school since completing the 3rd or 4th Grade… And it would be quite a bit less boring at the library I bet.

ZzzzZzz

Mondays are always tiring… thanks to the holidays that jobs now on Tuesdays until Janurary.. So I’m about ready to pass out.

Been playing with the prototype for configuring a port. Made some changes including actually writing out the options to a file. I also got to poke around at Pythons ways of handling file locking, nice to see that fcntl, flock, and lockf work more or less the same as in C.

At least work tomorrow work should be light and I might actually get home in time to rest.. The down side to having to reserve most of my time spent working on this to after dark, is I can only go about as long as it takes for my brain to crash.. And that is a bit constraining at times. Between a full days work and work the next morning.. Can’t wait for the weekends!

Tomorrow I need to finish work on the prototype and then start integrating it with the rest of the more complete code the day after. In between I’d like to experiment with a few things in QT, migh tbe a little easier to do the tests in C++ but that wouldn’t be totally helpful… hehe.

Spent about 2 hours tonight, mostly reading make files. I think I’ve read all of the bsd.port* makefiles in /usr/share/mk and /usr/ports/Mk. Along with most of /usr/ports/Tools/* and a ton of stuff in /usr/share/mk/. While there is a lot of leg-work there as far as ports goes it doesn’t seem to be as complicated as I would have thought.

Made some changes to the prototype for the dialog to handle our way to ‘make config’ but it’s to late to work on it further on it. I need to set it up to store what options have been checked / unchecked so they can be used to get things set up to run make.

if I didn’t have the small problem of getting up for work tomorrow I’d have that done tonight…

So far, work on this program has been fairly easy, the main problems I am anticipating is sorting out the final-form of the main GUI. I’ve been using the 2nd style that I posted awhile back, for the prototype but I can still replace it with another quite easy once the rest of the work is done.

I may have 0 formal education when it comes to Computers or Programming. But I am accustomed to doing things the way I am doing them now. Namely trying to implement things in as small and manageable sections as possible; where each section only handles as much as necessary and relies on as little as possible out-side of it’s related code. I like it when things can be tested as stand-alone as possible and then modified/rewritten as part of the larger program. You could say I’m not a fan of 10,000 line monolithic modules that do their job plus everything but recreate emacs lol.

I test the sections making sure that it does what I need it to. Then experiment with it to see how it reacts, and incorporate it into the rest of the project based on what was learned from that prototype. This is just the way I like to tackle the problem of writing a program.

I’m more used to dealing with C and a mess of .c/.h files but python is quite nice. And it has allowed me to spend most of my time concentrating on building a working program, solving the problems not tinkering around with GDB or malloc() more then thinking about how to best make stuff work.

I hate debuggers… But some times they are necessary. My favorite method of coping with some thing that doesn’t work as anticipated is go back to the code and apply brain power. Why doesn’t it work and why does it do what it does instead? Although I must admit GDB is invaluable when needed. But I’ve rarely needed to use a debugger outside of C and C++, and even then usually to track down a stray pointer or get a backtrace.

A short look

Well, almost missed my narrow window of code time… Was passed out on the couch with a full stomach before bed until nearly 0100 xD

Between My mom, my sister, the bird, and my nephew (a louder bird)… And helping with baking there is really not much I can do during the day.. Any attempt at even thinking about seriously trying to read or write goes out the window and in comes a headache. So I have to work at night, when every one else is alseep… Until I crash or the clock reaches a point where I need to go to sleep in order to be @ work on time.

I completed the mock up of a simple dialog for simulating make config. And I’ve almost finished a working prototype for the module but I don’t have time to handle writing the slots and associated code to deal with the check boxes right now. And I need to get that done and tested before I can incorporate the prototype unto the module it belongs.

I posted a screen shot of the mock up awhile back. I’ve fixed the display a bit (the leading variable name from the makefile is removed). And added an ok and cancel button with the beginnings of a more key-board friendly behavior when it comes to using the keys instead of the rat to use it hehe. I’ve also set the caption on the dialog to contain the ports name as category/program.

In doing this, I figured the most simple way was to just muck around with a string of our target (e.g. /usr/ports/net/samba3 and /usr/ports/www/links/ in my tests). Although now that I think of it, the same routine I wrote to get a list of options to create check boxes for. Could be used to look up the category and portname variables from the makefile instead because it’s not tied to looking up any specific variable. — TODO: compare both methods for speed
after the prototype is ready to be moved to alpha group.

In trying to figure out how to get the desired result from a string version of our working path, I remembered that most languages offer some way to obtain the basename of a file or directory and set to look for it in Pythons os module. But doing that would mean we’d have to cut off the trailing slash and refeed the path to the library routine then join them into a new thing. Not exactly my idea of fun when writing a quick helper subroutine. I was very happy that help(os.path.split) revealed a method that takes care of most of that legwork itself.

Python also has some thing called list compression and which allows the mapping of the contents from one list into another based on given criteria. I’ve generally avoided list compressions out of disfavor for some of the (large) examples I’ve seen before. But for this, I actually found it yielded both quick to write and easy to understand code.

p = name # '/usr/ports/category/program' for example
prog = os.path.split(p)
cat = os.path.split(prog[0])

path = ["%s/%s" % (c,p) for c,p in zip(cat, prog) if c and p]
return path.pop()

The above snippet splits name into two lists of two elements, each a string. Basically the 2nd line returns a list of [‘/usr/ports/category’, ‘program’]. So obviously line 3 likewise splits /usr/ports/category into a list of two elements, the last of which is category. I really love how os.path.split() speeds up reading this, it takes only a glance to read it without having to double check it.

The 5th line maps any elements of the two lists (cat and prog) into a single a list of strings, i.e. some thing like path => [ ‘/usr/ports/category’, ‘category/program’ ]. The test for each element being true could be ommited in this case but I wanted to show the possibility.

Since path is a list and we want to display a string (without using str() on the result!), we just pop() the last element off path, which is exactly the element we want ‘category/program’

mm, now for some sleep… It’s after 5am here and I’m exhausted.

tinkering

click to enlarge
Free Image Hosting at www.ImageShack.us

Last night I was trying to figure out a little bit about trying to display a suitable array of checkboxes for letting the user select the ports options. I checked out the QWidget class, looks like a nice time saver but a bit of over kill for what I need. I’m also sorry to see it’s deprecated in QT4, for the heck of it I started trying to write the examples in the QT4 doc’s for replacing it, in PyQT but I’ve got QT3 C++/Python bindings on my laptop and the same for QT4 on my Desktop so I didn’t have time to finish it completly. I find it rather werid using C++ doc’s for Python but I am starting to get pretty used to working with Python although I think Ruby and Perl handle regular expressions better along side strings. Not sure if QT3 has a Regex Engine le talong if PyQT has it if there is one =/. It is good to know how to use many different languages though.

For right now I’ve been tinkering with getting the checkboxes laid out and displayed, like in the screen shot of my mock up above. It is not done yet but once the mock up is finished I can set to work on doing more; most interestingly integrating it with the more robust code in the alpha directory (yes I’m to lazy to set up CVS for working from one machine).

In the end I’m likely to use QT Designer to take care of it, so it can be made to look nice. And after feeding it through pyuic use that and the mock up to create some thing to work in a more complete version. For right now though I need to get a little rest, it has been a very busy day. I would like to rest a little bit and finish the movie. Then get back to working on this stuff when I’m more relaxed but I am liable to fall asleep… I’m off until like Tuesday so I suppose it doesn’t really matter when I wake up… And a quick run of xgalaga is a good mental re focuser hehe

*Yawn*

wrapping make config

I was trying to work on creating a simple pop up window that would list of a ports configuration options, same as the dialog you get when doing a make config on a port with options. But for the life of me, I couldn’t figure out how on earth to nab it because of it being a multi line variable.

With the bird screaming my head off and Ma trying to bake (including calling for help). I gave up on even trying.

Tonight I sat down and tried to *concentrate*, it sucks that I’ve got to wait till everyone else is asleep to ensure I can work with my brain on my thoughts… But the anwser to the problem it me quite quickly once I started tonight.

Don’t think about it as grabbing the OPTIONS out of th Makefile, think of it as if I was the make program!

So obviously if I was make(1) I would want to look through the Makefile in order to find all variables declared. Storing them in a simple data structure and relying on the @ EOL continuation syntax that is used — So I would be getting the ‘whole’ thing when storing a variable for later reference.

Stepping back to the problem at hand, I made it look exclusively for Options by wrapping it in a regex check for the specific variable rather then any. I’ve converted the routine to be able to hunt for an indicated rather then specific variable so it is not so closely tied to the problem it is actually solving for me.

Next up, figuring out how QT handles check boxes and adusting the routine for a ‘nicer’ way to use it.

When I can just sit and work/read, I can in-hale a problem/book but when I’ve got to do things so late at night, I am limited to how long I can go until my brain just conks out of logical thinking… Do to the requirement for sleep.

Blissful reading, uhh cramming.

Finished off the last of my library books.

I only read the sections of Upgrading & Repearing Servers that I was interested in, didn’t have time to even open the book on TCP/IP… I probably don’t need that level of information right now, filled for ‘check out again later’ hehe.

Skipped most of the book on Java and like last time I read a book on Java. I’ve also managed to avoid writing a lot of Java code, not a language I am very fond off… but would likely prefer Java to C++, then again I generally prefer C to C++ too hehe. The goal however was not to get into writing Java programs but to see how much the language had changed since the mid-late 1990’s.

Most of my interest in the books I checked out was the Linux Programming Bible, which was useful for a few of the things common to Unix like OSes. For which the only good examples I have been able to dig up in the past, has been reading my Operating Systems user land programs.

The Security book, heck I would by that sucker if I had the cash; especially if there was a 5th edition xD

I generally inhale information when I am free to just do so, so polishing them off was not a problem.

Night module

Unfortunitly it is 0430 and I’ve got to get up early for work :'(

Finally had free time to day to shave off that infernal beard.. After so many months it is kind of nice to be reminded I’m more human the Wookie haha.

Spent tonight doing some light but thoughtful work,

Terry@Dixie$ pwd; /bin/ls -1                                               9:26
/home/Terry/code/Python/src/neo/testing/alpha
dialogs.py
main.py
neo.py
options.py
translate.py
Terry@Dixie$ 9:29

I have arranged things from the testing directory to it’s own work dir to clean up. Not to mention to continue work in a more module oriented pattern then the monolith and scratch files I have been using to test the first mock up of the programs UI.

  • dialogs -> will hold code common to user interaction through ‘pop up’ windows.
  • main -> the begining of it all, will probably be wrapped in a shell script
  • neo -> the GUI’s mock up code for the main window, need to split stuff off into sub modules when more work is done.
  • options -> the option subsystem, currently part of a config parser; to be expanded greatly
  • translate -> quick references to tools we will need later (subject to changes)

There is also a .neorc INI file for the testing 🙂

My near-future to do list right now is, in no particular order:

experiment with improving the wrapping paper on pkg_info; regex might be an improvement to the current process what ever the possible speed changes are. It might also be beneficial to look into using a pipe in order to ease start up times.

expanding dialogs into some thing more complete, currently it only has a routine for an error message.

Figure out how we can create a suitable display area for the output of a shell command (such as ‘make’).

I think I could probably do double the amount of work per day on this, if I didn’t have to do it before bed… Which has the bad limit of only being able to do things until my mind gets to the point that I just have got to sleep or the clock passes a point where I need to hit the hay, in order to get enough sleep for work in the morning.

I am lucky though, I’ve had almost 3 day’s off and only light duties tomorrow.

searchlet test level 7

Almost finished my work on npm from early this afternoon.

Took care of getting the mock up to actually search the ports tree for a given string, considering incorporating a simplified version of psearch but I can worry about that later.

Did most of the right view of installed software today, it still has a problem but this is good progress:

Click to enlarge

Some of the listings are screwed up and trigger a TypeError, I know I’m doing some thing wrong some where but not sure where yet…. At the moment I am about ready to hit the hay.

Some future tasks include fixing that, adding a toolbar to the mockup, a place holder (or working) msg display area. And moving on to other mock up’s.

So far, it is not pretty but the mock up is doing quite well. Hopefully fiddling with layout management and spacers will not be to much heartache under QT3 without having to use the Designer tool. Hmm, you know it might even be possible to make things a bit more flexibble rather then choosing one of the mock ups as the basise of the final forms overall appearance.

Any way one slices it though, it is time for BED.