Qute I/O

Been meandering about with Pythons subprocess and various IPC (inter Process Communication) focuses modules tonight. When I remembered I had noted that QT and KDE had classes for working with processes.

Looked up the class in the documentation and played with it. Then I found a small example program that they included. Here is the QT written example using C++, the primary language.


/****************************************************************************
** $Id: qt/process.cpp 3.3.7 edited Aug 31 2005 $
**
** Copyright (C) 1992-2005 Trolltech AS. All rights reserved.
**
** This file is part of an example program for Qt. This example
** program may be used, distributed and modified without limitation.
**
*****************************************************************************/

#include <qobject.h>
#include <qprocess.h>
#include <qvbox.h>
#include <qtextview.h>
#include <qpushbutton.h>
#include <qapplication.h>
#include <qmessagebox.h>

#include <stdlib.h>

class UicManager : public QVBox
{
Q_OBJECT

public:
UicManager();
~UicManager() {}

public slots:
void readFromStdout();
void scrollToTop();

private:
QProcess *proc;
QTextView *output;
QPushButton *quitButton;
};

UicManager::UicManager()
{
// Layout
output = new QTextView( this );
quitButton = new QPushButton( tr("Quit"), this );
connect( quitButton, SIGNAL(clicked()),
qApp, SLOT(quit()) );
resize( 500, 500 );

// QProcess related code
proc = new QProcess( this );

// Set up the command and arguments.
// On the command line you would do:
// uic -tr i18n "small_dialog.ui"
proc->addArgument( "uic" );
proc->addArgument( "-tr" );
proc->addArgument( "i18n" );
proc->addArgument( "small_dialog.ui" );

connect( proc, SIGNAL(readyReadStdout()),
this, SLOT(readFromStdout()) );
connect( proc, SIGNAL(processExited()),
this, SLOT(scrollToTop()) );

if ( !proc->start() ) {
// error handling
QMessageBox::critical( 0,
tr("Fatal error"),
tr("Could not start the uic command."),
tr("Quit") );
exit( -1 );
}
}

void UicManager::readFromStdout()
{
// Read and process the data.
// Bear in mind that the data might be output in chunks.
output->append( proc->readStdout() );
}

void UicManager::scrollToTop()
{
output->setContentsPos( 0, 0 );
}

int main( int argc, char **argv )
{
QApplication a( argc, argv );
UicManager manager;
a.setMainWidget( &manager );
manager.show();
return a.exec();
}

#include "process.moc"

Needless to say, I am not really a fan of C++ at times ^_^. I translated the example into Python for a simple test and I like it quite a bit.

#!/usr/local/bin/python

# I translated the example: process/process.cpp to python and boy do I love qt!
#
#############################################################################
##
## Copyright (C) 1992-2005 Trolltech AS. All rights reserved.
##
## This file is part of an example program for Qt. This example
## program may be used, distributed and modified without limitation.
##
##############################################################################

import sys
from qt import *

class UicManager(QVBox):
def __init__(self):
QVBox.__init__(self)
self.output = QTextView(self)
self.quitButton = QPushButton(self.tr('Quit'), self)
self.connect(self.quitButton, SIGNAL('clicked()'), qApp,
SLOT('quit()'))
self.resize(500,500)

# QProcess related code
self.proc = QProcess(self)

self.proc.addArgument("uic")
self.proc.addArgument("form1.ui")

self.connect(self.proc, SIGNAL('readyReadStdout()'),
self.readFromStdout)
self.connect(self.proc, SIGNAL('processExited()'),
self.scrollToTop)

if not self.proc.start():
sys.stderr.write('fatal error could not start uicn')
sys.exit(1)

def readFromStdout(self):
'read in proc data which may be in chucks'
self.output.append(str(self.proc.readStdout()))

def scrollToTop(self):
self.output.setContentsPos(0,0)

if __name__ == "__main__":
a = QApplication(sys.argv)
manager = UicManager()
a.setMainWidget(manager)
manager.show()
sys.exit(a.exec_loop())

QProcess is nice but I’m not sure about using enormous amounts of data though yet.

EDIT:

As a test I set up the Python script to pass a ‘ls -R /’ basically causing it to list all files on all mounted file systems. The code to scroll back to the top caused major lock up to the program and some performance loss for the entire laptop: probably screaming at being down to about a 150MB of swap space left.

Removing that and closing programs so only konsole running it and top + amarok playing music was running. At times top reported free memory over < 1MB about 30-40% swap usage. However without the scroll to top thing gone it didn't cause any real performance for the laptop; probably because there was much less swap usage used up. As a test I ran ls -R / direct afterwards and while it did pile on the stats in top, it still kept to about 50MB free and even less swap usage on the displays.
So far I think it would probably work for whatever is needed as long as the computer continues to provide adequate memory and the kernel doesn’t have a hissy fit about the pipes and things in the background.

ZzzzZzz

Mondays are always tiring… thanks to the holidays that jobs now on Tuesdays until Janurary.. So I’m about ready to pass out.

Been playing with the prototype for configuring a port. Made some changes including actually writing out the options to a file. I also got to poke around at Pythons ways of handling file locking, nice to see that fcntl, flock, and lockf work more or less the same as in C.

At least work tomorrow work should be light and I might actually get home in time to rest.. The down side to having to reserve most of my time spent working on this to after dark, is I can only go about as long as it takes for my brain to crash.. And that is a bit constraining at times. Between a full days work and work the next morning.. Can’t wait for the weekends!

Tomorrow I need to finish work on the prototype and then start integrating it with the rest of the more complete code the day after. In between I’d like to experiment with a few things in QT, migh tbe a little easier to do the tests in C++ but that wouldn’t be totally helpful… hehe.

Spent about 2 hours tonight, mostly reading make files. I think I’ve read all of the bsd.port* makefiles in /usr/share/mk and /usr/ports/Mk. Along with most of /usr/ports/Tools/* and a ton of stuff in /usr/share/mk/. While there is a lot of leg-work there as far as ports goes it doesn’t seem to be as complicated as I would have thought.

Made some changes to the prototype for the dialog to handle our way to ‘make config’ but it’s to late to work on it further on it. I need to set it up to store what options have been checked / unchecked so they can be used to get things set up to run make.

if I didn’t have the small problem of getting up for work tomorrow I’d have that done tonight…

So far, work on this program has been fairly easy, the main problems I am anticipating is sorting out the final-form of the main GUI. I’ve been using the 2nd style that I posted awhile back, for the prototype but I can still replace it with another quite easy once the rest of the work is done.

I may have 0 formal education when it comes to Computers or Programming. But I am accustomed to doing things the way I am doing them now. Namely trying to implement things in as small and manageable sections as possible; where each section only handles as much as necessary and relies on as little as possible out-side of it’s related code. I like it when things can be tested as stand-alone as possible and then modified/rewritten as part of the larger program. You could say I’m not a fan of 10,000 line monolithic modules that do their job plus everything but recreate emacs lol.

I test the sections making sure that it does what I need it to. Then experiment with it to see how it reacts, and incorporate it into the rest of the project based on what was learned from that prototype. This is just the way I like to tackle the problem of writing a program.

I’m more used to dealing with C and a mess of .c/.h files but python is quite nice. And it has allowed me to spend most of my time concentrating on building a working program, solving the problems not tinkering around with GDB or malloc() more then thinking about how to best make stuff work.

I hate debuggers… But some times they are necessary. My favorite method of coping with some thing that doesn’t work as anticipated is go back to the code and apply brain power. Why doesn’t it work and why does it do what it does instead? Although I must admit GDB is invaluable when needed. But I’ve rarely needed to use a debugger outside of C and C++, and even then usually to track down a stray pointer or get a backtrace.

A short look

Well, almost missed my narrow window of code time… Was passed out on the couch with a full stomach before bed until nearly 0100 xD

Between My mom, my sister, the bird, and my nephew (a louder bird)… And helping with baking there is really not much I can do during the day.. Any attempt at even thinking about seriously trying to read or write goes out the window and in comes a headache. So I have to work at night, when every one else is alseep… Until I crash or the clock reaches a point where I need to go to sleep in order to be @ work on time.

I completed the mock up of a simple dialog for simulating make config. And I’ve almost finished a working prototype for the module but I don’t have time to handle writing the slots and associated code to deal with the check boxes right now. And I need to get that done and tested before I can incorporate the prototype unto the module it belongs.

I posted a screen shot of the mock up awhile back. I’ve fixed the display a bit (the leading variable name from the makefile is removed). And added an ok and cancel button with the beginnings of a more key-board friendly behavior when it comes to using the keys instead of the rat to use it hehe. I’ve also set the caption on the dialog to contain the ports name as category/program.

In doing this, I figured the most simple way was to just muck around with a string of our target (e.g. /usr/ports/net/samba3 and /usr/ports/www/links/ in my tests). Although now that I think of it, the same routine I wrote to get a list of options to create check boxes for. Could be used to look up the category and portname variables from the makefile instead because it’s not tied to looking up any specific variable. — TODO: compare both methods for speed
after the prototype is ready to be moved to alpha group.

In trying to figure out how to get the desired result from a string version of our working path, I remembered that most languages offer some way to obtain the basename of a file or directory and set to look for it in Pythons os module. But doing that would mean we’d have to cut off the trailing slash and refeed the path to the library routine then join them into a new thing. Not exactly my idea of fun when writing a quick helper subroutine. I was very happy that help(os.path.split) revealed a method that takes care of most of that legwork itself.

Python also has some thing called list compression and which allows the mapping of the contents from one list into another based on given criteria. I’ve generally avoided list compressions out of disfavor for some of the (large) examples I’ve seen before. But for this, I actually found it yielded both quick to write and easy to understand code.

p = name # '/usr/ports/category/program' for example
prog = os.path.split(p)
cat = os.path.split(prog[0])

path = ["%s/%s" % (c,p) for c,p in zip(cat, prog) if c and p]
return path.pop()

The above snippet splits name into two lists of two elements, each a string. Basically the 2nd line returns a list of [‘/usr/ports/category’, ‘program’]. So obviously line 3 likewise splits /usr/ports/category into a list of two elements, the last of which is category. I really love how os.path.split() speeds up reading this, it takes only a glance to read it without having to double check it.

The 5th line maps any elements of the two lists (cat and prog) into a single a list of strings, i.e. some thing like path => [ ‘/usr/ports/category’, ‘category/program’ ]. The test for each element being true could be ommited in this case but I wanted to show the possibility.

Since path is a list and we want to display a string (without using str() on the result!), we just pop() the last element off path, which is exactly the element we want ‘category/program’

mm, now for some sleep… It’s after 5am here and I’m exhausted.

tinkering

click to enlarge
Free Image Hosting at www.ImageShack.us

Last night I was trying to figure out a little bit about trying to display a suitable array of checkboxes for letting the user select the ports options. I checked out the QWidget class, looks like a nice time saver but a bit of over kill for what I need. I’m also sorry to see it’s deprecated in QT4, for the heck of it I started trying to write the examples in the QT4 doc’s for replacing it, in PyQT but I’ve got QT3 C++/Python bindings on my laptop and the same for QT4 on my Desktop so I didn’t have time to finish it completly. I find it rather werid using C++ doc’s for Python but I am starting to get pretty used to working with Python although I think Ruby and Perl handle regular expressions better along side strings. Not sure if QT3 has a Regex Engine le talong if PyQT has it if there is one =/. It is good to know how to use many different languages though.

For right now I’ve been tinkering with getting the checkboxes laid out and displayed, like in the screen shot of my mock up above. It is not done yet but once the mock up is finished I can set to work on doing more; most interestingly integrating it with the more robust code in the alpha directory (yes I’m to lazy to set up CVS for working from one machine).

In the end I’m likely to use QT Designer to take care of it, so it can be made to look nice. And after feeding it through pyuic use that and the mock up to create some thing to work in a more complete version. For right now though I need to get a little rest, it has been a very busy day. I would like to rest a little bit and finish the movie. Then get back to working on this stuff when I’m more relaxed but I am liable to fall asleep… I’m off until like Tuesday so I suppose it doesn’t really matter when I wake up… And a quick run of xgalaga is a good mental re focuser hehe

*Yawn*

wrapping make config

I was trying to work on creating a simple pop up window that would list of a ports configuration options, same as the dialog you get when doing a make config on a port with options. But for the life of me, I couldn’t figure out how on earth to nab it because of it being a multi line variable.

With the bird screaming my head off and Ma trying to bake (including calling for help). I gave up on even trying.

Tonight I sat down and tried to *concentrate*, it sucks that I’ve got to wait till everyone else is asleep to ensure I can work with my brain on my thoughts… But the anwser to the problem it me quite quickly once I started tonight.

Don’t think about it as grabbing the OPTIONS out of th Makefile, think of it as if I was the make program!

So obviously if I was make(1) I would want to look through the Makefile in order to find all variables declared. Storing them in a simple data structure and relying on the @ EOL continuation syntax that is used — So I would be getting the ‘whole’ thing when storing a variable for later reference.

Stepping back to the problem at hand, I made it look exclusively for Options by wrapping it in a regex check for the specific variable rather then any. I’ve converted the routine to be able to hunt for an indicated rather then specific variable so it is not so closely tied to the problem it is actually solving for me.

Next up, figuring out how QT handles check boxes and adusting the routine for a ‘nicer’ way to use it.

When I can just sit and work/read, I can in-hale a problem/book but when I’ve got to do things so late at night, I am limited to how long I can go until my brain just conks out of logical thinking… Do to the requirement for sleep.

Night module

Unfortunitly it is 0430 and I’ve got to get up early for work :'(

Finally had free time to day to shave off that infernal beard.. After so many months it is kind of nice to be reminded I’m more human the Wookie haha.

Spent tonight doing some light but thoughtful work,

Terry@Dixie$ pwd; /bin/ls -1                                               9:26
/home/Terry/code/Python/src/neo/testing/alpha
dialogs.py
main.py
neo.py
options.py
translate.py
Terry@Dixie$ 9:29

I have arranged things from the testing directory to it’s own work dir to clean up. Not to mention to continue work in a more module oriented pattern then the monolith and scratch files I have been using to test the first mock up of the programs UI.

  • dialogs -> will hold code common to user interaction through ‘pop up’ windows.
  • main -> the begining of it all, will probably be wrapped in a shell script
  • neo -> the GUI’s mock up code for the main window, need to split stuff off into sub modules when more work is done.
  • options -> the option subsystem, currently part of a config parser; to be expanded greatly
  • translate -> quick references to tools we will need later (subject to changes)

There is also a .neorc INI file for the testing 🙂

My near-future to do list right now is, in no particular order:

experiment with improving the wrapping paper on pkg_info; regex might be an improvement to the current process what ever the possible speed changes are. It might also be beneficial to look into using a pipe in order to ease start up times.

expanding dialogs into some thing more complete, currently it only has a routine for an error message.

Figure out how we can create a suitable display area for the output of a shell command (such as ‘make’).

I think I could probably do double the amount of work per day on this, if I didn’t have to do it before bed… Which has the bad limit of only being able to do things until my mind gets to the point that I just have got to sleep or the clock passes a point where I need to hit the hay, in order to get enough sleep for work in the morning.

I am lucky though, I’ve had almost 3 day’s off and only light duties tomorrow.

searchlet test level 7

Almost finished my work on npm from early this afternoon.

Took care of getting the mock up to actually search the ports tree for a given string, considering incorporating a simplified version of psearch but I can worry about that later.

Did most of the right view of installed software today, it still has a problem but this is good progress:

Click to enlarge

Some of the listings are screwed up and trigger a TypeError, I know I’m doing some thing wrong some where but not sure where yet…. At the moment I am about ready to hit the hay.

Some future tasks include fixing that, adding a toolbar to the mockup, a place holder (or working) msg display area. And moving on to other mock up’s.

So far, it is not pretty but the mock up is doing quite well. Hopefully fiddling with layout management and spacers will not be to much heartache under QT3 without having to use the Designer tool. Hmm, you know it might even be possible to make things a bit more flexibble rather then choosing one of the mock ups as the basise of the final forms overall appearance.

Any way one slices it though, it is time for BED.

GET ME THE **** OUT OF HERE !

I’m sitting here across the building…

Trying to build a working mockup of my main widget that’ll handle searching ports for the entered string through the GUI and volia !

Family is driving me out of my fsck’ing mind.

Just for once in my life, I would love it if I didn’t have to stay up to after 4 in the damn morning just to get stuff done because I can’t be left alone during the afternoon !!!!!

Every one else asleep == I can get stuff done

Any one else awake == 75% less chance of being able to hear my brains internal dialogue, let along get work done (much lower chances).

Thinking…

A few concept drawings of what I have in mind

Style #1: simple interface (emphasis on searching and ease of performing common actions)

Free Image Hosting at www.ImageShack.us

Style #2: orthodox interface (commander like with toolbar, I like this one xD)
Free Image Hosting at www.ImageShack.us

Style #3: balanced interface (I think it would be better with the left being a |installed software | build messages | tab-thingy).

Free Image Hosting at www.ImageShack.us

Personally I think the first style is easier to use if your not familiar with what your doing, can you get more simple then ‘search, click, click’ with a big set of buttons?

The second I really like because I think it makes best use of space and presents the most important information quickly.

The third, I think is probably a better choice since it combines #1 and #2’s ways of doing things into a fairly simple yet compact amount of space.

To be honest, I would kind of love docking / undocking elements and a very ‘cool’ look, like XFire or XMMS L&F with a Gimp or Designer style of operation but I think that would probably be a lot of work and a bit confusing to new people =/