SSMTP/Getmail how-to part III

back to part II

The getmail documentation said that was the best way to automate it, and its bloody better then buling up getmail with ‘daemon mode’ code. But also like the documentation said if we want to ‘stop’ this daemon like mail checking we need a script for cron to run getmail through and program it to not run getmail if a file is pressent. Now we can do this many ways, heck we could set a enviroment variable if we want.

I’ve written a little script that can be run with a users cron jobs and skip over all mail or only the given account. You need one rcfile per account and you can tweak it to follow any conventions you want. My RCFiles follow the convention of getmailrc-shortaccountname, hence getmailrc-bell and getmailrc-sas for my bellsouth and sasclan accounts. This script should work on any system that has a bourneshell compatible /bin/sh. Just edit the shebang if you need to run it as another shell (such as /bin/ksh).

#!/bin/sh

# Use getmail to check my E-Mail account using the RC variable
# This script has the advantage that one can save it as another file, change
# one variable and set a cron job to check each account at different times (1
# cron job per script). Then not only use a file in their .getmail folder to
# stop all the scripts from running getmail or use a file to stop one script
# from checking its account. It also keeps a log which will be trimmed by
# another script

# Name of the getmailrc file to use
RC=getmailrc-sas

# log to a file when mail is checked
LOGFILE=${HOME}/.getmail/cronjobs.log

#
# If a nomail or nocheck file is found in ~/.getmail/ exit without checking
# else look for a no file. Where is equal to every thing that
# comes after the getmailrc- in $RC. If none of these files exsist check mail
# using the $RC file.
#
if [ -e ${HOME}/.getmail/nomail ]
then
LOG=$(echo "Skipping mail for $RC")
exit 1
elif [ -e ${HOME}/.getmail/nocheck ]
then
LOG=$(echo "Skipping mail for $RC")
exit 1
else
DIE=$(ls ${HOME}/.getmail| grep $RC | cut -d '-' -f 2)
if [ -e ${HOME}/.getmail/no${DIE} ]
then
LOG=$(echo "You have desided not to check this mailbox - $DIE")
else
LOG=$(echo `date` "checked mailbox with $RC")
getmail -r$RC
fi
fi

# Update log with the result
echo $LOG >> $LOGFILE 2> /dev/null

if you want to use the script copy and paste it into a text file and mark it executible. I saved them as ~/.getmail/check-.sh and chmod’d them 0700

Ok, let us make a cron job, because this is allready a big long post that took me forever to write with the way my house is. I’m not detailing cron(8) so please read the handbook or read the fine manual.

I created this crontab file to run my scripts to check my accounts every 5 and 4 hours respecfully and to ‘trim’ my log file every week.

# rstf's crontab
SHELL=/bin/sh
PATH=/etc:/bin:/sbin:/usr/bin:/usr/sbin
HOME=/usr/home/rstf/
MAILTO=""
#
#minute hour mday month wday who command
#
# check sas every 5hr
5 * * * * rstf ${HOME}/.getmail/check-sas.sh
#
# check bell every 4hr
4 * * * * rstf ${HOME}/.getmail/check-bell.sh
#
# trim log @weekly
0 0 * * 0 rstf ${HOME}/sh/trim-getmail-cronlog
#

The trim-getmail-cronlog script is thus

#!/bin/sh

# Rotate my logfile for running get mail via cron

LOGFILE=${HOME}/.getmail/cronjobs.log
TMPDIR=/tmp/

if [ -e $LOGFILE ]
then
tail $LOGFILE > $TMPDIR/gmlog.$$
rm $LOGFILE
mv $TMPDIR/gmlog.$$ $LOGFILE
else
exit 1
fi

To load my crontab file into crons system i.e. /var/run/tabs/${USER} all I have to do is run a simple command.

crontab ~/rstf-contrab

SSMTP/Getmail how-to part II

Back to Part I

It should’ve taken like two minutes to install or so. The getmail program is very nice and it follows the concept of do one ting and do it well + allows for a great level of modularity. We can use it with a number of protocols and other softwares like procmail, spamasassion, clamav e.t.c.

Setting up getmail is very easy and theres great documentation so far I’ve been very happy with it. Lets go to our home directory, we do *not* want to be the root user for this.

I’m gong to use a console because thats how i like it, so I’ll short how to do it via that way. If you want to use a GUI app like Konqueror to do it be my guest, you should be able to easy enough.

Make a directory your home directory called .getmail and set the permissions so that only you have access.

mkdir -m 0700 ~/.getmail That ammounts to the owner having read, write, and execute but no one else but the root user being able to enter the directory. Lets cd over to our ~/.getmail folder and create a rc file. By default getmail reads the ${HOME}/.getmail/getmailrc file but we can create multiple rc files and have getmail use the one we choose.

getmail -rRCFILENAME

if its not in our .getmail/ folder we need to supply the path to the rc file, if its in .getmail we can skip it.

The syntax of the file reminds me alot of .ini files on Win32, to be perfectly honest the getmailrc file syntax is the easist I've seen. Heres a copy of one of the RC files I use complete with anotations of what the ooptions do. After this I'll go into more detail about the options to help you get a owrking rc file.

[retriever]
# This file is getmailrc-sas which is for checking my @sasclan.org account
type = SimplePOP3SSLRetriever
server = mail.host.tld
username = my_emailaddr@sasclan.org
password = My_Password

[destination]

# This destination is just for my e-mail not the systems local mboxes.
type = Maildir
path = ~/Mail/
user = rstf
filemode = 0600

[options]
# Note that '0' means disable for all integar values.

# (int) 0-warn/err only, 1-retriv/delete only, 2-every thing
verbose = 1
# (bool) true = fetch all, false = only what getmail has not seen before
read_all = true
# (bool) true = delete messages after downloading, will override delete_after
delete = true
# (int) delete messages from server after int days
#delete_after = 1
# (int) max server reported message *total* size to download per session
max_bytes_per_session = 0
# (int) do not retreve messsages larger then this number of bytes
max_message_size = 0
# (int) max number of messages to process
max_messages_per_session = 0
# (bool) adds Delivered-To: header field to the message.
delivered_to = true
# (bool) add received: header field to the message
received = true
# (str) log to file
#message_log
# (bool) use system logger
message_log_syslog = false

as a reminder so I wouldn't have to check the documentation in /usr/local/share/doc/getmail/ or online. I put comment notes in the file briefing discribing what each option does and the type of setting, namely bool (i.e. true/false), int(eger) i.e. 0 1 or 435 e.t.c., or str(ing) likethis. Basically you need to have a [retriever] and a [destination] section. Under retriever we tell getmail what type of protocol to use, taken from the documentation heres the options.

  • SimplePOP3Retriever
    — for single-user POP3 mail accounts.
  • BrokenUIDLPOP3Retriever
    — for broken POP3 servers that do not support the
    UIDL
    command, or which do not uniquely identify messages; this provides basic
    support for single-user POP3 mail accounts on such servers.
  • SimpleIMAPRetriever
    — for single-user IMAP mail accounts.
  • SimplePOP3SSLRetriever
    — same as SimplePOP3Retriever, but uses SSL encryption.
  • BrokenUIDLPOP3SSLRetriever
    — same as BrokenUIDLPOP3Retriever, but uses SSL encryption.
  • SimpleIMAPSSLRetriever
    — same as SimpleIMAPRetriever, but uses SSL encryption.
  • MultidropPOP3Retriever
    — for domain mailbox (multidrop) POP3 mail accounts.
  • MultidropPOP3SSLRetriever
    — same as MultidropPOP3Retriever, but uses SSL encryption.
  • MultidropSDPSRetriever
    — for domain mailbox
    SDPS mail accounts,
    as provided by the UK ISP Demon.
  • MultidropIMAPRetriever
    — for domain mailbox (multidrop) IMAP mail accounts.
  • MultidropIMAPSSLRetriever
    — same as MultidropIMAPRetriever, but uses SSL encryption.

Odds are if you don't know what you need, its probably SimplePOP3Retriever. If you've ever set up a mail client before you should know it, your ISP or webhost should be able to tell you as well. Next we gotta tell getmail what server to fetch mail off of with the server option. If your ISP is some thing like charter, its probably mail.charter.net. I don't have charter but all the mail servers I've seen have been mail.ispname.topleveldomain lol.

We need to set the username and password so the server knows its us and which mailbox we want. Other wise it will tell us to go 'eff off.

Now we need to tell getmail what to do with our mail once it checks the incoming mail server. This is what the destination section is for. You basically have two big options here, Maildir or MBox. I've always used mboxrd since thats what Mozilla Mail&Newsgroups, Mozilla Thunderbird, and Seamonkey Mail&Newsgroups used. Plus the systems local mailboxes are mbox format as well. Theres various 'variations' of mbox and probably maildir but compatible enough for our needs I'd say. Other options for 'type' allow us to use an

External Message Delivery Agent (MDA) like procmail.
Mutilple Destinations, using multiple maildir/mbox/externMDAs e.t.c.
Mutiple message sorting
Sort mitple messages into geussed destinations
And to use qmail-local to deliver messages as instructed in the .qmail file.

The exact specifics and how to set getmail to use these features are in the manual, go read it if you want to know more. I suggest ether mbox or maildir personally.

Maildir is pretty simple there is a folder containing new, cur, and tmp directories full of e-mails. If you plan on checking e-mail often or automating it (as I do) this is probably for you. Each e-mail gets its own file in one of those directories which I personally think makes it better suited sharing messages but bad for FAT32 file systems (i.e. many small files).

We can make a mail directory like this on the command line, assuming we want ~/Mail. Or just make four directories in a GUI file manager.

mkdir -p ~/Mail/{new,cur,tmp}

You'll need to set the path to the mail directory as well to use Maildir. You can also set the user and file permissions to use. You've got to use the unix octal format, i.e. 0755 instead of u=rwx g=rx o=rw or some thing.

[destination]
type = Maildir
path = ~/Mail/
user = rstf
filemode = 0600

Setting filemode to 0600 means only I and the root user have read-write permission to my mail.

Now if we want to use mboxrd we have to specify the type and path to the mbox file as well. The user option works here too. Also you need to consider the locktype option. It takes a string argument and you have a choice of 'lockf' which uses fcntl locking or 'flock' default as of this writing (getmail v4.7.0) is lockf.

[destination]
type = Mboxrd
path = ~/Mail/inbox
user = rstf
locktype = lockf

We can also set up filter sections to use stuff like clamav & spam assasion on our e-mail. Considering that the odds of a virus or trojan that can invect a windows machine through being ssh'd into a freebsd box that is using mutt to view mail fetched with getmail from a server that filters spam (optional /w my ISP) and scans for viruses (nice ISP). I don't blood ythink I need to filiter things through an Anti-Virus ! But if you like go read the documentation on how to set that up.

I think I'll be looking into spam assasion for my ISP account though so maybe I'll have some thing topost there.

The options section I don't think is required but I'd suggest you set your read_all and ether delete or delete_after options.

My suggestions

[options]
# fetch all mail on the server
read_all = true
# then delete it after its in our Maidir or Mboxrd destination(s)
delete = true

[options]
# fetch mail getmail has not seen before
read_all = false
# then delete old messages after 1 day
delete_after = 1

The bottom option deletes the messages you download today from your mail server (not your destination) the next time getmail checks for mail and sees that the old messages are '1' day old. Any integer number will do but not a floating point number. i.e. 4675 will work but 2.43 will not.

I have two getmail rc files one for each account

$ ls -R ~/.getmail                                             20:21
getmailrc-bell
getmailrc-sas
oldmail-mail.host.tld-110-username
oldmail-mail.host.tld-995-username

So I can run getmail and tell it which file to use so Ican deside which mailbox to check. I've made a pair of shell aliases in my shells rc file to save typing.

alias gm-bell='getmail -rgetmailrc-bell'
alias gm-sas='getmail -rgetmailrc-sas'

Ok, lets run getmail (I'll skip the alias), it will take a few seconds bu t if it takes a really long time you might want to make sure your system is configured correctly to resolve the hostnames.

rstf@Vectra$ getmail -rgetmailrc-sas                                                       20:22
getmail version 4.7.0
Copyright (C) 1998-2006 Charles Cazabon. Licensed under the GNU GPL version 2.
SimplePOP3SSLRetriever:My_EmailAddr@sasclan.org@OurMailServer:
0 messages retrieved, 0 skipped

Looks like I have no new mail in the account.

Ok, lets try some automation we can set cron jobs to run getmail -rRCFILE when ever we want on one or all of our files.

SSMTP/Getmail how-to part I

This is a short how to for kicking sendmail in the buttocks and setting up a micro-replacement for send-only usage on a FreeBSD (post 5.x) system. Plus using the getmail utility to check for new messages. I spent long enough screwing with it after I miss-read some documentation (fbsd handbook/ssmtp).

The goal is to be able to use utilties such as mailx, mutt, and other MUA’s dependant on sendmail (or exteneral) MTA(s) and be able to check mail easy and automaticly. I assume you at least know how to setup a mail client such as Mozilla Thunderbird or Outlook Express (or can get the relivent info) and know how to edit files as root when needed.

First we have to kill sendmail. For this we need to edit the /etc/rc.conf file, since I’ve cleaned mine up to place various options in related ‘sections’ heres the relivent one from my rf.cofing. Note all the sendmail_* options at the bottom.

#########################################################################
# SERVICES #
#########################################################################


#cupsd_enable="YES"
#background_cupsd="YES"

samba_enable="YES"

ntpdate_enable="NO"
ntpd_enable="YES"
ntpd_falgs="-c /etc/ntp.conf -f /var/db/ntpd.drift"

# Completly kill sendmail
sendmail_enable="NONE"
sendmail_submit_enable="NO"
sendmail_outbound_enable="NO"
sendmail_msp_queue_enable="NO"

Save rf.config with those sendmail lines in it and we can stop sendmail from working. Sendmails probably the worlds most used Mail Transfer Agent but for a desktop we don’t really need it. While its well known for its history of security problems according to some OpenBSD people, at least they still patch it. Now with sendmail gone we have two problems.

  1. Daily run messages are usually mailed to root and this might break that a tad.
  2. MUA’s dependant on external MTA or utils dependant on sendmail may require configuration changes or stop working

Now to fix this we want some thing small that will allow us to send e-mail through some thing sendmail compatible’ish. I’ll assume that you have ports installed or know how – so go do it and update them. Personally I try to update my ports tree every few days or just before building a port if I rarly install stuff on the machine. I also prefer cvsup/csup 🙂

Ok, now that you have ports ready to go lets install some software. Open a shell (or use a GUI) to cd into /usr/ports/mail/sstmp/ so we can build it. Run the commands

make install replace clean

it shouldn’t take long to install so no need to go on coffee break 😛
In case your wondering what ssmtp is heres the pkg-descr:

A secure, effective and simple way of getting mail off a system to your
mail hub. It contains no suid-binaries or other dangerous things – no mail
spool to poke around in, and no daemons running in the background. Mail is
simply forwarded to the configured mailhost. Extremely easy configuration.

WARNING: the above is all it does; it does not receive mail, expand aliases
or manage a queue. That belongs on a mail hub with a system administrator.

WWW: http://packages.debian.org/testing/mail/ssmtp.html

Basically it doesn’t handle fetching or checking mail or reading it but it lets you send it. While some thing like sendmail, qmail, or postfix should at least provide send/recieve if you want to go bugger setting up a full blown mail server be my guest 🙂

sSMTP is really s send-only app that tries to emulate sendmail, most of the command line options to sendmail are accepted but many are just ignored. Some MUA’s can check mail for us, mutt springs to mind but I rather like using getmail so far. At first I thought I would use sendmail/fetchmail/ but I saw an app called ‘getmail’ in ports and checked it out. Its not as buggy as fetchmail is *said* to be, its got great documentation, and its really simple. So if your going to use getmail once ssmtp is done building start on /usr/ports/mail/getmail/ while we open another terminal to configure ssmtp while getmail compiles.

Using a shell or file manager like konqueror cd over to /usr/local/etc/ssmtp/ . We can ether use the .sample files here or create new ones from scratch. We to create a ssmtp.conf file that tells ssmtp how to send e-mail to our out going mail server. The config file syntax is a cake walk, its key=value and # for comments to end of line like many a language or file has.

# Your E-Mail _address_
root=email_username@isp.net
# Your outgoing mail server, normally the TLD is .net, .com, or .org
mailhub=mail.isp.topleveldomain
# Where mail looks like it came from, just stick in your e-mail address.
rewriteDomain=email_username@isp.net
# Your hostname, your e-mail address should be fine if your not a mail server
hostname=email_username@isp.net
# Set this to never rewrite the "From:" line (unless not given) and to
# use that address in the "from line" of the envelope.
FromLineOverride=YES

save as /usr/local/etc/ssmtp/ssmtp.config and that takes care of that. You might want to nuke sendmail or reboot of course for this to take effect. Now since we do want our daily run stuff to still work we will set up an alias so mail sent to our user account locally will actually be sent to our e-mail address. We’ll do this with the revaliases file in the same directory.

This ones a little bit og a bug but its not once you figure out how. Basically the syntax is like this

your_login_name_on_this_computer:your_email_address_to_send_to:the_out_going_mailhub_to_use

# Alias e-mail to the system root account to go to my private e-mail
root:myemailaddr@myisp.tld:mail.myisp.tld
# Alias my_username to my email address so I get my cron-job reports
my_username:myemailaddr@myisp.tld:mail.myisp.tld

This way the daily run and cron e-mail messages will still work and they will be in my e-mail rather then /var/mail/${USER} where I often forget to check 8=)

I’ve yet to figure out how to set up ssmtp for multiple outgoing mail servers but I’ll figure it out later & post it.

Ok, now lets get back to getmail

PC-BSD Backup-Howto

/****************************************************************************
* NOTICE *
****************************************************************************
* This is still under construction. Having gotton very tired before getting*
* this far. Makes this less then optimal. Also since I have little in the *
* way of practical exp. It will probably show. I've never had need or using*
* any of my backups. *
* *
* I did enjoy learning a few new things like basic cpio usage, well. *
* I've read about using tar/cpio/afio/dump before. But I've only really *
* used bsdtar with / without gzip/bzip2 for the most part. *
* I will get stuff cleaned up as soon as I can. Feedback/comments always *
* Welcome. *
* *
* Chow *
***************************************************************************/

The most important policy a computer owner ever works on. Is his/her backup
plan, what happens if that hard drive blows up or you erase all your mp3s ?

Since PC-BSD is a desktop based OS, I’ll think more so from a desktop users
perspective then a server. I also don’t know of many people that have ‘tape’
drives at home so I’m not saying any thing about what I don’t own.

Why should I back up my data ?

0.) In case of hard ware failure, namely the hard drive

1.) In case of disaster. Such as fire/flood/electric surge

2.) In case of accidental deletion, oops I just shredded my tax returns the
night before tax day !

3.) In case you have to reinstall the Operating System and want your data.

4.) Just in case you install another Operating System and it over writes some
data.

All good reasons to have a plan, further more its generally assumed that before
you go installing OSes, partitioning drives, e.t.c. that you got back ups.

Ok, so I need backups but where to put them?

Good question, depends on the type. I generally use a mixture of drives and
disks. You should keep them in a safe place, probably not in the same area as
the computer. In case of a fire or burst pipe. You might loose the computer but
not the backups on a shelf down the hall. A fireproof safe would be nice if you
got one.

Remember, while it can be secure on your computer behind passwords and access
controls. Once your data is backed up there is no protection. Although you can
backup encrypted user files with luck and decrypt them after restore. Such as
ones Documents. If your backups contain sensitive information be _sure_ to keep
them in a secure location. It can be just as easy for a smart thief to restore
a backup of yours on his/her computer as it is for you. If given access to the
backup media.

When to backup ?

The question is how much could you afford to loose? If you work on files
often such as a writer, programmer, or an artist. You might want to backup
daily. If your a casual user more worried about bookmarks and mailboxes. Ever
two weeks to two months might do it. Really your the only one who would know.

If you want to automate the task we will be looking at tools meant for that
purpose.

What to backup ?

This is all a matter of personal preference. The most important things for the
desktop user are files. Namely one would rather save a few 1000 family photos
then installed software that can be reinstalled. My suggestion is at the bare
_minimal_ that you backup your home directory and any major file storage
folders or collections.

System configuration can also be important. Maybe you had to make some changes
to get your system working right or theres just some stuff you don’t want to
tinker with again. Hopefully you did write down any changes you had to make,
like wireless configuration details e.t.c. But a backup never hurts ! I’d
suggest you backup the /etc directory, the loader.conf file and modules
directory in /boot/ if you have drivers there, as well relevant directories
under /usr/local/ if your software requires it (like global configuration
files). It might also be of interest to grab the /var directory and system logs.

At best

/boot/loader.conf
/boot/modules
/etc
/usr/home/

What to backup to!

Any proper safe location. If you do use a tape drive, I suggest you take _good_
care of the thing and don’t use it very often. If you don’t know what tape
drives are don’t worry, I’ve never used one ether.

The best storage medium you can get would be CD-R, DVD-R, CD-RW, DVD+R, or Hard
Drive. You will need a burner to write files to a disk but heres a run down.

CD-R, needs a CD Burner. Holds about 650MB of data and files burned to it are
stored for as long as the disk lasts.

CD-RW, needs a CD Burner, works just like a CD-R but you can erase files from
itand reuse the disk. Costs a little more but can be well worth it.

DVD-R / DVD+R, needs a DVD burner but tends to hold much more data, a bit over
4.3GB or over 4,400 MB. Most burners to day support both -R and +R formats, I’d
suggest DVD-R but what ever your systems support.

There are also DVD-RW and Dual Layed DVD media that your drives may support. A
DVD Burner will also burn CD-R and CD-RW 🙂

Another good medium is the external USB hard drive or large capacity USB
sticks. These are good in that they can be used just like hard drive space, but
taken else wheres for storage. If you often use a USB stick for file shares,
you should back it up as well in case of loss or damage.

You can also use hard drives, if you have several computers a reliable one can
be tasked with a high capacity hard drive for the purpose.

My greatest fear is generally for hardware failure rather then accidental
deletion, I’m kinda careful. So I generally make use of hard drives then
optical media for storing long-term backups.

How to backup ?

Interesting question, lets think a few. We have a few options we can

Archive the whole system and stick it some where safe.

Take a snapshot of the file system, good if your prone to erasing stuff

Copies of the disks, poor mans RAID.

If for some reason you are working with a Database, stop the thing before you
back it up. Or at least check the documentation on it before backing it up.
Trust me you don’t want to break that local copy of a website your developing
do you ? Hehehe. Our major software will be dump, cpio, and a few
archive/compression systems.

Dump, the real way to backup a toilet bowl !

Sorry just had to say it. Dump writes raw data blocks that make up
files/folders I’d suggest reading the man page on dump(8) and restore(8) if you
have need of it use it. If you know how to, I’d really say use it. The manual
pages are very good but feel tape-drive centric. Perhaps some one with more
experience with dump/restore can provide better info.

Here is a
good article

If you don’t understand a lick of that or the manual page, do not bother with
dump unless you have to. Trust me, you’ll only get a headache. Learning the
details of dump/restore is also on my to do list.

That being said, dump and restore are probably the _best_ thing you have in the
arsenal of backup and restore tools.

Tar and Friends, the zippy way.

This is probably my most used method and it should prove usable for most
desktop systems. The basic idea behind tar is you take many files and create
one. For example tar can take the files tom, dick, and harry. And turn them
into the file ‘names.tar’, when unpacked you get your tom, dick, and harry
files back. The problem here is tar does not compress data to save space. For
this if we are not working with just a few small files. We probably want to
archive it with tar, then compress it using a program like gzip or bzip2.

PC-BSD uses BSDTar but includes GNUTar as tar and gtar respectively. See the
man page tar(1) and the info file on tar (GNU info page). If your used to
advanced tar usage under GNU/Linux you might find gtar more to your liking.
For basic usage the differences are not worth noting.

We can create a tar archive like so in the konsole

tar -cf files_to_add tarfilename.tar

If we want to be quick about compressing this file we can do it in one line and
use ether gzip (tar.gz) or bzip2 (tar.bz2) like so:

tar -czf filename.tar.gz files_to_add
tar -cjf filename.tar.bz2 files_to_add

GZip is faster but BZip2 may offer smaller files but use more resources when
dealing with bigger files. One can also backup over the network using ssh or
rsh if necessary. For simple usage like this we can use the graphical program
‘ark’ but it may take longer to process the operation (imho). To extract the
files we can open it in ark and extract it or simply run

tar -xzf filename.tar.gz
tar -xjf filename.tar.bz2

A quick and dirty way to back up ones home directory would be to run ark as the
root user from the run dialog or konqueror in super user mode. Add the folder
/usr/home/ to a new archive of given format or to create it like so from
a root konsole.

cd /usr/home
tar -cpzf Terry home-Terry.tar.gz

That is, create a tar archive of my home directory, preserve my file
permissions, and gzip it from the file ‘Terry’ in the current directory.

A far bit of warning, the manual on tar will make your eyes bleed so its
probably best to look for examples or use ark until your used to the command.

See Pee Eye Oh, I see you.

The program cpio or ‘copy input / output’ is the backup tool I recommend along
with pax. Its usage is more complex then tar but more useful. The version
included with PC-BSD/FreeBSD is GNU cpio.

A simple way to create a file with cpio from a konsole is like this

find ./ -print | cpio -vo –format=crc > filename

Will make a ASCII cpio archive in the new portable SVR4 format with a checksum
of all files/folders in the current directory. To restore it to the current
directory.

cpio -vidm < filename The v option means be verbose, that is show us what its doing. The -o option is create, -i is extract. -d is make directories as needed, -m is preserve modification times, and --format= is the archive format. Thumb
around the manual page for more details. Sadly this lacks compression 🙁 The
sysutils/afio port is an updated cpio program with more options including built
in compression.

Any one seen K-PAX?

Since the many versions of tar and pax in use by commercial UNIX and Free Unix
likes such as HP-UP & Solaris on one side. And GNU and BSD on the other
userland side. Have various compatiblity issues from time to time. POSIX UNIX
standards people. Created their own program called ‘pax’

If your prone to ‘Distro’ hopping or having to deal with several different Unix
systems you probably would prefer pax to cpio. Dru Lavigne has a great article
about it, so rather then write about it I suggest you read
hers
🙂

Heres a simple usage of it just for the same of this document.

pax -wf archivename /what_to_archive

cd /where/to/restore/to
pax -rf /to/archivename

Direct Device Access
dd(1) is a utility to convert and copy files. It has more use then it sounds
really. dd is a powerful tool and a dangerous one. If your not used to it,
please don’t make a typo trust me it could be bad. This program can do some raw
disk copy jobs and even has use in computer forensics to get byte exact copies
of drives. It is also some what hated in that it is used very diffrent from
other UNIX programs. Rather then -options you use a option=value method for
setting options. Here is a demostration of coping a mounted floppy disk in drive fd0.

dd if=/dev/fd0 of/tmp/floppy.image

Which very well may take awhile, we can add the obs= option to write
‘bytes’ at bytes at once and ibs= to read at ‘bytes’ at bytes per once.
Or we can just use the bs= option to set both the same. Thus to speed up
stuff.

We now have a copy of the sucker as I’ll show with the file and Disk Usage
programs.

file /tmp/fd.image
fd.image: x86 boot sector, code offset 0x3c, OEM-ID “BSD 4.4”, sectors/cluster 8, root entries 512, sectors 2880 (volumes <=32 MB) , sectors/FAT 2, serial number 0x742c15e2, unlabeled, FAT (12 bit) du -h /tmp/fd.image 1.4M /tmp/fd.image I only had 96K of data on the floppy but it is a byte for byte copy of the disk. So the image is a full 1.4 megs. Now I will copy it to another blank disk, raw. At approx 16 Kilobytes per. First unmount the disk and insert a blank floppy. dd if=/tmp/fd.image of=/dev/fd0 bs=16384 Now we can mount it and bingo its the mirror image of our old disk complete with the free space on it. mount_msdosfs /dev/fd0 /mnt/fd0/ We can even use rsh or ssh to do this over a network. Remember if you want to 'dd' an image of your hard drive. It will be the same size. For example if I used my systems ad0s1a partition as the In File (if=/dev/ad0s1a) and my ad0s1a partition is 7GB in size with 4GB free space and 3GB used space. The resulting image would be 7GB in size _not_ 3GB. So one must have space to store the image. You would probably be better off with dump but dd has its value. Since dd will probably need to be run as root for much of its usage as a backup tool. I suggest that you _read_ this section of Wikipedia on *WHAT NOT TO DO WITH DD* before using the command and erasing data instead of cloning it! dd
Anti-Examples

Once PC-BSD has released a GUI program for making backups things will get
simpler. In the mean time here are some ports you might research:

rchivers/mtf             A Unix reader for the Microsoft Tape Format used by NT Backup
archivers/rvm An archive manager that uses rsync to manage backups
databases/usogres Real-time backup utility for PostgreSQL
deskutils/multisync-backup Multisync backup plugin
emulators/vmsbackup Reads VMS BACKUP tapes
misc/afbackup AF's backup system
misc/afbackup-client AF's backup system
misc/afbackup-server AF's backup system
multimedia/streamdvd A fast tool to backup Video DVDs 'on the fly'
palm/pilot-link PalmPilot communications utilities (backup/restore/install/debug/...)
sysutils/afio Archiver & backup program w/ builtin compression
sysutils/bacula-client The network backup solution (client)
sysutils/bacula-client-devel The network backup solution (client) - DEVELOPMENT Version
sysutils/bacula-server The network backup solution (server)
sysutils/bacula-server-devel The network backup solution (server) - DEVELOPMENT Version
sysutils/be_agent VERITAS Backup Exec (tm) UNIX Agent
sysutils/bksh Backup-only shell
sysutils/boxbackup An open source, completely automatic on-line backup system for UNIX
sysutils/cdbkup Simple but full-featured backup/restore perl scripts (uses gnu tar)
sysutils/cpbk Backup Copy programm
sysutils/dar A full featured command-line backup tool, aimed for disks
sysutils/dirvish Network backup system based off of rsync
sysutils/duplicity Untrusted backup using rsync algorithm
sysutils/dvdbackup Backup content from DVD to hard disk
sysutils/flexbackup Perl-based flexible backup system that can use dump/afio/cpio/tar/star
sysutils/fsbackup File system backup and synchronization utility
sysutils/hdup The little, spiffy, backup tool
sysutils/kdar KDar is KDE-based backuptool using libdar
sysutils/pdumpfs A daily backup system similar to Plan9's dumpfs
sysutils/pdumpfs-clean A utility to clean up old backup files of a pdumpfs archive
sysutils/rdiff-backup Local/remote mirroring+incremental backup
sysutils/rdiff-backup-devel Local/remote mirroring+incremental backup
sysutils/reoback Simple but flexible ftp/nfs backup script

Some links that may be of interest

Stress
testing the backup utilities

FreeBSD Backup Strategies
FreeBSD Backup Basics

QEMU

How to install PC-BSD using the QEMU Emulator

What is QEMU?

QEMU is software that allows you to emulate another computer in order to
install and use an operating system on your physical machine without actually
installing it. Much like a Famcom emulator uses software to create the
impression to a game ROM that it’s really running on a Nintendo 8-Bit console.
It’s really running on your PC. It’s basically the same thing.

For Windows you can download QEMU as a ZIP archive and unpack it to any working
directory. If your using an older version of windows you may want to install a
program such as 7-Zip or WinZip (I reccomend the former). In order to use the
accelerator module you need to download it as well. You can find a link to it
on the main QEMU web site. Un pack the archive (it’s tared and bzipped, try
7Zip or WinRar). Copy the kqemu.inf and kqemnu.sys files to your QEMU
installation.

If you are using a Unix/Linux system it is best to use what ever package
management system you have to install it. One should be able to find an RPM or
Deb for there system. PC-BSD users can install QEMU and KQEMU via PBI which
includes a simple GUI for setting it up.

The KQEMU module will make the installed system faster but it is not open
source software. Qemu is Open Source and Free Software. Installing this will
likey vary from system to system.

Qemu Website
http://fabrice.bellard.free.fr/qemu/

Qemu for Windows
http://www.h7.dion.ne.jp/~qemu-win/

I will deal with using QEMU at the command line since it is the only portable
interface (Win, Lin, BSD, Mac). If your using a Unix/Linux system strip off the
“.exe” every where.

Find the location of QEMU on your system once it’s installed. Open a terminal
emulator (command prompt window) and change directory to it’s location like
this, cd “C:Program FilesQEMU” or if using a Uinx/Linux system you can check
it’s installed location with the command “which qemu” -> C Shell users may have
to “rehash” after installing qemu before they can run it.

Now since QEMU does not use your actual hard ware (it emulates it’s own) you
need to create a file that QEMU can use to store data (acts like a hard drive
or swap file). We can create a harddisk image file in several different
formats. “raw” which should be portable between emulation software. “qcow”
which is probably the best choice for Windows systems. “cow” for Linux systems,
“vmdk” which makes a VMWare image, and cloop which ya don’t want to toy with
right now !

Depending on the format used the size of the file should grow till it reaches
the limit we give it. Now PC-BSD should fit on a 6GB hard drive but to get any
real work done 15GB would be more realistic, assuming you like to store a lot
of photos, music, and video files. Then again for getting real work done you
should install an operating system not use an emulator (in my humble opinion).
It is good for testing things though but a few things to remember.

0.) Emulated hardware does not relate to YOUR hardware. So some things may work
or not work after really installing an operating system. Just the same some
things may not work in the Emulator but may work on the real system as the
emulator is simulating hardware.

1.) The system will run much slower then had you really installed it. Not only
is it going to use an alloted amount of memory (RAM) your computer will all
ready be running it’s current operating system and other softwares. This is not
suitable for the “faint hearted” machines. The test system I have runs a
Pentium D 930 (2 x 3.0Ghz) CPU and 2048MB (2GB) of fast DDR2 Memory.

3.) With virtual solutions some times odd things about software just “don’t”
work but are fixed when installed on the real system. Like wise it can be the
other way around.

4.) A lot of things won’t work under emulation or not well, i.e. 3D Games e.t.c.

Ok phase one we will create a disk image to install an operating system on. I
suggest 5GB minimal, for this I will use an 8GB image file. Thanks to having a
500GB hard drive xD

qemu -create -f qcow acd.img 8G

The syntax as you can see is `qemu-img -option -f format filename sizeG. Now we
have to basic options on how to install, we can boot the emulator off a CD/DVD
or an ISO image.It’s rather nice if you can use the ISO image and save your
self from burning a CD-R just for testing the system.

If using a Windows system

qemu -L . -cdrom “\.E:” -hda acd.img -m 512 -boot d

If using a FreeBSD system

qemu -cdrom /dev/acd0 -hda acd.img -m 512 -boot d

The device names vary from Unix to Unix some tiems even Linux Distro to Distro.
It will probably be some thing like /dev/scd0 on a Linux based system.

Now this is very important the number after the -m is how many megabytes of
memory to give the emulated “virtual” PC. In my case I chose to allow 512MB of
Memory to run PC-BSD under emulation.

Now if you want to use the ISO image file it’s slightly different.

Windows
qemu -L . -cdrom Imagefile.iso -hda acd0.img -m 512 -boot -d

Unix/Linux
qemu -cdrom Imagefile.iso -hda acd0.img -m 512 -boot -d

After the system is installed you can run it like so

Windows
qemu -L . -hda acd0.img -m 512

Unix/Linux
qemu -hda acd0.img -m 512

To try and enable KQEMU for more speed add a “-kernel-kqemu” option like so

qemu -hda acd0.img -m 512 -kernel-kqemu

The system should now boot and you can see how it works.

I’ve installed the system but it is very slow, taking about 2 times longer to install then normal and many minutes to boot + without sound or network. I don’t have KQEMU so it makes emulated hardware feel faster then our first Pentium PC (with 32MB of memory). If my laptop wasn’t so slow I might try it out that way. For running WinVista I’d say go with KQEMU on a system with 4GB of RAM and a lot 3.5 GB for it hehehehahhA !

Vi User How-To

Welcome to an introduction and usage study of one of the worlds most common text editors. It is also my favorite style of editor. We cover several sections each detailing a given aspect of the Vi editor.

Table of Contents

Short history
Which Vi is my Vi ?
Initial Fear
Learning Vi
Required Commands
Helpful Functions
A few extra commands
Making sense of Vi options
Deletion Wars
Cut, Copy, and Paste the Vi style !
Options and configuration
How to make a exrc file

Document Version: 0.90.0

This post has the underlying assumption that you want to learn to use the Vi text editor but know jack about it. I won’t insult you but I’ll try to teach it without being to terse or “hand holding”. We all need help but one has to put in a little bit of effort to master a program.

I have always cared more for content and understandable English then for closely following the “laws of the English anguage” so forgive any grammatical errors and ether curse or help improve the content 😛

Short History
In 1976 a man named Bill Joy wrote the Vi text editor for an early BSD release. It took the existing ed and ex line editors and created a visual interface. While ex was a simple line editor vi gave us an interface much more like what we are accustomed to in this era.

Vi is a modal text editor which means that depending on what editing mode one is in at any given time what each key does changes. Vi has thankfully only two editing modes of importance command and insert. Unlike common editors such as Emacs or Kate commands are entered by shifting the entire keyboard into command mode and back into insert mode to continue inserting text. This modal concept is probably the biggest adjustment new users have to make when learning Vi. Some notable influences on the Vi command set was the ADM3A terminal which had the Esc and left Control keys where the Tab and Capslock keys are on modern US_QWERTY keyboards. There was also arrows on the H, J, K, and L keys which intern became the Vi movement keys.

Picture of ADM3A Keyboard

Which Vi is my Vi ?
There have been many vi implementations and clones through the years and on many systems but you can always count on an Unix to have a Vi. By high decree of the POSIX (Portable Operating System Interface for uniX). It would be best that before you try to start learning how to use Vi you learn what type of Vi editor you have. Many GNU/Linux systems will use a Vi clone in place of Vi while BSD Unix’s use nvi. This how to should be relivent to every Vi/Vi Clone but most Vi Clones have a lot more to offer then Vi does.

Vi is the classic BSD
implementation now free and open source under a BSD style license.

Nvi is a re-implementation of Vi used for the 4BSD release when the original Vi was considered encumbered code. This is what is like wise used on Free, Net, Open, PC, and Desktop BSD and the focus of this how to.

Elvis is a Vi clone that adds a number of features and commands. Elvis functions on Unix, Dos, NT, and OS/2 systems both as a console and graphical application. It is standard issue on Slackware Linux for the systems vi program.

Vim or Vi IMproved is a Vi clone that implements many additional features and options. It’s known to run on many platforms including Amiga, GNU/Linux, Mac OS X, Windows, OpenVMS, OS/2, and Unix like systems. Vim supports several graphical user interfaces including Motif, GTK1, GTK2, and Athena versions. It is also what this post was written with.

Vile was initially derived from an early version of Microemacs in an attempt to bring the “modern” benefits of the Emacs multi-window/multi-buffer editing paradigm to users more comfortable with the vi command-set. Vile functions on DOS, Windows, OS/2, GNU/Linux, Mac OS X, and Unix like systems.

Viper-mode of the popular Emacs editor tries to emulate many common vi commands and may prove interesting to emacs users and humorous.

It does not matter what Editor one uses as long as one is happy, productive, and effective but I would recommend using a Vi Clone with good documentation. So know any good jokes about Vims 25,000,000,000 page manual ? Hehehe.

You may invoke the vi editor from a console window by typing “vi”, if your using another type of editor or “vi clone” please consult it’s documentation.

Initial Fear
The traditional vi runs in a command window which is enough to scare off many new users. While newer Vi Clones such as Vim and Elvis have graphical user interfaces that can run with such luxurious features as tool bars and menus it is important to know that the editors are still available in the command line environments without loss of functionality.

One of the most confusing things to scare you about learning Vi is the concept of editing modes. Most users are used to an editor that starts with and is always in “insert-mode” that is you type on the keyboard it and writes text to the screen. Vi by default starts in a command mode where the keys are used to issue editing commands such as delete, replace, yank (copy), paste, cut and movement keys and such. When one wants to start writing text they have to enter insert mode. Once you get sed to this idea you will start to grasp Vi much better. When you are working in insert mode Vi will behave in a similar manor to what most users expect a few differences aside (like the Esc key and copy/paste commands). To indirectly quote a man named Jon Lasser about vi editing modes.

When you move your (mouse) pointer out of the screen area, you can’t type in text. You have, in effect, moved from insert mode to command mode. “It’s the same thing,”

Most implementations of Vi including nvi should be able to make use of ones cursor (arrow) keys and the insert/home/pgup/del/home/pgdwn clusters we take for granted but using the arrow keys are less effective then using command mode & the classic movement keys. I’d say you should try to use the cursor keys for simple movement at first because they work in any mode and are less foreign. When some day you find yourself without working cursor keys you will be bloody
happy you learned to use H,J,K,L instead of LEFT, UP, DOWN, RIGHT. Trust me it can happen, I once had a very bad system boot the kind your usually lucky to be able to fix without a rescue disk kind. I had to edit a file and my choices of editors were rather limited (ed, ex, vi, ee). When I ran vi it could not use my cursor keys forcing me to use the H,J,K,L cluster in it’s place. Never before was I so glad to know two ways of doing the same thing in a text editor !

Learning Vi

I will split this section into several groups of commands. When I first tried to learn Vi(m) I didn’t like it because I could not figure it out at the time. After a while on the FreeBSD command line I learned how to use it. In the process I also learned the easiest way to learn how to use Vi, aside from using Vim instead (:help) was to learn things in small pieces. It won’t do you any good to try and learn Vi all at once but if your able to just sit down and inhale it I’ll probably edit in a cheat sheet or a link to one at the end of this how-to.

Required Commands

So named because these are what I feel are the fewest commands one needs to know in order to use Vi. Maybe not use it well but use it as well as Notepad is a good editor. While we are on the subject of Notepad. Just to say it, notepad is a good editor for several reasons:

It is on nearly any Windows system, it’s very simple and light weight.

You don’t need a Ph.D to use it.

It’s also very week and very basic.

Vi like many Vi users prefer Speed, Power, and Efficiency over a small inertial learning curve. Many people say Vi is not intuitive to learn but I must disagree it is only a matter of adopting ones mind to shifting between insert and command modes. The more one learns with Vi the more productive they will be but no one need learn every thing on day one !

These are the core super simple commands you have to know to use Vi like most editors.

Key         Action

h Move left one character - the left arrow does this as well
j Move down one line - the down arrow does the same thing
k Move up one line - the up arrow does this too!
l Move right one character - Right arrow yet again.
Esc Enter Command Mode /or cancel commands
i Enter insert mode - the Insert key will also work
x Delete the character under the cursor - just like the Delete key.
ZZ Save file and quit Vi

To tell the truth compared to using Notepad you only learned 3 commands. Esc, i, and ZZ or enter command mode, enter insert mode, and write file and get me out of this crazy editor!

The h, j, k, l, and x keys are analogous to the arrow keys and the insert and delete keys on your keyboard. The only difference is these keys work when nothing else does. Some terminals don’t support luxuries well such as home and end keys so when working on a machine from a remote session or laptop these can be your best friends. You should be able to live with having to press ESC to use commands and go to insert mode to input text.

Helpful Commands

These are more keys and commands that are not necessary to edit a file but do make live a lot easier especially if you often find your self editing text not just creating it. If you have some experience with some thing called regular expressions a few of these might even seem familiar to you.

Key         Action

: Enter "ex" like commands in a "command line" mode
:w Write changes to file
:q Quit Vi
:e file Open file for editing

/word Find "word" much like Control+F in many other editors
n Go to next occurrence of the search term below the current
N Go to next occurance of the search term above the current one

a Enter insert mode right of the cursor
A Enter insert mode at the end of line
o Insert line below cursor
O Insert line above cursor
w Move forward one word
b Retreat one word
$ Move cursor to the end of the line
^ Move the cursor to the start of the line
Control+u Page up half a screen
Control+d Page down half a screen

u Undo previous command(s) - some Vi clones have unlimited undo/redo
Control+r Redo the undone command(s) - same as above ^

r Replace one char and return to command mode
R Enter Replace mode - works like pressing insert in most editors
c Change, takes an operator like d. example: cw=changes a word
d Delete using a following movement command (see below)
dd Delete entire line

None of these commands are necessary to know but they sorta make life easier. Personally I find these much easier on the wrists then Emacs which uses key combinations using the Control and Meta keys instead of a command mode.

A few words about the “:” key, it enters a little command line mode. When you press the : key in command mode it gives you a command line to type an ex (extended editor) type of command. After you have typed the command you press enter to run it and should return to normal mode. If you need to force a write or quit you can append the “!” Symbol to it. If you are a ex guru or need to enter many commands in this way you might try the “Q” command to enter Ex mode. For example the ZZ command is the same as :wg. One can mix and match actions and movements very well in Vi, for example if one wanted to delete to the end of the line you could use command mode to enter d$ and delete from the cursor to the end of the line. When one shifts between command and insert modes often in there editing style you soon find the a, A, r, o, and O keys very useful. I’ve mentioned a few other modes then command and insert but they are of little importance to you unless your into reading good ocumentation. Consult your Vi implementation for details, for users of nvi it’s nothing to fart about at this stage of learning. Many commands allow one to specify a location or a movement. This is especially true for using the commands for deletion, joining text, copy/cut/paste, and multiple editing buffers (windows) if your vi supports it.

You might want to get used to using most of these commands or at least the concepts be fore we try to tackle such tasks as more advanced delete or copy/paste commands. A coffee break about now is a very good idea.

A few extra commands

These are other commands useful to users but I broke them off from the above list to ease session based learning.

Key         Action

:!cmd Execute shell command cmd.
:command! Force command to run, such as force write-quit (:wq!)
:r!cmd Run cmd in shell and insert it's output in the file

G Go to end of buffer (the bottom of the file)
1G Go to line one in the file
nG Go to line n. So 10G means go to line 10

>> Shiftwidth text over one indentation level to the right
<< Shiftwidth text over one indentation level to the left

One cane execute commands from vi that one would usually run in a shell such as bash or cmd.exe. :!dir b “C:Program Files” would be executed just as if one ran the dir command from cmd.exe or command.com. If you want to nab the output of a command into your file use :r!command this is very good if one needs to quickly grab some data thats a fast pipe away. The G command is a go to without any arguments it goes to the bottom of the file. If one was to type a number before it it would go to that line number. The >> and << are useful for shifting text around but have likely little need for users not into programming or scripting e.t.c. So I have nothing to say about these commands. Making sense of Vi optionsOne of the worst things is that by default vi generally does not tell you what mode you are in or where the cursor is this can make life harder. Lucky for us Vi has the ability to set options. Really this belongs in a customizing vi section and I’ll probably paste it there but I think knowing some of this now will help you. You can set options thus turning them on, off, or setting a value :set option :set nooption :set option=10 If you want to see all the options available you can type :set all if you have a good Vi clone you should be able to get some documentation on what all of these do, for the sake of making Vi more friendly we will cover some good stuff here. :set showmode :set ruler ;set ignorecase These three options (in order of appearance) will make vi show you the current mode at the bottom of the screen, show you the cursors position as line,character at the bottom of the screen, and turn off case sensitivity when searching in a file. You can make a startup file that vi will read when it starts that will set options for you. Ex/Vi and Nex/Nvi use the /etc/exrc and ~/.exrc files but depending on your implementation it may be different such as Vim or Elvis. I don’t have much experience with Elvis but for Vim it reads the .vimrc file(s). You can turn these off pretty easy like so :set noshowmode :set noruler :set noignorecase If you want to make a .exrc file for your self you can store commands in it that are what you would type after hitting : so a .exrc file enabling showmode would look like this: set showmode and saved as ~/.exrc if you want to leave a comment in the file preside it with a single ” quote. The ” Comment marks that entire line after the ” character not to be read when starting up. A few options that may interest you are autoindent, tabstop, shiftwidth, wrapmargin, and warplen. By default when one tries to type past the end of the screen it rolls to the line below on screen but is treated as one long line. One can try and make this more like most editors word wrapping with some play time. I will only discuss the autoindent option since it is some thing you will ether long for or hate if you use vi for editing code, scripts, web sites e.t.c. When one sets autoindent on vi starts a new line with the same indentation as the last. The word indentation is a little odd in my opinion if your not a programmer so I’ll give a short example of how it goes.

if i is-greater-then 0
do this code
and this code
else
do this instead
endif

If typing that with autoindent on pressing return after typing “do this code” would automatically indent the next line to the same level. You can press Control+D to kill the autoindent such as if you wanted to place the “else” in our pseudo code at the same level as the “if” and “endif”. Deletion WarsIf you remember awhile ago when we both were still sane. I mumbled about being able to combine a command and a movement. This is pretty much true with most movement, deletion, and “visual” commands (visual as in cut, copy, paste). x deletes the letter under the cursor but what if we want to delete some thing else? We can use the d key along with a movement specifier if you recall the movement keys h, j, k, l, $, ^ we can combine them all with d to make it mean delete this way. So d^ really means delete to-start-of-line. This makes for very versatile work and it’s not just limited to deletion. Much like we can use a number and G to go to a line like 275G to go to the 275th line of a file. Nvi users will especially need to know this for dealing with copy/paste as it lacks a Visual mode that makes it more like other editors. A very fast way to delete an entire line is the dd command or deadly delete 🙂 You can prefix a number of lines to a command as well, so for example 10dd will delete the current line and the next ten for you. Unlike most editors Vi does not really know you have a mouse, although Vi Clones should if they have a GUI. Some like Elvis and Vim have a “visual” mode that can highlight text using movement keys and then delete it or do the cut, copy, and paste antics with it.

Key         Action

y Yank (copy) - selects text in a similar manner to the d command
yy Yank line - selects text in a similar manner to the dd command
yyp Duplicate line - copy/paste current line on the line below.
p Put (paste) text
v Enter visual mode - for some Vi Clones

One should really know both ways if possible. Personally I prefer visual mode, it’s more like what we are all used to. That is you select (aka highlight) text then do with it what you want, such as copy or delete. With classic Vi as included in FreeBSD at least one will get an error. If one wanted to copy the current line and the next tree. You would type 3y in command mode. Go to where you want to paste it and press p key in command mode. This is more or less how it works in traditional Vi. This is how it can also be done in a Vi clone like Vim. You can enter visual mode (v key). Once you do that the editor starts treating the cursor movements just like when you old the left mouse button and start dragging the pointer around. Only it works without a mouse ! Once you’ve high lighted some text you can act on it.So for example if you pressed “vwwd” you would have just deleted from the current cursor position to the next two words. You can use the “c” command to “cut” in visual mode. I have not tried elvis much but it should be similar to vim. Search and Replace If you are familiar with regular expressions, sed, awk, ed, or ex you should be at home right here. The search & replace takes syntax like this ‘n,m s/find expression/change expression/cmd’. So if I wanted to change every occurrence of ‘vi’ to ’emacs’ in this file I could do this. :1,$s/vi/emacs rules/g That is from line 1 to end of lines search for the string vi and replace it with emacs globally. If I only wanted to change it for n through m lines I could have tried some thing like this :260,285s/emacs/vi/g Regular expressions are similar to the wild cards used by the shell but they are different. Same concept (imho) but different syntax. Much like C and C++ maybe. A detailed Analise of regular expressions would be out side the scope of this how-to and totally irregular 🙂 I suggest the following links for more info about regex (regular expressions) http://en.wikipedia.org/wiki/Regex http://www.regular-expressions.info/ Back in the days before operating systems when software ran on the bare hardware. Stuff like this “regular expression” mumbo-jumbo was a thing just for the uber-nerdy scientist but now it’s in the power of your editor. It should also be worth learning how to use regex. As it is a common syntax with many uses. The better you know regular expressions the better you will be able to use tools like sed, grep, awk, find, perl, ex, and many more. Options and Configuration We touched on this back in section 4.4.Making Sense of Vi options but will go over a few things again. Saves your scroll bars eh? 🙂 Vi has a number of options that adjust the way it behaves, here is screen dump of the default setup of nvi. As used on FreeBSD 6.1-Release.

~
+=+=+=+=+=+=+=+
noaltwerase noextended matchtime=7 report=5 term="xterm"
noautoindent filec="" mesg noruler noterse
autoprint flash nomodeline scroll=12 notildeop
noautowrite nogtagsmode noprint="" nosearchincr timeout
backup="" hardtabs=0 nonumber nosecure nottywerase
nobeautify noiclower nooctal shiftwidth=8 noverbose
cdpath=":" noignorecase open noshowmatch warn
cedit="" keytime=6 optimize noshowmode window=24
columns=85 noleftright path="" sidescroll=16 nowindowname
nocomment lines=25 print="" noslowopen wraplen=0
noedcompatible nolisp prompt nosourceany wrapmargin=0
escapetime=6 nolist noreadonly tabstop=8 wrapscan
noerrorbells lock noredraw taglength=0 nowriteany
noexrc magic remap tags="tags"
directory="/tmp/"
msgcat="/usr/share/vi/catalog/"
paragraphs="IPLPPPQPP LIpplpipbp"
recdir="/var/tmp/vi.recover"
sections="NHSHH HUnhsh"
shell="/usr/local/bin/ksh93"
shellmeta="~{[*?$`'""
Press any key to continue [: to enter more ex commands]:

You can get a screen like this by using the :set all command. Every option or “variable” can be turned off by appending ‘no’ to the command. So if we wanted to turn on autoindent we would :set autoindent We could also have used the short form and typed :set ai but that makes it harder to read. Why this matters I’ll tell you in a sec 😛 To turn it off we would then type this. :set noautoindent Some options take a numeric value and are set like this :set shiftwidth=4 I’ll try to quickly document each of these options the best I can ones of value. So you can choose what you prefer. I’d suggest if you want to modify Vi’s behavior to test a few options to see how you like the changes. If I have no information about the option, it is not listed. Option: autoindent Short form: ai Value: on/off Default: noautoindent Description: When turned on ‘ai’ will automatically start the new line to at the same depth as the previous line when breaking lines (i.e. pressing enter). The value of the ‘shiftwidth’ variable is used when inserting tabs. When your done you can undo the autoindent by pressed the control+d. I’ve yet to figure out if it is a bug or a configuration error but if tabstop and shiftwidth have different values. Control+d will not un-indent the line correctly if sw and ts are not equal. Line one line two Autoindented Cntrl+D Option: autoprint Short Form: ap Value: on/off Default: noautoprint Description: I don’t see any effects but my reference says ‘Display changes after each command.’ Option: autowrite Short Form: aw Value: on/off Default: off Description: Automatically write changes to file after certain events. Option beautify Short Form: bf Value: on/off Default: off Description: Ignore all control characters during input (except tab, newline, formfeed). Pressing control+letter in insert mode when no command exists will result in text like this by default. How to delete a tpyo�� No I’m not trying to insult emacs users, I would probably use M-B M-D to fix such a typo when using Evil Macs. Option: columns Shot Form: col Value: numeric Default: varies Description: The number of columns to use for the text area. Example, how to set vi to 85 columns wide. :set columns=85 Option: edcompatible Short Form: ed Value: on/off Default: off Description: Use ed-like features on substitute, probably more useful for Bill Joy then you or me. Option: errorbells Short Form: eb Value: on/off Default: on Description: Sound the system bell when you make a booboo. Option: exrc Short Form: ex Value: on/of Default: off Description: Vi can read an exrc file that will run a set of commands at startup. Basically a configuration file for setting your options (see next chapter). Option: hardtabs= Short Form: ht= Value: numeric Default: 8 Description: Set boundary for hardware tabs, may be useful for a Teletype. Option: ignorecase Short Form: ic Value: on/off Default: off Description: Ignore case in regex expressions. Option: lisp Short Form: ? Value: on/off Default: off Description: Turn on lisp mode, I think it aids the formating of old lisp code. Option: list Short Form: ? Value: on/off Default: off Description: Display all tabs, end of lines. Turn this on for fun 🙂 Option: magic Short Form: ? Value: on/off Default: on Description: Enable more regex expressions Option: mesg Short Form: ? Value: on/off Default: on Description: Allows mesgs to be sent to terminal, see man mesg(1) Option: number Short Form: nu Value: on/off Default: off Description: Enables line numbering, this can help when debugging files. I usually have a mapping to enable it with the F2 key in my ~/.exrc file. Option: prompt Short Form: ? Value: on/off Default: on Description: Enalbes the ‘:’ prompt, if you disable this you can make ex behave more like ed. You probably don’t want this. Option: readonly Short Form: ro Value: on/off Default: off Description: Prevents you from writing the file unless you override it like so. :w! Option: redraw Short Form: ? Value: on/off Default: off Description: Redraw screen when edits are made, probably works like the Control+L command (refreshes screen). Option: report= Short Form: ? Value: numeric Default: =5 Description: Report changes if they effect more lines then report=n. Such as when using the delete/yank/join commands e.t.c. Option: shiftwidth= Short Form: sw= Value: numeric Default: 8 Description: How many characters to use when shifting width with the >> and << commands e.t.c. If you plan to use autoindent, the value of sw and ts should be the same (see below). Option: showmatch Short Form: sm Value: on/off Default: off Description: The cursor will flash and move to the openning {, }, (, ), [, or ] when typing a closing one on screen. Option: showmode Short Form: ? Value: on/off Default: off Description: Shows which mode you are in such as command, insert, append, or replace. This is so useful its a perfect choice for the beginners exrc file. Option: tabstop= Short Form: ts= Value: numeric Default: 8 Description: How many characters to display a 'tab' as, mostly of use for programmers. Common choices are 2, 4, 6, 8, 10, 12. It has been suggested that one should not change this setting but instead adjust the shiftwidth setting. I can only say from personal experience if you change tabstop but forget to change shiftwidth to the same. Auto-indent will be a bit off :-) Option: wrapmargin= Short Form: wm= Value: numeric Default: 0 Description: Set the right margin. Setting a value greater than 0 will word wrap n spaces from the edge of the screen. So :set wm=5 would make the display wrap lines when ever it hit the 5th to last line of the terminal. Option: wrapscan Short Form: ws Value: on/off Default: on Description: Searches wrap around end of file How to make a exrc fileThe exrc file is very simple. Commands are read from the ~/.exrc file just as they are from the ‘:’ command line in Vi. Placing this line in your exrc file set autoindent Is the same as tying :set autoindent in command mode. You can use the double qoute or ” character to comment a line. ” This line is not read by Vi set tabstop=6 Here is an example of what my ~/.exrc file might look like with extra comments.

" Tell me what mode I'm in between coffee breaks
set showmode
" Press F2 to turn on line numbering, make the ^M with ctrl+v ctrl+m
map :set number
" Alias ';' to :
map ; :

UNDER CONSTRUCTION

__            ___      ___  
| | / / / /
| || / / / /__/ /
| || / / / __/
| || / / / ___
| || / / / / /
| || / / / / / /
| || / / / / / /
| || / / / / / /
| || / / / / / /
| ||/ / / / / /
| |/ / / / / /
| / / / / /
| / / / / /
| / / / / /
|__/ / /__/ /
__/ __/