Local Subversion Repositories

Everyone should be using some kind of version control system for their own files. There are tons of options available, including mercurial, git, CVS and subversion. I've recently had to move some code to subversion so that I can share code with some other groups. You don't need to have a server set up out there somewhere on the interwebs. You can set up your own private repository on your own machine and use that to track changes to your code and documents. The first step is to create space on your drive

mkdir ~/svn

After that, you need to initialize the actual repository

svnadmin create ~/svn

That's it. Now you have a repository that you can use. The very first thing is to import your current code.

svn import /home/jbernard/temprepos/my_sources file:///home/jbernard/svn/my_sources -m "Initial Import"

After this initial import, you can checkout a working copy of the code to work with.

svn checkout file:///home/jbernard/svn/my_sources

All of the other commands assume that you are actually in the subdirectory holding the checked out code. You can check the surrent status of your code with

svn status

Any edited files will be flagged. You can check in these changes with

svn commit

where it will ask you for a comment about the commit. You can grab any changes committed by someone else with

svn update

I'll be adding more commands here as I think of them.

WiFi on the Command Line

More people than ever are using wireless networks as their primary networking medium. There are great programs available under X11 that give users a graphical interface to their wireless cards. Both Gnome and KDE include network management utilities, and their is a desktop environment agnostic utility called wicd which also offers great functionality. But what do you do if you aren't running X11 and you want to manage your wireless card? I won't be covering how to get your card installed and activated. For that, you should look at projects like madwifi or ndiswrapper, in order to properly configure your system and card. In the rest of this piece, I'll assume that you have your card properly installed and configured, and that it is called wlan0. Also, most of the utilities mentioned below have to talk directly to your wireless card, or at least the card driver, and so will need to be run with root privileges. Just remember to use sudo.

The first step is to see what wireless networks are available in your area. There is a utility called iwlist which provides this information. This utility can give you all sorts of information about your wireless environment. The first thing to do is to scan your environment for available networks. You would run
sudo iwlist wlan0 scan
and get output resembling
Cell 01 - Address: 00:11:22:33:44:55
ESSID:"network-essid"
Mode:Master
Channel:11
Frequency:2.462 GHz (Channel 11)
Quality=100/100 Signal level:-47dBm Noise level=-100dBm
Encryption key:off
.
.
.
The details (address and essid) have been changed to protect the guilty. Also, the ... is extra output that may or may not be available, depending on your hardware. You will get a separate cell entry for each access point within range of your wireless card. For each access point, you can find out the hardware address, the essid and the channel it is operating on. Also, you can find out what mode the access point is operating in (whether master or ad-hoc). In most cases, you will be most interested in the essid, and what encryption is being used.

Once you have information about what is available in your immediate environment, you need to configure your wireless card to use one of these access points. You can use the utility iwconfig to set these parameters for your wireless card. The first thing you will want to set is the essid, which identifies the network access point you are interested in. You would run
sudo iwconfig wlan0 essid network-essid
Depending on your card and its driver, you may have the option to set the essid to the special value "any". In this case, your card will pick the first available access point. This is called "promiscuous mode".

You may also need to set the mode to be used by your wireless card. This will depend on your network topology. You may have a central access point that all of the other devices connect to. Or you may have an ad-hoc wireless network, where all of the devices communicate as peers. You may wish to have your computer act as an access point. If so, you can set the mode to master using iwconfig. Or, you may want to simply sniff what is happening in the air around you. You can do this by setting the mode to monitor and passively monitor all packets on the frequency your card is set to. You can set the frequency, or channel, by running
sudo iwconfig wlan0 freq 2.422G
or
sudo iwconfig wlan0 channel 3

You can also set other parameters, but you should only consider doing so if you have a really good reason. One option of interest is the sensitivity threshold. This defines how sensitive the card is to noise and signal strength. You can set the behavior of the retry mechanism for the wireless card. You may need to play with this in very noisy environments. You can set the maximum number of retries with
sudo iwconfig wlan0 retry 16
or set the maximum lifetime to keep retrying to 300 milliseconds with
sudo iwconfig wlan0 retry lifetime 300m
In a very noisy environment you may also need to play with packet fragmentation. If entire packets can't make it from point to point without corruption, your wireless card may have to break packets down into smaller chunks to try and avoid this. You can tell the card what to use as a maximum fragment size with
sudo iwconfig wlan0 frag 512
This value can be anything less than the size of a packet. The last thing you may need to run is
sudo iwconfig wlan0 commit
Some cards may not apply these settings changes immediately. In these cases, you'll need to run this command to flush all pending changes to the card and get them applied.

Two other commands that may prove useful are iwspy and iwpriv. If your card supports it, you can collect wireless statistics by using
sudo iwspy wlan0
The second command gives you access to optional parameters for your particular card. iwconfig is used for the generic options available. If you run it without any parameters
sudo iwpriv wlan0
it will list all available options for the card. If there are no extra options, then you will get an output looking like
wlan0 no private ioctls
To set one of these private options, you would run
sudo iwpriv wlan0 private-command [private parameters]

Now that you have your card configured and connected to the wireless network, you'll need to configure your networking options to actually use it. If you are using DHCP on the network, you can simply run dhclient to query the DHCP server and get your IP address and other network settings. If you wish to set these options manually, you can do so through the command ifconfig. I would suggest that you give the man page for ifconfig a read. Hopefully this article will help those on the command line to use their wireless cards and networks.

Matlab OS Checks

Matlab, at least on Linux, is pretty firm on which versions it will run on. You may see glibc warnings when you try and run on older distributions. The problem is, as long as the distro isn't too old, Matlab will probably run fine. When you run Matlab interactively, you can simply accept the warning and continue on. But this doesn't work if you want to run scripts in batch mode. To get around this, you can set an environment variable to skip the OS check. If you use bash, simply use

   export OSCHECK_ENFORCE_LIMITS=0

This should let you go ahead and run your script without pausing. Hope this helps.

Putty timeouts

Some of my users were having problems with ssh connections dropping. Don't forget to go into the connection section of the preferences and set the keepalive options.

GoFredX Convention

I'm currently attending tne GoFredX Convention in Fredericton. The talks so far have been great. I think the talks after lunch are going to be even better.

Maintaining Your System from the Command line

Many Linux distributions use some form of packaging system to to organize the applications installed on a system. Using a formal packaging system lets you install, remove and in general, maintain the software on your system in a controlled and coherent way. The three main packaging systems are the Debian deb package, the Red Hat rpm package, and the Slackware pkg package. The vast majority of the distributions today will use one of these three packaging systems. They all have graphical utilities to interact with the packaging system. But what if you want to deal with the system on the command line? Say you are running a server, or are accessing a distant machine through ssh and don't want to deal with the overhead of X11? This month we'll take a look at how to do this for Debian based systems.

The first thing you will probably want to do is install some software on your system. The preferred way to do this is through the utility apt-get. apt-get is aware of the chain of dependencies between packages. Let's say you want to do some star gazing and want to install stellarium on your system. You would run
apt-get install stellarium
This would download the relevant package file, and all of its dependencies, from a repository. What if you don't know exactly what the package is named? You can query the package management system with the utility dpkg-query. If you know that the package name has "kde" in it, you can list all of the matching packages with the command
dpkg-query -l "*kde*"
Remember to quote any search strings that have a "*" in it so that you don't inadvertently have the shell try and expand them.

This works great for software available in the given repository. But what if you want something not available there? If you have a ".deb" file available for download, you can download it and install it manually. After downloading the file, you would install it by running
dpkg -i file_to_install.deb
The utility dpkg works with the deb packaging system at a lower level than apt-get. With it, you can install, remove and maintain individual packages. If you have a whole group of packages you would like to install, you might want to add the relevant repository to your list so that apt-get will know about it. The list of repositories is stored in the configuration file /etc/apt/sources.list. Each line has the form
deb http://us.archive.ubuntu.com/ubuntu/ karmic main restricted
The first field tells apt-get what is available at this repository: deb is for binary packages, deb-src is for source packages. The second field is the URL to the repository, in this example we're looking at the Ubuntu repository. The third field is the repository name, in this case the repository for the karmic version of Ubuntu. The last fields are the sections we want to look at to install packages from. This example will look at the sections main and restricted when trying to install applications or resolve dependencies.

Now that you have some applications installed, you will probably want to maintain them and keep them updated. Every piece of software will have bugs, or security issues, that come to light over time. Software developers are always releasing new versions to fix these issues, and updating the relevant packages in the repositories. To update the list of software and their versions on your system, you would run
apt-get update
Once you've updated the list, you can tell apt-get to install these updates by running
apt-get upgrade
If you want a list of what is about to be upgraded, add the command line option "-u"
apt-get upgrade -u
Sometimes, when a new version comes out for a package (like when your distribution releases a new version), the dependencies for said package might change, too. In cases like this, a straight upgrade might be confused. In these cases you can use the command
apt-get dist-upgrade
This command tries to intelligently deal with these changes in dependencies, adding and removing packages as necessary.

What do you do if you've installed a package to try it out and don't want it anymore? You can remove a package with the command
apt-get remove stellarium
This removes all of the files that were installed as part of the package stellarium, but leaves any configuration files intact and also doesn't deal with any extra packages installed because stellarium depended on them. If you wish to completely remove a package, including all configuration files, you would run
apt-get purge stellarium

All of this software installation and removal could result in cruft accumulating on your system. You may end up with unnecessary packages wasting space on your system. To start to recover some space you can run the command
apt-get autoclean
This command will remove the package ".deb" files from the local cache for packages that can no longer be downloaded. These would be mostly useless packages. If you want to completely clean out the local cache and recover more space, you can run
apt-get clean
While "remove" and "purge" will remove a package, what can you do about any dependencies installed for this package. If you run the command
apt-get autoremove
you can uninstall all packages that were installed as dependencies for other packages and aren't needed anymore.

Another way of finding packages that aren't needed anymore is through the utility deborphan. The first thing you'll need to do is install it, using
apt-get install deborphan
since most distributions don't install it by default. Once it is installed, running it with no command line options will give you a list of all packages in the libs and oldlibs sections that have no dependents. Since no other package depends on these packages, you can safely use apt-get to remove or purge them. If you want to look in all sections, you can use the option "-a". If you're trying to save space, you can ask deborphan to print out the installed sizes for these orphan packages by using the command line option "-z". You can then sort them out by running
deborphan -z -a | sort -n
This will give you a list of packages you can safely uninstall, sorted by installed size from smallest to largest. You can then free up space on your system by getting rid of the biggest space wasters.

Hopefully this gives you a good starting point to dealing with the Debian package management system. Each of the tools discussed above have lots of other options that you should research in the relevant man pages. Also, if you use a Red Hat based system, there are equivalent commands to these to help you manage the rpm files used there.

Using sed to clean up files

You can use sed to delete particular lines, based on some unique criteria. This would look like

sed -e "/search criteria/d" -i2 file1.txt

This happens "in place", replacing the file's original contents. If you want to process a number of files in the same way, you can use

ls *.txt | xargs -i{} sed -e "/search criteria/d" -i2 {}

This takes the results from ls and hands it in to xargs. xargs then runs sed on each filename, one at a time.