Mac / Linux cross-development

Introduction
Mac has always been the default alternative to Wintel systems. In the past, the Mac OS has been a fairly unique entity, without too much in common with the other OSes around, like Windows or the Unix variants. Cross-platform work has always been a bit more convoluted.
With the advent of the latest incarnation of the Mac OSX, or Darwin, we've got an alternative that is very comfortable for Linux geeks. Since Darwin is based on BSD Unix, we now have the possibility of using POSIX compliant applications on the Mac. This article will cover using Linux apps on a Mac, and some of the issues in running Mac apps on Linux. We'll also take a look at using the Mac as a development environment, developing applications that will end up on Linux.
Running Linux Code on Mac OSX
Apple provides a package called Xcode on their developer site. This packages gives you the tools you need to compile programs on the Mac. This includes a nice graphical IDE and lots of examples for developing your own applications for OSX. But that is not the only thing you can do. Xcode is based on the GNU toolset, providing tools like gcc, libtool, make, etc. This means that using Xcode, most command line applications can be compiled and run on the Mac. So our simple little hello world program

   #include <stdio.h>
   #include <stdlib.h>
   int main (int argc, char **argv) {
      printf("Hello World\n");
   }

compiles fine with gcc, giving us an executable which prints out "Hello World" on the command line. Basically, anything that is POSIX compliant should compile and run with no issues.
The process of getting graphical programs to run can be a bit more involved. Mac OSX does provide an X11 server, and all of the standard development libraries that you would need for a purely X11 application, like Xlib. However, none of the other standard libraries, like gtk or Qt, are available by default. You are responsible for getting them downloaded, compiled and installed on your system yourself. In this sense, it's a lot like the Linux From Scratch project. This works fairly well, but you do have to pick the correct configuration options, and collect all of the required dependencies. However, there's no reason why you need to go through this much pain. There are two projects in active development which provide some form of package management for GNU software, Fink and MacPorts. Using these, getting and installing GNU software is made as easy as most Linux distro.
The Fink project started in 2001. Fink is based on the Debian package management system. This means that you can use the Debian package tools like dpkg, dselect and apt-get. This makes it familiar for users of Debian-based distros, like Debian or Ubuntu. Once the base installation is done, you can start to install packages. If you like a text based manager, you can use dselect . If you like a graphical manager instead, you can use the command
sudo apt-get install synaptic
to get synaptic

 Using these applications, we can install many of the packages that we are used to in Linux. The package count, as of the writing of this article, is 10872. Not all packages are available as a binary install using these tools. For this class of packages, fink will install them directly from source, compiling and installing on your Mac. So, for example, if you wanted to install gramps and get some genealogy done, you would simply execute
sudo fink install gramps
Even installing from source, fink deals well with the dependency issues since it is still based on the Debian package management system.
The Mac Ports project started in 2002. This project decided to model itself after the BSD port packaging system. This means that you use the command port to manage the packages on your system. Once you have done the base install, you can install other software packages by simply running the command
sudo port install stellarium
There are several graphical interfaces available, as well, such as Porticus. However, they tend to be independent projects, as opposed to the Debian tools available in Fink. As such, their development cycle and behavior tend to be a bit more erratic and unstable then the older and more mature Debian tools. But still, they may be exactly what your are looking for if you prefer a graphical interface. Like the Fink project, it has both binary packages and source packages available. There are 5829 packages available in the Mac Ports project, and it is growing constantly.
Both projects provide the user with access to the full wealth of open source applications that has been available to Linux users, and the number of packages available continues to grow. This further blurs the line between what makes an OS. If the only difference between Mac OSX and, say Debian, is the kernel and maybe the GUI, are they effectively the same OS? I won't pretend to have an answer, but I think that we may have to start opening our ideas of what constitutes an OS.
Once you have one, or both, of these projects installed (they can peacefully coexist on your system) you will have all of the tools necessary to do your own code development. I have used anjuta

 on my MacBook to develop some small Gnome applications. These compile and run equally well on my MacBook, and my netbook running Ubuntu. While there isn't binary compatibility between OSX and Linux, with source compatibility we can essentially run the same code on both machines.
Running Mac OSX on Linux
This is not as easy as running Linux on Mac OSX. The real stumbling block is the graphical interface in the Mac OS. This is the area where Apple have been very secretive and proprietary. The graphical interface is based on a software technology called Quartz. While the kernel and most of the command line tools have been released as open source software, Quartz is still very closed. As of the writing of this article, I could not find any references to a reverse-engineered open source replacement for Quartz. So the only option available is running OSX inside a virtual machine. While this is not technically running Mac applications on Linux, it does provide the ability of running OSX on a Linux box.
Conclusion
Using the links and information in this article, user who straddle the Linux / Mac divide can narrow that gap a considerable amount. With projects like Fink and MacPorts, Linux users have access to the full spread of GNU and other open source software. As well, the Mac makes a good development environment, allowing a user to develop code that will run on both Linux and OSX. Hopefully, you will explore the possibilities and shrink the divide a little more.
Links

• http://developer.apple.com
• http://www.opensource.apple.com
• http://www.finkproject.org
• http://www.macports.org

What Hardware Do I Have?

There are many cases where you may not necessarily know what kind of hardware you have. You may have bought a no-name box from a smaller company. Or you may have bought a used machine. In any case you may not know what you have for hardware. This month we'll look at tools you can use to find out what you have installed.

The first tool to look at is lshw. This utility LiSts HardWare (lshw). If you run it as a regular user it will actually warn you to run it as root. So go ahead and run "sudo lshw". You should now see screens of information for your system. The first section will be general information, and should look something like

jbernard-eeepc
description: Notebook
product: 700
vendor: ASUSTeK Computer INC.
version: 0129
serial: EeePC-1234567890
width: 32 bits
capabilities: smbios-2.5 dmi-2.5 smp-1.4 smp
configuration: boot=normal chassis=notebook cpus=1 uuid=XXXXXX-XXXXX-XXXXX-XXXXX

This is what I get when I run it on my little Asus EeePC. So right away we can find out the manufacturer of this little beast (ASUSTeK), the BIOS version (0129), and the fact that it is a 32-bit machine with one CPU. There is more information broken down into the following categories

core
firmware - motherboard and BIOS information
cpu - CPU information
cache - cache information
memory - memory information
bank - specific bank memory information
pci - PCI bus information
display - PCI diplay adapter
multimedia - PCI audio adapter
pci - other PCI devices
network - PCI network adapter
usb - USB devices
ide - IDE information
disk - individual disks
volume - volumes on this disk

If we look at the section describing main memory, we get on my EeePC

*-memory
description: System Memory
physical id: 1f
slot: System board or motherboard
size: 512MiB
*-bank
description: DIMM DDR2 Synchronous 400 MHz (2.5 ns)
product: PartNum0
vendor: Manufacturer0
physical id: 0
serial: SerNum0
slot: DIMM0
size: 512MiB
width: 64 bits
clock: 400MHz (2.5ns)

we can get an idea of how much information is available.

This utility is kind of an all in one tool that spits out everything on your system in one go. But what if you want information only about specific subsystems in your machine. There is an entire suite of utilities you can use for this. These might be more useful when you want some specific piece of information, or you want to do some system querying in a script.

The first thing you may want to look at is the CPU. The utility lscpu will give you output similar to

Architecture: i686
CPU op-mode(s): 32-bit
CPU(s): 1
Thread(s) per core: 1
Core(s) per socket: 1
CPU socket(s): 1
Vendor ID: GenuineIntel
CPU family: 6
Model: 13
Stepping: 8
CPU MHz: 571.427

From this, you can find out the manufacturer, whether it is 32-bit or 64-bit, the exact version and model, as well as the current CPU frequency.

If you want to find out whether your video card is supported by X11, or whether you'll need to go and find a third-party driver, you can use lspci. This utility will give you a list of all of the devices plugged into you PCI bus. The output will look something like

00:02.0 VGA compatible controller: Intel Corporation Mobile 915GM/GMS/910GML Express Graphics Controller (rev 04)
00:02.1 Display controller: Intel Corporation Mobile 915GM/GMS/910GML Express Graphics Controller (rev 04)

With this information we can see that the video controller in my EeePC is an Intel controller. So, if you wanted to, you could now hit Google with this information to learn about your video card and how best to configure it. If you want to see what USB devices are on your system, you can use lsusb. On my EeePC I have an SD card installed, and it shows up as

Bus 001 Device 002: ID 0951:1606 Kingston Technology

The last piece of hardware we'll look at is the disk subsystem. You can find out what you have on your system with the utility blkid. This utility will print out all of the filesystems available to you, with an output format of

/dev/sda1: UUID="XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX" TYPE="ext2"
/dev/sda2: UUID="XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX" TYPE="swap"
/dev/sda3: UUID="XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX" TYPE="ext2"
/dev/sdb1: UUID="XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX" TYPE="ext2"

With this utility, you can find out what devices are available and what filesystems are being used on them. The associated UUID's are also available, in case you want to use them in the entries in /etc/fstab.

Now that we know what kind of hardware we have on our system, the last thig to check is to see if our kernel is actually using this available hardware. On most modern distributions, the kernel is compiled to use modules. You can check to see which modules are loaded by using the command lsmod. We get a list that looks like

agpgart 31788 2 drm,intel_agp
lp 7028 0
video 17375 1 i915
output 1871 1 video

We can see that the agpgart modules has a size of 31788 bytes, and is used by the drm and intel_agp modules.

Now that you know what you have installed for hardware, you can hopefully get it all configured and optimized so that you get the most out of it. If you find other utilites that aren't covered here, I would love to hear about them.

How To Write Unmaintainable Code

How To Write Unmaintainable Code: All programmers who want to guarantee job security really should read this.

'via Blog this'

The Web on the Console

Most people think graphical interfaces when they think of surfing the web. And under X11, there are lots of great programs, like firefox or chrome. But, the console isn't the wasteland that it might seem. There are lots of utilities available to surf the web, and also to download content from or upload content to the web.

The first thing you may want to do is surf around the web and find some content. The first utility to look at is also one of the oldest, the venerable lynx. Lynx was actually my first web browser, running on a machine that couldn't handle X11. Using it in its most basic form, you simply run it on the command line and give it a file name or a URL. So, if you wanted to hit Google, you would run
lynx http://www.google.com
The first thing you will notice is that lynx will ask you whether you want to accept a cookie that Google is trying to set. Once you either accept or reject the cookie, lynx will load the web page and render it. As you will no doubt notice, there are no images. But all of the links and the text box for entering your search query are all there. You can simply navigate around from link to link with your arrow keys. Because the layout is very simple and text-based, items will be in very different locations on the screen from what you will see using a graphical browser.

There are several options to lynx that might be handy to know. The first useful tip is that you can hand in more than one URL when you launch lynx. Lynx will add all of these URLs to the history of your session and render the last URL and display it. When you tested loading Google above, lynx asked you about whether or not to accept a cookie. Most sites these days use cookies. You may not want to hear about every cookie. You can use the option "-accept_all_cookies" to avoid all of these warning messages. You can use lynx to process web pages into a readable form by using the option "-dump". This will take the rendered output from lynx and write it out to standard out. This way you can simply process web pages to a readable format and dump them into a file for later viewing. You can choose what kind of key mapping you wish to use by using the options "-vikeys" or "-emacskeys". By using one of the options, you can use shortcut keys that match your editor of choice.

Lynx does have a few issues. It has a hard time with HTML table rendering and doesn't handle frames. Along comes the browser links. Links not only works in text mode on the command line, but can also be compiled to use a graphics display. The graphics systems supported include X11, svga, and framebuffer. You can select one of these graphics interfaces with the option "-g". Links can also write out the rendered web pages to standard output by using the option "-dump". If you need to use a proxy, you can tell links which to use with the option "-http-proxy host:port". Links is also able to deal with buggy web servers. There are several web servers out there which claim to be compliant to a particular HTTP version and aren't. To try and compensate for this, you can use the "-http-bugs.*" options. For example, "-http-bugs.http10 1" forces links to use HTTP 1.0, even when a server claims to support HTTP 1.1.

If you are looking for a strictly text replacement for the venerable lynx, there is elinks. Elinks supports colors, table rendering, frames, background downloading and tabbed browsing. One possibly useful option is "-anonymous 1". This option disables local file browsing and downloads, among other things. Another interesting option is "-lookup". When you use this, elinks will print out all of the resolved IP addresses for a given domain name.

Now that we can look at web content from the command line, how can we interact with the web? What I really mean is, how do we upload and download from the web? There are many instances where you may want an offline copy of some content from the web that you can read through at your leisure, off by the lake where you don't have any Internet access. One of the tools you can use to do this is curl. Curl can transfer data to or from a server on the Internet using HTTP, FTP, SFTP, even LDAP. It can do things like HTTP POST, SSL connections, and cookies. You can specify form name/value pairs so that the web server things you are submitting a form by using the option "-F name=value". One really interesting option is the ability to use multiple URLs through ranges. For example, you can specify multiple hosts with

curl http://site.{one,two,three}.com

which will hit all three sites. You can range through alphanumeric ranges with square brackets. The command

curl http://www.site.com/text[1-10].html

will download the files text1.html to text10.html.

But what if you want a copy of an entire site for offline browsing? The tool wget can help you out here. In this case, you will likely want to use the command

wget -k -r -p http://www.site.com

The option "-r" recurses through the links of the site starting at http://www.site.com/index.html. The option "-k" rewrites the downloaded files so that links from page to page are all relative so that you can navigate correctly through the downloaded pages. The option "-p" downloads all extra content on the page, such as images. This way you can get a mirror of a site on your desktop. Wget also handles proxies, cookies, HTTP authentication, along with many other conditions. If you're uploading content to the web, you can use wput. Wput pushing content up using ftp, with an interface like wget.

So now you should be able to interact with the Internet without ever having to use a graphical interface. Yet another reason to keep you on the command line.

Google Reader going away

Well, Google has announced that Reader is going away July 1. Argh! I was one of those who used almost religiously. Now I need to find a replacement. I think I have found it in Feedly. It seems to give me the same ability to keep multiple devices synchronized. The only think I haven't found yet is how to change the sort order so that I see the oldest articles at the top. Once I do, I think I'll be happy again.

Comparing Files

There are many times when you wish to know what has happened to a
file. You may be interested in comparing one version of a file to an
earlier one. Or you may need to check one file against a reference
file. Linux provides several tools for doing this, depending on how
deep a comparison you need to make.

The most common task involves comparing two text files. The tool of
choice for this task is diff. With diff, you can compare two files,
line by line. By default, diff will notice any differences between the
two text files, no matter how small. This could be as simple as a
space character being changed into a tab character from one file to
the next. The file will look the same to a user, but diff will find
that difference. The real power of diff comes from the options
available to ignore certain kinds of differences between files. In the
above example, you could ignore that change from a space character to
a tab character by using the option "-b" or
"--ignore-space-change". This option tells diff to ignore any
differences in the amount of whitespace from one file to the next. But
what about blank lines? The option "-B" or "--ignore-blank-lines"
tells diff to ignore any changes in the numbers of blank lines from
one file to the next. In this way, diff will effectively be only
looking at the actual characters and comparing them from one file to
the next. You have essentially narrowed the focus of diff to the
actual content.

What if that is not good enough for your situation? You may be
comparing files where one was entered with all capitals on, for some
reason. Maybe the terminal being used was misconfigured. In any case,
you may not want diff to report simple differences in case as "real"
differences. In this case, you can use the option "-i" or "--ignore-case".

What if you have files from a Windows box that you are working with?
Everyone who works on both Linux and Windows has run into the issue
with line endings on text files. Linux expects only a single newline
character while Windows uses a carriage return and a newline
character. diff can be told to ignore this with the option
"--strip-trailing-cr".

The output from diff can take a few different formats. The default
output contains the line which is different, along with a number of
lines just before and just after the line in question. These extra
lines are called context, and can be set with the option "-c", "-C" or
"--context=" and a number of lines to use for context. This default
output can be used by the program patch to change one file into the
other. In this way, you can create source code patches to upgrade code
from one version to the next. diff will also output differences
between files that can be used by ed as a script by using the option
"-e" or "--ed". diff will also output an RCS format diff by using the
option "-n" or "--rcs". The other option is to print out the
differences in two columns, side by side. The option "-y" or
"--side-by-side" will let you see each file side by side with the
differences between them highlighted.

The utility diff only compares two files. What if you need to compare
three files and see what changes exist moving from one to the others?
The utility diff3 comes to the rescue. This utility compares three
files and prints out the diff statements. Again, you can use the "-e"
option to print out a script suitable for the editor ed.

But what if you simply want to see two files and how they differ?
Another utility might be just what you are looking for, comm. With no
other options, comm takes two files and prints out three columns. The
first column contains lines unique to the first file, the second
column contains lines unique to the second file and the third column
contains lines common to both files. You can selectively suppress each
of these columns with the options "-1", "-2" and "-3". They suppress
columns 1, 2 or 3, respectively.

While this works great for text files, what if you need to compare two
binary files? You need some way to compare each and every byte in each
file. The utility that can be used for this is called cmp. It does a
byte by byte comparison of two files. The default output is a print
out of which byte and which line contains the difference. If you are
interested in seeing what the byte values are, you can use the option
"-b". The "-l" option gives even more detail, printing out the byte
count and the byte value from the two files.

Using these utilities, you can start to get a better handle on how
your files are changing. Here's hoping you keep control of you files.

SVN over SSH

There are many times when the only access to a server is over SSH. There are many reasons for this, including security. If you want to host an SVN repository on this machine, how do you access it? The first step is to create the repository. If you are creating a new one, then this easy.Just log in to the server and do the following

svnadmin create /path/to/repo

If you already have a repository that you want to move to the server, you should use svnsync to do so. First, you will want to allow a user to have svnsync access with

#!/bin/sh
USER=""
if [ "$USER" = "myusername" ]; then exit 0; fi
echo "Permission denied"
exit 1

This goes in the file /path/to/repo/hooks/pre-revprop-change. Make sure that it is set as executable

chmod +x ~/mymirror/hooks/pre-revprop-change

You can then tell svnsync to initialize the mirror.

svnsync init file:///path/to/repo
svn+ssh://username@svnmaster.domain.ext/newpath/to/repo

Once this initialization is done, you can sync the full data.

svnsync --non-interactive sync file:///path/to/repo

You can then interact with this new repository over SSH by using

ssh+svn://username@svnmaster.domain.ext/newpath/to/repo

as the connection details.

One problem that may come up when your SVN repository from one server to another is to fix up your local copy. There is a switch command that will allow you to do this.

svn switch --relocate old-url new-url

This way, you don't need to toss away your current working copy and checkout a fresh one.