Kok-Yan's blog

If you like watching TV shows involving meaningful relationship dramas, "The Secret Life of the American Teenager" is definitely worth watching.

You can get more information on the show here, or you can download them here.

If you like listening to weird sounds, the Boston Typewriter Orchestra is surely up your road.

They are a group of people who use old typewritters to make really interesting rhythmic beats. You can check out their website here and get some audio samples here.

I must admit, the past week had been very difficult and I was deeply upset by a chain of events that I cannot comprehend at the time. The following lecture allowed me to see the reasons behind those events and have changed my view on the situation entirely, and also life in general, for the better.

The lecture was given by professor Randy Pausch in September 2007 (after he had been diagnosed with terminal cancer) and is titled "Really Achieving Your Childhood Dreams.". In the lecture, he talks about his own life and his life philosophies, and is deeply meaningful and very positive.

Please spend some time watching the video and looking at the slides - I'm sure you'll find something really positive from it.

I don't usually write about personal stuff on here but recent events has taught me to never forget a very important rule - if you are not sure about saying something, don't say it - no exceptions.

The last 2 words "no exceptions" are very important; the last time this happened, I managed to refrain from pressing the "send" button and I was glad I did that. This time round, I let one such message slip through and it made a simple situation really complicated, and may have far-reaching and tremendous consequences because of this.

So please, for everyone who is reading this, if you can't refrain from writing that email, at least save whatever you're planning to send until the next morning. Think about it, think some more, and delete / archive it - but never send it.

Lol, this just cracked me up when I saw it.

It is actually availiable for sale at the NBC Universial store!

For those of you who are still learning Bash (including me...), I'm sure one of the things you would have asked yourself is "How on earth do I return an array from a bash function?".

Well I've written a small script that will explain this:

#!/bin/bash
IFS=$'\n\t'

function fnGo () {
	array=(
		a	s	d	f	
		"gh ij"	"kl mn"
	)
	echo "${array[*]}"
}

# -------- out - String variable --------
out=$(fnGo)
echo "\"out\" isn't an array: ${out[1]} - nothing"

echo $'\n'"\"out\" Works with iteration:"
for item in $out; do
	echo "item:"$'\t'"$item"
done

# -------- out2 - An array --------
out2=($(fnGo))
echo $'\n'"\"out2\" now an array:"
for ((i=0; i<${#out2[*]}; i++)); do
	echo "item $i:"$'\t'"${out2[i]}"
done

echo $'\n'"Though \"out2\" cannot be iterated anymore...:"
for item in $out2; do
	echo "item:"$'\t'"$item"
done

Basically, it's exactly as you'd do for returning a single value from a function (use echo) - but, you need to make sure you surround the variable with quotes (in the function - echo "${array[*]}"), and receive it as an array - out2=($(fnGo)).

Note you can only either choose to use an iterator method (out1), or an addressing method (out2), but not both - run the script and you'll see what I mean.

Oh, one more thing (just as a tip for those who don't already know) - pay attention to your IFS variable (which determines how parameters are separated)! This is especially important if you're taking in quoted (escaped) command-line parameters that may have a space in them (such as file names) - in that case, I normally use "\n".

A while back I attempted to write a wrapper script for mplayer to resume a file. Unfortunately, that script was rather limiting where:

• File name must be first entry
• You have to manually specify the "-profile" parameter
• You cannot specify a global profile (i.e. the profile you specify only applies to the last file)

Now I have a new script that resolves the above issues! Unfortunately, there are some new issues with the new script:

• It's slow at generating command for many files
• Extension-less file won't work...
• Per-file manual profile definition is not yet implemented - though automatic RegExp profile definition is working :-P !
• This script is very much in the alpha stage and more testing is needed - so use at your own risk!

Sounds cryptic? It'll be clearer after I've shown you some examples.

As with the previous script, an example usage (we'll assume we're trying to resume a file called "mediaFile.avi" at timecode 300, and the script is named "mp"):

echo 300 > mediaFile.avi.txt
mp mediaFile.avi

Also as with the previous script, you'll probably want to enable the "statusline" display by adding "msglevel=statusline=9" into your mplayer config file (at "~/.mplayer/config" | "/etc/mplayer/config"), and ensure "quiet=0".

Now, to use a profile for all files (at the moment, it doesn't matter where you place "-profile" and "-ss" - they just get applied to every file...):

mp -profile hd file1.avi "file 2.avi"

However, the real magic of the v2 script comes into play when you have multiple files, matching multiple profile switching definitions, with different resume files! For example:

$ ls -d *.avi*
file1.mts
file1.mts.txt
2x03 SomeEpisode.mkv
2x03 SomeEpisode.mkv.txt
file2.avi

$ less mp
...
profiles=(
	# [PF name]	[RegExp]
	"hd"	"(/|^)[0-9]{,2}x[0-9]{,2} .*\.mkv$"
	# ---- Generic profiles ----
	".mts"	"\.mts$"
)
...

$ mp file1.mts "2x03 SomeEpisode.mkv" file2.avi

The script will now use the ".mts" profile for the mts file, the "hd" profile for the .mkv file, the "global" profile for the .avi file, whilst picking the values out of the resume files and resume at the point specified in the .txt files!
(Just in case you're wondering, the "global" profile is defined in the "$defaultProfile" variable in script.)

Anyway, you can get the script here.

Of course, if anyone's interested in improving it, they're more than welcome to do so! Just don't forget to email me a copy!

A couple months ago, my brother's PC suddenly stopped sharing files via SMB (port 445). Since my brother told me he hasn't done anything, I was a bit mystified as to what exactly is going on.

Initially I though it was a firewall problem - so I checked the firewall config and tried to access the share from localhost (on his PC), which failed as if nothing is shared (i.e. "net view \\127.0.0.1" just times out). I then tried "netstat -an" to see if the SMB server is actually listening for a connection, and it turns out nothing is listening on port 445!

A restart of lanmanserver and lanmanworkstation (services) and a reboot didn't help, so in a last ditch attempt to get something shared I enabled NetBIOS over TCP/IP, which seems to have done the trick (i.e. I can now access the shares). However, port 445 is still no where to be seen in "netstat -an"...

After a lot of searching on Google, I found this article which more less describes the problem I've been having.

After turning off NetBT, applying the registry fix (adding HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\NetBT\Parameters:SmbDeviceEnabled = 1) and rebooting (I had also re-installed "File and Printer sharing" at the time), the shares were now accessible again via port 445.

Unfortunately, I never got to the bottom of what caused this problem - seeing it wasn't simply a missing registry value, as all my other systems don't have the SmbDeviceEnabled value. But I guess I can consider this one being resolved via a work-around as sharing is back to normal...

Note: if you prefer the bare-essential steps and code, please jump straight to the "The re-partition process" section.

Having used Xubuntu for a while now on my transition test rig, I've decided to re-partition the HDD to better suit my needs. Here's why I'm doing this:

• Data security - Not encryption, but rather to ensure the data is (most likely) safe on an OS re-installation
• The default swap partition is not flexible - It always takes up space even when you don't need it to
• Makes backup easier - I can simply image the small OS partitions and back them up onto several DVDs

However, the main drawback (and my original reason for not having a separate data partition) is that your HDD's capacity is effectively reduced (Data = HDD - OS - Swap), and not to mention when your OS partition runs out of your space, you're boned (alright, I know what you're thinking, but let's not get carried away with hacks like creating symbolic links to the data partition...).

First thing first, let's figure out what the setup should look like and how much space we need, then we can explore different options when we encounter them.

Partition, swap and hibernation config

As much I want to make a complete transition over to Linux, there are some situations where Windows is required (like certain applications and firmware updates), so we need a Windows partition.

The best way to judge how much space is required is to look at the existing space usage of "%SystemRoot%" and "%ProgramFiles%" - you can use a tool like "diruse" (from the Windows XP support tools) or folder properties in explorer.exe...

Windows: In a clean(ish) install of XP SP3, %SystemRoot% + %ProgramFiles% comes to about 2.7 GiB (say, 3GiB), and allowing 1 GiB each (yeah, I know it's rather generous) for MS Office, IDEs, .NET, JDK/JRE and AutoRoute the minimum size so far comes to about 8 GiB. Adding a 15% defrag free space requirement, overhead for old (un)installation files, system restore files, pagefile.sys, hiberfil.sys, and some free space, the minimum size for the Windows partition comes to 12 GiB (which also fits nicely onto 3 DVDs when imaged).

It is worth point out that if you have a lot of RAM and want to hibernate your Windows session, you'd obviously want more space to accommodate hiberfil.sys. A safe suggestion would be the next multiple of 4, say, 16 GiB (if you have 3 GiB of RAM) (to fit onto 4 DVDs). As for page file, it's up to your judgement / Windows's to determine how much is needed - and if you have a second HDD, put the page file there instead.

Linux: As for the Linux partition, it's a little bit more difficult to estimate (seeing I haven't really used it for that long) - but base install rounds up to about 3 GiB. For now, I've assigned 16 GiB to it as I plan on installing some of the software above using WINE (like MS Office), OpenOffice, IDEs. Though I'd probably use a 12 GiB partition when I partition my main system - but I'll just have to wait and see how it goes.

As per the Xubuntu default setup, the OS(es) should reside on a primary partition of its own, with the rest being on extended, logical partitions - you can only have up to 4 primary partitions on a volume.

As for swap and hibernation (suspend to disk), I've opted for a 512 MiB swap file and a 512 MiB swap partition dedicated for hibernation, for the 512 MiB of RAM (though there are usage problems with this setup - I'll talk about that next).

The reason behind using a swap file is because it is more flexible (can be resized dynamically) with no reported performance difference under the 2.6 kernel comparing to a swap partition.

As for the swap partition, it is used because there doesn't seem to be a way to configure Xubuntu's in-kernel hibernation implementation to use a hibernation file. The main problem with this setup is that hibernation would fail if the entire swap file had been filled and overflowed onto the hibernation swap partition. Unfortunately, swapoff-ing the swap partition will result in a hibernation failure.

The only usable hack around the the filled swap file problem is to run swapoff after boot and resume from hibernation, and swapon just before hibernation. Unfortunately, I was not able to find a way to do swapoff after resuming from hibernation (I'd welcome any suggestions on this!) - I could use upstart on boot for swapoff and modify "/usr/lib/hal/scripts/linux/hal-system-power-hibernate-linux" to run swapon (this is used by GDM (and the logout GUI) to hibernate), but I still need to run swapoff on resume from hibernation... However, saying that, you could modify and use "/etc/acpi/hibernate.sh" to do the swapon and swapoff as the script stops at "echo -n "disk" >/sys/power/state" and resumes execution afterwards on wake.

The only command I haven't had time to explore is "/usr/sbin/pmi" (called with parameters "action [hibernate|sleep]") - I found this in gdmsetup --> "Edit Commands...".

Update: 2008-05-27 @ 03:09
You can use "pmi" to put your system to sleep / hibernation and be able to do swapon and swapoff at the appropriate time - just do swapon before calling pmi, and swapoff after, since pmi doesn't detach itself. I'll post some scripts on how to do this later.

Hacks aside, there are actually solutions to enable proper hibernation to a file - such as TuxOnIce, but it requires patching and re-compilation of the kernel - which is something I don't plan on doing in this project.

While we're (kinda) still on the subject on partitions, there is something that can resolve the issues with partition sizes - LVM (Logical Volume Manager). In theory, pretty much everything (except "/boot") can be setup to use LVM, but I've decided against it because:

• My inexperience to Linux - I think it's best to keep it simple at this stage
• Compatibility - LVM volumes cannot be read by the "Ext2 Installable File System for Windows" FS driver
• Reliability - A disk failure or (somehow) a LVM Volume Group corruption would be disastrous
• Performance - Since the data goes through another abstract layer, there may be issues there

For now, I'll leave LVM alone until I need a heavy-duty file server with RAID (or equivalent) that runs only on Linux.

File system

So far, we've decided on the partition configuration and now, it's time to choose the file systems.

Since the aim of this project was to re-partition the drive, I've decided to stick with Ext3 in Linux as I don't see any benefits in using another FS for my needs. As for Windows (XP), it's a no-brainer - NTFS (linux can mount that as ntfs-3g type).

I have chosen Ext3 as the file system for the data partition as I will be using Linux primarily, and Windows XP can use the "Ext2 Installable File System for Windows" to access that partition.

Pre-requisites and tools

We're now almost at the point where we can start re-partitioning the drive - now that we know what the disk setup is going to be.

We obviously need a re-partitioning tool, but we'd also need something to backup and restore the drive (in case anything horrible were to happen), and somewhere to store the HDD images.

Not long ago, I came across the SystemRescueCd (possibly on Hak5 or this article on linux-mag.com (free subscription required) - can't remember exactly which...) which demoed the imaging and bare-metal restore capability of the live CD.

As well as the partimage tool on the live CD, it also contains GParted (for re-partitioning) and samba (for smb-based network backups). But there are so many more tools on this CD it is definitely worth keeping in your toolbox.

The re-partition process

Here's the target partition configurations (for a single HDD setup):

12GiB NTFS (primary):
	(Windows) C:\

16GiB ext3 (primary):
	(Xubuntu) /

* (extended):
	[*]GiB ext3 (logical):
	/home/[user]/	# Replace [user] with the users

	[RAM]GiB swap (logical):
	hibernation "swap" partition == RAM size

The essential steps of this project are:

1. Boot into SystemRescueCd
   1. ifconfig ... - If you need network access to store the partition image
   2. mount ... - Create a mount point to save the drive image
   3. partimage - Image the existing OS + Data partition and note down the partition sizes
   4. startx - We'll need a GUI
      1. gparted - Re-size and create partitions
         
         • IMPORTANT: Note down the existing partition sizes! - in case you need to do a restore
         • IMPORTANT: Make sure you have a backup of the partitions! - go back to step 3 if you don't
         • IMPORTANT: Keep actions to a minimum! - ALL actions will be replayed in the re-partitioning process!
         • IMPORTANT: Keep actions to a minimum! - Sorry, I cannot stress this enough - For I've been burnt by it and had to go through a lengthy restore process...
         • If you screw up, re-create the size of the original partitions and restore image, then try again
   5. shutdown -r 0 - Cross your fingers!
      (note the Xubuntu boot screen will exit with a segfault error, just ignore it for now, will fix later)
2. login and gain root access
3. dd if=/dev/zero of=/swap bs=1M count=512 - create a 512 MiB swap at /
4. mkswap /swap - Configure the swap file - note we'll edit fstab instead of using swapon to activate it on next boot
5. mkdir /mnt/h - Make a new mount point
6. fdisk -l - Note down which partition is for what. For instance:
   

   Disk /dev/sda: 60.0 GB, 60000000000 bytes
   255 heads, 63 sectors/track, 7294 cylinders
   Units = cylinders of 16065 * 512 = 8225280 bytes
   Disk identifier: 0x########
   
      Device Boot      Start         End      Blocks   Id  System
   /dev/sda1               1        2089    16779861   83  Linux
   /dev/sda2            2090        7294    41809162+   5  Extended
   /dev/sda5            7231        7294      514080   82  Linux swap / Solaris
   /dev/sda6            2090        7230    41295019+  83  Linux
   

7. cd /dev/disk/by-uuid/ && ls -lFa - Take note of uuids and the partitions they point to - we'll need this next. For instance:
   

   6db5296a-0bf4-4773-a9f5-22eb1abbd066 -> ../../sda1
   bc3216cf-32f3-49ff-87af-6e2fdb7ce376 -> ../../sda6
   c9e7c3c8-6f49-44be-9cdf-1387ce320b27 -> ../../sda5
   

8. cd /etc/ && cp fstab fstab.bak && vim /etc/fstab - Backup then edit fstab according to fdisk and ls output, don't forget to change the "pass" number for disk check - see man fstab for more information. The edited part of fstab should look something like:
   

   # /etc/fstab: static file system information.
   #
   # <file system> <mount point>   <type>  <options>       <dump>  <pass>
   ...
   # -------- HDD mounts --------
   # /dev/sda1 - avoid mounting it as /c ...
   UUID=6db5296a-0bf4-4773-a9f5-22eb1abbd066       /       ext3    defaults,errors=remount-ro      0       1
   # /dev/sda6 - Home drive - symbolic links are then made from /mnt/h
   UUID=bc3216cf-32f3-49ff-87af-6e2fdb7ce376       /mnt/h  ext3    defaults,errors=remount-ro      0       2
   #
   #
   #
   # -------- Swap --------
   /swap           none            swap    sw,pri=-1       0       0
   # ---- Update: 2008-05-27 @ 03:09 ----
   # You can use "pmi" to do this - you can ignore the swap partition configuration below
   # /dev/sda5 - swap partition for hibernation... - hence pri=-2
   #UUID=c9e7c3c8-6f49-44be-9cdf-1387ce320b27       none    swap    sw,pri=-2       0       0
   

   (Note I've left the other stuff like "proc", "/dev/scd*" out in the example above - keep them in fstab!)
   
   As you can see, I've set the hibernation swap partition as "pri=-2" (i.e. lower than the swap file's) in an attempt to not use it - as mentioned before, I still need a proper workaround for this... - Use "pmi" instead - see update description above.
9. vim /etc/initramfs-tools/conf.d/resume - Change the UUID to that of the new "swap" partition, e.g.:
   

   RESUME=UUID=c9e7c3c8-6f49-44be-9cdf-1387ce320b27
   

10. update-initramfs -u - Update (fixes) the boot screen
11. shutdown -r 0 - Make sure everything still works and boot screen fixed
12. Log back in as root
13. swapon -s - Check the output to ensure there is 1 mounted swap:
    

    Filename    Type       Size    Used    Priority
    /swap       file       524280  38216   -1
    

14. cp -r /home /mnt/h/home - Copy the existing home directory
15. mv /home /home_old - Make way for the new home
16. cd /mnt/ && chown -R [user]:[group] h - Makes the new user own everything in the data partition (since I'll be the only one using the data partition for now), then tweak it to your liking...
17. ln -s /mnt/h/home /home - Link the new home in - make sure you use the full path or you'll get link errors.

That's it! Simple eh? lol.

Some links to the interesting articles:

• LVM introduction article on linux.com
• Swap file addition article on linux.com
• System backup and restore using partimage (SystemRescueCd)
• Boot screen "segfault" problem resolution