This will most likely be the last post in this blog, as I have moved over to a site of my own hosting, which will be found at both http://n00bsys0p.co.uk.

Any of you who are subscribed to my RSS feed, will need to modify their reader to check the following URL:
http://n00bsys0p.co.uk/rss.xml

n00b

I found an interesting snippet of information today. In OSX Lion, scrolling is reversed, like we have seen on a lot of touch applications. I for one, find this form of scrolling much more natural than the standard method, particularly when using a laptop touchpad.

It is also, really easy to implement on any Linux system. I can’t claim credit for this entirely. I have paraphrased the necessary Linux terminal commands from this python app.

The only commands you will need are `xinput`, `grep` and `sed`. The original code used gawk to separate text fields, but I thought I’d use sed, as it gives me a chance to show you some more regex pattern matching, which I have decided will be a running theme through as many of my posts as possible.

The first thing to do is to extract the slave input device’s xinput id. This is done using `xinput list`, and grep/sed to extract the exact number we want. Here’s the command:
xinput list | egrep "slave.*pointer" | grep -v XTEST | sed -e 's/^.*id=//' -e 's/\s.*$//'
This extracts the id number by first matching the regex “slave.*pointer”, which means match the letters “slave”, then anything, then the letters “pointer”. It then makes sure that our output ignores any line which contains the characters “XTEST”. Now it has the line it wants, it needs to extract the correct field. As I mentioned, I’ve used sed instead of awk (which is possibly slightly more readable – I never said my option was the better one). I use sed -e, so I can search and replace multiple regex in the order I specify. The first sed expression is to match anything from the start of the line (^.*), until it reaches the end of the characters “id=”, and replace them with nothing (//). The second sed expression starts matching at the first whitespace found (\s), and continues on until the end of the line ($). It is not necessary to include the $, as sed will default to matching until the end of the line anyway. I have included it only for the purposes of explanation.

Now we have our id number (in my case “11″), we can find out what the current input order is. For this, we use the following command, replacing {idnum} with the number we extracted before:
xinput get-button-map {idnum}
This will most likely output a sequence of numbers, in order. If the numbers are in order, this means that the scrolling is not reversed. What we need to do to reverse the scrolling, is reverse numbers 4 and 5, which represent the scroll wheel. We do this by using a very similar command to the one we just used. In this, replace {order} with the same sequence you saw in the output of `xinput get-button-map`, with 4 and 5′s order reversed, and {idnum} with our extracted input id again:
xinput set-button-map {idnum} {order}
That’s it done! Go to any application with a scroll-wheel activated scroll function, and watch it in action! To switch back, just repeat the last command with 4 and 5 back in the correct order!

To simplify this whole process, check out the Python code at the link I mentioned previously. You may or may not be able to use it as it is, because it uses PyGTK, and therefore KDE users will have to port it to PyKDE in order to use it. It’s pretty simple as long as you have any programming experience. Even if not, I encourage you to have a look – Python’s an amazing language!

Anything you want to know in more detail, or any corrections/improvements you would like to suggest, please direct it to the comments section, and I’ll do my best to get back to you as soon as I can!

n00b

Got an interesting BASH script for you this time. I regularly use a BASH script to clone a HDD using cp, and other than perhaps using “-v”, I couldn’t think of a way to give it a GUI. A little research and thought into it came up with the following result:

#!/bin/bash
SRC="/mnt"
TGT="/mnt2"
echo "Copying $SRC to $TGT"
[[ -w /tmp/fifo ]] && rm /tmp/fifo
mkfifo /tmp/fifo
exec 5<>/tmp/fifo
(cat <&5 | dialog --guage "Copying OS and data files from source to target HDD" 7 30) &
CPPID=$!
TOTAL=$(df | grep $SRC | awk '{print $3}' | tr '\n' '+' | sed 's/.$/\n/' | bc)
while true; do
    COPIED=$(df | grep $TGT | awk '{print $3}' | tr '\n' '+' | sed 's/.$/\n/' | bc)
    echo 100*$COPIED/$TOTAL | bc >&5
    if [ ! "$(ps aux | awk '{print $2}' | grep $CPPID)" ]; then
        break
    fi
    sleep 10
done
5<&-
reset

Apologies for the truly horrible formatting. WordPress’s code tags aren’t the best! Now I’ll do my best to explain each stage…

First up, the script sets variables for the locations to copy from and to. These must be mounted volumes in this example, as we use `df` to detect them. This wouldn’t be too difficult to convert to use some other method of working out the size, such as `du`, if you aren’t using a mounted volume.

We then create a pipe called fifo in /tmp. This is how we will feed data to `dialog`.

Now we connect file descriptor 5 to the pipe, to facilitate the connection to our ncurses dialog, then pipe FD 5 into the `dialog` program, yet again forked into the background.

Next, we find the total size of the source partition, and set it to the variable $TOTAL.

Here we set the copy process off, and fork it into the background, so the script will jump through to the next section, which regularly updates our instance of `dialog`.

Here’s the interesting bit… We run a while true loop, which every 10 seconds, checks the cumulative size of the target partition ($COPIED), and compares it to $TOTAL by converting it to a percentage, using bc. This number is then fed into the pipe, which we have connected to dialog. Each loop, we also check to see if the copy process is still running, and if not, we break out of the loop.

To clean up, we close the file descriptor (5<&-), and use `reset`, as dialog has a nasty habit of breaking the terminal, and `clear` doesn't fix it.

As always, if you have any questions, tips, improvements or corrections, just direct them straight to the comments!

n00b

Recently, I needed to find out to which /dev entries a particular driver was attached. For this, I did the following.

First, navigate to /sys. This is where kernel information is stored about devices and drivers. Under here, you’ll notice folders like “block” (for storage devices) and “bus” for pluggable items such as USB devices and PCI cards.

We are going to be looking in “bus”, as the device I am looking for is connected via USB. From /sys/bus, navigate to usb/drivers, and you will see a list of all drivers currently attached to devices connected via USB.

From here, I needed the ftdi_sio driver, so I cd’d into the folder, and had a look around. Nothing which makes much sense so far, but bear with me. There will be one or multiple symlinks in this folder, which consists of numbers, hyphens, colons and dots. These link to each device’s sysfs folder. Run the command:
ls [0-9]*/

This will look in any folder which starts with a number (in this case our two device folders). Within these folders should be your information. In my case, there was a folder in each of them called ttyUSB? (where the ? was a number).

Now I knew that I just had to connect my program to /dev/ttyUSB? to communicate with the devices.

If there’s anything anybody doesn’t understand about this, feels that I could explain better, or simply wants to know more about, then feel free to let me know via the comments.

Have a great day!

n00b

The title’s pretty self explanatory, so I’ll just dive right in…

I needed to access data on a VM which had no network connectivity, and therefore needed a way to mount partitions from the host (Ubuntu 11.04 in this case).

After a little research, I found this article. It explains in simple steps how to go about this. Here’s my much shortened (but far less complete) version:

First, you need to have the package qemu-kvm installed. This can be done in Ubuntu by running the following command:
sudo apt-get install qemu-kvm

Next, load the nbd kernel module.
sudo modprobe nbd

Now you’re ready to make the partitions available to the host OS by running:
sudo qemu-nbd -c /dev/nbd0 {VDI FILE}

This will allow you to access the partitions as /dev/nbdp{number}. For example, to mount the first partition of the VDI in /mnt, you would run the following command:
sudo mount /dev/nbd0p1 /mnt.

For a whole bunch more information on this, click the link to the original article I mentioned before.

Any questions, just ask!

n00b

I had a problem starting Google Chrome on one of my servers today. It started with a whole bunch of errors, the most pronounced of which was as follows:

Failed to create a ProcessSingleton for your profile directory.
This means that running multiple instances would start multiple
browser processes rather than opening a new window in the
existing process. Aborting now to avoid profile corruption.

Try as I might, I couldn’t sort the problem. I tried deleting the config folder (/home/user/.config/google-chrome), and other standard random chances. The fix, however, was absurdly simple. I happened to use “df -h” during my research, and found that my / partition was completely full. I went into /tmp, found about 4Gb of useless cruft, and promptly deleted it. Problem sorted.

n00b

Just thought I’d post this little nugget of information, as it’s taken me a little while to work out how to do it. Before I start, I’ll mention that downloading copyrighted material may well be extremely illegal where you live, so make sure you only use this technique to download videos which contain your work entirely. This can be useful if you have, for example, lost your user details for a video upload site of some variety, and there is no facility to retrieve your videos without it.

Anyway, enough with the disclaimer, on with the hack…

As you may know, Flash Player used to store the temporary stream files in /tmp. They switched from this at the end of last year to storing them in the specific browser’s cache folder for reasons unbeknownst to the masses. This still made it pretty easy to locate any file you may have wished to download. After a recent update, I found that I was unable to locate the temporary caching folder anywhere.

My first step was to load the video in a browser, and check the output of the following command:
lsof | grep -i flash

This came out with a predictable, and very useful single line:
plugin-co 25646 n00b 17u REG 8,2 31286337 787220 /tmp/FlashXXepl6qa (deleted)

This showed me that there was a file descriptor open to a “deleted” file, /tmp/FlashXXepl6qa. I’m no programmer, so I have no idea how this works, but it seems that it’s adding chunks of data to this file descriptor (I imagine stored in RAM), while the file itself is technically nonexistent.

UPDATE (24/04/11):

Thanks to reader Raven Morris, I have some more information about what exactly is happening. The reason is that Linux does not have file locking, like Windows. Windows programs, when they open a file, will lock it, so no other programs can access it. In Linux, when a file is deleted, the operating system will keep track of it until there are no programs which have it open. Once the last program closes the last file descriptor to the file in question, the file becomes unrecoverable without relevant forensic tools, but until then, there is a record of it within the /proc/*/fd directory tree.

Anyway, the second field of the output tells us which process currently has the file descriptor open, and the fourth tells us which number the file descriptor has taken. This is all we need to access the file itself.

If you navigate to your /proc folder, you will see a bunch of folders all named numerically, including a folder which matches the number in the second field. Now navigate to this folder, then its subfolder “fd”. In this folder, you will see a whole selection of numbers. These relate to the file descriptors themselves. Run “ls -l” in this folder, and you will see that each of these numbers is linked to either pipes, sockets or files. Within this, the number from the fourth field will be symbolic linked to the /tmp/Flash* file we found before. To test that this is the right file, you can run it through mplayer or vlc (“mplayer filedescriptornumber”/”vlc filedescriptornumber”). If you’re having trouble finding the filename, try “ls -l | grep Flash”, as pointed out by reader Nobi.

Once the video is fully streamed, you can use a simple “cp” to copy the file from the file descriptor to a real location on your hard disk. (“cp filedescriptornumber ~/Videos/filename.flv”).

Another way to locate these files is to use the following command:
stat -c %N /proc/*/fd/* 2>&1|awk -F[\`\'] '/Flash/{print$2}'

I encourage you to play with the various sections of it to see how it works. If you’re having trouble getting it to work, make sure you have the apostrophes, backticks and spaces in the correct location.

Reader Robert submitted the following BASH alias, to automate the whole process. Here’s the script to insert into your bashrc:

cpflashvideo() { cd /proc/`lsof | awk ‘/Flash/&&/plugin-co/’ | awk //’{printf “%s”, $2}’`/fd/ && cp `ls -al | grep ‘\(deleted\)’ | awk ‘//{printf “%s”, $8}’` $* && cd – > /dev/null; }

UPDATE (13/07/11):

A couple of my friends were pondering the question of this, and came up with the following for those of you who have a few to many tabs open at one time… I’ve included a couple of versions, to show how this could be done using entirely awk, or a mixture of awk and sed.

This first one uses regexp matching in awk to ensure that the letter is stripped from the end of the fourth field:
for FILE in $(lsof -n | grep "Flash.*deleted" | awk '{printf "/proc/" $2 "/fd/"; sub(/[a-z]+/,"",$4); print $4}'); do
cp $FILE $(mktemp -u --suffix .flv $HOME/Videos/Video-XXX)
done

The next uses sed to strip off the last character of the fourth field:
for FILE in $(lsof -n|grep .tmp.Flash | awk '{print "/proc/" $2 "/fd/" $4}' | sed 's/.$//'); do
cp -v $FILE $(mktemp -u --suffix=.flv --tmpdir=$HOME/Videos/)
done

I hope this post is as interesting for others as I found it myself, and as always, direct any questions to me in the comments!

If you are a Spanish speaker, and have had trouble understanding this, Taringa.net user racsoprieto has translated the basics of the article here.

n00b

Found another link for you guys to peruse at your leisure. I have found BASH scripting to be pretty much the most useful thing to know when administering Linux systems. Debugging a script which won’t play fair can be really time-consuming, and these tricks might just save you a heck of a lot of time!

Anyway. Here’s the link.

n00b

Here’s the link to the page.

n00b

Just a link here for you. I’ve been working closely with a programmer, who has used Kylix 3 for the last 6 years, and needed to change to an IDE which is being actively developed. This required switching to the Lazarus Freepascal IDE. As all of the project files used Indy components, I was asked to find a way to install the Indy component into Lazarus. A bit of trial and error, followed by some excellent tips from the following link, and I was well away!

Anyway, here’s the link… http://forum.lazarus.freepascal.org/index.php?topic=5096

Hope it’s of use!

n00b