cpumemlog: Monitor CPU and RAM usage of a process (and its children)

Long time no see ...

Today I pushed the cpumemlog script to GitHub https://github.com/gregorgorjanc/cpumemlog. Read more about this useful utility at the GitHub site.

Functions dim, nrow, and ncol for shell

Ever wanted to find out quickly the number of rows and/or columns in file directly from terminal. There are many ways to skin this cat. Here is what I used for number of rows for quite a while:

wc -l filename

What about number of columns? "Easy", just combine head and awk commands:

head -n 1 filename | awk '{ print NF }'

not a big problem (there are likely better ways to do this), but is long and tedious.

I got sick of typing commands above and assembled them in three easy to use function with R-like names: nrow, ncol, and dim. Functions are simply a collection of above ideas and assume that the file is of "rectangular shape", i.e., a table, a matrix, etc.

dim()
{
  for FILE in $@; do
    NROW=$(nrow $FILE | awk '{ print $1}')
    NCOL=$(ncol $FILE | awk '{ print $1}')
    echo "$NROW $NCOL $FILE"
    unset NROW NCOL
  done
}
export -f dim

nrow ()
{
  for FILE in $@; do
    wc -l $FILE
  done
}
export -f nrow

ncol ()
{
  for FILE in $@; do
    TMP=$(head $FILE -n 1 | awk '{ print NF }')
    echo "$TMP $FILE"
    unset TMP
  done
}
export -f ncol

Add these files to your .bashrc or .profile or something similar and you can now simply type:

nrow filename

ncol filename

dim filename

A simple test:

touch file.txt

echo "line1 with four columns" >> file.txt

echo "line2 with four columns" >> file.txt

nrow file.txt

2 file.txt

ncol file.txt

4 file.txt

dim file.txt

2 4 file.txt

AWK one-liners

See here.

Compress PDF

See nice description here. In short using the command bellow on my Linux box (Windows users see PDFCreator) allowed me to reduce scanned color PDF for a factor of 4 to 5, e.g., from ~14 MB to  ~3 MB. Supposedly PDFCreator also 

gs -sDEVICE=pdfwrite -dMaxSubsetPct=100 -dPDFSETTINGS=/ebook -sOutputFile="out.pdf" -dNOPAUSE -dBATCH "in.pdf"


Update (2012-01-16): you can get a nice shell script called compressPDF.sh that saves some typing and can process multiple files, e.g., 


compressPDF.sh file.pdf

Sweave.sh update

A local version of Sweave.sh has been updated:

• fixed a buglet that caused weaving the file twice when --weaver option was used --> this led to a change how caching is now invoked (--cache invokes cacheSweave package, while --cache --weaver or --weaver invokes weawer package)
• quoting *APP variables to ensure that things work in case of "bad" filenames, i.e., spaces in filenames etc.
• small changes in the documentation

I will take this opportunity to show how PDFAPP and PSAPP environmental variables can be used "open" options. This is an example from the help, which shows that these two variables need to be exported in order to have any effect. I am still fiddling whith how to provide any meaningfull defaults. My defaults are 'acroread' for PDF and 'gv' for Postscript, but I am happy to set any other more meaningfull defaults if there will be enough demand!

# Create PDF via the texi2dvi (latex) tool and open
# a produced file
Sweave.sh -otld file.Rnw

# Create PDF via the texi2dvi (latex) tool and open
# a produced file with a "non-standard" viewer
Sweave.sh -otld=acrobat file.Rnw

# ... or
export PDFAPP=acrobat
Sweave.sh -otld file.Rnw

This is an example of launcing Acrobat Reader installed on MS Windows, while Sweave.sh is launched within Cygwin X-terminal:

export PDFAPP=/cygdrive/c/Program\ Files/Adobe/Reader\ 9.0/Reader/AcroRd32.exe
Sweave.sh -otld file.Rnw

Using Makefile to ease the repeated compilaton of LaTeX source

I like LaTeX, but it can be tedious if you want "instant" check of the produced output. Usually, there is no need to check the output very often - basically you can write the whole article/report, ... and then compile the LaTeX source. However, when I create a presentation, I often check what does it look like. LyX can also be very handy with "instant" checking! Two days ago I gave a LaTeX Beamer package a try and I use the following Makefile for the process of several compilations of the Sweave source file (fusing LaTeX for the creation of the presentation and R for doing the computations and plotting) - here are the PDF and the source file.

You need to be carefull with formatting of the Makefile, i.e., the lines bellow the target (say line two) needs to start with a TAB and not with spaces!

all:
      Sweave.sh --latex2pdf talk.Rnw; make rm

tex: # Sweave --> LaTeX
      Sweave.sh talk.Rnw; make rm

rm: # Remove some other files
      rm -f .pdf Rplots.pdf *.out *.nav *.snm *.log *.tex *.aux *.toc

Using this Makefile I only needed to type make or better pressing the "up" key to repeat the previous (make) command in the terminal to compile the Sweave source file.

Unix tools in MS Windows command terminal (prompt)

Previously, I wrote about using Rtools when one already has Cygwin installed on a MS Windows machine. The "solution" was to avoid putting the Cygwin into the PATH variable and to create a new script which adds Rtools to the PATH variable on the fly. This (the first thing) eventually means that I was not able to use Unix tools (that are installed with Cygwin) in the MS Windows command terminal (prompt). For example, I was prepairing the presentation using the LaTeX Beamer package and I usually use a Makefile to ease the repeated compilation. In order to be able to use make and other unix shell tools, I created another startup script (CMD_Cygwin.bat) with the content as shown bellow.

rem --- Add Cygwin and current folder to the PATH ---
set PATH=.;%PATH%;c:\cygwin\bin;c:\cygwin\usr\bin;c:\cygwin\sbin;c:\cygwin\bin;c:\cygwin\usr\local\bin

rem --- Start the Command Prompt ---
cmd

Rtools and Cygwin on MS Windows

Duncan Murdoch provides Rtools which ease the installation of tools that are needed to do R package development/testing on MS Windows. The Rtools is a collection of various tools. However, if you also use Cygwin on MS Windows, you can expect problems since Rtools also includes some tools from Cygwin. The problem is the version collision of fundamental Cygwin libraries. Basically, this means that you will not be able to use Cygwin and if you have C:/Cygwin/bin in the PATH envorinmental variable, you can expect problems also in the Command Prompt. You can try to fuse both "worlds" as described in the documentation, but is seems tricky. I just used the following to make my life easier:

• Install Rtools and do not modify the PATH envorinmental variable --> this means that you will still be able to use Cygwin without problems
  

• Create a BAT script (say on the Desktop) with the following content

rem --- Add RTools to the PATH ---
set PATH=c:\Programs\R\Rtools\bin;c:\Programs\R\Rtools\perl\bin;c:\Programs\R\Rtools\MinGW\bin;%PATH%

rem --- Start the Command Prompt ---
cmd

• Start the script with the double click and you will get a working environment for R package development/testing
  

Btw. under Linux or Mac you can test the functionality of R package under MS Windows using the win builder at http://win-builder.r-project.org provided by Uwe Ligges