LyX-Sweave on MS Windows

Jay Kerns has recently notified me that for LyX-Sweave on MS Windows one needs to add the bin directory of R installation (say contrib/extra/lyx/) to the PATH system variable. He is right, since LyX can not find R binary otherwise. On MS Windows I always add R to PATH variable, since I often use R CMD check or other commands in MS Windows Command Tool - terminal. New version of INSTALL file will be updated with this information. Goosh, administration of Linux is getting better and better in comparison to MS Windows, at least for me.

Embeding feeds on your site

See:

• http://cybernetnews.com/2008/06/01/google-lets-you-embed-feeds-on-your-site/
  
• http://www.google.com/uds/solutions/wizards/dynamicfeed.html

Program and videos from the Lush Visions Symposium - Animal Breeding Plans

Lush Visions Symposium - Animal Breeding Plans was held on April 25, 2008 at the Iowa State University, Ames, IA. Here are links to the programme and videos. Putting videos on the web is really a great thing as we can now watch conferences from home. Of course it is better to be there, but that involves costs, time, etc. I think that traking the videos and publishing them is also a great was to disseminate the work and finally, you at least know which person is behind the name.

Using optimized linear algebra libraries with R II

After installing ATLAS on my laptop, I also tried the same setup on machine in office having Intel(R) Core(TM)2 CPU 6300 @ 2x1.86GHz.

• default: user ~6.5, system ~0, elapsed, ~6.6
• Core2Duo: user ~0.8, system ~0.1, elapsed ~0.82

Whooa, that certainly is a gain!

Symmetric sparse matrices with Matrix R package

These tests are flawed! See here

I need to solve the following system of normal equations i.e. I want to calculate x from Ax=b, where A is a square symmetric positive definite matrix. Since I use R, I tried to do this with the new Matrix package. However, I wanted to test different approaches of building and storing a symmetric matrix.

## Load the package
library(Matrix)

## Create a 100 x 100 sparse matrix
n <- 100 test <- Matrix(data=0, nrow=n, ncol=n, sparse=TRUE)
## See the structure
str(test)

I will only printout, that this is a matrix of dsCMatrix class which means that this is symmetric matrix stored in column compressed format.

object.size(test)
## 1.3 kB

## Adding some values to the first row
test[1, 1:n] <- 1:n
str(test)

Class is now dgCMatrix, since the matrix is not symmetric anymore.

object.size(test)
## 2.4 kB

## Force it to become symmetric
test2 <- forceSymmetric(test)
str(test2)
## dsCMatrix --> OK
object.size(test2)
## 2.5 kB

The size practically did not change, which is good - values are stored only once.

## Adding to the first column
test[1:n, 1] <- 1:n
str(test)

The class is still dgCMatrix, which means that there is no automatic checking that the matrix is
symmetric. This makes sense, since automatic checking could be a formidable task.

object.size(test)
## 3.6 kB

That is also the reason for larger size of the object in memory.

## Convert the class
test <- as(test, "dsCMatrix")
str(test)
## dsCMatrix
object.size(test)
## 2.5 Kb

After conversion to dsCMatrix class, we again drop redundant info from one off-diagonal part of the matrix.

Now let us try some sums to see how matrices behave.

## Summation of two sparse symmetric matrices
test2 <- test + test
str(test2)
## dsCMatrix
object.size(test2)
## 2.5 kB

The symmetry is preserved as well as the size of the object in memory - the "identical" size is just because we summed the same object.

Now let us try what happens if one matrix has upper and other lower triangle filled.

##Summation of upper and lower "triangle"
n <- 100
test1 <- test2 <- Matrix(data=0, nrow=n, ncol=n, sparse=TRUE)
test1[1, 1:n] <- 1:n
test2[1:n, 1] <- 1:n
test1 <- forceSymmetric(test1)
test2 <- forceSymmetric(test2)
str(test1)
## dsCMatrix
object.size(test1)
## 2.5 kB
str(test2)
## dsCMatrix
object.size(test2)
## 1.3 kB

Whoops! these matrices are the same in terms of the content, but the second one is almost half the size. This is probably due to the column compressed format i.e. lower triangle is better suited for this. This is good to know!

## Sum them
test <- test1 + test2
str(test)
## dsCMatrix
object.size(test)
## 2.5

The size of the sum is influenced by the "big" matrix.

## Reverse the sum
test <- test2 + test1
str(test)
## dsCMatrix
object.size(test)
## 2.5 kb

The same result as before - it seem that the order does not matter!

## Now sum only the "small" matrix
test <- test2 + test2
str(test)
## dsCMatrix
object.size(test)
## 1.3 kB

Aha! Again lower triangle wins.

## Now sum only the "big" matrix
test <- test1 + test1
str(test)
## dsCMatrix
object.size(test)
## 2.5 kB

Using optimized linear algebra libraries with R

After reading blog posts by Yu-Sung and Gelman (see also comments) as well as recent additions to R on Debian page at RWiki, I decided to take a try with ATLAS on my laptop. I did not aspect much, since I use a poor IBM ThinkPad R50e with Intel(R) Celeron(R) M processor - 1400MHz. However, this laptop offers what I need when I move around, while I use a "hammer" in the office. I could not find the information about which instruction set (SSE, SSE2, ...) is appropriate for my processor. In order to figure this out and to evaluate the benefits I did:

• Launched the XUbuntu 7.10 (gutsy)
  
• Started R with: R --vanilla -q
• Used the following R script i.e. setting up the matrix and calculating its crossproduct
  

mm <- matrix(rnorm(4 * 10^6), ncol = 10^3); system.time(crossprod(mm))

• Tested packages: atlas3-base, (atlas3-sse, atlas3-sse-dev) and (atlas3-sse2, atlas3-sse2-dev), which is just a matter of installing and removing the packages via aptitude install or remove command

The results (output from system.time function - several calls) are:

• standard R installation: user ~13.4, system ~0.06, elapsed ~15.0
  
• atlas3-base: user ~4.1, system ~0.05, elapsed ~4.2
  
• atlas3-sse: user ~3.4, system ~0.03, elapsed ~3.7
  
• atlas3-ss2: user ~4.1, system ~0.03, elapsed ~4.1
  

It is clear that installing ATLAS is beneficial, but there is not much gain with specific instruction sets for my laptop. Nevertheless, I kept atlas3-sse since it was consistently showing the best performance.

Overview of scientific journals with Eigenfactor

I came across the Eigenfactor.org site, which (according to the description of the site) ranks and maps scientific knowledge. I like the display and the interactive browser --> go to mapping menu. When you click a field you can see which fields are related and on the right you get a list of top journals in the field. That is very nice and handy information. The bars at the top of the display lets you show more or less related fields.

Using Sweave with LyX customisation script

Gabor suggested I should write a short batch file or R script which would copy the various files needed to use Sweave in LyX so that one does not need to actually do it onself. That would be nice, but there are several things that I should take care of:

• I would need to find the LyX user/library directory. I do not know how to get his information in scriptable manner.

• If the preference file already exists, I would have to append new content. That is easy, but in case some definitions are already used I would first need to parse the old version of prefences. That would not be so easy.

• I would need to write a Unix shell script for Unix like systems and a batch file for MS Windows. I am not sure about Mac OS-X - I think Unix shell script should work there out of the box.

I think I will leave this step to someone else. I actually hope, that LyX developers might find a way to provide automatic customisation if R is installed on the computer. Some ideas were given here. Additionally, Gabors' {Sweave,Stangle}.bat could be used instead of {Sweave,Stangle}.sh that are shipped with R.

Using Sweave with LyX on MS Windows

Ian Holliday has informed me that my solution for using Sweave with LyX has additional requirement that I have not stated. Beside LyX & R (obviously) one needs also a variant of Unix shell. I have not realized this before since I usually use Linux, which always has installed a variant of Unix shell. Additionally, I always install a set of Unix tools (via Cygwin or MSYS) on my MS Windows boxes. Therefore, I could not realize that Unix shell is needed. However, I could have figured this requirement since R CMD Sweave and R CMD Stangle (commands that drive the weaving and the tangling process - this two commands are also inserted in LyXs' preferences file) use shell scripts Sweave.sh and Stangle.sh. I have provided additional info about this issue in INSTALL file. New version of the Rnews paper will appear soon.

Slides and videos from The 3rd International Conference of Quantitative Genetics

The 3rd International Conference of Quantitative Genetics was held at Zhejiang University, Hangzhou, China August 19 -24, 2007. The webpage of the conference is available at:

http://ibi.zju.edu.cn/icqg/

Now, slides and videos are available for download. Great!