Bloglines

I use bloglines as an RSS reader and today I got the message that they ar fixing something. This was the note:

Hi,

I'm the Bloglines Plumber. Bloglines is down for a little fixer upper. We will be back shortly. Bloglines will be all better when I'm done with it.

Thanks,

The Bloglines Plumber

OK. But the funny part is this picture

GeneticsPed in BioC

We have finally switched to BioC SVN for development of GeneticsPed package. Now we have results of daily builds and checks at different platforms. Neat.

Sweave (bash) shell script

I uploaded my Sweave.sh (bash) shell script to CRAN. It is available here.

Installing PyPedal under Debian

Here is my initial attempt about installing PyPedal on my Debian box. ## mark my comments and the rest are shell commands:


## Switch to root user
su -

## First install non-python stuff in Debian
##
## Software Debian package
## Graphviz graphviz
## SQLite sqlite3
##
## Note that there are sqlite and sqlite3 in Debian and
## I took the new one

aptitude install graphviz sqlite3

## Now we get to python modules that are
## already packaged in Debian
## Module Debian package
## elementtree python-elementtree
## matplotlib python-matplotlib
## NetworkX python-networkx
## NumPy python-numpy
## PIL python-imaging
## pydot python-pydot
## PyGraphviz python-pygraphviz
## pyparsing python-pyparsing
## pysqlite python-pysqlite2
## ReportLab python-reportlab
## setuptools python-setuptools

## Again, there are python-sqlite (for SQLite 2),
## python-pysqlite1.1 and python-pysqlite2
## (both for SQLite 3)

aptitude install python-elementtree\
python-matplotlib \
python-networkx \
python-numpy \
python-imaging \
python-pydot \
python-pygraphviz \
python-pyparsing \
python-pysqlite2 \
python-reportlab \
python-setuptools

## Other Python modules not in Debian

## According to Python inst. doc especially this section
## we should use the following way to install
## non-debianized modules
##
## python setup.py install --prefix=/usr/local

## PythonDoc
## Site: http://effbot.org/zone/pythondoc.htm
## At the time of writting I issued the following

wget http://effbot.org/downloads/pythondoc-2.1b6-20060406.tar.gz

tar -xzvvf pythondoc-*
cd pythondoc-*
python setup.py install --prefix=/usr/local
cd ..
rm -R -f pythondoc-*

## testoob
## Site: http://testoob.sourceforge.net/
## At the time of writting I issued the following

wget http://kent.dl.sourceforge.net/sourceforge/testoob/testoob-1.13.tar.gz

tar -xzvvf testoob-*
cd testoob-*
python setup.py install --prefix=/usr/local
cd ..
rm -R -f testoob-*

## PyPedal
## Site: http://pypedal.sourceforge.net/
## At the time of writting I issued the following

wget http://puzzle.dl.sourceforge.net/sourceforge/pypedal/PyPedal-2.0.0b21.tar.gz

tar -xzvvf PyPedal-*
cd PyPedal-*
python setup.py install --prefix=/usr/local

## Upps! The last one does not install cleanly as
## reported on SF mailist list for PyPedal.

Wich forge for R should I use

When developing software it is very handy to have some version control companied with e-mail alerts on commits, maybe a forum etc. I have used SourceForge for this for some of my packages i.e. in r-genetics and r-ggorjan "projects". I have even persuaded some of "R-BUGS" people to initiate bugs-r (I have not done any development recently there which is a pitty). What I miss with SourceForge is daily checks with R CMD check etc. I have written some scripts to do this and even played with (semi-)automatic publication on my homepage (follow software --> R). This was very tedious and it never worked in the way I wanted. CRAN provides daily check for all packages but only for those published at CRAN. BioC has pushed this idea a bit further with SVN and daily checks of code in SVN against several R versions (the last feature is also available by CRAN).

I noticed in the past a note by Gregory Warnes on site ala "R-Forge" - I do not remember the details. Now I have discovered r-forge.net by Simon Urbanek which seems to be great for what I want. I have browsed the site and wanted to join there, but then I discover the following note on that site:

"Note: This site is not related (except in spirit) to the R-Forge project at the WU Wien which can be found at http://r-forge.wu-wien.ac.at/."

Pheu. There seems to be another site with this functionality! Given that this site has r-project.org domain I guess that this will become official forge for R. I will ask on R-devel list.

RJDBC

Just found an R package RJDBC that allows connecting R to any database that has JDBC driver. This might be even easier than with ODBC as I have previously described for Oracle.

For example, see this blog post.

Tips in ESS mode for R functions

Sven has written handy function that shows arguments when you start typing function( . This is really handy and it was available only in some GUIs. There is some discussion here.

Bayesian inference on quantitative genetic (animal) model in BUGS & R

There is a paper by Lars Damgaard in Journal of Animal Science showing how to fit quantitative genetic (animal) model in WinBUGS and this of course extends also to other BUGS versions. This is really great. It is based on decomposition of additive relationship matrix A=TDT'. I will try to provide functions to ease the preparation of needed data for BUGS in my R package GeneticsPed. This will ease the exploration and first stage fitting of models on small or medium datasets. As noted by Damgaard, BUGS will not serve for genetic evaluation. I wonder if one can gain in speed with using Umacs or JAGS (site seems to be down at the moment).

Bayesian methods in animal breeding and theory

I have finally read the seminal paper by Gianola and Fernando (1986). I had it in my pile of must read papers for some time now. I guess around two years. Phuh! I am glad I read it since it is nice to see how thing conceptually have not changed a lot from that time. It is 20 year from then, but you can still find the same stuff is in the papers today. The only difference is that now problems are tackled with use of MCMC machinery, while the concept is the same. Is this a sign of ...? I also liked the paper since a lot of issues that you find in many different places are in one place. I would rate is as a must read paper for anyone working in field of animal breeding. I wonder if it can be recommended as an introductory paper to Bayesian way for animal breeding problems. I had some experience with Bayesian approach and have read a lot of literature on this topic so my impressimon might be distorted. Anyway, I found it nice to refresh the setup and to take a journey from Bayes theorem, posteriors, priors, likelihood, sequential inference of breeding values, sample data domination, Bayesian asymptotic theory, integration of nuisance parameters, a pinch of Jeffrey's priors, inference of breeding values with informative priors, a touch to Henderson's mixed model equations when priors are uniformative and variance components known, biased but better (in the sense of mean-square error) estimators, selection and assortative mating, Bayesian view of selection index (best linear predictor) and best linear unbiased predictor, nonlinear merit functions, nonlinear models, Bayesian estimation of (genetic) parameters - variances. Oddly, paper finishes with "a cut" - no real end. Weird. Blasco is more recent example of such a paper. It covers quite a lot of topics as Gianola and Fernando (1986) - I told you thing are repeating, but also adding comparison with frequentist/likelihood approach and is writen in a way that also people that are not everyday into the statistics.

Blasco, A. (2001) The {Bayesian} controversy in animal breeding. Journal of Animal Science. 79(8):2023--2046 http://jas.fass.org/cgi/reprint/79/8/2023

Gianola, D. and Fernando, R. L. (1986) Bayesian methods in animal breeding and theory. Journal of Animal Science, 63:217-244

Oracle instantclient + ODBC on Debian GNU/Linux

I have finally managed to properly install Oracle's instantclient and ODBC drivers on my Debian GNU/Linux box. Here is my mini HOWTO. Lines that start with # are comments.

# Change to root (as usual)

su -

# Go to link bellow and register

http://www.oracle.com/technology/software/tech/oci/instantclient/htdocs/linuxsoft.html

# Download

Instant Client Package - Basic: All files required to run OCI, OCCI, and
JDBC-OCI applications

oracle-instantclient-basic-10.2.0.3-1.i386.rpm

Instant Client Package - SDK: Additional header files and an example makefile for developing Oracle applications with Instant Client

oracle-instantclient-devel-10.2.0.3-1.i386.rpm

Instant Client Package - JDBC Supplement: Additional support for XA,
Internationalization, and RowSet operations under JDBC

oracle-instantclient-jdbc-10.2.0.3-1.i386.rpm

Instant Client Package - SQL*Plus: Additional libraries and executable for
running SQL*Plus with Instant Client

oracle-instantclient-sqlplus-10.2.0.3-1.i386.rpm

Instant Client Package - ODBC: Additional libraries for enabling ODBC applications

instantclient-odbc-linux32-10.2.0.3-20061115.zip

# Convert rpm to deb

alien oracle-instantclient-*

# Install

dpkg -i oracle-instantclient-*deb

# Remove deb and rpm files

rm -f oracle-instantclient-*rpm oracle-instantclient-*deb

# Installed files are

dpkg -L oracle-instantclient-basic
/usr
/usr/share
/usr/share/doc
/usr/share/doc/oracle-instantclient-basic
/usr/share/doc/oracle-instantclient-basic/copyright
/usr/share/doc/oracle-instantclient-basic/changelog.Debian.gz
/usr/lib
/usr/lib/oracle
/usr/lib/oracle/10.2.0.3
/usr/lib/oracle/10.2.0.3/client
/usr/lib/oracle/10.2.0.3/client/bin
/usr/lib/oracle/10.2.0.3/client/bin/genezi
/usr/lib/oracle/10.2.0.3/client/lib
/usr/lib/oracle/10.2.0.3/client/lib/libclntsh.so.10.1
/usr/lib/oracle/10.2.0.3/client/lib/libnnz10.so
/usr/lib/oracle/10.2.0.3/client/lib/libocci.so.10.1
/usr/lib/oracle/10.2.0.3/client/lib/libociei.so
/usr/lib/oracle/10.2.0.3/client/lib/libocijdbc10.so
/usr/lib/oracle/10.2.0.3/client/lib/ojdbc14.jar

dpkg -L oracle-instantclient-devel
/usr
/usr/share
/usr/share/doc
/usr/share/doc/oracle-instantclient-devel
/usr/share/doc/oracle-instantclient-devel/copyright
/usr/share/doc/oracle-instantclient-devel/changelog.Debian.gz
/usr/share/oracle
/usr/share/oracle/10.2.0.3
/usr/share/oracle/10.2.0.3/client
/usr/share/oracle/10.2.0.3/client/cdemo81.c
/usr/share/oracle/10.2.0.3/client/demo.mk
/usr/share/oracle/10.2.0.3/client/occidemo.sql
/usr/share/oracle/10.2.0.3/client/occidemod.sql
/usr/share/oracle/10.2.0.3/client/occidml.cpp
/usr/include
/usr/include/oracle
/usr/include/oracle/10.2.0.3
/usr/include/oracle/10.2.0.3/client
/usr/include/oracle/10.2.0.3/client/nzerror.h
/usr/include/oracle/10.2.0.3/client/nzt.h
/usr/include/oracle/10.2.0.3/client/occi.h
/usr/include/oracle/10.2.0.3/client/occiAQ.h
/usr/include/oracle/10.2.0.3/client/occiCommon.h
/usr/include/oracle/10.2.0.3/client/occiControl.h
/usr/include/oracle/10.2.0.3/client/occiData.h
/usr/include/oracle/10.2.0.3/client/occiObjects.h
/usr/include/oracle/10.2.0.3/client/oci.h
/usr/include/oracle/10.2.0.3/client/oci1.h
/usr/include/oracle/10.2.0.3/client/oci8dp.h
/usr/include/oracle/10.2.0.3/client/ociap.h
/usr/include/oracle/10.2.0.3/client/ociapr.h
/usr/include/oracle/10.2.0.3/client/ocidef.h
/usr/include/oracle/10.2.0.3/client/ocidem.h
/usr/include/oracle/10.2.0.3/client/ocidfn.h
/usr/include/oracle/10.2.0.3/client/ociextp.h
/usr/include/oracle/10.2.0.3/client/ocikpr.h
/usr/include/oracle/10.2.0.3/client/ocixmldb.h
/usr/include/oracle/10.2.0.3/client/odci.h
/usr/include/oracle/10.2.0.3/client/oratypes.h
/usr/include/oracle/10.2.0.3/client/ori.h
/usr/include/oracle/10.2.0.3/client/orid.h
/usr/include/oracle/10.2.0.3/client/orl.h
/usr/include/oracle/10.2.0.3/client/oro.h
/usr/include/oracle/10.2.0.3/client/ort.h
/usr/include/oracle/10.2.0.3/client/xa.h
/usr/lib
/usr/lib/oracle
/usr/lib/oracle/10.2.0.3
/usr/lib/oracle/10.2.0.3/client
/usr/lib/oracle/10.2.0.3/client/lib
/usr/lib/oracle/10.2.0.3/client/lib/libclntsh.so
/usr/lib/oracle/10.2.0.3/client/lib/libocci.so

dpkg -L oracle-instantclient-jdbc
/.
/usr
/usr/share
/usr/share/doc
/usr/share/doc/oracle-instantclient-jdbc
/usr/share/doc/oracle-instantclient-jdbc/copyright
/usr/share/doc/oracle-instantclient-jdbc/changelog.Debian.gz
/usr/lib
/usr/lib/oracle
/usr/lib/oracle/10.2.0.3
/usr/lib/oracle/10.2.0.3/client
/usr/lib/oracle/10.2.0.3/client/lib
/usr/lib/oracle/10.2.0.3/client/lib/libheteroxa10.so
/usr/lib/oracle/10.2.0.3/client/lib/orai18n.jar

dpkg -L oracle-instantclient-sqlplus
/.
/usr
/usr/share
/usr/share/doc
/usr/share/doc/oracle-instantclient-sqlplus
/usr/share/doc/oracle-instantclient-sqlplus/copyright
/usr/share/doc/oracle-instantclient-sqlplus/changelog.Debian.gz
/usr/bin
/usr/lib
/usr/lib/oracle
/usr/lib/oracle/10.2.0.3
/usr/lib/oracle/10.2.0.3/client
/usr/lib/oracle/10.2.0.3/client/bin
/usr/lib/oracle/10.2.0.3/client/bin/sqlplus
/usr/lib/oracle/10.2.0.3/client/lib
/usr/lib/oracle/10.2.0.3/client/lib/glogin.sql
/usr/lib/oracle/10.2.0.3/client/lib/libsqlplus.so
/usr/lib/oracle/10.2.0.3/client/lib/libsqlplusic.so
/usr/bin/sqlplus

# Configuration

# Create a general link so that env. variables bellow can be more general
# between different versions

cd /usr/lib/oracle/
ln -s 10.2.0.3/ default

# Add the following lines in '/etc/profile' so that all users will benefit

echo 'export SQLPATH=/usr/lib/oracle/default/client/lib' >> /etc/profile
echo 'export LD_LIBRARY_PATH="$LD_LIBRARY_PATH:$SQLPATH"' >> /etc/profile

# LD_LIBRARY_PATH can be clobbered by unknown system to me and it is
# probably nicer to add library location via the following way, which will
# work out of the box for all users.

echo '/usr/lib/oracle/default/client/lib' >> /etc/ld.so.conf
ldconfig

# Set TNS_ADMIN, which should point to map with 'TNSNAMES.ORA' file. Hmm,
# where do you get that file? You should ask database administrator. I got
# it from him and put it via commands bellow. Additionally, I also needed
# file 'SQLNET.ORA' in same place!

cd /usr/lib/oracle/
mkdir -p network/admin
mv TNSNAMES.ORA SQLNET.ORA /usr/lib/oracle/network/admin/.
## keep in uppercase !!!!
echo 'export TNS_ADMIN=/usr/lib/oracle/network/admin' >> /etc/profile

# Login to database with

sqlplus username/password@//machineName:port/databaseName

# Additionally, ODBC drivers where not available as RPM package so I need
# to do installation by hand.

# Unpack
unzip instantclient-odbc-linux32-10.2.0.3-20061115.zip
Archive: instantclient-odbc-linux32-10.2.0.3-20061115.zip
inflating: instantclient_10_2/ODBCRelnotesJA.htm
inflating: instantclient_10_2/ODBCRelnotesUS.htm
inflating: instantclient_10_2/ODBC_IC_Readme_Linux.html
inflating: instantclient_10_2/libsqora.so.10.1
inflating: instantclient_10_2/odbc_update_ini.sh

# Move files to proper place
mv instantclient_10_2/* /usr/lib/oracle/default/client/lib/.
cd /usr/lib/oracle/default/client/lib/.

# Create link - just to be sure
ln -s libsqora.so.10.1 libsqora.so

# We need system ODBC stuff
aptitude install odbcinst1

# Issue ODBC configuration via available shell script. I had a problem to
# understandwhat and where is driver manager directory. Looking in install
# script revealed that this is a root directory, since I should pass
# something like $DM_HOME/etc/odbc.ini. Since 'odbc.ini' is in '/etc',
# driver manager home is just '/'.
chmod a+x odbc_update_ini.sh
./odbc_update_ini.sh /

# Now let's take a look
cat /etc/odbcinst.ini

[Oracle 10g ODBC driver]
Description = Oracle ODBC driver for Oracle 10g
Driver = /usr/lib/oracle/10.2.0.3/client/lib/libsqora.so.10.1
Setup =
FileUsage =
CPTimeout =
CPReuse =

cat $HOME/.odbc.ini

[_yourDSN_]
Application Attributes = T
Attributes = W
BatchAutocommitMode = IfAllSuccessful
BindAsFLOAT = F
CloseCursor = F
DisableDPM = F
DisableMTS = T
Driver = Oracle 10g ODBC driver
DSN = OracleODBC-10g
EXECSchemaOpt =
EXECSyntax = T
Failover = T
FailoverDelay = 10
FailoverRetryCount = 10
FetchBufferSize = 64000
ForceWCHAR = F
Lobs = T
Longs = T
MetadataIdDefault = F
QueryTimeout = T
ResultSets = T
ServerName = _yourServerName_
SQLGetData extensions = F
Translation DLL =
Translation Option = 0
DisableRULEHint = T
UserID = _yourUserID_

# Exit root account
exit

# Copy roots' '.odbc.ini' to users home and modify it that it matches to your
# database.
cat /root/.odbc.ini >> .odbc.ini

# You can play with graphical tools

DataManager &
DataManagerII &
OBCDConfig & # I got crash here when I wanted to connect to database

# Or with command line

isql -v _yourDSN_ _yourUserName_ _yourPassword_

# Now slick stuff with RODBC

su - # again as root
aptitude install r-cran-rodbc
exit

# Start R

library(RODBC)

channel <- odbcConnect(dsn=_yourDSN_, uid=_yourUserID_, pwd=_yourPassword_,
case="oracle", believeNRows=FALSE)
odbcGetInfo(channel)

# Simple query
query <- "query1;"
zival <- sqlQuery(channel, query)

# Or a set of queries, where sprintf can be nicely used

query <- c(
"query1;",
sprintf("CREATE TABLE tmp AS
SELECT bla1,
bla2,
FROM tableX
WHERE bla3 = '%s';", x),
"query3;")

sapply(query, sqlQuery, channel=channel)