machine-envy » bioinformatics http://www.machine-envy.com/blog Tue, 27 Jul 2010 23:23:10 +0000 http://wordpress.org/?v=2.8.5 en hourly 1 Variant Call Format: really? http://www.machine-envy.com/blog/2010/07/27/variant-call-format-really/ http://www.machine-envy.com/blog/2010/07/27/variant-call-format-really/#comments Tue, 27 Jul 2010 23:23:10 +0000 James Casbon http://www.machine-envy.com/blog/?p=274 1000 genomes are making their genotypes available in variant call format (vcf). Now as others have noticed, vcf isn’t the prettiest format around. There are a few things to dislike:

  • The data is in ‘wide’ format which means that a file is fifteen screens wide and hides rare variation in a load of noise – it’s also ‘ungreppable’.
  • The use of tab delimiting with embedded semi-colon delimiting is an old trick, but these days everyone has a json parser handy so you should just use json for an structured field inside a tab delimited file.   Since strings, lists and key/value pairs all have simple json representations this will cover all cases.
  • The fact that the file headers dynamically describe the structure of the fields makes it less a format and more a family of formats.  The genotype fields have arbitrary keys.
  • Despite the flexibility defining genotypes, the locus information is not extensible so you cannot annotate the site with extra information.
]]> http://www.machine-envy.com/blog/2010/07/27/variant-call-format-really/feed/ 0 cogent: the unsung hero of bioinformatics and python http://www.machine-envy.com/blog/2009/10/27/cogent-the-unsung-hero-of-bioinformatics-and-python/ http://www.machine-envy.com/blog/2009/10/27/cogent-the-unsung-hero-of-bioinformatics-and-python/#comments Tue, 27 Oct 2009 16:57:53 +0000 James Casbon http://www.machine-envy.com/blog/?p=225 I recently started using cogent – the COmparative GENomics Toolkit and discovered that it is an excellent piece of kit. A google search for ‘python ensembl‘ doesn’t even show it at all, yet it definitely has the best bindings for ensembl avaiable in python – they’re based on sqlalchemy making it easy enough to pull of any query. Have a look at the full list of examples.

]]> http://www.machine-envy.com/blog/2009/10/27/cogent-the-unsung-hero-of-bioinformatics-and-python/feed/ 0 Installing python bioinformatics tools with virtualenv and pip http://www.machine-envy.com/blog/2009/07/11/installing-python-bioinformatics-tools-with-virtualenv-and-pip/ http://www.machine-envy.com/blog/2009/07/11/installing-python-bioinformatics-tools-with-virtualenv-and-pip/#comments Sat, 11 Jul 2009 13:56:52 +0000 James Casbon http://www.machine-envy.com/blog/?p=222 Python seems to have developed a decent set of tools for quickly building development environments. I want to store my notes on how to get a good environment for bioinformatics set up quickly.

First of all, if you haven’t already, install virtualenv and pip. Both are easy installable. Now install virtualenv wrapper.

Now we are going to setup a bioinformatics environment with both biopython and pygr installed so that you can hack on them. Firstly create a new virtualenv, passing the no site packages flag to keep this clean:

james@flapjack:~/Documents/virtualenvs$ mkvirtualenv --no-site-packages bio
New python executable in bio/bin/python
Installing setuptools............done.
(bio)james@flapjack:~/Documents/virtualenvs$ cdvirtualenv
(bio)james@flapjack:~/Documents/virtualenvs/bio$

Now, to install biopython we first use pip to install numpy:

(bio)james@flapjack:~/Documents/virtualenvs/bio$ pip -E . install numpy
Downloading/unpacking numpy
...

Important to remember the ‘-E’ flag which tells pip to use the virtualenv we are in (this should be added to virtualenv_wrapper IMHO). Now we can install biopython from our github fork, using the ‘-e’ flag to keep it editable (i.e we are hacking on it).

(bio)james@flapjack:~/Documents/virtualenvs/bio$ pip -E . install -e git://github.com/jamescasbon/biopython.git#egg=biopython
Obtaining biopython from git+git://github.com/jamescasbon/biopython.git#egg=biopython
Cloning git://github.com/jamescasbon/biopython.git to ./src/biopython
remote: Counting objects: 22719, done.
...

Next up, we want pygr so we need pyrex to build the c files:

(bio)james@flapjack:~/Documents/virtualenvs/bio$ pip -E . install -U pyrex -f http://www.cosc.canterbury.ac.nz/greg.ewing/python/Pyrex/Pyrex-0.9.8.5.tar.gz
Downloading/unpacking pyrex
Downloading Pyrex-0.9.8.5.tar.gz (242Kb): 242Kb downloaded
In the tar file /var/folders/Gn/GneSaDeKGaGpZXx+hcopdU+++TI/-Tmp-/tmpTEdhFd/Pyrex-0.9.8.5.tar.gz the member Pyrex-0.9.8.5/Demos/embed/Makefile is invalid: 'filename None not found'
Running setup.py egg_info for package pyrex
Installing collected packages: pyrex
Running setup.py install for pyrex
changing mode of build/scripts-2.5/pyrexc from 644 to 755
changing mode of /Users/james/Documents/virtualenvs/bio/bin/pyrexc to 755
Successfully installed pyrex

Now, to get and editable pygr:

(bio)james@flapjack:~/Documents/virtualenvs/bio$ pip -E . install -e git://github.com/jamescasbon/pygr.git#egg=pygr
Obtaining pygr from git+git://github.com/jamescasbon/pygr.git#egg=pygr
Cloning git://github.com/jamescasbon/pygr.git to ./src/pygr
remote: Counting objects: 6281, done.
...
Successfully installed pygr

Finally, ipython:

(bio)james@flapjack:~/Documents/virtualenvs/bio$ pip -E . install ipython
Downloading/unpacking ipython
Downloading ipython-0.9.1.tar.gz (2.8Mb): 2.8Mb downloaded

We now have a completely isolated environment, where pygr and biopython are editable:

(bio)james@flapjack:~/Documents/virtualenvs/bio$ bin/ipython
Python 2.5.4 (r254:67916, Mar 2 2009, 10:40:04)
Type "copyright", "credits" or "license" for more information.

IPython 0.9.1 -- An enhanced Interactive Python.
? -> Introduction and overview of IPython's features.
%quickref -> Quick reference.
help -> Python's own help system.
object? -> Details about 'object'. ?object also works, ?? prints more.

In [1]: import pygr

In [2]: pygr.__file__
Out[2]: '/Users/james/Documents/virtualenvs/bio/src/pygr/pygr/__init__.pyc'

]]> http://www.machine-envy.com/blog/2009/07/11/installing-python-bioinformatics-tools-with-virtualenv-and-pip/feed/ 0