Having worked with an excellent documentation tool like Sphinx, I started looking to static blog generators meant. It turned out that Manu Viera, a colleague working at Yaco with me, shared the same itch and had already looked several static web generators in Python, which is our main language at Yaco. Manu found pelican the best candidate but still I found it a bit immature, not something like something like Jekyll.

Then I found Octopress, a framework built on top of Jekyll with several plugins, including syntax highlighting or automatic support for disqus comments.

The migration from wordpress was not too painful. I used the default Jekyll script to import wordpress posts and disqus importer for the comments. After some sed commands I got nice markdown formatted scripts.

I had some trouble in the beginning configuring an isolated Ruby runtime in Arch Linux just for Octopress but after discovering rbenv, everything went smooth. (I prefer rbenv instead RVM with rbenv I know at any moment what it’s doing).

Deploying an Octopress generated site to github pages is as easy as pie.

Aside of nice Python syntax highlighting now I have some extra advantages I didn’t have with wordpress.com:

• Markdown syntax when writing my posts.

• I can use the best text editor to mankind: vim :P

• My blog data becomes more manageable. If at some point I don’t want to host it github, I could just to push it somewhere else with no modification.

• I got a very nice default theme for free, that aside of looking good, it’s also very easy to tweak and maintain.

• Now I have a good excuse to learn Ruby outside of RoR influence. Ruby is one of those languages I wish I would be better at, even if Python remains my main working language.

In any case, I must say the service provided by wordpress.com has been quite good but this one of those cases where you have to say: “Sorry, it’s not you, it’s just me”.

from pyramid.events import subscriber

class MyCustomEventType(object):
    def __init__(self, msg):
        self.msg = msg

@subscriber(MyCustomEventType)
def my_subscriber(event):
    print(event.msg)

def my_view(request):
    request.registry.notify(MyCustomEventType("Here it comes"))
    return {}

When running the application, every time a request goes through my_view, an event with a message is emitted, in this case, “Here it comes”. The subscriber then handles the event by printing the message, but it could do anything you want.

Notice that I’m using a decorator to hook my_subscriber. In order for the decorator to work you have to make sure you call the scan method when configuring the application.

Be aware though, that all these events are synchronous because Pyramid is primarily a request-response framework, all the events emitted block until the subscribers are done. If you want non-blocking events in Pyramid you could spawn a process from the subscriber or come with some other solution.

But the events in Pyramid are just another functionality that it offers. Pyramid is not a event-oriented framework, if you want to go all the way with async events you should look into Twisted or Tornado.

With Arch Linux there is very little in your system that you are not aware of. You have to configure everything yourself by editing config files. The process is not that hard because all those configuration files are meant to be tweaked. You also count with an excellent wiki to help you.

The Arch Linux philosophy doesn’t try to shield the user from complexity with extra layers. Instead it focuses on making the direct configuration as simply as possible. For example, writing a proper boot script is much straightforward than in other distros. At the same time if you are not careful you have more chances of really screw things up everything.

Arch Linux aggressively updates from upstream sources. This has the advantage and disadvantages of being always in the bleeding-edge. I also like the idea of putting more responsibility about the stability of software in developers than in packagers, as long as you are aware of this as a user. As a user you have to assume the responsibility of being at the cutting-edge. Things may not go always smooth but you count with excellent tools to manage chaos.

That brings me to the real killer feature that makes Arch Linux shine over the rest: the packaging system. Pacman, ABS, AUR, makepkg and the PKGBUILD format are just great. You usually don’t have to mess with packaging that much, everything installs nicely and dependencies are correctly handled, specially if you stick to the official repository.

But if you don’t like something about a package or need another version you have all the tools in place for the creation and introspection of packages without disrupting pacman bookkeeping (pacman is the equivalent of dpkg/apt-get in Debian).  Let me illustrate all this with something I had to deal with this week.

I decide to use Compass to make my stormy relationship with CSS smoother. Compass is a Ruby gem, the usual way to install gems is through Ruby packaging system but I don’t want to mess with the Ruby libraries already installed in the system with pacman. If I install those gems as root pacman will not be able to keep track of them, everything could break in the future, and most importantly, without an easy solution.

A way to deal with this issue is to install the Compass gem in some directory and handle the runtime somehow. You usually end up with a new runtime environment for each project you start. There are excellent tools to manage runtimes in Ruby like Rake, but boy, I already have enough managing my Python virtualenvs.

I see that Compass is already in AUR. AUR is a very liberal package repository where anyone can upload source packages. When you install from AUR you usually have a review the PKGBUILD, the comments of other users and check how many users have voted the package to be included in official repositories. With tools like yaourt the whole process is very smooth.

Alright, the ruby-compass PKGBUILD looks good to me so I install it. Now compass is a good system citizen and can be updated, installed and uninstalled through pacman. Compass works as expected but it turns out that the most interesting feature I wanted to use in Compass is only available in the latest version of Compass, the version in AUR is not the latest one.

No problem, it’ll probably be some version bumps and I’ll be done. I download the PKGBUILD, bump the versions and build the package again but then I realize that the new version depends on new Ruby gems that are not in AUR.

At this point I would avoid getting into a dependency hell and go for Rake, but wait, I’m using Arch Linux, let’s see what happens if I continue with the Arch flow.

I take the PKGBUILD of Compass as a template, which is generic enough for any Ruby gem, and use them for the Ruby dependencies. I update licences, versions checksums, build them and done, everything works. They are all coming from rubyforge and follow the same building conventions, making my life easy as a packager.

I upload the PKGBUILDs to AUR with just one burp command. Now I can install the latest version of compass through pacman without any issue. I then send my modified version of PKGBUILD to the original Compass packager who updates it. That’s it, now anyone can install the latest version of Compass with all its dependencies from AUR. I now can install Compass at home with just one command: just yaourt -Sy ruby-compass.

Now I just have to keep an eye in new updates on the dependencies I’m now maintaining in AUR but rubyforge offers an excellent notification system for gem updates.

That’s it. The whole thing took less than 30 minutes.

I don’t know if nowadays writing a DEB package spec is that hard, I acknowledge I never tried. The tutorials I found about them drove me away when I considered it some years ago.

It’s not only the packaging format itself, there is also the community and policy aspects. Editing your PKGBUILDs is something that every Arch Linux user does. For AUR there is very little regulating making the packaging smoother process at the expense of shifting the trust on the packages to the user. In general, most packages in AUR are good enough but for production machines I still value more the trust the Debian and Ubuntu package maintainers.

That’s where open source community shines, you have many choices.

<!-- <div>${context.name}</div> -->

However Chameleon will still evaluate what’s inside the ${…} block even if it’s within an HTML comment. Chameleon must do this because you might want to insert conditional comments.

This dummy tal:condition block will do the job:

<span tal:condition="None">
  <div>${context.name}</div>
</span>

Chameleon ignore anything inside the condition block.

This is an important shift in my career since I won’t be working on proteomics informatics and academic research anymore. I have mixed feelings about leaving proteomics. On one hand I like the area because there are plenty of tough challenges to be solved. But on the other hand I’m glad I can dedicate all my time to develop web applications, that might not be as sophisticated as proteomics software, but that will be immediately useful for the masses. I love web development and the Python community but within proteomics I could only intersect with the Python web development community quite sporadically. Now I’ll have the chance to be part of it full time.

Personally, The Netherlands is the most comfortable and easy-going country I ever lived. Here I had the chance to work with very smart people and made friends that will never forget. What I have learned during these years is priceless.

But I can’t deny my origins, Spain is where I feel at home even if sometimes I don’t find it too exciting because I’m too familiar with the culture. However Seville is quite far from my hometown, in the North of Spain. The culture in the South is very different from the North, so in a way, I’ll be another foreigner excited about the peculiarities I discover about Andalusian culture.

from multiprocessing import Process, Lock, Queue

job = 0
q = Queue(maxsize=3)
lock = Lock()

def work():
    import time; time.sleep(8)
    job = q.get()
    print("Job done: {0}".format(job))
    print("Queue size: {0}\n".format(q.qsize()))
    if not q.empty():
        work()
    else:
        lock.release()

def my_view(request):
    global job
    if not q.full():
        job += 1
        q.put(job)
        # Not running
        if lock.acquire(False):
            Process(target=work).start()
            print("Job {0} submitted and working on it".format(job))
        else:
            print("Job {0} submitted while working".format(job))
    else:
        print("Queue is full")
    print("Queue size: {0}\n".format(q.qsize()))
    return {'project':'asyncapp'}

With every request a job is sent. Here the queue accepts 3 jobs. The recursion in work makes sure there is only 1 process working at a time.

I will leave my previous example with Value because it’s easier to understand but this version is much safer.

Update: You can avoid the use of locks by using 2 queues.

My experience with pyramid has been really smooth until I had to write a form to parse huge input files in order to build experiment sites.

In an average view for an add form I would return a response redirecting to the newly created model resource. But in this case, the proteomics files can be quite large. I needed some way of having the view returning a response while the files were being parsed. Here is a simplified representation of how I made it work:

from multiprocessing import Value, Process

is_parsing = Value('B', 0)

def parse():
    print "Parsing started"
    import time; time.sleep(10)
    print "Parsing is done"
    is_parsing.value = False

def my_view(request):
    if not is_parsing.value:
        is_parsing.value = True
        Process(target=parse).start()
    else:
        print("Still parsing...")
    return {'project':'asyncapp',
            'is_parsing': is_parsing.value}

Here I’m launching another process to do the parsing while the web app returns the response without the parsing being done. The is_building variable is shared in both processes (no need of global statement). In my case I only want to run one parsing process at a time, so I didn’t have the need for locks or queues.

In the template of this view I can either offer a form to create an experiment or inform that there is already a experiment site being built. When building I can have the browser polling to check if the parsing is done.

That’s enough in my case, I don’t need to scale to thousands of users parsing multiple files at the same time, but I was curious about how to deal with that problem if I had to. I played a bit with different ideas I was given in the always supportive #repoze channel at freenode.

First, instead of multiple processes I tried OS threads, I know about the infamous GIL but want to see it with my own eyes. However I got some intimidating random errors from paster/pyramid that were enough to drive me off that path.

I also monkeypatched the standard library with eventlet so that the OS threads would become green threads. The dummy example I show above seemed to run fine but when trying in my real application I ran into more cryptic thread errors from monkeypatched ZODB, which is what I use in my real app. I also tried gevent with similar results. If you want to use eventlet or gevent you have to find another storage mechanism that works with green threads. Update: I was monkeypatching incorrectly, Andrey Popp’s comment explains how to do it.

Another potential source of problems when scaling with long polling, specially if you would like to add a nice responsive progress bar and a kind of log showing what is being done while parsing.

WebSockets are being regarded as the ultimate solution to deal with this kind of problems. First of all, let’s pretend websockets would be supported by all major browsers soon.

How can websockets be handled in pyramid? It turns out that dealing with websockets within the WSGI protocol is messy. However eventlet and gevent have ways to have websockets working within WSGI. Theoretically you could run a monkeypatched pyramid application behind gunicorn which can make the websockets accessible in the request.environ in pyramid. There is still some websocket protocol tasks (i.e. handshaking, closing socket, etc.) which would make writing something looking like a normal pyramid view hard.

But it happens that Ben Ford has a already written a wrapper to take care of that problem:  stargate (as he says, communication for pyramids). With stargate, in your pyramid app you create websocket view by subclassing from a base class that deals with the minutiae of the websocket protocol (up to v76, the latest version at the time of writing). In a websocket view, instead of returning anything from that view, you just write a handler to catch what is coming from the websocket. The great advantage of stargate is that you don’t need to run another process to deal with websockets, you can handle websockets from within pyramid. Additionally, stargate has 100% unit test coverage and some documentation.

While websockets look like the way forward I think it’s going to take some time for websockets to become mainstream in all browser, specially after mozilla announced it won’t support websockets in the next release of firefox because of security issues.

However, with node.js, it seems finally event driven web frameworks are becoming mainstream, bringing projects like Socket.IO. Socket.IO provides an abstraction layer to the developer to write event driven web applications. Socket.IO gives the same way of writing regardless of what the browser supports, being websockets, long polling or Flash; the developer writes the app the same way.

Although initially Socket.IO is meant to be used with node.js, there is something available for the server side in Python: SocketTornad.IO. It’s built on top of Tornado web framework. In spite of Tornado having some WSGI support, I’m afraid it won’t be easy to have the async features when in WSGI mode.

If I were to support now many concurrent users in a highly responsive application I would probably ditch pyramid  and go directly with SocketTornado.IO. Perhaps I will still be using pyramid for the non async part and have a front web server dispatching requests accordingly.

But it turns out that this is just a fun thought experiment, the multiprocess solution is fine for me because, like most web developers or bioinformaticians, for now I don’t need to write highly responsive applications for thousands of users.

Update: Marius Gedminas pointed out a better way to do this with locks. I will leave the code snippet using Value because is quite illustrative but you shouldn’t use Value if you to do something similar, instead check a better example I wrote in another post.

Update: Check pyramid_socketio for a newer version of async apps.

Some influential bioinformaticians group try to define the bioinformatics field as if it were precisely the research they are doing, frowning upon bioinformatics research not similar to theirs (or similar but superior to theirs). The followers of these groups try to imitate them so that they can be some day become experts in the field. I see also other bioinformaticians gathering together just because they work in biology using a computer, regardless of how little overlap there is in the things they do. It’s like group therapy, sharing experiences with people marginalized for the same reason.

Bioinformatics field is still in the very beginning. The field is very broad and will eventually be fragmented in multiple official fields. Working in an emerging field can be very exciting because you don’t have the constrains rules of an established field. But if social recognition is important to you, think twice when getting into bioinformatics. You’d likely feel out-of-place wherever you go.

I also acknowledge that fine-tuning all your UNIX tools to your exact requirements can be cumbersome. But you don’t need to customize everything from scratch. Open source is great because you can steal what other people are sharing and put it altogether as you like.

Lately I have been reading some great posts about customization ZSH. I consider the shell prompt a fundamental part of the UNIX IDE. In the screenshot below I show how my customized ZSH prompt plays nicely with bzr, git and virtualenvwrapper.

I barely have done anything from scratch. I just stitched configurations and tips from these sources:

1. Prompt decorations: http://aperiodic.net/phil/prompt/, http://git.sysphere.org/dotfiles/tree/zshrc
2. Zenburn theme: http://git.sysphere.org/dotfiles/tree/Xdefaults
3. git prompt hack: http://briancarper.net/blog/570/git-info-in-your-zsh-prompt
4. virtualenv prompt hack: http://www.doughellmann.com/docs/virtualenvwrapper/tips.html#zsh-prompt
5. bzr prompt hack: from scratch imitating git hack.

End results:

1. .zshrc
2. virtualenwrapper postactivate and postdeactivate

Isn’t open source great? I would be flattered if you also can steal my configuration files ;)

In another blog post I’ll write about my customizations for Vim, urxvt and awesomewm to reach the ultimate UNIX IDE.

Why do you reinvent the wheel? Why do you want to waste your time?

Well, if I hadn’t ‘wasted’ so much time during all these years I would still be doing terrible things like using regex to parse HTML with a crippled Perl scripts. The skills for what I’m getting my salary are mostly from wasted time. I wouldn’t be nowhere with only my formal training.

Information technology nowadays is mostly about about knowledge investment. Doing stuff is not as hard as learning how to do stuff orders of magnitude more efficiently. Learning is kind of accumulative, the more you learn the better you are at learning and the faster your efficiency grows. With computers is really hard to hit a physical limit where it’s not possible to improve significantly more by learning.

It’s well known that few highly productive people can beat large corporations made of people who work linearly with the skills they once learned in order to get a job.

Accessing genomics or transcriptomics data is more or less straightforward compared to proteomics data. You go to a data repository and download whatever you are looking for. Anyone who tries to do something similar in proteomics usually ends up downloading spreadsheets with incomplete data from supplemental material of published articles.

It’s frequent to hear proteomics informaticians complaining about experimentalists not making their data easily available and how they could be so greedy of not publishing the data for what they have been funded with public money. But jumping into conclusions about experimentalists being greedy or sloppy is not fair to me. The real issue is far deeper than that.

First of all, in order to share proteomics data the first requirement is to have the proper infrastructure. Lately there has been some projects aiming to provide the sharing infrastructure. I would say that the most popular proteomics repositories are PeptideAtlas, Proteinpedia, PRIDE and ProteomeCommons.

As far as I know, the main way Proteinpedia and PeptideAtlas handle data upload is with manual curators who study the format and the way the proteomics experiments were done. Then they come up with the best way to put the data in the repository. Usually is a combination of custom parsing with some manual editing to make experiments consistent in the repository. Obviously this kind of method is not that scalable for every proteomics experiment out there. In order to cope Proteinpedia focuses on human submissions only, whereas PeptideAtlas team is already looking into other solutions to facilitate data submission.

PRIDE, in the other hand, accept only data submission in PRIDE XML format. The generation of this XML format is not supported by most of the proteomics tools experimentalists use. However the PRIDE team provide an application that tries to convert every format out there and every experiment design to the PRIDE XML format. But because they try cover every single aspect of every experiment, I’ve seen experimentalists struggle when trying to fill all the forms. Moreover, they also don’t like the rigidity of the model to depict what the proteomics experiment was about. It’s true that there are some optional controlled vocabulary terms to give flexibility but still experiments have difficulties wrapping their head about how to use those terms.

This problem is not exclusive of PRIDE. Each proteomics lab uses its own mass spec terminology and frequently forms designed by developers with no first-hand experience making experiments don’t match the terms the experimentalists would understand. After all, experimentalists like to spend their time with experiments, they don’t like spending their time in things that are considered bureaucracy. The PRIDE team is aware of this problem a keeps trying with more familiar ways for experimentalists to fill form data.

ProteomeCommons seems to be the repository getting most traction. It’s currently where most proteomics experimentalists are submitting their data to fend off journal editors’ complains about the lack of published data. ProteomeCommons is built on top of the Tranche network, a kind of global distributed filesystem, that potentially offers infinite scalability to store data by just adding more nodes to the network. The tranche network looks just like a big hard disk with several files. Everybody can upload anything they like. That’s where ProteomeCommons comes in place, it’s the web gateway to upload the data, the web application offers some options to annotate the data, but it’s not as complete as what you have in the other databases. That’s understandable because if they bothered the experimentalists with thousands of forms with mandatory fields, the experimentalist wouldn’t submit their data to ProteomeCommons. It’s also worth mentioning that ProteomeCommons is not required to use Tranche network, any other proteomics repository could use Tranche network as a backend to store huge proteomics files. The rest of the repositories are currently looking into using the Tranche network to store the huge amounts of proteomics data, specially derived from raw data.

You might have notice the conspicuous absence of format standards in my description about the different repositories. If everybody used the same proteomics standards the infrastructural problems to share data would have been solved. Right?

The major effort to standardized proteomics formats is being carried out by HUPO-PSI. It’s a kind of consortium where they have regular meetings where representatives of different proteomics groups among the world agree on what has to go in the standard and how. You can follow the discussions and chip in for what would you like to have in the formats.

Aside of the typical problems of something designed by committee, it remains to be seen if mass spec vendors and proteomics software developers will fully embrace the standards. Proteomics data is highly heterogeneous by nature, there are very different kinds of proteomics experiments depending on what is the research being done. High quality proteomics experiments is not something that can be converted into an assembly line process where everything can be easily is fixed and standardized.

However the new stable format releases from HUPO PSI look good enough to me to at least start making the data exchange among repositories possible. There is also the promise from several mass spec vendors of future commitment to fully support the standards. I hope all those promises don’t end up in just that.

In my opinion all these infrastructural difficulties are going to be solved somehow relatively soon. The problem is that I don’t think that just by solving the infrastructural problems everybody will start sharing data transparently. There are other difficulties.

The main reason experimentalists are, at least, uploading their data to Tranche is to avoid being bugged by proteomics journals into making their proteomics data available. Many proteomics journals are getting really serious about this making the data available. Why proteomics journals are so interested in having the authors making their data available?

One could argue that editors of these journals believe in the moral imperative of making the data available but I don’t buy ethics as the main reason. The majority of biomedical scientific journals are still for profit companies, not academic institutions. In order to survive they have to make money selling something as every other company. For a journal publishing articles with lots of citations from other journals, with lots citations themselves, are the best way to guarantee that companies and institutions will keep renewing the yearly subscriptions.

It’s not something that I can’t demonstrate with facts, but lately I’m getting the feeling that proteomics is being disregarded by people in other biological fields as low quality research. After all proteomics is just a technology, a tool, to find out biological insights. Mass spec research by itself wouldn’t get so much funding if it couldn’t be used for biological research. What I think it’s happening is that biologists are taking less seriously biological findings in pure proteomics journals. Most published proteomics experiments are irreproducible and if you start digging into the published data you frequently find many false positives. That’s why proteomics journals editors are enforcing the experimentalists to release their data and make it as transparent as possible. They hope they can gain more credibility and get more citations from non-proteomics journals.

But still one can see that most experimentalists are reluctant to make their best data fully available. Many informaticians trying to analyze the data think that this attitude is because of cultural resistance to change. Many of these informaticians try to evangelize the experimentalists about why is so important to share data. Among evangelists the most notable group is the Fix Proteomics Campaign, which proposes some habits to make proteomics more credible.

But experimentalists are not dummies. I have seen them changing really quickly any habit if they find something better. The problem is that making their data transparently available is worse for them and here is where I think the campaigns miss the point. Let me explain something unique about mass spectrometry proteomics that seems easily forgotten by many people.

Mass spectrometers are really expensive instruments. Getting the adequate skills to operate them takes several years of training. To make things more costly these instruments become obsolete in a matter of few years because there new ones are constantly new ones coming up with better features. When a new instrument arrives to the lab a lot of time is spent optimizing it and learning how to troubleshoot it. If you don’t keep getting those new mass spectrometers you are left behind by the competitors because they can get advantage of more powerful instruments.

How in academics is possible to maintain a high funding inflow? In a company you have to sell a a service or a product but you can’t do so in an academic group. Usually you rely on grant agencies to provide funding. Granting agencies grant money by scientific productivity of the group. Publications in reputable journals are the main tangible measurement used by granting agencies for scientific productivity.

But most journals, as I said before, have to operate like companies. Proteomics data by itself is not publishable, if there is no story with some biological insight or some novel way to improve results, what will you write in a proteomics paper with just high quality data? Generating high quality proteomics data is damn difficult, I would say even more difficult than to come with fancy analysis of data. Let me explain.

People coming from genomics and transcriptomics fields sometimes forget that the chemical nature of proteins is much more diverse than DNA or RNA. After all, DNA and RNA have more or less homogeneous chemical properties regardless of its sequence. Proteins, in the other hand, are chemically completely different from each other depending on the sequence - that’s why they can carry out so many molecular functions. A proteins from the nucleus are completely different than the proteins from the cytoplasmic membrane.

The proteome is also much more dynamic than the genome. The same cell under different conditions show completely different proteome profile. You also have to take chemical modifications of proteins into account, which are only detectable by probing the proteins directly. Chemical modifications like phosphorylation act as functional switches for proteins, a protein with a modification has also different chemical properties than the same protein without modification. The heterogeneity of proteins makes protein purification, proper separation, and identification by mass spec an entire field by itself.

So an experimentalist, who has had years of training just to be able to identify proteins and chemical modifications, might, understandably, lack the skills for sophisticated analysis that will make the story sexier for proteomics journals. Software to analyze proteomics data as a power user, without programming knowledge, is still in the early days. As a developer I can see how difficult it is to make analysis software that covers every kind of proteomics experiment with a point and click interface.

The most logical step for an experimentalist when having good data would be to look for people specialized in analysis to come up with a powerful story. Usually the best people analyzing are independent proteomics informaticians that would do the analysis only if they get the credit for it. After all they have to also have to get funded to keep doing research and they can always claim they are the ones writing the paper. But even if proteomics informaticians give proper credit to the data generators in their papers - which I wouldn’t say it’s always true -, very few granting agencies will keep the experimental lab funded just to generate data that other people will use to write publications.

To tackle this problem the the most powerful experimental proteomics labs are trying to aggressively hire programmers who can do analysis in-house so that the credit remains within the group. The first problem these groups face is that there are almost no proteomics informaticians in the job market. They have to invest in people with programming skills who will eventually get the proteomics knowledge necessary to make useful programs for analysis or to be able to analyze the data themselves.

Also, because of the lack of programming knowledge of the experimentalists it can be tricky for them to envision which potential programmers will have the skills required to become a good analyzer, not to mention how to motivate the programmers with an excellent about why to join their field. There are also programmers, where I include myself, that precisely look actively for this highly experimental labs instead of pure informatics groups in order to get a closer understanding of experimental data by interacting directly with the experimentalists.

But this kind of setting where data generators and informaticians try to work together is propitious to get into Dilbert-kind of situations. I would say that it’s mainly because of the misunderstanding generated by the technological gap.

I feel fortunate of working in my current group because I think it’s one of the few experimental proteomics lab where people are aware of this problem and actively try to improve the communication with the informaticians.

The final point I want to make in this post is that if the data generators were rewarded properly for what they are good at, generating unbeatable high quality, sharing proteomics data transparently, and as soon as it’s generated, would become mainstream. I see granting agencies rewards as the main cause for not sharing data because they usually don’t reward the generation of data accordingly. They should also reward the labs that not only make the data available but make it as accessible as possible for data analyzers, so that proteomics research field would advance much faster than it’s currently doing. I acknowledge granting agencies are changing slowly for the better but there is still a long way to go.

But understanding how granting agencies work and what are their motivations is still something quite fuzzy to me. I think I’m still not old enough to understand the politics behind research funding.

However keeping the reviewers anonymous these days is becoming more difficult because many scientific articles include supplemental material that can’t be attached directly to the article, i. e. a huge list of proteins identified in a proteomics experiment. This poses a problem for editors and reviewers because by watching which IPs are accessing the machine which is hosting the data the original authors can guess who are the reviewers.

I have heard about this issue as a really huge problem for peer review process and from advocates of third party hosts with sophisticated technologies which anonymize reviewers. But it turns out that accessing any site in the web anonymously is not as obscure or complicated as it sounds.

A quick way - but maybe not so reliable - of anonymizing your web traffic is by googling for ’browse anonymously.’ You can find many web proxies that claim to anonymize your identity. Usually you have to paste the web site you want to access anonymously and then you’ll be redirected to the web page normally, albeit with a much slower load. The people who are hosting the server will see, at most, an IP address where the anonymous proxy is, that won’t probably correspond to a place where any reviewers are located.

But I don’t recommend using any proxy out there unless you really need to access something anonymously quickly and you have nothing in place. After all, who knows what they can do with your data, or if your IP leaks somewhere. I would recommend the use of the Tor network. Tor is an anonymity network where volunteers spread all over the world provide their machines to act as anonymizing proxies. Oversimplifying, Tor connects, encrypts and obfuscates the web traffic between you and the host with many of these proxies so that it becomes damn difficult to find out the original IP. When using a browser that goes through the Tor network the guys hosting the data will be seeing different random IPs all over the world with no relation whatsoever to each other.

There are different ways to setup Tor but if you want to use it without thinking too much go to the download page of Tor and get the Tor browser bundle. That will come with a firefox browser which is already configured for accessing the tor network. If in your institution or company, the network policy is controlled by a fascist network administrator who denies everything in the firewall regardless of the true danger for security is, go to settings and indicate you are behind a firewall.

By giving the link to the Tor bundle browser in the supplemental material the editor and the reviewers shouldn’t have any problem accessing self-hosted data. Downloading the Tor bundle browser shouldn’t be an obstacle, I haven’t seen anyone complaining when asked to download a propietary viewer to visualize closed formats for raw data, which is quite common in proteomics.

However with this post I’m not advocating to host your own data instead of sharing it. Hosting it yourself and sharing are not mutually exclusive. I think sharing your scientific data is a moral imperative when you are funded with public money. I encourage sending scientific data to as many public repositories as possible, but I also think individual researchers have the right to host their own data if they want to.

I know it’s hard to believe but many researchers who are funded with money coming from tax-payers are reluctanct to share their data, at least in the proteomics community where I work. If their data is stolen and other people find more interesting things the original authors missed they lose the relevance necessary to keep getting funded, specially when the analyzers don’t give enough credit to the generators of data which is quite frequent. But I will leave that for another post.

I started using routinely a feed reader around 4 years ago. Since then I’ve been constantly replacing the blog feeds I follow with others I found more interesting. Right now all the blogs I’m following are excellently written, and I’m not talking only about celebrity blogs, but also the kind of highly technical blogs that make it, for example, into Python planet. When I think about the quality of writing I would like for my blog several blogs from Python planet come to my mind. But even if I put myself a much lower quality threshold to get started, it turns out that writing something of acceptable quality, just the writing, is indeed harder than it seems. I can allow myself to suck but can’t go public with something I know is total crap. Recently I reached a writing level where I’m comfortable enough to go public.

But obviously publishing a decent blog is not only about writing properly. You need something to write about. For me it’s not worth writing superficially just for the sake of writing. In order to comment on something I need to have a deep understanding of the matter, otherwise I can’t confidently give a public opinion. But for that to happen I need something I didn’t get until recently: specialized knowledge.

When I started my Biochemistry degree back in Spain I liked computers as a hobby  (I was a relatively early Linux user) but my real passion was molecular biology, the wet lab. Soon I realized how much tedious manual work and luck influenced in successful biological experiments. In the other hand, experiments using the computer were quick and you could somehow understand much better what was going on.

Later I worked for Akhilesh Pandey developing HPRD, a human protein database that included (and still does) plenty of high quality manually curated from scientific literature not found in any other biological databases. My role was a kind of bridge between the programmers, the curators and the biological requirements for the project. My research career started totally different to what it’s expected from a young researcher, instead of specializing in something first and from there try to understand later how what you study contributes to science in general, I had to first understand the global picture before trying to get into the nitty-gritty. That didn’t mean I didn’t want to get deep into different areas. The problem was that I wanted to get into too many things at the same time. After some time the fields where I really ended up focusing have become fewer. Now, instead of working in large teams, I work more isolated in a very specialized projects within the area of proteomics informatics.

Then if finally I ended up working in a very specialized topic was the global view experience a waste of time? Absolutely not. My way of thinking has been critically shaped since then. I’m the kind of person who has to have a clear reason of everything I do. Now it’s clear for me on what I want to focus on, without losing the context of everything I do. I know what I want to learn and for what. I still maintain more or less the same goals that I had when I got into research, the difference now is how I want to reach those goals. But I’ll leave my goals for another post.

In the end the lack of specialized knowledge has been a important blocker to start a blog. I didn’t feel with enough authority to write acceptable posts for concrete topics. Now in Python and proteomics I’m more or less getting above the knowledge threshold where I can start writing something critically.

At the same type proteomics informatics is a blend of very different specialized topics. In proteomics I still don’t totally understand how a mass spectrometer works but of so much parsing, merging, querying, filtering, and plotting mass spec data I got a good handle of what mass spec proteomics data is like. About programming I’m far from writing something I would qualify as good code. Even if my coding capabilities have improved over the years I qualify my code worse than I used to. In reality my code is better now, it’s just that, with time, I’m becoming more critical about what I would call good code. However I’m fully aware that being so critical with my own code is a symptom of programming competence. I feel I’m on the right track, if I keep pushing like I’m doing I’ll eventually get to write something I could qualify as good enough. For now I feel competent enough in Python to start saying something meaningful about programming publicly.

The bottom line is that now I have some specific knowledge and a decent ability for writing. One would think that these factors provide the ideal scenario to start a blog. I know that having a blog is very important for my profession but that hasn’t be the last push to start now. I’m writing this post right now because I need it. Let me rephrase it: I’m not forcing myself to write a blot, I need to write.

What is really happening is that blogging is just part of a bigger transformation I’m going through. I’ve been lurking for quite some time different open source communities, initially just to learn more from people who are way better than me in programming. But lately I have been developing a great admiration for certain communities and individuals. I need to give them something back. I’m gradually participating more in the community. Now I need show who I am, how I see the world, what I want in life and what do I admire and why.

That’s why I’m blogging.