Naturally Obsessed: What if the scientists had used an electronic lab notebook?

Posted by Rory on August 25th, 2010 @ 7:00 am

The film

Over a year after it was released, the award winning film,  Naturally Obsessed:  The Making of a Scientist, continues to attract praise and stimulate discussion. The film chronicles the experiences of three PhD candidates in the laboratory of molecular biologist Lawrence Shapiro at Columbia University Medical Center.  Here’s the plot, as summarized on the Memorial Sloan-Kettering Cancer Center website

“In the film, shot mostly in Dr. Shapiro’s lab, the students are trying to beat worldwide competition in identifying the molecular workings of the protein AMPK, which controls whether fat is burned to produce energy or is stored as fat. The three scientists struggle with various personal challenges: Robert Townley has a history of rebellious behavior; Kilpatrick Carroll questions whether he should leave academia for industry; and Gabrielle Cubberley wrestles with self-doubt about succeeding in such a competitive environment. At the end of the film, Townley achieves success in his project and publishes the results, with Dr. Shapiro as the only other co-author, in the journal Science.”

Naturally Obsessed has been called “the best film ever to depict what goes on inside a real science lab — period.”  In my view it deserves all the accolades it has received;  gritty, realistic, entertaining, etc.  So it’s well worth watching.   Understandably, it’s the human dynamics of life in the lab that seems to have captured the imagination of most people who have commented on the film.   In this post I’d like to focus on an aspect of the film which has not received a great deal of attention, but IMHO is just as interesting as the human dynamics.  That is the light the film sheds on the research process in a lab, on the ‘research dynamics’ if you will.

Research dynamics in the Shapiro lab

Richard Rifkind, M.D.  co-producer/co-director of Naturally Obsessed: the making of a scientist (and Chairman Emeritus of the Sloan-Kettering Institute and founding Chairman of the New York Structural Biology Center,  observed in commenting on why he was inspired to make the film

“Scientific research . . .  is an investigation that starts with curiosity about a question that can only be answered by the collection of data, and  . . . must find the story in that data that resolves that original question. For the scientist, that story is buried somewhere in the myriad experimental data points collected in a lab notebook.”

Sure enough, Robert Townley’s lab notebook even features in the film.    Since  Townley and Shapiro’s paper was published in 2007, the period of filming must have been roughly 2004 – 2007.  That was before collaborative research tools like electronic lab notebooks were widely available.  If the film was remade today, and the Shapiro lab was using an electronic lab notebook rather than individual paper lab notebooks, how might that impact on the research process and the research dynamics in the lab?

Three kinds of research relationships are depicted in the film:  between Larry Shapiro, the lab head, and each graduate student, between the graduate students themselves and between each graduate student their own research.

Relationships between the lab head and individual graduate students

The key research links in lab appear to be a series of individual relationships between Larry Shapiro and each graduate student.  Although Larry says, “I don’t tell the students what to do, we are trying to train independent scientists,”, he also says, “As a PhD, you are an apprentice to the advisor,” and he clearly is closely involved with his students’ research.  So closely that Robert says at the end of the Science paper, “This is me and Larry’s paper.  Together we have solved this structure of this really important molecule.”

Relationships between the graduate students

There are close links in the Shapiro lab between at least some of the graduate students.  And these are not just social links.  Kil sums it up when he says, “It’s not just my data that enriches the experience, Rob has been sitting across from me, working here, and has done many experiments that I never would have thought to do, and they’ve turned out to be really useful.”

Relationship of each graduate student to their own research

This of course is the ‘obsession’ in the film’s title.  Some quotes give a flavor of the experience:

“Two and a half years of doing experiments and having them not work.”

“We did an experiment a year ago, the crystals were too disordered.  We went back to the drawing board.”

“You learn so much from failure, nothing from success.”

The lab

As depicted in the film, this lab appears to be more a collection of individuals each working independently on an aspect of a problem rather than a group working together on the problem.  So there isn’t — or at least does not  appears to be — a strong common research agenda.

How might the research dynamics change (improve!) if the Shapiro lab adopted an electronic lab notebook?

Relationships between the lab head and individual graduate students

If everyone in the lab was using an electronic lab notebook Larry would find it much easier to keep track of each student’s research.  He could, for example, check their records of their experiments — any time, from anywhere — and leave comments.  The students would also find it easier to communicate with Larry.  They would not need to track him down or wait until he was available to ask a question about a procedure or an experiment in progress.  They could simply send him a message, with a link to the experiment, protocol, etc., posing their question, and expect to get an answer back promptly.

Relationships between the graduate students

With an electronic lab notebook opportunities for exchange of experience and learning from each other –something the students value highly — would be massively increased.  They could if they wanted give selected of their fellow  lab members view or even edit permission on their experiments, so that their colleagues would be able to comment on experiments as they progressed, and in their own time.  Communication would no longer be limited to chance encounters in the lab.

Relationship of each graduate student to their own research

The organized, searchable record of their research that would come with an electronic lab notebook would prove a major plus for the students in the two biggest challenges they describe:  how to learn from failure and how to manage the mass of data they accumulate as years of experiments pile up.  With an electronic lab notebook their past research would be immediately available — to themselves, to Larry and to other lab members — and searchable.  It would be much easier to keep track of what worked and and what did not, and to understand why.  And the chance of ‘making progress’ more quickly by applying lessons learned to future experiments would be materially increased.

The lab

In the absence of an electronic lab notebook, the Shapiro lab appeared to consist of a series of one to one relationships, between Larry and each student, and between individual students.  An electronic lab notebook would provide a convenient environment where the lab, as a group, could share their research and ideas about their research.  Since they are all working around the common theme of better understanding the workings of AMPK, the lab as a group and also each individual would benefit from access to a greater base of knowledge.  Even better, this base of knowledge would accumulate over time, so future members of the lab, and Larry, would have convenient access to the work not only of current, but also past, members of the lab.

5 Things PIs want in an electronic lab notebook — other suggestions?

Posted by Rory on July 28th, 2010 @ 7:00 am

What PIs want in an electronic lab notebook is often different from what postdocs and graduate students want because PIs are looking for a tool for recording the entire lab’s work, rather than an individual note taking tool.  I looked around the web at recent discussions of what PIs are looking for in an ELN, and identified five common themes:

  1. Something that’s easy to learn and easy to use in order to ensure (relatively stress free) lab-wide buy in and take up.  Joshua Shaevitz, at Princeton, has a good description of the considerations that went into adopting an ELN, and the adoption process, in his recent  post on My Lab’s Wiki-based Electronic Lab Notebook System.  He says, “Before implementing our wiki system, I setup a mock wiki ELN on my laptop and presented it during a  lab meeting to show everyone the benefits firsthand. I especially wanted to convince them that the new system would not generate extra work, but would instead make their lives easier.”
  2. Something that’s flexible in terms of providing for, on the one hand, common structures for group records and records that need to be accessed by multiple members of the group, and, on the other hand, scope for individuals to ‘do their own thing’ in terms of both research style and having their own private space.  Joshua Shaevitz again: “I didn’t want to impose too much structure on each lab member, as I think notebook style is very personal thing. But, I also wanted to ensure that the results would be compatible with features such as search and would work well with our archiving strategies.”
  3. Something that facilitates integrated handling of  experimental data (i.e. the lab notebook function) in the same environment as other information the lab deals with, e.g. protocols, meeting notes, etc. Alex Swarbrick at the Garvan Institute: we use our electronic lab notebook “to compile the diverse collections of data that we generate as biologists, such as images and spreadsheets, and to take minutes of meetings.”
  4. Related to the previous point, something that provides the capacity to manage physical inventory as well as data in electronic form, and the ability to link the two together.  This point is brought out by Cameron Neylon in a thread accessible in a great recent discussion started by Jonathan Eisen at U.C. Davis, Possible electronic lab notebook systems – update.  In discussing what kinds of data a system needs to able to handle, Cameron says, “generating, storing, analysing and publishing research objects, explicitly including samples and other physical objects.”  And Alex Swarbrick again: “the ability to link records, reagents and experiments. For example, to connect an experimental mouse with the tube containing its tissues in the freezer, to the 6 different experiments (conducted over a year) that analysed those tissues in different ways. Managing this kind of ‘metadata’ is absolutely essential to our work.”
  5. Something that can “help to deal with information and data overload (sorting and filtering)” — a scientist interviewed in a recent study of the research practices of seven life sciences research labs Patterns of information use and exchange:  case studies of researchers in the life sciences.

How does this list sound?  Is it an accurate reflection of what others want in an ELN? Is it comprehensive?  Are key requirements missing?  Comments welcome!

What is an electronic lab notebook II — how do wikis measure up?

Posted by Rory on July 14th, 2010 @ 7:00 am

In the last post I poked a bit of fun at the wikipedia definition of  electronic lab notebook — “a software program designed to replace paper lab notebooks”.  Since that could mean just about anything, beauty is in the eye of the beholder.  The big dividing line is between people — like postdocs and graduate students — looking for a note taking tool for themselves, and others — like PIs — looking for a collaborative research tool for the lab.

This time I’m going to take a look at wikis — how do they measure up to the challenge faced by PIs looking for a collaborative research tool:  organizing and keeping track of a wide range of types of research data generated by lab members, present, past and future?   The first point to make is that there are all sorts of wikis, with varying degrees of sophistication, power and capabilities.    I’m going to use the most developed wikis as a point of comparison here — Confluence and PBWorks are good examples — wikis with a fully developed feature set.

As a recent study looking in depth at the work practices of seven life sciences research labs pointed out, a growing number of labs have turned to wikis as convenient environments for storing and sharing general information like meeting notes and protocols.  Wikis have a number of attractions to labs, in that they are easy to learn and use, online, and provide good support for sharing and collaboration.  In addition, the more sophisticated wikis have integrated messaging systems and some, like PBWorks, even have voice conferencing capabilities.

At the top end, then, wikis are becoming fully fledged knowledge management tools.  But features like voice conferencing are aimed at businesses, not labs.  For labs the key issue is managing their data. The study notes that notwithstanding the trend towards organizing general information in wikis, all the labs studied still maintain paper lab notebooks.  Paper lab notebooks stand out as an island of tradition in the midst of a growing ocean of online information sharing.    An island perhaps but a pretty big island, Australia rather than Fiji if you will, because paper lab notebooks are the repositories for the most important information labs deal with, their research data.

On this evidence wikis are falling short as a software program that replaces paper lab notebooks, and hence are not functioning as electronic lab notebooks per the wikipedia definition.  Why are labs staying with paper lab notebooks even as they adopt wikis to share information other than research data?  Inertia no doubt is a big part of the reason.  But the other big barrier to adoption of tools in labs — they have to be easy to learn and easy to use — is probably less of a factor.  Wikis  are coming into general use and it’s not the wiki per se that is being resisted, its the use of the wiki specifically as a place for entering and sharing research data.

Here’s a hypothesis:  the reason labs are sticking with paper lab notebooks for dealing with experimental data and not moving their experimental data into wikis along with general information like meeting notes and protocols is that wikis are unable to provide structure for the data.  With a wiki all you get is the wiki page.  It has no more support for structure than a Word document, and even less structure than a spreadsheet, without  doubt the most popular electronic repository for experimental research data.  Next time I will look in more detail at how electronic lab notebooks provide support for structuring research data and the benefits this can bring to collaboration and communication in the lab.