Naturally Obsessed: What if the scientists had used an electronic lab notebook?

Posted by Rory on August 25th, 2010 @ 7:00 am

The film

Over a year after it was released, the award winning film,  Naturally Obsessed:  The Making of a Scientist, continues to attract praise and stimulate discussion. The film chronicles the experiences of three PhD candidates in the laboratory of molecular biologist Lawrence Shapiro at Columbia University Medical Center.  Here’s the plot, as summarized on the Memorial Sloan-Kettering Cancer Center website

“In the film, shot mostly in Dr. Shapiro’s lab, the students are trying to beat worldwide competition in identifying the molecular workings of the protein AMPK, which controls whether fat is burned to produce energy or is stored as fat. The three scientists struggle with various personal challenges: Robert Townley has a history of rebellious behavior; Kilpatrick Carroll questions whether he should leave academia for industry; and Gabrielle Cubberley wrestles with self-doubt about succeeding in such a competitive environment. At the end of the film, Townley achieves success in his project and publishes the results, with Dr. Shapiro as the only other co-author, in the journal Science.”

Naturally Obsessed has been called “the best film ever to depict what goes on inside a real science lab — period.”  In my view it deserves all the accolades it has received;  gritty, realistic, entertaining, etc.  So it’s well worth watching.   Understandably, it’s the human dynamics of life in the lab that seems to have captured the imagination of most people who have commented on the film.   In this post I’d like to focus on an aspect of the film which has not received a great deal of attention, but IMHO is just as interesting as the human dynamics.  That is the light the film sheds on the research process in a lab, on the ‘research dynamics’ if you will.

Research dynamics in the Shapiro lab

Richard Rifkind, M.D.  co-producer/co-director of Naturally Obsessed: the making of a scientist (and Chairman Emeritus of the Sloan-Kettering Institute and founding Chairman of the New York Structural Biology Center,  observed in commenting on why he was inspired to make the film

“Scientific research . . .  is an investigation that starts with curiosity about a question that can only be answered by the collection of data, and  . . . must find the story in that data that resolves that original question. For the scientist, that story is buried somewhere in the myriad experimental data points collected in a lab notebook.”

Sure enough, Robert Townley’s lab notebook even features in the film.    Since  Townley and Shapiro’s paper was published in 2007, the period of filming must have been roughly 2004 – 2007.  That was before collaborative research tools like electronic lab notebooks were widely available.  If the film was remade today, and the Shapiro lab was using an electronic lab notebook rather than individual paper lab notebooks, how might that impact on the research process and the research dynamics in the lab?

Three kinds of research relationships are depicted in the film:  between Larry Shapiro, the lab head, and each graduate student, between the graduate students themselves and between each graduate student their own research.

Relationships between the lab head and individual graduate students

The key research links in lab appear to be a series of individual relationships between Larry Shapiro and each graduate student.  Although Larry says, “I don’t tell the students what to do, we are trying to train independent scientists,”, he also says, “As a PhD, you are an apprentice to the advisor,” and he clearly is closely involved with his students’ research.  So closely that Robert says at the end of the Science paper, “This is me and Larry’s paper.  Together we have solved this structure of this really important molecule.”

Relationships between the graduate students

There are close links in the Shapiro lab between at least some of the graduate students.  And these are not just social links.  Kil sums it up when he says, “It’s not just my data that enriches the experience, Rob has been sitting across from me, working here, and has done many experiments that I never would have thought to do, and they’ve turned out to be really useful.”

Relationship of each graduate student to their own research

This of course is the ‘obsession’ in the film’s title.  Some quotes give a flavor of the experience:

“Two and a half years of doing experiments and having them not work.”

“We did an experiment a year ago, the crystals were too disordered.  We went back to the drawing board.”

“You learn so much from failure, nothing from success.”

The lab

As depicted in the film, this lab appears to be more a collection of individuals each working independently on an aspect of a problem rather than a group working together on the problem.  So there isn’t — or at least does not  appears to be — a strong common research agenda.

How might the research dynamics change (improve!) if the Shapiro lab adopted an electronic lab notebook?

Relationships between the lab head and individual graduate students

If everyone in the lab was using an electronic lab notebook Larry would find it much easier to keep track of each student’s research.  He could, for example, check their records of their experiments — any time, from anywhere — and leave comments.  The students would also find it easier to communicate with Larry.  They would not need to track him down or wait until he was available to ask a question about a procedure or an experiment in progress.  They could simply send him a message, with a link to the experiment, protocol, etc., posing their question, and expect to get an answer back promptly.

Relationships between the graduate students

With an electronic lab notebook opportunities for exchange of experience and learning from each other –something the students value highly — would be massively increased.  They could if they wanted give selected of their fellow  lab members view or even edit permission on their experiments, so that their colleagues would be able to comment on experiments as they progressed, and in their own time.  Communication would no longer be limited to chance encounters in the lab.

Relationship of each graduate student to their own research

The organized, searchable record of their research that would come with an electronic lab notebook would prove a major plus for the students in the two biggest challenges they describe:  how to learn from failure and how to manage the mass of data they accumulate as years of experiments pile up.  With an electronic lab notebook their past research would be immediately available — to themselves, to Larry and to other lab members — and searchable.  It would be much easier to keep track of what worked and and what did not, and to understand why.  And the chance of ‘making progress’ more quickly by applying lessons learned to future experiments would be materially increased.

The lab

In the absence of an electronic lab notebook, the Shapiro lab appeared to consist of a series of one to one relationships, between Larry and each student, and between individual students.  An electronic lab notebook would provide a convenient environment where the lab, as a group, could share their research and ideas about their research.  Since they are all working around the common theme of better understanding the workings of AMPK, the lab as a group and also each individual would benefit from access to a greater base of knowledge.  Even better, this base of knowledge would accumulate over time, so future members of the lab, and Larry, would have convenient access to the work not only of current, but also past, members of the lab.

Electronic lab notebooks: how ‘social’ should they be?

Posted by Rory on August 18th, 2010 @ 10:22 am

The web has gone social, with Facebook and Twitter leading the way.  And applications that were not designed with social networking as a core element are scrambling to add social features.  Often, however, that’s not as straightforward as it seems and can have unintended consequences, of which  Google Buzz is perhaps the highest profile example.  As Robin Wauters at Techcrunch pointed out, “Merging something designed for public broadcasting (Buzz) with something inherently private (Gmail) was just looking for trouble.”

The issue of how many and what kind of social features to include also faces developers of collaborative tools like electronic lab notebooks.  Although collaborative tools are designed to help people work together, they are not  necessarily ‘social’.   But how can something that is designed to help people collaborate not be ‘social’?   As a way into looking at that question, let’s examine what Tom Coates, until recently head of product at Yahoo’s Brickhouse lab, was getting at it when he was quoted in the August 16 issue of Fortune as saying,

“Google is very good at building these utility-type products — search, email, and messaging . . .  But what they lack is a sense of how people share and collaborate.”

When I read that I thought, hang on a minute, what about Google Docs?  OK, Google Docs is not Google’s highest profile offering, but it is used by millions of people, so it surely must be included when considering Google’s ability or propensity to build ‘social’ products .  Document sharing is clearly a social activity, and the express purpose of Google Docs is to facilitate collaboration, so it must be the way that Google Docs is set up that, in Tom Coates’ view, excludes it from the ‘social’ category.

The Fortune article goes on to say, “Coates’s point is that you don’t have friends on Google, you have contacts and tasks.  These services reflect an engineering culture that is all about utility, but one that makes it hard for the company to create something that’s friendly and social.”   So in Fortune’s view in order to be ‘social’ it’s not enough to involve a group, in addition there needs to be an element of ‘friendliness’, and a focus on utility is implied to be anti-social.  Whether or not that is correct, it does constitute an explanation for Coates’s apparent oversight about Google Docs.  Google Docs is very much a utilitarian tool, and it is not designed to be particularly ‘friendly’.  So, even though Google Docs is intended to be used by groups of individuals, for these reasons it is not ‘social’.

Collaborative research tools, including electronic lab notebooks, are like Google Docs primarily designed for a utilitarian purpose — in the case of ELNs, to provide a platform for sharing and structuring experimental data and other information of interest to the research group.  On the Coates/Fortune view, this is at least one strike against their counting as ‘social’.  But since they are used by people not only to collaborate, but also to communicate, would they not benefit from being more ‘friendly’ and ‘social’?  Here are some capabilities, found in Facebook, that Fortune describes as the ‘social layer’.  Which of them would make an electronic lab notebook more ‘friendly’ (and useful!)?

What social features should an electronic lab notebook include?











Provenance in electronic lab notebooks

Posted by Rory on August 11th, 2010 @ 7:00 am

In this post I’d like to stimulate some discussion about provenance in electronic lab notebooks, and more generally in documenting biomedical research. This issue is of interest to various groups of  people, but they usually don’t talk to each other.  I’ll begin with observations on the issue from three people.  One is a biochemist who is also a leading commentator on documenting and communicating about biomedical research, the second is a thoughtful scientist who works in a lab and is is constantly looking for ways to get better organized in capturing her research, and the third is an informatician working on a project to bring the benefits of databases — including provenance — to wikis with a particular focus on biomedical research. Perhaps this post will stimulate some cross fertilization of ideas that otherwise might not take place.

The first person is Cameron Neylon.  Cameron has written a lot about different aspects of provenance in research, and helped organize a workshop on the issue in April where he delivered a presentation called In your worst nightmare:  how experimental scientists are doing provenance for themselves.  For the purposes of this discussion I’m going to focus on some comments Cameron made recently in a discussion started by Jonathan Eisen about possible electronic lab notebook systems.  Commenting on versioning and provenance, Cameron said,

“. . .versioning systems (generally) fail to provide a good way of capturing or thinking about the process that converts one thing to another. So I think the provenance problem or the process problem is the more interesting one.”

The second person is Kim Martin, who works at the Division of Pathway Medicine at Edinburgh University.  Kim has a strong interest in organizing her research and communicating with colleagues in an efficient way. Like Cameron, she feels that a simple audit trail showing all past versions of a particular record provides only a very limited perspective on the research process.

Kim has developed the idea of a journal view or ‘journalling’ in an electronic lab notebook as a way of being able to look back at the process of her work during a particular period of time.   To do this she wants to be able to very easily create a snapshot of everything she was doing on a particular day.  Here is Kim’s sketch of how such a ‘journal view’ might look:

Kim’s concept is that the electronic lab notebook would, through automatic linking, support the creation with a single click of a’ journal view’ of research and related activity undertaken on any given day.  One of Kim’s key objectives is to gain insights on the process of research which may have been undertaken some time ago, as a mnemomic device.  I think she shares this objective with Cameron — it would be interesting to get Cameron’s views on this.

The third person is James Cheney, at the Laboratory for Foundations of Computer Science, Edinburgh University.  I met James when we both spoke at the Biomedical-data day held at Edinburgh University in June. James gave a presentation called Databases + Wikis = Curated Databases.  Among the core areas of expertise of James’ group, which is led by Peter Buneman, is provenance for database queries and updates. They are working on a project aimed at bringing the benefits of databases, including the ability to deploy more sophisticated provenance, to wikis.  The project involves developing a “database wiki” which includes support for provenance and user queries about provenance, including the following planned features:

  • Basic Provenance: Record basic information about changes (userids of logged-in users, IP addresses of unknown users).
  • [DONE 0.2] Copy-Paste Provenance: Record provenance links relating data in consecutive versions of the tree.
  • Provide the ability to import data from other sources (including other DatabaseWikis) while automatically recording source information.
  • Query provenance: Propagate provenance along with queries embedded in pages, to support user queries about provennace
  • Bulk update provenance: Provide the ability to rearrange data within DBWiki pages or data using bulk updates while automatically recording provenance for these transformations.

With that background, it would be great to hear more from Cameron, Kim, James and others about:

  1. The nature and details of  the research  ‘process’ that needs to be captured.
  2. Reactions to Kim’s journalling idea — general reactions and also views on whether it provides a good (or at least a useful) angle on the research process, and how it might be modified to capture other aspects of the research process.
  3. Reactions to James’ planned provenance features — e.g. are these features likely to be useful to biomedical researchers, what other kinds of provenance would be useful in capturing the research process?
  4. Other thoughts on process and provenance in biomedical research stimulated by the above.