Naturally Obsessed: What if the scientists had used an electronic lab notebook?

Posted by Rory on August 25th, 2010 @ 7:00 am

The film

Over a year after it was released, the award winning film,  Naturally Obsessed:  The Making of a Scientist, continues to attract praise and stimulate discussion. The film chronicles the experiences of three PhD candidates in the laboratory of molecular biologist Lawrence Shapiro at Columbia University Medical Center.  Here’s the plot, as summarized on the Memorial Sloan-Kettering Cancer Center website

“In the film, shot mostly in Dr. Shapiro’s lab, the students are trying to beat worldwide competition in identifying the molecular workings of the protein AMPK, which controls whether fat is burned to produce energy or is stored as fat. The three scientists struggle with various personal challenges: Robert Townley has a history of rebellious behavior; Kilpatrick Carroll questions whether he should leave academia for industry; and Gabrielle Cubberley wrestles with self-doubt about succeeding in such a competitive environment. At the end of the film, Townley achieves success in his project and publishes the results, with Dr. Shapiro as the only other co-author, in the journal Science.”

Naturally Obsessed has been called “the best film ever to depict what goes on inside a real science lab — period.”  In my view it deserves all the accolades it has received;  gritty, realistic, entertaining, etc.  So it’s well worth watching.   Understandably, it’s the human dynamics of life in the lab that seems to have captured the imagination of most people who have commented on the film.   In this post I’d like to focus on an aspect of the film which has not received a great deal of attention, but IMHO is just as interesting as the human dynamics.  That is the light the film sheds on the research process in a lab, on the ‘research dynamics’ if you will.

Research dynamics in the Shapiro lab

Richard Rifkind, M.D.  co-producer/co-director of Naturally Obsessed: the making of a scientist (and Chairman Emeritus of the Sloan-Kettering Institute and founding Chairman of the New York Structural Biology Center,  observed in commenting on why he was inspired to make the film

“Scientific research . . .  is an investigation that starts with curiosity about a question that can only be answered by the collection of data, and  . . . must find the story in that data that resolves that original question. For the scientist, that story is buried somewhere in the myriad experimental data points collected in a lab notebook.”

Sure enough, Robert Townley’s lab notebook even features in the film.    Since  Townley and Shapiro’s paper was published in 2007, the period of filming must have been roughly 2004 – 2007.  That was before collaborative research tools like electronic lab notebooks were widely available.  If the film was remade today, and the Shapiro lab was using an electronic lab notebook rather than individual paper lab notebooks, how might that impact on the research process and the research dynamics in the lab?

Three kinds of research relationships are depicted in the film:  between Larry Shapiro, the lab head, and each graduate student, between the graduate students themselves and between each graduate student their own research.

Relationships between the lab head and individual graduate students

The key research links in lab appear to be a series of individual relationships between Larry Shapiro and each graduate student.  Although Larry says, “I don’t tell the students what to do, we are trying to train independent scientists,”, he also says, “As a PhD, you are an apprentice to the advisor,” and he clearly is closely involved with his students’ research.  So closely that Robert says at the end of the Science paper, “This is me and Larry’s paper.  Together we have solved this structure of this really important molecule.”

Relationships between the graduate students

There are close links in the Shapiro lab between at least some of the graduate students.  And these are not just social links.  Kil sums it up when he says, “It’s not just my data that enriches the experience, Rob has been sitting across from me, working here, and has done many experiments that I never would have thought to do, and they’ve turned out to be really useful.”

Relationship of each graduate student to their own research

This of course is the ‘obsession’ in the film’s title.  Some quotes give a flavor of the experience:

“Two and a half years of doing experiments and having them not work.”

“We did an experiment a year ago, the crystals were too disordered.  We went back to the drawing board.”

“You learn so much from failure, nothing from success.”

The lab

As depicted in the film, this lab appears to be more a collection of individuals each working independently on an aspect of a problem rather than a group working together on the problem.  So there isn’t — or at least does not  appears to be — a strong common research agenda.

How might the research dynamics change (improve!) if the Shapiro lab adopted an electronic lab notebook?

Relationships between the lab head and individual graduate students

If everyone in the lab was using an electronic lab notebook Larry would find it much easier to keep track of each student’s research.  He could, for example, check their records of their experiments — any time, from anywhere — and leave comments.  The students would also find it easier to communicate with Larry.  They would not need to track him down or wait until he was available to ask a question about a procedure or an experiment in progress.  They could simply send him a message, with a link to the experiment, protocol, etc., posing their question, and expect to get an answer back promptly.

Relationships between the graduate students

With an electronic lab notebook opportunities for exchange of experience and learning from each other –something the students value highly — would be massively increased.  They could if they wanted give selected of their fellow  lab members view or even edit permission on their experiments, so that their colleagues would be able to comment on experiments as they progressed, and in their own time.  Communication would no longer be limited to chance encounters in the lab.

Relationship of each graduate student to their own research

The organized, searchable record of their research that would come with an electronic lab notebook would prove a major plus for the students in the two biggest challenges they describe:  how to learn from failure and how to manage the mass of data they accumulate as years of experiments pile up.  With an electronic lab notebook their past research would be immediately available — to themselves, to Larry and to other lab members — and searchable.  It would be much easier to keep track of what worked and and what did not, and to understand why.  And the chance of ‘making progress’ more quickly by applying lessons learned to future experiments would be materially increased.

The lab

In the absence of an electronic lab notebook, the Shapiro lab appeared to consist of a series of one to one relationships, between Larry and each student, and between individual students.  An electronic lab notebook would provide a convenient environment where the lab, as a group, could share their research and ideas about their research.  Since they are all working around the common theme of better understanding the workings of AMPK, the lab as a group and also each individual would benefit from access to a greater base of knowledge.  Even better, this base of knowledge would accumulate over time, so future members of the lab, and Larry, would have convenient access to the work not only of current, but also past, members of the lab.

Electronic lab notebooks: how ‘social’ should they be?

Posted by Rory on August 18th, 2010 @ 10:22 am

The web has gone social, with Facebook and Twitter leading the way.  And applications that were not designed with social networking as a core element are scrambling to add social features.  Often, however, that’s not as straightforward as it seems and can have unintended consequences, of which  Google Buzz is perhaps the highest profile example.  As Robin Wauters at Techcrunch pointed out, “Merging something designed for public broadcasting (Buzz) with something inherently private (Gmail) was just looking for trouble.”

The issue of how many and what kind of social features to include also faces developers of collaborative tools like electronic lab notebooks.  Although collaborative tools are designed to help people work together, they are not  necessarily ‘social’.   But how can something that is designed to help people collaborate not be ‘social’?   As a way into looking at that question, let’s examine what Tom Coates, until recently head of product at Yahoo’s Brickhouse lab, was getting at it when he was quoted in the August 16 issue of Fortune as saying,

“Google is very good at building these utility-type products — search, email, and messaging . . .  But what they lack is a sense of how people share and collaborate.”

When I read that I thought, hang on a minute, what about Google Docs?  OK, Google Docs is not Google’s highest profile offering, but it is used by millions of people, so it surely must be included when considering Google’s ability or propensity to build ‘social’ products .  Document sharing is clearly a social activity, and the express purpose of Google Docs is to facilitate collaboration, so it must be the way that Google Docs is set up that, in Tom Coates’ view, excludes it from the ‘social’ category.

The Fortune article goes on to say, “Coates’s point is that you don’t have friends on Google, you have contacts and tasks.  These services reflect an engineering culture that is all about utility, but one that makes it hard for the company to create something that’s friendly and social.”   So in Fortune’s view in order to be ‘social’ it’s not enough to involve a group, in addition there needs to be an element of ‘friendliness’, and a focus on utility is implied to be anti-social.  Whether or not that is correct, it does constitute an explanation for Coates’s apparent oversight about Google Docs.  Google Docs is very much a utilitarian tool, and it is not designed to be particularly ‘friendly’.  So, even though Google Docs is intended to be used by groups of individuals, for these reasons it is not ‘social’.

Collaborative research tools, including electronic lab notebooks, are like Google Docs primarily designed for a utilitarian purpose — in the case of ELNs, to provide a platform for sharing and structuring experimental data and other information of interest to the research group.  On the Coates/Fortune view, this is at least one strike against their counting as ‘social’.  But since they are used by people not only to collaborate, but also to communicate, would they not benefit from being more ‘friendly’ and ‘social’?  Here are some capabilities, found in Facebook, that Fortune describes as the ‘social layer’.  Which of them would make an electronic lab notebook more ‘friendly’ (and useful!)?

What social features should an electronic lab notebook include?











Provenance in electronic lab notebooks

Posted by Rory on August 11th, 2010 @ 7:00 am

In this post I’d like to stimulate some discussion about provenance in electronic lab notebooks, and more generally in documenting biomedical research. This issue is of interest to various groups of  people, but they usually don’t talk to each other.  I’ll begin with observations on the issue from three people.  One is a biochemist who is also a leading commentator on documenting and communicating about biomedical research, the second is a thoughtful scientist who works in a lab and is is constantly looking for ways to get better organized in capturing her research, and the third is an informatician working on a project to bring the benefits of databases — including provenance — to wikis with a particular focus on biomedical research. Perhaps this post will stimulate some cross fertilization of ideas that otherwise might not take place.

The first person is Cameron Neylon.  Cameron has written a lot about different aspects of provenance in research, and helped organize a workshop on the issue in April where he delivered a presentation called In your worst nightmare:  how experimental scientists are doing provenance for themselves.  For the purposes of this discussion I’m going to focus on some comments Cameron made recently in a discussion started by Jonathan Eisen about possible electronic lab notebook systems.  Commenting on versioning and provenance, Cameron said,

“. . .versioning systems (generally) fail to provide a good way of capturing or thinking about the process that converts one thing to another. So I think the provenance problem or the process problem is the more interesting one.”

The second person is Kim Martin, who works at the Division of Pathway Medicine at Edinburgh University.  Kim has a strong interest in organizing her research and communicating with colleagues in an efficient way. Like Cameron, she feels that a simple audit trail showing all past versions of a particular record provides only a very limited perspective on the research process.

Kim has developed the idea of a journal view or ‘journalling’ in an electronic lab notebook as a way of being able to look back at the process of her work during a particular period of time.   To do this she wants to be able to very easily create a snapshot of everything she was doing on a particular day.  Here is Kim’s sketch of how such a ‘journal view’ might look:

Kim’s concept is that the electronic lab notebook would, through automatic linking, support the creation with a single click of a’ journal view’ of research and related activity undertaken on any given day.  One of Kim’s key objectives is to gain insights on the process of research which may have been undertaken some time ago, as a mnemomic device.  I think she shares this objective with Cameron — it would be interesting to get Cameron’s views on this.

The third person is James Cheney, at the Laboratory for Foundations of Computer Science, Edinburgh University.  I met James when we both spoke at the Biomedical-data day held at Edinburgh University in June. James gave a presentation called Databases + Wikis = Curated Databases.  Among the core areas of expertise of James’ group, which is led by Peter Buneman, is provenance for database queries and updates. They are working on a project aimed at bringing the benefits of databases, including the ability to deploy more sophisticated provenance, to wikis.  The project involves developing a “database wiki” which includes support for provenance and user queries about provenance, including the following planned features:

  • Basic Provenance: Record basic information about changes (userids of logged-in users, IP addresses of unknown users).
  • [DONE 0.2] Copy-Paste Provenance: Record provenance links relating data in consecutive versions of the tree.
  • Provide the ability to import data from other sources (including other DatabaseWikis) while automatically recording source information.
  • Query provenance: Propagate provenance along with queries embedded in pages, to support user queries about provennace
  • Bulk update provenance: Provide the ability to rearrange data within DBWiki pages or data using bulk updates while automatically recording provenance for these transformations.

With that background, it would be great to hear more from Cameron, Kim, James and others about:

  1. The nature and details of  the research  ‘process’ that needs to be captured.
  2. Reactions to Kim’s journalling idea — general reactions and also views on whether it provides a good (or at least a useful) angle on the research process, and how it might be modified to capture other aspects of the research process.
  3. Reactions to James’ planned provenance features — e.g. are these features likely to be useful to biomedical researchers, what other kinds of provenance would be useful in capturing the research process?
  4. Other thoughts on process and provenance in biomedical research stimulated by the above.

5 Things PIs want in an electronic lab notebook — other suggestions?

Posted by Rory on July 28th, 2010 @ 7:00 am

What PIs want in an electronic lab notebook is often different from what postdocs and graduate students want because PIs are looking for a tool for recording the entire lab’s work, rather than an individual note taking tool.  I looked around the web at recent discussions of what PIs are looking for in an ELN, and identified five common themes:

  1. Something that’s easy to learn and easy to use in order to ensure (relatively stress free) lab-wide buy in and take up.  Joshua Shaevitz, at Princeton, has a good description of the considerations that went into adopting an ELN, and the adoption process, in his recent  post on My Lab’s Wiki-based Electronic Lab Notebook System.  He says, “Before implementing our wiki system, I setup a mock wiki ELN on my laptop and presented it during a  lab meeting to show everyone the benefits firsthand. I especially wanted to convince them that the new system would not generate extra work, but would instead make their lives easier.”
  2. Something that’s flexible in terms of providing for, on the one hand, common structures for group records and records that need to be accessed by multiple members of the group, and, on the other hand, scope for individuals to ‘do their own thing’ in terms of both research style and having their own private space.  Joshua Shaevitz again: “I didn’t want to impose too much structure on each lab member, as I think notebook style is very personal thing. But, I also wanted to ensure that the results would be compatible with features such as search and would work well with our archiving strategies.”
  3. Something that facilitates integrated handling of  experimental data (i.e. the lab notebook function) in the same environment as other information the lab deals with, e.g. protocols, meeting notes, etc. Alex Swarbrick at the Garvan Institute: we use our electronic lab notebook “to compile the diverse collections of data that we generate as biologists, such as images and spreadsheets, and to take minutes of meetings.”
  4. Related to the previous point, something that provides the capacity to manage physical inventory as well as data in electronic form, and the ability to link the two together.  This point is brought out by Cameron Neylon in a thread accessible in a great recent discussion started by Jonathan Eisen at U.C. Davis, Possible electronic lab notebook systems – update.  In discussing what kinds of data a system needs to able to handle, Cameron says, “generating, storing, analysing and publishing research objects, explicitly including samples and other physical objects.”  And Alex Swarbrick again: “the ability to link records, reagents and experiments. For example, to connect an experimental mouse with the tube containing its tissues in the freezer, to the 6 different experiments (conducted over a year) that analysed those tissues in different ways. Managing this kind of ‘metadata’ is absolutely essential to our work.”
  5. Something that can “help to deal with information and data overload (sorting and filtering)” — a scientist interviewed in a recent study of the research practices of seven life sciences research labs Patterns of information use and exchange:  case studies of researchers in the life sciences.

How does this list sound?  Is it an accurate reflection of what others want in an ELN? Is it comprehensive?  Are key requirements missing?  Comments welcome!

What is an electronic lab notebook III: the benefits of structure

Posted by Rory on July 21st, 2010 @ 8:43 pm

The last post and the one before that looked at different views on who electronic lab notebooks are for — individuals or the lab — and how wikis measure up as environments that  enable lab members to enter and share experimental data.  Notwithstanding their attraction as convenient online tools for sharing general information, wikis lack structure, and it is primarily this which has kept even labs that use wikis wedded to the paper lab notebook for documenting experiments.

In this post we’ll look at why the ability to add structure to research data  is the key enabler permitting the transition from paper lab notebooks to electronic lab notebooks.

Paper lab notebooks support as much structure as you like.  You can create sections, paste copies of images, make notes in the margin, draw diagrams — the only limit to adding structure to a paper lab notebook is the scientist’s imagination.    Unlike a wiki, an electronic lab notebook allows you to replicate the structure that you put into a paper lab notebook.  Why?  Because an electronic lab notebook allows the creation of records with different kinds of fields.  This supports structuring your research data in two ways. First, the different types of fields support entry of information in differing ways, e.g. by date or time, by entry of text, by number, with radio buttons signalling  series of mutually exclusive options, etc.  Second, different classes of records can be put together with different combinations of various kinds of fields, so creating types of records that are appropriate to different aspects of research, e.g. a CHiP experiment, a freezer, a particular protocol,or an antibody.   This is in stark contrast to the wiki, which has only one type of record — the wiki page — and an undifferentiated one at that with no support for separation into different fields.

The benefits of this structure extend further to the other  things you use in your research like images and spreadsheets.  Like wikis, electronic lab notebooks have the advantage over paper lab notebooks of being able to make links to images and spreadsheets, which can also be inserted into the electronic repository — wiki or electronic lab notebook.  But electronic lab notebooks offer superior structuring capabilities in this respect too, because with an electronic lab notebook, unlike a wiki, you can associate a spreadsheet, image or other electronic item with a particular field of a particular kind within a record.

Making use of an electronic lab notebook’s ability to create records with different kinds of fields allows you to put structure into the record of your research in an online electronic environment much as you did with a paper lab notebook and at the same time gain the benefits of associations between bits of information which can only be made in an online environment, so they actually enable taking structuring of research data to a new and higher level.  It is this element – the ability to add structure to research data – which explains why that electronic lab notebooks — and not wikis — provide the best platform for labs  wishing to move from paper to electronic recording and management of their research data.  This is the unstated driver that lies behind wikipedia’s definition of electronic lab notebook as “a software program designed to replace paper lab notebooks“.    And so, I would revise that definition and say that an electronic lab notebook is an online environment that provides a sufficient capability for structuring research data to enable scientists to document and share their research data in that environment without the need to also resort to a paper lab  notebook.

What is an electronic lab notebook II — how do wikis measure up?

Posted by Rory on July 14th, 2010 @ 7:00 am

In the last post I poked a bit of fun at the wikipedia definition of  electronic lab notebook — “a software program designed to replace paper lab notebooks”.  Since that could mean just about anything, beauty is in the eye of the beholder.  The big dividing line is between people — like postdocs and graduate students — looking for a note taking tool for themselves, and others — like PIs — looking for a collaborative research tool for the lab.

This time I’m going to take a look at wikis — how do they measure up to the challenge faced by PIs looking for a collaborative research tool:  organizing and keeping track of a wide range of types of research data generated by lab members, present, past and future?   The first point to make is that there are all sorts of wikis, with varying degrees of sophistication, power and capabilities.    I’m going to use the most developed wikis as a point of comparison here — Confluence and PBWorks are good examples — wikis with a fully developed feature set.

As a recent study looking in depth at the work practices of seven life sciences research labs pointed out, a growing number of labs have turned to wikis as convenient environments for storing and sharing general information like meeting notes and protocols.  Wikis have a number of attractions to labs, in that they are easy to learn and use, online, and provide good support for sharing and collaboration.  In addition, the more sophisticated wikis have integrated messaging systems and some, like PBWorks, even have voice conferencing capabilities.

At the top end, then, wikis are becoming fully fledged knowledge management tools.  But features like voice conferencing are aimed at businesses, not labs.  For labs the key issue is managing their data. The study notes that notwithstanding the trend towards organizing general information in wikis, all the labs studied still maintain paper lab notebooks.  Paper lab notebooks stand out as an island of tradition in the midst of a growing ocean of online information sharing.    An island perhaps but a pretty big island, Australia rather than Fiji if you will, because paper lab notebooks are the repositories for the most important information labs deal with, their research data.

On this evidence wikis are falling short as a software program that replaces paper lab notebooks, and hence are not functioning as electronic lab notebooks per the wikipedia definition.  Why are labs staying with paper lab notebooks even as they adopt wikis to share information other than research data?  Inertia no doubt is a big part of the reason.  But the other big barrier to adoption of tools in labs — they have to be easy to learn and easy to use — is probably less of a factor.  Wikis  are coming into general use and it’s not the wiki per se that is being resisted, its the use of the wiki specifically as a place for entering and sharing research data.

Here’s a hypothesis:  the reason labs are sticking with paper lab notebooks for dealing with experimental data and not moving their experimental data into wikis along with general information like meeting notes and protocols is that wikis are unable to provide structure for the data.  With a wiki all you get is the wiki page.  It has no more support for structure than a Word document, and even less structure than a spreadsheet, without  doubt the most popular electronic repository for experimental research data.  Next time I will look in more detail at how electronic lab notebooks provide support for structuring research data and the benefits this can bring to collaboration and communication in the lab.

What is an electronic lab notebook?

Posted by Rory on July 6th, 2010 @ 9:30 pm

Welcome to the electronic lab notebook blog.  This will be a space for discussing electronic lab notebooks from every angle:  what benefits do they bring? how do they compare with alternatives? what kinds of features do they have and should they have? what issues do people face in using them?  how can you get the most out of them?

In this first post I’m going to start at the beginning:  what is an electronic lab notebook?

Wikipedia defines electronic lab notebook as “a software program designed to replace paper lab notebooks“.  That could be just about anything — beauty is in the eyes of the beholder!   I’ll take a look in a second at who the relevant beholders are and what each of them thinks, but taking wikipedia as the starting point it’s  fair to say first that all of them are looking to move away from this:

Postdocs and graduate students

So who are the relevant beholders?  They can be divided into two categories.  The first is people looking for an electronic note taking device for themselves.  They tend to be interested in convenience and simplicity.  Something which is easy to use and also helps them get organized.     But many postdocs and grad students want something which also provides  support for research.  Here is one description of the ‘dream app’ over on an Apple forum about electronic lab notebooks:

  • easy copy-pasting/drag ‘n drop
  • ability to re-open the files
  • metadata and search (tags, keywords, …)
  • possiblity to link to older notes and graphs
  • store PDF (or TIFF) representation of external files along with the original file: preview files without their originating application
  • automated backup mechanism
  • encryption on disk

The thread following that post contains a good discussion of the varieties of things people want in an electronic lab notebook for their own use, examples of what they have tried, what they like and dislike, and the limitations of the available tools.

PIs

The second category of people looking for an electronic lab notebook want a collaborative tool rather than one aimed at individuals.  They tend to be PIs, or in some cases others in the lab who’ve been asked by the PI to identify a suitable tool. Like those looking for an individual tool, PIs want something which is simple to use and easy to learn.  But beyond that their needs diverge from that of individual scientists. Professor Mike Shipston of the University of Edinburgh provides a good summary of the kinds of challenges that drive PIs to look at adopting an electronic lab notebook:

“We generate a wide variety of types of data sets, for example data from molecular analysis, quantitative analysis, for example quantitative RTPCR, gene cloning, through to electrophysiological analysis, for example from confocal images and total internal reflective microscopy right up to behaviorial assays in animals.  So its really about coordinating those types of data sets that fit together, keeping them contained within projects, because the data sets are derived from different people within the lab.  Also we have a very big extended network both in the UK and across Europe and the US.  It’s about keeping that information together. We have a large number of people coming in and out of the lab, the challenge is keeping track of that data and integrating it in with data from existing projects.”

Well that’s it for this first post — I realize I haven’t yet provided an answer to the question of what an electronic lab notebook is!  The next post is going to look at the tool that has been most widely adopted by labs looking for a collaborative tool, wikis.  I’ll discuss the strengths and limitations of wikis and whether they do – or should – qualify as electronic lab notebooks.  And don’t worry, I promise to come back with a specific answer to the question of what an electronic lab notebook is.