On December 10th, 2014, I was formally awarded tenure at UC Davis, where I will start as an Associate Professor in the School of Veterinary Medicine on January 5th, 2015. In my research statement for my job application, I wrote: Open science and scientific reproducibility: I am a strong advocate of open science, open source, …
I was a reviewer on Determining the quality and complexity of next-generation sequencing data without a reference genome by Anvar et al., PDF here. Here is the top bit of my review. One interesting side note – the authors originally named their tool kMer and I complained about it in my review. And they renamed …
The apocalypse is nigh. Soon, binary executables and containers in object stores will join the many Web-based pipelines and the several virtual machine images on the dystopic wasteland of “reproducible science.” Anyway. I had a conversation a few weeks back with a senior colleague about container-based approaches (like Docker) wherein they advocated the shipping around …
Dear <chairs>, I am resigning my Assistant Professor position at Michigan State University effective January 2nd, 2015. Sincerely, CTB. Anticipated FAQ: Why? I’m moving to UC Davis. Do you have an employment contract with UC Davis?? Nope. But I’m starting there in January, anyway. Or that’s the plan. And yes, that’s how this kind of thing happen. …
Brian O’Shea (a physics prof at Michigan State) asked me the following, and I thought I’d post it on my blog to get a broader set of responses. I know the answer is “Python 3”, but I would appreciate specific thoughts from people with experience either with the specific packages below, or in teaching Python …
A few months ago, I wrote a short description of how we make our papers replicable in the lab. One problem with this process is that for complex pipelines, it’s not always obvious how to connect a number in the paper to the steps in the pipeline that produced it — there are lots of …
As we think about the next few years of khmer development, it is helpful to explore what khmer is, and what our goals for khmer development are. This can provide guiding principles for development, refactoring, extension, funding requests, and collaborations. Comments solicited! Links: khmer github repo: https://github.com/ged-lab/khmer/ khmer docs: http://khmer.readthedocs.org/ Definition khmer is an open …
Here’s an excerpt from an e-mail to a student whose committee I’m on; they were asking me about a comment their advisor had made that they shouldn’t put a result in a paper because “It’ll confuse the reviewer.” One thing to keep in mind is that communicating the results _is_ important. “It’ll confuse the reviewer” …
A colleague just e-mailed me to ask me how I felt about journal impact factor being such a big part of the Academic Ranking of World Universities – they say that 20% of the ranking weight comes from # of papers published in Nature and Science. So what do I think? On evaluations I’m really …
Sean Eddy wrote an interesting blog post on how scripting is something every biologist should learn to do. This spurred a few discussions on Twitter and elsewhere, most of which devolved into the usual arguments about what, precisely, biologists should be taught. I always find these discussions not merely predictable but rather besides the point. …
Since being chosen as a Moore Foundation Data Driven Discovery Investigator, I’ve been putting together the paperwork at UC Davis to actually receive the money. Part of that is putting together a budget and a Statement of Work to help guide the conversation between me, Davis, and the Moore Foundation. Here’s what I sent Chris Mentzel …
Yesterday I gave my third keynote address ever, at the Australasian Genomics Technology Association’s annual meeting in Melbourne (talk slides here). On my personal scale of talks, it was a 7 or 8 out of 10: I gave it a lot of energy, and I think the main messages got across, but I ended up …
A few weeks back, Nick Loman (via Manoj Samanta) brought MEGAHIT to our attention on Twitter. MEGAHIT promised “an ultra-fast single-node solution for large and complex metagenome assembly” and they provided a preprint and some open source software. This is a topic near and dear to my heart (see Pell et al., 2012 and Howe …
I would like to build a community site. Or, more precisely, I would like to recognize, collect, and collate information from an already existing but rather diffuse community. The focus of the community will be academic data science, or “data driven discovery”. This is spurred largely by the recent selection of the Moore Data Driven …
I am very, very happy to announce that I have been selected to be one of the fourteen Moore Data Driven Discovery Investigators. This is a signal investment by the Moore Foundation into the burgeoning area of data-intensive science, and it is quite a career booster. It will provide my lab with $1.5m over a period of …
At the NIH ADDS meeting, we had several breakout sessions. Michelle Dunn and I led the training session. For this breakout, we had the following agenda: First, build “sticky-note clouds”, with one sticky-note cloud with notes for each of the following topics: Desired outcomes of biomedical data science training; Other relevant training initiatives and connections; …
In Extracting shotgun reads based on coverage in the data set, we showed how to get a read coverage spectrum for a shotgun data set. This is a useful diagnostic tool that can be used to estimate total genome size, average coverage, and repetitive content. Uses for this recipe include estimating the sequencing depth of …
Here is a links roundup and some scattered thoughts on the recent meeting on “the next five years of data science at the NIH”; this meeting was hosted by Phil Bourne, the new Associate Director for Data Science at the NIH. Before I go any further, let me make it clear that I do not …
As I mentioned, I am hoping to significantly scale up my training efforts at UC Davis; it’s one of the reasons they hired me, it’s a big need in biology, and I’m enthusiastic about the whole thing! A key point is that, at least at the beginning, it may replace some or all of my …
If you’re into metagenomics, you may have heard of CAMI, the Critical Assessment of Metagenome Interpretation. I’ve spoken to several people about it in varying amounts of detail, and it seems like the CAMI group is working to generate some new shotgun metagenome data sets and will then encourage tool developers to bang on them. …
This recipe provides a time-efficient way to determine whether you’ve saturated your sequencing depth, i.e. how much new information is likely to arrive with your next set of sequencing reads. It does so by using digital normalization to generate a “collector’s curve” of information collection. Uses for this recipe include evaluating whether or not you …
This is a recipe that provides a time- and memory- efficient way to loosely estimate the likely size of your assembled genome or metagenome from the raw reads alone. It does so by using digital normalization to assess the size of the coverage-saturated de Bruijn assembly graph given the reads provided by you. It does …
Inspired by Sarah Bisbing’s excellent post on her first year as a faculty member, here are the questions I remember asking myself during my first six years: Year 0: What science do I want to do? Year 1: What the hell am I doing all day and why am I always so exhausted? Year 2: …
The below is a recipe for subsetting a high-coverage data set to a given average coverage. This differs from digital normalization because the relative abundances of reads should be maintained — what changes is the average coverage across all the reads. Uses for this recipe include subsampling reads from a super-high coverage data set for …
In recent days, we’ve gotten several requests, including two or three on the khmer mailing list, for ways to extract shotgun reads based on their coverage with respect to the reference. This is fairly easy if you have an assembled genome, but what if you want to avoid doing an assembly? khmer can do this …
I just finished reading Svante Paabo’s autobiography, Neanderthal Man: In Search of Lost Genomes. The book is perfect — if you’re a biologist of any kind, you’ll understand most of it without any trouble, and even physicists can probably get a lot out of the story (heh). The book describes Svante Paabo’s journey towards sequencing …
Hello, my name is Titus Brown, and I’m at Michigan State University where I run a biology group whose motto is “better science through superior software”. I’m going to tell you about my vision for building infrastructure to support data-intensive biology. Our research is focused on accelerating sequence-based biology with algorithmic prefilters that help scale …
Our lab is part of the ongoing online conversation about how to properly credit software and algorithms; as is my inclination, we’re Just Trying Stuff (TM) to see what works. Here’s an update on our latest efforts! A while back (with release 1.0 of khmer) we added a CITATION file to the khmer repository and …
These are the talk notes for my opening talk at the 2014 Bioinformatics Open Source Conference. Normally my talk notes aren’t quite so extensive, but for some reason I thought it would be a good idea to give an “interesting” talk, so my talk title was “A History of Bioinformatics (in 2039)”. My thought was …
Nik Sultana, a postdoc in Cambridge, asked me some questions via e-mail, and I asked him if it would be OK for me to publish them on my blog. He said yes, so here you go! How is the quality of scientific software measured? Is there a “bug index”, where software loses points if it’s …
We just released khmer v1.1, a minor version update from khmer v1.0.1 (minor version update:220 commits, 370 files changed. Cancel that — _I_ just released khmer, because I’m the release manager for v1.1! As part of an effort to find holes in our documentation, “surface” any problematic assumptions we’re making, and generally increase the bus factor of the khmer project, …
How has sharing code, data, R methods helped scientific research in one or two specific examples? One personal story — we have a technical approach (called digital normalization) that we have yet to publish. We posted a paper draft on a preprint site and made the software available almost 3 years ago, however. The approach …
As part of the 2-day Mozilla Science Labs hackathon in late July, the khmer project will be providing a “mentored open source contributathon” experience. This will provide an opportunity for people interested in trying out our instance of the “github flow” model, in which contributions are submitted for review using a pull request. Since our …
tl;dr? I played around with building a CountMin Sketch that is dynamic in size, based on a scalable Bloom Filter approach. I’m not sure it worked. Thoughts, suggestions, help? Bloom Filters In our research, we’ve made some hay using Bloom filters. They’re remarkably easy to implement; I’ve talked about them a couple of times on …
About 10 days ago, I gave a talk in Manchester to Carole Goble’s group, hosted by Aleksandra Pawlik. The talk title was “Six ways to Sunday: Approaches to computational reproducibility in non-model sequence analysis.” I’ve posted the slides (here). For the talk, I put together a list of five things that I felt were “myths” …
I’m on a European trip that involves several plane flights accompanied by long airport stays, and I just used some of that time to do a bit of tedious coding on khmer. The coding I did was to add proper exception handling to khmer’s internal file loading routines (see the pull request). The old behavior …
My second-round Data Driven Discovery application is due on Monday, and my first draft contained the following story. I don’t think I’ll include it in the actual application, but it was entertaining enough to write that I thought I’d post it here. A vision of the future I would like to enable In 2020, a …
There were lots of problems with PyCon this year. For example, the free, hi-speed wifi made you log in each day. And it was in Montreal, so one of my foreign students couldn’t come because he didn’t get a visa in time. The company booths were not centrally located. And, worst of all, the PyCon …
tl;dr? The Software Carpentry train-the-trainers workshop in Toronto this past M-W was just fantastic. I can’t recommend it enough. A bit of background: Software Carpentry is a project to teach scientists to use computing more effectively. Started by Greg Wilson about 16 years ago, the project has progressed through many different moults, including attempts to …
I recently finished two books that really stood out in my memory of recent reading. Lighter fare than I have recommended in the past, but more representative of what I read day to day :) The first book, The Martian, is a near-future book about an astronaut who wakes up on Mars, having been left …
So my daughter just participated in her first science fair, at the age of 6. (“Conclusion: science can be fun! and sticky!”) Over dinner, my wife and I came up with some ideas for her next fair. She was having trouble dissolving sugar in ice water, so we suggested maybe something where we measured the …
Links, software, thoughts — all solicited! Send ’em to me, t@idyll.org. — Imagine… a rolling 48 hour hackathon, internationally teleconferenced, on reproducing analyses in preprints and papers. Each room of contributors could hack on things collaboratively while awake, then pass it on to others in overlapping timezones and go to sleep. The next day, they …
The khmer team is pleased to announce the release of khmer version 1.0. khmer is our software for working efficiently with fixed length DNA words, or k-mers, for research and work in computational biology. Links: khmer documentation PyPI link Announcement ChangeLog Citation handle khmer v1.0 is the culmination of about 9 months of development work …
I’m pleased to announce the publication of “Tackling soil diversity with the assembly of large, complex metagenomes”, by Adina Howe, Janet Jansson, Stephanie Malfatti, Susannah Tringe, James Tiedje, and myself. The paper is openly available on the PNAS Web site here (open access). External links: Joint Genome Institute press release GenomeWeb article MSU press release …
A few months back, I announced the khmer protocols project, an effort to write down an explicit, open protocol for transcriptome and metagenome assembly. This project was started during the summer of 2013 at the Woods Hole Marine Biological Lab, in collaboration with Joshua Rosenthal. I’d met Josh at the CephSeq meeting the year before, …
This term, I’m once again teaching my upper-division CSE undergrad course in Web Dev here at MSU. For the second time, I’m requiring students to use github for their homework; unlike last year, I now understand pull requests and have integrated them into the process. How does it work, basically? The instructions for handing in …
A few months back, we submitted a paper, These are not the k-mers you are looking for: efficient online k-mer counting using a probabilistic data structure, to PLoS One. We got the (signed) reviews back in December, and I asked the reviewers if I could post their reviews publicly. They said yes! So here they …
On January 27th, 2014, the MSU BEACON Center graduate students held a panel on how to review. The panel was organized by Emily Weigel (@choosy_female) and Jory Schossau, Bjorn Ostman (@CarnyEvolution), Kristin Parent, Rich Lenski (@RELenski), and Arend Hintze were panel members, together with me. I put together the following notes, somewhat amended in response …
I’ve just posted the narrative for a recently funded USDA grant on improving the quality of the chick genome assembly on the lab’s research page. The issues are laid out in detail in the grant, but, basically, the question is: how can we improve the quality of the assembly? The answer, we think, is to …
Over the past year of PyCon, #sciox, and other gender imbalance/harassment/sexism discussions that I’ve seen on Twitter and blogging, I’ve run into a few posts on these matters that stood out to me in terms of clarity, logic, and/or good instruction for me. I’m sharing them below because I think they’re really good, and because …