New software aims to make science more replicable

But if everyone agrees on the need for specialized equipment in research, shouldn't the same mindset apply to every aspect of the scientific process? Why is it, then, that when writing scientific reports, most researchers still use the same program that came with their first computer?

"Most people write their papers in Word, which is not a good scientific publishing environment," said John Kitchin, professor of Chemical Engineering at Carnegie Mellon University. "For example, Word has no way to log your data as you go or record exactly how it was analyzed. It isn't practical to see where the data in a figure came from. So when researchers are making their supporting information file-which contains much of the data and analysis that explains just how the study was done-they're forced to reconstruct what they think they did from memory."

Kitchin, whose research focuses mainly on software development for modeling materials and solving problems in engineering, saw this unfortunate trend in his own scientific report writing, and decided to do something about it. That's why he created SCIMAX: a software specifically designed for the purpose of writing scientific reports.

SCIMAX is an open-source, fully integrated program that combines the function of Word-namely, the narrative portion of the report-with everything else that one might need when writing a research paper. The data, the code, the bibliography-there's a place in the program for everything, already perfectly formatted and ready for input. And this is key, because doing these things in other programs often leads to serious problems.

"I saw a paper where the author had copied their MATLAB code into Word," Kitchin said. "Even that is better than what many people do. But because they copied it and didn't run it from Word, they forgot to copy two of the files that you need to actually run it. So suddenly, the code in the paper-the same code they claim is what helped them run their study-flat out doesn't work."

It may sound like semantics-whether you use one program or another to write your reports-but according to Kitchin it has widespread implications for the integrity of science.

"So when I write a paper, I can tell you in the paper that I did something, and I can tell you that it worked," Kitchin said. "But without the accurate dataset, and in many cases the actual code used to analyze it, there's no way for you to take the paper and replicate my work. And that's a fundamental failing on our part."