Showing posts with label git. Show all posts
Showing posts with label git. Show all posts

Wednesday, 5 February 2014

Making figures for scientific papers

The next few blog posts (when I get round to doing them) are going to be a series of tutorials into common but neglected aspects of the academic process, such as how to make figures (this post), and how to submit/prepare data for GenBank (coming later). Very few students get taught how to do these things, so we just muddle our way through and learn along the way. Considering that these kind of activities are a central part of our research time, however, a little advice at the early stages can save a lot of effort.

What I won't be covering today though, is how to design good figures in the first place. There's enough information on that out there already. See, for example, the paper "A brief guide to designing effective figures for the scientific paper"; this article runs through dos and don'ts of making a figure to express your results clearly and effectively. What I will talk about, however, is how you actually put your figures together—what tools/processes are used, and how to be organised.

The basics

The really basic stuff is probably worth repeating. There are essentially two types of digital graphics: "raster" graphics (also known as "bitmap") and "vector" graphics. Raster graphics are comprised of entirely of coloured/shaded pixels, and as a result lose quality when you zoom in. Vector graphics (e.g. SVG, PDF, EPS) are comprised of instructions to draw objects, and so are scalable and therefore do not lose quality when zoomed in. Raster graphic formats (e.g. JPG, PNG, TIF) are for photographic images, and vector formats are for graphs, plots, diagrams, and cartoons. Raster images can be inserted into vector files (where they remain as raster), but not vice versa. Importantly in vector plots, the text remains as text. It amazes me how many rasterised phylogenetic trees I still see in journal articles. It might seems a trivial detail, but when you rasterise a tree, the names of the taxa in those trees become invisible to searching both within the PDF article, or by Web search engines.

Fig 1. Difference between raster and vector graphics

Software

So, what programs do we use to create/edit our figures? Well first of all, I will not be recommending any proprietary software. The reasons are threefold: (a) a lot of students work on their own personal machines, so it's unfair to make them spend their limited money on these expensive software packages; (b) with open-source software you can often use the same program on any operating system (Mac, Windows, Linux); and (c) there's no need, as the free tools are good enough. So, for generating the data plots I use the R statistical package with ggplot2 (the sort of equivalent of Microsoft Excel), for editing raster images I use Gimp (the equivalent of Adobe Photoshop), for maps I use QGIS (the equivalent of ArcGIS), and for putting it all together and making adjustments to vector figures I use Inkscape (the equivalent of Adobe Illustrator). Now I'm not pretending that Gimp, QGIS, and Inkscape are better than their paid-for equivalents, but they are free, and for academic use I think they do pretty much all that is needed.

Before we get started, though, here's one useful piece of advice: go to the Web site of the journal that you're intending to submit your manuscript to, and read the instructions for authors. Each journal has slightly different requirements, and knowing the specifics at the beginning makes things a lot easier. They will tell you the dimensions of the figures, the label formats, the fonts to use, etc. If you don't know where you will submit the article, just do something generic, but importantly, make it easy for yourself to change it as a later date; you will have to. The usual format journals want for vector graphics is EPS (sometimes PDF), while for pure raster graphics, it is usually TIFs at > 300 pixels per inch (ppi, but often also called dpi).

For plots/graphs

So, for a simple plot, my workflow is straightforward. In basic R graphics I just run the code (see below) which saves my plot as a PDF (I'd prefer to save directly as SVG, but R Cairo does not currently support editable text objects in SVG). You may have to make minor changes to the plot—such as trimming up margins, moving the axis labels or changing fonts etc—before it's good enough for publication. Here, we have two choices: (a) do all the plot polishing in R, and export as a final version EPS (to save as EPS, just change "pdf" to "postscript" in the code below); or (b) save as PDF, import into Inkscape, and make minor adjustments there. Generally I find that it's easier to do this kind of thing in Inkscape, rather than messing around with details too much in R. However, there is a law of diminishing returns here, which means that only very minor changes are worth doing by hand instead of taking the time to perfect the code in R. There's a good chance you will certainly have to make the plot several times in the end—co-author doesn't like it, reviewers don't like it, need to submit to different journal, etc—so think about this in advance and do the bulk of the work in R. If working on a plot in Inkscape, it is best to save it as a master copy in SVG (the native format of Inkscape), or if it's a big file, as a compressed SVG (SVGZ). You can export to EPS, PDF and a variety of other vector formats in Inkscape via the "File > Save a Copy" menu.

data(mtcars)#load up the car data that comes with R
pdf("carplot.pdf", useDingbats=FALSE)#open PDF file
plot(mtcars$mpg~mtcars$wt)#create plot
abline(lm(mtcars$mpg~mtcars$wt), col="red")#fit linear model
dev.off()#close PDF file

Fig 2. A basic, no frills R plot showing car MPG (efficiency) as function of car weight.
Fig 3. After editing the plot in Inkscape to reduce the margins and change the axis font
(please never use comic sans, it's just for illustrative purposes).

In a ggplot2 session, I do this (ggplot2 plots are pretty, so usually good-to-go with no editing at all):

library(ggplot2)#load ggplot2 library
data(mtcars)#load up the car data that comes with R
c <- ggplot(mtcars, aes(mpg, wt))#create plot
c + stat_smooth(method="lm", se=TRUE) + geom_point()#layers
ggsave("carplotgg.pdf")#save as PDF

Fig 4. A ggplot2 plot.

For mixed figure types such as multi-panel figures

Sometimes you'll need to combine several photographic images onto one figure. Again, I use Inkscape for this, but if I need to make any alterations such as contrast, brightness, or resolution, I do this first in Gimp (Inkscape cannot edit the image in this way). What I don't do is crop the photo in Gimp first. This can be done in Inkscape, and importantly, if you need to change the proportions of the figure, it can be reversed or re-edited at any later date from within the same file. A useful thing worth mentioning at this point, is the value of keeping a backed-up history of each version of the figure. This can be really handy, not only for getting out of trouble if you make mistakes, but also to keep your working folders free of dozens of copies of the same file. I recommend looking as git for this purpose.

Now, to import the raster images into Inkscape, go to "File > Import", choose the file and select the "embed" box. Align the imported objects using "View > Grid". You can also resize them by selecting them, and going to the "Object > Transform > Scale" menu, remembering to "scale proportionally" to keep the width/height ratio. Add any arrows or text boxes that are required. The format/colour of these can be changed by selecting the object and going to "Object > Fill and Stroke". A useful thing to know when making arrows in Inkscape, is that you need to go to "Extensions > Modify Path > Colour Markers to Match Stroke" in order to make the arrow head match the arrow shaft.

Journals generally want minimal margins on plots. In Inkscape I use "File > Document Properties > Resize page to drawing or selection" to trim the plot to just a few pixels round each of the edges. Experiment to see what looks best. Here's one I made earlier.

Fig 5. Two pygmy seahorse images edited onto a single figure with arrows and labels.

Again, I save this as a master copy SVG and work in that format. The figure can be exported as an EPS or PDF for the final version, but pay attention to the "resolution for rasterisation" option of the internal raster elements when you save (journals usually require 300 ppi or more, even if embedded in a vector file). Sometimes the file sizes can end up being quite big here, so what I often do when emailing figures to co-authors, or even when submitting a manuscript at the peer review stage, is to generate a low-resolution raster image for this purpose (high quality images are submitted later on in the review process). You can do this by going to "File > Export Bitmap" and select the page as the export area, and change the dpi to, say, 90. This will export a PNG file. Repeat at lower resolution if file size is still too big. Follow this process if the journal, for some strange reason, require the figure to be a raster image; if they do not accept PNG, you will need to open Gimp and export the PNG as a TIF.

In conclusion, I hope this provides some useful advice to get people started on making figures for their publications. Please don't hesitate to add any of your own nuggets of advice in the comments.

Saturday, 17 August 2013

Writing scientific papers with git and LaTeX


I wrote my last paper using the 'git' version control software.



You may have heard of git, and you may have even downloaded a program or R script from GitHub. It's been around a while (8 years), but it's only recently starting to be used by non-technical types (= biologists!). It's mainly used by programmers and web designers to keep track of changes to their code, but this applies equally to writing a manuscript. The principles are the same.

No matter how organised you are, everyone must have had at some time folders containing files called 'final.doc', 'finalfinal.doc', 'finalfinalfinal.doc', 'finalfinalfinal-version2_july12.doc', finalfinalfinal-version2_aug12_TJedit3_submitted.doc'. You get the picture. With git, this is a thing of the past. You have one file for your manuscript, and one file only. Git's magic happens in the background. Should you need to, you can roll back to any previous version of your manuscript, and it instantly changes in your working directory.

One of the things with writing manuscripts is different journals will require different formatting, or even an entirely different structure/focus of your work. Using git, we can accommodate this using the branching functions. To set up a new branch is simple, and it acts effectively as an additional, independent copy of your manuscript (although only the changes are actually stored by git—it's very efficient like that). The beauty is in the way that git allows you to transfer changes from one version to another, or merge them completely should you want.

Lets say you want to submit to Nature, but realistically you have to admit that they're unlikely to publish your important research on the length of ant's legs (but it's worth a try just in case). You branch off from 'master' into a new version called 'nature' and alter all the formatting to their requirements, but it's not the final version and you notice some typos or something more you want to change. It's easy to switch between branches, so you make the corrections, and using the 'cherry-pick' tool in git, you send only these specific changes you made in the nature branch, back to master while ignoring the new formatting. Unfortunately, you get rejected by Nature, and you decide that perhaps the Bulgarian Journal of Myrmecology is more appropriate. No problem, your master is up to date, and you just create another new branch from master (which can also be cherry picked if needed). If you are organised and wrote informative commit messages, you can even do the cherry picking at a much later date.

I hope I've demonstrated that this is a more intelligent way than copy/pasting, but one of the key features is that all authors can work on the same document at the same time, without fear of screwups. This is where git really shines. No longer will you have to email drafts out to all authors and then clean up the mess afterwards using track changes. Each person can independently work on the project at the same time and changes can be incorporated as desired.

However, there's a big snag in adopting this git approach, and as usual, it's other people. Lets be honest, it's not easy to persuade busy/important people to drop what they're doing to learn how to do something new, even if they are genuinely curious. This could make collaborating on a paper hard, which is ironic, as this is one of git's big strengths. So in my case I sent a pdf to my co-authors and received back comments annotated on the pdf. I was relatively fortunate in that my co-authors only wanted minor changes to the text, so this was not a problem to do manually. If they needed to get really stuck in, then the pdf option would have been a no-go (same goes for the dead tree option if they are in the same building).

But what's this about pdfs and this 'LaTeX' thing? Why can't git manage my Word documents? Well, git can track word documents, but it's a bad idea. Word stores its content as binary or compressed data, and while git can in theory be set to handle this, it gets complicated and unless you know exactly what you're doing, you can lose the main benefits of git—i.e. how it tracks differences between files and effortlessly merges them. Git works best with a plain text file, and therefore the LaTeX system is the obvious choice, as it stores the content of your manuscript in this format. You simply run the text file through compiler software, and a fully typeset pdf is produced. The formatting relies on 'markup language', so for example italic text would be presented as follows: \textit{Homo sapiens}. If you've ever written anything in html it's a similar idea, and not as difficult as it sounds.

However, again, the big problem with LaTeX is other people. If the journal you want to submit to is a nice modern one, then a LaTeX template will be available on their site. Conforming to their punctilious formatting rules is a doddle—you just use the template, and all is good. I've submitted to PLoS, Springer, and Elsevier* journals, and each was very straightforward (almost a pleasure). If your chosen journal does not accept LaTeX, however, you're in a world of pain. Converting to .rtf and then .doc via latex2rtf is straightforward enough, but how do you conform to their ridiculous rules (nobody could possibly peer review a manuscript if the subfigures are numbered with lower case rather than upper case letters, right?). You could do this by hand in the Word doc, but our time is just too valuable to be wasted like that. Changing these things in LaTeX is possible, but it's a royal pain the arse sometimes, especially when you need to change minutiae in the reference formatting. Besides, it goes against the LaTeX mantra of letting LaTeX take care of these things for you.

So, if I haven't put you off, how does it all work? First you install git. Next you create a local folder on your machine to hold your documents. This would just be the same as for any other file. Next, you need to set up a repository, or 'repo' as it's known in the trade. This repo is usually online, but need not be; it could be another folder on the same computer. Obviously if you wish to work on multiple machines, or you wish to collaborate, it needs to be online. There are a few options out there for that. GitHub is the most well known; public accounts are free, but if you want a private project you'll need to pay (I think some academics can apply for a free private account). BitBucket is another option that does offer free, private repos. So, what do I use? I use Dropbox. If your reading this with any knowledge of git, you'll know that Dropbox is not recommended to be used as a git repo, as it is simply not designed for it. However, the main problem lies with the fact that it can't deal with two people working on the same files at the same time—it becomes corrupted. But in my experience, if you are the only user of the repo, then that isn't a problem and it works fine.

Git essentially works by tracking your files and noticing when they have been changed. Once this occurs, the changes are now sitting in what's called the 'staging area'. When you are happy with the changes, you can 'commit' them to git, and they are assigned a unique 'hash', which acts as a permanent record of those changes. At the end of the day, you can 'push' your commits to the remote repo, and they can be accessed by you on another machine later, or by a collaborator. The main advantage of this three step system is that you can craft exactly who sees which changes and when. Git is a command line program, and although GUIs are available, it is good to start familiarising yourself with the basic commands when you learn. They are very simple (see below), and any problems/questions can be easily Googled. There's tons of information out there.

So, I will definitely be using the git/LaTeX combination in future, assuming I can convince people to join me. There's a lot more to it than I've mentioned here, but here's a few commonly used commands below, mainly to illustrate how simple it is. For further information, read these helpful git tutorials here, here, and here. If you're interested in LaTeX, the Wikibook is here.

Git is not limited to dealing with manuscripts either. I also added my figures and data there too. In fact, any version of the whole project at any time can be accessed with a single command. Another cool feature is that a repo such as GitHub can double as a preprint server, should you wish to share your results with the research community prior to journal submission.

#adds file(s) to be tracked by git
#all files in directory can be added with 'git add .'
git add manuscript.tex

#you can make some changes to several files and commit these changes (e.g. a day's work) all at once with one message (the -a specifies all)
#you can alternatively tailor your commits to apply to just one file, or just one specific edit, and this makes rolling back a specific file a lot easier
#your future self will thank you for informative commit messages!  
git commit -a -m "a message describing what you did"

#view the history of commits
git log

#send your commits to the repo
#can be set up to push automatically with just 'git push'
git push remotename branchname

#to create a new branch called 'newbranch'
git branch newbranch

#switch to new branch
git checkout newbranch

#switch back to master
git checkout master

#rollback to a previous commit
#the commits are stored as unique alphanumerical 'hashes' and can be accessed with 'git log'
#they can be truncated too.
git checkout c96c8009

#permanently reset to a previous commit: you lose all later commits
git reset --hard c96c8009

#cherry pick a specific commit and incorporate into your current branch
#need to have checked out the branch you want to cherry-pick IN to to do this.
#a tip for using with LaTeX is to write each sentence on a separate line. This will minimise conflicts (the same line getting edited in different places by different people).
git cherry-pick c96c8009

#for significant points in your timeline, add a version tag to a commit
git tag -a v2.0 -m 'version submitted to Nature' c96c8009

#compare two versions of the same file
#there are many additional options including colouring and word differences
git diff <commit1> <commit2> <file_name>


*Say what you like about Elsevier, their LaTeX support is very good.