Writing a research proposal – first steps

Science takes a lot of hard work, which can occasionally be a little repetitive and even uninspiring. At other times, it is an incredibly creative process, which requires a lot of insight into seeing how pieces of information are connected and what the bigger picture might look like.

As I mentioned before, I’ve recently been thinking a lot about post-PhD options and trying to come up with ideas for a possible project proposal. Although reading has been immensely useful for seeing what has been done already in my subject of interest, it’s been a lot trickier to identify potential gaps in the literature. This kind of creative idea generation is difficult as it can’t really be forced and just takes time and perhaps some luck. I’m reminded of the early days of my PhD, when I was trying to decide on specific analyses I could do within my chosen topic.

So, how exactly are you supposed to come up with a brilliant proposal idea? Although I don’t really know the answer to that, here is my attempt at muddling through and having a go at it.

blinking cursor

The first step of idea generation is getting something, anything down on paper, i.e. doing some brainstorming. A reasonable way to begin could be by jotting down any random ideas that come to mind and trying to think about possible gaps (no matter how frivolous or vague) regarding the existing literature in a fairly narrow topic of interest. It might also be useful to draw some diagrams. It sounds simple enough, but the longer you stare at an empty white page, the clearer it becomes that starting from scratch is daunting. I began by writing a literature summary of any studies that were relevant to an interesting observation I’ve noticed. This was a good starting point but I felt like I spent hours staring at my computer screen, trying (and most days, failing) to think of an insightful or clever way of exploring this observation further.

Once you do have some vague ideas, it’s useful to share them with colleagues, supervisors or even family and friends (particularly if they are academically-inclined) to get some feedback and have a discussion. It’s really easy to feel like you are going around in circles thinking about an idea and so having a general chat with someone else can give you new insights. If they don’t understand the connections you are trying to make, this might suggest that you need to think of the issues in simpler terms or take a step or two back. The media and popular beliefs paint a picture of science being about grand discoveries and great leaps in knowledge. In reality, we know that science is a cumulative process, typically involving many much smaller steps towards increasing our knowledge and understanding.

After some initial brainstorming and informal chats, a possible helpful next step is to do a short presentation and have a longer discussion with several supervisors/colleagues. Although this may feel slightly daunting to begin with, in my experience the prospect of doing a presentation is good motivation for transforming random half-formed ideas and jotted-down thoughts into a more coherent whole. Within the first 3 months of starting my PhD, I had to give a couple of departmental talks (the first on a research paper relevant to my PhD topic for a journal club and a second talk introducing my topic and research plans). I spent the first few months of my PhD doing little other than reading papers and preparing these 2 talks. The talks gave me a target for finalising my aims and the feedback was really useful.

The feedback from my more recent talk about my fellowship proposal ideas has similarly helped to steer me in the right direction. I’m now at the no-less-difficult stage of formulating specific aims and hypotheses and trying to design a method to address them. I have a long way to go with my fellowship proposal and I am conscious that all this effort might not even result in any funding, as the schemes are all extremely competitive. Part of my thinking was always that I would just try my best with this and learn something from the process.

Getting your paper through peer review

Whether you like it or not, productivity in academia/science is judged (at least in part) based on your publication rate. This focus on quantity is associated with various problems – see for example “Problem 4: Pressure to publish” of Chris Chamber’s recent post. Although the goal of doing science should not be the publications themselves, they remain an important method of disseminating our results and ideas to other researchers and the public. Learning how to write a concise, informative, thorough, accurate and all-round good research paper is clearly an important skill to acquire during your PhD, particularly if you hope to continue your academic career.

However, learning how to get your paper through the peer review process is also a skill that you need to acquire. The quality of your writing and your ability to include important details in your manuscript (i.e. the content) are of course directly linked to this but there are other important things too. I’m glad that the attitude of my supervisors and more generally, in my department, is that I should be publishing my PhD work when ready, as this gives me targets for writing up my thesis. Being involved in analyses and manuscripts outside of my core PhD has given me additional experience in navigating the peer review process.

Peer-Review-cartoon

(Source)

Step 1: Choosing a journal

Once you are happy with your analysis and have started writing a manuscript, you need to decide on a target journal for your work. I have written about this before and I have to agree with my previous thoughts on the matter: it just takes experience and time to get a sense of what journals publish what kinds of papers, in terms of their specific focus etc. When I’m reading papers, I tend to ignore what journal they came from, though sometimes I make a conscious effort to check this. Journal homepages frequently have a little blurb about what sorts of papers they will publish and scanning through the titles/abstracts of some of their recent issues can help. There are a couple of online tools which allow you to paste your abstract in and then scan it for keywords to give you some suggestions of appropriate journals. I’ve also tended to chat to my supervisors to get their more experienced thoughts on the matter.

Step 2: Choosing reviewers

Perhaps my least favourite part of submitting a paper is the need to identify potential reviewers. Some journals even require six different reviewers. How do you know who to suggest? Ideally, you want to identify people who have a good understanding of the background to your work and the methods you have used, are fair and have a good reputation and don’t have a clear conflict of interest. For my first few papers I relied heavily on my supervisors to suggest people based on their research interests and reputations. I kept a list of these suggestions and have re-used names for other submissions, where appropriate. Although in cases where my paper was rejected, unsurprisingly, I’ve tried to come up with alternatives.

I think the main way of identifying potential appropriate peer reviewers is by checking the author lists of the papers that are most closely linked to your own work. I’ve also found that going to seminars, talks and conferences is also useful. Seeing somebody present their work, especially if it is relevant to yours, means you are much more likely to remember their name. If you get the opportunity to speak to them, for example if you are lucky enough for them to visit your poster presentation at a conference, you can even get a sense of what kind of person they are and what they think about your work. I think this may also be particularly helpful for trying to identify potential PhD thesis examiners.

One other thing to bear in mind is that the anonymity of the peer review process means you are unlikely to find out which of the people you suggested did end up reviewing your work and which of them were the ones who gave the positive and useful comments or indeed, the less polite ones.

Dealing with rejection – back to step 1

Even after writing as a good a paper as you can, having it internally reviewed by your supervisors and co-authors, proof-read by your colleagues and settling on a journal and possible reviewers, the chances of having your publication rejected can feel depressingly high. It happens to the best of academics, sometimes for the wrong reasons (e.g. null results from a well-powered good study design) and sometimes because of bad luck in having reviewers who have misunderstood your work and not given you a chance to clarify it. I think that being able to cope  with the occasional rejection and carry on without letting it get to you personally is an important thing to learn too.

Step 3: Responding to reviewers’ comments

It’s important to bear in mind that (almost) regardless of what the comments are, being asked to do revisions is good news! I don’t think I’ve ever heard of a paper being accepted for publication without at least some minor revisions. More frequently, major revisions are called for, oftentimes multiple rounds of these. Whether it is the anonymity of peer review or tactlessness to blame, reviewers’ comments can sometimes be a little upsetting in their negativity. For some examples of particularly harsh comments, see FemaleScienceProfessor’s recent “Fake Review Contest” blog posts. Alternatively, for some slightly more humorous examples, see this Storify of the recent #SixWordPeerReview Twitter meme.

In an ideal world, the reviewers’ comments would all be very insightful, sensible and constructive, with a view to improving the manuscript. Although many comments will be quite reasonable, there are a few things to learn about writing your response to them. What should you do if you disagree on a certain point or if the reviewer is asking for something that you had actually already mentioned? I’ve been told by colleagues that if a reviewer is asking for something that is already there, you might need to make this item clearer and more prominent (especially if it’s hidden away in supplementary text for instance). As for disagreeing, if you have a genuine scientific reason, you should probably defend your work and the decisions you made, although perhaps also amend your manuscript to discuss the issue further.

Another question concerns the tone of your response. Should you be excessively polite, thanking the reviewers for their useful comments multiple times, or just get straight to the point of addressing them? You don’t want to come across as abrupt or rude. Generally speaking, you probably need to be professional and polite, but assertive if you have solid reasons backing up how you performed your experiment or chose to present the results. It can be quite useful to look at how other people write their responses to reviewers, which you can do by looking at the “peer review history” of articles published at PeerJ.

Having a little bit of experience with how peer review works may help you anticipate the sorts of comments that you might get. Being a reviewer yourself (something I have only a little experience of so far) and discussing research articles during a journal club can also highlight what kinds of things will make a good publishable research article.

Getting to know your academic ‘family’

Last week, I went to a brilliant science meeting run by the International Society for Psychiatric Genetics. In terms of making you feel like you belong to something bigger than your research group, there’s probably nothing better in academia than an international conference.

Based on my experience so far (N=2), a big benefit of conferences is seeing talks by people whose names you recognise from all those papers you’ve been reading. I occasionally get an idea of where someone works and who with from hearing them mentioned in a meeting, happening to look this up if I really like some of their work or being involved in an email conversation with them. Mostly though, people are names on a paper (or more often on a screen) to me until I see them in real life. Putting a face and personality to a name is really interesting, as is getting a better sense of the networks linking researchers in your field.

Although presumably not critical for getting on with day-to-day PhD life, I find this sort of thing intriguing and I expect it’s worth knowing for the future. If only to help with deciding who to suggest as a reviewer when submitting a paper – something I was struggling with for my first PhD paper recently. It’s also nice to see where you and your group fit in. For instance, my PhD supervisor introduced me to my “academic grandfather” (i.e. her PhD supervisor), which gave me something of a feeling of continuity. Seeing who other members of your group talk to and how people react when you say which group you are part of can also be enlightening.

Another (related) benefit of attending conferences may or may not be specific to this particular one. By all accounts, this (20th) World Congress of Psychiatric Genetics was one of the most exciting years so far, what with significant recent technological and methodological advances spurring on research in this area. Arguably a more important reason for these huge advances is the development of big international collaborations.

The human genome is marvellously complex. We are only slowly beginning to get even a vague idea (via ENCODE and smaller projects) of what the bits of the genome that don’t explicitly get translated into proteins (i.e. the exome) actually do. And yet, so many of the medical diseases and disorders humans suffer from are highly familial and heritable. Surely, pinpointing the potential genetic origin of these problems has the potential to shed at least some light on disease mechanisms and eventually lead to more understanding and better treatment options?

It’s a neat and simple idea but a lot more difficult to deliver trustworthy and meaningful results in practice. This is largely because there are a lot of genetic variants to consider, with variability in how commonly/rarely they occur. In order to have the statistical power to detect anything of genuine interest, apparently you need samples of tens of thousands of individuals – many more than even the most generous research grants are likely to allow a single research group to collect data on in a reasonable amount of time… This is where the international collaborations of research groups pooling and (generously) sharing their samples have resulted in huge progress. Although such analyses are not without their problems (like having to understand and deal with the issue of population genetic differences), seeing the results of this work and being linked into this network (albeit mostly by association) is pretty cool.

Thinking in numbers

(Source)

I read an intriguing and slightly baffling quote by Ernest Rutherford the other day; apparently, he once said: “If your experiment needs statistics, you ought to have done a better experiment.” I guess in his day the fields of psychology/psychiatry weren’t considered scientific. Statistics are so much at the centre of what I do, they’re really the sharpest tools in my PhD toolbox. However, statistics without knowledge, conceptual understanding and a thoughtful study design are worse than useless, potentially leading to all sorts of trouble like the dreaded Type 1 error (i.e. a false positive finding), which can easily occur if you simply run enough tests – see here for a good illustration of this. So in effect, statistics are a vital set of tools (at least in psychology/psychiatry – perhaps physicists will have some insight into what Ernest Rutherford was talking about) which need to be used in the right environment and in an appropriate way.

Having come a long way since learning about t-tests in my first year as an undergrad, it seems that hardly a day passes without me finding out something new about statistics. The funny thing is that I don’t really especially go out of my way to do so. It’s just that the more I look at real data sets full of real numbers describing real people, the clearer it becomes that those “textbook” examples you first see when you come across a new stats technique, can be fairly unhelpful. The problem is that so many methods rely on a number of important assumptions, like the residuals of the model you run being distributed at random (i.e. normally around the mean, like a bell curve) or variables not being overly related to one another in a given test. If these assumptions are violated, you need to backtrack and take another route. In the last couple of months, I’ve seen massively zero-inflated distributions, bi-modal distributions, heavily skewed distributions and not-positive-definite matrices. It gets to the point where seeing a normal distribution gives me a warm and fuzzy feeling inside.

In trying to learn how to correctly tackle data with these quirks, I occasionally get the impression that some other researchers either don’t notice these problems or perhaps ignore them. A bit like the impression I had when I discussed the problem of missing data previously. Though, to be fair, some of the techniques of how to address such problems and discussions around how best to use them are relatively new. Another problem is when there just aren’t enough details in a method section to provide a comprehensive “recipe” that you can follow, which is somewhat frustrating when you’re trying to learn how others use a given stats method. It would seem that my PhD examiners may be in for some pretty tedious, if thorough, methodological sections! But at least I feel like I’m putting in the effort to use the statistical technique the data call for and to understand what I’m doing.

When being organised backfires

(Source)

When it comes to making holiday or travel arrangements I’m quite well-organised, generally finding and booking a good early deal. However, my excessive zeal of planning things far in advance has now backfired twice in a row in terms of making conference arrangements. So here’s my attempt at working out why and what to do differently in future.

For my first conference, I booked flights and hotels pretty much as soon as I thought everything was confirmed (i.e. the conference organisers had emailed to accept both of my abstract proposals for poster presentations and my departmental permission to travel form was approved). I was on the verge of claiming my expenses for the flights when I received a completely unexpected invitation to also present one of my posters as a talk to a small group on the first day of the conference. Part of the invitation was a travel award (hotels and flights covered); really exciting but I’d jumped the gun and already booked and paid for things… With some wild organising, emailing, form-filling and internet-searching I managed to sort it out somewhat. The main complication from the whole situation was that I ended up staying in 2 hotels and having to move on the first day of the conference, which in hindsight, was completely impractical.

For the next conference, I took making travel arrangements a bit more slowly. There are other people I know travelling to this one so I’ve coordinated with them on what to do. We booked our flights and I had just submitted my expense claim for them when the airline decided to cancel the outbound flight and offer us a terrible indirect journey instead. It was either go with the new annoying roundabout route or try to find something better but have to undo the expense claim. Not the best choice. I ended up just sticking with everyone else and will now be travelling for the better part of a day just to get from the UK to Germany.

I guess no matter how organised you try to be, you can never prepare for the unexpected. It seems like a desire to arrange things and claim expenses back quickly and efficiently ended up complicating things for me. The trick is you need to get your timing right: do things too early and you might end up out of pocket for longer, the timetable for the conference might not be finalised and you might miss out on something at the very start or end and people you know who you were hoping on travelling/staying with might not have decided what they’re doing yet. On the other hand, delaying in booking and paying for things results in higher registration fees and more expensive hotels/flights; this can be a big deal if your budget is limited.

There is a potentially brilliantly simple solution to this issue, suggested to me by a post-doc friend of mine. Her trick is to have a dedicated credit card which is used for all conference-related expenses. Afterwards, she just prints her statement and gets all her money back, including any charges incurred. I’ve always been against the idea of having a credit card for fear of getting into debt but given the stress I feel at spending 100s of pounds of money I don’t really have on travel arrangements….well it’s kind of a moot point. I think getting a conference-only credit card may be an idea worth pursuing.

Also, for anyone out there who has yet to arrange their travel plans to an academic conference, I’ve written this short and hopefully handy guide. It may seem excessively obvious but there are those couple bits in it which could potentially confuse you at first, the way they did me.

Step 1: First you should really get some form of approval from your supervisor(s) to submit the abstract; depending on how much relevant work you’ve done, your supervisor(s) might suggest submitting more than 1 abstract; that’s pretty cool, but bear in mind that both/all of them may be accepted which = more work.

Step 2: Write an abstract in the style requested by the conference and double check with your supervisors that it’s decent enough and then submit it. Now’s also the time to apply for a travel award if you are planning on doing that.

Step 3: Wait (while maybe having a vague look at how convenient/expensive flights and hotels are going to be).

Step 4: Hopefully, you will have heard that your abstract was accepted to do the talk or poster (whichever you asked for).

Step 5: Now is the time to get official permission to travel from the head of department or postgraduate office, if your university has such a procedure – at my university, this form also involves agreeing a budget for your expected expenses.

Step 6: You should probably wait for the approval before doing anything else really…

Step 7: Register for the conference.

Step 8: Book the hotel and flights, conferring with other people who you know are also going.

Step 9: Finally, claim your expenses!

Gathering data

One of the cool things about doing a PhD in science is that the types of things you need to learn and the set of steps you go through are fairly similar to those other science (and some non-science) PhD students are going through, pretty much regardless of the actual subject and specific topic of your PhD. This can make for a great sense of belonging to a bigger group of people with common experiences, as well as providing opportunities for sharing and learning from each other’s experiences (online as well as IRL).

However, there is still a fair amount of variability and one of the ways in which individual experiences can differ drastically appears to be at one of the earlier stages, namely data collection/acquisition. This step in the process of a PhD can take up a really substantial chunk of time and energy. On the other hand, for some (arguably lucky) students it can be quite minimal, if they have access to an already existing data-set. Some subjects and topics are more suited to running your own individual experiments and collecting your own data, whereas other research questions might be impractical or even impossible to address within the scope of a PhD without access to a pre-existing database. In effect, the choice of topic for one’s PhD might automatically dictate which route (data collection or collaboration/use of other samples) would be ideal, if not necessarily available.

In the field of psychiatric genetics, there is something of a disagreement about the relative importance of quality vs. quantity – it is clear that there is so much variation in DNA and human behaviour that large sample sizes are necessary to rule out chance findings but equally, thorough measurements of each individual are essential to adequately measure the psychiatric problems of interest. The need for both of these (quality and quantity) makes personally collecting sufficient data to address even basic questions during a PhD in this field nearly impossible, hence the need for large data-sets. Although I count myself as extremely lucky to have access to some data which will be indispensable for the analyses I have in mind, I’m at the point where I’m about to set out to collect some other data which would also be really useful. This fills me with a sense of dread for two reasons:

1) as I am planning on collecting sensitive clinical information from people, I am going to need to go through an ethics panel to approve the project – having spoken to a number of friends and other researchers, I can tell this has the potential to be an exceedingly complicated and time-consuming process (see Dorothy Bishop’s post on the subject for a particularly tough account)

2) also, my target sample is a set of individuals who have already been seen previously and trying to track them down and get them to take part in a second bit of research may prove quite challenging, not to mention that those who refuse or are impossible to track down, are likely to be the ones with the most severe problems (i.e. the ones I would like to track down the most!)

Although my concerns are real, I can’t help but feel that I have been spoiled so far by not having to spend my PhD time collecting data and I know others who have had a much tougher time of it than I am likely to. There are some definite positives to be said about collecting your own data though. I know of these to some extent first-hand as I worked as a research assistant collecting some of the data that I am using for my PhD before I started it. The time and effort taken to collect the data, however tough at the time, makes you really appreciate what you have and makes you unlikely to take it for granted. Secondly, collecting the measures yourself really improves your understanding of what they are and how reliable they are.

The other potential advantage, depending on your viewpoint, is that spending a large amount of time collecting data means you have less time to do other things and the impression I get is that if you have access to data and are not collecting it yourself, you are expected to do more complicated analyses and more other types of work. The combination of having access to some data and needing to collect some is potentially quite a good one as you get the advantages and experience of both. I may change my mind though once I get started with the collection…

When science fiction becomes reality

a picture of glasses and a fake eye

(Source)

This post is a little different to my usual ramblings about PhD life, with some thoughts on how science fiction always seems just a short step ahead of science & technology – enough to captivate and tantalise some of us with the awesome (and occasionally disturbing) possibilities.

Given the combination of my long-standing interest in psychology and the constant awe and gratitude I feel at some of our day-to-day technological wonders, I really love when psychology and technology collide through art. By art, I mostly mean films and books, but there is currently an exciting-looking exhibition on at the Wellcome Trust in London, which I really want to check out – it deals with the concept of how people are continually developing new ways to enhance our bodies (from physical features such as height, to our senses and minds).

Sometimes it seems that the fictional worlds and technologies I like to watch or read about are all too realistic. Last December, I watched Charlie Brooker’s 3-part Black Mirror TV series. The last episode (which you can apparently still watch here, but probably only if you live in the UK) was about a world where everyone seems to have an implant in their brains which records everything they see and hear, with the possibility of replaying it in your own head or on a screen for others to see. Arguments are changed forever when it is so easy to just go back in time and prove to someone what exactly they said and did, in a parallel way to how nowadays, disagreements in the pub about what year something happened or whatever are quickly followed by someone putting an end to the conversation by looking up the answer online.

In a very similar vein, I remember really enjoying The Final Cut, a film about brain implants which record everything you see, so that a film of the highlights of your life can be created once you die and the implant is salvaged. Morbid, yes, but a good story nonetheless.

More recent fictional ideas I’ve seen have gone some steps further, beyond implants recording information to reality augmentation. Last week, H+, a new web series of (very) short episodes started. It’s set in a world where people have a brain implant connecting their minds to the internet. On the other hand, in a short film project, called Sight, the fictional technology allows you to play games and watch TV in your head, get nutritional information, plan your wardrobe and have access to all sorts of other information.

Which brings me to Google glasses, a real item, which looks like it might be on the market someday very soon. These appear to be a version of a smartphone worn as glasses, with a little screen and camera in one corner. They give you access to a seemingly infinite number of apps, as well as recording and being able to share what you see. Sound familiar? Although I’m not convinced how practical these will actually be and definitely not keen on the potential for marketing and ads getting in the way, I can easily envision a future where they are as common as smartphones, which themselves can be viewed as an enhancement – a way to store information without needing to try to remember things and a combination of unlimited tools, given that there is basically an app for everything nowadays. I wonder if the Wellcome Trust exhibition will have mention of Google glasses and other such reality augmenting technology.