Significance - NIH Style

Courtesy of Nick Page on Unsplash

​The significance section of an R01 is probably the most misunderstood part of the entire NIH grant. Perhaps this should come as no surprise, as this section has been a bit of a moving target over the past several years. Another source of confusion may be that Webster’s definition of significance and the NIH’s definition are “significantly” different. A brief history lesson may shed some light on why the significance section is a bit murky and will put today’s directives in context.
 


A brief history of significance
            Way back in the dark ages, when getting your NIH review in the mail as a “pink sheet”, the R01 proposal was 25 pages long. Significance was included with background, and given the overall length of the grant, background often expanded into a verbal upchuck of information about the topic at hand. In 2009, the powers that be decided to give the reviewers a break and the 25 page limit was reduced to 12. The background and significance section slimmed down and was intended to highlight key findings rather than review the world’s literature. In 2016, spurred on by the need for public accountability, NIH grants underwent a major overhaul and the significance section had a new set of instructions. Significance was to include the word premise, which was defined by the NIH as “the quality and strength of the prior research used as the basis for the proposed research question or project; this is distinct from the hypothesis or justification.” The significance section was also to describe the strengths and weaknesses of the prior research. Since these initial instructions, the word premise has been dropped and quality and strength of prior research is addressed as, wait for it…., rigor. 
Webster defines significance as “the quality of being important”. Investigators must like this definition, as most significance sections that I have read include something about the importance of the work. However, the NIH forgot to check Webster’s, and if you only include statements about the importance of the work your significance section will miss the mark. The NIH assumes success of your specific aims and wants to know “how will your work, successfully carried out, impact the field?”. I put the word successfully in italics, because if your work is founded on weak preliminary data and/or a poor approach, successful completion of aims will not have a major impact on the field.  
 
Significance today
      Knowing the evolution of the significance section might explain the potential differences that you’ll see either as a reviewer or in reading examples of successful grants. Now let’s head back to the future. What are reviewers looking for in today’s version? The current significance section looks for impact based on the rigor of prior research.  What is rigor of prior research you ask?  Right from the horse’s orifice of choice, rigor of prior research is defined as “the quality and strength of the research being cited by the applicant as crucial to support the application; this is distinct from the hypothesis or justification.” It’s premise without the word premise. 
            As part of the rigor of prior research, the applicant is supposed to address the “strengths and weaknesses of the prior research used to support the application and describe how the proposed research will address weaknesses or gaps identified by the applicant. This may include the applicant’s own preliminary data, data published by the applicant, or data published by others.” Here is your opportunity to discuss the strengths of your preliminary data (for an R01) or published data (R21, R03) that are the foundation of your hypothesis. The weaknesses you will, of course, be addressing in the grant.  
 
Structuring your significance section
            Although the exact order isn’t critical, the elements described above must be included in your significance section. I suggest starting with a brief paragraph restating the critical need your proposal addresses and how successful completion of your proposal will change the field. Remember, this is the NIH definition of significance. If you have a surprising statistic that supports the need here’s the spot.  In the next paragraph restate the hypothesis, followed by the strengths of your key supporting data. I don’t think it’s over the top to include a sub-header entitled “Rigor of prior research”.  It’s hard for a reviewer to claim this wasn’t addressed when you have it written as a separate section.  From there you discuss the weaknesses, which prepares the reviewer for the approach section where they will be addressed. 
 
Here’s an example of a significance section, which is taken from a proposal using e-yarn as a way to detect lost socks. This example highlights my approach, but as the saying goes, there are many ways to skin a significance section.
 
Socks are the number one lost laundry item in the United States. Every household is, at some point, plagued by the problem of lost socks. Lost socks generate substantial psychological and financial burdens on many households, which escalate exponentially in families with 8 feet or more. Last year alone, replacement costs for lost socks reached 1.2 million dollars, and this cost is on the rise. Experts from the Simpson Family Foundation, a think tank focused on family issues, project that by 2040 the problem of lost socks will surpass health care as the number one drain on family budgets. New ways to find lost socks must be found now to avert this future financial catastrophe.
 
Rigor of prior research: We hypothesize that socks made with e-yarn will be detectable within an average size US home (2000 sq feet) using a barcode cell phone application. This hypothesis is built on strong data demonstrating that detectability of e-yarn is resistant to dye (Figure 1), water (Figure 2), and that 1 g of e-yarn is currently detectable up to 2000 feet (Figure 3). Based on these data, the proposal that follows will develop a unique, cost effective way to find lost socks. Successful completion of this proposal will represent a significant step towards eliminating the financial and psychological burden of lost socks. A side benefit of this approach is that, if desired, family members with similar sized feet can identify their own socks. Finally, this technology can be applied to other problematic clothing items such as cufflinks and earrings. 
This proposal will address the following weaknesses in preliminary and published data. Our preliminary data shows that 1 g of e-yarn is detectable by RFID up to 2000 feet. However, we have not yet tested smaller amounts of e-yarn. Since it is not cost effective to make an entire sock of e-yarn, we will determine the smallest amount of e-yarn that can be woven into a sock and remain detectable. This question will be addressed in Aim I. 
Many studies have taken a compensatory approach to the problem of lost socks rather than finding the sock itself, which is the goal of this proposal. Some studies, for example, suggest that purchasing a 6 pack of socks can address the lost sock problem.2-5 However, these studies have only used white athletic socks and do not take into account more expensive dress socks. Another approach, reported by Hanes, et al, was to wear mismatched socks.6 However, their study was limited to 15-25 year olds, who have been shown to enjoy wearing mismatched socks.7-9 A similar approach, wearing only one sock, was supported by findings of Scholl, et al.10 However, this study involved a more sedentary population. When tested in a younger, more active population, Adidas et al found that the one sock approach was not feasible due to foot sores and odor, particularly when studied in student athletes traveling together on buses.11,12
 
As outlined above, this example starts with a restatement of the problem, adding a little more detail than what was presented on the specific aims page. From there, I restate the hypothesis, so that it is fresh in the mind of the reviewer as he/she goes on read the key data supporting your hypothesis. These key data are the “strength” of the strength and weaknesses of supporting data. Make sure that you demonstrate the strength of your data with figure legends that support rigor and reproducibility (positive and negative controls, statistics, numbers of replicates and experiments/animals, etc). After listing the key supporting data, I describe what will happen upon successful completion of the proposal. Because this section focuses on the proposal, rather than the problem, the successful completion part flows nicely from here. I then finish by discussing the weaknesses/gaps that will be addressed.
 
Where’s the background?
            Since the NIH got rid of the “background and significance” section it’s dealer’s choice as to where to the background. Do you even need a background section and if so, how much? Yes, you need some background information. I think of background as a “need to know” section. Include what the reviewer needs to know so that he/she understands your work and its context. Some investigators still put this information in the significance section, and others put it at the beginning of the approach. I do not recommend putting your background with the significance, because you risk reviewers missing the information the NIH asks for (and you are scored on) by getting caught up in background details. 
 
If nothing else, remember that the significance is the impact of your proposal, if completed successfully, on the field. Reviewers are instructed to score grants with this idea in mind. Now time to do some laundry.

---