Like other database resources, we collate measures of impact for various uses. These include our own internal bi-annual IUPHAR/GtoPdb review meetings, documenting outputs of past grants and applications for new ones. While there is a widening choice of these metrics (including website accesses, data downloads, hits on Open Access papers, papers linked to grant numbers and Altmetrics scores) citation counts remain an important component. Recently, we were asked to include these in our application to become an ELIXIR Europe Core resource (following on from our successful incorporation into the ELIXIR UK node). In addition, related to a recent funding pre-application, we have gratefully received recommendation letters, one of which happened to highlight our citation achievements. What we now need to explain here, underpinned by data, is why some of our recent citation counts appear anomalously high (notwithstanding the fact that different sources generate different numbers anyway). We will also broaden the scope of this post by mentioning other metrics and promotional strategies.
As a technical preamble, the good news is that PubMed (including PubMed Central, PMC) and European PubMed Central (EPMC) now have divergent but powerfully complementary bibleometric feature sets for the same 22 million abstracts and 4 million full texts (e.g. Entrez and MeSH x-refs on one side vs EBI x-refs on the other) . The bad news is, as usual, that anyone wanting to explore bibliometrics in detail is now faced with a two-stop-shop (actually a de facto transatlantic split). This is manifest below where I have to hop between the two, depending on what filters I need. The fact that EPMC citation are significantly lower than for PubMed, Thomson WOS or Google Scholar (in ascending order) may be related to the PMC embargo periods for BJP.
Our Nucleic Acids Research Database papers
Including our previous incarnation as IUPHAR-DB, the Edinburgh team and their collaborators are a tad proud to have accumulated five papers in Nucleic Acids Research Annual Database issues. These are listed below, along with the citation counts from EPMC (which was what the ELIXIR form specified).
||Harmar et al., IUPHAR-DB: the IUPHAR database of G protein-coupled receptors and ion channels
||Sharman et al., IUPHAR-DB: new receptors and tools for easy searching and visualization of pharmacological data
||Sharman et al.., IUPHAR-DB: updated database content and new features
||Pawson et al., The IUPHAR/BPS Guide to PHARMACOLOGY: an expert-driven knowledgebase of drug targets and their ligands
||Southan et al., The IUPHAR/BPS Guide to PHARMACOLOGY in 2016: towards curated quantitative interactions between 1300 protein targets and 6000 ligands
Up until 2013 we consider our citation counts for the three papers (depending on comparative benchmarks of course) as not only realistic but also quite respectable. However, even though it was associated with our IUPHAR-DB to GtoPdb re-branding campaign in 2012, the 10-fold citation jump between our 2013 and 2014 efforts is clearly anomalous. As it happens, the causality is easy to spot from PubMed (because the Boolean result set combinations cannot be done directly via EPMC). Performing the appropriate intersects in PubMed shows that, from 621 citations, no less than 588 (i.e. 95%) come from the British Journal of Pharmacology (BJP). This reason for this is because our paper was selected as one of the reference citations derived from the Tables of Links (ToLs). These valuable connections between entities specified by authors and GtoPdb entries, were initiated in November 2014. An example legend from within a 2015 paper is shown below.
The selected references in the table legend include our 2014 NAR, along with whichever reviews from the Concise Guide series matched the theme of the specific article. The key point is that the BJP Wiley Editors added these to the author citation lists as obligatory “reference citations”. Thus, for this article from 2015, these would typically include PMID 24234439 , PMID 24517644 and PMID 24517644. Since the ToLs are included in most BJP papers these rapidly accumulated citations with EMPC counts of 477, 421 and 388, respectively. As of 2016, the updated reference publications have notched up by two years and the table legends thus point to the newer titles of PMID 26464438, PMID 26650445 and PMID 26650439. Consequently these have also accumulated citations quickly with counts of 76, 189 and 147, respectively. These may even eventually outpace the previous set since the ToLs (although for a lower proportion of articles) have also recently been introduced into the British Journal of Clinical Pharmacology (BJCP). Thus, from 128 PubMed cites for PMID 26464438, BJCP has contributed 25 and BJP 85.
BJP citation rankings
To get an overview of BJP internal reference citations (i.e. not the external NAR papers) we can view their 24321 papers ranked by citation count (a selectable setting in EPMC but not in PubMed). Unsurprisingly, first and third places are taken by reference citations in the form of two successive editorials on “Animal research: reporting in vivo experiments: the ARRIVE guidelines” as PMID 20649561 , with 898 cites and PMID 20649560 with 830. A subsequent section of these results is shown below, from fifth to 11th place.
Related to GtoPdb and earlier Edinburgh outputs, we can see the 2011 and 2008 versions of “Guide to Receptors and Channels” (GRAC) were also selected as reference citations, consequently ranking then at fifth and seventh. As is implicit from above, since the introduction of ToLs, the “Concise Guides” (effectively the successors of GRAC) have now replaced these as reference citations (n.b. depending on the time elapsed from Jan 2017 the live citation numbers in the table above will have increased in the links). Additional aspects on this citation theme are expounded in this blog post.
In addition to keeping an eye on citations per se we also folow up on some of the newer ways of increasing the findability and connectivity of our work in the ever more complex bibleometrics/Social Media ecosystem. These efforts are modest (compared to what can be done) since we have our heads down for the Day Job but some of them have become necessary house-keeping . These include grant linking, the addion of ORCHIDs for team members (both of these as EPMC functionality) and making sure papers are entered into our very own Edinburgh Research Explorer (actually highly ranked in Google for title searches).
Two other aspects may be of interest (they can’t be detailed here but background is in the links). The first of these is the use of PubMed Commons. that has several utilities for us, including being able to “daisy chain” forward citaion pointers (but you wont see them in EPMC yet). For example, amoung the 73 PubMed citations for our 2009 NAR paper, 7 are 2015 and 2 from 2016. Thus, some recent authors are still citing our oldest paper (we see this across the series in fact but, to be fair, some of them could be giving us the courtesy of multiple NAR cites although I have not checked). We therefore came up with the strategy of adding discrete pointers in PubMed Commons. As it happened, the last one (pictured below) was added most recently, even thought it is first in the chain by abstract date.
So, if the scholars in question happen to check PubMed (n.b. but not BJP authors, since the most recent NAR reference would have been added by the Editors anyway) we have now have a set of comments to point the four older papers forward to the fifth 2016 paper (and should we be fortunate enough to get accepted for a future NAR Database issue, we would then add a new comment to the chain). Consequent to the posting above, an unexpectedly prominent ping appeared below, on the 2nd of Feb, highlighted in yellow.
In a nutshell, “Featured comments” just happend to automatically select ours (but it seems like an actual human edited it) which consequently featured on the PubMed front page, no less, giving us 24 hours of micro-fame! As icing on the cake, the concomitant dailly auto-tweet of the heuristic chart-toppers, shown below, reached 4394 followers of the PMCom account (and was re-tweeted by us of course)
Continuing on the metrics theme, in the panel below you can see Altmetrics scores for our same five NAR papers, plus the one Kudos entry.
These outlinks can be found under the right-most “External links” tab on any EPMC entry that has them. The Altmetrics Rosette and sub-scores give a general measure of interest asssociated with a paper (but dont forget this may not necessarily be completely positive) broken down by category, as you can see for our 2016 NAR below;
Interesting aspects of Almetic scores include that they are faily immediate (i.e. accumulating within the first month or so) and tend to move in the oposite direction to the slower accumulation of cites (i.e.they flatten off). Here again, we alow ourselves a little warmth of feeling to see that the Altmetrics hueristics (while not incontravertable) puts us close to the top-10% of comparable publications for both our GtoPdb NARs (i.e. we got the word out). The older papers, published during LBA (Life Before Altmetrics), clearly pick up lower scores. To conclude by putting it on the record, we are most appreciative of colleagues and compatriots who explicitly draw positive attention to our work in both traditional and altenative ways.