Help

Overview

iCite is a powerful web application that provides a panel of bibliometric information for journal publications within a defined analysis group (where an analysis group can consist of a single article or a very large group of articles). The data produced by iCite can be downloaded as a customized report from the dashboard and could be used to understand the influence of articles within an analysis group. An example application for iCite might be to compare how the influence of a portfolio of articles compares to the remaining articles that come out of grants funded by the NIH.

The following data are produced using iCite:

The Relative Citation Ratio is a new metric developed within the Office of Portfolio Analysis (OPA) that represents a citation-based measure of scientific influence of one or more articles. It is calculated as the cites/year of each paper, normalized to the citations per year received by NIH-funded papers in the same field and year. A paper with an RCR of 1.0 has received the same number of cites/year as the median NIH-funded paper in its field, while a paper with an RCR of 2.0 has received twice as many cites/year as the median NIH-funded paper in its field. The displayed values are the average and standard deviation of the papers in the group along with the median.

iCite is limited to analyzing only articles that appear in PubMed; users upload the PubMed IDs for articles within the analysis group of interest (articles identified from either SPIRES or PubMed searches). At present, only PubMed citations are included, so citations appearing from journals outside PubMed are not counted. iCite allows for the analysis of just one group of articles or for the comparison of two separate groups as appropriate.

Gathering data from PubMed

Use this method to search for papers in a topic area or for individual authors. The following method is for identifying articles about "neuronal migration" published between 2010 and 2013. Be thoughtful! Newly published articles will not have any citation statistics associated with them, but the default sorting option in PubMed is newest to oldest. Use date filters to help select articles that have been around long enough for meaningful citation counts to accrue.
  1. Go to PubMed.
  2. Search for the topic of interest ("neuronal migration" for example)
  3. On the left sidebar, find "Publication dates" and choose "Custom range"
    Publication Dates, Custom range
  4. Type in a year range (2010 to 2013) and click "Apply"
    Custom date range of 01/01/2010 to 12/31/2013.
  5. There are two options for retrieving the PubMed IDs (PMIDs).
    • If you just want the top 20, you can click on "Summary" and change the view format to "PMID List":
      To get a list of PMIDs, change the Summary drop-down menu to PMID List instead.
    • This displays a list of PubMed IDs that can be copied and pasted into iCite.
    • If you want the full list, choose "Send to", "Choose Destination: File", and then change "Format: Summary" to "Format: PMID List" and click "Create File"
      To export up to 5000 PMIDs, select the Send To drop-down menu. Choose Send To File. Choose the PMID List format instead of Summary (text).
    • This will download a file called "pubmed_result.txt" containing up to 5000 PubMed IDs to whichever folder is the default save location for this browser. Generally, this is in the user's "Download" folder.

Loading data into iCite

  1. Go to iCite's New Analysis page.
  2. For analyzing a single group of papers, either:
    • Paste the PMIDs into the text box,
      Paste PMIDs that you have copied from another program into the text box.
    • or upload your single-column file with the list of PMIDs to be analyzed.
      Upload a text or spreadsheet file containing your PMIDs using the file browser.
    • Then click "Process".
      Click Process to process your data.
  3. For analyzing two or more groups of papers, upload your two-column spreadsheet (PMIDs in one column and group label in the other).
    For analyzing two or more groups simultaneously, upload a two column spreadsheet using the file browser.
    • Then click "Process".
      Click Process to process your data.

Analyzing a single group with iCite

Once the data have been processed, two summaries appear on screen: a table with summary data, and graphs with a year-by-year breakdown.

Table:

Screenshot of the summary statistics table, which shows Total Pubs, Pubs per Year, Cites per Year, Relative Citation Ratio (RCR), and Weighted RCR.

Metrics describing the set of selected papers that were uploaded are displayed in this table. The metrics given in the table are Total Pubs, Pubs/Year, Cites/Year, RCR and Weighted RCR. Total Pubs, Pubs/Year and Weighted RCR are given as single values, but for Cites/Year and RCR, the maximum, mean, Standard Error of the Mean (SEM), and median are all shown. Users can click on the column labels to see detailed descriptions of how these metrics are calculated. These are summary statistics for the whole group, but yearly breakdowns are shown for Total Pubs, Mean RCR, and Weighted RCR in the graphs below.

Below are the descriptions that are shown when users click on the column labels:

Graphs:

Histograms show a box-and-whisker chart of the article RCRs, Pubs per Year, and Weighted RCR per Year.

These histograms show:

Basic selection options:

There are some options for selecting and deselecting subsets of the data based on publication year or type. These options are available if there are fewer than 1000 publications.

Drop down boxes let you change the year range of analysis.

The year ranges give the option of selecting only documents in certain time windows. The "Exclude non-articles" check box deselects papers flagged as non-articles. Values from these de-selected papers will not be included in the calculations below. After making changes here, the summary information and graphs are automatically updated.

The Exclude non-articles checkbox lets you exclude any articles that may be derivative work like reviews or news articles.
Histograms automatically update as these selections change.

Working definition of 'articles'

To be counted as an 'article', a document must have one of the below PubMed "Publication Type" tags indicating a possible research article, and none of the below "Publication Type" tags indicating derivative or non-research documents:

Must have at least one to be counted as 'article': Cannot have any to be counted as 'article':

Based on manual curation of a random sample of articles published in 2013, this classification strategy yields 92% accuracy. 8% of 'article' assignments on average were incorrect, while 5% of 'non-article' assignments were incorrect. Most errors tend to occur in journals that are not indexed in MedLine, suggesting that persons whose publication records are concentrated in non-MedLine journals may have disproportionately high error rates in this classification schema. We are working to address this issue.

Coverage information

Note that in some cases, PMIDs cannot be matched to citation records. In this case, a warning appears:

Not all requested data was found.

Clicking "View details" displays the list of missing articles and common reasons why the PMIDs were not matched:

Clicking View Details next to the warning shows which PMIDs were not found, and highlights common reasons why.

Most often, this occurs because articles are outside the year range for available citation data. We currently have citation data for papers from 1995-present, but note that papers published in the most recent year (some of which are only a few months old) have not had enough time to accrue a meaningful citation count. For this reason, only articles that are at least a year old are available for calculating RCR. Recent papers may also show stronger seasonality patterns and discrepancies because of online publication preceding print publication. Papers that are published early in the calendar year or with long online-only publication periods will have a longer period in which to accrue citations.

Customization options for small groups (<1000)

If your dataset is larger than 1000 papers, you are limited to the basic selection options described above. However, if your dataset has fewer than 1000 papers, there are further customization options. You can see the title, authors, article type, journal, and RCR of each paper:

Click 'Download selected results' to download the raw data.

Users can select and deselect individual papers with the checkbox on the left (for example, deselecting a person's middle-author papers). In addition, whole years can be selected or deselected. "Select/deselect all" can be used to reset the individual selections.

Downloading article-level data

iCite users can download the article-level data underlying the summary statistics. Use the "Download selected results" button to download the selected results as a spreadsheet.

The article selection section lets you select or deselect articles individually to include or exclude for analysis.

Clicking this button will download the selected (but not the deselected) articles as a spreadsheet with additional bibliometric information. This will appear in your web browser's default download folder.

Differences when analyzing two or more groups with iCite

To compare two or more groups in iCite, users must upload a two-column spreadsheet: one column with PMIDs and one column with group labels. For this example, publications were selected that were linked to NIH grants falling into one of two Program Class Codes (a neuro-development set and a neuro-endocrinology set):

Two-column spreadsheets should have one column with PMIDs and one column with the group name for that PMID.

Table:

As with an analysis of a single group, a table appears toward the top of the page with bibliometric information displayed:

The summary table shows summary stasistics for each group separately.

Graphs:

For multiple groups, summary statistics are calculated for each group separately. If there are multiple groups, toggle between graphs for each group by selecting the eye icon for your chosen group.

Separate histograms are shown for each group.

Downloading data

The procedure for downloading data is the same for multiple groups as it is for single analysis groups. However, the downloaded file will contain a column with the group labels so that the publications can be analyzed outside of iCite (for example, with Pivot Tables in Excel).

Support

What browsers are supported?

We seek to support all modern browsers and platforms. If you experience a problem using the current version of any of the following, please let us know:

How can I send feedback?

Send an email to iCite@mail.nih.gov.