Content and Bibliometric Analysis of Articles Published in the Journal of Orthopaedic & Sports Physical Therapy

Content Analysis

Eligibility Criteria The primary author (R.A.C.) screened all manuscripts for inclusion that were published online on the JOSPT website ( www.jospt.org) from the years 1980 through 2009. Articles classified as research reports, topical reviews, or case reports were included for further analysis (TABLE 1), and uncertainty regarding inclusion was resolved by discussion with a second reviewer (S.Z.G.). Of the 2233 potential JOSPT publications screened, 1732 (77.6%) articles met inclusion criteria for use in addressing the primary aim. TABLE 2 lists the type, number, and percent of articles included and excluded from this bibliometric analysis.

Coding System In the current analysis, we utilized the same coding system as that used in our previous study,⁴ the domains of which included type of article, type of participants, type of research design, purpose of article, clinical condition, and type of therapeutic intervention. One domain from our previous study, physical therapy focus, was not included because all JOSPT articles would likely have been uniformly coded (musculoskeletal) in this system.

TABLE 1 lists the domain codes and definitions for type of article, type of participants, and purpose of article. Additional information about the participant sample was obtained with 3 subdomains: sample size, mean/median age of sample, and number of males and females included. This information was extracted from the article and manually entered. If a mean/median age was not provided, an attempt was made to calculate this based on the information provided. We adapted a decision tree and algorithm from the Centre for Evidence Based Medicine website ( www.cebm.net) for coding of the research design domain.³FIGURE 1 illustrates the decision tree used and the final domain codes used for each article. We adapted search terms from the Physiotherapy Evidence Database website ( http://www.pedro.org.au/) for coding of type of intervention domain.¹⁰ Nineteen intervention codes were used, which included acupuncture, education, electrotherapies, heat and cold, fitness training, gait and locomotor training, health promotion, hydrotherapy, manual therapy, mechanotherapy, neurodevelopmental treatment or neurofacilitation, other, psychosocial/behavior, respiratory therapy, skill or task training, stretching and range of motion, strength training, work hardening, and wound therapy. Finally, the clinical condition domain involved open-ended entry by the reviewer. Clinical conditions were ultimately combined by the primary author, to reduce redundancy and to provide a concise list of common pathologies.

Download FigureDownload PowerPoint

Only 1 domain code could be selected for type of article, type of participants, purpose of article, and research design. A maximum of 2 intervention codes could be selected and 3 clinical condition entries made. Readers interested in more detail on the intervention codes are encouraged to refer to the previous publication,⁴ which contains a more thorough description and expanded definitions.

Reliability of the Coding System We conducted 3 separate intertester reliability trials, similar to those described in our previous study.⁴ Two blinded reviewers (W.A.W. and C.B.S.) coded articles for the domains of type of article, type of participants, research design, and purpose of article. Prior to the reliability coding, the primary author (R.A.C.) and reviewers (W.A.W. and C.B.S.) met for 1 hour to discuss and review the coding system. Open communication between each reviewer and the primary author was allowed, but coding between the reviewers was blinded.

The reliability trials occurred during the first week (n = 15), at 2 weeks (n = 50), and at 4 weeks (n = 25) of independent coding. The total independent coding time period lasted approximately 8 weeks. Percentage agreement, kappa value, and 95% confidence interval for kappa were calculated from the combined data of the second and third reliability trials.²² These times were chosen as representative data because training occurred after the first reliability trial and no additional modifications were made after this stage. There was good agreement between the reviewers for all coding domains: type of article (percentage agreement, 99%; kappa, 0.97; 95% CI: 0.92, 1.00), type of participants (percentage agreement, 94%; kappa, 0.91; 95% CI: 0.83, 1.00), research design (percentage agreement, 75%; kappa, 0.70; 95% CI: 0.58, 0.83), and purpose of article (percentage agreement, 86%; kappa, 0.81; 95% CI: 0.70, 0.93).

Bibliometric Analysis

Bibliometric information was obtained by identifying all included JOSPT articles in the ISI Web of Science database ( http://apps.isiknowledge.com). A search of this database for JOSPT documents published from 1980 to 2009 yielded 1793 articles. This list was cross-referenced and matched with the list of included articles (n = 1732) obtained from the JOSPT website ( www.jospt.org). Six hundred nine documents from the ISI Web of Science search were excluded due to being either previously excluded from our content analysis or published abstracts not listed on the JOSPT website. As only articles published after July 1992 were available on the ISI Web of Science database (n = 1184), data for articles previous to 1992 (n = 548) were not included in the bibliometric analysis. We exported the resultant list of 1184 articles from ISI Web of Science into HistCite (Thompson Reuters, New York, NY) bibliometric analysis and visualization software. HistCite is a software program with applications that can further analyze bibliographic information (eg, citation analysis, institution frequency, author frequency, etc). Finally, we exported the HistCite results into Microsoft Excel (Microsoft Corporation, Redmond, WA) for data management. Data management primarily included adding number of publications for author and institution, because the HistCite software did not account for slight differences in name (eg, Smith A versus Smith AB). All other data are presented as tallied by HistCite software.

Statistical Analysis

We used SPSS for Windows, Version 17 (SPSS Inc, Chicago, IL) and Microsoft Excel 2007 for analysis of descriptive information and trends. Linear regression was conducted for each of the 4 domains (type of article, type of participants, type of research design, purpose of article) to determine if positive or negative trends were identified during the 30-year period. Because a linear function was assumed for these trend analyses, lack of a trend was associated with higher probability values (eg, P>.05) for the regression models. Visual inspection of the trends for each domain confirmed our linear assumptions, so only linear trends were tested in these analyses. Standardized beta coefficients and R² values were computed. The R² value, which generally indicates the total variance explained by the prediction model, reflected in this analysis (a) whether a linear trend was likely to be present for publication year and number of publications, and (b) the magnitude of the linear trend (if present). The standardized beta coefficients reported in this analysis then indicated whether there was a positive (increasing) or negative (decreasing) trend for publication year and number of publications. Specifically, the standardized beta coefficients reflected how many standard deviations of publications would change as a result of an increase of 1 standard deviation in the publication year.

Content Analysis

Of the 1732 articles (1980–2009) included in the analysis, 1172 (67.7%) were research reports, 351 (20.3%) were topical reviews, and 209 (12.1%) were case reports. There was a mean of 58 articles published per year (range, 21–81), with peak publications in 1994 (81), 1998 (78), and 2007 (77). An increase in the number of articles published was observed over the 30-year period (standardized beta, 0.691; R² = 0.478; P = .00002) (FIGURE 2).

Download FigureDownload PowerPoint

JOSPT Publication Content FIGURE 3 represents the percentage of the 1732 articles, classified by type of participants (FIGURE 3A), research design (FIGURE 3B), purpose of article (FIGURE 3C), and therapeutic intervention (FIGURE 3D). Percentages for type of participants, research design, and purpose of article were computed from the total sample of 1732, while percentages based on therapeutic intervention were computed only from articles coded as either treatment or anatomy/physiology (eg, mechanistic studies) (n = 1200). The highest percentage of papers for each domain was asymptomatic adults (type of participants), cross-sectional studies (research design), anatomy/physiology (purpose of article), and strength training (intervention).

Download FigureDownload PowerPoint

The top 25 clinical conditions reported in JOSPT from 1980 through 2009, also computed from the total sample of 1732, are listed in TABLE 3. The top 5 most common clinical conditions were low back pain, anterior cruciate ligament (ACL) pathology, patellofemoral pain syndrome, rotator cuff disease/shoulder impingement, and ankle sprain.

The median sample size for articles reporting on human participants (n = 1331) was 24. Articles reporting on a sample size of 50 participants or less (78.5%) was the most prevalent, followed by 51 to 200 participants (9%), 201 to 1000 participants (2.1%), and 1001 and greater participants (0.4%). Of the articles reporting sex of participants (n = 1020), 48.5% of participants were female. Of the articles reporting mean age (n = 1197), the majority included human participants between the ages of 25 and 44 years (48.2%), followed by those aged 19 to 24 years (31.1%), 45 to 65 years (10.4%), birth to 18 years (7.6%), and finally those over 65 years (2.8%).

JOSPT Publication Trends A percentage increase over the 30-year period was noted for research reports (standardized beta, 0.427; R² = 0.182; P = .019), while a percentage decrease was noted for topical reviews (standardized beta, −0.615; R² = 0.378; P = .0003). There was no trend for case reports over this same period (standardized beta, 0.129; R² = 0.017; P = .497). For type of participants, we observed an increased percentage over time in studies including symptomatic participants (standardized beta, 0.795; R² = 0.632; P = .0000002). There was no trend in percentage of studies including asymptomatic participants (standardized beta, −0.298; R² = 0.089; P = .11).

For research design, there has been an increased percentage in systematic reviews (standardized beta, 0.648; R² = 0.42; P = .0001) and a concurrent decreased percentage in nonsystematic reviews (standardized beta, −0.613; R² = 0.375; P = .0003). No change in percentage trends was observed in the other research design domains, including randomized-controlled trials (standardized beta, 0.309; R² = 0.095; P = .097) or cross-sectional studies (standardized beta, 0.238, R² = 0.057, P = .206). For purpose of article, an increased percentage in prognostic studies (standardized beta, 0.599; R² = 0.359; P = .0005) and a decreased percentage of anatomy/physiology studies (standardized beta, −0.589; R² = 0.347; P = .001) was noted over the 30-year period.

Bibliometric Analysis

Results from the bibliometric analysis are presented in TABLES 4 and 5. These are top 20 lists for most frequently published authors, institutions, cited journals, cited JOSPT articles, and cited references from the available articles published from 1992 to 2009 (n = 1184, 68.4% of articles published from 1980 to 2009). Over this time span, the most commonly published author and institution have been Gross MT and the University of Pittsburgh, respectively (TABLE 4). The most frequently cited journals (excluding JOSPT) have been the American Journal of Sports Medicine, Physical Therapy, Spine, Journal of Bone and Joint Surgery — American, and Medicine & Science in Sports & Exercise (TABLE 4). Roos's paper on knee injury and osteoarthritis outcome score was the most frequently cited article published in JOSPT, while Shrout and Fleiss's paper on intraclass correlations published in Psychological Bulletin was the most frequently cited reference found in articles published in JOSPT (TABLE 5).

JOSPT Publication Content and Trends

There has been significant publication growth in JOSPT since 1980.²³ First, the total number of publications has increased, which may be associated with a change in the number of issues printed per year. JOSPT shifted from quarterly issues to monthly issues in 1987.²³ Second, the proportion of research reports to total publications has increased, while the proportion of topical reviews to total publications has decreased. This observation may be an indirect measure of change in the JOSPT's emphasis or a reflection of the evolution of physical therapy research.

Topical nonsystematic reviews are lower level evidence as compared to empirical studies, and a trend of this nature is encouraging for a profession with explicit interest in evidence-based practice.³ However, topical reviews, of which expert opinion and commentary are a part, are still of interest to JOSPT readers. First, these reviews can thoroughly describe individual perspectives on current clinical or research topics. Second, expert opinions and commentaries can be provocative pieces that spark future analytical research through generation of novel hypotheses or by highlighting current controversies in clinical management.

The increased percentage of research reports over time for JOSPT is similar to findings observed in prior bibliometric studies.^4,10,16,17 Interestingly, this observation did not occur with a concomitant increase in percentage of randomized controlled trials. Few studies have examined analytical studies published in physical therapy journals outside of randomized controlled trials.^13,17,19 Both Paci et al¹⁷ and Robertson¹⁹ found an increase in the proportion of analytical studies, including randomized controlled trials, within physical therapy journals. While randomized controlled trials are heralded as the highest standard for intervention research, it is unclear whether increasing the number or percentage of randomized controlled trials is ideal or realistic.¹² Publishing more randomized controlled trials within a journal with limited space yields a smaller proportion of other types of articles that may also impact clinical practice, for example, diagnostic or prognostic studies. Also, there is a recent emphasis that both subjects enrolled and treatment effects reported in randomized controlled trials may not be representative of routine clinical practice, so different standards may have to be explored for guidance of intervention research.⁸

Maher et al¹⁰ and Moseley et al¹⁶ found increasing numbers of systematic reviews within the Physiotherapy Evidence Database. A similar increase in percentage of systematic reviews in JOSPT is noteworthy, as these reports provide a comprehensive synthesis of previously published research. Results from the current analysis suggests an increase in intervention-based literature syntheses in a specialty practice journal (JOSPT), as well as a general journal within physical therapy PTJ).⁴

JOSPT Bibliometric Results

The focus of bibliometrics is on assessing patterns and trends in published literature within a given field.²⁴ Citation analysis is probably the most recognized form of bibliometrics and has been conducted previously on rehabilitation literature.^2,6,19,21,26 The interest of citation analysis is in the frequency with which certain authors and publications are cited. The results of such analyses provide measures of impact (eg, impact factor).

In JOSPT, several institutions are the most impactful. Interestingly, the University of Pittsburgh was the most prolific institution in both JOSPT and PTJ. Determining whether the University of Pittsburgh has maintained a steady rate of publication or produced more within recent years is beyond the scope of this paper, as we do not have yearly data for each institution. Seven institutions (University of Pittsburgh, University of Southern California, Washington University, University of Iowa, University of Delaware, University of Florida, and University of North Carolina) were not only listed in our previous study⁴ but also by Richter et al¹⁸ in their study on publication productivity.

There have been previous bibliometric studies of core journals of physical therapy.^1,2,5,11,21 In TABLE 4, we present what may be considered core journals of orthopaedic and sports physical therapy, which include those within the fields of sports medicine, orthopaedic rehabilitation and surgery, manual therapy, athletic training, pain research, podiatry, physical medicine and rehabilitation, and biomechanics. The journal representation suggests that authors draw from a variety of sources, including a roughly equal proportion of orthopaedic and sport-focused journals. While JOSPT represents the field of orthopaedic and sports physical therapy broadly, observation of top JOSPT articles suggests a certain level of specificity. Based on citation counts of JOSPT articles, it appears this journal has demonstrated a strong expertise for biomechanics and issues related to the ankle, knee, and shoulder.

Selective Comparisons to Pediatric Physical Therapy

Svien et al²⁵ reported growth in the quality and quantity of research published in Pediatric Physical Therapy from 1989 to 2003. Similar to JOSPT, Pediatric Physical Therapy has experienced growth in the number of both experimental and nonexperimental research reports published over the last 15 years. While not statistically significant, the authors reported an increased proportion of experimental studies within Pediatric Physical Therapy. Additionally, increases in studies focused on intervention and metrics were also reported.

The publication trends observed in both pediatric and orthopaedic and sports specialty may be indicative of larger research focus within these specialty areas. Further, it's important to consider that journal focus may be shifting towards experimental and/or clinically relevant research. If so, these trends support the advancement of evidence-based physical therapy practice resources for specialty clinicians.

Selective Comparisons to PTJ

The use of the same methodology in completing these analyses allows for direct comparisons between JOSPT and PTJ. We observed similar trends in JOSPT and PTJ for research reports (increased percentage), topical or nonsystematic reviews (decreased percentage), articles including symptomatic participants (increased percentage), and articles focused on prognosis (increased percentage). Globally, these results support emphasis on evidenced-based practice in physical therapy, and the current analysis confirms that the orthopaedic and sports specialty fields reflect this emphasis. These similarities may be in part due to the fact that 43% of the papers reviewed for the bibliometric analysis of PTJ had a musculoskeletal emphasis.

Comparison of journal content within each of the 4 domains (type of article, type of participants, type of research design, purpose of article) also revealed similar patterns between both journals. No trend in the proportion of randomized controlled trials was observed in either journal. This finding conflicts with prior studies^10,16,17,19 examining publication content and may be due to a difference in analytical methodology (eg, examining total number of publications versus proportion within journal).

There were notable differences in journal content and trends between JOSPT and PTJ. First, participant characteristics were different between the 2 journals. Specifically, the most frequently published participant type for PTJ was symptomatic adults, while for JOSPT it was asymptomatic adults. The proportion of studies admitting symptomatic participants relative to all studies published over the same period in PTJ was only slightly higher than that of JOSPT (44% versus 38%). While similar in representation of symptomatic participants, asymptomatic participants were not higher in PTJ, possibly due to a larger proportion of studies in PTJ with animals, healthcare professionals, or no participants. As such, the overall percentage of asymptomatic participants in JOSPT was higher. We consider a key finding from both of these analyses is that less than 50% of published studies in these 2 key physical therapy journals included symptomatic participants, despite both journals having, as a primary mission, to serve a rehabilitation profession. A second difference was in publication trends for diagnostic, prospective, and metric-focused articles. PTJ showed increasing trends in these article categories, while JOSPT did not.

Interpretation of trends in publication can be difficult, as there is no set standard for the optimal mix of studies to be published in a given journal or field. For example, research in a specific practice area may warrant a higher proportion of investigations with asymptomatic participants. Another consideration is that, based on our coding system, most mechanistic research would be classified as being on asymptomatic subjects, even when investigating the physiology of common interventions (such as manual therapy). It is an accepted progression of research that these investigations often begin with a nonclinical population and transition into clinical populations. In addition, because our analysis did not explicitly rate study quality, increasing trends of any article type would not constitute a favorable trend if accompanied by lower quality studies.

Future Implications

There are several implications for orthopaedic and sports physical therapy research, based on the findings from this study. First, randomized controlled trials are considered the gold standard study design of the efficacy/effectiveness of interventions. Given the trend for lack of growth in these types of publications in JOSPT, there may be a need to encourage the submission and publication of high-quality randomized controlled trials. Additionally, approximately 10% of all JOSPT articles were related to diagnosis and prognosis. In our opinion, there is a need for an increase in these studies, along with those related to the metrics of diagnostic/prognostic tools (eg, reliability, validity, responsiveness) for orthopaedic and sport-related conditions. We observed similar proportions in PTJ, so the lack of these studies does not appear to be an issue specific to a specialty area.⁴ Although JOSPT did not exhibit increased trends in the proportion of treatment studies, one fifth of all articles were related to treatment, representing a consistent emphasis on intervention studies.

Few studies in JOSPT included participants greater than 65 years of age, and the limited number of citations from geriatric journals is an indication that orthopaedic and sport conditions related to the aging population have been underreported in JOSPT. Thus the publication of orthopaedic and sports-related research in geriatric populations may be an area in need of greater emphasis in JOSPT.

Examining publication trends within physical therapy specialty journals is one way to measure whether researchers are meeting the criteria listed in the APTA's Research Agenda.⁷ For example, one area of interest in the Research Agenda focuses on measurement development and validation. Future research should examine relationships between profession-driven initiatives, such as the Research Agenda, and trends in the types of clinical conditions studied and papers published in core physical therapy journals. Additionally, it is important to consider whether professional journals like JOSPT are meeting the criteria within a broader context. For example, Rose et al²⁷ examined the frequency with which articles published in the Journal of Hand Therapy focused on the International Classification of Function domains from the World Health Organization. Much of the emphasis in Journal of Hand Therapy has remained on body functions and structures, and less on activities, participation, and contextual factors. Examining whether JOSPT and other physical therapy journals demonstrate higher representation of patient-centered research is another potential area of future study.

Limitations

This study had limitations that should be considered when interpreting these results. First, it was not our intention to measure the quality of articles published in JOSPT. We presented data related only to content and changes in content over time. We are aware that trends in quantity of published material may not be directly related to a journal's quality and have, therefore, refrained from discussing implications related to research quality. We further acknowledge that there is no established standard for what constitutes an optimal proportion of study designs to be published in a peer-review journal. Therefore, our suggestions for what these trends imply are based solely on author opinion. Future research should examine changes in the quality of research published in JOSPT, as in prior studies in the physical therapy literature.^10,15,16 Second, while we used a comprehensive coding system, we were still limited in the information captured. For example, our coding system was not designed to distinguish between papers that might be unique to orthopaedics or sports areas. Due to the amount of content overlap in these areas, such distinctions would be difficult to make with most available coding systems. Additionally, we were unable to include citation data prior to 1992, as earlier articles were not listed within ISI Web of Science. Given this, there is potential for bias in the bibliometric results because early authors, institutions, and articles prior to 1992 were not included. Interpretation of bibliometric data should be made in light of this limitation.

Findings

In JOSPT there were significant increases in the proportion of research reports, systematic reviews, articles focused on prognosis, and articles including symptomatic participants, and concomitant decreases in the proportion of topical or nonsystematic reviews and articles focused on anatomy/physiology.

Implication

There is growth in higher level publication types in JOSPT. Authors should be encouraged to submit randomized controlled trials and papers focused on diagnosis, prognosis, and metrics.

Caution

Interpretation should be limited to changes in quantity of published article types. Quality assessment of published JOSPT articles was not performed.