Proposals and Announcements

2015/10/30

Dear TES Global/Times Higher Education World University Ranking


Proposal for
Times Higher Education World University Ranking Indicators



This is a proposal to revise the indicators of the THE (Times Higher Education) World University Ranking, so they can reflect the actual situation of a more wide and diverse range of university activities.


It would be important to review the current indicators, particularly regarding the perspective of human resources development. The motivation behind producing research papers is not only limited to creating highly-cited papers that directly contribute to science and other disciplines, but it also aims to assist the development of human resources through personnel training of academic paper writing. Collaborative work with industries has clear economic benefits, but it also is important for human resources development under academic-industry collaboration. Because human resources development is an important aspect of university activities, we are afraid that too much emphasis on citations could lead to the underestimation of such actual university activities.


Especially, we propose that the list below should be applied to next year's Asia University Rankings.


Also, in regards to the announcement of annual university rankings, we would suggest not only publishing the ranking but also expressing comments regarding the previous year's ranking indicators, what the analysis was back then and what has changed since. This would be most helpful. Since such changes affect this year's ranking, it would be important to emphasize that this year's results should not be directly compared with those in the previous year.


1. Research and Citation Indicators

A) We propose the additional use of “citation per faculty numbers” with the use of FWCI (Field Weighted Citation Impact) for citation evaluations. A university has a responsibility to produce high quality papers, but sometimes a paper with low citation numbers can be regarded as significant to future generations. As a citation indicator, FWCI has been used for correction of averaged citation numbers, but we are afraid that this could be still a threat by emphasizing too much on citations. Placing too much emphasis on citations might lead scientists to believe that papers not able to get citations are papers not worth publishing. In regards to this, a better way to show a university's potential would be to calculate the total number of citations divided by faculty numbers (citation per faculty numbers) in addition to using FWCI.


B) In regards to the use of Kilo-author paper, we agree that it is necessary to rethink how to evaluate Kilo-author papers with more than 1000 authors. However, an undesirable effect would be the possibility of eliminating important papers that use large-scale collaborative equipment, and those in a scientific field traditionally involving several institutions and between different countries. We propose an alternative solution, where citations would be made to the author's registered university or research institute (citations divided by the number of authors or author institutions).


C) Whether a university is capable of producing a Top 10% paper also depends on the university's research capabilities. We suggest an indicator that looks at the number of Top 10% papers in regard to faculty number (Top 10% paper per faculty).


D) In general, it has been thought international research capability could be measured by citations in English-written journals. Usually, non-English journals have difficulty with adapting to this system, since papers in their native language only reach out to a limited number of readers. We recommend removing non-English journals for evaluation of not only citations but also research productivity, because appropriate correction between English and non-English journals is not applied yet.



2. Reputation Indicator

A) Current methods to measure reputation have tended to favor a small portion of major universities that cover a wide range of research fields. As an indicator, discriminant validity is low, and the reputation indicator should be considered less important.


B) In the coming Asia University Rankings, the reputation survey should be redone among Asian universities alone to enable a more accurate evaluation of top, middle, and low-level university reputations.


C) A correction is needed in order to measure the scale of research fields provided by an individual university and how active they are, since large scale universities have an advantage to draw popular attention from a wide range of research fields.



3. Industry/Innovation Indicator

A) Currently, industrial collaboration is measured by “industry income” only. Many universities today are connected with industries, not only for financial profit, but also for human resources development including staff exchange programs and research collaborations. We propose the additional use of “academic corporate collaboration” (number of academia-industry collaboration papers) to become an indicator along with industry income in regard to evaluation in terms of human resources development.


B) Other emerging points of interest include the number and quality of patents or royalties produced by universities, and number of product development cases, all of which may become more significant.



4. Introduction of a Diversity Indicator

A) We request an improvement in evaluating international staffs' and students' ratio. For example, if a university were made up of 100 per cent foreign staffs and students, the university would achieve 100 per cent international outlook value. Even if a university were able to achieve such a high number, it does not always mean there will be a good international environment. It is important to be able to maintain a good balance between domestic and international students in order to create a quality


international environment. For that reason, we propose the introduction of an “entropy factor” to count the number of international staffs and students. With this, it ensures that the highest marks are allocated to universities where the percentage of international to domestic staffs and students is half-half.


B) We also recommend the introduction of an indicator measuring a university's gender balance. An “entropy factor” can also be applied here to the indicator.


Research University Network of Japan


Task Force on Academic Ranking Index of World Universities




  • Okayama University (Chair)
  • Tokyo University of Agriculture and Technology
  • The University of Electro-Communications
  • Niigata University
  • Kanazawa University
  • Hiroshima University
  • Research Organization of Information and Systems
  • National Institutes of Natural Sciences (Facilitator)

Contact:
Amane Koizumi, Professor
National Institutes of Natural Sciences
+81-3-5425-1301, nins-ura@nins.jp