Author: Sholpan Kalanova, Professor, Doctor of Sciences
The ranking of higher education institutions (HEIs) in Kazakhstan in 2008 was conducted by the Independent Kazakhstan quality assurance agency. Experience of the leading international ranking agencies was used when determining the methodology of ranking. In ranking of the HEIs, quantitative as well as qualitative criteria were followed. The ranking of HEIs was based on two major survey questionnaires: survey №1 concerned the academic, technical, social and financial resources of the HEIs; survey №2 provided data from employers: governmental organisations, major national companies, non-governmental organisations, etc.. The results of the ranking were then published in the major mass media of the country, in order to inform future students at the time of admission to universities and society in general. The ranking of HEIs in Kazakhstan was conducted for the third time in history of Kazakhstan and for the first time by the independent agency.
One of the ways of advancing quality in higher education is the ranking of higher education institutions (HEIs). Publishing the results of ranking of HEIs in the mass media is important way to let the populace know of the quality of education of HEIs. It is vital to use trusted and effective methodologies for determining credible and scientifically justified ranking.
The significance of ranking of 2008 in Kazakhstan is based on the fact that for the first time the ranking of higher education institutions of the Republic of Kazakhstan (RK) is conducted by the initiative of non-governmental organization "Association of higher education institutions of Kazakhstan". The conduction of ranking was entrusted to the Independent Kazakhstan Quality Assurance Agency in Education (IQAA). More than sixty out of one hundred twenty six higher education institutions participated in the ranking of 2008.
IQAA bases its activity in the accordance with the Berlin Principles, ESG standards in Higher Education, which state that the evaluation of activity of HEIs must be conducted by organizations independent of the Ministry of Education, HEIs and other related entities.
The activity of the agency is based on the principles of social partnership with HEIs, Association of HEIs of Kazakhstan, associations of employers, international organisations, and the consumers of educational services. The work of the agency is directed to the perfection of mechanisms and instruments which improve the quality of higher education, and to the assurance of systematization of process of receiving and evaluating objective data of activity of HEIs.
One of the aims of IQAA is based on the improving competitive ability of HEIs of Kazakhstan on national and international levels, by the means of ranking of HEIs, conducting accreditation, offering consultations to HEIs, etc.
At the current moment, there is the established international experience in the formulation of approach and requirements, demanded from the rankings of HEIs. The methodology of IQAA is developed with the consideration of the latest achievements in the sphere of quality assurance and rankings. IQAA has followed in its activity the Berlin Principles on ranking of higher education institutions, based on 16 standards of good practice.
The aims of the ranking of HEIs of the Kazakhstan are the following:
- to assist decision-making by future students and their parents, to provide relevant information
- to government organizations, funding organisations, employers and international organisations;
- to promote competition between HEIs;
- to stimulate the creation and development of centers of quality assurance within HEIs.
IQAA had carried out significant work of collecting information, checking the correctness of information received, and processing the material when conducting the ranking of the HEIs of Kazakhstan. Not only quantitative-qualitative criteria were used, characterizing the quality of education of HEI, but also sociological studies of various groups of employers. The criteria by which the ranking was conducted consisted of two independent categories:
- the first category provides quantitative data and consists of seven criteria and forty-two indicators (Table №1) describing the qualitative composition of students and faculty, the financial and scientific research potential, the admission criteria for students and the results of the study process (the student grades and the results of national tests for students), the social-cultural conditions of students, and the level of international cooperation. The maximum number of points awarded in this category is 600.
- the second category provides external qualitative data by assessing the level of preparation of graduates from employers' perspectives (governmental organizations, leading national companies, NGO's, etc.) The maximum number of points awarded in this category is 300.
For survey №1, to calculate the points there have to be presented objective information on fifty three positions by HEIs of Kazakhstan, on the base of which 7 criteria and 42 indicators are formed. More detailed information about survey №1 is presented in Table №1. Survey №1 consists of the following criteria:
Criteria №1 - qualitative composition of contingent of students (80 points awarded)
Criteria №2 - qualitative composition of faculty (100 points awarded)
Criteria №3 - scientific-research potential of HEI (120 points)
Criteria №4 - financing, education expenses (70 points) and the availability of a high quality web-site, portal (30 points)
Criteria №5 - selectiveness of the admission of students and the results of study chievements of students (such as the student grades and the results the national tests held for students)
Criteria №6 - social-cultural conditions of students (45 points)
Criteria №7 - the level of international cooperation (55 points).
In 2008, in relation to the previous rankings of HEIs in Kazakhstan, the raising of points for survey №1 was made, based on the higher attention to the academic life of HEI and its activity. Apart from that, attention to international cooperation was raised. The aim is not only to be able to evaluate the activity of HEIs in Kazakhstan, but also to make their evaluation compatible on the international level.
Criteria and indexes used for ranking determination according to survey №1 (maximal score is 600)
|
Criteria |
№ |
Indexes |
Weighting coefficient in scores |
|
Qualitative composition of students: (0-80 scores) |
1.1 |
The proportion of full-time students to the total number of students: Number of full-time students / (Number of full-time students + Number of part-time students) x 100% |
10 |
|
1.2 |
The proportion of students awarded the State Educational Grant to the total number of full-time students: Students awarded the State Educational Grant / (Students awarded the State Educational Grant + number of full-time students) x 100% |
30 |
|
|
1.3 |
The proportion of foreign students to the number of full-time students: Number of foreign students + number of full students) |
15 |
|
|
1.4 |
The proportion of graduate and post-graduate students to the total number of full-time students: Number of graduate students + Number of post-graduate students / (Number of graduate students + Number of post-graduate students + Number of baccalaureate students) x 100% |
15 |
|
|
1.5 |
HEI ranging index, defined by the formula: I = N / N average, where I - is HEI ranging index, N - is the number of students of one HEI; N average - average number of students for one HEI in Kazakhstan, N average = Sum of all students in Kazakhstan / number of HEIs in Kazakhstan: I = N / N average |
10 |
|
|
Qualitative composition of faculty (0-100 scores)
|
2.1 |
The proportion of Doctors of Science and Candidate Doctors of Sciences to the total number of faculty: ((Doctors of Sciences + Candidate Doctors of Sciences) / Total number of faculty) x 100% |
30 |
|
|
2.2 |
The proportion of faculty, awarded the State Grant "The best faculty of the HEI in Kazakhstan" to the total number of faculty: (The number of faculty awarded the State Grant "The best faculty of the HEI in Kazakhstan" / Total number of faculty) x 100% |
30 |
|
|
2.3 |
The proportion of faculty awarded other State Awards to the total number of faculty (The number of faculty awarded other State Awards / Total number of faculty) x 100% |
20 |
|
|
2.4 |
The proportion of Doctors of Sciences and Candidate Doctors of Sciences to the total number of faculty, who work part-time in different branches of work, beside their work at the HEI |
5 |
|
|
2.5 |
The proportion of Doctors of Sciences and Candidate Doctors of Sciences to the total number of faculty, who work part-time in different branches of work, beside their work at the HEI |
15 |
|
Research potential (0-120 scores) |
3.1 |
Funding of research work per one faculty, in one thousand Tenges (Tenge - national currency in Kazakhstan): Funding of research work / per one faculty |
15 |
|
|
3.2 |
The number of scientific publications per one faculty in Kazakhstan during the last two years: Scientific publications in Kazakhstan / Total number of faculty) x 100% |
10 |
|
|
3.3 |
The number of international publications per one faculty during the last two years: International publications / Total number of faculty |
15 |
|
|
3.4 |
The number of dissertation councils |
20 |
|
|
3.5 |
The number of scientific specialties |
5 |
|
|
3.6 |
The number of published scientific manuals and monographs per one faculty during the last two years, which were quoted at the Kazakh State Institute of Scientific-Technical Research: Scientific manuals and monographs / Per one faculty |
15 |
|
|
3.7 |
Number of students who presented scientific publications, on the HEI level, national level, or international level |
15 |
|
|
3.8 |
The number of scientific international programs in which the HEI participates |
15 |
|
|
3.9 |
The number of the scientific national and regional programs in which the HEI participates |
15 |
|
Financing, study process expenses (0-100 scores) |
4.1 |
The area of scientific laboratories per one student, in square meters: Scientific laboratories total area, in square meters / Per one student |
15 |
|
|
4.2 |
Number of computers to the total number of students: Number of computers / Number of students |
10 |
|
|
4.3 |
The availability of educational web-site in three languages (Kazakh, Russian, English) |
30 |
|
|
4.4 |
Library resources measured by the number of text books, manuals, published since 1998 for the number of students: Number of books, manuals / Number of students |
15 |
|
|
4.5 |
The number and the title of Kazakhstan journals which were subscribed in 2007 per each ten faculty (Number of Kazakhstan journals / Per each ten faculty) |
5 |
|
|
4.6 |
The number and the title of foreign journals, including CIS journals (CIS - Commonwealth of Independent States), which were subscribed in 2007 per each ten faculty: Number of foreign journals / Per each ten faculty |
10 |
|
|
4.7 |
The proportion of expenses of study process per one student (excluding participations in State competitions): Expenses of study process / Per one student |
15 |
|
Student admissions criteria and the results of study process (0-100 scores) |
5.1 |
Average score of National Complex Test results for future students at the enrolment process for 2004-2005 academic year |
5 |
|
|
5.2 |
Average score of United National Test results for future students at the enrolment process for 2004-2005 academic year |
5 |
|
|
5.3 |
Average score of Unified National Test results for future students at the enrolment process for 2006-2007 academic year |
10 |
|
|
5.4 |
Average score of Intermediate State Control Test in 2006-2007 academic year (takes place after students complete second year of studying) |
40 |
|
|
5.4.1 |
Average score of Intermediate State Control Test of students in 2005-2006 academic year |
20 |
|
|
5.4.2 |
Consideration of ranging of the number of students who passed Intermediate State Control Test, by the formula: I = N / N average, where I - index of consideration of ranging of students who passed Intermediate State Control Test, N - number of students of one HEI, who passed Intermediate State Control Test, N average - average score of students, who passed Intermediate State Control Test, for each HEI in Kazakhstan (Intermediate State Control Test is a test for students who finish second year of studying) |
20 |
|
|
5.5 |
The proportion of students who passed the Intermediate State Control Test in 2005-2006 academic year |
10 |
|
|
5.6 |
The consideration of the improved knowledge of students for two years, based on the difference of the results of United National Test/National Complex Test and Intermediate State Control Test, which takes place two years later |
20 |
|
|
5.7 |
Correlation of student number to the amount of bachelor programs |
5 |
|
|
5.8 |
Correlation of student number to the amount of graduate programs |
5 |
|
Social-cultural conditions of students: (0-45 scores) |
6.1 |
Total area of dormitory per one non-resident student, in square meters: Total area of dormitory, in square meters / Per one non-resident student |
15 |
|
|
6.2 |
Opportunity for student cultural development: the number of student organisations, choruses, theatres, sport teams, etc, at the HEI (including the participation in intellectual competitions on national and international levels) |
10 |
|
|
6.3 |
The availability of medical centers, sport halls, recreation zones, etc, for students |
5 |
|
|
6.4 |
The proportion of students who took part in the work of social development of regions in Kazakhstan: "Green Earth" Student Building Brigades and other arrangements for 2005-2006 and 2006-2007 academic years in the percent correlations |
10 |
|
The level of international cooperation (0-55 scores) |
7.1 |
The cooperation with foreign HEIs: the number of the international exchanges for one full-time student (bachelor, graduate and post-graduate) for 2005-2006 and 2006-2007 academic years |
15 |
|
|
7.2 |
The number of the international exchanges, business trips per one faculty for 2005-2006 and 2006-2007 academic years |
15 |
|
|
7.3 |
The number of joint educational programs with foreign HEIs |
15 |
|
|
7.4 |
Membership in international associations of HEIs |
10 |
In June of 2008, survey №2 was sent to all Ministries, major governing bodies of the Republic of Kazakhstan and national companies (Parliament of RK, Supreme Court of RK, various Ministries, members of higher education system such as presidents and vice presidents of HEIs, regional bodies, national companies as well as average-size and small-size business companies, etc), for employers evaluation. The distribution of surveys for the regions was conducted proportionally to the population of each region, only for the major cities of Astana and Almaty, considering the number of central administrative bodies placed there in, the coefficient of 2 was used for them.
Surveys were conducted by different ways: by sending surveys by mail, by personally visiting employers, by phone and by e-mail. More than 3,500 surveys were collected from different groups of employers.
Survey №2 presents the list of all 126 civilian HEIs of Kazakhstan, in which the points from 0 to 10 were assigned. Very high level of preparation is awarded ten points, the high level of preparation - seven points, adequate level of preparation - four points, inadequate level of preparation - zero points. Each of the employers could have marked any number of HEIs. The results for each HEI were calculated after summing of all the questionnaires of survey №2 collected from employers.
The maximum amount of points in survey №2 is 300 points. It is appointed to the university which receives the highest results, other HEIs receive points in accordance to the coefficient, the relation of which is determined between significance of which was determined the result of the HEI and the maximum result of the first-ranked HEI. If HEI A collected the maximum amount of scores of 10,180 it was awarded 300 points. If HEI B received in survey №2 2700 scores, than it would get (2700/10180)*300=79,57 points. The calculation of all surveys was conducted with computers with optical scanning of survey results.
Results of two surveys were summed and the final results were determined, in accordance with which, ranking positions were determined for each group of HEIs. If the results of final results of two HEIs were within 3% range, than HEIs were awarded the same place in General ranking by directions of preparation of specialists. The results of ranking were published in the leading mass media of Kazakhstan, in newspapers such as "Express K", "Liter", "Panorama" and others, on the web-site of the agency www.iqaa.kz, and had a high resonance and interest of society.
We would like to quote some positions of "Berlin principles of ranking of higher education institutions (Berlin, May 2006), formed by the International Ranking Expert Group:
- rankings must be one of the different ways of assessment of higher education when entering the HEI, control of internal current processes, and also evaluation of higher education when leaving the HEI.
- Rankings can affirm relative information and better understanding of higher education, but they mustn't be the main methods of evaluation.
It is important to note that although rankings are powerful and important instruments of improving quality of HEIs, which present comparative information, they should also be combined with other ways of assessment of HEIs, such as accreditation, to ensure the full and better ways of quality assurance of higher education.