conclusions

conclusions

Read also

7.3. Methods for assessing the scientific and technical effectiveness of research

The result of research is the achievement of scientific, scientific-technical, economic and social effects. The scientific effect is characterized by the acquisition of new scientific knowledge and reflects the growth of information intended for "intra-scientific" consumption. The scientific and technical effect characterizes the possibility of using the results of ongoing research in other R&D and provides the information necessary to create new products. The economic effect characterizes the commercial effect obtained by using the results of applied R&D. The social effect is manifested in the improvement of working conditions, the improvement of economic characteristics, the development of culture, health care, science, and education.

Scientific activity is multifaceted, its results, as a rule, can be used in many areas of the economy for a long time.

Evaluation of the scientific and scientific and technical effectiveness of R&D is carried out using a system of weighted scoring. For fundamental R&D, only the coefficient of scientific effectiveness is calculated (Table 7.3), and for exploration work, the coefficient of scientific and technical effectiveness is calculated (Table 7.4). Estimates of coefficients can only be established on the basis of the experience and knowledge of scientists who are used as experts. The evaluation of the scientific and technical effectiveness of applied R&D is carried out on the basis of a comparison of the technical parameters achieved as a result of the R&D with the basic ones (which could have been implemented before the R&D was completed).

Table 7.3

Characteristics of factors and signs of scientific effectiveness of R&D

Scientific performance factor	Coef. the significance of the factor	Factor quality	Factor characteristic	Coef. achieved level
The novelty of the results obtained			Fundamentally new results new theory, the discovery of a new regularity
			Some general patterns, methods, ways to create a fundamentally new products
		Insufficient	Positive decision based on simple generalizations, analysis of factor relationships, extension of known principles to new objects
		Trivial	Description of individual factors, dissemination of previous results, abstract reviews
Depth of scientific study			Performing complex theoretical calculations, testing on a large amount of experimental data
			Low complexity of calculations, verification on a small amount of experimental data
		Insufficient	Theoretical calculations are simple, the experiment was not carried out
Degree of probability of success
		Moderate

Table 7.4

Characteristics of factors and signs of scientific and technical effectiveness of R&D

Factor of scientific and technical effectiveness	Coef. the significance of the factor	Factor quality	Factor characteristic	Coef. achieved level
Prospects for using the results		Paramount	The results can be applied in many scientific areas.
			The results will be used in the development of new technical solutions
		Useful	The results will be used in subsequent research and development
Scale of implementation of results		National economy
			Implementation time: up to 3 years, up to 5 years, up to 10 years, over 10 years
		Individual firms and enterprises	Implementation time: up to 3 years, up to 5 years, up to 10 years, over 10 years
Completeness of results			Terms of Reference for R&D

		Insufficient	Overview, information

In this case, the coefficient of scientific and technical effectiveness is determined by the formula

where k is the number of estimated parameters;

coefficient of influence of the i-th parameter on scientific and technical performance;

– coefficient of relative increase of the i-th parameter in comparison with the base value.

For the convenience of performing calculations, the data are summarized in Table. 7.5.

Table 7.5

Assessment of the scientific and technical effectiveness of applied R&D

Parameter	Unit of measurement	Coef. influence	Parameter values
			achieved

THEORY AND PRACTICE OF MANAGEMENT

Dr. tech. Sciences A. V. Ryzhakova

cand. economy S. V. Manakhov

EVALUATION OF THE EFFICIENCY OF THE RESEARCH

WORKS IN UNIVERSITY1

The article considers the fundamental factors of efficiency (efficiency) of fundamental and applied research work carried out by modern university. A brief description of some methods for evaluating the results of research work is given, a method for calculating the integral indicator that reflects the quality of the results of the scientific activity of the university is proposed. A list of indicators is given, on the basis of which it is possible to evaluate the dynamic rating of the effectiveness of R&D.

Keywords and phrases: research work (R&D), fundamental and applied R&D, R&D performance, integral indicator, scientific and technical level indicator, dynamic assessment of R&D effectiveness.

Scientific work is one of the most important types activities of the teaching staff, graduate students and students of any higher educational institution. Holding scientific work at the university ensures continuous improvement of the educational process on the basis of fundamental and applied research in the areas of training specialists and implementation in educational activities modern methods and pedagogical technologies.

The university status of a higher educational institution implies:

Personnel training highest qualification in a wide range of areas of science, production and public life through postgraduate and doctoral studies;

Participation in the educational process of scientific and teaching staff higher qualification;

The presence of generally recognized scientific schools, significant achievements in the field of fundamental and applied research, scientific and methodological innovations;

Raising the scientific qualifications of scientific and pedagogical personnel;

Dissemination of scientific knowledge, cultural and educational activities.

Thus, one of the main tasks of universities is to organize and conduct fundamental and applied scientific research. The results of fundamental scientific research in this case are

1 The article was prepared as part of the research work "Development of a mechanism for project management of the scientific activities of the university" under the state assignment of the Ministry of Education and Science Russian Federation in 2013

theory, method, hypothesis, and applied scientific research - methodology, algorithm, technology, device, installation, device, mechanism, substance, material, product, system (management, regulation, control, design, information), software, database .

Availability of a well-established mechanism for the implementation of regular analysis and evaluation of the results of research work - essential element effective management of scientific activities of the university. At the same time, if the quantitative assessment of scientific research conducted at the university is easily feasible, then characterizing the quality of research work performed is a difficult task.

The quality and efficiency of research work, the composition of the indicators used and the accuracy of the calculation of the latter depend on the following factors:

The nature of the results obtained;

Liaison with practical bodies;

The scale of R&D implementation;

The duration of the period of research and development and the stage at which the calculation is made (planning, implementation, implementation).

There are features of determining the effectiveness of research work, depending on their type. According to fundamental research, in the course of a comprehensive analysis, the scientific effect is first of all established. If the results of R&D in the future may affect the social side of public life (which is important, for example, for the penitentiary system), then the content of a comprehensive analysis of R&D should include a qualitative assessment of these components of the overall effect.

In addition, the characterization of the effect of fundamental work carried out at the university should include the established impact of these works on the quality of education at the university. The economic effect and economic efficiency of R&D expenditures on fundamental research usually cannot be calculated. In some cases, only a qualitative forecast of the economic rationality of R&D, possible in the future, can be given.

Applied (exploratory) R&D primarily reveals scientific and technical effects. The possibility of analyzing the organizational feasibility of R&D depends on its type. For applied work related to material production, performed on the basis of the use of new phenomena and patterns discovered during the fundamental research, the economic effect of implementing their results and the economic efficiency of R&D costs, as a rule, cannot be calculated. Only the production and other possibilities of applying R&D are clarified, and on the basis of a qualitative analysis, a characteristic of their alleged economic significance is given. This determines the range of indicators that change under the influence of the implementation of the results of the work, as well as the possible range of these changes.

Thus, it should be noted that far from every research work, the analysis of rationality can be brought to the calculation of indicators of economic effect.

Currently, a fairly large number of different methods for assessing the effectiveness of research work are known, but there are no unified methods for assessing the results of research work, so the problem of their development is very relevant for modern universities.

The development and application of methods for assessing the economic efficiency of research and development work is one of the most important levers for accelerating scientific and technological progress, concentration of scientific potential on the most important directions development of the national economy and dispersion of material, financial and human resources.

The main indicator of the quality of the results of the scientific activity of the university, in our opinion, is the indicator of scientific and technical level (STL) as a generalizing characteristic of the prospects and progressiveness of scientific and technical solutions included in the development, in comparison with existing and previously created ones, which belong to one scientific industry. The definition of NTU is carried out by a complex method based on an integrated assessment as a function of private assessments (indicators) and weight coefficients characterizing the importance of each indicator.

The result of research is the achievement of a scientific and scientific-technical effect. Their quantitative values are determined using a system of weighted scoring for a group of scientific performance factors. Moreover, these factors differ for fundamental and applied R&D. The most general list of brief description and possible values of the coefficients are presented in Table. 1 and 2.

For fundamental R&D, only the coefficient of scientific effectiveness is calculated (in accordance with the initial data of Table 1), and for applied work, the coefficient of scientific and technical effectiveness is also calculated (in accordance with the initial data of Table 2).

The coefficient of scientific (scientific and technical) effectiveness is determined by the formula

E = £ ErKr,

where k is the number of estimated parameters;

E7 - coefficient of significance of the factor (influence of the 7th parameter on scientific (scientific and technical) performance);

K is the coefficient of the achieved level of the 7th parameter.

In accordance with the proposed methodology, the coefficients are estimated only on the basis of the experience and knowledge of scientists who are used as experts. The assessment of the scientific and technical effectiveness of applied R&D is carried out on the basis of a comparison of the technical parameters achieved as a result of the R&D with the basic ones (which could have been implemented before the R&D was completed).

Table 1

Characteristics of factors and signs of scientific effectiveness of fundamental R&D

Factor of scientific effectiveness Coefficient of significance of the factor Quality of the factor Characteristics of the factor Coefficient of the achieved level

Novelty of the obtained results 0.5 High Fundamentally new results, new theory, discovery of a new regularity 1.0

Medium Some general patterns, methods, ways to create a fundamentally new product 0.7

Insufficient Positive decision based on simple generalizations, analysis of factor relationships, extension of known principles to new objects 0.3

Trivial Description of individual factors, dissemination of previous results, abstract reviews 0.1

Depth of scientific study 0.35 High Performance of complex theoretical calculations, verification on a large amount of experimental data 1.0

Medium Low complexity of calculations, verification on a small amount of experimental data 0.6

Insufficient Theoretical calculations are simple, the experiment was not carried out 0.1

Degree 0.15 Large - 1.0

probability of success Moderate - 0.6

Small - 0.1

Table 2

Characteristics of factors and signs of scientific and technical performance

applied research

Factor Coefficient Quality Characteristic Coefficient

scientific and technical factor factor factor factor

the effectiveness of the significance of the factor of the achieved level

Promising 0.5 Highly Possible Results 1.0

use find application

results in many scientific fields

Important The results will be used in the development of new technical solutions 0.8

Helpful Results will be used in subsequent research and development 0.5

Implementation scale 0.3 National Implementation time:

economic results up to 3 years up to 5 years up to 10 years over 10 years 1.0 0.8 0.6 0.4

Industry Implementation period: up to 3 years up to 5 years up to 10 years over 10 years 0.8 0.7 0.5 0.3

Individual firms and Implementation time: up to 3 years 0.4

enterprises up to 5 years up to 10 years over 10 years 0.3 0.2 0.1

Completion 0.2 High Terms of Reference- 1

results for experimental design work

Insufficient Overview, information 0.4

Evaluation of the effectiveness of research should be carried out at different stages of the implementation of a research project.

The initial assessment of efficiency (planned efficiency) takes place at the stage of formation of the terms of reference (TOR). It is reflected in the explanatory note and the draft TOR, which necessarily describe the planned results and analyze their value in terms of the overall objectives of the university.

Evaluation of effectiveness at the time of delivery and defense of work (or stage) is reflected in the minutes of meetings of commissions for the acceptance of research results, in particular, in the form of recommendations for further use with an assessment of expected beneficial effects.

Evaluation of the effectiveness of research after its formal completion is of particular relevance and significance for modern Russian universities. Unfortunately, many R&D ends at the stage of a research report and payment for the work of its participants. The further scientific activity of the department or scientific unit in most universities of the country is not tracked in relation to those R&D that were performed at the department or with the participation of the department or scientific unit. This type estimates will be called the dynamic R&D rating.

Number of monographs on research topics;

Volume printed sheets in published monographs highlighting the results of a specific R&D;

Number of articles published on research topics;

The number of Ph.D. theses defended using the results of research, indicating the coefficient of use of research results for each dissertation (from 0 to 1);

The number of doctoral dissertations defended using the results of research, indicating the coefficient of use of research results for each dissertation (from 0 to 1);

The volume of work performed on R&D in financial terms (total cost of R&D);

Number of textbooks and teaching aids using the results of research;

Quantity academic disciplines using the results of research;

The number of patents registered based on the results of research;

Number of registered software products created on the basis of R&D results;

Number of new bids submitted based on R&D results;

Number of winning applications created based on results

total amount winning applications created based on the results

Dynamic evaluation of effectiveness may fall if there are no publications, patents, no development, no implementation, the results did not form the basis of dissertations, there are no customers for the results, etc.

Publication indicators;

Indicators of scientific qualification of scientific and pedagogical workers (candidate and doctoral dissertations);

The quality of the educational process;

Degrees of commercialization of research results;

The activity of the most qualified part of the scientific and pedagogical workers (the desire to enter and further improve the rating scientific supervisors) regarding the search for the most promising research topics;

The volume of research carried out, in particular, with the involvement of external sources of financing.

The use of a dynamic R&D rating by higher education institutions will make it possible to more fully ensure continuity between the stages of research work, in particular, between the stage of research development and the stage of commercialization of scientific knowledge.

Bibliography

1. Butenko Ya. A. Implementation of project management in the scientific activity of the university // Bulletin of the Russian University of Economics named after G. V. Plekhanov. - 2013. - No. 9 (63).

2. Manakhov S. V. The development of scientific activity in universities - a new priority of the state scientific and technical policy of Russia // Bulletin of the Plekhanov Russian University of Economics. - 2013. - No. 8 (62).

3. Maslennikov V.V. Organizational Models project management of scientific activities in Russian universities// Bulletin of the Plekhanov Russian University of Economics. - 2013. - No. 9 (63).

4. Saginova O. V., Zavyalova N. B., Shtykhno D. A. Communication of scientific research and educational process in universities // Bulletin of the Plekhanov Russian University of Economics. - 2012. - No. 12 (54).

The effectiveness of scientific research is established taking into account various kinds manifestations of a beneficial effect that can be obtained by using the results of research. These types of effects can be: a) social effect - is manifested in increasing safety for human life and health, labor safety, in improving working conditions, reducing the likelihood of occupational diseases, increasing environmental safety and in obtaining other socially significant results b) defensive effect - characterizes the significance of the research results for ensuring the country's defense capability. Performance indicators in this case can be: the probability of completing a combat mission, the degree of protection of objects from enemy damage, the level of preservation of state, industrial, military secrets, etc. d.; in) economical effect - characterizes the valuation of the effectiveness of scientific research, manifested in a decrease in the cost of products, works, services, profit growth; G) scientific and technological effect - characterized as the accumulation of new knowledge in the field of properties and phenomena of the material world, manifested in the form of scientific publications, reports, discoveries, defended dissertations, inventions.

Quantification usually lends itself to the economic impact expected from applied research and development. The basis of such an assessment is a comparison of costs (estimated cost of the topic) and results (expected reduction in the cost of production). When calculating other types of effect, the peer review method is often used. For example, when assessing the scientific and technical effect, it may be recommended to use the scientific and technical effect coefficient Ht:

where m is the number of signs of scientific and technical effect;

ri - scoring of the i-th attribute of the scientific and technical effect;

bi is the level of significance of the i-th feature of the scientific and technical effect. Tables 1 and 2 show typical signs of scientific and technical effect and approximate values of ri, bi.

Table 1 - Point values of the signs of the scientific and technical effect

Signs of scientific and technological effect		Point values ri
1. Expected level of novelty of R&D results	a) fundamentally new
	b) relatively new
	c) not new
2. Theoretical level	a) establishment of a new law, theory
	b) deep study of the problem
	c) development of a method, method, program
3. Possibility of practical use of the results	a) within 1 - 2 years
	b) within 3 - 5 years
	c) for 5 years or more
	d) indefinite

Table 2 - Significance of signs of scientific and technical effect

10. Organization and procedure for the implementation of research

10.3. Methods for assessing the scientific and technical effectiveness of research

Evaluation of the scientific and scientific and technical effectiveness of R&D is carried out using a system of weighted scoring. For fundamental R&D, only the coefficient of scientific effectiveness is calculated (Table 10.3), and for exploration work, the coefficient of scientific and technical effectiveness is calculated (Table 10.4). Estimates of coefficients can only be established on the basis of the experience and knowledge of scientists who are used as experts. The evaluation of the scientific and technical effectiveness of applied R&D is carried out on the basis of a comparison of the technical parameters achieved as a result of the R&D with the basic ones (which could have been implemented before the R&D was completed).

Table 10.3

Characteristics of factors and signs of scientific effectiveness of R&D

Factor of scientific effectiveness	Coef. factor significance	Factor quality	Factor characteristic	Coef. reached level
The novelty of the results obtained			Fundamentally new results, new theory, discovery of a new regularity
			Some general patterns, methods, ways to create a fundamentally new product
		Insufficient	Positive decision based on simple generalizations, analysis of factor relationships, extension of known principles to new objects
		Trivial	Description of individual factors, dissemination of previous results, abstract reviews
Depth of scientific study			Performing complex theoretical calculations, testing on a large amount of experimental data
			Low complexity of calculations, verification on a small amount of experimental data
		Insufficient	Theoretical calculations are simple, the experiment was not carried out
Degree of probability of success
		Moderate

Table 10.4

Characteristics of factors and signs of scientific and technical effectiveness of R&D

Factor of scientific and technical effectiveness	Coef. the significance of the factor	Factor quality	Factor characteristic	Coef. reached level
The prospect of using the results		paramount foam	The results can be applied in many scientific areas.
			The results will be used in the development of new technical solutions
		Useful	The results will be used in subsequent research and development
Scale of implementation of results		National economy	Implementation time: over 10 years
			Implementation time: over 10 years
		Individual firms and enterprises	Implementation time: over 10 years
Completeness of results			Terms of Reference for R&D

		Insufficient	Overview, information

In this case, the coefficient of scientific and technical effectiveness is determined by the formula

where k is the number of estimated parameters; - coefficient of influence of the i-th parameter on scientific and technical performance; - coefficient of relative increase of the i-th parameter in comparison with the base value.

For the convenience of performing calculations, the data are summarized in Table. 10.5.

Table 10.5

Assessment of the scientific and technical effectiveness of applied R&D

The result of research is the achievement of a scientific, scientific-technical, economic or social effect. The scientific effect characterizes the acquisition of new scientific knowledge and reflects the increase in information intended for internal scientific consumption. The scientific and technical effect characterizes the possibility of using the results of ongoing research in other research and development projects and provides the information necessary to create new technology. The economic effect is characterized by the value of living and materialized labor in social production, obtained by using the results of applied research. The social effect is manifested in the improvement of working conditions, the improvement of environmental characteristics, the development of health care, culture, science, education, and so on.

The assessment of the social effect of R&D should be based on a long-term forecast.

When performing technical and economic calculations of this section, a quantitative assessment can be made, or a qualitative analysis can be carried out. It is advisable to make a quantitative assessment of the scientific effect by calculating the scientific effectiveness, scientific and technical effect - scientific and technical effectiveness. A qualitative analysis of possible types of R&D effect consists in comparing the advantages and disadvantages of the results obtained based on the estimates of "higher-lower", "better-worse", "more-less", and so on.

Evaluation of scientific and scientific and technical performance is carried out using coefficients calculated according to the formulas:

Significance coefficient of the i-th factor used for evaluation;

The coefficient of the achieved level of the i -th factor;

n, m - number of factors.

The following factors can be taken as factors in assessing scientific performance: the novelty of the obtained or expected results; depth of scientific study; the degree of probability of success (in case of incomplete work). The following factors can be used as factors in assessing scientific and technical effectiveness: prospects for use; scale of implementation; completeness of the obtained results. For each of the factors, the numerical value of the significance coefficient is established by expert means. In this case, the sum of the significance coefficients for all factors should be equal to one. The coefficient of the achieved level of the factor is also established by expert means, and its numerical value is determined taking into account the quality of the factor attribute and its characteristics. In this case, the value of k j - i 1. The maximum possible value of kn.r. and kn.t.r is equal to one. The closer the value of kn.r. and kn.t.r to one, the higher the scientific and scientific and technical effectiveness of the ongoing research. Factors affecting quantification scientific and scientific and technical performance are summarized in tables 7.10 and 7.11.

Table 7.10

Scientific performance factor	Factor significance factor	Factor quality	Characteristic	Achievement ratio

The novelty of the results obtained			Some regularities have been established that allow creating fundamentally new types of equipment
Depth of scientific study			The complexity of theoretical calculations is low, the results are verified on a limited number of experiments
Degree of probability of success			Success is very possible, there is a high probability of a positive solution to the tasks

Table 7.11

Characteristics of factors and signs of scientific effectiveness of ongoing research

We obtain the coefficients of scientific performance factors:

The novelty of the expected results k zn1 =0.5, k du1 =0.7;

The depth of scientific study k zn2 =0.35, k dn2 = 0.6;

Degree of probability of success k zn3 =0.15, k du3 = 1.0;

As a result, the coefficient of scientific effectiveness:

k n.r. = 0.5 0.7 + 0.35 0.6 + 0.15 1.0 = 0.71;

Coefficients of scientific performance factors:

Prospects for using the results k ntr1 =0.5, k d.y1 =0.8

The scale of the possible implementation of the results k ntr2 = 0.3, k d.y2 = 0.8

Completion of the obtained results k ntr3 = 0.2, k d.y3 = 0.6

As a result, the coefficient of scientific and technical effectiveness:

k n.t.r = 0.5 0.8+0.3 0.8+0.2 0.6 = 0.76

Some social results achieved on the basis of R&D and R&D can be quantified in the appropriate units adopted for measurement. given result. These results include: the purity of the atmosphere in locality and industrial zone, air purity in the production room; characteristics of industrial waste; noise level, lighting and so on. The value of the social result in these cases is determined by the change in the quantitative characteristics achieved on the basis of research.

The social effect of this research and development work can be attributed to the absence of pollution environment, simplicity and ease of use. When implementing the results of research aimed at solving social tasks, along with the main social result, a concomitant economic effect is often achieved - an increase in labor productivity, saving labor resources.

As a result of the research, it can be concluded that the further introduction of the designed milling cutter into production will significantly reduce the energy consumption for milling coatings, we will get a road milling machine with higher productivity, low unit costs and capital investments. Increased productivity reduces milling toxicity, more efficient use working time, the level of environmental pollution is reduced. Based on the technical and economic indicators of the cutter, we can safely say that the developed equipment not only can, but should be introduced into public utilities.

unit of measurement

Parameter values