Skip to main content
SpringerLink
Log in

A multiple criteria information system for pedagogical evaluation and professional development of teachers

  • General Paper
  • Published:
Journal of the Operational Research Society

Abstract

Information systems for the evaluation of pedagogical practices aim to improve teaching activity and encourage the pedagogical development of teachers. The implementation of these systems has been interpreted in different ways, depending on each school’s administration board and/or the person responsible for the evaluation process. This ambiguity (and inherent subjectivity) is a barrier to the principles of transparency, equity and fairness in the pedagogical performance evaluation of teachers. From this premise, the main objective of this paper consists in proposing a multiple criteria information system to support the evaluation of pedagogical practices. By combining cognitive mapping and the measuring attractiveness by a categorical based evaluation technique (MACBETH), our framework allows for the conception of a more transparent evaluation system, bringing new insights to the pedagogical appraisal and professional development of teachers. The strengths and weaknesses of our framework are also discussed.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Figure 1
Figure 2
Figure 3
Figure 4
Figure 5
Figure 6
Figure 7

Similar content being viewed by others

References

  • Altet M (2000). Análise das Práticas dos Professores e das Situações Pedagógicas. Porto Editora: Porto.

    Google Scholar 

  • Amado C, Santos S and Marques P (2012). Integrating the data envelopment analysis and the balanced scorecard approaches for enhanced performance assessment. Omega: The International Journal of Management Sciences 40 (3): 390–403.

    Article  Google Scholar 

  • Ananda J and Herath G (2009). A critical review of multi-criteria decision making methods with special reference to forest management and planning. Ecological Economics 68 (10): 2535–2548.

    Article  Google Scholar 

  • Avalos B and Assael J (2006). Moving from resistance to agreement: The case of the chilean teacher performance evaluation. International Journal of Educational Research 45 (4/5): 254–266.

    Article  Google Scholar 

  • Bana e Costa C and Oliveira M (2012). A multicriteria decision analysis model for faculty evaluation. Omega: The International Journal of Management Sciences 40 (4): 424–436.

    Article  Google Scholar 

  • Bana e Costa C, Corrêa E, Corte J and Vansnick J (2002). Facilitating bid evaluation in public call for tenders: A socio-technical approach. Omega: The International Journal of Management Sciences 30 (3): 227–242.

    Article  Google Scholar 

  • Bana e Costa C, Corte J and Vansnick J (2012). MACBETH. International Journal of Information Technology & Decision Making 11 (2): 359–387.

    Article  Google Scholar 

  • Bana e Costa C, De Corte J and Vansnick J (2005). On the mathematical foundations of MACBETH. In: Figueira J, Greco S and Ehrgott M (eds). Multiple Criteria Decision Analysis: The State of the Art Surveys. Springer: New York, pp 409–442.

    Chapter  Google Scholar 

  • Bana e Costa C, Ensslin L, Corrêa E and Vansnick J (1999). Decision support systems in action: integrated application in a multicriteria decision aid process. European Journal of Operational Research 113 (2): 315–335.

    Article  Google Scholar 

  • Bana e Costa C, Fernandes T and Correia P (2006). Prioritisation of public investments in social infrastructures using multicriteria value analysis and decision conferencing: A case study. International Transactions in Operational Research 36 (3): 279–297.

    Article  Google Scholar 

  • Bana e Costa C, Oliveira C and Vieira V (2008). Prioritization of bridges and tunnels in earthquake risk mitigation using multicriteria decision analysis: Application to Lisbon. Omega: The International Journal of Management Sciences 36 (3): 442–450.

    Article  Google Scholar 

  • Bana e Costa C, Silva F and Vansnick J (2001). Conflict dissolution in the public sector: A case-study. European Journal of Operational Research 130 (2): 388–401.

    Article  Google Scholar 

  • Belton V and Hodgkin J (1999). Facilitators, decision makers, DIY users: Is intelligent multicriteria decision support for all feasible or desirable? European Journal of Operational Research 113 (2): 247–260.

    Article  Google Scholar 

  • Belton V and Stewart T (2002). Multiple Criteria Decision Analysis: An Integrated Approach. Kluwer Academic Publishers: Dordrecht, MA.

    Book  Google Scholar 

  • Desselle S, Mattei T and Vanderveen R (2004). Identifying and weighting teaching and scholarship activities among faculty members. American Journal of Pharmaceutical Education 68 (4): 1–11.

    Article  Google Scholar 

  • Dyson R (2000). Strategy, performance and operational research. Journal of the Operational Research Society 51 (1): 5–11.

    Article  Google Scholar 

  • Eden C (2004). Analyzing cognitive maps to help structure issues or problems. European Journal of Operational Research 159 (3): 673–686.

    Article  Google Scholar 

  • Eden C and Ackermann F (2001a). Group decision and negotiation in strategy making. Group Decision and Negotiation 10 (2): 119–140.

    Article  Google Scholar 

  • Eden C and Ackermann F (2001b). SODA—The principles. In: Rosenhead J and Mingers J (eds). Rational Analysis for a Problematic World Revisited: Problem Structuring Methods for Complexity, Uncertainty and Conflict. John Wiley & Sons: Chichester, pp 21–41.

    Google Scholar 

  • Eden C and Ackermann F (2004). Cognitive mapping expert views for policy analysis in the public sector. European Journal of Operational Research 152 (3): 615–630.

    Article  Google Scholar 

  • Elmore H (2008). Toward objectivity in faculty evaluation. Academe 94 (3): 38–40.

    Google Scholar 

  • Ferreira F (2013). Measuring trade-offs among criteria in a balanced scorecard framework: Possible contributions from the multiple criteria decision analysis research field. Journal of Business Economics and Management 14 (3): 433–447.

    Article  Google Scholar 

  • Ferreira F, Santos S and Rodrigues P (2011). Adding value to branch performance evaluation using cognitive maps and MCDA: A case study. Journal of the Operational Research Society 62 (7): 1320–1333.

    Article  Google Scholar 

  • Ferreira F, Santos S, Marques C and Ferreira J (2014a). Assessing credit risk of mortgage lending using MACBETH: A methodological framework. Management Decision 52 (2): 182–206.

    Article  Google Scholar 

  • Ferreira F, Santos S, Rodrigues P and Spahr R (2014b). Evaluating retail banking service quality and convenience with MCDA techniques: A case study at the bank branch level. Journal of Business Economics and Management 15 (1): 1–21.

    Article  Google Scholar 

  • Ferreira F, Santos S, Rodrigues P and Spahr R (2014c). How to create indices for bank branch financial performance measurement using MCDA techniques: An illustrative example. Journal of Business Economics and Management 15 (4): 708–728.

    Article  Google Scholar 

  • Ferreira F, Spahr R, Santos S and Rodrigues P (2012). A multiple criteria framework to evaluate bank branch potential attractiveness. International Journal of Strategic Property Management 16 (3): 254–276.

    Article  Google Scholar 

  • Gul R and Boman J (2006). Concept mapping: A strategy for teaching and evaluation in nursing education. Nurse Education in Practice 6 (4): 199–206.

    Article  Google Scholar 

  • Huber M (2002). Faculty evaluation and the development of academic careers. New Directions for Institutional Research 2002 (114): 73–83.

    Article  Google Scholar 

  • Inácio M and Salema M (2011). The competence of learning to learn in vocational training: Promoting reflective thinking of the trainer of trainers. Procedia: Social and Behavioral Sciences 29 (1): 611–619.

    Google Scholar 

  • Junior H (2008). Multicriteria approach to data envelopment analysis. Pesquisa Operacional 28 (2): 231–242.

    Google Scholar 

  • Keeney R (1992). Value-Focused Thinking: A Path to Creative Decisionmaking. University Press Harvard: Harvard, MA.

    Google Scholar 

  • Keeney R, See K and von Winterfeldt D (2006). Evaluating academic programs: With applications to U.S. graduate decision science programs. Operations Research 54 (5): 813–828.

    Article  Google Scholar 

  • Kolb DA (1984). Experiential Learning. Prentice-Hall: Englewood Cliffs, NJ.

    Google Scholar 

  • Liu S and Teddlie C (2003). The ongoing development of teacher evaluation and curriculum reform in the People’s Republic of China. Journal of Personnel Evaluation in Education 17 (3): 243–261.

    Article  Google Scholar 

  • Maskit D (2011). Teachers’ attitudes toward pedagogical changes during various stages of professional development. Teaching and Teacher Education 27 (5): 851–860.

    Article  Google Scholar 

  • Mills M and Hyle A (1999). Faculty evaluation: A prickly pair. Higher Education 38 (3): 351–371.

    Article  Google Scholar 

  • Murias P, Miguel J and Rodríguez D (2008). A composite indicator for university quality assessment: The case of Spanish higher education system. Social Indicators Research 89 (1): 129–146.

    Article  Google Scholar 

  • Politis Y and Siskos Y (2004). Multicriteria methodology for the evaluation of a Greek engineering department. European Journal of Operational Research 156 (1): 223–240.

    Article  Google Scholar 

  • Ramli N, Mohamad D and Sulaiman N (2010). Evaluation of teaching performance with outliers data using fuzzy approach. Procedia—Social and Behavioral Sciences 8 (1): 190–197.

    Article  Google Scholar 

  • Ramos G and Moraes R (2000). Avaliação do desempenho de professores numa perspectiva qualitativa: Contribuições para o desenvolvimento profissional de professores universitários. Revista Iberoamericana de Educación 108: 1–18.

    Google Scholar 

  • Richter D, Kunter M, Klusmann U, Lüdtke O and Baumert J (2011). Professional development across the teaching career: Teachers’ uptake of formal and informal learning opportunities. Teaching and Teacher Education 27 (1): 116–126.

    Article  Google Scholar 

  • Sahlberg P (2011). Developing effective teachers and school leaders: The case of Finland. In: Darling-Hammond L and Rothman R (eds). Teacher and Leader Effectiveness in High-Performing Education Systems. Alliance for Excellent Education and Stanford Center for Opportunity Policy in Education, pp 13–22.

    Google Scholar 

  • Santiago P, Roseveare D, Amelsvoort G, Manzi J and Matthews P (2009). Teacher evaluation in Portugal - OECD Review. OECD Review: Paris.

  • Santos S, Belton V and Howick S (2002). Adding value to performance measurement by using systems dynamics and multicriteria analysis. International Journal of Operations & Production Management 22 (11): 1246–1272.

    Article  Google Scholar 

  • Santos S, Belton V and Howick S (2008). Enhanced performance measuring using OR: A case study. Journal of the Operational Research Society 59 (6): 762–775.

    Article  Google Scholar 

  • Simões G (2000). A Avaliação do Desempenho Docente: Contributos para uma Análise Crítica. Texto Editora: Lisbon.

    Google Scholar 

  • Smith P and Goddard M (2002). Performance management and operational research: A marriage made in heaven? Journal of the Operational Research Society 53 (3): 247–255.

    Article  Google Scholar 

  • Sparks D and Loucks-Horsley S (1989). Five models of staff development for teachers. Journal of Staff Development 10 (4): 40–57.

    Google Scholar 

  • Spooren P (2010). On the credibility of the judge: A cross-classified multilevel analysis on students’ evaluation of teaching. Studies in Educational Evaluation 36 (4): 121–131.

    Article  Google Scholar 

  • Sproule R (2002). The underdetermination of instructor performance by data from the student evaluation of teaching. Economics of Education Review 21 (3): 287–294.

    Article  Google Scholar 

  • Stoklasa J, Talaŝová J and Holeček P (2011). Academic staff performance evaluation: Variants of models. Acta Polytechnica Hungaric 8 (3): 91–111.

    Google Scholar 

  • Tegarden D and Sheetz S (2003). Group cognitive mapping: A methodology and system for capturing and evaluating managerial and organizational cognition. Omega: The International Journal of Management Sciences 31 (2): 113–125.

    Article  Google Scholar 

  • Timoštšuk I and Ugaste A (2010). Student teachers’ professional identity. Teaching and Teacher Education 26 (8): 1563–1570.

    Article  Google Scholar 

  • Uzoka F (2008). A fuzzy-enhanced multicriteria decision analysis model for evaluating university academics’ research output. Information Knowledge Systems Management 7 (3): 273–299.

    Google Scholar 

  • Weber M and Borcherding K (1993). Behavioral influences on weight judgments in multiattribute decision making. European Journal of Operational Research 67 (1): 1–12.

    Article  Google Scholar 

  • Wolansky W (2001). A multiple approach to faculty evaluation. Education 97 (1): 81–96.

    Google Scholar 

  • Woods N (2011). Evaluation matters: Lessons learned on the evaluation of surgical teaching. The Surgeon 9 (1): S43–S44.

    Article  Google Scholar 

  • Zavadskas E and Turskis Z (2011). Multiple criteria decision making (MCDM) methods in economics: An overview. Technological and Economic Development of Economy 17 (2): 397–427.

    Article  Google Scholar 

  • Zopounidis C and Doumpos M (2002). Multicriteria classification and sorting methods: A literature review. European Journal of Operational Research 138 (2): 229–248.

    Article  Google Scholar 

Download references

Acknowledgements

A previous and unpublished version of this paper was presented at 8th International Forum on Knowledge Assets and Dynamics, held in Zagreb, Croatia. The authors wish to thank the superb contribution and infinite willingness of the panel members: Anabela Gonzaga, Carla Jesus, Celina Justino, Liliana Gens, Vanda Martins and Vera Vieira. The authors gratefully acknowledge use of the services and facilities of the School of Management and Technology of Santarém, Portugal. The third author also acknowledges financial support from Fundação para a Ciência e a Tecnologia and FEDER/COMPETE (grant PEst-C/EGE/UI4007/2013).

Author information

Authors and Affiliations

  1. Polytechnic Institute of Santarém, Santarém, Portugal

    Marlene N M Filipe

  2. University Institute of Lisbon (ISCTE-IUL), Lisbon, Portugal

    Fernando A F Ferreira

  3. University of Memphis, Memphis, TN, USA

    Fernando A F Ferreira

  4. University of Algarve, Faro, Portugal

    Sérgio P Santos

  5. Center for Advanced Studies in Management and Economics of the University of Évora, Évora, Portugal

    Sérgio P Santos

Authors
  1. Marlene N M Filipe
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Fernando A F Ferreira
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Sérgio P Santos
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Fernando A F Ferreira.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Filipe, M., Ferreira, F. & Santos, S. A multiple criteria information system for pedagogical evaluation and professional development of teachers. J Oper Res Soc 66, 1769–1782 (2015). https://doi.org/10.1057/jors.2014.129

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1057/jors.2014.129

Keywords

Search

Navigation