1. Introduction

To maintain secure and economic operations in power systems in China, some advanced facilities and technologies for power systems control, such as energy management (EMS) and distribution management systems (DMS), have been introduced to business operations since 1995. Generation scheduling and network analysis, such as transient and voltage-stability analysis, are the basic functions of EMS (Matsuzawa et al, 1990). DMS is in charge of monitoring network security and managing voltage and reactive power for 35 kV (and above) sub-transmission networks or 10 kV radial distribution networks (Liu et al, 2000). On average, EMS or DMS can be operated less than 10 years in China. Accordingly, most of the existing systems have reached expiration, and the power companies have a heavy task to either update or rebuild the supervisory control and data acquisition (SCADA) systems. In addition, during tap change, tap controls, under load transformer and capacitor switching, network reconfiguration and maintenance scheduling are the major areas of concern for power networks. Therefore, some special functions for area network dispatching centres have been developed in recent years, and some off-line management functions, such as automated mapping (AM), facilities management (FM), and geographic information systems (GIS) have been considered as well. AM/FM/GIS have been integrated with customer information systems, load management systems, and automatic dispatching system, and formed an integrated computer management information system called the feeder management system (FMS). It is forecasted that the integration of DMS and FMS in China will become the main trend in the coming years. However, no literatures have presented such an important topic before. Although IT is an important cornerstone for a firm to take the leading position in the market, at least 40% of IT projects realize no benefits (Willcocks and Lester, 1994) and there is no standard method to evaluate performance of IT (Fitzgerald, 1998). Even by adopting a particular IT technology through duplication by other firms, it often does not provide a sustainable competitive advantage for the duplicating firms. Accordingly, the primary objectives of this paper are to provide some new perspectives in explaining how IT technology can create a sustainable competitive advantage and how the methodological model can effectively evaluate the performance of IT projects.

In recent years, intensive cooperation among firms has increased, especially in knowledge-intensive industries. In such an environment, knowledge has become a critical factor for a firm to survive (Ozman, 2006). However, because the skills of management leading to innovation, efficiency, and effectiveness are different and contradictory (Hoegl and Wagner, 2005), firms still struggle to find efficient and effective management processes associated with innovation for project operations (Salomo et al, 2007). In order to retain a sustainable competitive advantage for firms adopting IT technology, this paper proposes a new conceptualization of IT capabilities as a higher-order construct, consisting of four dimensions: innovation, knowledge exchange, coordination, and activity integration. Operating IT projects both innovatively (innovation and knowledge exchange) and effectively (coordination and activity integration) require different forms of knowledge management (KM) and suitable management processes associated with KM methods. Based on the knowledge creation mode and maturity of the project, appropriate project development management is adopted at each phase to transform the value of the project into sustained competitive advantages. In addition, four aspects of a balanced scorecard (BSC) cannot simply fit overall characteristics of IT projects (Willcocks and Lester, 1994) because critical factors of IT projects are variant and dynamic (Asrilhant et al, 2006). In order to precisely display different facets of IT project performance, a modified BSC requires further development. Accordingly, this paper also identifies the critical success criteria of the FMS, and constructs an evaluation model, instead of a conventional BSC, to help power companies evaluating the performance of FMS projects. Conventional analytic hierarchy process (AHP) and analytic network process (ANP), two commonly used multiple criteria decision-making models, adopt pairwise comparisons of criteria (or alternatives) to evaluate the final performance. However, considering the aspects of benefits (B), opportunities (O), costs (C), and risks (R) of an alternative, and synthesizing the positive criteria of benefits (B) and opportunities (O), and the negative criteria of costs (C) and risks (R) with a rating calculation (not pairwise comparison) by methods such as additive, subtractive, and multiplicative, are more comprehensive and distinctive methods in real practice. Accordingly, ANP associated with BOCR is applied in this paper to address positive and negative criteria simultaneously in evaluating IT projects.

This paper is organized as follows. In Section 2, the characteristics of project management with suitable KM methods are introduced. Performance evaluation of new project development is examined in Section 3. The ANP model with BOCR for evaluating FMS projects is constructed in Section 4, and a real case is examined in Section 5. Some conclusion remarks and discussions are provided in the last section.

2. Project management with suitable KM methods

Process management philosophy and associated procedures, which focus on effectiveness and efficiency, need to be comprehensively process-oriented, customer-focused, fact-based, and participative throughout a firm (Winter, 1994). More generally, studies in learning and evolution have suggested that increased routine and coordination in an organization's activities may expedite responsiveness in stable environments. However, it also contributes to resistance to change, competency traps, and inadequate or inappropriate responses in changing environments (Levitt and March, 1988). Efforts toward tighter horizontal coordination have created greater interdependencies and interactions, increased congruence among organizational routines, and created system-wide benefits of continued incremental change, leading to further stability and focus on incremental change (Siggelkow, 2002). On the contrary, exploratory innovation requires flexibility, breaking existing rules, autonomy, risk taking, and tolerance for mistakes in the pursuit of new knowledge (Levinthal and March, 1993). Exploration and exploitation have been characterized as fundamentally different search modes (March, 1991). While exploitation involves a local search that builds on a firm's existing technological capabilities, exploration involves a more distant search for new capabilities (Weick, 1979). Exploitative innovations involve improvements on existing components and architectures, and build on the existing technological trajectory, whereas exploratory innovation involves a shift to a different technological trajectory (Rosenkopf and Nerkar, 2001). Empirical literatures reflecting this dichotomy often distinguish between innovations that leverage a firm's existing knowledge and innovations that rely on no previous firm knowledge (Rosenkopf and Nerkar, 2001). From clear controversy between these two arguments above, a firm with a limited budget and focused technology can adopt a trade-off strategy between exploitation and exploration. The trade-off between efficiency and innovation can be eliminated when an integrated project development is monitored to ensure that they follow a specific fluctuation pattern (Naveh, 2005). The specific fluctuation pattern of a new project means that innovation must be emphasized in the early stages, and efficiency must be emphasized in the later stage to facilitate control and coordination. However, while this may create effectiveness, efficiency, and exploitative innovation at the same time, it will also crowd out exploratory innovation. Obviously, these approaches emphasize on compromise and balance in strategies with some specific limitations. A recent research concluded that the proficiency of process management is the mediator for new project performance, and is the moderator for new project innovation (Salomo et al, 2007). Although innovation is widely recognized as a potentially vital source of competitive advantage, firms still struggle to find efficient and effective process management methods associated with innovation for new project development.

Business performance, which is related to immediate gain from the utilization of a firm's knowledge base, can be regarded as the outcome of exploitative activities, while knowledge performance, which measures the degree of new knowledge creation, can be regarded as the outcome of exploratory activities. Since exploration and exploitation are different in terms of logic, and both logics are competing for a firm's limited resources, there will inevitably be tensions between them, making it difficult for the firm to achieve a satisfactory balance. Once potentially valuable knowledge and skills have been acquired through exploration, a firm can turn to exploitation activities. As a result, the superior knowledge can enable incremental innovations that lead to better quality and lower costs, and thus contribute to better business performance (Ahn et al, 2006). Because such exploitation depends on prior exploration, the knowledge performance of a given project can positively affect business performance of subsequent projects. Accordingly, this offers a relationship with no contradiction, but with harmony. On the other hand, it was found that business service improvements and new service introductions are significantly associated with shared knowledge, such as codified service solutions or team-based competences and procedures. The results also suggested that tacit collective knowledge is more closely associated with new service introductions, whereas explicit collective knowledge is associated with service improvements (Leiponen, 2006). A managerial implication is that new service introductions necessitate team competences and routines, whereas incremental service improvements are more likely to occur if procedures are in place to codify services into explicit solutions or technologies. Thus, the KM approach should depend on the service toward continuous improvement of existing services or development of completely new services. Although the targets and methods of new service introductions and incremental service improvements are different, they exist in time sequence. Accordingly, this is an alternative way to show that exploitation and exploration are not in conflict.

To distribute and transfer knowledge within project development, three forms of organization for KM exist (Nonaka, 1994; Brown and Duguid, 2000; Söderquist, 2006):

1. A central KM function: a team of specialists headed by a chief knowledge officer is in charge of KM-related activities in all development projects.
2. A project-decentralized KM task force: a task force headed by a leader reporting to the project manager is in charge of KM-related activities that are internal to each development project.
3. Functionally located KM cells: developing knowledge in support of development projects and disseminating knowledge to project groups, it is the responsibility of a cell within the specialized functional departments.

Stating the KM mission clearly, the centralized structure provides the strongest alignment between R&D strategy and KM initiatives, the best communication and coordination of KM activities, and the clearest responsibilities of different players involved. These factors lead operational staff to commit strongly to KM initiatives. Project-decentralized KM task forces fail to deliver strong strategic alignment. Instead, the mission of KM activities becomes subordinated to those of the projects, leading to different initiatives developed across each project and little cumulative effect. Conversely, the strength of this structure is its focus on KM efforts as driven by operational needs, and this enables rapid and pragmatic testing of the contributions of the initiatives in the field. The functionally located KM cells structure results in a blurred KM vision and high uncertainty of KM mission, both at the strategic R&D management and the operational project management levels. Its KM mission and vision are subordinate to the priorities and politics of the specialized department (Dyer and Nobeoka, 2000). In terms of the impact on knowledge transferring and sharing, the centralized structure provides good overview of project needs and individuals' knowledge, and rapid transfer of knowledge between similar problem-solving activities from team to team, and from project to project. Project-decentralized KM task forces strongly drive inter-functional knowledge sharing, and provide the highest potential for supporting transfers and sharing of tacit knowledge inside, as well as explicit knowledge outside. The functionally located KM cells focus on knowledge deep in fields of expertise and strong transfer within the engineering functions (Hammer et al, 2004). In addition, even though knowledge capture and sharing can exploit existing knowledge or distribute new knowledge, the creation of new knowledge is the most beneficial to innovation (Armbrecht et al, 2001). For the adoption of the four knowledge creation modes, such as socialization, externalization, combination, and internalization, during the concept and development phases of new project development, some researchers concluded that socialization during the concept phase and combination during the development phase are positively related to new project success. However, externalization during the concept phase, as well as socialization and internalization during the development phase, is negatively related to new project success (Nonaka et al, 2000; Schulze and Hoegl, 2006).

Based on the above literatures, this research proposes that exploratory and exploitative innovation and process management can be independently developed as long as a critical factor, 'the knowledge under specific environment is being managed', can be analysed and understood in advance, and then suitable management methods for the development of new projects can be adopted at a specific time and place. The development of new project based on the knowledge creation mode, as shown in Figure 1, and mature level of the project, as shown in Figure 2, can be divided into three sequential steps, as shown in Table 1: (i) conceptual phase: idea generation, research, and analysis; (ii) development phase: prototyping and technical engineering; (iii) implementation phase: integration, installation, and final availability test. Step 1 focuses on creating exploratory innovation using socialization without process management. Step 2 stresses on creating exploitative innovation using externalization, combination, and creating effectiveness and efficiency using process management. Step 3 concerns process innovation using combination and internalization, and creating effectiveness and efficiency using process management. In addition, to effectively disseminate and transfer knowledge, functionally located KM cells are applied inside each phase while a project-decentralized KM task force is adopted within the interfaced region between each step. A central KM function is also installed to monitor overall KM performance, and stipulate strategic objectives. A full understanding of KM methods, the development of new projects, and their utilization at a specific situation is necessary; otherwise, negative and chaotic effects may be resulted.

Figure 1.

Different knowledge creation mode for each phase.

Full figure and legend (12K)

Figure 2.

Three sequential steps of project management with variant KM methods.

Full figure and legend (29K)

Table 1 - The sequential procedures of new project development.

Full table (54K)

3. Performance evaluation of new project development

Today, no one would dispute that IT has become the most important cornerstone of an enterprise's ability to successfully compete in the global marketplace. As the resource commitments to IT investment continue to escalate, the following questions are being asked more frequently than ever before: is the investment in IT worthwhile? Is the IT application we implemented a success? Is our IT department, or function, productive and effective? Hochstrasser and Griffiths (1991) found that only 18% of the organizations in their samples relied on rigorous methods to calculate the benefits of investment in IT. At least 22% of expenditure on IT was wasted, and 40% of IT projects realized no net benefits (Willcocks and Lester, 1994). The reason for these failures can be a complex interaction among technical, human resource, environmental, organizational, and management issues. A consensus among academia and practitioners is that IT investments should be carefully justified, measured, and controlled (Fitzgerald, 1998). In practice, the traditional capital-investment appraisal techniques are by far the most commonly used. Nevertheless, serious doubts concerning the fitness of these techniques in an IT environment arise. IT investments have special characteristics such as high risks, long time return, large proportion of intangible costs and benefits, which make the use of most techniques very difficult and the reliability of the outcome very uncertain. Efforts have been made to develop more appropriate techniques; however, neither adjusted techniques nor new techniques are frequently used. This might be explained by the outcome of these techniques, which are difficult to interpret and use, and the fact that some significant problems, such as the estimation of hidden costs remain unsolved. Since all techniques have their drawbacks, it is safe to say that reliance on a sole technique may lead to sub-optimal solution or even failure. Therefore, a combination of techniques to eliminate or diminish the weaknesses of each of the techniques is recommended. Kaplan and Norton (1992) developed a BSC framework that helps top management to select a set of measures that provide an integrated insight at a company. This framework is a mixture of traditional (financial perspective) and new evaluation methods, and encourages a shift from financial-based evaluation techniques to strategy and vision. Thus, the metrics used in a BSC framework are aligned to the corporate strategy and business missions. Milis and Mercken (2004) recommended a multi-layer evaluation process, which was derived from the BSC, for the appraisal of major IT investment projects. The multi-layer evaluation process uses different evaluation techniques, which are more or less ordered in a hierarchical manner. In the first stage, all investments that do not contribute to the corporate strategies or missions are rejected (Willcocks and Lester, 1996). In the second stage, NPV technique is used for tangible parts, and pairwise comparison is adopted for intangible costs and benefits (Earl, 1989). In the last stage, an analysis of the risks and uncertainties is made before making a final decision (Meredith and Hill, 1987). It is believed that the ANP is a versatile multi-attribute decision methodology that can be adapted to a wide range of BSC decision environments (Leung et al, 2006). Nevertheless, there are some disadvantages and pitfalls when using BSC for the evaluation of IT investments (Willcocks and Lester, 1994). First of all, there are no generic IT measures and models that fit all organizations. Second, when using BSC for IT purposes, the perspective might be too narrow if the scorecard is only prepared from the IT department perspective. Accordingly, using ANP with BSC to evaluate final performance of IT projects needs to be further revised.

A key player in the project management performance is the critical factors. Critical factors should receive constant and careful attention from management because they drive the organization to focus attention on the success of the project (Asrilhant et al, 2006). Accordingly, in order to evaluate IT project performance of FMS, one needs to identify critical factors of the project and then integrate them with a modified BSC. Based on extensive literatures (Mulebeke and Zhang, 2006; Yusuf et al, 2006) and interview with experts, the most important factors for the FMS project under benefits (B), opportunities (O), costs (C), and risks (R) are described in this paper. Under benefits (B), the criteria are: functionality (to what extent the finished product complies with all functionality targets, including all planned features), reliability (to what extent the finished product meets all reliability objectives, such as accuracy, number of errors in the product, quality, etc), and usability (to what extent the finished product meets user-friendly characteristics, such as ease of use, automated, easy to repair, easy fault identification, etc). The criteria under opportunities (O) are technology improvement and learning, quick and easy access of information, and sharing knowledge with different experts. The criteria under costs (C) are: customer complaint (severe levels of faults, and number of complaints), extra expenditure required, and failure of target attainment (handling time and quality assurance). The criteria under risks (R) are: difficulties of extension and expansion (to what extent the finished product does not meet the traits of easy expansion including scan point number, hardware and software, and site numbers), difficulties of technology attainment, and inflexibility and incompatibility (incompatibility of hardware support, application software to provide ability to interface, support and managing files, performing storage, retrieval, manipulation and transmission functions). In order to evaluate performance of FMS project in the subsequent case study, the authors constructed a BOCR framework with 12 critical success criteria, as shown in Table 2. The analysis is presented in Section 5.

Table 2 - Structure of BOCR network.

Full table

4. An ANP model with BOCR

The AHP, proposed by Saaty (1980), is a simple, mathematically based multi-criteria decision-making tool to deal with complex, unstructured, and multi-attribute problems. The ANP, also proposed by Saaty (1996), is a generalization of the AHP, and is used to solve problems that can be represented by networks. Under either AHP or ANP, Saaty (1996) further proposed a method to let decision makers to deal with the benefits, opportunities, costs, and risks (the BOCR merits) of a decision.

A systematic ANP model with BOCR is proposed to solve the FMS project evaluation problem (Lee, 2008). The steps are summarized as follows:

Step 1:

Form a committee of experts in the industry and define the FMS project evaluation problem.

Step 2:

Construct a control network for the problem. A control network, as depicted in Figure 3, contains strategic criteria, the very basic criteria used to assess the problem, and the four merits, benefits, opportunities, costs, and risks.

Step 3:

Determine the priorities of the strategic criteria. A questionnaire with Satty's nine-point scale is prepared to obtain pairwise comparison results of the importance of strategic criteria toward achieving the overall objective (Saaty, 1980). For each expert's questionnaire results, form a pairwise comparison matrix. Calculate the priorities of the strategic criteria, and examine the consistency property of the matrix. If an inconsistency is found, the expert is asked to revise the questionnaire, and the calculation is done again. Aggregate the priorities from the experts by a geometric mean method.

Step 4:

Determine the importance of benefits, opportunities, costs, and risks with respect to each strategic criterion. A five-step scale is used, and the values of each scale is assigned to be very high, 0.42; high, 0.26; medium, 0.16; low, 0.10; and very low, 0.06 (Saaty, 2004; Erdomu et al, 2005,2006). Geometric mean method is applied to aggregate experts' opinions.

Step 5:

Determine the priorities of the merits. Calculate the priority of a merit by multiplying the score of a merit on each strategic criterion from Step 4 with the priority of the respective strategic criterion from Step 3 and summing up the calculated values for the merit. Normalize the calculated values of the four merits, and obtain the priorities of benefits, opportunities, costs, and risks, that is, b, o, c, and r, respectively.

Step 6:

Decompose the FMS project evaluation problem into a network with four sub-networks. Based on literature review and experts' opinions, a network in the form as in Figure 4 is constructed. Four merits, benefits (B), opportunities (O), costs (C), and risks (R), must be considered in achieving the overall goal. A sub-network is formed for each of the merits. For instance, for the sub-network for benefits (B) merit, there are criteria that are related to the achievement of the benefits of the ultimate goal, and the lowest level contains the FMS projects that are under evaluation.

Step 7:

Formulate a questionnaire based on the networks to pairwise compare elements, or factors, in each level with respect to the same upper level element, and the interdependence among the elements. Forbenefits (B) and opportunities (O), the question is to ask what gives the most benefit or presents the greatest opportunity to influence fulfillment of the criterion (sub-criterion). For costs (C) and risks (R), the question is to ask what incurs the most cost or faces the greatest risk. Experts in the field are asked to fill out the nine-point-scale questionnaire.

Step 8:

Calculate the relative priorities in each sub-network. A similar procedure as in Step 3 is applied to establish relative importance weights of criteria with respect to the same upper level merit, the interdependence priorities among the criteria that have the same upper-level merit, and the relative performance weights of alternatives with respect to each sub-criterion.

Step 9:

Calculate the priorities of alternatives for each merit sub-network. Using the priorities obtained from Step 8, form an unweighted supermatrix, a weighted supermatrix, and a limit supermatrix for each sub-network by ANP, which is proposed by Saaty (1996). The priorities of the alternatives under each merit are calculated by normalizing the alternative-to-goal column of the limit supermatrix of the merit.

Step 10:

Calculate overall priorities of alternatives by synthesizing priorities of each alternative under each merit from Step 9 with corresponding normalized weights b, o, c and r from Step 5. There are five ways to combine the scores of each alternative under B, O, C and R (Saaty, 2003).

1. Additive
   
   
   
   where B_i, O_i, C_i and R_i represent the synthesized results of alternative i under merits B, O, C and R, respectively, and b, o, c and r are normalized weights of merits B, O, C and R, respectively.
2. Probabilistic additive
   
   
   
   
3. Subtractive
   
   
   
   
4. Multiplicative priority powers
   
   
   
   
5. Multiplicative
   
   
   
   

Figure 3.

The control hierarchy.

Full figure and legend (31K)

Figure 4.

The BOCR network.

Full figure and legend (83K)

The above model, integrated with 12 critical factors described in Section 3, will be applied to evaluate performance of FMS projects in the next section.

5. Case study

FMS integrates resources of existing DMS to build an overall distribution feeder automatic system in order to: (i) remotely control/monitor normal-open, and normal-close power distribution systems; (ii) suspend the normal-open feeder when a fault occurs, isolate the fault area, and then restore the power supply to the blackout area (fault identification and service restoration); and (iii) help power dispatchers to improve power supply quality and reliability through variant technologies in advanced feeder automation. In order to examine the practicality of the proposed models and methodologies, an anonymous power company in China running FMS project since the beginning of 2006 was used as an example. The FMS project, containing four regional projects, was implemented by a joint-venture company, and each of the projects has been executed for more than 9 months. The execution of the project contains three steps: conceptual phase, development phase, and implementation phase. Appropriate KM methods and process management, as described in Table 1 and Figure 2, were applied into each phase of the two regional FMS projects, titled FMS(A) and FMS(D). The other two regional projects, titled FMS(B) and FMS(C), only adopted conventional project management techniques, such as quality control, schedule control, etc. In order to evaluate the final performance of different regional FMS projects, seven senior managers, including technology development manager, research manager, operations manager, marketing manager, purchasing manager, dispatching manager and controller, contributed their professional experience and formed the evaluation committee. Their first task was to verify the critical success criteria introduced in Section 3. The committee also confirmed the firm's strategic criteria as performance, business drivers, and marketing needs, based on previous literatures and practical experiences. The goal, strategic criteria, merits, criteria, and evaluated projects for the FMS evaluation problem are listed in Table 2.

An ANP with BOCR approach described in Section 4 is applied here to compare the development performance of FMS projects that do not undertake KM methods and process management including FMS(B) and FMS(C) and that undertake the methods including FMS(A) and FMS(D). The structure for determining the project's overall performance is shown in Table 2 and Figures 3 and 4. The relationship can be divided into two networks: the control network and the BOCR network. The first level of the control network contains the goal, the evaluation of the best FMS project. In the second level, three strategic criteria are considered, namely, performance, business driver, and market needs (Mulebeke and Zhang, 2006). Performance concerns the capabilities of the technology for delivering the expected results in variant processing environments, such as functionality and usability. Business drivers are defined as business expectations of the firm assessed by business managers, engineers or developers, for instance, time to business operations, learning and innovation. Market needs consider whether the firm possesses advanced technology to satisfy customer needs in comparison with other competitors. In the third level, there are four merits: benefits (B), opportunities (O), costs (C), and risks (R). The purpose of the control network is to calculate the priorities of the four merits, b, o, c and r.

The BOCR network has the same goal as the control network does, and the purpose of this network is to calculate the priorities of the FMS projects. The second level of the network composes of the four merits. The BOCR network can be further divided into four sub-networks: benefits sub-network, opportunities sub-network, costs sub-network, and risks sub-network. In the third level of the network, 12 selected criteria described in Section 3 and shown in Table 2 are applied here to evaluate the performance of each FMS project. Under benefits merit, there are three criteria, group factors (a)–(c). Under opportunities merit, there are three criteria, group factors (d)–(f). Group factors (g)–(i) are the criteria of costs merit, and group factors (j)–(l) are the criteria of risks merit. Four evaluated projects are in the last level of the network.

A questionnaire was constructed, and the members of the evaluation committee were invited to contribute their professional experience. Based on the collected opinions of the experts and the proposed model, the performance of the four regional projects could be generated. In the first part of the model, experts were asked to evaluate the priorities of benefits, opportunities, costs, and risks. Based on each expert's opinion, a pairwise comparison matrix was formed to evaluate the three strategic criteria, and the priorities of the strategic criteria were calculated. The consistency property of the matrix was also examined. Geometric mean method was applied to synthesize experts' opinions. The final pairwise comparison of the experts on the three strategic criteria with respect to the goal is as shown in Table 3, and the priorities of performance, business drivers, and market needs are 0.5960, 0.2964, and 0.1076, respectively.

Table 3 - Comparison matrix for the strategic criteria.

Full table

Next, experts were asked to assess BOCR according to strategic criteria by the five-step scale. The ratings of the four merits on strategic criteria by geometric mean method are shown in Table 4. The normalized priorities of BOCR are calculated and shown in the last column of Table 4.

Table 4 - Priorities of benefits, opportunities, costs, and risks.

Full table

In the second part of the model, the priorities of the alternatives under each merit are calculated. There are four sub-networks, namely benefits, opportunities, costs, and risks. The relative importance weights of criteria with respect to the same upper level merit, and the interdependence priorities among the criteria that have the same upper-level merit are calculated using the geometric mean of the pairwise comparison results. The priorities of criteria, without considering the inter-relationship among criteria, are shown in the third column of Table 5. After taking into account the inter-relationship, the synthesized priorities of criteria are shown in the last column of Table 5.

Table 5 - Priorities of merits and criteria.

Full table

The importance of criteria in evaluating the FMS projects should be understood by the management. Under the benefits merit, the most important criterion, out of the three criteria, is usability, with a priority of 0.5710, after the consideration of interdependence among criteria. The second most important criterion is functionality, with a priority of 0.2732. This means that the major benefit concern for the firm in having the FMS project is to have a usable functional system to operate. Under the opportunities merit, technology improving and learning (0.5656) is the most important criterion. This implies that whether a new system can lead to the improvement and learning of technology is essential for the future prospect of the FMS project. Under the costs merit, customer complaint (0.5250) is the major concern while the target attainment ranks the second (0.3733). This means that a new system concerns most about customer satisfaction. Under the risks merit, inflexibility (0.5569) is the problem the firm worries most about. This means that a new system is essential to have the compatibility of hardware support, application software, and data storage when the system is expanded or extended in the future.

The performance results of different contractors under various criteria, however, are collected from each expert individually in order to limit the number of pairwise comparisons (Saaty, 1980). All criteria, except extra expenditure required, are qualitative criteria and are rated in a range from zero to a hundred. For the criteria under benefits and opportunities merits, the higher the score, the better the performance of the FMS project is. On the other hand, for the criteria under costs and risks merits, the higher the score, the worse the performance of the FMS project is. Extra expenditure required is a quantitative criterion under costs merit. The larger the estimated amount is, the worse the performance of the FMS project is. The synthesized performance value of each FMS project on each criterion is calculated by geometric averaging the results from all the experts. The results are shown in Table 6. These performance values are further transformed so that the values of FMS projects on the same criterion are summed up to be one. The above performance values of FMS projects, the priorities of criteria, and the interdependence among criteria are entered into appropriate places in the unweighted supermatrix for each merit sub-network. As an example, the unweighted supermatrix for the benefits sub-network is as shown in Table 7. The weighted and limit supermatrices for each sub-network are calculated. The normalized performances of FMS projects under the four merits are obtained as shown in Table 8.

Table 6 - The qualitative and quantitative results of different criteria under different FMS.

Full table

Table 7 - Unweighted supermatrix for benefits.

Full table

Table 8 - Priorities of alternatives under four merits.

Full table

The final ranking of FMS projects is calculated by the five methods to combine the scores of each alternative under B, O, C and R. The results are as shown in Table 9. Under four out of the five methods of synthesizing the scores of alternatives, FMS(D) and FMS(A) (adopting appropriate KM methods and project process management) rank the first and the third respectively, while FMS(C) and FMS(B) (without adopting KM methods and project process management) rank the second and the fourth. Because the final performance will also be affected by organizational culture, existing process management, and leadership style, it does not guarantee that a new project with the adoption of KM methods and process management, needs to outperform a new project without KM methods and process. However, in average, the performance of FMS(D) and FMS(A) is better than that of FMS(C) and FMS(B). Although the ranking of FMS(C) is better than that of FMS(A), their overall scores are rather close. In addition, in the opportunities merit, the performances of FMS(D) and FMS(A) (both with proposed methodologies) is much better than that of FMS(B) and FMS(C) (both without proposed methodologies). That means adopting the proposed methodologies provides the opportunities to technology improving and learning, quick and easy access of information, and sharing knowledge with different experts. All these opportunities are essential for the competition in the future. Also note that under the multiplicative method, which does not concern the weights of B, O, C and R (b, o, c and r), FMS(A) (with proposed methodologies) performs better than FMS(C) (without proposed methodologies). The ranking of the two projects may be changed under the other four methods when the weights of B, O, C and R (b, o, c and r) are changed. This can be observed by carrying out sensitivity analysis.

Table 9 - Final synthesis of priorities of alternatives.

Full table

The results of the sensitivity analysis are shown in Table 10. When the priorities of merits are changed, the ranking of projects may be different too. Use benefits as an example. No matter how the priority of benefits (b) decreases or increases, the best project remains to be FMS(D). However, the second best projects become both FMS(C) and FMS(A) when b decreases to 0.1292, 0.2100, 0.2140 and 0.1750 under the additive, probabilistic additive, subtractive, and multiplicative priority powers method, respectively. The second best alternative remains to be FMS(C) no matter how much b increases under the above four methods. Note again that the priorities of merits have no effect on the solution under the multiplicative method. Based on Table 10, we can see that FMS(D) remains to be the best alternative no matter how the priorities of merits change, except when c increases to a very large number, which is unlikely to happen. The second best project is FMS(C); however, it may become FMS(A) in some instances. Therefore, the importance of the four merits will determine whether FMS(C) performs better or worse than FMS(A).

Table 10 - Sensitivity analysis under different priorities of merits.

Full table

6. Conclusion

In past literatures, compromised strategies were recommended to deal with project management with conflicted innovation, effectiveness, and efficiency goals. This research proposed the analysis of knowledge creation modes and project maturity. The most appropriate project management, including suitable forms of organizations for KM and suitable project process management according to the characteristics of project, such as creation modes and mature level, was proposed to achieve innovative, efficient, and effective new project management.

The proposed models are recommended to practitioners in developing new projects. However, our proposed models cannot diminish the gap between strategic targets and the performance of evaluated projects since the models only evaluate the performance of the strategic selected project, not strategic targets. Accordingly, our future research direction is to identify the characteristics of the dynamic link in order to diminish the gap.

References

1. Ahn JH, Lee DJ and Lee SY (2006). Balancing business performance and knowledge performance of new product development. Long Range Plann 39: 525–542. | Article |
2. Armbrecht FMR Jr., Chapas RB, Chappelow CC and Farris GF (2001). Knowledge management in research and development. Res Technol Mngt 44(4): 28–41.
3. Asrilhant B, Meadows M and Dyson R (2006). Techniques to support successful strategic project management in the UK. Eur Mngt J 24(2/3): 214–225.
4. Brown JS and Duguid P (2000). Knowledge and organization: A social practice perspective. Org Sci 12(2): 198–213. | Article |
5. Dyer JH and Nobeoka K (2000). Creating and managing a high-performance knowledge-sharing network: The Toyota case. Strat Mngt J 21(3): 345–367. | Article |
6. Earl JM (1989). Management Strategies for Information Technology. Prentice Hall: Englewood Cliffs.
7. Erdomu S, Kapanoglu M and Koç E (2005). Evaluating high-tech alternatives by using analytic network process with BOCR and multiactors. Eval Prog Plann 28: 391–399. | Article |
8. Erdomu S, Aras H and Koç E (2006). Evaluation of alternative fuels for residential heating in Turkey using analytic network process (ANP) with group decision-making. Renew Sust Energy Rev 10: 269–279. | Article |
9. Fitzgerald G (1998). Evaluating information systems projects: A multidimensional approach. J Inform Technol 14: 17–30.
10. Hammer M, Leonard D and Davenport T (2004). Why don't we know more about knowledge? MIT Sloan Mngt Rev 45(4): 14–18.
11. Hochstrasser B and Griffiths C (1991). Controlling IT Investment: Strategy and Management. Chapman & Hall: London.
12. Hoegl M and Wagner SM (2005). Buyer–supplier collaboration in project development projects. J Mngt 31(4): 530–548.
13. Kaplan RS, Norton DP (1992). The balanced scorecard: Measures that drive performance. Harvard Bus Rev January/February: 71–80.
14. Lee AHI (2008). A fuzzy supplier selection model with the consideration of benefits, opportunities, costs and risks. Expert Syst Appl, in press, doi: 10.1016/j.eswa.2008.01.045.
15. Leiponen A (2006). Managing knowledge for innovation: The case of business-to-business services. J Prod Innovat Mngt 23: 238–258. | Article |
16. Leung LC, Lam KC and Cao D (2006). Implementing the balanced scorecard using the analytic hierarchy process & the analytic network process. J Opl Res Soc 57: 682–691. | Article |
17. Levinthal D and March JG (1993). The myopia of learning. Strat Mngt J 14: 95–112. | Article |
18. Levitt B and March JG (1988). Organizational learning. Ann Rev Sociol 14: 319–340. | Article |
19. Liu Y, Zhang P and Qiu X (2000). Optimal reactive power and voltage control for radial distribution system. IEEE Trans Power Syst 2(1): 85–90.
20. March JG (1991). Exploration and exploitation in organizational learning. Org Sci 2: 71–87.
21. Matsuzawa K, Suzuki M, Okuhara Y and Kishida Y (1990). New energy management system for central dispatching center of Tokyo Electric Power Co., Inc. IEEE Trans Power Syst 5(4): 1112–1117. | Article |
22. Meredith JR and Hill MM (1987). Justifying new manufacturing systems: A managerial approach. Sloan Mngt Rev 28(4): 49–61.
23. Milis K and Mercken R (2004). The use of the balanced scorecard for the evaluation of information and communication technology projects. Int J Proj Mngt 22: 87–97. | Article |
24. Mulebeke JAW and Zhang L (2006). Analytic network process for software selection in product development: A case study. J Eng Technol Mngt 23: 337–352.
25. Naveh E (2005). The effect of integrated product development on efficiency and innovation. Int J Prod Res 43(13): 2789–2808. | Article |
26. Nonaka I (1994). A dynamic theory of organizational knowledge creation. Org Sci 5(1): 14–37. | ISI |
27. Nonaka I, Toyama R and Konno N (2000). SECI, Ba and leadership: A unified model of dynamic knowledge creation. Long Range Plann 33: 5–34. | Article |
28. Ozman M (2006). Knowledge integration and network formation. Technol Forecast Soc Change 73: 1121–1143. | Article |
29. Rosenkopf L and Nerkar A (2001). Beyond local search: Boundary spanning, exploration, and impact in the optical disc industry. Strat Mngt J 22: 287–306. | Article |
30. Saaty RW (2003). Decision Making in Complex Environment: The Analytic Hierarchy Process (AHP) for Decision Making and the Analytic Network Process (ANP) for Decision Making with Dependence and Feedback. Super Decisions: Pittsburgh.
31. Saaty TL (1980). The Analytic Hierarchy Process. McGraw-Hill: New York.
32. Saaty TL (1996). Decision Making with Dependence and Feedback: The Analytic Network Process. RWS Publications: Pittsburgh.
33. Saaty TL (2004). Fundamentals of the analytic network process – multiple networks with benefits, opportunities, costs and risks. J Syst Sci Syst Eng 13(3): 348–379. | Article |
34. Salomo S, Weise J and Gemünden HG (2007). NPD planning activities and innovativeness performance: The mediating role of process management and moderating effect of product innovation. J Prod Innovat Mngt 24: 285–302. | Article |
35. Schulze A and Hoegl M (2006). Knowledge creation in new product development projects. J Mngt 32(2): 210–236.
36. Siggelkow N (2002). Evolution toward fit. Admin Sci Quart 47: 125–159. | Article |
37. Söderquist KE (2006). Organizing knowledge management and dissemination in new product development. Long Range Plann 39: 497–523. | Article |
38. Weick KE (1979). The Social Psychology of Organizing. McGraw Hill: New York.
39. Willcocks L and Lester S (1994). Evaluating the feasibility of information systems investments: Recent UK evidence and new approaches. In: Willcocks L. (ed). Information Management: The Evaluation of Information Systems Investment. Chapman & Hall: London, pp. 49–77.
40. Willcocks L and Lester S (1996). The evaluation and management of information systems investments: From feasibility to routine operations. In: Willcocks L (ed). Investing in Information Systems: Evaluation and Management. Chapman & Hall: London.
41. Winter S (1994). Organizing for continuous improvement: Evolutionary theory meets the quality revolution. In: Baum J. and Singh J. (eds). Evolutionary Dynamics of Organizations. Oxford University Press: New York, pp. 90–108.
42. Yusuf Y, Gunasekaran A and Wu C (2006). Implementation of enterprise resource planning in China. Technovat 26: 1324–1336. | Article |