Applying Analytical Hierarchy Process to a WiMAX Performance Evaluation Model

1 Introduction
To satisfy the need for multi-media data access any time and anywhere, demand has grown for high speed wireless data transmission. Expansion of the smart phone and embedded device market requires high speed data networks to cope with different application suites.

    Worldwide Interoperability for Microwave Access (WiMAX) is based on the IEEE 802.16 standard. It was initially designed for fixed and nomadic connection; and since 2001, has been continually updated and enhanced. With the introduction of 802.16e in 2005, it started to evolve towards support for fixed and mobile connection. Important technologies such as Multiple-Input Multiple-Output (MIMO), Orthogonal Frequency Division Multiplexing (OFDMA), and Media Access Control (MAC) have been introduced to enable high data rate, enhanced spectral efficiency, and to improve efficiency and coverage in Non-Line of Sight (NLOS) situations.

 
    A number of studies[1] have been carried out to evaluate different WiMAX techniques, and to individually assess uplink/downlink data rate, spectral efficiency, and Quality of Service (QoS). However, such studies have failed to reference overall design objectives. These assessments do not address the fact that WiMAX technology should support multiple types of applications. Thus, evaluation of any individual feature does not help engineers improve design effectiveness in a quantitative manner.

    As a result, there is no consensus about how good WiMAX technology and network performance can actually be. There is a lack of systematic and analytical method to improve overall network performance so that long term high-speed data network development plans can be met.
Analytical Hierarchy Process (AHP) and Data Envelopment Analysis (DEA) can be used to develop a new WiMAX performance model. The purpose of such a model is to establish an agreed-upon performance index (defined as a technological value) that can be measured and compared with design objectives. It is built on the features of broadband wireless access networks, but is not only applicable to WiMAX technology. Indeed, the model can easily be modified for Evolved High Speed Packet Access (HSPA+) and Long Term Evolution (LTE) technologies.

 
2 Methodology of AHP-Based WiMAX Performance Evaluation Model

2.1 AHP Modeling
In the field of technology management, many studies[2] have been carried out in which AHP has been adopted to evaluate or assess technology by prioritizing different criteria. AHP is a comprehensive framework designed to cope with intuitive, rational, and irrational factors when multi-objective, multi-criterion decisions must be made, and a number of alternatives are possible.

    The AHP approach was designed to help decision makers integrate qualitative and quantitative aspects of complex problems. Complex problems are solved by decomposing the structure of the problem into hierarchies, and pair-comparisons are made to determine the priorities in each hierarchy.

    Typically, an hierarchical model comprises three or four levels—Objectives, Criteria, Sub-Criteria, and Alternatives. Objectives are on the top level while alternatives are on the lowest level, as shown in Figure 1. Pair-comparisons determine the importance of design criteria to the design objective. The results are represented as a relative value which indicates the technical value of a criteria compared to others. For example, technology 1 may be twice as important as technology 2 in meeting design objectives. These results are useful but insufficient because they do not provide a holistic measurement of the overall performance of WiMAX technology. They do not show how the technology should progress in meeting objectives.

    As a result, the conventional approach of constructing an AHP hierarchical model with technological alternatives at the lowest level is insufficient for measuring overall WiMAX performance. A new hierarchical approach called DEA[3] is proposed in which the lowest level is replaced with technological efficiency, as shown in Figure 2. Technological efficiency is a semi-absolute value with a maximum of 100%. This enables technologies to be measured in a simple and comparable format.

2.2 Integrating DEA into AHP Model
DEA is one of the key methods for measuring technological efficiency of WiMAX or Broadband Wireless Access (BWA). It is a multi-factor efficiency analysis model for determining the relative efficiencies of a set of decision making units or criteria.

    Technological efficiency is represented as a ratio of the maximum sum of output to the relevant input bundle. With this approach, technological criteria are evaluated according to a semi-absolute value; ideal technological efficiency in terms of network performance is 100.
Telecom systems—including wireless networks—can be compared in this way. However, never before has DEA been applied to WiMAX or BWA technology where multiple design criteria are used to assess efficiency.

    Unlike traditional wireless communications in which capacity or spectral efficiency is the benchmark for assessing technology, multiple design criteria must be considered when measuring the performance of WiMAX technology that supports not only voice, but also converged data, multi-media, and other IP applications.

    By combining AHP and DEA methodology, design criteria can be weighed and prioritized before being integrated into the DEA framework for deriving technological efficiency. The technological efficiencies for each design criteria are then aggregated and transformed into the technology value, which is a quantifiable yardstick for measuring overall WiMAX performance.

 
    The methodology is as follows:
    (1)  Performance measuring criteria for WiMAX technology must be identified and categorized into main performance groups. Main design criteria and sub-criteria are used to build the AHP hierarchical model as shown in Figure 3.

    (2)  Using AHP, the basic elements of each level—such as the measuring criteria and sub criteria—are then compared in pairs with respect to the design objectives. The comparison is based on a 9 point scale that indicates the relative importance of each
criteria/sub-criteria.

    (3) DEA framework is used to determine the technological efficiency for each design criteria. These technological efficiencies can then be used to evaluate the technology based on the degree to which its design criteria satisfy the desirability on the measure of effectiveness of each design factor.

    (4) By integrating AHP priority with technological efficiency within the DEA framework, the technological value of WiMAX can be derived. This is a measure of WiMAX performance in relation to technological expectations, and can be represented by the mathematical model:

    In Equation (1),

• TVn: Technological value for measuring the performance of WiMAX in relation to technological expectations;
• Wk: Relative priority of main criterion (k) with respect to the
  company’s design objective;
• fk, j : Relative priority of sub criterion (j ) with respect to the main criteria (k);
• tn,k, j: Performance and physical characteristics of design factor (n) along with main criteria (k) and sub criterion (j );
• tn,k: Performance and physical characteristics of design factor (n) along with main criteria (k);
• V(t  ,k, j): Desirability value of the performance and physical characteristics of technology (n) along with sub criterion (j ) for main criterion (k);
• V(t  ,k): Desirability value of the performance and physical characteristics of technology (n) for main criterion (k).

3 Characteristics of WiMAX Technology
WiMAX is a BWA technology, and BWA is defined as wireless access with broadband connection capabilities. Broadband should have instantaneous bandwidths greater than 1 MHz and support data rates greater than 1.5 Mbit/s (IEEE standard for Local and Metropolitan area networks Part 16 2009). This is the bare minimum for widespread success. Broadband wireless systems must deliver multi- megabit per second throughput to end users,  and have robust QoS to support voice, data, and multimedia services. Indeed, it should be able to support IP-based applications and services in a highly efficient manner.

    To meet stringent service requirements and to overcome constraints imposed by wireless, there are several technological challenges that BWA must overcome[4].

    (1) Data Rate
    Reliable transmission and reception schemes need to be developed to push broadband data through hostile wireless channels. Typically, services require high data rates and high-speed mobility. Wireless communication relies on complex radio wave propagation mechanisms so that signals can traverse through the intervening environment. 

    Several factors affect or impair transmission:

• Decay of signal power in an NLOS environment;
• Inter-symbol interference in a multipath environment when time delay between variou signal paths becomes significant;
• Multipath fading caused by large variations in the amplitude of the received radio signals;
• Doppler effect due to relative motion between transmitter and receiver.

    (2) Spectral Efficiency
    High spectral efficiency and broad coverage must be achieved because only a limited amount of spectrum is available to deliver broadband services to a large number of users. The capacity of a wireless system is closely related to its frequency usage. For higher capacity and spectral efficiency, frequency reuse must be maximized.  But this can significantly increase the potential for interference. The challenge, therefore, is to design a transmission scheme that can operate with a low Signal to Interference plus Noise Ratio (SINR).
The signal level depends on cell radius and the number of base stations within the coverage area. In order to maintain a certain spectral efficiency within the coverage area, the following should be considered:

• The cell radius of the base station;
• The number of base stations and cost required for the coverage area;
• An effective method of ensuring reliability under low signal to SINR.

    (3) Policy Based QoS
    Broadband Wireless networks must support a variety of voice, data, and multimedia applications. Each application has a different traffic pattern and QoS requirement[5].  Therefore, policy-based QoS should be enforced so that service provision levels are appropriate to different subscriber plans.

    QoS must be delivered end to end across both the wireless link and underlying IP network. The challenge lies in accommodating variable QoS requirements across application, services, and user profile, while maintaining the quality levels as defined for all users. Design factors affecting QoS include:

• Throughput that handles multiple traffic flow;
• Packet loss of different applications;
• RAN latency in handover between cells;
• Jitter in the time between idle to transition.

    (4) Mobility Management
    Mobility is a key element for wireless services. However, support for a subscriber station moving over a large coverage zone leads to two important design considerations[6],

• Roaming: A means of reaching inactive users for session initiation and packet delivery must be provided regardless of their location within the network;
• Handoff: A continuous, uninterrupted session must be maintained with the base station while on the move—even at vehicular speeds.

    (5) Portability and Device Management
    Portability requires the subscriber device to be lightweight, battery powered, and energy efficient so that re-charging is not necessary for an extended period of time[7].

    To reduce power consumption, WiMAX technology should support  a power-efficient transmission scheme, power saving protocol for fast switching technology, signal processing algorithm, and it must be interoperable with other radio channels (e.g. EVDO, Wi-Fi) within the subscriber station.

    (6) Security
    Security is always a major consideration in wireless communication systems. End users are often concerned about privacy and data integrity while network providers are primarily concerned about preventing unauthorized use of network services. WiMAX networks should support:

• Data encryption to avoid eavesdropping over the communication link;
• Authentication of legitimate end users who access the network;
• Access control over subscription applications or services.

    (7) Converged IP-Based Architecture
    IP-based architecture can facilitate rapid development of new services and applications because there is a large development community for leveraging. An IP-based system also tends to be cheaper because there are more supplies in markets that have already been adopted by wired communication systems.

    Although IP-based protocol is simple and flexible, it is not necessarily efficient and robust enough to operate with limited bandwidth in wireless environments. Thus, making IP protocol more bandwidth efficient and guaranteeing QoS is of major concern in WiMAX systems.

    (8) Integration with Legacy Infrastructure
    For service operators with existing wireless infrastructure, it is important to leverage the existing platform and roll out wireless broadband services in a timely and cost effective manner.

    Although WiMAX employs a different RF access technology, integration of existing base stations, Access Services Network (ASN), and Connectivity Service Network (CSN) can bring about seamless upgrade and easy migration of the existing customer base[8].

4 AHP Hierarchical Model for WiMAX Performance Evaluation
AHP prioritizes and assesses network performance criteria/sub-criteria in relation to design objectives. It is particularly useful where certain design criteria such as converged IP architecture and security are not quantifiable. Paired comparisons between criteria support logical and rational decision making.

    AHP formulates a model by structuring network performance criteria in hierarchies consisting of three levels: services operator’s design objectives at the highest level, followed by categorized performance criteria, and then performance sub-criteria at the lowest level.

5 Case Study—An AHP Model for Evaluating Performace of an Operator’s WiMAX Network
The AHP model was adopted by a leading WiMAX service provider to measure their network performance against their desired target. The data—which was collected in 2009—included three years (2009-2011) desired performance data for each design criteria/sub-criteria. The model converted performance data into technological efficiency, which became an integral part of the technological value of the company’s WiMAX network. The technological value was an index for measuring the extent to which the WiMAX network was meeting the provider’s desired performance level.

5.1 Prioritization of the AHP Model
Technological characterization was defined by a group of WiMAX service providers, who also provided judgement on the relative priority of design criteria/sub-criteria. An online evaluation tool was used during face to face interviews. Operators prioritized the importance of each criteria by paired comparisons with other criteria on the same level. The comparison warranted the relative importance of each criteria and sub-criteria, and the results are summarized in Table 1.

    The relative priority for the main criteria (Wk) and the sub-criteria (fk,j ) is an integral part of the AHP model.

5.2 Technological Assessment with Desired Performance Targets
The WiMAX operators estimated their desired network performance levels from 2009 to 2011, and this became the desirable output under the DEA framework. The data was then integrated into the AHP model to become the technological metrics for each of the measuring criteria, as shown in Table 2.

    Desirability graphs representing operators’ preferences for the technological metrics of each design criteria were drawn. Figure 4 shows examples of desirability graphs for data rate and spectral efficiency.

    The technological value represents the overall performance of the AHP model after integrating all the technological efficiencies of the design criteria. For the WiMAX study in 2009, this was 50%. As WiMAX technology continues to improve, the technological value is expected to improve to 70% by 2010, and then to 100% by 2011.

5.3 Formation of the Performance Evaluation Model 
The growth rate of technological value shows WiMAX service providers how design criteria should be improved in order to meet desired performance targets.

    High data rate (0.312) is still the dominant factor affecting overall performance. Any improvement in NLOS coverage and/or interference cancellation would also greatly improve technological value. For example, technological value increases from 8% to 16% between 2009 and 2010, and reaches 31% by the end of 2011.

    High Spectral efficiency (0.163) is the second most important factor. Given that frequency spectrum is the most limited resource in a wireless network ecosystem, operators are concerned about enhancing efficiency and maximizing the coverage and number of end users. Technological value for the operator increased from 4% to 8% between 2009 and 2010, and will reach 16% by 2011.

    The contribution of the above two criteria alone affects the overall WiMAX network performance target by 47%. The other six design factors affect the overall performance target by 53%. This model provides a clear indication of what resource investments need to be made into the network, and monitors performance improvement in a quantitative way.

6 Conclusions
The performance evaluation model provides a quantitative approach to measuring an operator’s WiMAX network performance against their desired targets. It is not designed for comparing one mobile operator’s network performance with another’s, as performance targets differ according to market strategy and investment model.

    However, achieving targets very much depends on equipment suppliers delivering products that align with the operator’s timetable. This model can also be extended to equipment suppliers for the purpose of validating performance targets of individual criteria over some years of study, as well as to enhance supplier roadmaps for future network development. The enhanced model could strengthen partnerships between mobile operators and equipment suppliers.

References
[1] WiMAX system evaluation methodology, V1.0 [S]. WiMAX Forum, 2007.
[2] GERDSI N, KOCAOGLU D F. Applying the Analytic Hierarchy Process (AHP) to build a strategic framework for technology roadmapping [J]. Mathematical and Computer Modelling, 2007,
46(7/8): 1071-1080.
[3] WANG Zhiqiang, ZHANG Lixin, QUE Huakun, et al. Evaluation of wireless access modes in distribution network communication based on weighted DEA [C]// Proceedings of the 1st International Conference on Sustainable Power Generation and Supply (Supergen’09), Apr 6-7, 2009, Nanjing, China. Los Alamitos, CA, USA: IEEE Computer Society, 2009: 6p.
[4] ANDREWS J G, GHOSH A, MUHAMED R. Fundamentals of WiMAX understanding broadband wireless networking [M]. Upper Saddle River, NJ, USA: Prentice-Hall, 2007.
[5] CHARILAS D, MARKAKI O, NIKITOPOULOS D, et al. Packet-switched network selection with the highest QoS in 4G networks [J]. Computer Networks, 2008, 52 (1): 248-258.
[6] TUNG Hoi Yan, TSANG Kim Fung, LEE Lap To, et al. On the handover performance of a tri-threshold bandwidth reservation CAC scheme [J]. ETRI Journal, 2007, 29(1): 113-115.
[7] KIM Min Gon, CHOI Jung Yul, KANG Minko. Trade-off guidelines for power management mechanism in the IEEE 802.16e MAC [J]. Computer Communications, 2008, 31(10): 2063-2070.
[8] A comparative analysis of mobile WiMAX deployment alternatives in the access network [S]. WiMAX Forum, 2007.

[Abstract] Evaluating performance of individual features of WiMAX technology is a topic of widespread discussion. Currently, there is no quantitative way of measuring WiMAX technology so that wireless operators can meet their design objectives. This paper outlines a set of design criteria for WiMAX and provides a decision-making aid that ranks the importance of criteria using Analytic Hierarchy Process (AHP). This ranking should sufficiently reflect market expectations of the relative importance of various design criteria. A model integrating AHP priorities with enhanced Data Envelopment Analysis (DEA) is the basis for formulating a technological value in simple, comparable format. A case study is provided to show how this technological value is used to evaluate a three year network deployment plan. In the future, this model could be extended to WiMAX equipment suppliers for the purpose of validating performance targets of individual criteria, and enhancing supplier roadmaps for future network development.