Cognitive Engine Technology

The work was supported by the National Basic Research Program of China("973" Program) under Grant No. 2009CB320403, and the National Natural Science Foundation of China under Grant No. 60832008.

   Cognitive Radio (CR), an intelligent radio communication system, is able to sense the surrounding electromagnetic environment, wireless channel features and user requirements, and then as a result of reasoning and learning from past experiences, it adjusts its internal configuration and optimizes system performance to adapt to the changes of environment and requirements[1].

   As the abovementioned concept implies, besides the capabilities of sensing and adapting to environment, CR features the learning capability, which is very important to intelligent activities and is what differentiates CR from other traditional radio, sensing radio and adaptive radio. Learning is a basic cognitive activity, a process of experience and knowledge accumulation, in which CR could understand the external environments increasingly. With learning, CR is able to improve the performance of radio communication system.

   Software radio technology lays the foundation for CR implementation. The software reconfigurability of radio modules makes the adaptive adjustment of CR possible. Meanwhile, the intelligent learning capability of CR depends on the artificial intelligence technology. All these make it understandable that Joseph Mitola noted that CR, in principle, is the combination of software radio technology and artificial intelligence technology, when he firstly proposed the concept of CR in 1999[1].

   Cognitive Engine (CE), which realizes intelligent reasoning and learning on the software radio platform and implements the whole cognitive loop, is the key component of CR. CE can be deemed as the "brain" of CR, therefore, cognitive engine technology is the core technology for CR implementation[2-4].

1 Current Implementations of CE
   Most CR-related studies stay theoretical at present. Few implementations of cognitive engine have been reported and they are mainly on the cognition and optimization at physical and link layers only. Among current cognitive engine implementations, the typical ones are developed by the Center for Wireless Telecommunications (CWT) of Virginia Polytechnic Institute and State University (VT), and the Laboratory for Telecommunications Sciences (LTS) of US Department of Defense (DoD).

1.1 VT-CWT CE
   Researchers of VT-CWT propose a general CR architecture[5] as shown in Figure 1. The cognitive engine is designed as an individual module to optimize and control the communication system through its cognitions to the information from the user domain, wireless domain and policy domain.

   According to CWT researchers, CR, to adapt to specific spectrum environments, needs to make adjustments on waveform parameters including frequency, power, modulation modes, constellation sizes, encoding modes and encoding rates. Therefore, CR’s adapting to the radio environment is a process of multi-objective optimization. The Genetic Algorithm (GA) is an effective algorithm for fulfilling multi-objective optimization. Based on this general CR architecture, CWT designs the GA-based cognitive engine[4-5], as shown in Figure 2.

   CE can adjust waveform-related parameters to meet the requirements of Bit Error Rate (BER), signal bandwidth, spectrum efficiency, power, data rate and interference avoidance. So far the cognitive engine shown in Figure 2 has passed the Matrix Laboratory (Matlab) simulation and the test conducted on CR test platforms of VT.

1.2 DoD-LTS CE
   When working with LTS, Charles Clancy and Erich Stuntebeck developed the Open Source Cognitive Radio (OSCR)[7] on the basis of Software Communication Architecture (SCA) and designed a CE[8] to maximize channel capacity through adjustments of modulation and encoding modes in various channel conditions.

   Charles and his workmates believe that CR is the combination of a software radio and a CE, and a suite of Application Program Interface (API) has to be defined between the software radio and the CE. CE includes a knowledge base, reasoning engine and learning engine, as shown in Figure 3, and its purpose is to reconfigure the software radio to fulfill some objectives.
In CE, the knowledge base stores the current status of the external environment (signal-to-noise ratio, SNR, and BER) and software radio (modulation mode and encoding rate), and at the same time it stores some rules that comprise conditions and operations determining what waveform needs to be selected under what status in order to maximize channel capacity. OSCR aims at a single rather than multiple objectives and has few adjustable waveform parameters, hence, only reasoning on the basis of preset rules (not GA) is required.

   The learning function of the OSCR cognitive engine is realized through
trial-and-error search. According to Reference [8], the capacity of the Additive White Gaussian Noise (AWGN) channel can be figured out with the Shannon formula, and the waveform’s BER can also be figured out theoretically. Therefore, the optimal waveform can be obtained from reasoning. However, as for  non-AWGN channel, neither the channel’s capacity nor the actual performance of various waveforms are obtainable; hence proper waveforms can be concluded only from "learning" rather than reasoning.

   Specifically, the channel capacity is a convex function of waveform’s constellation size, encoding rate and BER. With the increase of constellation size and encoding rate, the BER will increase accordingly, and the channel capacity will not necessarily grow. The lower the encoding rate and constellation size are, the more reduced BER will be, and this will possibly lead to greater channel capacity. BER can be obtained from the feedback of receiver, and then the actual channel capacity of the waveform can be calculated. Through continuous adjustment of waveform parameters and communication performance analysis, the cognitive engine is able to find the waveform with maximal channel capacity.

2 CE Functional Model

2.1 Cognitive Loop
   The functional elements for cognition in CR include environment sensing, reasoning, learning and status control as well, forming a process in the mode of excitation-experience-response. CR executes the loop from Observe to Act continuously, and the cognitive information circulates to form a complete cognitive loop.

 
   To realize the functions of an ideal CE, Mitola defines a cognitive loop that contains the stages of Observe, Orient, Plan, Decide, Act and Learn. These stages can be thought as the response sequence[8] of CE to realize its functions. The cognitive process in CR can be taken as the transfer process of a finite state machine.

   As shown in Figure 4, the loop is started with the Observe stage (observe external environment) and ended with the Act stage, passing through the following two processes[5]:

• The sensing and adaptation loop that is made of Observe, Orient, Plan, Decide and Act;
• The second loop that is called learning loop is the process of finding, forming, using and accumulating knowledge.

   The cognitive loop defines the behaviors and the corresponding functions of CE only, no specific realizations are involved. CE is responsible for realization of different cognitive loops under different conditions. This explains why the abovementioned cognitive engines feature different realization modes to suit different applications. However, they contain the same basic cognitive elements, i.e. reasoning and learning, through which to drive the circulation and repetition of the whole cognitive loop.

2.2 Functional Model
   In view of the CR definition, cognitive loop and functions of CE, the functional model of CE mainly includes the following elements:

   (1) Modeling System
   It is the input part of CE, modeling the information of user domain, wireless (network included) domain and policy domain. That is, it identifies, integrates and abstracts the input information and prior knowledge to form
machine-understandable models. It has the objective modeling, policy modeling, service modeling, network status modeling and wireless environment modeling components. The system involves the Observe stage of the cognitive loop, and performs sensing functions.

   (2) Knowledge Base
   As the core of CE, the knowledge base stores both the long-term knowledge from experiences and temporary status/environment knowledge, and it is the basis for reasoning and learning. All knowledge stored in the base in machine-understandable form.

   (3) Reasoning Engine
   As the "logical thinking" part of CE, the reasoning engine performs Orient, Plan, Decide and Act based on the knowledge in the knowledge base thus realizing the adaptation loop. The adaptation capability requires the reasoning and decision making techniques from artificial intelligence.

   (4) Learning Engine
   As the "creative thinking" part of the cognitive engine, the learning engine can find, form and accumulate knowledge through the feedback of action result, and store the new knowledge to the knowledge base. The machine learning techniques of artificial intelligence is required to realize the learning capability.

   Other necessary elements include various interfaces and resource monitoring modules. They can be implemented on the software radio platform.

   Spectrum management is an important application for CR. Here the policy engine is integrated into the cognitive engine. The spectrum policy inputting and sensing is implemented in the modeling system and the dynamic spectrum policy management and application is decomposed into reasoning and leaning modules.

3 Key Technologies of Cognitive Engine
   The cognitive engine in CR is a functional component separable from the communication system. In fact, the communication system addresses the problem of uncertainty, with the purpose of transmitting or restoring precisely the information carried with the waveforms, and reducing the uncertainty caused by noises and interferences. The cognitive engine, however, addresses the problem of complexity, with the purpose of learning user’s requirement and environment status in light of the data from the sensor, and optimizing the communication system to meet the demands and adapt to the environment. Since artificial intelligence is the strategy and method for addressing the complex problems, the cognitive engine mainly involves the artificial intelligence technologies. The key technologies for CE implementation include knowledge representation, machine reasoning and machine learning.

3.1 Knowledge Representation Technology
   Knowledge representation refers to describing the knowledge and expressing the knowledge in a data structure that can be treated by computers[9]. According to human’s various exploration perspectives and different understanding of problems, the knowledge representation methods can be either declarative or behavioral. The declarative knowledge representation is used to describe factual knowledge, for example, the SNR of a channel, and current working parameters of a radio. The behavioral knowledge representation is used to describe rule-related knowledge and control knowledge, for example, the waveform parameter configuration rule of CR in a specific environment.

   In CE, the knowledge representation technology is used to construct the knowledge base. The CR knowledge base consists of short-term and long-term knowledge. The short-term knowledge means the current status, including external environment status and internal working parameters, while the long-term knowledge includes the bases of rules and cases. The rule base stores the general knowledge related with radio and radio communication, for example, deduction of parameters such as a waveform’s information rate and bandwidth, and estimation of BER. This type of knowledge is used for the reasoning process. The case base stores the experience of CR. On one hand, the case base stores the sensed data of external environment, including the status of local electromagnetic environment and the transmission characteristics of channels. These data are the object for learning and the basis to implement learning. By processing these data, CE is able to obtain helpful experience and knowledge, such as how the local spectrums are used and which channel is usually available. On the other hand, the case base stores the operations used in each communication process and their results as well. Similar to engineers solve problems by drawing on past experiences, or avoid mistakes by learning from past failures. The case base helps CE come up with optimized decisions rapidly.
At present, most description languages, such as Specification and Description Language (SDL), Unified Modeling Language (UML), Interface Definition Language (IDL), and Extensible Markup Language (XML), lack of accuracy and flexibility. Radio Knowledge Representation Language (RKRL), developed by the Royal Swedish Academy of Sciences, and the ontology-based knowledge representation techniques have been studied in depth.

3.2 Machine Reasoning Technology
   Reasoning refers to the process of using the mastered knowledge to deduce implied factual conclusion or induce some new conclusions according to current facts[9]. The reasoning engine, based on the contents of short-term knowledge base, performs matching and selection on the contents in the rule base and case base. The produced interim results then stores back into the short-term knowledge base. The reasoning engine repeats the matching and selection operation on the rule base and case base again and again until a final result is obtained.
As the long-term knowledge, short-term knowledge and reasoning engine are the foundation of the modern rule-based expert system[10], CE is firstly an expert system in the field of radio communication. The reasoning engine uses the knowledge base to perform rule- or case-based reasoning. Currently, the reasoning function is provided in many general environments for developing artificial intelligence or in the expert system shells, such as C Language Integrated Production System (CLIPS) [10] and SOAR[11]. The developers only need to develop the professional knowledge base according with the grammatical definitions to construct an expert system in a specific field. The cognitive engines in Reference [7] and Reference [8] are developed on the basis of SOAR.

   At present, available machine reasoning algorithms are: state space model and search, rule-based system, case-based reasoning, neural network, fuzzy logic, GA and knowledge-based reasoning.

3.3 Machine Learning Technology
   Machine learning is a discipline that studies how to use the computer to simulate the learning activities of humankind. Strictly speaking, it researches the methods with which the computer obtains new knowledge and techniques, identifies current knowledge, to gradually improves the performance and itself [9]. Herbert Simon defines learning as: Any change that makes the system carry out the same task, or another task of the same quantity, in a better way than it does in the previous time. It is the capability of learning from experiences that differentiates CR from other radio; and it is the most important feature of CR.

   To learn different knowledge, CE needs to integrate multiple learning algorithms. Three major algorithms are as follows:

   (1) Supervised learning, used for learning the external environment, that is, to use the measured data to train the estimator. For example, in the situation of sensing channel, CR does not know the specific channel parameters until the communication begins. Hence, CR has to select some parameters based on prediction. Once the communication ends, related channel parameters can be obtained through estimation. CE can use the estimated parameters to train the channel predication model so as to better match the current channel.

   (2) Non-supervised learning, used for learning the external environment, that is, to extract the change characteristics of related environment parameters. For example, in situation of finding the busy/idle state of spectrums, it is required to obtain the temporal and frequency distribution of spectrum holes according to the case base.

   (3) Reinforcement learning, used for learning internal rules or behaviors. It is to use encouragement and punishment to highlight the rules or behaviors that best suit current environment and discard those that do not suit current environment. For example, reinforcement learning can be used to select one waveform that suit current channel conditions from several waveforms.

   At present, available machine learning algorithms are: Bayesian logic, decision tree, Q-learning, time difference method, neural network, game theory and GA.

4 Conclusion
   Cognitive engine is the source of intelligence in CR. Realized on the software radio platform and equipped with artificial intelligence, CE, with good interface definition，can support more the possible CR applications, including dynamic spectrum sharing, dynamic waveform reconfiguration, cooperative and relay communication, cognitive network, and heterogeneous network convergence. With the development of cognitive engine technologies, CR is promised to apply in much wider area in the future.

References
[1] HYKIN S. Cognitive radio: brain-empowered wireless communications [J]. IEEE Journal on Selected Areas in Communications, 2005, 23(2): 201-220.
[2] MITOLA J III, MAGUIRE G Q JR. Cognitive radio: making software radios more personal [J]. IEEE Personal Communications, 1999, 6(4): 13-18.
[3] MAHMOUD Q S. Cognitive networks, towards
self-aware networks [M]. New York, NY, USA: John Wiley & Sons, 2007.
[4] RIESER C J. Biologically inspired cognitive radio engine model utilizing distributed genetic algorithms for secure and robust wireless communications and networking [D]. Blacksburg, VA, USA: Virginia Polytechnic Institute and State University, 2004.
[5] RONDEAU T W. Application of artificial intelligence to wireless communications [D]. Blacksburg, VA, USA: Virginia Polytechnic Institute and State University, 2007.
[6] RONDEAU T W, RIESER C J, LE B, et al. Cognitive radios with genetic algorithms: intelligent control of software defined radios [C]//Proceedings of SDR Forum Technical Conference (SDR’04), Nov 15-18, 2004, Phoenix, AZ, USA. 2004: C3-C8.
[7] STUNTBECK E, O’SHEA T, HECKER J, et al. Architecture for an open-source cognitive
radio [C] //Proceedings of SDR Forum Technical Conference (SDR’06), Nov 13-17, 2004, Orlando, FL. 2006.
[8] CLANCY C, HECKER, STUNTEBECK E, et al. Application of machine learning to cognitive radio networks [J]. IEEE Wireless Communications, 2007, 14(4): 47-52.
[9] 张仰森,黄改娟. 人工智能教程 [M]. 北京: 高等教育出版社, 2008.
[10] GIARRATANO J, RILEY G. 专家系统原理与编程 [M]. 印鉴, 陈忆群, 刘星成, 译. 北京: 机械工业出版社, 2006.
[11] LEHMAN J, LAIRD J, ROSENBLOOM P. A gentle introduction to SOAR, an architecture for human cognition: 2006 update [EB/OL].
http://ai.eecs.umich.edu/soar/sitemaker/docs/misc/ GentleIntroduction-2006.pdf.

   The cognitive engine of OSCR makes decisions based on the long-term knowledge in the knowledge base. The reasoning engine works similarly to an expert system that uses the knowledge base to make intelligent decisions. The leaning engine, however, obtains knowledge from experiences and updates the knowledge base. The knowledge stored by the learning engine will in turn be used by the reasoning engine for subsequent work.

   In this design, the Wireless Channel Genetic Algorithm (WCGA) module models the wireless channel and environment with GA; the Wireless System Genetic Algorithm (WSGA) module generates new waveform with GA; the cognitive system monitoring module consists of a knowledge base. The long-term knowledge in the knowledge base refers to various channels processed by CE and related WSGA initialization parameters, while the short-term knowledge in it includes the cases extracted from the long-term knowledge base that are suitable for current channel. To avoid repeated GA-based optimizations under the same channel condition, a Case-Based Decision (CBD)[6] maker is implemented. That is, if a case already exists in the knowledge base, the past optimization result will be applied directly, otherwise a new optimization process will be performed. The cognitive system monitoring module uses the long-term knowledge to match channels in an integrated way or operate the short-term knowledge in view of application requirements.

   In this general architecture, CR has three input domains. The user domain is responsible for the input of performance requirements for applications and services to CE, such as the QoS requirements of delay and rate. The wireless domain refers to the external propagation environment and Radio Frequency (RF) channel conditions related to radio station transmission and reception. The wireless domain is important for decision optimization and waveform selection. The policy domain is responsible for the input of spectrum allocation policy and entrance permission policy for the location of CR. The policies are explained and managed by policy engine.

[Abstract] Cognitive Radio (CR) is an intelligent radio communication system, whose intelligence mostly comes from the Cognitive Engine (CE). Based on the techniques of software-defined radio and with the support of machine reasoning and learning in artificial intelligence, cognitive engine implements the cognitive loop to realize the abilities of sensing, adaptation and learning in CR. Cognitive engine consists of the modeling system, knowledge base, reasoning engine, learning engine and interfaces. The key techniques are knowledge representation, machine reasoning and machine learning.