Big Data Brings Opportunities to Telecom Operators

Cisco has forecasted that between 2011 and 2016 global mobile internet data traffic will increase by a factor of 18 (to 10.8 exabytes per month) and global IP traffic will reach 110.28 exabytes per month by 2016. A greater amount of data requires more data storage, processing, and analysis. Big data technology has thus been introduced to effectively handle data, ensure adequate scalability, and lower deployment costs. It has also been introduced to expand applications of intelligent data analysis. Big data technology allows enterprises to quickly adapt to changes and enhance their competitiveness.

According to Wikipedia, big data includes data sets that cannot be captured, managed, and processed by common software tools within a reasonable timeframe. Big data involves a huge amount and variety of data from different sources that is rapidly fed in and out.

Big data technology is closely connected to cloud computing and is considered an extension of it. The technology ranges from mass data storage and processing to applications. These all involve mass distributed file systems, parallel computing framework, NoSQL database, real-time streaming processing, and intelligent pattern recognition, natural language understanding, and expert knowledge base.

Big data technology brings new opportunities to telecom operators faced with increasingly complex networks. This increased complexity has come as a result of the rapid adoption of smart phones and competition in the internet ecosystem. Operators can use big data technology to improve operational efficiency and network intelligence. In-depth insight into customers and prompt decision-making based on big data can bring value to operators. Operators can also provide results of data analysis to third parties. This creates new market opportunities. Big data applications include business management, such as strategic and competitive analysis; operation analysis, such as user and traffic analysis; network management and maintenance, such as signaling monitoring and network quality analysis; and marketing analysis, such as precision marketing and personalized recommendations. Some typical applications are

●    Network management,maintenance, and optimization

As the amount of data traffic grows, revenue from data services accounts for an increasing proportion of total operator revenue. However, data traffic grows much faster than data revenue. This imbalance has become prominent, and operators have arrived at the consensus that smart pipes and optimized operation is the way out. An important basis for optimized pipes is network management, maintenance and optimization.

Traditional signal monitoring, especially data signal monitoring, is affected by a bottleneck. A telecom operator might produce 1TB of raw packet-switched signal data per day and store it in a file format. After processing, 550 GB xDR signal data is generated per day and is saved in a database format. This data is often saved for a few days or a few months. Handling such a large amount of data is quite difficult for the traditional file system and relational database. Big data technology can address problems associated with rapid data growth, scalability, and high cost. A mass distributed file system can store an almost unlimited amount of data and can expand the data as required. An NoSQL database can handle up to a petabyte of data, and a stream handling and analysis platform can handle huge amounts of event data in real time (Fig. 1).

Intelligent analysis based on big data technology is important in network OAM. Real-time network maintenance is greatly enhanced so that a pre-prevention mechanism is set up. For example, historical traffic data and expert knowledge base are combined to generate an early warning model that identifies abnormal traffic and prevents network congestion or viruses.

●    User behavior analysis

User behavior analysis is important in data traffic operation. By analyzing user profiles, product packages, services, billing, and financial information, operators can obtain precisely control policies. Web pages, messages, pictures and movies, and other traffic delivered through pipes can also be analyzed to better understand user behavior.

Data collection and processing is a significant bottleneck. Fig. 2 shows an operator marketing portal that is built for marketing staff, managers, and technical support staff at all levels. The portal provides daily and monthly statistical reports on data flow, revenue, subscriber development, warnings, and summary tree structure. The amount of data added each month can reach up to 4 TB. Usually, it takes 26 hours to analyze 4 TB of data using a traditional method that is inefficient and cannot adequately deal with system expansion. However, when parallel distributed processing is used, only two hours are needed for analysis and reporting. These technologies allow good system scalability and availability so that deadlines for statistical reports can be met.

●    Personalized recommendations

Operators recommend services and applications according to user preferences. In applications such as Appstore and IPTV, large amounts of structured and unstructured data are handled in real time, and big data technology is key to service-recommendation systems. An IPTV program recommendation system involves analyzing not only user logs, comments, and rating but also relevant videos and comments obtained from the internet using a web crawler. Technologies that can be used for personalized recommendations include parallel computing framework; distributed file systems; text classification, clustering, and correlation algorithm; and intelligent analysis algorithms such as text abstract extraction, semantic analysis, and text mining.

●    Data as a service (DaaS)

The heaviest traffic in mobile internet sector is video data. With the development of social networks, mobile payment, and internet of things, the real economy and virtual world are much more intermingled, and the value of data continues to rise. Telecom operators can obtain valuable information by analyzing user profile, location, terminal status, call record, and web log. The information belongs exclusively to the operators and can be provided to third parties such as SPs/CPs, research institutions, and enterprises. Operators can offer companies and public sector bodies analytical insights based on real-time, location-based data service. Mapping the movement of crowds can also help with city and transportation planning and can help retailers with promotions and choosing store locations. When providing these data services, user privacy protection and legal permission need to be considered.