Database and bigdata processing system for analysis of AIS messages in the netBaltic research project

A specialized database and a software tool for graphical and numerical presentation of maritime measurement results has been designed and implemented as part of the research conducted under the netBaltic project (Internet over the Baltic Sea – the implementation of a multi-system, self-organizing broadband communications network over the sea for enhancing navigation safety through the development of e-navigation services.) The developed software allows tracing graphs of radio-connections between shore stations and vessels (offshore units), based on historical data including the traffic of ships and their specific parameters collected on the Baltic Sea during the last four years. It also enables preparation of data for network simulation experiments using AIS (Automatic Identification of Ships) and GPS (Global Positioning System) loggers installed on shore stations and vessels, taking into account a number of input parameters, such as: time range, coast station selection, ship flags based on MMSI numbers and types and ranges of possible communication technologies used (WiFi, WiMax, Radwin, LTE,etc.). The created tool has a multi-layer architecture that utilizes the Maria DB SQL database, the Apache2 WEB server, and a number of PHP applications. The runtime environment has been built on Linux Debian version 8 and the HP C7000 cluster of the 16 CPU x86_64 architecture. The modularity of the application allows parallel processing and, therefore, optimization of the computing cluster. The database contains more than 70 million records which enables simulation of various topologies (with multi-hop transmissions) and network operations depending on the transmission techniques being used. The database is fully scalable, and allows easy adding of further data collected during subsequent measurement sessions. Additionally, the use of virtualization tools facilitates the future migration to more efficient processing environments, in case of a significant increase in the volume of data. The data recorded in the database allows calculation of statistics for the surveyed networks, and determining the incidence of potential network nodes (e.g. by flag) complete with their available communication techniques – information which is important in determining structures of possible multi-hop networks and their performance. The software finds routes for datagrams according to accepted criteria and exports results to a network traffic simulator, and as such is an important part of the framework used for planning next measurement campaigns and determining which communications equipment would be more suitable for vessels.