Dataset. Commercial ships periodically send out messages containing their position details (ship identifier, latitude, longitude, speed, ...) . These AIS messages are collected and visualized in systems such as MarineTraffic. We have 2.6 TB of AIS messages (TXT/AIVDM format) in the years 2016 and 2017.

S1: Shipping Safety. Find almost-collisions in English channel, visualize on interactive map. Do so by reconstructing ship paths and distance computation in latitude longitude and time dimensions. Take ship information into account, e.g. by getting information by IMO number, and the maneuverability that results from the ship-type.

Summary. The shipping dataset needs some decoding logic which was coded in Java and deployed using Spark. The ship data was subsequently grouped by grid to detect collisions using SparkSQL. Collision detection inside the grid tiles was implemented in Java and again deployed via Spark. This data involves the close-by trajectory and is visualized on a map.

Disclaimer: this visualization seems to be really memory hungry and has hung up many a browser panel.

Data curiosity: ****
Paper writing: ***
Technical difficulties mastered: **
Visualization coolness: **