Big Data - Data defined Big data is a buzzword and a vague term, but at the same time an obsession Big data is an evolving term that describes any voluminous amount of

  • Published on
    24-Mar-2018

  • View
    212

  • Download
    0

Transcript

  • Big Data

    Paula Santos

    paula.santos@nav.pt

    Example of SUR Data Application

  • Summary

    What is Big Data?

    Where is Big Data?

    Why use Big Data?

    How is it used (usable)?

    Brussels 7th December 2017 paula.santos@nav.pt 2

  • Big Data defined

    Big data is a buzzword and a "vague term",

    but at the same time an "obsession"

    Big data is an evolving term that describes any voluminous amount of structured, semistructured and unstructured data that has the potential to be mined for information.

    Big Data = extremely large data sets that may be analyzed computationally to reveal patterns, trends, and associations, especially relating to human behavior and interactions.

    Big Data represents the Information assets characterized by such a High Volume, Velocity and Variety to require specific Technology and Analytical Methods for its transformation into Value

    Brussels 7th December 2017 paula.santos@nav.pt 3

  • Big Data defined

    From Wikipedia:

    Big data usually includes data sets with sizes beyond the ability of commonly used software tools to capture, curate, manage, and process data within a tolerable elapsed time.

    size" is a constantly moving target, as of 2012 ranging from a few dozen terabytes to many petabytes of data. (1 PB = 1000000000000000B = 1015bytes = 1000 terabytes).

    Gartners definition

    "Big Data is high-volume, high-velocity and/or high-variety information assets that demand cost-effective, innovative forms of information processing that enable enhanced insight, decision making, and process automation.

    Brussels 7th December 2017 paula.santos@nav.pt 4

    Is Surveillance data Big Data?

  • Big Data Where?

    Existing open surveillance data sources (live data):

    https://www.flightradar24.com/

    https://flightaware.com/live/

    https://www.radarbox24.com/

    https://global.adsbexchange.com/VirtualRadar/desktop.html

    Is this data stored, kept for analysis?

    Brussels 7th December 2017 paula.santos@nav.pt 5

    https://www.flightradar24.com/https://www.flightradar24.com/https://flightaware.com/live/https://flightaware.com/live/https://www.radarbox24.com/https://www.radarbox24.com/https://global.adsbexchange.com/VirtualRadar/desktop.htmlhttps://global.adsbexchange.com/VirtualRadar/desktop.html

  • Big Data Where?

    Is this data stored, kept for analysis?

    https://opensky-network.org/

    https://www.adsbexchange.com/

    Brussels 7th December 2017 paula.santos@nav.pt 6

    https://opensky-network.org/https://opensky-network.org/https://opensky-network.org/https://opensky-network.org/https://www.adsbexchange.com/https://www.adsbexchange.com/https://www.adsbexchange.com/https://www.adsbexchange.com/

  • Big Data Why?

    Tracking is an area of interest for several domains: taxis, crime,

    There is recent progress in this area with new algorithms for Trajectory processing.

    Trajectory pre-processing

    Trajectory data mining uncertainty, classification, anomaly detection,

    Trajectory pattern mining

    Graphs, Matrixes and Tensors for trajectories

    Brussels 7th December 2017 paula.santos@nav.pt 7

    How can the Surveillance services products benefit from this research?

  • Big Data Why use it?

    What is the service provided by an ANSP?

    Expedite and maintain a safe and orderly flow of traffic

    How can this service be evaluated?

    Via the unwanted events , e.g. separation infringements

    Checking parts of the system against target values, e.g. Probability of detection.

    Other ways?

    Brussels 7th December 2017 paula.santos@nav.pt 8

  • Big Data Why use it?

    What are the outputs of the ANSP service?

    Instructions

    Position & Guidance information (from Navigation Aids)

    Publications

    Where can the results of an effective service be seen?

    Brussels 7th December 2017 paula.santos@nav.pt 9

  • Surveillance data Used

    ATM seminar Lisbon June 2015

    Topological characteristics of air traffic situation Hongyong Wang, Ruiying Wen, Yifei Zhao - Civil Aviation University of China http://www.atmseminar.org/seminarContent/seminar11/papers/358-WANG_0111150846-Final-Paper-7-5-15.pdf

    Air traffic is complex and changeable!

    How difficult/complex does a given air traffic situation look like?

    Air traffic situation can be mapped into a network structure, whose structural characteristics

    are significant for describing the air traffic complexity in a sector.

    The results show that network node degree can distinguish the key aircraft in the sector; the network connection rate reflects the proximity of aircraft; the clustering coefficient identifies the presence of high-density aircraft group; the network structure entropy reflects the homogeneity of aircraft node degrees.

    Brussels 7th December 2017 paula.santos@nav.pt 10

    How can traffic complexity be measured?

    http://www.atmseminar.org/seminarContent/seminar11/papers/358-WANG_0111150846-Final-Paper-7-5-15.pdfhttp://www.atmseminar.org/seminarContent/seminar11/papers/358-WANG_0111150846-Final-Paper-7-5-15.pdfhttp://www.atmseminar.org/seminarContent/seminar11/papers/358-WANG_0111150846-Final-Paper-7-5-15.pdfhttp://www.atmseminar.org/seminarContent/seminar11/papers/358-WANG_0111150846-Final-Paper-7-5-15.pdfhttp://www.atmseminar.org/seminarContent/seminar11/papers/358-WANG_0111150846-Final-Paper-7-5-15.pdfhttp://www.atmseminar.org/seminarContent/seminar11/papers/358-WANG_0111150846-Final-Paper-7-5-15.pdfhttp://www.atmseminar.org/seminarContent/seminar11/papers/358-WANG_0111150846-Final-Paper-7-5-15.pdfhttp://www.atmseminar.org/seminarContent/seminar11/papers/358-WANG_0111150846-Final-Paper-7-5-15.pdfhttp://www.atmseminar.org/seminarContent/seminar11/papers/358-WANG_0111150846-Final-Paper-7-5-15.pdfhttp://www.atmseminar.org/seminarContent/seminar11/papers/358-WANG_0111150846-Final-Paper-7-5-15.pdfhttp://www.atmseminar.org/seminarContent/seminar11/papers/358-WANG_0111150846-Final-Paper-7-5-15.pdfhttp://www.atmseminar.org/seminarContent/seminar11/papers/358-WANG_0111150846-Final-Paper-7-5-15.pdfhttp://www.atmseminar.org/seminarContent/seminar11/papers/358-WANG_0111150846-Final-Paper-7-5-15.pdfhttp://www.atmseminar.org/seminarContent/seminar11/papers/358-WANG_0111150846-Final-Paper-7-5-15.pdf

  • Surveillance data Used

    ASMT Automatic Safety Monitoring Tool (From EUROCONTROL)

    Inputs: Surveillance data, Flight Plans, Alarms (STCA, APW, ACAS-RA)

    Output: Safety occurrences

    Proximity (STCA)

    Rate of closure (STCA)

    Airspace penetration (APW)

    Altitude deviation (CLAM)

    Used for

    Safety monitoring

    Safety analysis (hot spots)

    Surveillance evaluation

    Tool evaluation (Safety nets)

    Brussels 7th December 2017 paula.santos@nav.pt 11

  • Surveillance data Used

    SASS-C Surveillance Analysis Support System for ATC-Centre

    Inputs: Surveillance data, Surveillance infrastructure description

    Output: Sensor and tracker performance metrics

    Probability of detection

    Accuracy

    False targets

    Used for

    Sensor evaluation

    Tracker evaluation

    Event analysis

    Compliance demonstration (SPI-IR)

    Brussels 7th December 2017 paula.santos@nav.pt 12

    How can SASS-C profit from Big Data research?

  • Potential usage

    It is in the air that the QoS of an ANSP can be measured.

    What could be measured?

    Service performance metrics to be researched

    Traffic complexity

    Impact of changes, e.g. improved service performance

    Impact of unexpected events, e.g. war breakout

    Surveillance infrastructure performance

    Requirements:

    Efficient processing - innovative forms of information processing

    New analytical methods (to show the data and analysis results)

    Brussels 7th December 2017 paula.santos@nav.pt 13

    Surveillance data is the basis for these measures.

  • Questions

    Brussels 7th December 2017 paula.santos@nav.pt 14

Recommended

View more >