perfSONAR Architecture: Design, Usage, Extension and Next Steps

  • Published on

  • View

  • Download


perfSONAR Architecture: Design, Usage, Extension and Next Steps. Presented by Prof. Martin Swany University of Delaware / Internet2 05 August, 2008 26 th APAN Conference Queenstown, New Zealand. What is perfSONAR. A collaboration - PowerPoint PPT Presentation


  • perfSONAR Architecture: Design, Usage, Extension and Next StepsPresented by Prof. Martin SwanyUniversity of Delaware / Internet2

    05 August, 200826th APAN ConferenceQueenstown, New Zealand

  • What is perfSONARA collaborationProduction network operators focused on designing and building tools that they will deploy and use on their networks to provide monitoring and diagnostic capabilities to themselves and their user communities.An architecture & a set of protocolsWeb Services ArchitectureProtocols based on the Open Grid Forum Network Measurement Working Group SchemataSeveral interoperable software implementationsJava, Perl, PythonA Deployed Measurement infrastructure

  • perfSONAR ArchitectureInteroperable network measurement middleware (SOA):ModularWeb services-basedDecentralizedLocally controlledIntegrates:Network measurement tools and data archivesData manipulationInformation ServicesDiscoveryTopologyAuthentication and authorizationBased on:Open Grid Forum Network Measurement Working Group schemaCurrently attempting to formalize specification of perfSONAR protocols in a new OGF WG (NMC)Network topology description being defined in the Network Markup Language WG

  • Decouple 3 phases of a Measurement Infrastructure

    Analysis & Visualization

    Measurement Infrastructure

    Data Collection

    Performance Tools

    Analysis & Visualization

    Measurement Infrastructure



  • perfSONAR ServicesMeasurement Point (MP) ServiceEnables the initiation of performance testsMeasurement Archive (MA) ServiceStores and publishes performance monitoring resultsTransformation ServiceTransform the data (aggregation, concatenation, correlation, translation, etc)Resource protectorArbitrate the consumption of limited resourcesOther services delegate a limited portion of the authorization decision here

    These services are specifically concerned with the job of network performance measurement and analysis

  • *(perfSONAR) Information ServicesLookup ServiceAllows the client to discover the existing services and other LS services.Dynamic: services registration themselves to the LS and mention their capabilities, they can also leave or be removed if a service goes down.Topology ServiceMake the network topology information available to the framework.Find the closest MP, provide topology information for visualisation toolsAuthentication ServiceBased on Existing efforts: Internet2 MAT, GN2-JRA5Authentication & Authorization functionality for the frameworkUsers can have several roles, the authorization is done based on the user role.Trust relationship between networks

    These services are the infrastructure concerned with discovering federating the available network services

  • Where is link utilization for - IPs d,e,f?Example perfSonar client interaction

    ClientNetwork ANetwork BLS ALS BMA AMA BabcdefWhere is link utilization for IPs a,b,c?a,b,c : Network A, MA AGet link utilization d,e,fHere you goGet Link utilization a,b,cHere you goUseful graphgLSWhere can I get more about networkDoman B/IP d,e,f and Domain A/IP a,b,c?LS A, LS Bd,e,f : Network B, MA B

  • *perfSONAR works E2E when All Networks ParticipateFNAL (AS3152)[US]ESnet (AS293)[US]GEANT (AS20965)[Europe]DFN (AS680)[Germany]DESY (AS1754)[Germany]measurement archivemeasurement archivemeasurement archivemeasurement archivemeasurement archiveperformance GUIuserAnalysis toolMany collaborations are inherently multi-domain, so for an end-to-end monitoring tool to work everyone must participate in the monitoring infrastructure

  • perfSONAR Architecture OverviewperfSONAR schema and protocolUnderstanding the structure of the schema is informative when discussing the architecture and extensibility mechanisms Information ServicesThe Lookup Service and Topology Service utilize the base model and provide system glue that allows for service discovery and contextualization of both data and servicesExtensibility and ongoing work

  • perfSONAR SchemaKey Goals: Extensibility, Normalization, ReadabilityBreak representation of performance measurements down into basic elementsData and MetadataMeasurement Data A set of of measurement events that have some value or values at a particular timeMeasurement MetadataThe details about the set of measurement data

  • Schema NormalizationCan simplify the database representation for many types of measurement dataWhile optimizations are possible, many measurement types can be viewed as one value measured over timeAssists Combination/Concatenation of metricsCreating derived metricsNormalization helps with inferring relationships between types of metrics

  • Schema Basic Elements - MetadataSubject (Noun)The measured/tested entityEventType (Verb)What type of measurement, value, or event occurredCharacteristic, tool output, or generic eventParameters (Adjectives and Adverbs)How, or under what conditions, did this event occur?

  • Schema Basic Elements - DataSome sort of value - DatumExistence of an event might point to the case where there no additional valueAs in Link up/down or threshold eventsTimeIs extensible since various representations are appropriate in different casesE.g. UNIX timestamp vs NTP time

  • A MessageMessageMessage

  • An Object StoreStore

  • A Data is Linked to a MetadataMetadata



  • A Metadata may be linked to anotherMetadata



  • Schema NamespacesAll measurements have some sort of Data and TimeAll measurements can be described by the Metadata identifying who, what and howThe specific structures of the Data and Metadata elements depend on the measurementApproach: Consistently use Data and Metadata elements and vary the namespaces of the specific elements

  • Schema Namespaces - 2We encode the measurement/event type in the namespaceAnd as a standalone elementSome components of the system can pass Data and Metadata elements through without understanding their specific structureAllows and implementation to decide whether it supports a particular type of data or notAllows validation based on extended (namespace-specific) schemata

  • Schema Namespaces and ExtensibilityOne key to extensibility is the use of hierarchy with delegationSimilar to OIDs in the IETF management worldThe NM-WG has a hierarchy of network characteristicsGood starting pointHowever, not all tools are cleanly mapped onto the Characteristic spaceOften a matter of some debate

  • Schema Namespaces and Extensibility - 2Organization-rooted tools namespace addresses thisSome top-level toolsping, tracerouteEasy to add new tools in organization-specific namespacesPerformance Event RepositoryAdd a schema and get a URIAdd Java classes

  • Linking MetadataMetadata can be linked in series in two waysMerge chaining allows for elements to be reused and a complete metadata can be builtOperation chaining requests or describes operations on data sets


  • Operation MetadataFunctions applied to the data have URI-based namesCommon parametersIdeally, a series of these metadata elements can completely describe the provenance of any resultant datasetAs well as requesting selection and reduction operations at query time

  • Topology SchemaTopology schema grew from network measurement descriptionReusable Subject elements for common casesAlso reduces redundancy Relationships between measurement SubjectsSame basic structure at all layersNetworks are graphsDefine:NodePort (Interface)LinkDomainNetworkPathService

  • Topology

  • Topology - Recursive Links

  • Topology SchemaStructured by layers and the same elements recurring thereVaried by namespacesReuse visualization logic, etc.Validate layer- or technology-specific attributes4 Layers: Base (both abstract and L1), L2, L3, L4Also technology-specific layers like Ethernet, SONET/SDH

  • Relationships between Subjects To completely capture the relationships, we need to do a few more thingsRecursive definition of linksLogical links consist of physical linksOrdered lists of links - PathLike above, but we need to introduce an Index attributeNetworksPhysically consist of links but that is not always the most convenient logical viewSpecial element to which Interfaces or Links belong

  • Relationships between SubjectsElements at the same layer have relationshipsA link references two ports/interfacesAt Layer2 or Layer3Elements of the same sort have relationships between themselves at different layersA Layer 1 Interface (physical NIC) can have one or more Layer 2 Interfaces, which can each have one or more Layer 3 InterfacesNode is special Since a Node doesnt really have any higher-layer characteristic independent of its Interfaces

  • Schema - Network Element IdentifiersA scheme for identifying network elementsEach network element gets a unique identifierThis identifier will be included with any measurement associated with that element.

  • Network Element IdentifiersUse Cases:A topology service can be used to find the identifier for a network elementAn LS could then be queried to find all measurements associated with that elementDynamic service path-finding can be integrated with ongoing measurements

  • Network Element IdentifiersIdentifiers use URN notationPrefixed with urn:ogf:network:Consists of name/value pairs separated by colonsPossible field names: domain, node, port, link, path, networkSet of rules defined for each field to keep identifiers compact and finiteReferred to as NURNs

  • Network Element to ATLA OC192urn:ogf:network:path=anna-11537-176

  • perfSONAR ProtocolUtilizes the basic message containerWith type attributeContains various data and metadata elementsUses a message-level parameter element to communicate message-level optionsMessages return a result datum with a type system identical to that of a measurement datum

  • perfSONAR Information ServicesThe Lookup Service and the Topology Service comprise the perfSONAR ISTopology and Service Lookup information are linked with references and NURNsNot necessarily a single instance of each information serviceThese services are being utilized outside the measurement world

  • perfSONAR Lookup ServiceDirectory service of perfSONAR deploymentsAccept service registrationsHandles queries for service location and capabilities and location of available dataManage the lifetimes of data and services to keep information up to dateWeb Service interface to XML DatabaseSleepycat/eXist XML DatabasesService Info/Data kept in native formatsDraw away complex query tasks from otherwise 'busy' services

  • Lookup ServiceAlso use an XML based configuration/protocolNative storage/query mechanisms [Xpath/XQuery]Uses LS-specific messagesTargeted at single domain deploymentSingle instance to manage multiple servicesClient components and applications use the LS to find services

  • perfSONAR Topology ServiceProvides a queryable repository for obtaining topology information about a domainCan obtain the entire networkXquery interface allows the construction of complex queries about the networkTopology is specified according to the topology schemaVarious Uses as a Subject repository for measurement informationTopology discovery for dynamic circuit pathfinding

  • Distributed/Global Lookup ServiceFederation of individual LS instances into a global systemMeta-lookup phase allows a query to find the specific LS that has relevant informationOr perhaps the relevant LSes that have said infoThe specific query is sent directly to the LS in question

  • Distributed Lookup ServiceService and measurement metadata is summarized for propagation to distant domainsIP addresses in service and measurement metadata are compressed into network/netmask pairs in the same way that routes are advertised (CIDR-style)These summarized metadata elements are advertised to external scopesA scope is a set of LSes that are related by e.g. being in the same administrative domain (although multiple scopes within a single domain are possible)

  • Global Lookup ServiceCurrently deployed solution involves a static set of root Lookup ServicesThis works very much like DNS with peer to peer information propagationSummarization is based on the stored eventTypes and serviceTypes

  • Using perfSONARHow do I share my data using perfSONAR?perfSONAR MAs can publish data gathered with MRTG, Cacti Download one or more distributionsMDM Release from GEANT2Most services in JavaAvailable as RPMsperfSONAR-PSServices in PerlAvailable as RPMs, via CPAN, Knoppix live cdDownload various services a la carteRegister with the Global Lookup Service

  • perfSONAR ServicesMeasurement Archive (MA) ServicesBoth RRD and SQL, java and perlFlowSA MA (java)HADES MA (perl)Layer2 Circuit Status MA (java and perl)perfSONAR-BUOYMeasurement Point (MP) ServicesCLI MPSNMP MPTelnet/SSH MPFlow SubscriptionBWCTLE2EMonLookup/Topology/Authentication Services

  • Extending perfSONARHow can I extend perfSONAR?Definition of metric schemaIf you are publishing a new type of data, schema definition is the first stepReuse or re-implement protocol processingExamples in Java and PerlRegister with the Lookup ServiceDefining a new service typeAnalysis modules are extended by assigning a URI, defining parameters

  • Ongoing WorkDeployment and refinement of the global LSRecently funded by the US National Science FoundationDeployment of (live CD) appliancesInitially to support LHC networkingIntegration with dynamic circuit networks like Internet2s DCN, GEANT2s AutoBAHN and ESnets SDNLeverage common components for dynamic, visible network

  • Joining perfSONARHow can I join this effort?Join the mailing lists at perfsonar.netParticipate in OGF working groupsParticipate in Internet2 working groupsLet us know that you are interested!

  • AcknowledgmentsGRNETHEAnetIndiana UniversityInternet2ISTFPOZNANRedIRISRenaterRNPSLACSURFnetSWITCHUNINETTUniversity of Delaware

    ARNESBELNETCARNETCESNETCYNETDANTEDFNESnetFCCNFermilabGARRGEANTGeorgia Institute of TechnologyUS National Science Foundation, OCI-0721902Collaborators:

  • endQuestions?Visit

    *To address our goals and vision, we have started (along with others) the perfSONAR project***The services register themselves to the LS and mention their capabilities. They can also leave or be removed from a LS if a service gets down (keepalives).

    *The services register themselves to the LS and mention their capabilities. They can also leave or be removed from a LS if a service gets down (keepalives).



View more >