Introducing ITNM Stitchers

What is a stitcher?

A stitcher is an ITNM process used for the collection and processing of network data. Data collection stitchers are utilised during the initial phases of the discovery process, passing data between finders and discovery agents. Data Processing stitchers build the network topology layers during the final phase of the discovery, ultimately writing the data to the database table scratchTopology.entityByName, from where the data is transferred to the topology database Model by the stitcher SendTopologyToModel.

Stitchers0

Why use a custom Stitcher?

Custom stitchers are generally used to extend the data retrieved by the discovery process, processing the details in the table scratchTopology.entityByName. The stitchers can be used to add devices and links that cannot be discovered or associate customer specific data to devices.

For example, a firewall that cannot be discovered can be added to the topology and links added from connecting devices. Similarly, the location and owner of a device can be extracted from an external database, and associated with a device so it is visible from the TIP.

Basics about Stitchers

Stitchers are located on the ITNM Server, in the directory $NCHOME/precision/disco/stitchers, and have the extension stch. A stitcher can be domain specific, as with all ITNM configuration files, achieved by inserting the ITNM domain name between the filename and the extension, for exampleMyStitcher.My_Domain.stch.

Stitchers may be initiated in a number of different ways:

  • On completion of a discovery agent
  • On completion of a discovery phase
  • On completion of a specified stitcher or stitchers
  • When data is inserted into a specific database table
  • On a time based interval
  • On request from another stitcher

User defined stitchers are text files, and commonly fall into the last category, being initiated from the stitcher PostScratchProcessing. This is the last chance to process the topology before it is sent to Model.

Basic Scratch Topology Schema

To enable an ITNM administrator to develop a new stitcher it is useful to have a basic understanding of the database table scratchTopology.entityByName. This table contains the data that will be transferred to the Model database, and ultimately the NCIM database. It is the NCIM database that TopoViz uses for topology visualisation and where devices are identified for polling.

Each row of the scratch topology table details information on a specific entity, anf includes the field EntityType, an integer that distinguishes between eight potential entity types:

  • 0: Unknown
  • 1: Chassis
  • 2: Interface
  • 3: Logical interface
  • 4: Vlan object
  • 5: Card
  • 6: PSU
  • 7: Subnet
  • 8: Module

Stitchers will often use the field EntityType to restrict which rows it processes. For example, when adding location details it is recommended that this is only attached to a chassis or module, as the other entities are contained entities, i.e. a sub-component of the chassis or module.

The field entityName uniquely identifies the entity and will generally be derived from a hostname. Where hostname resolution is not available the TCP/IP address is likely to appear within the field.

MAC Address and TCP/IP Addresses are detailed in the field Address. This field is of the data type List of Text, and hence it is necessary to use an numeric index to reference a specific value, for example Address(2) for the primary TCP/IP address or Address(1) for the primary MAC address of a chassis.

The ExtraInfo field is used to store information that is specific to an entity type. This field is of data type vblist, where the data is referenced using a string based key, in a similar fashion to a Perl associative array. For example, the field may include additional SNMP information sysLocation, SysUpTime and sysName for a chassis, referenced by the indexes m_sysLocation, m_SysUpTime and m_sysName. To reference a specific value within the list, use the syntax ExtraInfo->IndexName, for example ExtraInfo->m_sysLocation.

Certain entity types are components of other entities, for example an interface is part of a chassis. This relationship is defined within the field UpwardConnections. Don’t confuse this with the field RelatedTo that identifies a list of entities that are connected to the network entity.

Stitchers5

These and the other fields are fully documented within the ITNM 3.8 Discovery Guide.

Querying the Database Table scratchTopology.entityByName

It is useful to familiarise oneself with the data stored within the database table scratchTopology.entityByName before attempting to process it. There are two methods for querying the data, through the OQL command line interface and from the Tivoli Integrated Portal (TIP).

From the TIP, select the navigator option Administration->Network->Management Database Access, as demonstrated in the figure below.

Stitchers1

The administrator must select the relevant ITNM domain, the DISCO service and input the required SQL statement, for example select * from scratchTopology.entityByName where EntityType =1; as demonstrated below.

Stitchers2-big

To run the same query from the OQL command line, execute:

ncp_oql -domain CORE_DOMAIN -service DISCO -username admin -password ‘password’ -latency 120000 -query “select * from scratchTopology.entityByName where EntityType =1;”
Executing query:
select * from scratchTopology.entityByName where EntityType = 1;
.
( 25 record(s) to follow )
{
EntityName=’route2.ccna.com’;
Address=[”,”,’10.201.1.21′];
Description=’Cisco Internetwork Operating System Software ..IOS ™ C2600 Software (C2600-JS-M), Version 12.1(8), RELEASE SOFTWARE (fc1)..Copyright (c) 1986-2001 by cisco Systems, Inc…Compiled Tue 17-Apr-01 05:38 by kellythw’;
EntityType=1;
EntityOID=’1.3.6.1.4.1.9.1.208′;
IsActive=1;
Status=1;
ExtraInfo={
m_IpForwarding=1;
m_SysName=’router2′;
m_DNSName=’route2.ccna.com’;
m_SysLocation=”;
m_SysContact=”;
m_SysUpTime=8744984;
m_SysServices=78;
m_IfNumber=13;
m_OSPF={
m_IsBdrRtr=0;
m_OspfDomain=0;
};
m_AssocAddress=[{m_IfIndex = 1, m_IpAddress = ‘10.201.1.21’, m_Protocol = 1, m_IfOperStatus = 1 }];
m_BaseName=’route2.ccna.com’;
m_AddressSpace=NULL;
m_AccessProtocol=1;
m_Location=’Swansea’;
m_Department=’Sales’;
m_DeviceType=’unknown’;
m_Continent=’unknown’;
m_City=’unknown’;
};
Contains=[‘route2.ccna.com[ 0 [ 1 ] ]’,’route2.ccna.com[ 0 [ 14 ] ]’,’route2.ccna.com[ 0 [ 5 ] ]’,’route2.ccna.com[ 0 [ 6 ] ]’,’route2.ccna.com[ 0 [ 7 ] ]’,’route2.ccna.com[ 0 [ 8 ] ]’,’route2.ccna.com[ 0 [ 9 ] ]’];
}

Adding Basic Data to the ExtraInfo Field

The value of the ExtraInfo field within the example above includes a number of custom details, for example m_Location and m_Department. This information was added by custom stitchers, initiated from the PostScratchTopology stitcher, designed to analyse the subnet of each chassis, and append the data accordingly. To achieve this it is necessary to understand the basic format of a text stitcher.

A stitcher includes two main sections, defining stitcher triggers and stitcher rules. The triggers identify when the stitcher is run, the rules identify the logic of the processing. Hence the shell of a text stitcher is as follows:

//Start Trigger definition
UserDefinedStitcher
{
  //When does this trigger run?
  StitcherTrigger
  {
  //Triggers entered here
  }
  // What does this trigger do?
  StitcherRules
  {
  //Rules entered here
  }
}
//End trigger definition

For this example, the trigger will be initiated from the PostScratchProcessing stitcher, hence the first section is defined as follows:

StitcherTrigger
{
ActOnDemand();
}

The rules section updates the ExtraInfo field, adding the m_Location value for specific subnets:

StitcherRules
{
ExecuteOQL("update scratchTopology.entityByName
set ExtraInfo->m_Location='Swansea' where ((EntityType = 1 or
EntityType = 8) and Address(2) like '10.201.1.');
update scratchTopology.entityByName set
ExtraInfo->m_Location='Cardiff' where ((EntityType = 1 or EntityType =
8) and Address(2) like '10.202.2.');
update scratchTopology.entityByName set
ExtraInfo->m_Location='Llanelli' where ((EntityType = 1 or EntityType
= 8) and Address(2) like '10.203.3.');
update scratchTopology.entityByName set
ExtraInfo->m_Location='Port Eynon' where ((EntityType = 1 or
EntityType = 8) and Address(2) like '172.16.31.');");
}

The extract above demonstrates the ExecuteSQL rule being used to run multiple update commands, each updating the ExtraInfo->m_Location value dependent on the value of the subnet (the where statement). The full stitcher syntax is as follows:

// Stitcher to add the location to a Chassis or module
// based on the TCP/IP subnet
UserDefinedStitcher
{
  //When does this trigger run?
  StitcherTrigger
  {
    ActOnDemand();
  }
  // What does this trigger do?
  StitcherRules
  {
    ExecuteOQL(“update scratchTopology.entityByName set ExtraInfo->m_Location=’Swansea’ where ((EntityType = 1 or EntityType = 8) and Address(2) like ‘10.201.1.’);
    update scratchTopology.entityByName set ExtraInfo->m_Location=’Cardiff’ where ((EntityType = 1 or EntityType = 8) and Address(2) like ‘10.202.2.’);
    update scratchTopology.entityByName set ExtraInfo->m_Location=’Llanelli’ where ((EntityType = 1 or EntityType = 8) and Address(2) like ‘10.203.3.’);
    update scratchTopology.entityByName set ExtraInfo->m_Location=’Port Eynon’ where ((EntityType = 1 or EntityType = 8) and Address(2) like ‘172.16.31.’);”);
  }
}
//End of the trigger

As indicated, this stitcher is designed to be run from the PostSratchProcessing stitcher. This is achieved by creating a domain specific copy of the stitcher, and adding the following line to the end of the file:

ExecuteStitcher("AddLocationFromTCPIPAddress");

This assumes that the custom stitcher is named AddLocationFromTCPIPAddress.domain_name.stch.

The stitcher updates will automatically be loaded, as the stitcher directory is monitored for changes every minute (by default). The additional information will be added during the next discovery. Alternatively, the existing data in the scratchTopology database can be resent to Model by running the command:

ncp_oql -domain <domain_name> -service
disco -username admin -password 'password' -latency 120000 -query
"insert into stitchers.actions values ('SendToModel');"

Mapping Data into the NCIM Database

The stitcher updates discussed so far will transfer the extra information to Model, to the table master.entityByname, but further modifications are required to transfer that information to the NCIM database, and hence make it available to TopoViz for display.

The easiest method for achieving this end is to transfer the data to the pre-defined NCIM database table entityDetails. This is made relatively simple as such mappings can be defined within a Model database table, also called entitiesDetails.

Stitchers3

The Model table entitiesDetails has two fields, entityType and entityDetails, identifying the numeric for the entity type that the map applies to, and a list of the mappings. These additions should be made to the domain specific configuration file DbEntityDetails.domain.cfg in the directory $NCHOME/etc/precision. The following update is required to map ExtraInfo->m_Location to an NCIM field Location:

insert into dbModel.entityDetails
(

EntityType,
EntityDetails
)
values
(

1, -- chassis
{
Location = "eval(text, '&ExtraInfo->m_Location')"
}
);

As the field EntityDetails is of the data type list, its value or values are assigned comma-separated within the curly-brackets, for example:

{
Location = "eval(text, '&ExtraInfo->m_Location')",
Contact = "eval(text, '&ExtraInfo->m_Contact')"
}
N.B.: As of version 3.9, below syntax is valid:

Contact = "eval(text, '&m_ExtraInfo->m_Contact')"

 

Displaying the Extra Information

Once the extra information has been transferred to the NCIM database, the information can be displayed from the TIP, using the structured browser, as demonstrated in the figure below:

Stitchers4-big

Conclusions

This above example is a very basic example of how a stitcher can be used to add additional information to devices for display through the Tivoli Integrated Portal. The example used static information, defined within the stitcher itself, and the use of the information was limited to display through the structured browser. Future articles will discuss more practical examples, where the extra information is derived from a database, is utilised for dynamic view filters and how the data can be added to events inserted into the ObjectServer.

by Nick Lansdowne

Visits: 491