Solarwinds is a great network polling solution, and offers some event collection capabilities. However, googling ‘Solarwinds event correlation’ reveals the product’s shortcomings in the correlation and problem isolation space.

For example, many problems can occur in the IT environment of a business:

• Silent failures: many issues can go unnoticed such as link flapping, spanning tree changes, etc.
• High level of event noise, requiring time and resources to diagnose.
• No conciliation of collected events with performance degradation, and
• No correlation against application performance problems reported by 3rd party tools such as Microsoft SCOM, Hyperic, and CA Wily – essentially no cross-tool correlation.

RiverMuse Pro, our commercial IT operations management solution, accepts any event from any source. That includes Solarwinds Network Performance Manager, Syslog and SNMP traps from network devices, and alarms from other systems and application management tools. RiverMuse Pro addresses the correlation gap in Solarwinds.

A customer of mine recently experienced the scenario of a critical network link flapping. The problem went undetected for 60 minutes. Solarwinds was polling the interfaces every 5 minutes, and pinging for availability every 2 minutes. Coincidentally, the interface happened to be up during each poll. Syslog messages from the devices were noisy, but link down and link up traps were reported. However, given the volume of messages they went unnoticed. Without a real-time operations tool in place, it was not evident that the interface was down or flapping. On the 15-minute mark poll, Solarwinds reported heavy interface errors and packet discards. But the fault was cleared on the subsequent 20-minute mark poll. At the 60 minute mark, the link was down again, which coincided with the Solarwinds poll. It was only then that some smart operators manually sifted through Syslog messages, and determined that the link was flapping.

In contrast, RiverMuse offers a throttle based correlation that can detect link flaps. If 2 or more link down events are reported for the same interface in 3 minutes, RiverMuse can escalate the issue, and generate a summary event indicating the link flap. In addition to that, RiverMuse can perform event priority based correlation. Solarwinds events reporting high packet discards and errors would be correlated as symptoms of the ‘Link flapping’ event.

Lastly, RiverMuse can provide business context and automation on the ‘Link Flapping’ event. A dynamic lookup from an inventory management system would identify the carrier and circuit id. A second lookup from a change management system can provide the escalation contact name and email address. All this information can be viewable in the alert within the RiverMuse console. The escalation contact would also be automatically emailed.

With RiverMuse, a problem that went unnoticed could be detected in real-time. And previously manual procedures would also be automated. Both would safeguard business operation and save time and effort of the operations staff.

Download RiverMuse Pro  free for 30 days and check it out. And if there are additional types of correlations that you would like to set up in your environment, let us know. It would be great to discuss that in our RiverMuse Pro Community.

IT Operations Consoles can be a great asset to an organization, as all pertinent events are consolidated and accessible from a single pane of glass. An effective IT Operations Console also requires comprehensive correlation capabilities. This allows organizations to reduce alarm noise to only those that are meaningful and actionable. It also allows organizations to ensure that operator actions are efficient and impactful in business terms.

RiverMuse Pro offers a broad rules-based correlation engine. One of those rules can perform topology-based correlation by looking up relationships from a CMDB, then performing upstream or downstream suppression. Another rule can lookup procedures from a 3^rd party knowledgebase, and execute them through an external action. Through my years in data center operations, perhaps the 3 most popular correlations include:
-       Suppressing duplicate events
-       Sliding time window based correlation (escalate if X events occur in Y minutes), and last but not least
-       Mapping business impact of outages.

De-duplication has been so commonplace with enterprise-class event management systems that it can easily be taken for granted. But the fact remains: de-duplication is the first and primary method of reducing alarm volumes to manageable levels. Some leading tools in the market lose granularity when de-duplicating events. All they know is the first time the problem occurred, the last time it occurred, and how many times it occurred. RiverMuse addresses this issue with a 2-tier event/alert model. Raw incoming ‘events’ are managed separate from processed de-duplicated ‘alerts’, and reside in separate subsystems. This allows RiverMuse to scale and de-duplicate events without sacrificing granularity.

How many times have we seen a link flapping? Or numerous CPU spikes? The problems are auto-remedied within seconds or minutes, but their frequency hides other problems. User transactions might hang or fail. The 3-hour file transfer had 10 seconds left for completion, only to stop unexpectedly. Deploying a time-series or ‘throttle’ based correlation through RiverMuse Pro can detect these recurring flaps, and escalate the problem(s).

In most IT Operations Consoles, there is zero to little notion of business context. Perhaps the most popular correlation method is determining business impact of an outage. Under the covers, this correlation is based on parent-child relationships. These relationships can be based on a service topology present in a CMDB, or other service management tools. Through the use of dynamic variables, RiverMuse can query external tools, such as CMDBs to lookup impacted business services, and other dependent Configuration Items (CIs). Business impact analysis can then be performed.

RiverMuse offers a wealth of correlation capabilities to address an organization’s business and operational needs. Download RiverMuse available free for 30 days and check it out. And if there are specific type of correlations that you would like to set up in your environment let us know. It would be great to discuss that in our RiverMuse Pro Community.

Until ITIL V3 came to fruition, there was a common misconception that any reported event was an incident, and required a ticket be opened. That could be a CPU threshold exceedance, a process restarting, or a change in network topology. Service Desk vendors drooled like Tasmanian Devils, and some went as far as redefining terms such as Business Service Management.

On the opposite end of the spectrum, many organizations still send email notifications or pager alerts whenever any exception occurs in the monitoring infrastructure. IT administrators receive numerous notifications to the point that many emails are ignored or deleted – since the notifications were too common to be taken seriously. Some infrastructure monitoring tools incorporate a certain level of correlation, but they only correlate the data captured by the tool itself; and often, the correlation capabilities are too narrow or limited. Naturally, these tools cannot correlate information reported by other toolsets. So, the IT administrators are now bombarded with emails from multiple tools, not just one.

With an event management platform in place, correlation first occurs prior to taking any secondary action. Correlation can occur whether the events originated from the same source, or multiple sources. Tickets are then opened on real incidents and problems.  Email notification volumes are dramatically reduced. When an administrator receives an email, he/she can rest assured it’s regarding a real problem. It ensures that those 3 AM calls are actually worth getting up for.

Aside from correlation, an event management platform also acts as a central location to consolidate events across the infrastructure. It is ideal for data center operators since they can see the status of any component in real-time in a single pane of glass. That can include power supplies, HVAC units, servers, virtual machines, the network, the cloud, and virtually any other component. Operators can launch “in context” to other tools once incidents have been correlated to problems. This facilitates troubleshooting, and ensures operators drilldown to the right tool, rather than wasting their time swiveling between multiple tools, and manually trying to correlate problems.

Last but not least, a good event management system also makes business sense out of IT events. In a previous life, I recall instructing operators to ignore a set of messages from 1 – 2 AM due to system maintenance. And that was just for my systems! What about every other application, system, and network device out there? Alert enrichment to the rescue! By looking up the maintenance schedule from a change control system, alerts can be suppressed during maintenance periods, or automatically escalated otherwise. What if escalation contacts are automatically populated into alerts, but the contact information resides in an external inventory management system? Alert enrichment can automate the lookup, and automatically send notifications to the contact. How about populating carrier Circuit ID for ‘link down’ events? Or customers impacted by an outage? Perhaps looking up procedures from a knowledgebase, and automatically executing the verification steps? Alert enrichment allows for such intelligence, which facilitates correlation, automation, and visualization.

RiverMuse, the next generation real-time IT Operations platform enables operational efficiency through event consolidation, alert reduction, and business context enrichment. Rather than swiveling chairs from one tool to another, and looking up procedural information in spreadsheets and other systems, all pertinent information is automatically mapped to alerts. In the past such systems were only afforded by large corporations with large budgets. RiverMuse completely breaks through that barrier by delivering sophisticated event management technology at the right price and ease-of-use that can appeal to mid-market companies too.  Join our RiverMuse Pro Community to discuss your thoughts on IT event management.

RiverMuse PRO includes a Reusable Business Logic (RBL) engine to streamline the creation of all configuration components in a single reusable package.

The configuration is loaded through a text file; it is then parsed and converted by a back-end engine that updates the configuration of multiple components within the RiverMuse product. This is a vast change from traditional Manager of Managers (MoM) solutions as configurations from multiple components are incorporated in a single configuration.

The RBL engine serves to:
– Map organizational processes into the product through one configuration channel
– Fuel the creation of a community driven repository (aka App Store). Configuration packages can be shared or bought. (i.e. a configuration package for grouping events by event type, and performing isolated problem correlation; a configuration package for interpreting Cisco alarms and integrating with Cisco inventory tools to populate RiverMuse dynamic variables).

RiverMuse PRO also incorporates a presence management engine that can discover entities on demand.

This is most useful when new alarms are reported for devices/entities not yet present in a CMDB or inventory management system. A RiverMuse Business Logic Package can first lookup Configuration Item information from the CMDB, and if nothing is found, attempt to perform a discovery using the RiverMuse Presence Management System. This will provide many additional variables that can be leveraged through correlations, automations, and escalations.

RiverMuse PRO includes a centralized rules management wizard.

Whether there are 1 or 50 remotely deployed collectors, rules are configured in a central location through a GUI. Within a traditional Manager of Managers (MoM) solution, the process of obtaining events, performing correlation, and providing business context are usually separate and distinct. Legacy MoM architectures typically require business logic rules to be updated at various levels and multiple components within their system and frequently using different, proprietary scripting languages. This creates a management challenge – and makes it hard for operations teams to keep up with infrastructure shifts.

RiverMuse PRO provides the facility to consolidate your Data Center Operations in a single pane of glass, and achieve Operational Excellence by automating tasks and streamlining processes.

RiverMuse Core, the first enterprise-class open source Real-time Consolidated Operations Console system ideally collects information via SNMP traps and Syslog messages out-of-the-box. Additionally it supports 8 standards-based APIs to obtain data from virtually any source (gSOAP, Perl, Java, C++, XML, PHP, Python, and Ruby on Rails). RiverMuse PRO builds on top of RiverMuse Core and provides a presence management discovery engine, a powerful enterprise desktop console, dynamic alert enrichment from external systems, enhanced scalability, and additional functionalities to streamline organizational processes and dramatically simplify system maintenance.

Perhaps the most compelling reason for pursuing a Consolidated Operations Console solution is being buried in a myriad of tools. And more importantly having little or no business context mapped to the results. The need for a so-called Manager-of-Managers MOM solution becomes more evident the more complex and dynamic an infrastructure becomes – thus requiring various tools to manage and monitor the environment. While multiple monitoring tools are great for specific tasks such as application monitoring, transaction monitoring, or communication device monitoring, problems that affect more than one silo take longer to identify and isolate.

RiverMuse PRO solves this problem by centralizing data across all the different tools, and retrieving events directly from devices when needed. To perform this, RiverMuse PRO includes passive as well as active collectors such as the RiverMuse VMWare agent. This active collector natively interprets CIM (Common Information Model) formatted data streams. Events from different environments are consolidated in one repository, where cross-domain correlation can occur. This allows operations personnel to quickly identify the problem and associated symptoms.

RiverMuse PRO incorporates the event processing scalability of leading commercial Manager-of-Manager solutions without sacrificing granularily.

In other words, all events related to a specific alert are kept in our system and made available on demand including through a launch in context tool. Additionally, correlation can occur against events and alerts. Other leading Manager of Managers tools are also resource intensive and require several install instances to provide value. In contrast, RiverMuse PRO was incorporates a small footprint to curb the overhead and maintenance requirements of legacy MOM solutions without sacrificing elegance and functionality.