For any IT service provider the service is the business, and assuring it is the first line of customer and revenue protection. Until now mid-sized service providers have had no choice but to resort to a multiplicity of tools and consoles to manage and monitor their services from the application down to the underlying infrastructure.  

Some illustrative areas covered by these tools include network monitoring, storage monitoring, log monitoring, system monitoring, virtualization monitoring, application monitoring, database monitoring, VoIP monitoring and others. Despite claims to the contrary by so called suite vendors, no single monitoring system provider can cover all aspects of management, across all technologies and across all layers, at any given time. There is simply too much out there to address and this market is too dynamic for any one vendor to address it all.

While higher order event consolidation frameworks exist, they are highly expensive, take months and years to configure and need armies of staff to maintain. Most mid-size service providers have bypassed using these framework platforms altogether. 

Consequently, mid-size service providers have ended up with a combination of monitoring tools whether they like it or not. These tools produce their own specific IT events and alarms in their own environments, leading to multiple consoles that command centers operators have to inevitably negotiate through inefficient ‘swivel chair’ management.

Furthermore, because of scalability constraints -most of the existing mid market tools apply only a rudimentary logic of IT event elimination and consolidation, and offer even more limited coverage of real-time events.

In some cases, they may simply discard events rather than process them so that these are neither seen, nor controlled by operators – leading to missed alarms that are eventually business impacting. Operators learn about such events when customers call in with a problem.

In other cases, they may easily overwhelm operations teams with non-correlated alarm floods that often lack business context – lengthening the time to prioritization, escalation and ultimately remediation.

In summary, most current monitoring systems in use in mid-size service providers sacrifice accuracy, granularity, and enrichment to operate within a limited scale. While this worked for low velocity of change environments, this is an unacceptable proposition as we enter a new era of computing with increasing demand for intelligence, elasticity and compliance.

In my next post I will discuss a new approach to Real Time IT operations that meets the needs of the mid size Service providers in this time of change.

Over the last few years, the heightened pace of innovation in service delivery technologies has increased the complexity of IT Operations Management for organizations of all sizes. This problem is particularly acute for Small and Medium Enterprises (SME’s) including regional managed service providers – which have traditionally operated with a lean staff and simpler but silo-based management systems. As long as the technologies they managed were relatively isolated and did not change rapidly, this management and monitoring structure worked fine. However, with the rapid advent of new technologies and the increasing pressure to do more with less – the very fabric of enterprise services has undergone a sea change.  

For example, converged communication services today no longer carry only data, but latency-sensitive voice and video packets that require considerable systems and application processing. Virtualized systems have completely overturned the conservative “one application per server” rule that was a costly, but safe choice for many IT managers. Expectations on utilization rates of virtualized servers are now pegged at more than fifty percent, instead of single digit rates.  Return on investment expectations has likewise increased considerably, and focuses on the effective delivery of end-to-end services (like say web apps, voice or video) rather than individual infrastructure availability. And lately, cloud based architectures are making resource allocation and monitoring even more dynamic and distributed.

While all of these changes have helped reign in IT budgets and do more with less, they also have introduced more specialized service components within scalable multi-tier architectures that are far more complex to manage and monitor. Many of these components are dynamic and can be moved or re-allocated based on advanced automated tools – for e.g. the movement of virtual machines across a server farm or entire applications from the enterprises premises to the cloud. Interestingly enough, while the first adopters of virtualization were large enterprises, recent analyst reports predict that x86 virtualization adoption in SME’s will outstrip large enterprise penetration levels in the next few years.

The implication of the structural shift in the nature of service delivery technologies and management is that SME’s will have to contend with vastly different kind of operational challenges going forward.

If you work in an SME or mid tier MSP organization – what kind of challenges do you see? On our part we will try and relate what we hear from our customers on this topic in later posts.

Over the past five years we have witnessed an acceleration of open source software (OSS) adoption by both service provider and enterprises alike. As Gartner stated in a November 2008 report, the primary advantages for customers adopting OSS tools were lower cost of ownership, ease of implementation, investment protection against a single vendor and faster time to market.

These key advantages have held true for the open source industry time and again, enabling it to gain significant market share across the entire IT stack, from operating system to middleware to tools to business applications and across enterprises and service providers (see the discussion on this subject on the microSperience blog). In fact, in the Network and System Management (NSM) space, there is already a wide selection of open source monitoring tools that have gained a broad following.

 And with RiverMuse last year, we have now launched the first and only open source IT Operations Management platform. By our definition, Operations Management includes at its core the disciplines of IT event and fault management. This is the primary platform that operations teams rely upon to keep their services and infrastructure running and healthy 24/7.

The founders at Rivermuse recognized early on that the IT event and fault management market is at the cusp of a major shift. Technology innovation in this market has stagnated for nearly a decade while the infrastructure environment has gone through transformational changes. Existing tools have become overly complex and costly, and return on investment has become questionable. In fact, customers today demand a number of things that legacy vendors are unable to fulfill. These include built-in support for today’s dynamic infrastructures, marked simplification of current management toolsets, significant TCO reduction as required by the new software economics, and the desire of many IT organizations to be liberated from management vendor lock-in. Let’s take a brief look at each of these points.

Most of the legacy operations/fault management platforms were developed to manage network-centric infrastructures that were relatively slow to change. Hence they were built to deal with network-related faults but not much more. Ten years later the entire scenario has completely changed. Today’s infrastructure is abstracted away from the underlying physical assets and governed by policies that relate to business and service priorities. It has evolved to support the need for business agility – new projects can be deployed rapidly, resources can be dialed up or down as needed, and change is easily accommodated throughout a virtualized fabric of computing, storage and networking. The legacy fault management platforms, with their proprietary design, are ill conceived to cope with such a diverse and dynamic environment in an elegant and nimble way.

Second, these management toolsets have extremely rigid architectures built for a different era in computing. Rather than redesigning their products to meet technology and business needs, legacy vendors have responded with bolt-on products from acquisitions or partnerships. Consequently they have multiple user interfaces and programming languages, and do not support standard reference architectures that are needed to fully capture the modern IT infrastructure. This in turn limits their integration and automation capabilities. The number of products and options presented by a legacy vendor to a typical IT organization is a bewildering and convoluted list that only the largest customers can afford. Yet, there is no reason why IT Operations Management platforms can’t be simpler. Vendors like VMWare have delivered simplicity in complex environments of their own. To meet market demands, NSM tools must facilitate agility in IT operations, not get in the way as they often do today and that requires a complete change of attitude and engineering design.

Third, the overall solution cost of legacy operations / fault management tools is prohibitive, particularly in the areas of support and maintenance and ongoing administration. How is an IT organization to achieve ROI on the top of these exorbitant costs? For example, IBM Tivoli Netcool, the most widely deployed enterprise fault management solution requires months of setup, has minimal configuration automation capabilities among other limitations – all contributing to push its TCO through the roof. This is a losing proposition. As more enterprises, Tier 2 and 3 service providers, and managed service providers recognize the need to adopt a new class of IT Operations Management tools, they mandate a level of affordability that only a fully functional open source platform like RiverMuse can deliver.

RiverMuse arrived and will meet market demand because IT organizations have asked for change. The IT world evolves quickly and these organizations simply can’t afford to keep waiting for promised features and integration that never make it past the roadmap slides of many IT vendors. Too much ‘lock-in’ power rests with the legacy vendors. It is time to shift the balance of innovation and control from a few oligopolistic vendors to the many practitioners,  and end users who ultimately know best what IT operations management capabilities they really need and when. It’s time for RiverMuse – and its open source roots to shake up the status quo and move the needle forward.