APM in OpsMgr 2012: for Dev and for Ops

I recently wrote a couple of technical posts about the object model we have chosen for APM in OpsMgr 2012 and how to author granular alerting rules for APM in XML. That’s more the type of post that pertains on the momteam blog.

This one you are reading now, instead, is more “philosophical” than technical – I think that, going forward, I’ll keep more of this distinction by posting my rants here on my personal blog, as they are only partially related to the products and more about my point of view on things. The reasons explained below are just those that I perceive and what drives me – I don’t mean in any way to be speaking on behalf of my company, our strategists or product planners.

I have heard statements from customers such as “AVIcode is a developer tool” or “APM is for QA/Test environments – if you need it in production you have not done your QA work well”and similar statements. People asked why we did bring together the two, for example, on the TechNet forums. Sure, it can be useful to employ such a tool also in a development and QA/test environment… but why not in production? With frequent deployments that the agile business demands, change control alone can’t slow down the business and sometimes bad things happen anyway – so we need solid monitoring to keep an eye on the behavior and performance on the system, exposed in a way that can quickly pinpoint where issues might be – be them in the infrastructure or in the code – in a way that enables people to efficiently triage and resolve them. Sergey points out how APM in OpsMgr 2012 is much easier to setup, simpler to configure and cheaper to maintain than the standalone AVIcode product ever was, and hints at the fact that a comprehensive solution encompassing both “traditional” systems management approach as well as Application Performance Monitoring is a good one. It is a good one, in its simplest form, because we have a simplified, unified and more cost-effective infrastructure. It is a good one – I add – because we can extract a lot of useful information from within the applications, only when those are running; when they are down altogether, APM is not very useful on its own, when it is not complemented by “traditional” OS and platform checks: before I wonder if my application is slow, I’d better ask “is IIS actually up and running? is my application running at all?”. Operations Manager has been historically very good, with its management packs, in answering those questions. APM adds the deep application perspective to it, to provide rich data that Developers and Operations need to have an overall picture of what is going on in their systems and applications.

In my opinion, in this world of continuous services improvement and cloud services, IT management is tearing down the walls between what traditionally has been two separate worlds of “Operations” (Ops) teams and Development (Dev) teams. So, while people ask why we brought what was more of a Developer tool into a pure System Management tool, it is clear to me that those areas are converging, and even other vendors who start from the opposite approach (APM) eventually go “back to the basics” and begin implementing server-level systems management such as showing disk space and CPU utilization, meaning that, whatever your starting point was or has been, everybody wants and feels the need to bring those two worlds and disciplines together.

This line of thoughts has even been given a name: “DevOps”.

What is this DevOps things anyway is one famous post that can be found on the web, where Stephen Nelson-Smith writes:

[…] On most projects I’ve worked on, the project team is split into developers, testers, release managers and sysadmins working in separate silos. From a process perspective this is dreadfully wasteful. It can also lead to a ‘lob it over the wall’ philosophy – problems are passed between business analysts, developers, QA specialists and sysadmins […] The Devops movement is built around a group of people who believe that the application of a combination of appropriate technology and attitude can revolutionize the world of software development and delivery […] these people understand the key point – we’re all on the same side! All of us – developers, testers, managers, DBAs, network technicians, and sysadmins – are all trying to achieve the same thing: the delivery of great quality, reliable software that delivers business benefit to those who commissioned it. […]

DevOps – the war is over if you want it is a presentation by Patrick Debois which I also encourage you to check out, as it is also very evocative thru images:

The War is over if you want it

DevOps – 6 steps for improved collaboration

[…] The DevOps movement is a modern push from the software industry to instill better interaction and productivity between development (Dev) and IT operations (Ops). Instead of throwing applications “over the fence” blindly to operations, a fluid and much more effective DevOps process inserts transparency, efficiency and ownership into the art of developing, releasing and the production use of critical applications. It also binds the two traditionally siloed teams together. […]

Last but not least, 10+ Deploys Per Day: Dev and Ops Cooperation at Flickr (another presentation from a conference) is a real-world example of a large scale web site (Flickr) and how those practices are adopted.

When it comes to the DevOps ideas and concepts within Microsoft products, for what I can see, some customers really “get“ it, and would like to see more in this sense. For example I found this interesting blog post by James Dawson:

[…] The bulk of my work revolves around the Microsoft platform and to put it bluntly it is very much a second class citizen in terms of the available tooling.

Now I’ve fanned the flames, let me put some context around that. I don’t mean that as a criticism, in fact I view the status quo as an entirely natural result given where the movement grew out of and, to be frank, the mindset of the typical Microsoft IT shop. In a Microsoft environment there tends to be far greater reliance on big vendor products, whereas in the Linux/BSD world it is far more common to integrate a series of discrete tools into a complete tool chain that meets the needs for a given scenario. […]

I think James is right when saying this: he “gets” it, but we also have a vast user base of more “traditional” enterprise customers where the concepts have not been digested and understood yet. When it comes to traditional enterprises, what sometimes happens is well explained in this other article by Paul Krill:

[…] To protect the infrastructure, IT ops can put in place processes that seem almost draconian, causing developers to complain that these processes slow them down, says Glenn O’Donnell, an analyst at Forrester Research. Indeed, processes such as ITIL (IT Infrastructure Library) that provide a standardized way of doing things, such as handling change management, can become twisted into bureaucracy for its own sake. But sometimes, people "take a good idea too far, and that happens with ITIL, too." […]

And I think that is exactly one of the reasons why, even if many of our teams “get” it, we need to talk more of the DevOps culture in those places where it hasn’t arrived yet, so that these integrated products are more successful and can help them solve problems – because some of these customers haven’t yet realized that it takes a culture shift before these new tools can be adopted. DevOps does not have critical mass today, but could have it tomorrow. Even Gartner says:

[…] by 2015, DevOps will evolve from a niche strategy employed by large cloud providers into a mainstream strategy employed by 20% of the Global 2000 organizations”. […]

So, back to suggesting that Microsoft produces more of this “goodness”, James again writes:

[…] I want to see the values espoused by DevOps spread far and wide, including the quietest backwaters of corporate IT, where Windows, Office and IE 6 reign supreme. To that end, the Microsoft infrastructure community needs to take a similar approach as the .NET community did and start bringing some of the goodness that we see in the Linux world to the Microsoft platform in a way that facilitates adoption for all and actually takes advantage of the platform’s innate richness and strengths. […]

So do I. And, for what I can tell, we are actually trying to bridge gaps and push the culture shift – integrating APM in OpsMgr is definitely an effort in this direction. But it might take some time. Is it too an “utopian” a vision? I don’t think it is; I think we can get there. But it will take some time. As this other article was saying:

[…] The DevOps approach is so radical it will take some time to cross the chasm, and indeed it will be actively resisted by many organizations where it threatens traditional delivery models and organizational structures. […]

Let’s get Dev and Ops talking to each other, also in the Enteprise! I am all for it.

Disclaimer

The information in this weblog is provided "AS IS" with no warranties, and confers no rights. This weblog does not represent the thoughts, intentions, plans or strategies of my employer. It is solely my own personal opinion. All code samples are provided "AS IS" without warranty of any kind, either express or implied, including but not limited to the implied warranties of merchantability and/or fitness for a particular purpose.
THIS WORK IS NOT ENDORSED AND NOT EVEN CHECKED, AUTHORIZED, SCRUTINIZED NOR APPROVED BY MY EMPLOYER, AND IT ONLY REPRESENT SOMETHING WHICH I’VE DONE IN MY FREE TIME. NO GUARANTEE WHATSOEVER IS GIVEN ON THIS. THE AUTHOR SHALL NOT BE MADE RESPONSIBLE FOR ANY DAMAGE YOU MIGHT INCUR WHEN USING THIS INFORMATION. If you want to see the official info from my employer about the topic above, go to http://www.microsoft.com/presspass/presskits/cloud/default.aspx

Inversely Proportional

Inversely Proportional

Some time ago I was reading www.caffeinatedcoder.com/book-review-the-c-programming-la…

[…] Since a good portion of the C# books are between the 500 and 1000 page range, it was refreshing to read a book that was less than 200 pages. Partly this is because when the book was published the surface area of the reusable API was a small fraction of what it is now. However, I also wonder if there was an expectation of disciplined conciseness in technical writing back in the late 80’s that simply no longer exists today. […]

I think this is a very important point. But then, again, it was no secret – this was written in the Preface to the first edition of that book:

[…] is not a “very high level” language, nor a “big” one, and is not specialized to any particular area of application. But its absence of resrictions and its generality make it more convenient and effective for many tasks than supposedly more powerful languages. […]

I think it all boils down to simplicity, as Glenn Scott says in glennsc.com/start-a-revolution-with-confident-simplicity

[…] To master this technique you need to adopt this mindset that your product is, say, simple and clean, and you just know this, and you are confident and assured of this. There is no urgent need to “prove” anything. […]

Another similar book on a (different) programming language, is “Programming Ruby, the pragmatic programmer’s guide” which starts with

[…] This book is a tutorial and reference for the Ruby programming language. Use Ruby, and you’ll write better code, be more productive, and enjoy programming more. […] As Pragmatic Programmers we’ve tried many, many languages in our search for tools to make our lives easier, for tools to help us do our jobs better. Until now, though, we’d always been frustrated by the languages we were using. […]

Of course that language is simple and sweet, very expressive, and programmers are seen as having to be “pragmatic”. No nonsensical, incredibly complex cathedrals (in the language itself and in the documentation) – but quick and dirty things that just WORK.

But way too often, the size of a book is considered a measure for its quality and depth.
I recently read on Twitter about an upcoming “Programming Windows Phone 7” book that would be more than a thousand pages in size: twitter.com/#!/MicrosoftPress/status/27374650771

I mean: I do understand that there are many API’s to take a look at and the book wants to be comprehensive…but…. do they really think that the sheer *size* of a book (>1000 pages) is an advantage in itself? it might actually scare people away, for how I see things. But it must be me.

In the meantime the book has been released and can be dowloaded from here blogs.msdn.com/b/microsoft_press/archive/2010/10/28/free-…

I have not looked at it yet – when I will have time to take a look at it I’ll be able to judge better…

for now I only incidentally noticed that a quick search for books about programming the iPhone/iPad returns books that are between 250 and 500 pages maximum…

And yet simplicity CAN be known to us, and some teams really “Get it”: take Powershell, for example – it is a refreshing example of this: the official powershell blog has a subtitle of “changing the world, one line at the time” – that’s a strong statement… but in line with the empowerment that simplicity enables. In fact, Bruce Payette’s book “Powershell in Action” is also not huge.
I suppose it must be a coincidence. Or maybe not.

Early Adoptions, Health Checks and New Year Rants.

Generations

Two days ago I read the following Tweet by Hugh MacLeod:

“[…] Early Adopter Problem: How to differentiate from the bandwagon, once the bandwagon starts moving faster than you are […]”

That makes me think of early adoption of a few technologies I have been working with, and how the community around those evolved. For example:

Operations Manager… early adoption meant that I have been working with it since the beta, had posted one of the earliest posts about how to use a script in a Unit Monitor back in may 2007 (the product was released in April 2007 and there was NO documentation back then, so we had to really try to figure out everything…), but someone seems to think it is worth repeating the very same lesson in November 2008, with not a lot of changes, as I wrote here. I don’t mean being rude to Anders… repeating things will surely help the late adopters finding the information they need, of course.

Also, I started playing early with Powershell. I posted my first (and only) cmdlet back in 2006. It was not a lot more than a test for myself to learn how to write one, but that’s just to say that I started playing early with it. I have been using it to automate tasks for example.

Going back to the quote above, everyone gets on the bandwagon posting examples and articles. I had been asked a few times about writing articles on OpsMgr and Powershell usage (for example by www.powershell.it) but I declined, as I was too busy using this knowledge to do stuff for work (where “work” is defined as in “work that pays your mortgage”), rather than seeking personal prestige through articles and blogs. Anyway, that kind of articles are appearing now all over the Internet and the blogosphere now. The above examples made me think of early adoption, and the bandwagon that follows later on… but even as an early adopter, I was never very noisy or visible.

Now, going back to what I do for work, (which I mentioned here and here in the past), I work in the Premier Field Engineering organization of Microsoft Services, which provides Premier services to customers. Microsoft Premier customer have a wide range of Premier agreement features and components that they can use to support their people, improve their processes, and improve the productive use of the Microsoft technology they have purchased. Some of these services we provide are known to the world as “Health Checks”, some as “Risk Assessment Programs” (or, shortly, RAPs). These are basically services where one of our technology experts goes on the customer site and there he uses a custom, private Microsoft tool to gather a huge amount of data from the product we mean to look at (be it SQL, Exchange, AD or anything else….). The Health Check or RAP tool collects the data and outputs a draft of the report that will be delivered to the customer later on, with all the right sections and chapters. This is done so that every report of the same kind will look consistent, even if the engagement is performed by a different engineer in a different part of the world. The engineer will of course analyze the collected data and write recommendations about what is configured properly and/or about what could or should be changed and/or improved in the implementation to make it adhere to Best Practices. To make sure only the right people actually go onsite to do this job we have a strict internal accreditation process that must be followed; only accredited resources that know the product well enough and know exactly how to interpret the data that the tool collects are allowed to use it and to deliver the engagement, and present/write the findings to the customer.

So why am I telling you this here, and how have I been using my early knowledge of OpsMgr and Powershell for ?

I have used that to write the Operations Manager Health Check, of course!

We had a MOM 2005 Health Check already, but since the technology has changed so much, from MOM to OpsMgr, we had to write a completely new tool. Jeff  (the original MOM2005 author, who does not have a blog that I can link to) and me are the main coders of this tool… and the tool itself is A POWERSHELL script. A longish one, of course (7000 lines, more or less), but nothing more than a Powershell script, at the end of the day. There are a few more colleagues that helped shape the features and tested the tool, including Kevin Holman. Some of the database queries on Kevin’s blog are in fact what we use to extract some of the data (beware that some of those queries have recently been updated, in case you saved them and using your local copy!), while some other information are using internal and/or custom queries. Some other times we use OpsMgr cmdlets or go to the SDK service, but a lot of times we query the database directly (we really should use the SDK all the times, but for certain stuff direct database access is way faster). It took most of the past year to write it, test it, troubleshoot it, fix it, and deliver the first engagements as “beta” to some customers to help iron out the process… and now the delivery is available! If a year seems like a long time, you have to consider this is all work that gets done next to what we all have to normally do with customers, not replacing it (i.e. I am not free to sit on my butt all day and just write the tool… I still have to deliver services to customers day in day out, in the meantime).

Occasionally, during this past calendar year, that is approaching its end, I have been willing and have found some extra time to disclose some bits and pieces, techniques and prototypes of how to use Powershell and OpsMgr together, such as innovative ways to use Powershell in OpsMgr against beta features, but in general most of my early adopter’s investment went into the private tool for this engagement, and that is one of the reasons I couldn’t blog or write much about it, being it Microsoft Intellectual Property.

But it is also true that I did not care to write other stuff when I considered it too easy or it could be found in the documentation. I like writing of ideas, thoughts, rants OR things that I discover and that are not well documented at the time I study them… so when I figure out things I might like leaving a trail for some to follow. But I am not here to spoon feed people like some in the bandwagon are doing. Now the bandwagon is busy blogging and writing continuously about some aspect of OpsMgr (known or unknown, documented or not), and the answer to the original question of Hugh is, in my opinion, that it does not really matter what the bandwagon is doing right now. I was never here to do the same thing. I think that is my differentiator. I am not saying that what a bunch of colleagues and enthusiasts is doing is not useful: blogging and writing about various things they experiment with is interesting and it will be useful to people. But blogs are useful until a certain limit. I think that blogs are best suited for conversations and thoughts (rather than for “howto’s”), and what I would love to see instead is: less marketing hype when new versions are announced and more real, official documentation.

But I think I should stop caring about what the bandwagon is doing, because that’s just another ego trip at the end of the day. What I should more sensibly do, would be listening to my horoscope instead:

[…] “How do you slay the dragon?” journalist Bill Moyers asked mythologist Joseph Campbell in an interview. By “dragon,” he was referring to the dangerous beast that symbolizes the most unripe and uncontrollable part of each of our lives. In reply to Moyers, Campbell didn’t suggest that you become a master warrior, nor did he recommend that you cultivate high levels of sleek, savage anger. “Follow your bliss,” he said simply. Personally, I don’t know if that’s enough to slay the dragon — I’m inclined to believe that you also have to take some defensive measures — but it’s definitely worth an extended experiment. Would you consider trying that in 2009? […]