Freeware IT monitoring tools are used by thousands of organisation worldwide however using them to monitor complex network, server and application installations can be quite a challenge. This blog post takes the basic capabilities of one such tool, Nagios® Core, and shows how you can scale it with Opsview for use in enterprise environments.

Distributed monitoring

Building and managing a complex distributed monitoring environment with Nagios Core is no mean feat. With Opsview you get distributed monitoring that’s easy to setup and simple to maintain. You can monitor your devices and applications from a central location and grow the system without growing the monitoring complexity.

Slave server clustering

Opsview can automatically load-balance across multiple slaves and reallocate monitoring duties if a slave server fails, giving you high availability and scalability without additional overhead.

Example Clustering Model

Master server clustering

Management of Opsview is performed on a single master server, however master servers can be clustered giving you the high availability and redundancy needed for mission critical monitoring.

Separate database server

Opsview can be run on a separate database server so you can move intensive reporting activity to a dedicated machine and fine tune the server for better performance.

Efficient configuration UI

Nagios Core is capable of monitoring thousands of devices, but maintaining configuration on expanding systems can quickly become a problem. Opsview handles this with an easy to use interface and middleware layer which tackles the complexity of configuring individual software components.

‘Single pane of glass’ monitoring

Unlike Nagios Core where data may be gathered from a number of systems and presented in different ways, Opsview’s intuitive web interface displays all your monitoring information in one place, with a top down view on system status. Devices and applications can be easily grouped by business process and their status displayed using simple ‘traffic lights’ so you can easily see the health of critical and non-critical groups. This makes monitoring and maintaining large, complex systems less time consuming and more efficient with a scalable architecture to cover all your systems and locations.

Opsview's Host Group Hierarchy View

Distributed alerting

Slave servers monitored by Opsview can handle their own notifications, allowing autonomy if communication is lost between master and slave servers. Alerts can be sent by the Master server or slave server by email / sms so you’re always in touch with the health of your system, no matter the location or your systems.

Automated APIs

Opsview APIs speed up system configuration by automatically populating and updating host information saving you time and effort as your system grows.

Example use cases for Opsview's RESTful API

SNMP trap processing

Nagios Core has no native support for SNMP trap processing. Opsview’s SNMP engine accepts incoming traps, analyses the data and decides how to handle them. In-built SNMP discovery allows SNMP objects to be detected and monitored easily and rules can be configured through the management UI.

Notification profiles

With Nagios Core you can be inundated with monitoring information, not all of it useful. In Opsview you can set-up notification profiles so the right people get the right information at the right time. Only want to know about email server status during business hours? No problem. Need SMS alerts about your webstore? It’s covered. Notification profiles can also combined with Opsview’s service desk module to automatically assign support tasks to engineers, helping streamline incident management.

If you’re managing any web sites or web applications Selenium can be used with Opsview’s monitoring platform to ensure your customers get the very best user experience and increase your online conversion rates.

What is Selenium?

Selenium is a suite of tools to automate web application testing, it is also the 34th element in the periodic table. You can use the Selenium IDE to record web browser actions in the form of test cases. These test cases can be saved in a number of different formats and replayed when needed. The Selenium IDE is implemented as a Firefox extension.

What is Opsview?

Opsview provides comprehensive monitoring and management capabilities for a very wide range of web applications. Our Community Edition is free, easy to install and simple to use. More advanced features are available with Opsview Enterprise. Check out our screencasts

User Experience monitoring

In 2001, Zona Research reported that 30% of surfers would abandon a Web site if it took more than 8 seconds to load, creating the widely-quoted ´8-second´ standard. Ten years later, with wide adoption of broadband services users become frustrated if sites take more than a couple of seconds to load.

Monitoring your web applications from an user’s perspective is vital for ensuring they get the best experience and allows you to get early warning of any application performance issues. You can use the Selenium IDE to record common paths through your web applications and the have Opsview replay these paths on a regular basis. Opsview will alert if any URLs fail to load in a pre-defined timescale or if the entire path takes too long to complete. These checks can be replayed from multiple locations (Opsview Slaves) to give you a complete picture of how your customers are experiencing the applications.

End-to-end testing

Using Selenium to perform end-to-end web application testing ensures that the whole system is being monitored in concert. This can be combined with individual component checks to provide a coherent overview of system performance and health. The Selenium IDE can be used to record transactions performed through the web applications. These are then replayed on a regular cycle to ensure the transaction completes successfully in a given timeframe. By replaying key transactions all components and sub-systems that comprise the application are exercised to ensure correct operation.

Selenium in action

Ten uses for Selenium with Opsview

1. Measure how your applications are performing for end-users
2. Keep tabs on your shopping cart / order processing application
3. Check that your landing page forms are working correctly
4. Confirm that web site authentication is working
5. Check that you’re not getting any pesky database errors
6. Track your cloud hosting provider’s performance against their SLA
7. Make sure your boss can get to his web mail account
8. Be the first to know that Opsview’s web UI is offline
9. Ensure your load balancer is working correctly
10. Get alerted if your homepage has been hacked

What next?

Installation and configuration instructions can be found in Opsview’s documentation along with instructions for downloading the selenium plugin.

Our recommended approach is to configure Selenium on an Opsview slave server, either by updating an existing slave or by deploying a new one. Selenium can place an appreciable load on your server so we recommend allocating another 1GB RAM. For heavy Selenium usage you should consider allocating an additional CPU core.

Thanks

Thanks to Alan Wijntje @ Ziggo, Rohit Deshmukh @ Opsview and Steve Burt @ Opsview for their work on the Selenium plugin for Opsview and related documentation.

Hindsight is a wonderful thing.

In hindsight, Opsview would have always had access controls for objects at the role level – since roles also define which parts of the Opsview application you can get to, it would make sense to also put all the host and service objects into this definition.

(In our defence, we wanted to make it as obvious as possible for a contact when you were changing access information.)

The downside of our design decision many years ago is that Opsview administrators who have lots of their users – or contacts in Opsview terms – with the same sort of access and were having to change each user individually. This was painful and error prone if you had 40 “similar” users.

So Opsview Enterprise 3.12.0 now has access information in the role definition. You change access at the role level and it automatically affects all users of that role. Even better, we ensure that a user’s notification profile only has references to the objects they are allowed – change the role definitions and object references will be automatically removed from all aspects of that contact.

Now you can administer users in a much simpler way.

Hindsight – we wish we had more of it.

But the next best thing to hindsight – migratability.

Migratability?

The Opsview configuration database is normalised and models the data as it is. Some application designers like to use key-value pairs to describe their data, which is nice from an expandable point of view, but rubbish when it comes to actually accessing that data in a meaningful way. When it comes to modelling our data, we model it – we don’t “meta-model” it.

We care a lot about people’s data because we build up a trust with our users that they can upgrade and bring their platform right up to date with the latest versions of Opsview. So we spend a lot of time getting the upgrade scripts just right.

If you are upgrading to Opsview 3.12, here’s what the upgrade scripts do:

• for each contact, it will see if this contact’s role has access information applied
• if it hasn’t, it will use this contact’s access information to populate the role
• if the role already has access information (from another contact), then it will compare all the current roles with their access information and if there is a match, this role is used
• if it isn’t exactly the same, then a new role is created called “old role name – contact”

So for example, let’s say you have 4 contacts – John, Paul, George and Ringo – all using the “Liverpool administrators” role.

The first three have their access information the same, but Ringo has missed out the Production – Slicehost Servers host group he should have had access to.

After the upgrade, the first three would still have the role of Liverpool administrators, but Ringo would have the role of a newly created Liverpool administrators – ringo. You could easily tell from this that Ringo should belong to the Liverpool administrators, so you can set him to this. Otherwise, maybe he is different after all, so you can rename the role to Liverpool drummers.

This means that most of the configuration for this new functionality is done for you automatically and you can think of your users in their role groups, rather than individually. Another step to making Opsview easier to use.

We present the 2nd part to our Catalyst advent calendar entry about internationalising your Catalyst application, complete with some really neat scripts such as automatically translating your string using Google! See it here. Enjoy!

We love Catalyst. This perl-based web application framework is at the heart of Opsview’s web interface and is crucial for the REST API we’ve developed. So it is with great pleasure that we’ve written a Catalyst Advent Calendar entry about internationalising your Catalyst application. See it here!

The JNRPE server provides an open source Java implementation of the Nagios Remote Plugin Executor (NRPE). This is much more efficient for performing JMX checks than regular NRPE as you only need to start one JVM rather than a JVM instantiation per check (as performed by check_jmx invoking java -jar JMXQuery.jar).

However this efficiency gain comes with a few compromises, it doesn’t handle composite data as well as other variants of check_jmx and doesn’t support performance data. It would be nice to see some community action to produce a definitive check_jmx.

Luckily with Opsview, you can work around the lack of performance data using the map.local file so you can get performance graphs to assist with correlation or view rate of change over time.

In JNRPE version 0.6.3 there is also an integer overflow bug in the check_jmx base plugin.

The maximum value for a Java Integer is 2^31 – 1 (or 2,147,483,647) – if a number exceeds this, as an Integer is a signed number, it will wrap around to a negative number. For example on a JVM set with a 5GB heap, we saw the following output:

JMX OK - HeapMemoryUsage.used is -2037300184

Amusingly with the initial warning/critical threshold values set too low, this caused the state to flap between OK (when the heap was larger than a 32 bit number and hence negative) and CRITICAL after garbage collection had brought it beneath 2GB.

This has been raised with the JNRPE project as issue 3131380 – though unfortunately having raised it without signing in, I now can’t attach the following simple patch to the parseData method.

Index: CCheckJMX.java
===================================================================
--- CCheckJMX.java (revision 243)
+++ CCheckJMX.java (working copy)
@@ -334,7 +334,7 @@
- private int parseData(Object o)
+ private long parseData(Object o)
{
if (o instanceof Number)
- return ((Number) o).intValue();
+ return ((Number) o).longValue();
else
- return Integer.parseInt(o.toString());
+ return Long.parseLong(o.toString());
}

Since Grails incorporated the testing plugin into core it provides good unit & integration testing support (via the test-app script). There are also additional plugins to support BDD tools (e.g. EasyB) and functional testing (e.g. Canoo WebTest).

One of the useful roles that Hudson fulfils is helping to manage quality. Consequently it has plugins available for most of the popular testing tools (we saw an example of the Violations plugin in part 1 of this series).

We’ll start with the standard unit & integration tests, then add in test coverage and functional tests.

Unit tests Assuming you already have unit/integration tests in your Grails project – we need to add the targets into the Grails builder options. clean compile "test-app -unit -integration"

Notice how the server port has been set to prevent port collision with the servlet container running Hudson.

Tip: you’ll need to set this differently on each job too.

Then set the post-build action:

After a couple of builds (so that there is a trend), Hudson will show:

With blue for tests that passed and red for failed tests. You can also drill into the test results just as with the standard HTML junit reports.

Test coverage As we’re looking to use Hudson to help ensure the quality of the software project, we’ll also inspect the level of test coverage as one of our quality metrics.

Warning: determining test coverage isn’t foolproof and high levels of coverage don’t guarantee good code or good tests. It is possible to achieve a high percentage with poor tests (I’m writing a separate post on this topic).

For this post, we’ll be using Cobertura to check the code coverage. There are other options, but we’ll be using Cobertura because:

1. There is a Grails plugin
2. There is a Hudson plugin too.

Install the Grails code-coverage plugin grails install-plugin code-coverage

You may want to configure additional exclusions in BuildConfig.groovy, e.g. to avoid testing 3rd party code such as a plugin taglib that isn’t used. (the plugin scripts/_Events.groovy includes a default list)

Install the Hudson Cobertura plugin This follows the same process that we covered in part 2 of this series (we’ll omit the screenshots this time):

1. From the main Hudson dashboard, click on the “Manage Hudson” menu item
2. Click on “Manage plugins”
3. Go to the available tab, find & select Cobertura, click on the install button.
4. When the plugin has downloaded and installed, you will need to restart Hudson (you can use the button provided).

Configure your Hudson job There are 2 things that need to be done here:

1. Set up the Grails build action
2. Enable the post-build action for Cobertura

1. This is a case of adding “-coverage -xml” into the build targets within the test-app double quoted section. 2. Enable the Cobertura post-build processing, tell it where the XML report is and set thresholds:

Leverage the information

Watch the trends:

and drill down to see the in-context coverage:

When you have the insight as to where you are missing coverage, then you can take the appropriate action to remedy the situation…

Functional tests There are a number of options for functional testing your application. We’ll cover Canoo WebTest in this post for the following reasons:

1. WebTest is htmlunit based – portable across different environments
2. Canoo have written a Firefox WebTest Recorder add-on – allowing less technical users to create the basis of the tests
3. There is also a WebTest Presenter plugin for Hudson.

Note that as WebTest isn’t browser-based, you may encounter some issues with JavaScript on particular test scenarios.

Install the Grails webtest plugin grails install-plugin webtest If you are using older versions of Grails, you may need to do an interactive plugin install on the Hudson server (the plugin used to download a WebTest bundle, rather than having it as a managed dependency).

Install the Firefox WebTest Recorder add-on You can install the WebTest Recorder in Firefox from: https://www.canoo.com/webtestrecorder-dist/webtestrecorder.xpi

Create some web tests You can come back to this bit later

The process we’ll follow here uses the WebTest Recorder to create the initial test steps.

1. Run “grails create-webtest name” e.g. grails create-webtest HomepageLogin

This plugin script will create us the placeholder test file under test/webtest.

2. grails run-app

3. Bring up Firefox 4. Navigate to the starting page for your test scenario. 5. Enable the WebTest Recorder Sidebar (Tools > Webtest Recorder Sidebar) 6. Click on the Groovy tab.

This should already contain “invoke http://localhost:8080/YourGrailsApp/” (you’ll probably want to trim this down to just a relative path e.g. /YourGrailsApp/)

Using the Webtest Recorder Sidebar, you can click around the site and also easily add verification of content on pages (as a test isn’t a test if you don’t verify that the output is correct for the given input). This can then be placed into a method in the groovy file we created earlier (note: you may need to trim some instructions e.g. where the recorder has created different versions for the same input field). For more information using Webtest, see the manual.

I’d strongly advise you to run your tests locally first before checking them in – in the interests of time/space we’ll assume you’ve done that using: grails test-app -functional

Install the Hudson WebTest presenter plugin This follows the same process as the Cobertura plugin above (see part 2 of this series for screenshots):

1. From the main Hudson dashboard, click on the “Manage Hudson” menu item
2. Click on “Manage plugins”
3. Go to the available tab, find & select WebTest Presenter, click on the install button.
4. When the plugin has downloaded and installed, you will need to restart Hudson (you can use the button provided).

Configure your Hudson job There are 2 things that need to be done here:

1. Set up the Grails build action
2. Enable the post-build action for the WebTest Presenter

1. This is a case of adding “-functional -headless” into the build targets within the test-app double quoted section (The headless option tells Webtest to not run the test monitor nor to load the test results into the browser).

2. The Webtest Presenter post-build action needs enabling and telling where to find the results

After the Hudson job has executed your webtests, you’ll see that these have been included in your overall test count (and may even have improved your test coverage if you’ve followed the Cobertura section above).

If you click onto an individual build, the left hand menu will now have “Webtest Results”

This will give you an overview report:

and a detailed step-by-step report for each test (including resulting pages and any errors):

So there you go, a whistle stop tour of setting up test execution, coverage analysis and result reporting with trends.

Next in the series, we move into using Hudson for continuous deployment of Grails projects.

The first part of a series of posts on Grails and Hudson leading up to a presentation at the London Groovy & Grails User Group. Subsequent instalments will include testing (unit, integration, functional), test coverage, automatic war deployment and monitoring Hudson with Opsview Enterprise.

Grails has a rich plugin eco-system with over 400 hundred plugins – so it’s easy to miss something useful. If you’re serious about software craftsmanship, then using static code analysis tools should be part of your quality regime as it gives further insight into the code base (and if you insist, yes it’ll help with your Technical Debt management).

CodeNarc provides static code analysis for Groovy and the CodeNarc plugin for Grails allows you to perform this analysis with the “grails codenarc” script. Behind the scenes this uses the CodeNarc ant task and settings from grails-app/conf/Config.groovy and produces an HTML report by default.

Until recently, if you used the codenarc target within a continuous integration server such as Hudson – then the HTML report would be generated and sit in the workspace waiting for a diligent developer to check it. You can imagine how often that happens in practice with all the other demands of a project!

However, I’ve now integrated the CodeNarc XML output with the Hudson Violations plugin so that an overview trend line is shown against the Hudson job. Then the team quickly fixed the violations…

You can also get a breakdown by priority:

And in-context views of the violations so you know what to fix:

This is how you do it…

Grails config.groovy codenarc { reportName = 'target/test-reports/CodeNarcReport.xml' reportType = 'xml' // any further settings like maxPriority1Violations=0 }

Hudson

Set up your Grails build step – normally you’d add the ‘codenarc’ target:

The codenarc.properties file can be used to configure specific exclusions, the location of this file can be passed in as a system property (shown above).

You need to have installed the violations plugin (Manage Hudson > Manage Plugins > Available and search for Violations). As this is a recent addition, I’m using a patched version of the Violations plugin, though the patch has been integrated into trunk (I’ll update when it is released). Configure violations:

Note the ‘Faux project path’ – you may need to set this to get an in-context view working properly due to path (e.g. if your code is checked out to workspace/trunk)

I also had a contribution to CodeNarc accepted at the weekend to add aninlineXml report type – this will, with a minor tweak to the CodeNarc parser, allow the Hudson Violations plugin to give the rule description on the pop-up message.

This post is based on research we’ve undertaken to develop a pilot mash-up style, charting dashboard for our monitoring solution, Opsview Enterprise. However the concepts we discuss could be used when building a dashboard that displays information from many other enterprise solutions. The assumption we make is that the reader is familiar with Java and the Google Web Toolkit (GWT). For more information about GWT and the other libraries used in this blog please see the Resources section at the end.

Introduction

The Opsview Enterprise platform is very flexible and allows us to extract monitoring information via a REST based API. This API returns monitoring data as JSON objects or XML so it is easy to integrate with many UI frameworks. In this example we will develop a mini portal-like application that displays status for different host groups in one of its portlets.

The client will be using the ‘GXT’ framework that is based on GWT for the user interface and some Java libraries for HTTP communication and JSON parsing.

Using the REST based API to talk to Opsview Enterprise

Let’s first have a look at how you can call Opsview Enterprise from a browser to collect monitoring data. In this case we can use the following URL to fetch host groups:

http://<hostname>/opsview/api/status/hostgroup

If we try using this URL directly from a browser it will not work, a 403 access denied will be returned even if we login with correct credentials. A couple of extra headers are needed for the authentication to work; we will look at them later on.

To get around this problem we must first login to Opsview Enterprise the normal way via the Opsview Enterprise UI and then hit the URL:

https://<hostname>/opsview/status/hostgroup

This is basically the first URL you get to. Now you will have a session in the browser so you can test the API URL again.

It will return JSON looking something like this:

{"hostgroup":{
"summary":{"handled":782,"unhandled":60,"total":842,
   "service":{"ok":558,"critical":206,"handled":747,"unknown":29,
"unhandled":58,"warning":12,"total":805},
  		   "host":{"handled":35,"unhandled":2,"down":7,"up":30,"total":37}},
"list":[
        {
            "hostgroup_id":"7",
            "hosts":{ "handled":2, "unhandled":0, "down":{"handled":1},
"up":{"handled":1}, "total":2},
"services":{"ok":{"handled":24},"critical":{"handled":21,"unhandled":3},
"handled":50,"highest":"critical","unknown":{"handled":5}, "unhandled":3,"total":53},
            "name":"ProjectX",
            "downtime":null
        },
        {
            "hostgroup_id":"13",
            "hosts":{"handled":2,"unhandled":1,"down":{"handled":2,"unhandled":1},"total":3},
            "services":{"critical":{"handled":73},"handled":86,"highest":"critical",
			"unknown":{"handled":13},"unhandled":0,"total":86},
            "name":"Internal",
            "downtime":"2"
        },
        {
            "hostgroup_id":"14",
            "hosts":{"handled":31,"unhandled":1,"down":{"handled":2,"unhandled":1},
"up":{"handled":29},"total":32},
            "services":{"ok":{"handled":534},"critical":{"handled":66,"unhandled":43},
"handled":611,"highest":"critical","unknown":{"handled":9,
"unhandled":2},"unhandled":55,"warning":{"handled":2,"unhandled":10},
"total":666},
            "name":"Hosting",
            "downtime":"2"
        }
    ]}}

So we can see by looking at it that it is nested JSON and in this case it returned the current status for 3 host groups: ProjectX, Internal, and Hosting.

Because the JSON is nested we will benefit from doing some processing on it before giving it to the UI routines that in GXT which only handles flat JSON.

The Architecture for the Mini-Dashboard Application

The following picture shows an overview of the architecture and the different components that will be used in this solution:

The GXT client will not call the Opsview Enterprise server directly but instead go via a proxy server. This has the benefit that the JSON response from the Opsview Enterprise server can be processed and converted into Serializable Java objects. In this case the JSON is turned into a Java object called HostGroup, which in turn contains a HostStatus object. The HostGroup object will contain enough information to be able to draw a pie chart showing the status for the different host groups.

The HostStatus object will be used when the user clicks on one of the host group pie slices and wants to drill down to see more detailed status for a particular host group. Because we send the HostStatus object at the same time as the HostGroup object we do not have to make another remote call when the user drills down to see individual host group status.

Another benefit of having the proxy server is that it enables us to have the Mini Dashboard client web app and the Opsview Enterprise server web app on different hosts in different domains. If we did not have the proxy server it would not be possible to access content in Opsview Enterprise as for security reasons you cannot open URL connections to a different host to the host from where the client is downloaded. Instead, you would have to install the Dashboard web app on the same host as Opsview Enterprise is running on.

The Mini-Dashboard Proxy Server

Let’s start implementing the proxy server using the Apache Commons HTTP Client and the Jackson JSON parser. For the remote calls between the Mini-Dashboard client and the proxy server (i.e. AJAX based client calls) we will implement a com.google.gwt.user.client.rpc. RemoteService as provided by Google Web Toolkit. It will handle all marshalling of calls for us.

Here is a UML Diagram giving you an overview of the involved proxy server classes:

First setup the OpsviewServerProxyService interface as follows:

/**
  *
  * The client side stub for the Opsview Proxy Service RPC service.
  * @author Martin Bergljung, Opsera Ltd.
  */
@RemoteServiceRelativePath("opsviewproxy")
public interface OpsviewServerProxyService extends RemoteService {
    public static final String SERVICE_NAME = "OpsviewServerProxyService";

    public List getHostGroups();
}

The RemoteServiceRelativePath annotation associates a RemoteService with a relative path. This annotation will cause the client-side proxy to automatically invoke the ServiceDefTarget.setServiceEntryPoint method with GWT.getModuleBaseURL() + value() as its argument. Subsequent calls to ServiceDefTarget.setServiceEntryPoint will override this default path.

We define one method called getHostGroups that will return a list of HostGroup objects. Create a HostGroup object as follows:

/**
 * Host Group (method level javadoc omitted for simplicity).
 *  
 *  {
 *   "hostgroup_id":"14",
 *   "hosts":{"handled":31,"unhandled":1,"down":{"handled":2,"unhandled":1},
 *            "up":{"handled":29},"total":32},
 *   "services":{"ok":{"handled":534},"critical":{"handled":66,"unhandled":43},
 *              "handled":611,"highest":"critical","unknown":{"handled":9,"unhandled":2},
 *              "unhandled":55,"warning":{"handled":2,"unhandled":10},"total":666},
 *   "name":"Internal",
 *   "downtime":"2"
 *   }
 * 
 * @author Martin Bergljung, Opsera Ltd.
 */
public class HostGroup implements Serializable {
    private int m_downtime;
    private String m_hostgroupId;
    private HostStatus m_hostStatus;
    private String m_name;
    private String m_services; // not used

    public HostGroup() {}

    //Getters, Setters, equals, hashCode, and toString have intentionally been left out to save space 

}

Next step is to create an asynchronous version of the interface as AJAX calls are asynchronous by nature. This is the actual interface that the client will come in contact with:

/**
 * The async counterpart of OpsviewServerProxyService.
 */
public interface OpsviewServerProxyServiceAsync {
    public void getHostGroups(AsyncCallback
> callback);
}       

The extra element here is the callback parameter that the client will have to implement. Now let’s move on to the actual implementation of this interface. Create the following class that implements the getHostGroups method:

/**
 * The server side implementation of the Opsview Proxy Server RPC service.
*/
public class OpsviewServerProxyServiceImpl extends RemoteServiceServlet
implements OpsviewServerProxyService {
   public static final String OPSVIEW_API_BASE_URI =
            "https:///opsview/api";
   public static final String OPSVIEW_API_HOSTGROUP_STATUS_URI =
            OPSVIEW_API_BASE_URI + "/status/hostgroup";

   public List getHostGroups() {
        String json = callOpsview(OPSVIEW_API_HOSTGROUP_STATUS_URI);

        if (json != null && json.length() > 0) {
            return readHostGroupsJSON(json);
        }

        return new ArrayList();
   }

Here you need to update the hostname in the OPSVIEW_API_BASE_URI constant to reflect your installation. Also, if you are not using SSL then change to http.

Then add the readHostGroupsJSON method that takes JSON returned from Opsview Enterprise and turns it into a list of HostGroup objects:

    private List readHostGroupsJSON(String json) {
        List hostGroups = new ArrayList();

        try {
            ObjectMapper mapper = new ObjectMapper(); // can reuse, share globally
            JsonNode rootNode = mapper.readValue(json, JsonNode.class);
            JsonNode hostGroupNode = rootNode.path("hostgroup");
            JsonNode listNode = hostGroupNode.path("list");

            for (JsonNode hostGroup : listNode) {
                HostGroup hg = new HostGroup();
                hg.setName(hostGroup.path("name").getTextValue());
                hg.setDowntime(hostGroup.path("downtime").getIntValue());
                String hostGroupId = hostGroup.path("hostgroup_id").getTextValue();
                hg.setHostGroupId(hostGroupId);
                hg.setServices("0"); // not used at the moment

                JsonNode hostStatus = hostGroup.path("hosts");
                HostStatus hs = new HostStatus();
                hs.setHostGroupId(hostGroupId);
                hs.setHandled(hostStatus.path("handled").getIntValue());
                hs.setUnhandled(hostStatus.path("unhandled").getIntValue());
                hs.setTotal(hostStatus.path("total").getIntValue()); // only one needed now

                JsonNode down = hostStatus.path("down");
                JsonNode up = hostStatus.path("up");
                hs.setDownHandled(down.path("handled").getIntValue());
                hs.setDownUnhandled(down.path("unhandled").getIntValue());
                hs.setUpHandled(up.path("handled").getIntValue());
                hs.setUpUnhandled(up.path("unhandled").getIntValue());

                hg.setHosts(hs);
                hostGroups.add(hg);
            }
        } catch (IOException e) {
            System.err.println("Fatal JSON Parsing error: " + e.getMessage());
            e.printStackTrace();
        }

        return hostGroups;
    }

What we do here is use Jackson JSON parser (org.codehaus.jackson) to first lookup the hostgroup node in the JSON response and then with this node we go on to lookup the list node. When we have the list node we can step through all the host groups and setup corresponding HostGroup objects. The HostStatus object is also setup by looking up the hosts node for each hostgroup node.

Here is a snippet of the JSON structure that we are talking about:

{"hostgroup":{
...
"list":[
        {
            "hostgroup_id":"7",
            "hosts":{

Finally we also need to implement the callOpsview method that will use Apache Commons HTTPClient to call Opsview Enterprise. Here is how that is done:

private String callOpsview(String url) {
        String username = ;
        String password =
;

        HttpClient client = new HttpClient();
	// Apache is adding basic authentication in Opsera’s installation so
// the following 2 lines are necessary
        Credentials defaultcreds = new UsernamePasswordCredentials(username, password);
        client.getState().setCredentials(AuthScope.ANY, defaultcreds);
        GetMethod getMethod = new GetMethod(url);
        getMethod.setRequestHeader("X-Username", username);
        getMethod.setRequestHeader("X-Password", password);
        getMethod.setRequestHeader("Accept", "application/json");

        try {
            int statusCode = client.executeMethod(getMethod);
            if (statusCode == HttpStatus.SC_OK) {
                String contents = getMethod.getResponseBodyAsString();
                System.out.println(contents);
                return contents;
            } else {
                System.err.println("Got Error " + statusCode +
                        " when calling Opsview API: " +url);
            }
        } catch (Exception e) {
            System.err.println("Fatal transport error: " + e.getMessage());
            e.printStackTrace();
        } finally {
            // Release the connection.
            getMethod.releaseConnection();
        }

        return "";
    }

When you implement this the username and password need to be specified according to the user account in your Opsview Enterprise installation. Here we can also see that we are setting the following headers:

• X-Username – Username for Opsview user account
• X-Password – Password for Opsview user account
• Accept – indicates that we will accept and handle JSON responses (can also be set to accept XML)

The Opsview Enterprise proxy server implementation also needs to be registered to a URL. We do this in the GWT module definition file as follows for testing purposes:

  <!— Setup the Opsview Server Proxy Service for hosted mode testing -->
    <servlet path="/opsviewproxy" class= 
			"com..server.OpsviewServerProxyServiceImpl" />

And as follows for the real web application, open up web.xml and add:

      <servlet>
      <servlet-name>opsviewServerProxyServiceServlet</servlet-name>
      <servlet-class>com.<your package>.server.OpsviewServerProxyServiceImpl</servlet-class>
    </servlet>

    <servlet-mapping>
      <servlet-name>opsviewServerProxyServiceServlet</servlet-name>
      <url-pattern>/dashboardApp/opsviewproxy</url-pattern>
    </servlet-mapping>

The Mini-Dashboard Client

The client is implemented as Google Web Toolkit application. So first create the main entry point / class for the application:

public class DashboardApp implements EntryPoint {

    public void onModuleLoad() {
        OpsviewServerProxyServiceAsync service = (OpsviewServerProxyServiceAsync)
                GWT.create(OpsviewServerProxyService.class);
        Registry.register(OpsviewServerProxyService.SERVICE_NAME, service);

        LayoutContainer container = new LayoutContainer();
        final BorderLayout layout = new BorderLayout();
        container.setLayout(layout);
        container.setWidth(1024);
        container.setHeight(768);
        container.add(createMainMenuAndToolbarPanel(), createMainMenuAndToolbarLayoutData());
        container.add(new Dashboard(2), createDashboardLayoutData());
        container.add(createBottomStatusBar(), createBottomStatusBarLayoutData());

        RootPanel.get().add(container);
    }

This creates the stub that handles communication with the Opsview Enterprise proxy server and registers this stub object locally. Then the layout for the GXT application is setup. Now create the individual panels and layout data:

    private ContentPanel createMainMenuAndToolbarPanel() {
        ContentPanel mainMenuAndToolbarPanel = new ContentPanel();
        mainMenuAndToolbarPanel.setTopComponent(new MainMenu());
        mainMenuAndToolbarPanel.setHeading("Opsview Dashboard...");
        return mainMenuAndToolbarPanel;
    }

    private ContentPanel createBottomStatusBar() {
        ContentPanel panel = new ContentPanel();
        ToolBar toolbar = new ToolBar();
        panel.setHeaderVisible(false);
        Label opsviewVersion = new Label();
        opsviewVersion.setStyleName("Arial");
        opsviewVersion.setText("Opsview Dashboard Example");
        Label loggedInInfo = new Label();
        loggedInInfo.setText("Logged in as Opsview Admin");
        toolbar.add(opsviewVersion);
        toolbar.add(new FillToolItem());
        toolbar.add(loggedInInfo);
        toolbar.setHeight(25);
        panel.add(toolbar);
        return panel;
    }

    private BorderLayoutData createMainMenuAndToolbarLayoutData() {
        BorderLayoutData breadcrumbsLayoutData =
new BorderLayoutData(Style.LayoutRegion.NORTH, 50);
        breadcrumbsLayoutData.setHideCollapseTool(true);
        breadcrumbsLayoutData.setMargins(new Margins(5, 5, 0, 5));
        return breadcrumbsLayoutData;
    }

    private BorderLayoutData createDashboardLayoutData() {
        BorderLayoutData data = new BorderLayoutData(Style.LayoutRegion.CENTER);
        data.setMargins(new Margins(0, 5, 0, 5));
        return data;
    }

    private BorderLayoutData createBottomStatusBarLayoutData() {
        BorderLayoutData breadcrumbsLayoutData =
new BorderLayoutData(Style.LayoutRegion.SOUTH, 25);
        breadcrumbsLayoutData.setHideCollapseTool(true);
        breadcrumbsLayoutData.setMargins(new Margins(0, 5, 0, 5));
        return breadcrumbsLayoutData;
    }

We are also going to need a menu and you can implement it if you like as it is not going to be used in this example. Here is a starting point:

public class MainMenu extends ToolBar {
   public MainMenu() {....
   }
}

Last thing we need for the client is the actual Dashboard class that will call the Opsview Enterprise proxy and draw the pie chart with host group status:

public class Dashboard extends Portal {
   public static final String OFC_FLASH_LOCATION = "dashboardApp/chart/open-flash-chart.swf";

   private Chart m_hostGroupsChart;
   private List m_currentHostGroups;

   public Dashboard(int columns) {
        super(columns);
        setBorders(true);
        setStyleAttribute("backgroundColor", "white");
        setColumnWidth(0, .50);
        setColumnWidth(1, .50);

        add(createHostGroupsChartPortlet(), 1);
        loadHostGroupsChartPortlet();
   }

Here we start off by implementing the Dashboard as a GXT Portal. We add one Portlet that will show the Host Group status pie chart. Then we call a method that will load the pie chart.

Next implement the method that creates the Host Group chart portlet as follows:

   private Portlet createHostGroupsChartPortlet() {
        Portlet portlet = new Portlet();
        portlet.setHeading("Status - Host Groups Hierarchy");
        portlet.setLayout(new FitLayout());
        portlet.setHeight(400);

        String url = !isExplorer() ? "../../" : "";
        url += OFC_FLASH_LOCATION;
        m_hostGroupsChart = new Chart(url);
        m_hostGroupsChart.setBorders(true);

        //portlet.setIcon(ICONS.hostgroup());
        portlet.setTopComponent(createChartPortletToolBar());
        portlet.add(m_hostGroupsChart);

        configurePortlet(portlet);

        return portlet;
    }

First we create the portlet and set its layout and heading etc. We then create a chart component and add it to the portlet. The chart component uses Flash to draw the chart so we need to supply the chart component with the URL for the Flash component. We also create a little toolbar for the portlet to show that you could implement configuration of what you want to see this way:

    private ToolBar createChartPortletToolBar() {
        ToolBar toolbar = new ToolBar();

        Button searchButton = new Button("Config");
        //searchButton.setIcon(ICONS.properties());
        searchButton.setBorders(true);
        searchButton.setToolTip("Click here to configure the host group status chart");

        toolbar.setBorders(true);
        toolbar.setHeight(25);
        toolbar.add(new FillToolItem());
        toolbar.add(searchButton);

        return toolbar;
    }

The loading of the Chart Portlet is done as follows by calling the getHostGroups method of the Opsview Enterprise proxy server stub and implementing the onSuccess and onFailure methods:

    private void loadHostGroupsChartPortlet() {
        // Get the Opsview Server Proxy Service from the Registry
        final OpsviewServerProxyServiceAsync opsviewServerProxyService =
		(OpsviewServerProxyServiceAsync)
                Registry.get(OpsviewServerProxyService.SERVICE_NAME);

        // Make sure we got the service
        if (opsviewServerProxyService == null) {
            showInfoMsg("Opsview Server Proxy service could not be detected!");
        } else {
            opsviewServerProxyService.getHostGroups(new AsyncCallback
>() {
                public void onSuccess(List hostGroups) {
                    m_currentHostGroups = hostGroups;

                    ChartModel cm = new ChartModel("Hosts by Host Group",
                            "font-size: 14px; font-family: Verdana; text-align: center;");
                    cm.setBackgroundColour("#fffff5");

                    PieChart pie = new PieChart();
                    pie.addChartListener(listener);
                    pie.setAlpha(0.5f);
                    pie.setTooltip("#label#
#percent#");
                    pie.setColours("#ff0000", "#00aa00", "#0000ff", "#ff9900", "#ff00ff");

                    for (HostGroup hostGroup : hostGroups) {
                        int totalHosts = hostGroup.getHosts().getTotal();
                        String name = hostGroup.getName();
                        pie.addSlices(new PieChart.Slice(totalHosts, name + " (" +
totalHosts + ")", name));
                    }

                    cm.addChartConfig(pie);

                    m_hostGroupsChart.setChartModel(cm);
                }

                public void onFailure(Throwable throwable) {
			throw new RuntimeException(throwable);
                    showInfoMsg("Opsview Server Proxy
getHostGroups call failed: " + throwable.getMessage());
                }
            });
        }
    }

When the Opsview Enterprise server proxy responds we store the current host groups in a global variable so we can access it later on when the user clicks on one of the slices. We then do not have to call the server proxy as we have the HostStatus for the clicked on host group slice. We then create the PieChart and add a slice for each host group that was returned. A listener is also set for the pie chart and it will load the host status pie chart. The listener implementation looks like this:

    private ChartListener listener = new ChartListener() {
        public void chartClick(ChartEvent ce) {
            PieChart.Slice hostGroup = (PieChart.Slice) ce.getDataType();
            loadHostStatusChartPortlet(hostGroup.getText());
            Info.display("Chart Clicked", "You selected {0}.", "" + ce.getValue() +
", " + hostGroup.getLabel());
        }
    };

The Host Status Chart is loaded like this:

    private void loadHostStatusChartPortlet(String hostGroupName) {
        ChartModel cm = new ChartModel("Host Status for Host Group " + hostGroupName,
                "font-size: 14px; font-family: Verdana; text-align: center;");
        cm.setBackgroundColour("#fffff5");

        PieChart pie = new PieChart();
        pie.addChartListener(listener);
        pie.setAlpha(0.5f);
        pie.setTooltip("#label#
#percent#");
        pie.setColours("#ff0000", "#00aa00", "#0000ff", "#ff9900", "#ff00ff");

        for (HostGroup hostGroup : m_currentHostGroups) {
            if (hostGroupName.equals(hostGroup.getName())) {
                HostStatus hostStatus = hostGroup.getHosts();
                pie.addSlices(new PieChart.Slice(hostStatus.getDownUnhandled(),
"Down - unhandled (" + hostStatus.getDownUnhandled() + ")", "Down - unhandled"));
                pie.addSlices(new PieChart.Slice(hostStatus.getUpHandled(),
"Up - handled (" + hostStatus.getUpHandled() + ")", "Up - handled"));
                pie.addSlices(new PieChart.Slice(hostStatus.getDownHandled(),
"Down - handled (" + hostStatus.getDownHandled() + ")", "Down - handled"));
                pie.addSlices(new PieChart.Slice(hostStatus.getUpUnhandled(),
"Up - unhandled (" + hostStatus.getUpUnhandled() + ")", "Up - unhandled"));
            }
        }

        cm.addChartConfig(pie);

        m_hostGroupsChart.setChartModel(cm);
    }

When the Host Status chart is loaded it is not possible to go back to the Host Groups chart. There is no menu item for that or any implemented code to do that. If you like go ahead and implement this solution by yourself.

The little toolbar in the Chart Portlet is implemented like this:

    private void configurePortlet(final ContentPanel panel) {
        panel.setCollapsible(true);
        panel.setAnimCollapse(false);
        panel.getHeader().addTool(new ToolButton("x-tool-gear"));
        panel.getHeader().addTool(
                new ToolButton("x-tool-close", new SelectionListener() {
                    @Override
                    public void componentSelected(IconButtonEvent ce) {
                        panel.removeFromParent();
                    }
                }));
    }

The last couple of methods are helpers:

    public static boolean isExplorer() {
        String test = Window.Location.getPath();
        if (test.indexOf("pages") != -1) {
            return false;
        }
        return true;
    }

    public static void showInfoMsg(String msg) {
        MessageBox box = new MessageBox();
        box.setButtons(MessageBox.OK);
        box.setIcon(MessageBox.INFO);
        box.setTitle("Information");
        box.setMessage(msg);
        box.show();
    }
}

Running the Mini-Dashboard Client

To run this application after installing it under for example Tomcat we use a URL like this:

http://localhost:8080/opsviewui/dashboardApp.html

When we run this we should see a Host Groups Chart looking something like this:

Clicking on one of the Host Groups will show Host status as follows:

Resources

The Java libraries that were used in this example can be found here:

• GWT – http://code.google.com/webtoolkit/
• GXT – http://www.extjs.com/products/gwt/
• Apache Commons HTTP Client – http://hc.apache.org/httpclient-3.x/
• Jackson JSON Parser – http://jackson.codehaus.org/