A lap around ServicePulse (2 of 3)

This is part 2 of my short series about the new Particular Service Platform… specifically about ServicePulse. In my first post about ServicePulse, you can read about the overall architecture and the problems that ServicePulse solves. Now I want to focus on one specific feature: the ability to add custom checks to your services.

After having set up ServiceControl and ServicePulse, you are able to monitor any NServiceBus service in your architecture. Just drop in the Heartbeat plugin and ServicePulse will see your service. The Heartbeat basically says: the service is there, it’s running and it’s able to send messages (since the Heartbeat is sent through a message to the ServiceControl input queue).

In general, this will not be sufficient, as you will also want to know how the service is behaving from a functional point of view. Or maybe you even want to monitor some deeper technical dependencies, such as whether it can reach its database, whether the config is OK, whether it can reach that external web service you rely upon, et cetera.

This is where custom checks come in, and they’re easy to implement. A custom check is very similar to the Heartbeat check: you just drop in a DLL that contains one or more checks, and the messages will be sent to ServiceControl. We end up with the following deployment overview:

ServiceControl-Deployment-CustomChecks

The custom check plugins will report through the same channel as the Heartbeat plugin.

Implementing a custom check
A custom check is pretty easy to implement, and you can choose between two scenarios: a periodic check and a one-time check.

Either way, you need to add a reference to the ServiceControl.Plugin.CustomChecks Nuget package and you’re good to go.

Startup check
A Startup custom check is executed when the service starts, so basically only once. A check like this is useful if you want to check some configuration settings after deployment, or if you want to check some environment dependencies that the service needs to run properly. You implement a one-time check by inheriting from ServiceControl.Plugin.CustomChecks.CustomCheck.

public class StartupCheck : ServiceControl.Plugin.CustomChecks.CustomCheck
{
    public StartupCheck()
        : base("StartupCheck", "Categoryname") // the name of the check and its category are specified here 
    {
        if (ConfigurationManager.AppSettings["TestSetting"] != "1")
        {
            ReportFailed("TestSetting must be 1!");
        }
        else
        {
            ReportPass();
        }
    }
}

The default constructor passes a check name and a category to the base constructor, which bootstraps the custom check. CustomCheck is an abstract base class, so you need to implement the StartupCheck method. All you have to do is put in your check, and call either ReportPass if all is well, or ReportFailed if there’s a problem. ReportFailed accepts a fault reason, which will be visible in ServicePulse.

Periodic check
A periodic check runs at a self defined interval, so it’s ideally suited to monitor things that might change over time, such as the availability of certain services or databases, or functional scenarios such as: “has all Point of Sale data arrived yet?”.

Again, implementing such a check is pretty easy. This time, inherit from ServicePulse.Plugin.CustomChecks.PeriodicCheck.

public class CheckHealth : PeriodicCheck
{
    public CheckHealth()
        : base("Healthcheck", "CategoryName", TimeSpan.FromMinutes(10))
    {

    }

    public override CheckResult PerformCheck()
    {
        // Fake a failure once in a while
        // TODO: think of a useful check to implement here.
        if (DateTime.Now.Second % 2 == 0)
        {
            return CheckResult.Failed("This is a sample failure report");
        }
         return CheckResult.Pass;
    }
}

Besides the check name and category name, the base constructor also excepts a TimeSpan. You can specify here at what interval the check runs. Each 10 minutes in the example.

Next, we override the PerformCheck method, which returns a CheckResult object. Case of success, report CheckResult.Pass, otherwise use CheckResult.Failed. Again, a reason or description for the feature must be supplied.

So what does admin/ops see?
After a custom check is deployed and activated, we can see the results in the ServicePulse front end. On the Dashboard, it shows:

Custom checks

And upon further inspection on the Custom Checks screen:

Custom checks - overview

In my current project, we make heavy use of custom checks to monitor the health of our system, and whether customers are efficiently using it. For now, we do this through a custom built monitoring service (based on NServiceBus), but I can see these migrating to ServiceControl plugins over time.

Next time: dealing with failed messages!

Advertisements

5 thoughts on “A lap around ServicePulse (2 of 3)”

  1. Great post, Roy. Have you figured out a way to alert you (email, etc.) when an endpoint goes down, or do you have to be looking at SP to notice? Also, any way to be alerted when a custom check *starts* failing? For example, you create a custom check that runs every 5 mins to confirm your DB is responding. If your DB goes down, it would be nice to get an email when that check first fails, then to perhaps get another email when the custom check starts to succeed again? You wouldn’t want errors every 5 minutes. This kind of feels like a monitoring saga.

    1. Hi Danny, thanks for your comment and sorry for the delayed response!

      I can relate to your question, as I have similar requirements. AFAIK there’s no way to do this out-of-the-box at the moment. My guess would be to do this yourself off of a feed from SP, or maybe ServiceControl.

      1. Hey Roy, I’m going to be presenting my alerting solution at NSBCon next month, as well as putting my code in GitHub. We’ve been using it in production for almost a year now and it’s working really well!

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s