Netcool Event Rate

From neil.tappsville.com
Jump to navigationJump to search

Netcool OMNIbus - Accurately measuring event rates

Hello all, My name is David Morton and I'm a developer for OMNIbus core. I've decided to kick off my blogging career with a guide on how to use OMNIbus triggers to get an accurate event rate in your ObjectServers!

I have talked to a few customers that use their event rate as a performance indicator for their ObjectServers. I plan on outlining an efficient way to measure this using only triggers and tables in an ObjectServer. I'll try my best to keep this guide version agnostic so that anyone should be able to use it. If you use other metrics as performance indicators for OMNIBus please comment below. I always like to hear about the different ways people measure their ObjectServers.

All the code below can be entered via nco_sql (put it into a file and pipe it into nco_sql). eg ./nco_sql -server NCOMS -user root -password < test.sql


OK, let's begin.

OK, let's begin.

  • First we need to set up our ObjectServer to store a few bits of information. Namely:
   A snapshot of the event count
   The time that snapshot was taken
   The event rate (this will be the output of our trigger)

So let's set up a table in the ObjectServer to do this. We will only let this table have 1 row (you can modify this if you want to track event rate over time) We will put it into the master database. We will use a varchar field for our primary key and use a static string (I happen to use 'alerts' because I made this for alerts events)

create table master.my_perf_table virtual
(
DatabaseName varchar(64) primary key,
EventCount int,
LastUpdate UTC,
EventRate Real
);
go

Trigger happy

Ok, so now we have a virtual table (will be cleared when ObjectServer restarts) that holds the 3 things we need to calculate event rate. Let's make our trigger.

First let's take advantage of triggers which already exist and understand what they do. The trigger group "stats_triggers" has a number of triggers which collect stats, we will take advantage of them. First before we begin, remember to enable this trigger group.

The stats triggers maintain some event counts in the master.activity_stats table. There is a trigger for new inserts, dedups, journals and details tables. All these triggers do is increment counters in the master.activity_stats table. If you want to see how they do this, check out $OMNIHOME/etc/automation.sql (these are the default automation triggers).

So in that table we now have a running total of all the events that have fired at alerts.status, alerts.details and alerts.journals let's put them to use in another trigger.

create or replace trigger my_objserv_eventrate
group stats_triggers
priority 20
comment 'Calculate the event rate on the object server'
every 60 seconds
declare
elapsed time;  --to store the time elapsed
firstRun boolean;  --Is this the first run?
eventCount integer; --temp event count
begin
    set firstRun = true;
    for each row mrow in master.my_perf_table --this is our table
    begin
        set firstRun = false; --if there are no rows in my_peft_table, this will not happen
        set elapsed = getdate() - mrow.LastUpdate; --work out how much time has passed since the last run
                                                     (by default this will usually be 60)

        set eventCount = 0;
        for each row srow in master.activity_stats where srow.DatabaseName = 'alerts'
        begin
            set eventCount = srow.StatusNewInserts + srow.StatusDedups + srow.JournalInserts + srow.DetailsInserts;
                                                       --add up all the events going into alerts database
        end;
        -- This next step can be done (more efficiently) with one update. For ease of reading I have split it
        update master.my_perf_table via 'alerts' set EventRate = to_real((eventCount - mrow.EventCount)) / to_real(elapsed);
                                                          --events now minus events before divided by time
        update master.my_perf_table via 'alerts' set EventCount = eventCount; --update this count for the next run
        update master.my_perf_table via 'alerts' set LastUpdate = getdate(); --update the time for the next run

        --In one command it looks like the following
        --update master.my_perf_table via 'alerts' set EventRate = to_real((eventCount - mrow.EventCount))
                                                         / to_real(elapsed) , EventCount = eventCount, LastUpdate = getdate();
    end;

    if(firstRun = true) then
        -- This will only run when our table is empty (first run)
        set eventCount = 0;
        for each row srow in master.activity_stats where srow.DatabaseName = 'alerts'
        begin
            set eventCount = srow.StatusNewInserts + srow.StatusDedups + srow.JournalInserts + srow.DetailsInserts;
        end;
        insert into master.my_perf_table values('alerts', getdate(), eventCount, 0); --first event rate will be 0.0
    end if;
end
go