Instrumenting your applications with StatsD

Many people don’t know this, but the first implementation of the StatsD daemon came from Etsy about 10 years ago. It was written specifically to collect metrics from instrumented code on the e-commerce platform, and this first version was written in Node.js. Over the years StatsD has become very popular and it is now the industry standard for opensource inside-the- app monitoring.

StatsD has a number of advantages that makes it ideal for measuring application performance but without those measurements influencing the application:

• It’s very light-weight text-based protocol
• The data is aggregated and stored outside of your application

The original version of StatsD only supported UDP. The publishing application sent the data and didn’t have to worry if it was consumed or not. Our version supports UDP, but also adds TCP for an additional level of reliability.

Starting with Geneos GA5.1 every Netprobe comes with a StatsD implementation inside the StatsD plugin.

The plugin has been tested with a number of 3rd party client libraries. Your existing library should work with our implementation as long as your publishing the following metric types.

• Gauge: these are metrics that represent a distinct value in time, such as CPU usage
• Counters: just like the name suggest, these are used to count things like number of logins, for example
• Unique Set: count unique values in a set
• Timers: These are essentially histograms and we translate these into synthetic metrics of percentiles
• Attributes: This is a special message type for non-metric information
• Events: These could be custom events or messages that would typically be logged to log files

But you don’t have to use 3rd party libraries, my favorite StatsD instrumentation libraries are our own. ITRS conveniently provides instrumentation libraries in Java and Python, with .Net and Node.JS soon to follow. The libraries are available on Maven or PyPi respectively, or on our website: https://resources.itrsgroup.com/downloads/collection

Now, let’s look at a real example of our StatsD plugin in action! For everything to work you’ll need a GA5.1 Gateway and Netprobe. If you don’t have sandbox Geneos environment, I highly recommend MiniGeneos, which pulls together all the necessary Geneos components in one neat package. Jeremy wrote a great blog about how to get started: https://www.itrsgroup.com/blog/minigeneos

Once you have Geneos up and running let’s start by writing a simple python app for trying out the different metric types:

If you execute this snippet you should get a Dataview in Geneos that looks like this:

Figure 1 - StatsD metrics in a dataview

You will notice that we sent gauges, counters, timers, events and attributes.

This Python example looks simple enough, but to truly appreciate the simplicity of StatsD let’s see what it looks like when we add a metric to this dataview using just a simple Linux shell command line.

As you can see, executing this command now added a new gauge named my_gauge to our dataview:

Figure 2 - New column my_gauge added

But how does the Netprobe know where to put the data and what dataview to update? It’s all based on the dimensions of each metric. In both examples all the metrics we sent had the following dimensions:

When the metrics arrive at the Netprobe, they are mapped based on their dimensions, using mapping rules, that map dimensions to Geneos constructs like Managed Entity names, Dataview names, Row names, etc. The beauty of all this is that the Managed Entities and Dataviews do not need to exist on the Gateway prior to the data being sent. Everything is created dynamically, on demand as the data arrives. This makes our new StatsD plugin perfect for monitoring highly dynamic environments.

While you are welcome to create your own mapping rules that best meet your needs, we do provide pre-packaged mapping rules for all of our new plugins, including StatsD in this GitHub repository: https://github.com/ITRS-Group/dynamic-entities-examples. In this blog we used the mapping rules found in basic_mapping.xml

Figure 3 - Example mapping rules

It’s really that simple! You can instrument your applications and be up and running in no time. The industry standard for inside-the-app monitoring is now available in your favorite Netprobe. StatsD plugin will provide you with greater insight into the performance and health of your applications.

I invite all of you to follow this blog and try it for yourself!

Figure 4 - Graphing StatsD metrics in the WebConsole