Add support for -loop and -every to collect data on regular basis

README.md

@@ -1,35 +1,103 @@
-# JMXQuery
-
-This project provides a command line tool written in Java and packaged as a Jar to allow you to connect to and query a JMX endpoint on a Java Virtual Machine.
-
-The command line tool can be used standalone, or with the Python module also included in this project, if you want to provide a way to query JMX from Python.
-
-This project was originally written in December 2014 and has been used in the Outlyer monitoring agent since to provide monitoring for JVM applications via Nagios plugins.
-
-Outlyer plugins use the Jar via the Python module to query JVM metrics via JMX and provide those for dashboards and alerts on Outlyer.
-
-However, this module can also be used standalone with any other monitoring tool that can run shell commands or include the Python module.
-
-There are two folders under this repo:
-
-## Java 
-This contains all the source to build and compile the Jar command line tool that connects to the JVM JMX endpoint.
-
-## Python
-This contains all the source to connect to the Jar with a simple to use Python class that handles all the communication via the command line.
-
-## Usage
-Full instructions on how to run the JAR directly on the command line are provided under the Java folder. To use the Python module just use the pip installer:
-
-```
-pip install jmxquery
-```
-
-__Note: The Python module only supports Python 3 at this time__
-
-## Contributing
-
-This project is released under the MIT License so you are free to use it in your own projects. However contributions are welcome and can be made via a fork/PR for review.
-
-Issues can also be raised in this repo if you find any, so please report them here for our attention.
-
+JMX Query
+=========
+
+A simple jar to query JMX data from a JVM and return in a format that can easily be used in Nagios check scripts.
+It has been extended to loop for a specified number of times, every n seconds.
+
+Requires Java 1.5 or above.
+
+
+Usage
+------
+
+```
+jmxquery [-url] [-username,u] [-password,p] [-query,q] [-incjvm] [-json] [-count -every] [-help]
+```
+
+options are:
+
+-help, h
+	Prints help page
+
+-url 
+	JMX URL, for example: "service:jmx:rmi:///jndi/rmi://localhost:1616/jmxrmi"
+
+-username, u
+	jmx username if required
+
+-password, p
+	jmx password if required
+
+-query, q
+        List of metrics to fetch in following format: {mBeanName}/{attribute}/{attributeKey};
+        For example: "java.lang:type=Memory/HeapMemoryUsage/used"
+        {attributeKey} is optional and only used for Composite metric types. 
+        Use semi-colon to separate metrics.
+
+-incjvm
+        Will add all standard JVM metrics to the -metrics query if used under java.lang domain
+        Useful utility function to add JVM metrics quickly and also for testing connections if
+        used by itself
+
+-json
+        Will output everything in JSON format, otherwise will be human readable text. Useful
+        for passing output to scripts.
+
+-count n will loop n times.
+
+-every n seconds.   This is used with -count to run the query periodically, and so collect 
+       the data over a period.  If using -json the data can easily be post processed into charts.
+       If using -count ... extra information is provided
+       date 2019/08/25
+       time 12:11:38.764
+       seconds since start, so 0,10,20, second etc - to make it easier to plot
+       loop     0,1,2,3 etc   
+
+Example Usage
+-------------
+
+### Listing available metrics
+
+List all metrics:
+
+```
+java -jar jmxquery.jar -url service:jmx:rmi:///jndi/rmi://localhost:1616/jmxrmi -q "*:*"
+```
+
+To filter on a particular domain so you only see the JMX metrics available under that (i.e. java.lang) you can use the following command:
+
+```
+java -jar jmxquery.jar -url service:jmx:rmi:///jndi/rmi://localhost:1616/jmxrmi -q "java.lang:*"
+```
+
+If you want to filter on attribute name you could use the following query:
+
+```
+java -jar jmxquery.jar -url service:jmx:rmi:///jndi/rmi://localhost:1616/jmxrmi -q "*:*/HeapMemoryUsage"
+```
+
+This will list any MBean attributes that have that attribue name in the JVM.
+
+### Get a metric value
+
+```
+java -jar JMXQuery.jar -url service:jmx:rmi:///jndi/rmi://localhost:1616/jmxrmi -q "java.lang:type=ClassLoading/LoadedClassCount"
+```
+
+You can get multiple values by joining the mbeans together with semi colons.
+
+```
+java -jar JMXQuery.jar -url service:jmx:rmi:///jndi/rmi://localhost:1616/jmxrmi -q "java.lang:type=ClassLoading/LoadedClassCount;java.lang:type=ClassLoading/UnloadedClassCount"
+```
+
+Building the Jar
+----------------
+
+Simply run the ./build.sh, modifying the build parameters for your environment in the script. This will compile the code for Java 1.5 and build the Jar ready to run.
+
+License & Credits
+-----------------
+
+This tool was inspired by https://code.google.com/p/jmxquery/ but has been completely rewritten by David Gildeh from Outlyer (www.outlyer.com).
+
+It is licensed under the MIT License (https://opensource.org/licenses/MIT)

java/src/main/java/com/outlyer/jmx/jmxquery/JMXConnector.java

@@ -3,7 +3,7 @@
 import com.outlyer.jmx.jmxquery.tools.JMXTools;
 import java.io.IOException;
 import java.util.ArrayList;
-import java.util.Arrays;
+// import java.util.Arrays;
 import java.util.HashMap;
 import java.util.Iterator;
 import java.util.Map;
@@ -28,6 +28,7 @@
  * JMX interface for values
  * 
  * @author David Gildeh (www.outlyer.com)
+ * Additions by Colin Paice
  */
 public class JMXConnector {
 
@@ -240,7 +241,13 @@ private ArrayList<JMXMetric> getAttributes(JMXMetric attribute, Object value) {
                     foundKey.setAttributeType(cData.get(attribute.getAttributeKey()));
                     foundKey.setmetricName(attribute.getmetricName());
                     foundKey.setmetricLabels(attribute.getmetricLabels()); 
-                    attributes.addAll(getAttributes(foundKey, cData.get(attribute.getAttributeKey())));                    
+                    //  Fix recursive, endless loop if attribute is missing
+                    //  Specify" Missing instead of calling itself again
+                    Object oValue = cData.get(attribute.getAttributeKey());
+                    if (oValue == null) oValue = "Missing";
+                    attributes.addAll(getAttributes(foundKey, oValue));
+                    // previous was
+                    // attributes.addAll(getAttributes(foundKey, cData.get(attribute.getAttributeKey())));                    
                 } catch (InvalidKeyException e) {
                     // Key doesn't exist so don't add to list
                 }    

java/src/main/java/com/outlyer/jmx/jmxquery/JMXMetric.java

@@ -15,6 +15,7 @@
  * E.g. jvm.memory.heap.used<>=java.lang:type=Memory/HeapMemoryUsage/used
  * 
  * @author David Gildeh (www.outlyer.com)
+ * Updated Colin Paice to prefix JSON with da
  */
 public class JMXMetric {
 
@@ -296,11 +297,11 @@ public String toString() {
     }
 
     /**
-     * Returns JSON representation of metric
+     * Returns JSON representation of metric, appended with any additional values passed in
      * 
      * @return  JSON String
      */
-    public String toJSON() {
+    public String toJSON(String[] passedStrings) {
 
         String beanName = this.mBeanName.replace("\"", "\\\"");
 
@@ -334,7 +335,13 @@ public String toJSON() {
             } else {
                 json += ", \"value\" : \"" + this.value.toString() + "\"";
             }
+        } 
+        // add any passed in values
+        for (int iStringArray =0; iStringArray < passedStrings.length; iStringArray++)
+        {
+            json += ","+passedStrings[iStringArray];      
         }
+
         json += "}";
 
         return json;

java/src/main/java/com/outlyer/jmx/jmxquery/JMXQuery.java

@@ -9,12 +9,20 @@
 import java.util.ArrayList;
 import java.util.Arrays;
 import javax.management.MalformedObjectNameException;
+// add the support for the time of day formatting etc
+import java.sql.Timestamp;
+import java.text.SimpleDateFormat;
+import java.lang.System;
+import java.lang.Runtime;
+
+
 
 /**
  *
  * JMXQuery is used for local or remote request of JMX attributes
  *
  * @author David Gildeh (www.outlyer.com)
+ * updates by Colin Paice to support loop, and date time support
  *
  */
 public class JMXQuery {
@@ -27,16 +35,38 @@ public class JMXQuery {
     String username = null;
     String password = null;
     boolean outputJSON = false;
+    long  loopCount = 0;
+    long  every = 0;
 
     /**
      * @param args
      */
     public static void main(String[] args) throws Exception {
-
+        SimpleDateFormat sdfdate = new SimpleDateFormat("yyyy/MM/dd");
+        SimpleDateFormat sdftime = new SimpleDateFormat("HH:mm:ss.SSS");                
+        Timestamp timestampStart = new Timestamp(System.currentTimeMillis());
+        Timestamp timestampCollectTime = timestampStart;
+        Timestamp timestampEndPreviousCollectTime = timestampCollectTime;
+        String dateNow ;
+        String timeNow ;
+        long secondsFromStart = 0;
+
+        String arrayPrefix = "";
+        String passedStrings[];
+        String header = "";
+
         // Initialise
         JMXQuery query = new JMXQuery();
         query.parse(args);
-
+
+        // if we have a loop, and we are doing JSON output trap CTRL-C to write a closing ]
+        if (query.outputJSON)
+        	Runtime.getRuntime().addShutdownHook(
+        		new Thread() {
+        		  public void run() { System.out.println("]");}
+        	       }
+        		);
+
         // Initialise JMX Connection    
         try {
             query.connector = new JMXConnector(query.url, query.username, query.password);
@@ -50,23 +80,74 @@ public static void main(String[] args) throws Exception {
             }
         }
 
-        // Process Query
-        try {
+
+
+
+   if (query.outputJSON)  System.out.println("[");
+   // check we have sensible values for loop time and loop count
+   if (query.every == 0 ) query.every = 10000; // millisecond value 
+   if (query.loopCount == 0 ) query.loopCount = 1;
+   for (long iLoop = 0;; iLoop ++)
+   {
+	  //  check to see if we have been asked to loop
+      if (iLoop >= query.loopCount)
+          break;
+      if (iLoop > 0) 
+      {
+    	// wait for the user specified period
+        // try to keep sleep time as close to specified time by excluding the time
+        // getting the data
+    	// if it takes 2 second to collect the data, and the interval between collect data is 10 seconds
+    	// then we should wait for 10 - 2 seconds
+    	// if the time to collect is longer than specified interval just use the sleep time as specified
+    	// by the end user
+
+        long sleepTime = query.every - (timestampEndPreviousCollectTime.getTime()-timestampCollectTime.getTime());
+        if ( sleepTime < 1) sleepTime = query.every;
+         Thread.sleep(sleepTime);
+      }
+      timestampCollectTime = new Timestamp(System.currentTimeMillis());
+      dateNow  = sdfdate.format(timestampCollectTime);
+      timeNow  = sdftime.format(timestampCollectTime);
+      // convert time delta from milliseconds to seconds
+      secondsFromStart = (timestampCollectTime.getTime() - timestampStart.getTime())/1000;
+
+      if (query.loopCount > 0)
+      {
+    	  passedStrings = new String[]{ " \"Date\" : \"" + dateNow + " \"",
+    			  " \"Time\" : \"" + timeNow + " \"",
+    			  " \"secondsFromStart\" : " + secondsFromStart,
+    			  " \"loop\" : " + iLoop
+    	  };
+    	  header = "Date:"+dateNow+ " Time:"+ timeNow+" Seconds from start:"+secondsFromStart +" loop:" +iLoop;
+      }
+      else
+      {
+    	  passedStrings = new String[] {}; // empty, no additional data appended
+      }
+      // Process Query
+      try {
             ArrayList<JMXMetric> outputMetrics = query.connector.getMetrics(query.metrics);  
+
             if (query.outputJSON) {
-                System.out.println("[");
+                System.out.println(arrayPrefix + "["); // either '' or ,
+                arrayPrefix = ",";   // separate each array with a ,
                 int count = 0;
+
                 for (JMXMetric metric : outputMetrics) {
                     metric.replaceTokens();
                     if (count > 0) {                        
-                        System.out.print(", \n" + metric.toJSON());
+                        System.out.print(", \n" + metric.toJSON(passedStrings));
                     } else {
                         count++;
-                        System.out.print(metric.toJSON());
+                        System.out.print(metric.toJSON(passedStrings));
                     }
                 }
                 System.out.println("]");
+                System.out.flush();  // so it gets passed on to the next stage
             } else {
+            	if (query.loopCount > 0)
+                    System.out.println(header);
                 for (JMXMetric metric : outputMetrics) {
                     metric.replaceTokens();
                     System.out.println(metric.toString());
@@ -99,9 +180,15 @@ public static void main(String[] args) throws Exception {
                 System.out.println(Arrays.toString(e.getStackTrace()));
                 System.exit(2);
             }
-        }
-
-        // Disconnect from JMX Cleanly
+        } // end of try
+        timestampEndPreviousCollectTime = new Timestamp(System.currentTimeMillis());
+
+     }  // end of for iloop      
+     //  Do not put out trailing ] as the shutdown exit does it
+     //   if (query.outputJSON) 
+     //                System.out.println("]");
+
+    // Disconnect from JMX Cleanly
         query.connector.disconnect(); 
     }
 
@@ -181,6 +268,12 @@ private void parse(String[] args) throws ParseError {
                     username = args[++i];
                 } else if (option.equals("-password") || option.equals("-p")) {
                     password = args[++i];
+                    // additional variables for looping and duration of loop
+                } else if (option.equals("-count") || option.equals("-c")) {
+                    loopCount = Long.parseLong(args[++i]);
+                } else if (option.equals("-every") || option.equals("-e")) {
+                    every = 1000 *Long.parseLong(args[++i]); // in millseconds
+
                 } else if (option.equals("-query") || option.equals("-q")) {
 
                     // Parse query string to break up string in format:
@@ -232,4 +325,4 @@ private void printHelp(PrintStream out) {
             }
         }
     }
-}
+}