DataDog and JMX
Jump to navigation
Jump to search
External
Internal
Configuration
The JMX configuration is located in /etc/dd-agent/conf.d/jmx.yaml.example.
After modification and removal of the ".example" suffix, restart datadog as root:
service datadog-agent restart
JMX integration with JBoss
Server Preparation
JBoss instances have to be configured to allow JMX agent access. Details on how to configure JBoss various JBoss versions running in various modes are available here:
Agent Classpath and Startup Options
Working jmx.yaml configuration:
init_config: custom_jar_paths: - /opt/jdg/bin/client/jboss-cli-client.jar instances: - jmx_url: "service:jmx:remote://1.2.3.4:4447" user: jmx password: *** java_bin_path: /opt/jdk/bin/java name: jdg1 tags: env: stage newTag: test conf: - include: domain: jboss.datagrid-infinispan bean: - type=Cache,name=gameStateCache(dist_sync),manager=clustered,component=Statistics attribute: attribute1: metric_type: counter alias: numberOfEntries
Make sure the JMX user has been configured on target JBoss instances, for symptoms see "JMX User Has not Been Configured on the Target JBoss Instance" section.
Connect to the JBoss Instance
Use the following URLs:
- For Standalone EAP 6
- For EAP 6 Host Controller
- For EAP 6 Domain Mode Server Node
- For Standalone EAP 7
- For EAP 7 Host Controller
- For EAP 7 Domain Mode Server Node
Troubleshooting
Agent Process Signature
ps -ef | grep java | grep datadog dd-agent 14889 14841 0 Oct15 ? 00:05:47 \ /opt/jdk/bin/java -Xms50m -Xmx200m \ -classpath /opt/jdk/lib/tools.jar:/opt/datadog-agent/agent/checks/libs/jmxfetch-0.11.0-jar-with-dependencies.jar org.datadog.jmxfetch.App \ --check jmx.yaml --check_period 15000 --conf_directory /etc/dd-agent/conf.d --log_level INFO \ --log_location /var/log/datadog/jmxfetch.log --reporter statsd:localhost:8125 --status_location /opt/datadog-agent/run/jmx_status.yaml collect
Agent Reinitialization
service datadog-agent restart
Metric List
sudo /etc/init.d/datadog-agent jmx list_not_matching_attributes
Errors
JMX User Has Not Been Configured on the target JBoss Instance
2016-10-19 15:34:53,433 | ERROR| App | Cannot connect to instance service:jmx:remoting-jmx://1.2.3.4:4447 javax.security.sasl.SaslException: Authentication failed: all available authentication mechanisms failed: DIGEST-MD5: Server rejected authentication
TODO
2016-10-19 16:20:43,683 | INFO | Connection | Connecting to: service:jmx:remote://1.2.3.4:4447 2016-10-19 16:20:43,731 | INFO | xnio | XNIO version 3.3.6.Final-redhat-1 2016-10-19 16:20:43,777 | INFO | nio | XNIO NIO Implementation Version 3.3.6.Final-redhat-1 2016-10-19 16:20:43,970 | INFO | remoting | JBoss Remoting version 4.0.18.Final-redhat-1 2016-10-19 16:20:44,663 | INFO | Instance | Connected to JMX Server at service:jmx:remote://1.2.3.4:4447 2016-10-19 16:20:44,664 | ERROR| Instance | Unable to compute a common bean scope, querying all beans as a fallback java.lang.ArrayIndexOutOfBoundsException: 1 at org.datadog.jmxfetch.Configuration.getIncludeFiltersByDomain(Configuration.java:89) at org.datadog.jmxfetch.Configuration.getGreatestCommonScopes(Configuration.java:227) at org.datadog.jmxfetch.Instance.getBeansScopes(Instance.java:284) at org.datadog.jmxfetch.Instance.refreshBeansList(Instance.java:301) at org.datadog.jmxfetch.Instance.init(Instance.java:132) at org.datadog.jmxfetch.App.init(App.java:354) at org.datadog.jmxfetch.App.main(App.java:92)