DataDog and JMX

From NovaOrdis Knowledge Base
Jump to navigation Jump to search

External

Internal

Configuration

The JMX configuration is located in /etc/dd-agent/conf.d/jmx.yaml.example.

After modification and removal of the ".example" suffix, restart datadog as root:

service datadog-agent restart

JMX integration with JBoss

Server Preparation

JBoss instances have to be configured to allow JMX agent access. Details on how to configure JBoss various JBoss versions running in various modes are available here:

WildFly and JMX

Agent Classpath and Startup Options

Working jmx.yaml configuration:

init_config:
    custom_jar_paths:
      - /opt/jdg/bin/client/jboss-cli-client.jar

instances:
    - jmx_url: "service:jmx:remoting-jmx://1.2.3.4:4447"
      user: jmx
      password: ***
      java_bin_path: /opt/jdk/bin/java
      name: jdg1
      tags:
        env: stage
        newTag: test
      conf:
        - include:
            domain: jboss.datagrid-infinispan
            bean:
              - type=Cache,name=gameStateCache(dist_sync),manager=clustered,component=Statistics
            attribute:
              attribute1:
                metric_type: counter
                alias: numberOfEntries

Make sure the JMX user has been configured on target JBoss instances, for symptoms see "[[]]" section.

Connect to the JBoss Instance

Use the following URLs:

Troubleshooting

Agent Process Signature

ps -ef | grep java | grep datadog

dd-agent 14889 14841  0 Oct15 ? 00:05:47 \
  /opt/jdk/bin/java -Xms50m -Xmx200m \
  -classpath /opt/jdk/lib/tools.jar:/opt/datadog-agent/agent/checks/libs/jmxfetch-0.11.0-jar-with-dependencies.jar org.datadog.jmxfetch.App \
  --check jmx.yaml --check_period 15000 --conf_directory /etc/dd-agent/conf.d --log_level INFO \
  --log_location /var/log/datadog/jmxfetch.log --reporter statsd:localhost:8125 
  --status_location /opt/datadog-agent/run/jmx_status.yaml collect


Agent Reinitialization

service datadog-agent restart

Errors

JMX Users Has Not Been Configured on the target JBoss Instance

2016-10-19 15:34:53,433 | ERROR| App | Cannot connect to instance service:jmx:remoting-jmx://1.2.3.4:4447 javax.security.sasl.SaslException: Authentication failed: all available authentication mechanisms failed:
   DIGEST-MD5: Server rejected authentication