Using Kafka

By default, we use the embedded ActiveMQ broker to communicate with the Minions. This broker is used for both issuing remote procedure calls (RPCs) i.e. ping this host, and for transporting unsolicited messages such as SNMP traps and Syslogs.

Apache Kafka can be used as an alternative to ActiveMQ for transporting the unsolicited messages.

Kafka cannot currently be used for handling the RPCs. This means that ActiveMQ is still required when the Kafka support is enabled.

Kafka must be enabled on both OpenNMS HORIZON and Minion to function.

The Kafka server must be compatible with Kafka client version 0.10.1.1_1.

Consumer Configuration

Enable and configure the Kafka consumer on OpenNMS HORIZON using:

echo 'org.opennms.core.ipc.sink.strategy=kafka
org.opennms.core.ipc.sink.kafka.bootstrap.servers=127.0.0.1:9092' > "$OPENNMS_HOME/etc/opennms.properties.d/kafka.properties"

Restart OpenNMS HORIZON to apply the changes.

Additional Kafka consumer options can be set by defining additional system properties prefixed with org.opennms.core.ipc.sink.kafka. For example, you can customize the group ID using org.opennms.core.ipc.sink.kafka.group.id=MyOpenNMS.

A list of all the available options can be found here in New Consumer Configs.

Producer Configuration

Enable the Kafka producer on Minion using:

echo '!opennms-core-ipc-sink-camel
opennms-core-ipc-sink-kafka' > "$MINION_HOME/etc/featuresBoot.d/kafka.boot"
The snippet above prevents the opennms-core-ipc-sink-camel feature from starting when Minion is started, and loads the opennms-core-ipc-sink-kafka feature instead.

Next, configure the Kafka producer on Minion using:

echo 'bootstrap.servers=127.0.0.1:9092
acks=1' > "$MINION_HOME/etc/org.opennms.core.ipc.sink.kafka.cfg"

Restart Minion to apply the changes.

Additional Kafka producer options can be set directly in the org.opennms.core.ipc.sink.kafka.cfg file reference above. A list of all the available options can be found here in Producer Configs.