Introduction

Syslog-ng, a flexible and scalable log management solution, can integrate seamlessly with OpsRamp for efficient log collection and monitoring. This integration supports two primary methods: Agent-based and Direct forwarding.

  • Agent-Based Integration: With agent-based integration, Syslog-ng forwards logs to OpsRamp through an installed agent. The agent collects syslog-ng logs and transmits them securely to OpsRamp, where they are processed for monitoring, alerting, and analysis.
  • Direct: Syslog-ng can directly forward logs to OpsRamp without requiring an intermediary agent. This method is beneficial for environments where direct communication is preferred or where agent deployment may be impractical.
Tabbed Interface with Table
Agent-based
Direct

Configuration

Below is the default configuration for integrating Syslog-ng with OpsRamp in an agent-based setup:

# supported version for this configuration is Syslog-NG 3.25.1-3
syslog_server:
  type: syslog
  source: syslog_server
  address: ""
  mode: tcp
  protocol:

Syslog-ng configuration

  1. To forward logs using syslog-ng, add the following configuration to your /etc/syslog-ng/syslog-ng.conf file.
  2. The following configuration specifies that all traffic coming in on port 515 should be forwarded to the OpsRamp agent, which is listening on port 514.
  3. The source of the input can vary depending on the use case, but to export logs to the OpsRamp portal, the destination must be configured as shown below:
   source s_net {tcp(ip(0.0.0.0) port(515)); };
destination d_net { tcp("0.0.0.0" port(514) log_fifo_size(1000)); };  # This line specifies the destination
log { source(s_net); destination(d_net); };
  1. Execute the following command in the terminal to restart Syslog-NG:
systemctl restart syslog-ng.service

Custom Configuration

  1. Create a custom configuration file at /opt/opsramp/agent/conf/log.d/log-config.yaml by copying the content from the sample config file /opt/opsramp/agent/conf/log.d/log-config.yaml.sample.
  2. Update (add/edit/remove) the configurations as needed for the syslog_server source, using the guidelines for field definitions provided below:
  type: "" # "syslog" is the type for syslog
  source: "" # specify the name of the application(if empty the source name is used)
  mode: # The type of socket to use. Valid Sockets are ["tcp","udp","unix"]
  address: # specify the "ip:port" on which the syslog server must run (only applicable if mode is set to "tcp" or "udp")
  protocol: # The protocol to parse the syslog messages as. Options are rfc3164 and rfc5424
  filters: # (optional) This section is used for filtering out logs by attribute_type. Execution happens in the order in which the filters are specified.
    - attribute_type: "body" # Possible values for attribute_type [ "body", "attributes", "resource" ] (defaults to "body" if the field is ignored).
      key: "" # The tag for which the respective filtering rule must be applied.
      include: "" # keeps the records which match the specified pattern.
    - key: "" # attribute_type defaults to "body" if the field is ignored.
      exclude: "" # removes the records which match the specified pattern
  masking: # (optional) This section is used to specify rules to mask any sensitive data in the logs.
    - text: "" # The text which needs to be masked.
      placeholder: "" # The string to replace the text to be masked.
  labels: # (optional) Allows to set up to 5 resource labels. If more than 5 are specified then the first five are considered and others are ignored.
    key: "{{value}}"
  attributes: [ ] # (optional) Specify parsed fields which need to be set as record attributes.
  resource_attributes: [ ] # (optional) Specify parsed fields which need to be set as resource attributes.
  1. Save the updated configuration file at /opt/opsramp/agent/conf/log.d/log-config.yaml.
  2. Restart the agent by executing the following command in the terminal:
systemctl restart opsramp-agent

Supported labels

Following are some of the supported labels:

  • source
  • resourceUUID

Configuration

To transmit logs from syslog-ng to OpsRamp, set up a new HTTP destination and utilize it to forward logs from the necessary sources in syslog-ng.

  1. Create a new configuration file. It is suggested to generate a new configuration file in the conf.d directory within the syslog-ng configuration folder. touch /etc/syslog-ng/conf.d/10-opsramp.conf
  2. Add the below content to /etc/syslog-ng/conf.d/10-opsramp.conf file created in Step 1.
destination d_opsramp_http {
    http(
    url("")
    method("PUT")
    batch-lines(1000)
    batch-bytes(1024Kb)
    batch-timeout(5000)
    user-agent("syslog-ng User Agent")
    headers("Content-Type: application/syslog.rfc5424")
    body-suffix("\n")
    body("<${PRI}>1 ${ISODATE} ${HOST:--} ${PROGRAM:--} ${PID:--} ${MSGID:--} ${SDATA:--} ${MESSAGE}")
    );
};

The destination configuration above includes the HTTP endpoint for exporting logs, necessary headers, and default batching options for transmitting logs to OpsRamp.

3. Add the following content to the /etc/syslog-ng/conf.d/10-opsramp.conf file after the destination is configured. The source configuration is named s_src as shown below:

log { source(s_src); destination(d_opsramp_http); };`
  1. Restart the syslog-ng service using the following command:
sudo service syslog-ng restart

Supported labels

Supported labels refer to the specific identifiers or attributes that can be associated with log messages in syslog-ng. These labels provide additional context or metadata about the log messages, helping to search, sort, and process log messages efficiently.
Following are some of the supported labels:

  • source
  • host
  • level
  • facility
  • severity