Introduction

This section provides instructions on integrating a custom application.

Prerequisite

To set up log configuration for a custom application, you must add a corresponding custom app definition in the App Definitions. See App Definitions for more details.

Configure custom integration

Once you have added a custom app definition in the APP DEFINITIONS, you can set up log configuration for the custom application from the INGESTION tab.

To set up log configuration:

  1. Click the custom app you have created.
    The LOG CONFIGURATION page is displayed.

    You can perform any of the following actions from the LOG CONFIGURATION page.

    • Default Configuration - This page provides the predefined configurations that describe how logs are collected from various sources and subsequently analyzed for insights and patterns.
    • +ADD - This page refers to the process of adding new settings or parameters to the default configuration based on requirements or preferences.

Default Configuration

  1. Select Default Configuration to make the changes to the default template.
    You can make changes to the template by providing values against the data provided in the template.
  type: "file"
  source: "Custom"
  include:
    - 
  exclude:
    - 
  multiline:
    line_start_pattern: 
    line_end_pattern: 
  parser_type: 
  parse_from: 
  parse_to: 
  regex:  
  start_at: 
  timestamp:
    layout_type: 
    layout: 
  fingerprint_size:
  severity_from: 
  severity_priority_order:
  severity_mapping: 
    warn: [ ] 
    info: [ ] 
    debug: [ ]
    trace: [ ]
    error: [ ]
    fatal: [ ]
  custom_formatting:  
  filters: 
    - attribute_type:  
      key:  
      include: 
    - key: 
      exclude: 
  masking: 
    - text:  
      placeholder:  
  labels: 
    key: 
  attributes: [ ] 
  resource_attributes: [ ]

Default configuration attributes explained

type

Description

Specifies the type or category of logs being configured.

Example

Possible values [ "file", "journald", "syslog", "kubernetes_logs", "windowseventlog", "docker"].

source

Description

Identifies the source of the logs.

include

Description

Specifies patterns to include certain log entries.

exclude

Description

Specifies patterns to exclude certain log entries.

multiline

Description

Defines patterns for handling multiline log entries. The multiline configuration block must include either line_start_pattern or line_end_pattern. These regex patterns identify either the start or end of a log entry.
See Multiline Configuration for more details.

parser_type

Description

Specifies the type of parser to be used for parsing the log entries. Only applicable for source file and can accept values ["regex", "json"].
See File Log Collection Using Regex for more details.
See File Log Collection Using JSON for more details.

Example

Possible values ["regex", "json"].

parse_from

Description

(optional) Specifies where to start parsing within the log entry.

Example

Possible values [ "body" , "attributes", "resource" ] (defaults to body if the field is ignored).

parse_to

Description

Specifies where to stop parsing within the log entry.

Example

Possible values [ "body" , "attributes", "resource" ] (defaults to body if the field is ignored).
See Setting ParseTo for more details.

regex

Description

Specifies a regular expression pattern for matching log entries.

start_at

Description

Specifies where to start collecting logs.

Example

Possible values ["end", "beginning" ].

timestamp

Description

Specifies how timestamps in log entries are formatted.

  • layout_type: This attribute determines the type of timestamp layout being used.
  • layout: This attribute specifies the format of the timestamp string within the log entries.
    See Supported Timestamps for more details.

Example

Possible values ["strptime" , "gotime", "epoch" ].

fingerprint_size

Description

Specifies the size of the fingerprint for log entries. This attribute specifies the quantity of bytes utilized from the start of a file to uniquely identify the file.
See Fingerprint Size for more details.

severity_from

Description

Specifies the location of the severity level within the log entry. Accepts values within the “attributes”, “body”, and “resource” sections.
See Severity From for more details.

severity_priority_order

Description

Specifies the priority order for severity levels. This list must contain the following six values, ordered based on their priority:

  • error
  • fatal
  • warn
  • info
  • debug
  • trace

The list should include all the six values.

Example

Possible values [ "error", "fatal", "warn", "info", "debug", "trace" ].

severity_mapping

Description

Defines how the severity of a log should be established when the logs are not formatted correctly. It can assign multiple values to the same severity level.
See Severity Mapping for more details.

custom_formatting

Description

(optional) Specifies any custom formatting rules for log entries.
See Custom Formatting for more details.

filters

Description

Specifies filtering rules for log entries based on attributes or keys within the log entry.

  • attribute_type - Specifies the type of attribute being filtered, with possible values being “body”, “attributes”, or “resource”. If this field is not provided, it defaults to “body”.
  • key - Represents the tag for which the respective filtering rule must be applied. This is the identifier used to determine which part of the log entry the filter should be applied to.
  • include - Keeps the records that match the specified pattern.
  • exclude - Removes the records that match the specified pattern.
    See Filter Logs for more details.
masking

Description

(optional) This attribute is used to specify rules to mask any sensitive data in the logs.

  • text - Specifies the sensitive text that needs to be masked within the logs.
  • placeholder - Determines the string used to replace the sensitive text in the logs.
    See Masking Logs for more details.
labels

Description

(Optional) Allows the setup of up to five resource labels. If more than five labels are specified, only the first five are considered, and the rest are ignored.

  • key - This attribute denotes the placeholder {{value}} used for the labels.
    See Labels for more details.
attributes

Description

(Optional) Used to specify parsed fields that need to be set as record attributes.

resource_attributes

Description

Specifies additional resource attributes related to the log entry.


See Logs Configuration for detailed information on the supported attributes.

  1. By default, the Enable Configuration checkbox is checked. If you want to disable the log ingestion for the app, you can uncheck the Enable Configuration checkbox.
  2. Click SAVE once you have made the changes.
    The data is now saved, and logs will be ingested based on the configuration settings.

Add Configuration

  1. In the Add Custom Configuration page:

    • Enter the name for the configuration.

    • Select the priority value from the Priority field. The priorities are assigned with a numerical value, the configuration with the highest value will be considered first.

    • To assign resources to the log configuration, click FILTER.
      The RESOURCES page is displayed.

    • You can filter the resources using LOGQL attributes. Once you select the resources, click DONE.

    • You can make changes to the template under LOG CONFIGURATION YAML by providing values against the data provided in the template.
      Note: See Default configuration for more details on the available configuration attributes.

  2. Click SAVE.
    The new log configurations have been added and will be displayed on the LOG CONFIGURATIONS page.