0

I am sending logs from my applciation logstash successfully, i started with this tutorial http://www.andrew-programming.com/2018/09/18/integrate-springboot-application-with-elk-and-filebeat/ then implemented my code in to my own apps easily.

The question I have is that I haven't mentioned filebeats anywhere in my app, how is it being used?

Everything is working but curious to know where filebeats comes is, is it through the logstash dependency in the pom file?

logback-spring.xml

<!DOCTYPE configuration>
<configuration>
  <appender name="LOGSTASH" class="net.logstash.logback.appender.LogstashTcpSocketAppender">
    <destination>localhost:4560</destination>
    <encoder charset="UTF-8" class="net.logstash.logback.encoder.LogstashEncoder">
    </encoder>
  </appender>

  <include resource="org/springframework/boot/logging/logback/base.xml"/>

  <root level="INFO">
    <appender-ref ref="LOGSTASH" />
    <!--<appender-ref ref="CONSOLE" />-->
  </root>

</configuration>

application.properties

logging.file=/tmp/filebeatDemoApp.log

pom dependency

        <dependency>
            <groupId>net.logstash.logback</groupId>
            <artifactId>logstash-logback-encoder</artifactId>
            <version>5.1</version>
        </dependency>

logstash.conf

input {
  tcp {
  port => 4560
  codec => json_lines
  }
  beats {
    host => "127.0.0.1"
    port => "5044"
  }
}
output{
  stdout { codec => rubydebug }
  elasticsearch {
  hosts => ["localhost:9200"]
  index => "app-%{+YYYY.MM.dd}"   
  document_type => "%{[@metadata][type]}"
  }
}
1

You are not using Filebeat. With <destination>localhost:4560</destination> you are sending directly to Logstash. It's nice because you don't need to care about logfiles, parsing them, or filling up the disk. Downside is that you won't receive any messages if the network is down and Logback will only buffer 200MB of logs from what I remember — so you might lose logs during extended outages.

PS: When using this configuration you can remove the beats { block from the logstash.conf since you aren't using it (and Filebeat too).

  • Thanks for the reply, the thing that confuses me though is that the tutorial is called 'Integrate Springboot Application With ELK And Filebeat' Even goes to the trouble of downloading filebeat and editing the .yml file ?? Any idea how I would add filebeats in? – Mick O'Gorman Apr 12 at 8:42
  • 1
    Well, if you log to a file (ideally JSON so you don't have to parse anything), Filebeat could just pick up that file and forward it to Elasticsearch directly (no Logstash required). Example for the logback.xml (repo also contains the Filebeat config): github.com/xeraa/microservice-monitoring/blob/master/java/… – xeraa Apr 12 at 23:13

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.