Logstash 8.15.0

Logstash is part of the Elastic Stack along with Beats, Elasticsearch and Kibana. Logstash is a server-side data processing pipeline that ingests data from a multitude of sources simultaneously, transforms it, and then sends it to your favorite "stash." (Ours is Elasticsearch, naturally.). Logstash has over 200 plugins, and you can write your own very easily as well. For more info, see https://www.elastic.co/products/logstash

Tags log log-viewer java
License
State stable

Homepage Download

Recent Releases

8.15.008 Oct 2024 11:05 major feature: snmp-ga-8.15.0 . ==== Announcing the new ls SNMP integration plugin. The new `logstash-integration-snmp` plugin is available and bundled with ls 8.15.0 (and later) by default. This plugin combines our classic `logstash-input-snmp` and `logstash-input-snmptrap` plugins into a single Ruby gem at v4.0.0 and later. The `logstash-integration-snmp` plugin combines the. `logstash-input-snmp` and `logstash-input-snmptrap` plugins into one integrated plugin that encompasses. the capabilities of both. This integrated plugin package provides better alignment in snmp processing, better resource management, easier package maintenance, and a smaller installation footprint. breaking-8.15.0 Breaking change with release of `logstash-integration-snmp`. Prior to 8.15.0, ls bundled stand-alone versions of the `input-snmp` and `input-snmptrap` plugins by default. When you upgrade to 8.15.0, the stand-alone plugins are replaced by the 4.0.0+ version contained in the new integration. IMPORTANT: Before you upgrade to ls 8.15.0, be aware of logstash-ref /plugins-integrations-snmp.html#plugins-integrations-snmp-migration behavioral and mapping differences between stand-alone plugins and the new versions included in `integration-snmp`. If you need to maintain current mappings for the `input-snmptrap` plugin, you have options to logstash-ref /plugins-integrations-snmp.html#plugins-integrations-snmp-input-snmptrap-compat preserve existing behavior . featured-8.15.0 . ==== New features and enhancements. wolfi-8.15.0 Wolfi-flavored Docker Image. Starting with 8.15.0, ls provides an extra Docker image flavor based on the Wolfi Linux distribution: https://github.com/wolfi-dev. (https://github.com/elastic/logstash/pull/16189). To use the image append "-wolfi" to the image name: ``. docker run docker.elastic.co/logstash/logstash-wolfi:8.15.0 ``. This new image flavor builds on top of a smaller and more secure base image, and is planned to become the default fl
8.14.313 Jul 2024 05:25 minor bugfix: notable-8.14.3 . ==== Enhancements and notable. Ensure pipeline metrics are cleared on shutdown https://github.com/elastic/logstash/pull/16264. logstash-8-14-2 .
8.14.205 Jul 2024 17:25 minor bugfix: notable-8.14.2 . ==== Notable. a regression from Logstash 8.7.0 that prevented pipelines from starting when they included plugins with unicode ids https://github.com/elastic/logstash/pull/15971. a regression from Logstash 8.12.0 that prevented pipelines from starting when they included a geoip filter that used the managed databases feature after the databases had been updated https://github.com/elastic/logstash/pull/16222. an with the dead-letter queue that could cause it to emit superfluous warning messages when age-based retention was enabled while determining whether a fully-consumed segment that had already been removed was also expired https://github.com/elastic/logstash/pull/16204. ==== Plugins. *Fluent Codec - 3.4.3*. : reduce overhead of unpacking packforward-payloads by reusing a single instance https://github.com/logstash-plugins/logstash-codec-fluent/pull/32 #32 . *Elastic_integration Filter - 0.1.10*. Handling of array-type event fields by treating them as lists https://github.com/elastic/logstash-filter-elastic_integration/pull/146 #146 . Syncs with Elasticsearch 8.14, including support for new user-provided GeoIP database types `ConnectionType`, `Domain` and `Isp` https://github.com/elastic/logstash-filter-elastic_integration/pull/147 #147 . *Elasticsearch Input - 4.20.3*. DOC Update link to bypass redirect, resolving directly to correct content https://github.com/logstash-plugins/logstash-input-elasticsearch/pull/206 #206 . *Elasticsearch Output - 11.22.7*. DOC `ssl_key` requires PKCS#8 format https://github.com/logstash-plugins/logstash-output-elasticsearch/pull/1181. logstash-8-14-1 .
8.14.122 Jun 2024 00:05 minor bugfix: JSON serialization of payloads that are not UTF-8, eliminating an where the Elasticsearch Output could get stuck in a retry loop https://github.com/elastic/logstash/pull/16168. Persistent Queue in which a PQ configured with `queue.max_bytes` equal to its `queue.page_capacity` could become permanently blocked when _precisely_ full https://github.com/elastic/logstash/pull/16178. a regression in multi-local pipeline loader that caused variable-references in a configured `pipelines.yml` to not be replaced by their values in the environment and/or keystore https://github.com/elastic/logstash/pull/16201. ==== Plugins. *Elastic_integration Filter - 0.1.9*. - DOC Removes Tech Preview label and adds link to extending integrations topic in LSR https://github.com/elastic/logstash-filter-elastic_integration/pull/142 #142 . *Azure_event_hubs Input - 1.4.7*. DOCS Clarify examples for single and multiple event hubs https://github.com/logstash-plugins/logstash-input-azure_event_hubs/pull/90 #90 . DOCS Add outbound port requirements for Event Hub https://github.com/logstash-plugins/logstash-input-azure_event_hubs/pull/88 #88 . *Jdbc Integration - 5.4.11*. an in which any one instance of a JDBC input plugin using `jdbc_default_timezone` changes the behaviour of plugin instances that do _not_ use `jdbc_default_timezone`, ensuring that timezone offsets remain consistent for each instance of the plugin _as configured_ https://github.com/logstash-plugins/logstash-integration-jdbc/pull/151 #151 . an exception that could occur while reloading `jdbc_static` databases when the underlying connection to the remote has been broken https://github.com/logstash-plugins/logstash-integration-jdbc/pull/165 #165 . *Kafka Integration - 11.4.2*. Add default client_id of logstash to kafka output https://github.com/logstash-plugins/logstash-integration-kafka/pull/169 #169 . *Http Output - 5.6.1*. Added body logging for non 2xx responses https://github.com/logstash-plugin
8.14.007 Jun 2024 00:05 major feature: featured-8-14-0 . ==== Announcing the new SNMP integration plugin (Technical Preview). The new logstash-ref /plugins-integrations-snmp.html#plugins-integrations-snmp-migration `logstash-integration-snmp` plugin is now available in _Technical Preview_, and can be installed on ls 8.13. experimental . The `logstash-integration-snmp` plugin combines our. classic `logstash-input-snmp` and `logstash-input-snmptrap` plugins into a single Ruby gem at v4.0.0. Current 1.x versions of the `input-snmp` plugin are bundled with ls by default, and will soon be replaced by the 4.0.0+ version contained in this new integration. If you want to try the new `integration-snmp` plugin while it is in Technical Preview, run `bin/logstash-plugin install logstash-integration-snmp`. IMPORTANT: Before you install the new integration, be aware of logstash-ref /plugins-integrations-snmp.html#plugins-integrations-snmp-migration behavioral and mapping differences between current stand-alone plugins and the new versions included in `integration-snmp`. notable-8.14.0 . ==== Enhancements and notable. a that created duplicated `io.netty.allocator.maxOrder` system property when using environment variable `LS_JAVA_OPTS` in Docker https://github.com/elastic/logstash/pull/16079. ==== Plugins. *Jdbc Integration - 5.4.10*. DOC Added database-specific considerations https://github.com/logstash-plugins/logstash-integration-jdbc/pull/167 #167 . *Kafka Integration - 11.4.1*. Added `message_headers` option to set headers of record for Kafka output https://github.com/logstash-plugins/logstash-integration-kafka/pull/162 #162 . dependencies-8.14.0 . ==== Updates to dependencies. Update JRuby to 9.4.7.0 https://github.com/elastic/logstash/pull/16125. logstash-8-13-4 .
8.13.408 May 2024 14:45 minor feature: No user-facing changes in Logstash core. ==== Plugins. *Elasticsearch Output - 11.22.6*. DOC Logstash output.elasticsearch index can be alias or datastream https://github.com/logstash-plugins/logstash-output-elasticsearch/pull/1179. logstash-8-13-3 .
8.13.304 May 2024 02:05 minor bugfix: No user-facing changes in Logstash core. ==== Plugins. *Beats Input - 6.8.3*. Updated netty to 4.1.109 https://github.com/logstash-plugins/logstash-input-beats/pull/495 #495 . *Http Input - 3.8.1*. Updated netty to 4.1.109 https://github.com/logstash-plugins/logstash-input-http/pull/173 #173 . *Tcp Input - 6.4.2*. Updated netty to 4.1.109 https://github.com/logstash-plugins/logstash-input-tcp/pull/220 #220 . *Multiline Codec - 3.1.2*. a race condition in periodic runner that blocks clean up process and pipeline shutdown https://github.com/logstash-plugins/logstash-codec-multiline/pull/72 #72 . logstash-8-13-2 .
8.13.209 Apr 2024 17:45 minor bugfix: Update Dockerfile to publishing of docker images to Docker Hub registry https://github.com/elastic/logstash/pull/16059. ==== Plugins. *Elasticsearch Output - 11.22.5*. DOC Correctly document default data stream name (`logs-generic-default`) https://github.com/logstash-plugins/logstash-output-elasticsearch/pull/1140. DOC Add note that ecs-compatibility is required for data streams to work properly https://github.com/logstash-plugins/logstash-output-elasticsearch/pull/1174. logstash-8-13-1 .
8.13.027 Mar 2024 23:05 major feature: featured-8-13-0 . ==== New ls plugin: Elastic_integration filter. With the ls Elastic_integration filter, you can take advantage of the features in integrations-docs Elastic integrations without missing out on the advanced processing capabilities offered by ls . Together, Elastic integrations and ls can help you solve a wide range of use cases, including air gapped environments, data privacy processing, and multiple destinations. Your ls pipeline would include the elastic_agent input, the elastic_integration filter, and the elasticsearch output, in addition to any other plugins you want to use. Check out and the plugin docs for details. notable-8.13.0 . ==== Enhancements and notable. Flow metrics added `worker_utilization` to pipeline-level https://github.com/elastic/logstash/pull/15912. Ensure that the order of JVM options from both files and environment variables is respected https://github.com/elastic/logstash/pull/15997. plugin-changes-8.13.0 . ==== Changes to bundled plugins. Packaged `filter-elastic_integration` as a bundled plugin in preparation for GA. https://github.com/elastic/logstash/pull/15769. Unbundled `logstash-input-imap` plugin https://github.com/elastic/logstash/pull/15985. ==== Plugins. *Elastic_integration Filter - 0.1.6*. The elastic_integration filter is generally available and bundled with Logstash 8.13.0. Using this filter, ls can process data collected by integrations-docs Elastic integrations . + When you configure the filter to point to an es cluster, it auto-detects the event s data-stream to determine what integration processing (if any) should be executed for each event. It performs that processing inside ls without transmitting the event to es . Events that are successfully handled are tagged so that any downstream es output in the ls pipeline will not re-run the integration in es . where configured `username`/`password` credentials was not sent to Elasticsearch instances that ha
8.12.225 Feb 2024 01:25 minor bugfix: Set Netty's maxOrder options to previous default value of 11 https://github.com/elastic/logstash/pull/15928. Add "openssl" to UBI docker images https://github.com/elastic/logstash/pull/15929. ==== Plugins. *Jdbc Integration - 5.4.8*. Update Derby to 10.15.2.1 https://github.com/logstash-plugins/logstash-integration-jdbc/pull/155 #155 . Update sequel version to = 5.74.0, that allows the generic jdbc adapter to better handle disconnect errors https://github.com/logstash-plugins/logstash-integration-jdbc/pull/153 #153 . *Kafka Integration - 11.3.4*. "retries" and "value_serializer" error handling in output plugin https://github.com/logstash-plugins/logstash-integration-kafka/pull/160 #160 . logstash-8-12-1 .
8.12.107 Feb 2024 07:25 minor bugfix: Updates bundled JDK https://github.com/elastic/logstash/pull/15840. ==== Plugins. *Http Filter - 1.5.1*. Don't process response when the body is empty. https://github.com/logstash-plugins/logstash-filter-http/pull/50 #50 . *Syslog_pri Filter - 3.2.1*. Remove spurious leftover text from "use_labels" docs https://github.com/logstash-plugins/logstash-filter-syslog_pri/pull/15 #15 . *Logstash Integration - 1.0.2*. : input plugin now correctly applies common event decorators `type`, `tags`, and `add_field` to events after receiving them https://github.com/logstash-plugins/logstash-integration-logstash/pull/21 #21 . logstash-8-12-0 .
8.12.021 Jan 2024 03:15 major feature: features-8.12.0 . ==== New features and enhancements. Add support for adding and removing multiple keystore keys in a single operation https://github.com/elastic/logstash/pull/15739. Docker: Update Iron Bank base image to ubi9.2 https://github.com/elastic/logstash/pull/15490. Internal: extract GeoIP database manager to stand-alone feature https://github.com/elastic/logstash/pull/15348. notable-8.12.0 . ==== Notable. Add missing method of logger wrapper for puma https://github.com/elastic/logstash/pull/15640. logstash-keystore multiple keys operations with command flags https://github.com/elastic/logstash/pull/15737. Separate scheduling of segments flushes from time https://github.com/elastic/logstash/pull/15697. Add system properties to configure Jackson's stream read constraints https://github.com/elastic/logstash/pull/15763. with Jackson 2.15: Can not write a field name, expecting a value https://github.com/elastic/logstash/pull/15564. dependencies-8.12.0 . ==== Updates to dependencies. Add bigdecimal 3.1 dependency. https://github.com/elastic/logstash/pull/15384. Update Guava dependency to 32.1.2 https://github.com/elastic/logstash/pull/15394. Swap dataformat-yaml with snakeyaml https://github.com/elastic/logstash/pull/15606. Bump Puma to 6.4.2+ https://github.com/elastic/logstash/pull/15776. Update jackson to 2.15.3 https://github.com/elastic/logstash/pull/15477. docs-8.12.0 . ==== Documentation enhancements. Add https://www.elastic.co/guide/en/logstash/8.12/running-logstash-kubernetes.html info and link to ls running on a k8s cluster through. eck (ECK) https://github.com/elastic/logstash/pull/15565. Add info for sending https://www.elastic.co/guide/en/logstash/current/serverless-monitoring-with-elastic-agent.html ls monitoring data to Elastic serverless-short https://github.com/elastic/logstash/pull/15636. Add docs for https://www.elastic.co/guide/en/logstash/current/ea-integrations.html extending integrations with
8.11.417 Jan 2024 18:11 minor bugfix: No user-facing changes in Logstash core. Plugins Netflow Codec - 4.3.2 Updates the milliseconds rounding for IPFIX start/end milliseconds fields. Fix the test to run on Logstash 8 with microseconds precision. #206 Fixed unable to initialize the plugin with Logstash 8.10+ #205 Json Filter - 3.2.1 Fix tag on failure test #52 File Input - 4.4.6 Change read mode to immediately stop consuming buffered lines when shutdown is requested #322 Twitter Input - 4.1.1 Bumped public_suffix gem version to 4 6 #77 Csv Output - 3.0.10 Extend spreadsheet_safe prefix guard to '-', '+', and '@' #27