forked from elastic/beats
-
Notifications
You must be signed in to change notification settings - Fork 0
/
CHANGELOG.asciidoc
9750 lines (6750 loc) · 443 KB
/
CHANGELOG.asciidoc
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
// Use these for links to issue and pulls. Note issues and pulls redirect one to
// each other on Github, so don't worry too much on using the right prefix.
:issue: https://github.com/elastic/beats/issues/
:pull: https://github.com/elastic/beats/pull/
[[release-notes-8.8.2]]
=== Beats version 8.8.2
https://github.com/elastic/beats/compare/v8.8.1\...v8.8.2[View commits]
==== Bugfixes
*Affecting all Beats*
- Make sure k8s watchers are closed when closing k8s meta processor. {pull}35630[35630]
- Upgraded apache arrow library used in `x-pack/libbeat/reader/parquet` from v11 to v12.0.1 in order to fix cross-compilation issues. {pull}35640[35640]
- Fix panic when the disk queue's MaxRetryInterval configuration is specified, but RetryInterval is not. {pull}35820[35820]
*Filebeat*
- Fix syslog message parsing for fortinet.firewall to take into account quoted values. {pull}35522[35522]
- [Filebeat GCS input] Fixed an issue where `bucket_timeout` was being applied to the entire bucket poll interval and not individual bucket object read operations. Fixed a map write concurrency issue arising from data races when using a high number of workers. Fixed the flaky tests that were present in the GCS test suit. {pull}35605[35605]
- Fix handling of IPv6 unspecified addresses in TCP input. {issue}35064[35064] {pull}35637[35637]
- Improve error reporting and fix IPv6 handling of TCP and UDP metric collection. {pull}35772[35772]
- Fix input reload on autodiscover. {issue}34388[34388] {pull}35645[35645]
*Heartbeat*
- Fix serialization of processors when running diagnostics. {pull}35698[35698]
*Metricbeat*
- Fix calculation of the `host.cpu.usage` metric for EC2. {pull}35717[35717]
==== Added
*Affecting all Beats*
- Update Go version to 1.19.10. {pull}35751[35751]
*Filebeat*
- [GCS] Added scheduler debug logs and improved the context passing mechanism by removing them from struct params and passing them as function arguments. {pull}35674[35674]
*Packetbeat*
- Add AF_PACKET metrics. {issue}35428[35428] {pull}35489[35489]
==== Deprecated
*Heartbeat*
- Removed support for `zip_url` and `local` browser sources. {pull}35429[35429]
[[release-notes-8.8.1]]
=== Beats version 8.8.1
https://github.com/elastic/beats/compare/v8.8.0\...v8.8.1[View commits]
==== Bugfixes
*Affecting all Beats*
- 'add_cloud_metadata' processor: add `cloud.region` field for GCE cloud provider
- 'add_cloud_metadata' processor: update Azure metadata API version to get missing `cloud.account.id` field
*Filebeat*
- Fix "Can only start an input when all related states are finished" error when running under Elastic Agent {pull}35250[35250] {issue}33653[33653]
- [system] Sync system/auth dataset with system integration 1.29.0. {pull}35581[35581]
- Fix filestream false positive log error "filestream input with ID 'xyz' already exists" {issue}31767[31767]
- Fix error when trying to use `include_message` parser {issue}35440[35440]
==== Added
*Filebeat*
- Add sanitization capabilities to azure-eventhub input {pull}34874[34874]
*Auditbeat*
- Migration of system/package module storage from gob encoding to flatbuffer encoding in bolt db. {pull}34817[34817]
*Metricbeat*
- Support collecting metrics from both the monitoring account and linked accounts from AWS CloudWatch. {pull}35540[35540]
- Add new parameter `include_linked_accounts` to enable/disable metrics collection from multiple linked AWS Accounts {pull}35648[35648]
[[release-notes-8.8.0]]
=== Beats version 8.8.0
https://github.com/elastic/beats/compare/v8.7.1...v8.8.0[View commits]
==== Bugfixes
*Affecting all Beats*
- Fix race condition when stopping runners {pull}32433[32433]
- Fix concurrent map writes when system/process code called from reporter code {pull}32491[32491]
- The Elasticsearch output now splits large requests instead of dropping them when it receives a StatusRequestEntityTooLarge error. {pull}34911[34911]
- In cases where the matcher detects a non-string type in a match statement, report the error as a debug statement, and not a warning statement. {pull}35119[35119]
- `add_cloud_metadata` processor: Add `cloud.region` field for GCE cloud provider.
- `add_cloud_metadata` processor: Update Azure metadata API version to get missing `cloud.account.id` field.
*Filebeat*
- [GCS Input] Added missing locks for safe concurrency. {pull}34914[34914]
- Fix the `ignore_inactive` option being ignored in Filebeat's filestream input. {pull}34770[34770]
- Add input instance ID to request trace filename for httpjson and cel inputs. {pull}35024[35024]
- Sanitize filenames for request tracer in httpjson input. {pull}35143[35143]
- Sanitize filenames for request tracer in cel input. {pull}35154[35154]
- Fix the grok expression outputs of log files. {pull}35221[35221]
- Move repeated Windows event channel not found errors in winlog input to debug level. {issue}35314[35314] {pull}35317[35317]
- Fix crash when processing forwarded logs missing a message. {issue}34705[34705] {pull}34865[34865]
- Fix crash when loading azurewebstorage cursor with no partially processed data. {pull}35433[35433]
*Heartbeat*
- Fix panics when parsing when HTTP URL is not parseable. {pull}34702[34702]
- Fix broken state ID location naming. {pull}35336[35336]
- Fix project monitor temp directories permission to include group access. {pull}35398[35398]
- Fix output pipeline exit on `run_once`. {pull}35376[35376]
- Fix formatting issue with socket trace timeout. {pull}35434[35434]
*Metricbeat*
- Make generic SQL GA. {pull}34637[34637]
- Collect missing `remote_cluster` in Elasticsearch CCR metricset. {pull}34957[34957]
- Add context with timeout in AWS API calls. {pull}35425[35425]
*Osquerybeat*
- Adds the `elastic_file_analysis` table to the Osquery extension for macOS builds. {pull}35056[35056]
*Packetbeat*
- Fix BPF filter setting not being applied to sniffers. {issue}35363[35363] {pull}35484[35484]
*Winlogbeat*
- Move repeated channel not found errors to debug level. {issue}35314[35314] {pull}35317[35317]
- Fix panic due to misrepresented buffer use. {pull}35437[35437]
- Allow program termination when attempting to open an absent channel. {pull}35474[35474]
==== Added
*Filebeat*
- Add metric `sqs_messages_waiting_gauge` for aws-s3 input. {pull}34488[34488]
- Add support for Okta debug attributes, `risk_reasons`, `risk_behaviors` and `factor`. {issue}33677[33677] {pull}34508[34508]
- Add `nginx.ingress_controller.upstream.ip` to `related.ip` {issue}34645[34645] {pull}34672[34672]
- Include NAT and firewall IPs in `related.ip` in Fortinet Firewall module. {issue}34640[34640] {pull}34673[34673]
- Add UNIX socket log parsing for NGINX `ingress_controller`. {pull}34732[34732]
- Add metric `sqs_worker_utilization` for aws-s3 input. {pull}34793[34793]
- Register MIME handlers for CSV types in CEL input. {pull}34934[34934]
- Add MySQL authentication message parsing and `related.ip` and `related.user` fields. {pull}34810[34810]
- Mention `mito` CEL tool in CEL input docs. {pull}34959[34959]
- Add nginx ingress_controller parsing if one of upstreams fails to return response. {pull}34787[34787]
- Allow neflow v9 and ipfix templates to be shared between source addresses. {pull}35036[35036]
- Add support for collecting IPv6 metrics. {pull}35123[35123]
- Add Oracle authentication messages parsing {pull}35127[35127]
*Heartbeat*
- Add status to monitor run log report.
- Remov Beta label for browser monitors. {pull}35424[35424].
*Metricbeat*
- Add GCP Carbon Footprint metricbeat data. {pull}34820[34820]
- Add event loop utilization metric to Kibana module. {pull}35020[35020]
*Winlogbeat*
- Add `event.category` and `event.type` to Sysmon module for EventIDs 8, 9, 19, 20, 27, 28, 255. {pull}35193[35193]
[[release-notes-8.7.1]]
=== Beats version 8.7.1
https://github.com/elastic/beats/compare/v8.7.0\...v8.7.1[View commits]
==== Bugfixes
*Affecting all Beats*
- Fix Beats started by agent not respecting the allow_older_versions: true configuration flag {issue}34227[34227] {pull}34964[34964]
- Beats started by agent now configure one global instance of each default add_x_metadata processor instead of creating one processor per input. Fixes a significant performance degradation for the aws-s3 input. {issue}35000[35000] {pull}35031[35031]
*Filebeat*
- [GCS Input] Added missing locks for safe concurrency {pull}34914[34914]
- Add input instance id to request trace filename for httpjson and cel inputs {pull}35037[35037]
- Fix panic in TCP and UDP inputs on Linux when collecting socket metrics from OS. {issue}35064[35064]
- Correctly collect TCP and UDP metrics for unspecified address values. {pull}35111[35111]
- Fix base for UDP and TCP queue metrics and UDP drops metric. {pull}35123[35123]
- Sanitize filenames for request tracer in httpjson and cel inputs. {pull}35143[35143]
- Fix handling of MySQL audit logs with strict JSON parser. {issue}35158[35158] {pull}35160[35160]
- decode_cef processor: Fix ECS output by making `observer.ip` into an array of strings instead of string. {issue}35140[35140] {pull}35149[35149]
- Fix accidental error overwrite in defer statement in entityanalytics Azure AD input. {issue}35153[35153] {pull}35169[35169]
*Heartbeat*
- Fix issue using projects in airgapped environments by disabling npm audit. {pull}34936[34936]
*Packetbeat*
- Fix documentation for `flows.period` related to flow reporting. {pull}35009[35009]
==== Added
*Affecting all Beats*
- Allow users to enable features via configuration, starting with the FQDN reporting feature. {issue}1070[1070] {pull}34456[34456]
*Winlogbeat*
- Add note in documentation about 21 event ID clause limit {issue}35048[35048] {pull}35049[35049]
[[release-notes-8.7.0]]
=== Beats version 8.7.0
https://github.com/elastic/beats/compare/v8.6.2\...v8.7.0[View commits]
==== Bugfixes
*Affecting all Beats*
- Fix dropped events when monitor a beat under the agent and send its `Host info` log entry. {pull}34599[34599]
- Fix panics when a processor is closed twice {pull}34647[34647]
- Update elastic-agent-system-metrics to v0.4.6 to allow builds on mips platforms. {pull}34674[34674]
*Filebeat*
- [Auditbeat System Package] Added support for Apple Silicon chips. {pull}34433[34433]
with the ecs field name `container`. {pull}34403[34403]
automatic splitting at root level, if root level element is an array. {pull}34155[34155]
- Prevent Elasticsearch from spewing log warnings about redundant wildcard when setting up ingest pipelines. {issue}34249[34249] {pull}34550[34550]
- Gracefully handle Windows event channel not found errors in winlog input. {issue}30201[30201] {pull}34605[34605]
- Fix the issue of `cometd` input worker getting closed in case of a network connection issue and an EOF error. {issue}34326[34326] {pull}34327[34327]
- Fix for httpjson first_response object throwing false positive errors by making it a flag based object {issue}34747[34747] {pull}34748[34748]
- Fix errors and panics due to re-used processors {pull}34761[34761]
- Add missing Basic Authentication support to CEL input {issue}34609[34609] {pull}34689[34689]
*Heartbeat*
- Fix integration hashing to prevent reloading all when updated. {pull}34697[34697]
- Fix release of job limit semaphore when context is cancelled. {pull}34697[34697]
with the ecs field name `container`. {pull}34403[34403]
automatic splitting at root level, if root level element is an array. {pull}34155[34155]
- Fix broken mapping for state.ends field. {pull}34891[34891]
*Filebeat*
- Allow the `misp` fileset in the Filebeat `threatintel` module to ignore CIDR ranges for an IP field. {issue}29949[29949] {pull}34195[34195]
- Remove incorrect reference to CEL ext extensions package. {issue}34610[34610] {pull}34620[34620]
- Fix handling of RFC5988 links' relation parameters by `getRFC5988Link` in HTTPJSON. {issue}34603[34603] {pull}34622[34622]
- Drop empty API response events for Microsoft module. {issue}34786[34786] {pull}34893[34893]
*Metricbeat*
- Fix kafka dashboard field names {pull}33555[33555]
*Winlogbeat*
- Fix handling of event data with keys containing dots. {issue}34345[34345] {pull}34549[34549]
- Gracefully handle channel not found errors. {issue}30201[30201] {pull}34605[34605]
- Clarify query term limits warning and remove link to missing Microsoft doc page. {pull}34715[34715]
- Improve documentation for event_logs.name configuration. {pull}34931[34931]
*Functionbeat*
- Fix Kinesis events timestamp to use timestamp of the event record instead of when the record was processed {pull}33593[33593]
==== Added
*Filebeat*
- Add backup to bucket and delete functionality for the `aws-s3` input. {issue}30696[30696] {pull}33559[33559]
- Add support for polling system UDP stats for UDP input metrics. {pull}34070[34070]
- Add support for recognizing the log level in Elasticsearch JVM logs {pull}34159[34159]
- Add new Entity Analytics input with Azure Active Directory support. {pull}34305[34305]
- Added metric `sqs_lag_time` for aws-s3 input. {pull}34306[34306]
- Add metrics for TCP packet processing. {pull}34333[34333]
- Add metrics for unix socket packet processing. {pull}34335[34335]
- Add beta `take over` mode for `filestream` for simple migration from `log` inputs {pull}34292[34292]
- Add pagination support for Salesforce module. {issue}34057[34057] {pull}34065[34065]
- Allow users to redact sensitive data from CEL input debug logs. {pull}34302[34302]
- Add support for new Rabbitmq timestamp format for logs {pull}34211[34211]
- Allow user configuration of timezone offset in Cisco ASA and FTD modules. {pull}34436[34436]
- Allow user configuration of timezone offset in Checkpoint module. {pull}34472[34472]
- Fill okta.request.ip_chain.* as a flattened object in Okta module. {pull}34621[34621]
- Fixed GCS log format issues. {pull}34659[34659]
- Add Basic Authentication support on constructed requests to CEL input {issue}34609[34609] {pull}34689[34689]
- Add string manipulation extensions to CEL input {issue}34610[34610] {pull}34689[34689]
- Improve CEL input documentation {pull}34831[34831]
- Add metrics documentation for CEL and AWS CloudWatch inputs. {issue}34887[34887] {pull}34889[34889]
- Metrics hosted by the HTTP monitoring endpoint for the `aws-cloudwatch`, `aws-s3`, `cel`, and `lumberjack` inputs are now available under `/inputs/` instead of `/dataset`.
*Heartbeat*
- Users can now configure max scheduler job limits per monitor type via env var. {pull}34307[34307]
- Remove host and port matching restrictions on hint-generated monitors. {pull}34376[34376]
*Metricbeat*
- Remove GCP Compute metadata cache {pull}33655[33655]
- Add GCP Redis regions support {pull}33728[33728]
- Changed cloudwatch module to call ListMetrics API only once per region, instead of per AWS namespace {pull}34055[34055]
- Add beta ingest_pipeline metricset to Elasticsearch module for ingest pipeline monitoring {pull}34012[34012]
- Handle duplicated TYPE line for prometheus metrics {issue}18813[18813] {pull}33865[33865]
*Packetbeat*
- Reduce logging level for ENOENT to WARN when mapping sockets to processes. {issue}33793[33793] {pull}33854[33854]
- Add metrics for TCP and UDP packet processing. {pull}33833[33833] {pull}34353[34353]
- Allow user to prevent Npcap library installation on Windows. {issue}34420[34420] {pull}34428[34428]
- Add metrics documentation for TCP and UDP protocols. {issue}34887[34887] {pull}34889[34889]
*Winlogbeat*
- Add metrics for log event processing. {pull}33922[33922]
- Add metrics documentation for event processing. {issue}34887[34887] {pull}34889[34889]
- Added processing for Windows Event ID's 4797, 5379, 5380, 5381, and 5382 for the Security Ingest Pipeline {issue}34293[34293] {pull}34294[34294]
- Added processing for Windows Event ID's 5140 and 5145 for the Security Ingest Pipeline {pull}34352[34352]
[[release-notes-8.6.2]]
=== Beats version 8.6.2
https://github.com/elastic/beats/compare/v8.6.1\...v8.6.2[View commits]
==== Bugfixes
*Affecting all Beats*
- Fix lockfile logic, retry locking {pull}34194[34194]
- Log errors from the Elastic Agent V2 client errors channel. Avoids blocking when error occurs communicating with the Elastic Agent. {pull}34392[34392]
- Only log publish event messages in trace log level under elastic-agent. {pull}34391[34391]
- Fix issue where updating a single Elastic Agent configuration unit results in other units being turned off. {pull}34504[34504]
*Auditbeat*
*Filebeat*
- [Azure blob storage] Changed logger field name from `container` to `container_name` so that it does not clash
- [GCS] Added support for more mime types & introduced offset tracking via cursor state. Also added support for
- [httpsjon] Improved error handling during pagination with chaining & split processor {pull}34127[34127]
- Fix EOF on single line not producing any event. {issue}30436[30436] {pull}33568[33568]
- Fix handling of error in states in direct aws-s3 listing input {issue}33513[33513] {pull}33722[33722]
- Fix `httpjson` input page number initialization and documentation. {pull}33400[33400]
- Fix reporting of `filebeat.events.active` in log events such that the current value is always reported instead of the difference from the last value. {pull}33597[33597]
- Fix splitting array of strings/arrays in httpjson input {issue}30345[30345] {pull}33609[33609]
- Fix Google workspace pagination and document ID generation. {pull}33666[33666]
- Update mito CEL extension library to v0.0.0-20221207004749-2f0f2875e464 {pull}33974[33974]
- [azure-eventhub input] Switch the run EPH run mode to non-blocking {pull}34075[34075]
- Fixing system tests not returning expected content encoding for azure blob storage input. {pull}34412[34412]
- [Azure Logs] Fix authentication_processing_details parsing in sign-in logs. {issue}34330[34330] {pull}34478[34478]
*Heartbeat*
- Fix broken zip URL monitors. NOTE: Zip URL Monitors will be removed in version 8.7 and replaced with project monitors. {pull}33723[33723]
- Fix bug where states.duration_ms was incorrect type. {pull}33563[33563]
- Fix handling of long UDP messages in UDP input. {issue}33836[33836] {pull}33837[33837]
- Fix beat capabilities on Docker image. {pull}33584[33584]
- Fix serialization of state duration to avoid scientific notation. {pull}34280[34280]
- Enable nodejs engine strict validation when bundling synthetics. {pull}34470[34470]
*Metricbeat*
- Fix metrics split through different events and metadata not matching for aws cloudwatch. {pull}34483[34483]
- Fix metadata enricher with correct container ids for pods with multiple containers in container metricset. Align `kubernetes.container.id` and `container.id` fields for state_container metricset. {pull}34516[34516]
*Packetbeat*
- Fixed a race condition in Packetbeat that could cause crashes or instability {pull}34514[34514]
==== Added
*Filebeat*
- Added support for HTTP destination override to Google Cloud Storage input. {pull}34413[34413]
[[release-notes-8.6.1]]
=== Beats version 8.6.1
https://github.com/elastic/beats/compare/v8.6.0\...v8.6.1[View commits]
==== Bugfixes
*Affecting all Beats*
- Fix unnecessarily restarting every time a new configuration was received from the Elastic Agent. {pull}34346[34346]
*Filebeat*
- [google_workspace] Fix pagination and cursor value update. {pull}34274[34274]
- Fix handling of quoted values in auditd module. {issue}22587[22587] {pull}34069[34069]
- The `close.on_state_change.inactive` default value is now set to 5 minutes, matching the documentation.
- Fix handling of null or empty arrays during split with keep parent option. {pull}34322[34322]
*Metricbeat*
- Support Oracle-specific connection strings in SQL module {issue}32089[32089] {pull}32293[32293]
- Remove deprecated metrics from controller manager, scheduler and proxy {pull}34161[34161]
*Osquerybeat*
- Fix data_stream configuration, enforce the default values used before 8.6.0. {pull}34246[34246]
*Winlogbeat*
- Corrects issue with security events with source IP of "LOCAL" or "Unknown" failing to ingest {issue}19627[19627] {pull}34295[34295]
==== Added
*Filebeat*
- Allow user configuration of keep-alive behaviour for HTTPJSON and CEL inputs. {issue}33951[33951] {pull}34014[34014]
- [GCS] Added support for more mime types & introduced offset tracking via cursor state. Also added support for automatic splitting at root level, if root level element is an array. {pull}34155[34155]
[[release-notes-8.6.0]]
=== Beats version 8.6.0
https://github.com/elastic/beats/compare/v8.5.3\...v8.6.0[View commits]
==== Bugfixes
*Affecting all Beats*
- Fix Windows service install/uninstall when Win32_Service returns error, add logic to wait until the Windows Service is stopped before proceeding. {pull}33322[33322]
- Support for multiline zookeeper logs. {issue}2496[2496]
- Allow `clock_nanosleep` in the default seccomp profiles for amd64 and 386. Newer versions of glibc (e.g. 2.31) require it. {issue}33792[33792]
- Disable lockfile when running under elastic-agent. {pull}33988[33988]
*Filebeat*
- [httpsjon] Improved error handling during pagination with chaining & split processor. {pull}34127[34127]
- [Azure blob storage] Added support for more mime types & introduced offset tracking via cursor state. {pull}33981[33981]
- Fix handling of error in states in direct aws-s3 listing input. {issue}33513[33513] {pull}33722[33722]
- Fix PANW handling of messages with event.original already set. {issue}33829[33829] {pull}33830[33830]
- Rename identity as identity_name when the value is a string in Azure Platform Logs. {pull}33654[33654]
- Fix 'requires pointer' error while getting cursor metadata. {pull}33956[33956]
- Fix input cancellation handling when HTTP client does not support contexts. {issue}33962[33962] {pull}33968[33968]
- Update mito CEL extension library to v0.0.0-20221207004749-2f0f2875e464. {pull}33974[33974]
- Fix CEL result deserialisation when evaluation fails. {issue}33992[33992] {pull}33996[33996]
- Fix handling of non-200/non-429 status codes. {issue}33999[33999] {pull}34002[34002]
- [azure-eventhub input] Switch the run EPH run mode to non-blocking. {pull}34075[34075]
*Heartbeat*
- Fix browser monitor summary reporting as up when monitor is down. {issue}33374[33374] {pull}33819[33819]
*Packetbeat*
- Fix panic on memcache transaction with no request or response. {issue}33852[33852] {pull}33853[33853]
- Fix termination logic. {pull}33979[33979]
==== Added
*Affecting all Beats*
- Add `http.pprof` config options for enabling block and mutex profiling. {issue}33572[33572] {pull}33576[33576]
- Add `add_formatted_index` processor that allows the resulting index for an event to be changed based on content from the event. {pull}33800[33800]
- deps: Updated to github.com/elastic/go-sysinfo v1.9.0. {pull}33864[33864]
- Fix panic due to close of already closed channel during shutdown. {pull}33971[33971]
*Auditbeat*
- Add file parser processor to file_integrity module. {pull}28802[28802]
- Improve documentation for symlink handling behaviour in file integrity module. {pull}33430[33430]
- Ensure file integrity module watch paths are absolute. {pull}33430[33430]
*Filebeat*
- Add `text/csv` decoder to `httpjson` input. {pull}28564[28564]
- Update `aws-s3` input to connect to non AWS S3 buckets. {issue}28222[28222] {pull}28234[28234]
- Add support for '/var/log/pods/' path for add_kubernetes_metadata processor with `resource_type: pod`. {pull}28868[28868]
- Add documentation for add_kubernetes_metadata processors `log_path` matcher. {pull}28868[28868]
- Add support for parsers on journald input. {pull}29070[29070]
- Add support in httpjson input for oAuth2ProviderDefault of password grant_type. {pull}29087[29087]
- threatintel module: Add new Recorded Future integration. {pull}30030[30030]
- Allow iptables module to parse ulogd v2 TOS field in logs. {pull}32126[32126]
- Fix handling of invalid UserIP and LocalIP values. {pull}32896[32896]
- Allow http_endpoint instances to share ports. {issue}32578[32578] {pull}33377[33377]
- Improve httpjson documentation for split processor. {pull}33473[33473]
- Added separation of transform context object inside httpjson. Introduced new clause `.parent_last_response.*`. {pull}33499[33499]
- Cloud Foundry input uses server-side filtering when retrieving logs. {pull}33456[33456]
- Add `parse_aws_vpc_flow_log` processor. {pull}33656[33656]
- Update `aws.vpcflow` dataset in AWS module have a configurable log `format` and to produce ECS 8.x fields. {pull}33699[33699]
- Modified `aws-s3` input to reduce mutex contention when multiple SQS message are being processed concurrently. {pull}33658[33658]
- Disable "event normalization" processing for the aws-s3 input to reduce allocations. {pull}33673[33673]
- Add Common Expression Language input. {pull}31233[31233]
- Add support for http+unix and http+npipe schemes in httpjson input. {issue}33571[33571] {pull}33610[33610]
- Add support for http+unix and http+npipe schemes in cel input. {issue}33571[33571] {pull}33712[33712]
- Add `decode_duration`, `move_fields` processors. {pull}31301[31301]
- Add metrics for UDP packet processing. {pull}33870[33870]
- Convert UDP input to v2 input. {pull}33930[33930]
- Improve collection of risk information from Okta debug data. {issue}33677[33677] {pull}34030[34030]
- Adding filename details from zip to response for httpjson. {issue}33952[33952] {pull}34044[34044]
*Heartbeat*
- Upgrade node to 18.12.0.
*Metricbeat*
- Add Data Granularity option to AWS module to allow for for fewer API calls of longer periods and keep small intervals. {issue}33133[33133] {pull}33166[33166]
- Update README file on how to run Metricbeat on Kubernetes. {pull}33308[33308]
- Add per-thread metrics to system_summary. {pull}33614[33614]
- Add GCP CloudSQL metadata. {pull}33066[33066]
- Add support for multiple regions in GCP. {pull}32964[32964]
- Add namespace metadata to all namespaced kubernetes resources. {pull}33763[33763]
*Packetbeat*
- Add fragmented IPv4 packet reassembly. {issue}33012[33012] {pull}33296[33296]
[[release-notes-8.5.3]]
=== Beats version 8.5.3
https://github.com/elastic/beats/compare/v8.5.2\...v8.5.3[View commits]
==== Breaking changes
*Filebeat*
- Fixed error spam from `add_kubernetes_metadata` processor when running on AKS. {pull}33697[33697]
==== Bugfixes
*Heartbeat*
- Fix broken zip URL monitors. NOTE: Zip URL Monitors will be removed in version 8.7 and replaced with project monitors. {pull}33723[33723]
- Fix handling of long UDP messages in UDP input. {issue}33836[33836] {pull}33837[33837]
[[release-notes-8.5.2]]
=== Beats version 8.5.2
https://github.com/elastic/beats/compare/v8.5.1\...v8.5.2[View commits]
[[release-notes-8.5.1]]
=== Beats version 8.5.1
https://github.com/elastic/beats/compare/v8.5.0\...v8.5.1[View commits]
==== Bugfixes
*Affecting all Beats*
- Re-enable build optimizations to reduce binary size and improve performance. {pull}33620[33620]
- Expand fields in `decode_json_fields` if target is set. {issue}31712[31712] {pull}32010[32010]
- Fix in AWS related services initialisation relying on custom endpoint resolver. {issue}32888[32888] {pull}32921[32921]
- Keep `orchestrator.cluster.name` if `kubeconfig` is not returned in GKE metadata. {pull}33418[33418]
*Filebeat*
- Fix `httpjson` input page number initialization and documentation. {pull}33400[33400]
- Add handling of AAA operations for Cisco ASA module. {issue}32257[32257] {pull}32789[32789]
- Fix gc.log always shipped even if gc fileset is disabled. {issue}30995[30995]
- Fix handling of empty array in httpjson input. {pull}32001[32001]
- Fix reporting of `filebeat.events.active` in log events such that the current value is always reported instead of the difference from the last value. {pull}33597[33597]
- Fix splitting array of strings/arrays in httpjson input. {issue}30345[30345] {pull}33609[33609]
- Fix Google workspace pagination and document ID generation. {pull}33666[33666]
*Heartbeat*
- Fix bug affecting let's encrypt and other users of cross-signed certs, where cert expiration was incorrectly calculated. {issue}33215[33215]
- Fix broken disable feature for kibana configured monitors. {pull}33293[33293]
- Fix states client support for output options. {pull}33405[33405]
- Fix states client reloader under managed mode. {pull}33405[33405]
- Fix bug where states.duration_ms was incorrect type. {pull}33563[33563]
==== Bugfixes
*Heartbeat*
- Fix bug affecting let's encrypt and other users of cross-signed certs, where cert expiration was incorrectly calculated. {issue}33215[33215]
- Fix broken disable feature for kibana configured monitors. {pull}33293[33293]
- Fix states client support for output options. {pull}33405[33405]
- Fix states client reloader under managed mode. {pull}33405[33405]
*Metricbeat*
- Add tags to events based on parsed identifier. {pull}33472[33472]
- Skip over unsupported filesystems in the system.filesystem metricset instead of failing immediately. Fix debug statement in system.fsstat metricset. {pull}33646[33646]
==== Added
*Affecting all Beats*
- Beats will now attempt to recover if a lockfile has not been removed. {pull}[33169]
[[release-notes-8.5.0]]
=== Beats version 8.5.0
https://github.com/elastic/beats/compare/v8.4.3\...v8.5.0[View commits]
==== Breaking changes
*Affecting all Beats*
- Upgrade to Go 1.18. Certificates signed with SHA-1 are now rejected. See the Go 1.18 https://tip.golang.org/doc/go1.18#sha1[release notes] for details. {pull}32493[32493]
- Fix formatting of MAC hardware addresses populated by the add_host_metadata processor. {issue}32264[32264] {pull}32265[32265]
==== Bugfixes
*Affecting all Beats*
- Fix metric namespacing for self-monitoring to correct some process incorrectly reading as zero. {pull}32336[32336]
*Auditbeat*
- Fix rendering of MAC addresses to conform to ECS. {issue}32621[32621] {pull}32622[32622]
*Filebeat*
- Fix rendering of MAC addresses to conform to ECS. {issue}32621[32621] {pull}32622[32622]
- Import dashboards from CEF integration. {pull}32766[32766]
- Fix how to handle IPv6 addresses in the fileset `nginx/ingress_controller` for Filebeat. {pull}32989[32989]
- Fix requestID parsing in AWS cloudtrail fileset. {pull}33143[33143]
- Fix input metrics not being unregistered when an input closes. This led to panics when configuration was reloaded for the aws-s3, aws-cloudwatch, and lumberjack inputs. {pull}33259[33259]
- Add handling of AAA operations for Cisco ASA module. {issue}32257[32257] {pull}32789[32789]
- Fix gc.log always shipped even if gc fileset is disabled {issue}30995[30995]
- Fix handling of Cisco 302020 messages in ASA and FTD modules. {pull}33089[33089]
*Heartbeat*
- Fix bug affecting Let's Encrypt and other users of cross-signed certs, where cert expiration was incorrectly calculated. {issue}33215[33215]
- Fix broken disable feature for Kibana-configured monitors. {pull}33293[33293]
*Metricbeat*
- Fix GCP storage field naming {pull}32806[32806]
- In `module/windows/perfmon`, changed collection method of the second counter value required to create a displayable value {pull}32305[32305]
- Change max query size for GetMetricData API to 500 and add RecentlyActive for ListMetrics API call. {pull}33105[33105]
- Add GCP CloudSQL region filter. {pull}32943[32943]
- Fix Logstash cgroup mappings. {pull}33131[33131]
- Remove unused `elasticsearch.node_stats.indices.bulk.avg_time.bytes` mapping. {pull}33263[33263]
*Packetbeat*
- Fix formatting of debug logs. {pull}32698[32698]
- Fix rendering of MAC addresses to conform to ECS. {issue}32621[32621] {pull}32622[32622]
*Winlogbeat*
- Reduce severity of message salvage failure logging. {pull}32697[32697]
==== Added
*Filebeat*
- Import dashboard from Fortinet Fortigate firewall integration. {issue}19810[19810] {pull}33003[33003]
*Heartbeat*
- Add new states field for internal use by new synthetics app. {pull}30632[30632]
*Packetbeat*
- Add option to allow sniffer to change device when default route changes. {issue}31905[31905] {pull}32681[32681]
- Add option to allow sniffing multiple interface devices. {issue}31905[31905] {pull}32933[32933]
- Bump Windows Npcap version to v1.71. {issue}33164[33164] {pull}33172[33172]
==== Deprecated
*Heartbeat*
- Deprecate `zip_url` and local monitor options. {pull}33123[33123]
[[release-notes-8.4.3]]
=== Beats version 8.4.3
https://github.com/elastic/beats/compare/v8.4.2\...v8.4.3[View commits]
==== Bugfixes
*Filebeat*
- Fix handling of Cisco 302020 messages in ASA and FTD modules. {pull}33089[33089]
- Fix handling of empty array in httpjson input. {pull}32001[32001]
==== Added
*Affecting all Beats*
- Added `--enable-all-filesets` to the `setup` command to simplify loading all ingest pipelines. {issue}30916[30916] {pull}33114[33114]
[[release-notes-8.4.2]]
=== Beats version 8.4.2
https://github.com/elastic/beats/compare/v8.4.1\...v8.4.2[View commits]
==== Bugfixes
*Affecting all Beats*
- Fix in AWS related services initialisation relying on custom endpoint resolver. {issue}32888[32888] {pull}32921[32921]
*Filebeat*
- Fix for pagination at root level not working when used with chaining. {pull}32722[32722]
*Metricbeat*
- Fix and improve AWS metric period calculation to avoid zero-length intervals. {pull}32724[32724]
- Add missing cluster metadata to k8s module metricsets. {pull}32979[32979] {pull}33032[33032]
==== Added
*Metricbeat*
- Allow filtering on AWS tags by more than 1 value per key. {pull}32775[32775]
- Azure Billing: switch to Cost Management API for forecast data. {pull}32589[32589]
[[release-notes-8.4.1]]
=== Beats version 8.4.1
https://github.com/elastic/beats/compare/v8.4.0\...v8.4.1[View commits]
==== Known Issue
// tag::credentials-error[]
*Filebeat*
Filebeat agents configured to read from AWS inputs may return an error similar to the following:
[source,shell]
----
sqs ReceiveMessage failed: operation error SQS: ReceiveMessage, https response
error StatusCode: 403, RequestID: cb57783a-505f-5099-9160-23b8eea8ddbb,
api error SignatureDoesNotMatch: Credential should be scoped to a valid region.
----
This error was introduced by a breaking change in the AWS library.
IMPORTANT: This issue also affects FIPS-enabled endpoints. **If you rely on FIPS,
do not upgrade until version 8.4.2 of the {stack} is available.** The workaround
documented here will not resolve this problem.
Suggested resolution: In the Filebeat configuration, if an AWS input or module
configuration sets `endpoint` to a non empty string, set it to an empty string
instead. Also make sure the default AWS region is set in an environment
variable, credentials or instance profile, or in the `default_region` setting in
the configuration. For example:
[source,yaml]
----
filebeat.inputs:
- type: aws-s3
...
endpoint: "" <1>
default_region: us-east-1
----
<1> You can set this value to an empty string or remove the configuration setting.
Or for modules:
[source,yaml]
----
s3access:
enabled: false
...
var.endpoint: "" <1>
var.default_region: us-east-1
----
<1> You can set this value to an empty string or remove the configuration setting
// end::credentials-error[]
==== Bugfixes
*Auditbeat*
- Fixes a bug with the auditd module where data is corrupted because it was not copied before the byte slice was reused. {issue}32818[32818] {pull}32823[32823]
*Filebeat*
- Update `cloud.region` parsing in cloudtrail fileset. {pull}32763[32763]
- Fix file.path field in cloudtrail fileset to use json.digestS3Object. {pull}32759[32759]
- Fix not parsing as json when `json` and `ndjson` content types have charset information in `aws-s3` input {pull}32767[32767]
==== Added
*Filebeat*
- httpjson input: Add `toJSON` helper function to template context. {pull}32472[32472]
[[release-notes-8.4.0]]
=== Beats version 8.4.0
https://github.com/elastic/beats/compare/v8.3.3\...v8.4.0[View commits]
==== Known Issues
*Auditbeat*
Auditbeat/auditd integration will send malformed data and may crash in version 8.4.0. {issue}32818[32818]
Suggested resolution:
Do not start Auditbeat or auditd integration on Elastic Agent at version 8.4.0. Skip 8.4.0 and upgrade directly to 8.4.1.
This issue is resolved in 8.4.1 and later.
//include credentials error from 8.4.1 section
include::CHANGELOG.asciidoc[tag=credentials-error]
==== Breaking changes
*Heartbeat*
- Browser monitors (beta) now write to the `synthetics-*` index prefix. {pull}32064[32064]
- Setting a custom index for a given monitor is now deprecated. Streams are preferred. {pull}32064[32064]
- Browser monitors now default to a max concurrency of two. {pull}32564[32564]
==== Bugfixes
*Affecting all Beats*
- Fix namespacing for agent self-monitoring, CPU no longer reports as zero. {pull}32336[32336]
- Expand fields in `decode_json_fields` if target is set. {issue}31712[31712] {pull}32010[32010]
*Auditbeat*
- auditd module: Fix parsing of audit rules where arguments are quoted (like file paths containing spaces). {pull}32421[32421]
- auditd module: Fix minimum AuditStatus length so that library can support kernels from 2.6.32. {pull}32421[32421]
- system/socket: Reduce memory usage of the dataset. {issue}32191[32191] {pull}32192[32192]
*Filebeat*
- Fix counter for number of events published in `httpjson` input. {pull}31993[31993]
- Fix handling of Checkpoint event for R81. {issue}32380[32380] {pull}32458[32458]
- gcp-pubsub input: Restart Pub/Sub client on all errors. {issue}32550[32550] {pull}32712[32712]
*Heartbeat*
- Send targetted error message for unexpected synthetics exits. {pull}31936[31936]
- Reduced memory usage slightly for browser monitors. {pull}32317[32317]
- Automatically kill zombie-ish node processes. {pull}32393[32393]
- Added timeout for browser monitors. {pull}32434[32434]
- Fix bug with browser jobs that had missing check groups or sent empty events. {pull}32542[32542]
*Metricbeat*
- Update Kubernetes apiserver metricset to not collect deprecated metrics and fix dashboard. {pull}31973[31973]
- Check for nil metadata in GCP. {pull}32281[32281]
- Update Kubernetes controllermanager metricset to not collect deprecated metrics and fix dashboard. {pull}32037[32037]
- Fix ARN parsing for Cloudwatch resource names with leading slashes. {pull}32358[32358]
- Fix an infinite loop in AWS billing metricset. {pull}32626[32626]
- Add missing metrics in AWS Transit Gateway module {pull}32617[32617]
- Replace internal expiring cache used by the Kubernetes module with in-memory dictionary. {pull}32539[32539]
- Oracle Module: Refactor module to use existing host parsers instead of doing its own parsing of hosts. {issue}31611[31611] {pull}31692[#31692]
- Oracle Module: Correctly handle special characters in the connection string. {issue}24609[24609] {pull}31368[#31368]
*Winlogbeat*
- Powershell: Fix processing of parameter details. {pull}31833[31833]
- Security: Fix processing of sidlist, access list and access mask. {pull}31833[31833]
- Fix fatal invalid memory write on Windows 11. {issue}32469[32469] {pull}32519[32519]
- Fix handling of event formatting when no metadata is available on Windows 11. {issue}32468[32468] {pull}32519[32519]
==== Added
*Affecting all Beats*
- Improve performance of disk queue by coalescing writes. {pull}31935[31935]
*Auditbeat*
- Add `immutable` option to the auditd module. {issue}8352[8352] {pull}32381[32381]
*Filebeat*
- Add `auth.oauth2.google.jwt_json` option to `httpjson` input. {pull}31750[31750]
- Add authentication fields to RabbitMQ module documents. {issue}31159[31159] {pull}31680[31680]
- Add template helper function for decoding hexadecimal strings. {pull}31886[31886]
- Add new `parser` called `include_message` to filter based on message contents. {issue}31794[31794] {pull}32094[32094]
- Extend list of mapped record types in o365 Audit module. {pull}32217[32217]
- Add references for CRI-O configuration in input-container and in our Kubernetes manifests. {issue}32149[32149] {pull}32151[32151]
- httpjson input: Add `replaceAll` helper function to template context. {pull}32365[32365]
- Optimize grok patterns in system.auth module pipeline. {pull}32360[32360]
- Checkpoint module: add authentication operation outcome enrichment. {issue}32230[32230] {pull}32431[32431]
- Add documentation for decode_xml_wineventlog processor field mappings. {pull}32456[32456]
*Metricbeat*
- Oracle Module: New sysmetric metricset. {issue}30946[30946] {pull}31462[#31462]
- AWS Fargate: Added support for DesiredStatus and KnownStatus. {issue}32077[32077] {pull}32342[#32342]
- Enable Generic SQL merge metrics to a single event for sql_queries using new flag. {pull}32394[32394]
- Add distribution type metrics for GCP. {pull}32170[32170]
*Packetbeat*
- Add support for specifying default route interface sniffing. {issue}31905[31905] {pull}31950[31950]
- Add support for TCP transport to the SIP protocol. {issue}28166[28166] {pull}32346[32346]
[[release-notes-8.3.3]]
=== Beats version 8.3.3
https://github.com/elastic/beats/compare/v8.3.2\...v8.3.3[View commits]
==== Bugfixes
*Affecting all Beats*
- Fix OS name reported by add_host_metadata on Windows 11. {issue}30833[30833] {pull}32259[32259]
- Fix race condition when reloading running inputs. {pull}32309[32309]
*Filebeat*
- Fix Cisco AMP rate limit and pagination. {pull}32030[32030]
- Fix wrong state ID in states registry for awss3 s3 direct input. {pull}32164[32164]
- cisco/asa: fix handling of user names when there are Security Group Tags present. {issue}32009[32009] {pull}32196[32196]
*Metricbeat*
- Update elasticsearch node_stats metricset to use keyword for cgroup memory instead of long. {pull}32197[32197]
==== Added
*Metricbeat*
- Azure Billing: upgrade Usage Details API to version 2019-10-01. {pull}31970[31970]
[[release-notes-8.3.2]]
=== Beats version 8.3.2
https://github.com/elastic/beats/compare/v8.3.1\...v8.3.2[View commits]
==== Bugfixes
*Filebeat*
- Fix handling of stale log message handling in the winlog input {issue}32168[32168] {pull}32176[32176]
*Heartbeat*
- Fix regression where we write a dotted (non-nested) key `event.type`. {pull}32097[32097]
*Metricbeat*
- Fix for unintentionally reporting the libbeat "handles" metrics under "system" instead of "beat" as they were in previous releases. https://github.com/elastic/elastic-agent-system-metrics/pull/37[system-metrics#37]
*Winlogbeat*
- Fix handling of stale log message handling {issue}32168[32168] {pull}32176[32176]
==== Added
*Metricbeat*
- Differentiate between actual idle CPU states and an uninterruptible disk sleep. https://github.com/elastic/elastic-agent-system-metrics/pull/32[system-metrics#32]
[[release-notes-8.3.1]]
=== Beats version 8.3.1
https://github.com/elastic/beats/compare/v8.3.0\...v8.3.1[View commits]
==== Bugfixes
*Affecting all Beats*
- Improve syslog parser/processor error handling. {issue}31246[31246] {pull}31798[31798]
*Auditbeat*
- Fix handling of audit status messages for modern kernels with reduced audit message feature support. {issue}31616[31616] {pull}32141[32141]
*Filebeat*
- Fix handling and mapping of syslog priority, facility and severity values in Cisco module. {pull}32025[32025]
- Fix deduplication in Google workspace module by changing fingerprint processor target field from `@metadata.id` to `@metadata._id`. {pull}31898[31898]
- Adding a fix for threatintel module where MISP was paginating forever. {pull}31784[31784]
- Fix http_endpoint input TLS handshake failures. {pull}32105[32105]
- Fix last write pagination commit checkpoint on `aws-s3` input for s3 direct polling when using the same bucket and different list prefixes. {pull}31776[31776]
[[release-notes-8.3.0]]
=== Beats version 8.3.0
https://github.com/elastic/beats/compare/v8.2.3\...v8.3.0[View commits]
==== Bugfixes
*Affecting all Beats*
- Allow loading secrets that contain commas from the keystore {pull}31694[31694].
*Auditbeat*
- Fix audit status collection on kernels prior to version 5.9. {issue}31616[31616] {pull}31710[31710]
*Filebeat*
- Do not emit error log when filestream reader reaches EOF and `close.reader.on_eof` is enabled. {pull}31109[31109]
- sophos.xg: Update module to handle new log fields. {issue}31038[31038] {pull}31388[31388]
- Fix MISP documentation for `var.filters` config option. {pull}31434[31434]
- Fix type mapping of client.as.number in okta module. {pull}31676[31676]
- If a file is ignored by `filestream` because of ignore_older settings, when it is updated, only the new lines are shipped to the output. {issue}31924[31924] {pull}31972[31972]
*Heartbeat*
- Fix unintentional use of no-op logger. {pull}31543[31543]
*Metricbeat*
- make `system/filesystem` code sensitive to `hostfs` and migrate libraries to `elastic-agent-opts` {pull}31001[31001]
- Fix kubernetes module's internal cache expiration issue. This avoid metrics like `kubernetes.container.cpu.usage.limit.pct` from not being populated. {pull}31785[31785]
- add missing HealthyHostCount and UnHealthyHostCount for application ELB. {pull}31853[31853]
*Winlogbeat*
- Sysmon: Drop fields with "-" value (unset) {pull}31556[31556]
==== Added
*Affecting all Beats*
- Add new config option `timestamp.precision` to configure timestamps. {pull}31682[31682]
*Auditbeat*
- Add `backlog_wait_time_actual` to the output of the `auditbeat auditd show-status` command. {pull}31535[31535]
*Filebeat*
- Support SASL/SCRAM authentication in the Kafka input. {pull}31167[31167]
- checkpoint module: Add `network.transport` derived from IANA number. {pull}31076[31076]
- Add URL Encode template function for httpjson input. {pull}30962[30962]
- Add `application/zip` decoder to the `httpsjon` input. {issue}31282[31282] {pull}31304[31304]
- Default value of `filebeat.registry.flush` increased from 0s to 1s. CPU and disk I/O usage are reduced because the registry is not written to disk for each ingested log line. {issue}30279[30279]
- Cisco ASA/FTD: Add support for messages 434001 and 434003. {pull}31533[31533]