Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[exporter/elasticsearch] Add sanitization utils for datastream fields #35494

Open
wants to merge 11 commits into
base: main
Choose a base branch
from

Conversation

rubvs
Copy link

@rubvs rubvs commented Sep 30, 2024

Description: Adds util functions to sanitize data stream fields

Link to tracking Issue: Fixes #34285

Testing: Added unit tests

Copy link

linux-foundation-easycla bot commented Sep 30, 2024

CLA Signed

The committers listed above are authorized under a signed CLA.

Copy link
Contributor

@carsonip carsonip left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, looks good functionality wise. A couple of minor comments.

exporter/elasticsearchexporter/data_stream_router.go Outdated Show resolved Hide resolved
exporter/elasticsearchexporter/data_stream_router.go Outdated Show resolved Hide resolved
exporter/elasticsearchexporter/util.go Outdated Show resolved Hide resolved
exporter/elasticsearchexporter/exporter_test.go Outdated Show resolved Hide resolved
exporter/elasticsearchexporter/exporter_test.go Outdated Show resolved Hide resolved
exporter/elasticsearchexporter/exporter_test.go Outdated Show resolved Hide resolved
exporter/elasticsearchexporter/util.go Outdated Show resolved Hide resolved
exporter/elasticsearchexporter/exporter_test.go Outdated Show resolved Hide resolved
@rubvs
Copy link
Author

rubvs commented Oct 1, 2024

@carsonip I have improved the sanitizeDataStreamField to leverage inlining. I also merged the .otel suffix extension into the sanitization check, since I have to take the extension into account when doing the 100 char check. Let me know if you approve this approach.

@rubvs rubvs requested a review from carsonip October 1, 2024 14:27
@rubvs
Copy link
Author

rubvs commented Oct 1, 2024

@carsonip I also added a test to model_test.go. I do not know if this is needed, but I think it won't hurt to add it either. If you want me to keep this, I'll update the test to get rid of the disallowedRunes and replace if with the hardcoded runes, as you suggested in the previous comments.

@rubvs rubvs changed the title Add sanitization utils for datastream fields [exporter/elasticsearch] Add sanitization utils for datastream fields Oct 1, 2024
Copy link
Contributor

@carsonip carsonip left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nice catch about the otel suffix! a comment about refactoring

exporter/elasticsearchexporter/data_stream_router.go Outdated Show resolved Hide resolved
exporter/elasticsearchexporter/data_stream_router.go Outdated Show resolved Hide resolved
exporter/elasticsearchexporter/data_stream_router.go Outdated Show resolved Hide resolved
exporter/elasticsearchexporter/data_stream_router.go Outdated Show resolved Hide resolved
@rubvs rubvs marked this pull request as ready for review October 1, 2024 16:40
@rubvs rubvs requested a review from a team as a code owner October 1, 2024 16:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[exporter/elasticsearch] Data stream routing does not sanitize values
3 participants