Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(datadog grok): enable the DOTALL mode by default #1022

Merged
merged 7 commits into from
Sep 26, 2024
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
correct comments
  • Loading branch information
vladimir-dd committed Sep 8, 2024
commit 118557ef2cb6f1c936e9d1520cecf9622dcfdb0a
18 changes: 9 additions & 9 deletions src/datadog/grok/parse_grok.rs
Original file line number Diff line number Diff line change
Expand Up @@ -1021,33 +1021,33 @@ mod tests {
#[test]
fn parses_with_new_lines() {
test_full_grok(vec![
// multiline mode is enabled by default
// the DOTALL mode is enabled by default
(
"%{data:field}",
"a\nb",
Ok(Value::from(btreemap! {
"field" => "a\nb"
})),
),
// (?s) enables the DOTALL mode
(
"%{data:line1}\n%{data:line2}",
"(?s)%{data:field}",
"a\nb",
Ok(Value::from(btreemap! {
"line1" => "a",
"line2" => "b"
"field" => "a\nb"
})),
),
// (?s) is not supported by the underlying regex engine(onig) - it uses (?m) instead, so we convert it silently
(
"(?s)%{data:field}",
"%{data:line1}\n%{data:line2}",
"a\nb",
Ok(Value::from(btreemap! {
"field" => "a\nb"
"line1" => "a",
"line2" => "b"
})),
),
// disable DOTALL mode with (?-s)
// disable the DOTALL mode with (?-s)
("(?s)(?-s)%{data:field}", "a\nb", Err(Error::NoMatch)),
// disable and then enable DOTALL mode
// disable and then enable the DOTALL mode
(
"(?-s)%{data:field} (?s)%{data:field}",
"abc d\ne",
Expand Down
5 changes: 3 additions & 2 deletions src/datadog/grok/parse_grok_rules.rs
Original file line number Diff line number Diff line change
Expand Up @@ -176,12 +176,13 @@ fn parse_pattern(
) -> Result<GrokRule, Error> {
parse_grok_rule(pattern, context)?;
let mut pattern = String::new();
// In Oniguruma the (?m) modifier is used to enable the DOTALL mode(dot includes newlines),
// as opposed to the (?s) modifier in other regex flavors.
// \A, \z - parses from the beginning to the end of string, not line(until \n)
pattern.push_str(r"(?m)\A"); // (?m) enables multiline mode
pattern.push_str(r"(?m)\A"); // (?m) enables the DOTALL mode by default
pattern.push_str(&context.regex);
pattern.push_str(r"\z");

// our regex engine(onig) uses (?m) mode modifier instead of (?s) to make the dot match all characters
pattern = pattern.replace("(?s)", "(?m)").replace("(?-s)", "(?-m)");

// compile pattern
Expand Down
Loading