feat: JSON Masking

[jamfor352]

Hi!

AKHQ is fantastic, but the data masking using regex doesn't really scale well if you have complex personal data (pii) requirements (for example, if you need to mask all fields by default apart from carefully selected fields in a production environment). Crafting a regex for this can be flaky. Especially if you need to deal with multiple lines and account for integers, arrays, all sorts of data types. Or if you want to be able to expose some fields on some topics, but not others, etc.

I've been trying my fork locally which introduces a new JSON filtering system (while keeping the regex stuff entirely in place) and it works great, and I'd like to contribute it upstream. I think I've added more than sufficient tests as well, and retained backwards compatibility by defaulting to using the regex filtering if not defined (which current users won't, since mode is a new property).

It has two modes - json_mask_by_default (where everything is masked by default and you have to specifically opt in to see specific fields - useful for regulation-heavy environments) and json_show_by_default where everything shows by default but you can mask specific bits of data.

[jamfor352]

I just pushed a commit fixing datetime parsing in AvroSerializer as I realised it led to some flaky tests as well. Not related to my changes here, but a nice to have (just noticed it when checking the workflow results)

[AlexisSouquiere]

@jamfor352 I will take time to review it and give you my feedback next week

[emerfan]

Hi, have been using this build locally and it's awesome - following the PR

[jamfor352]

Hi, have been using this build locally and it's awesome - following the PR

Thank you :) Glad to hear it's worked well for you!