KAFKA-4108: Improve DumpLogSegments offsets-decoder output format #1937
Conversation
vahidhashemian
force-pushed
the
KAFKA-4108
branch
3 times, most recently
from
21c0f9b
to
a1a70aa
Compare
vahidhashemian
force-pushed
the
KAFKA-4108
branch
3 times, most recently
from
732ca82
to
7d88e71
Compare
vahidhashemian
force-pushed
the
KAFKA-4108
branch
from
7d88e71
to
d952770
Compare
|
Refer to this link for build results (access rights to CI server needed): |
|
Refer to this link for build results (access rights to CI server needed): |
|
Refer to this link for build results (access rights to CI server needed): |
|
@hachikuji, does this seem OK to you? |
| val keyString = Json.encode(Map("metadata" -> groupId)) | ||
| val valueString = Json.encode(Map( | ||
| "protocolType" -> protocolType, | ||
| "groupMetadata.protocol" -> group.protocol, |
There was a problem hiding this comment.
Out of curiosity, why do we use the "groupMetadata" prefix?
There was a problem hiding this comment.
Hmm, it's been a while :) Perhaps to somehow reflect the object/value they are describing. I could remove the prefix in the next update.
| "protocolType" -> protocolType, | ||
| "groupMetadata.protocol" -> group.protocol, | ||
| "groupMetadata.generationId" -> group.generationId, | ||
| "assignment" -> assignment)) |
There was a problem hiding this comment.
Should we jsonify the assignment too, or do you think it's ok as is?
There was a problem hiding this comment.
This assignment value is already stringified in the previous block. So I think we could keep it.
This PR improves the output format of DumpLogSegments when the `--offset-decoder` option is used for consuming `__consumer_offsets`, especially when it comes to group metadata.
An example of the partial output with existing formatting:
```
key: metadata::console-consumer-40190 payload: consumer:range:1:{consumer-1-20240b92-fbf4-44d5-bf8c-66b6d70c9948=[foo-0]}
```
An example of the same output with suggested formatting:
```
key: {"metadata":"console-consumer-40190"} payload: {"protocolType":"consumer","groupMetadata.protocol":"range","groupMetadata.generationId":1,"assignment":"{consumer-1-20240b92-fbf4-44d5-bf8c-66b6d70c9948=[foo-0]}"}
```
There was a problem hiding this comment.
@hachikuji Thanks for taking a look. I'll submit an update shortly to address your comments.
| val keyString = Json.encode(Map("metadata" -> groupId)) | ||
| val valueString = Json.encode(Map( | ||
| "protocolType" -> protocolType, | ||
| "groupMetadata.protocol" -> group.protocol, |
There was a problem hiding this comment.
Hmm, it's been a while :) Perhaps to somehow reflect the object/value they are describing. I could remove the prefix in the next update.
| "protocolType" -> protocolType, | ||
| "groupMetadata.protocol" -> group.protocol, | ||
| "groupMetadata.generationId" -> group.generationId, | ||
| "assignment" -> assignment)) |
There was a problem hiding this comment.
This assignment value is already stringified in the previous block. So I think we could keep it.
vahidhashemian
force-pushed
the
KAFKA-4108
branch
from
d952770
to
5ace58d
Compare
|
Anything preventing this from being merged? Code freeze is tomorrow. |
|
Thanks for the heads-up. Just waiting for @hachikuji's feedback on my responses above and the new commit. |
|
LGTM. I went back to the JIRA and was surprised to find that I filed this issue 😝 . |
This PR improves the output format of DumpLogSegments when the
--offset-decoderoption is used for consuming__consumer_offsets, especially when it comes to group metadata.An example of the partial output with existing formatting:
An example of the same output with suggested formatting: