-
Notifications
You must be signed in to change notification settings - Fork 1.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add full version matrix produce/consume test #1040
Merged
Merged
Changes from 1 commit
Commits
Show all changes
11 commits
Select commit
Hold shift + click to select a range
e42dc76
Add missing Kafka versions
horkhe 1596275
Add timeout to async_producer_test
horkhe 857b221
Add full version matrix produce/consume test
horkhe d2959fa
Report ErrIncompleteResponse if nothing is retrieved only
horkhe 6a83383
Fail Travis CI builds on tests failure
horkhe 7579f89
Add LZ4 specific version matrix text
horkhe 7ad9a62
Fix TestProduceRequest
horkhe dfdbc3f
Increase test timeout
horkhe 973e75c
Simplify messageSet and recordBatch parsing
horkhe 780ea15
Fix gzip compression encode/decode test
horkhe 6ce9e92
Fix flaky TestAlterConfigsRequest
horkhe File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Add LZ4 specific version matrix text
- Loading branch information
commit 7579f896a21cfdd5b574bcc094183216411dce5c
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It might be simpler to run this test with an AsyncProducer so you can avoid spinning up a bunch of goroutines and tracking them with a waitgroup in order to get parallelism
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The point was to have several producers produce different types of messageSets/batchRecords concurrently. I certainly can use AsyncProducers for that although then I would need to store them somewhere and collect produced messages from all of their Success() channels, not sure if this implementation is going to be shorter though. I can implement it this way if you want.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Oh ok, I missed the
wg.Wait()
being fully outside the outer loop and thought the purpose was only for batching. This is fine then.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You know what, I cannot reproduce
kafka: response did not contain all the expected topic/partition blocks
now at all. Looks like #1041 fixed the root cause of this issue too. So I suggest simplifying messageSet/recordBatch parsing a bit to make it easier to understand.Note that I left:
intact. But honestly I think it would be better just ignore this case. Because what do you expect the library user to do about that error? Nothing but retry, and sarama can do that on its own.