Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feature Request]:[Go SDK] Improve "bad return type" messaging #31717

Open
1 of 16 tasks
lostluck opened this issue Jun 28, 2024 · 1 comment
Open
1 of 16 tasks

[Feature Request]:[Go SDK] Improve "bad return type" messaging #31717

lostluck opened this issue Jun 28, 2024 · 1 comment

Comments

@lostluck
Copy link
Contributor

What would you like to happen?

During analysis of DoFn types, when there's an issue with a type such as having an unencodable Exported field type, it can end up causing a message bad return type for <DoFn>: <type> but not including why it's a bad return type.

Message comes from here: https://github.com/apache/beam/blob/master/sdks/go/pkg/beam/core/funcx/fn.go#L468

Eg The following can trigger this message through this analysis.

type myRet struct{
  Error error // unencodable interface type as field type.
}

The complexity in fixing this though is that it may be possible that the wrapper type (myRet) has a coder registered for it, so it's permitted in this position. This might not be known at the functional Go type analysis level funcx operates on.

The ideal solution may be to change the analysis somewhat: The funcx analysis should return a list of suspect types, where they're coming from (this is critical), and why it's possibly a bad type, which a higher level validator can that inspect and validate against the higher level information (such as availability of coders for that type). This would prevent false rejections, and provide complete debugging information without iterating the pipeline one problem at a time.

But it would be sufficient to simply have the current message be more detailed about why the type is bad, instead of the current, less obvious situation.

Issue Priority

Priority: 3 (nice-to-have improvement)

Issue Components

  • Component: Python SDK
  • Component: Java SDK
  • Component: Go SDK
  • Component: Typescript SDK
  • Component: IO connector
  • Component: Beam YAML
  • Component: Beam examples
  • Component: Beam playground
  • Component: Beam katas
  • Component: Website
  • Component: Spark Runner
  • Component: Flink Runner
  • Component: Samza Runner
  • Component: Twister2 Runner
  • Component: Hazelcast Jet Runner
  • Component: Google Cloud Dataflow Runner
@mls3odp
Copy link
Contributor

mls3odp commented Jul 5, 2024

.take-issue

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants