Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

transport: validate http 200 status for responses #4474

Merged
merged 23 commits into from
Jul 14, 2021
Merged

transport: validate http 200 status for responses #4474

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
23 commits
Select commit Hold shift + click to select a range
fe87cc5
Validate http 200 for all non-end-of-stream messages
JNProtzman May 21, 2021
154e687
Add additional test
JNProtzman May 21, 2021
e437a72
Merge branch 'master' of https://github.com/grpc/grpc-go into http_200
JNProtzman May 24, 2021
9cb2546
Add test for bad http status code in gRPC mode
JNProtzman May 25, 2021
17d7508
Merge branch 'master' of https://github.com/grpc/grpc-go into http_200
JNProtzman May 25, 2021
71ad202
fix error messages
JNProtzman May 27, 2021
29c60a1
Merge branch 'master' of https://github.com/grpc/grpc-go into http_200
JNProtzman May 27, 2021
0e104a3
new test case, fix error messaging
JNProtzman May 27, 2021
17758ef
address pr comments
JNProtzman May 28, 2021
6de802f
Merge branch 'master' of https://github.com/grpc/grpc-go into http_200
JNProtzman Jun 4, 2021
399b76e
Fix test log, pr comments
JNProtzman Jun 4, 2021
253fd40
Merge branch 'master' of https://github.com/grpc/grpc-go into http_200
JNProtzman Jun 5, 2021
1446920
Use status in error instead of proto, add String method to Status
JNProtzman Jun 7, 2021
fa0bff4
Merge branch 'master' of https://github.com/grpc/grpc-go into http_200
JNProtzman Jun 9, 2021
efaf7ab
return status directly
JNProtzman Jun 9, 2021
c0cda22
Merge branch 'master' of https://github.com/grpc/grpc-go into http_200
JNProtzman Jun 17, 2021
6230a62
pr comment
JNProtzman Jun 17, 2021
43990b6
test updates
JNProtzman Jun 20, 2021
8b0e761
fix go vet issue
JNProtzman Jun 25, 2021
472d502
test endstream and not endstream
JNProtzman Jul 2, 2021
6642a86
Merge branch 'master' of https://github.com/grpc/grpc-go into http_200
JNProtzman Jul 2, 2021
adfc275
test grpc-status, not http status
JNProtzman Jul 2, 2021
9207e93
Minor suggestions
JNProtzman Jul 13, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
address pr comments
  • Loading branch information
@JNProtzman
JNProtzman committed May 28, 2021
commit 17758ef8f3f2573c9bbce012931db420958a989a
65 changes: 42 additions & 23 deletions internal/transport/http2_client.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,7 @@ import (
"io"
"math"
"net"
"net/http"
"strconv"
"strings"
"sync"
Expand Down Expand Up @@ -1269,18 +1270,22 @@ func (t *http2Client) operateHeaders(frame *http2.MetaHeadersFrame) {
contentTypeErr = "missing content-type"
dfawley marked this conversation as resolved.
Show resolved Hide resolved
JNProtzman marked this conversation as resolved.
Show resolved Hide resolved
grpcMessage string
statusGen *status.Status

httpStatus string
rawStatus string
httpStatusCode *int
httpStatusErr string
rawStatus string
// headerError is set if an error is encountered while parsing the headers
headerError string
)

if !endStream {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually I believe this is incorrect, as it will not validate that a trailers-only response contains the HTTP status. Changing this to if initialHeader should fix this.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ping on this since it's still something we should fix before merging.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed! Sorry for the delay.

httpStatusErr = "malformed header: missing HTTP status"
}

for _, hf := range frame.Fields {
switch hf.Name {
case "content-type":
if _, validContentType := grpcutil.ContentSubtype(hf.Value); !validContentType {
contentTypeErr = fmt.Sprintf("transport: received the unexpected content-type %q", hf.Value)
contentTypeErr = fmt.Sprintf("transport: received unexpected content-type %q", hf.Value)
break
}
contentTypeErr = ""
Expand All @@ -1299,7 +1304,27 @@ func (t *http2Client) operateHeaders(frame *http2.MetaHeadersFrame) {
headerError = fmt.Sprintf("transport: malformed grpc-status-details-bin: %v", err)
}
case ":status":
httpStatus = hf.Value
if hf.Value == "200" {
httpStatusErr = ""
statusCode := 200
httpStatusCode = &statusCode
break
}

c, err := strconv.ParseInt(hf.Value, 10, 32)
if err != nil {
se := status.New(codes.Internal, fmt.Sprintf("transport: malformed http-status: %v", err))
t.closeStream(s, se.Err(), true, http2.ErrCodeProtocol, se, nil, endStream)
return
}
statusCode := int(c)
httpStatusCode = &statusCode

httpStatusErr = fmt.Sprintf(
"unexpected HTTP status code received from server: %d (%s)",
statusCode,
http.StatusText(statusCode),
)
default:
if isReservedHeader(hf.Name) && !isWhitelistedHeader(hf.Name) {
break
Expand All @@ -1314,31 +1339,25 @@ func (t *http2Client) operateHeaders(frame *http2.MetaHeadersFrame) {
}
}

if !isGRPC || (httpStatus != "200" && !endStream) {
var (
code = codes.Internal // when header does not include HTTP status, return INTERNAL
httpStatusCode *int
)

if httpStatus != "" {
c, err := strconv.ParseInt(httpStatus, 10, 32)
if err != nil {
se := status.New(codes.Internal, fmt.Sprintf("transport: malformed http-status: %v", err))
t.closeStream(s, se.Err(), true, http2.ErrCodeProtocol, se, nil, endStream)
return
}
statusCode := int(c)
httpStatusCode = &statusCode
if !isGRPC || httpStatusErr != "" {
var code = codes.Internal // when header does not include HTTP status, return INTERNAL

if httpStatusCode != nil {
var ok bool
code, ok = HTTPStatusConvTab[statusCode]
code, ok = HTTPStatusConvTab[*httpStatusCode]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Interesting. I just was pointed to this:

https://github.com/grpc/grpc/blob/master/doc/http-grpc-status-mapping.md

This table is to be used only for clients that received a response that did not include grpc-status. If grpc-status was provided, it must be used

So, I think this part should only be happening if there is no grpc-status (rawStatusCode), but using rawStatusCode otherwise. Do you want to take a shot at this?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've put up a PR for if here

if !ok {
code = codes.Unknown
}
}

var errs []string
if httpStatusErr != "" {
errs = append(errs, httpStatusErr)
}
if contentTypeErr != "" {
errs = append(errs, contentTypeErr)
}
// Verify the HTTP response is a 200.
se := status.New(code, constructHTTPErrMsg(httpStatusCode, contentTypeErr))
se := status.New(code, strings.Join(errs, "; "))
t.closeStream(s, se.Err(), true, http2.ErrCodeProtocol, se, nil, endStream)
return
}
Expand Down
21 changes: 0 additions & 21 deletions internal/transport/http_util.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -173,27 +173,6 @@ func decodeGRPCStatusDetails(rawDetails string) (*status.Status, error) {
return status.FromProto(st), nil
}

// constructErrMsg constructs error message to be returned in HTTP fallback mode.
// Format: HTTP status code and its corresponding message + content-type error message.
func constructHTTPErrMsg(httpStatus *int, contentTypeErr string) string {
var errMsgs []string
if httpStatus == nil {
errMsgs = append(errMsgs, "malformed header: missing HTTP status")
} else {
errMsgs = append(errMsgs, fmt.Sprintf(
"unexpected HTTP status code received from server: %d (%s)",
*httpStatus,
http.StatusText(*(httpStatus)),
))
}

if contentTypeErr != "" {
errMsgs = append(errMsgs, contentTypeErr)
}

return strings.Join(errMsgs, "; ")
}

type timeoutUnit uint8

const (
Expand Down
8 changes: 6 additions & 2 deletions internal/transport/transport_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2008,7 +2008,7 @@ func (s) TestClientDecodeHeaderStatusErr(t *testing.T) {
// no error
JNProtzman marked this conversation as resolved.
Show resolved Hide resolved
wantStatus: status.New(
codes.Unknown,
"unexpected HTTP status code received from server: 200 (OK); missing content-type",
"missing content-type",
),
},
{
Expand All @@ -2034,7 +2034,7 @@ func (s) TestClientDecodeHeaderStatusErr(t *testing.T) {
},
wantStatus: status.New(
codes.Internal,
"malformed header: missing HTTP status; transport: received the unexpected content-type \"application/json\"",
"transport: received unexpected content-type \"application/json\"",
),
},
{
Expand Down Expand Up @@ -2115,6 +2115,10 @@ func (s) TestClientDecodeHeaderStatusErr(t *testing.T) {
got := ts.status
want := test.wantStatus
if got.Code() != want.Code() || got.Message() != want.Message() {
t.Logf("got: %s", got.Message())
t.Logf("got: %d", got.Code())
t.Logf("want: %s", want.Message())
t.Logf("want: %d", want.Code())
t.Fatalf("operateHeaders(%v); status = %v; want %v", test.metaHeaderFrame, got, want)
dfawley marked this conversation as resolved.
Show resolved Hide resolved
}
})
Expand Down
10 changes: 5 additions & 5 deletions test/end2end_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7254,7 +7254,7 @@ func (s) TestHTTPHeaderFrameErrorHandlingInitialHeader(t *testing.T) {
":status", "403",
"content-type", "application/grpc",
},
errCode: codes.Unknown,
errCode: codes.PermissionDenied,
},
{
// malformed grpc-status.
Expand All @@ -7263,7 +7263,7 @@ func (s) TestHTTPHeaderFrameErrorHandlingInitialHeader(t *testing.T) {
"content-type", "application/grpc",
"grpc-status", "abc",
},
errCode: codes.Internal,
errCode: codes.Unavailable,
},
{
// Malformed grpc-tags-bin field.
Expand All @@ -7273,7 +7273,7 @@ func (s) TestHTTPHeaderFrameErrorHandlingInitialHeader(t *testing.T) {
"grpc-status", "0",
"grpc-tags-bin", "???",
},
errCode: codes.Internal,
errCode: codes.Unavailable,
},
{
// gRPC status error.
Expand All @@ -7282,7 +7282,7 @@ func (s) TestHTTPHeaderFrameErrorHandlingInitialHeader(t *testing.T) {
"content-type", "application/grpc",
"grpc-status", "3",
},
errCode: codes.InvalidArgument,
errCode: codes.Unavailable,
},
} {
doHTTPHeaderTest(t, test.errCode, test.header)
Expand All @@ -7305,7 +7305,7 @@ func (s) TestHTTPHeaderFrameErrorHandlingNormalTrailer(t *testing.T) {
// trailer missing grpc-status
":status", "502",
},
errCode: codes.Unknown,
errCode: codes.Unavailable,
},
{
responseHeader: []string{
Expand Down