Bool parameters in json responses are not properly unmarshalled

[nlowe]

Version of AWS SDK for Go?

v0.9.0

Version of Go (go version)?

go version go1.12.1 windows/amd64

What issue did you see?

When sending a GetMethodRequest to the API Gateway API, bool request parameters are incorrectly deserialized.

Steps to reproduce

Given this sample program:

package main

import (
	"context"
	"flag"
	"fmt"

	"github.com/aws/aws-sdk-go-v2/aws/external"
	"github.com/aws/aws-sdk-go-v2/service/apigateway"
)

func main() {
	restApiID := flag.String("rest-api", "", "The Rest API ID")
	resourceID := flag.String("resource", "", "The Resource ID")
	method := flag.String("method", "ANY", "The HTTP Method")

	flag.Parse()

	cfg, err := external.LoadDefaultAWSConfig()
	if err != nil {
		panic(err)
	}

	apigw := apigateway.New(cfg)

	result, err := apigw.GetMethodRequest(&apigateway.GetMethodInput{
		RestApiId:  restApiID,
		ResourceId: resourceID,
		HttpMethod: method,
	}).Send(context.Background())

	if err != nil {
		panic(err)
	}

	fmt.Printf("%+v\n", result)
}

I get the following output:

{
  ApiKeyRequired: false,
  AuthorizationType: "NONE",
  HttpMethod: "ANY",
  MethodIntegration: {
    CacheKeyParameters: ["method.request.path.proxy"],
    CacheNamespace: "***",
    ConnectionType: INTERNET,
    ContentHandling: ,
    HttpMethod: "ANY",
    IntegrationResponses: {
      200: {
        ContentHandling: ,
        ResponseTemplates: {
          application/json: ""
        },
        StatusCode: "200"
      }
    },
    PassthroughBehavior: "WHEN_NO_MATCH",
    RequestParameters: {
      integration.request.path.proxy: "method.request.path.proxy"
    },
    TimeoutInMillis: 29000,
    Type: HTTP_PROXY,
    Uri: "http://***.us-east-1.elb.amazonaws.com/***/{proxy}"
  },
  RequestParameters: {
    method.request.path.proxy: false
  }
}

Notice how RequestParameters["method.request.path.proxy"] is false.

However, when I use the AWS CLI, you can see that it is set to true:

{
    "httpMethod": "ANY",
    "authorizationType": "NONE",
    "apiKeyRequired": false,
    "requestParameters": {
        "method.request.path.proxy": true
    },
    "methodIntegration": {
        "type": "HTTP_PROXY",
        "httpMethod": "ANY",
        "uri": "http://***.us-east-1.elb.amazonaws.com/***/{proxy}",
        "connectionType": "INTERNET",
        "requestParameters": {
            "integration.request.path.proxy": "method.request.path.proxy"
        },
        "passthroughBehavior": "WHEN_NO_MATCH",
        "timeoutInMillis": 29000,
        "cacheNamespace": "***",
        "cacheKeyParameters": [
            "method.request.path.proxy"
        ],
        "integrationResponses": {
            "200": {
                "statusCode": "200",
                "responseTemplates": {
                    "application/json": null
                }
            }
        }
    }
}

Notice that RequestParameters["method.request.path.proxy"] is actually true.

[nlowe]

I think this is because jsonutil.unmarshalScalar only tries to unmarshal *bool, not bool:

aws-sdk-go-v2/private/protocol/json/jsonutil/unmarshal.go

Lines 231 to 236 in da6472f

	case bool:
	switch value.Interface().(type) {
	case *bool:
	value.Set(reflect.ValueOf(&d))
	default:
	return errf()

by patching in a case for bool it behaves as expected:

	case bool:
		switch value.Interface().(type) {
		case *bool:
			value.Set(reflect.ValueOf(&d))
		case bool:
			value.Set(reflect.ValueOf(d))
		default:
			return errf()
		}

The error returned by jsonutil.unmarshalScalar is checked by jsonutil.unmarshalAny, but not by jsonutil.unmarshalMap:

aws-sdk-go-v2/private/protocol/json/jsonutil/unmarshal.go

Lines 165 to 170 in da6472f

	for k, v := range mapData {
	kvalue := reflect.ValueOf(k)
	vvalue := reflect.New(value.Type().Elem()).Elem()
	unmarshalAny(vvalue, v, "")
	value.SetMapIndex(kvalue, vvalue)
	}

So the default of false is used. I believe this will affect any bool field, not just this specific instance that I initially opened the issue for.

[nlowe]

Indeed, it looks like it does:

package main

import (
	"fmt"
	"strings"

	"github.com/aws/aws-sdk-go-v2/private/protocol/json/jsonutil"
)

type Foo struct {
	Fizz map[string]bool
}

func main() {
	payload := `{"Fizz": {"Buzz": true}}`

	v := &Foo{}
	jsonutil.UnmarshalJSON(v, strings.NewReader(payload))

	fmt.Printf("%+v\n", v)
}

Produces: &{Fizz:map[Buzz:false]} without the patch and &{Fizz:map[Buzz:true]} with it.

[jasdel]

Thanks for reporting this detailed issue @nlowe. I'm looking into reproduce this issue, but I think the patch you suggested makes sense. The v2 SDK refactored updated list and maps to be values instead of pointers and this looks like a case that might of been missed.