为什么解组此API响应会返回意外的EOF?

时间:2018-05-13 09:05:24

标签: go protocol-buffers grpc

问题

我正在使用协议缓冲区和gRPC在Go中创建微服务。它与第三方API(Snooth)交互,我试图将JSON响应解组为我使用proto包创建的protobuf结构。

Unmarshalling返回unexpected EOF错误。

我尝试过什么

  • 使用json.newDecoder代替json.unmarshal
  • jsonpb.Unmarshal代替proto.Unmarshal(返回bad value in StructValue for key错误)
  • 限制使用io.LimitReader
  • 读取的响应

我还读过一些关于为proto类型添加大小标签或其他内容的东西吗?但我不确定那是什么,或者它是否相关。这是Github上的回购。

问题

导致此unexpected EOF错误的原因是什么以及如何修复它,以便API响应成功解组到proto结构中?

旁注:我是Go的新手,也非常感谢以下代码的任何反馈/改进。谢谢!

代码

Proto

message Response {
    message Meta {
        int32 results = 1;
        int32 returned = 2;
        string errmsg = 3;
        int32 status = 4;
    }

    Meta meta = 1;
    repeated google.protobuf.Struct wines = 2;
    repeated google.protobuf.Struct actions = 3;
}

main.go

func fetchFromSnooth(e string, qs string, c chan response) {
    defer func() {
        if r := recover(); r != nil {
            log.Printf("Error fetching from Snooth: %s", r)
            errmsg := fmt.Sprint(r)
            c <- response{nil, snoothApiError{errmsg}}
        }
    }()

    v := url.Values{"akey": {os.Getenv("SNOOTH_API_KEY")}}
    requestUrl := fmt.Sprintf("%s%s/?%s%s", snoothRoot, e, v.Encode(), qs)
    log.Printf("Fetching: %s", requestUrl)

    res, err := httpClient.Get(requestUrl)
    if err != nil {
        panic(err)
    }
    defer res.Body.Close()

    body, err := ioutil.ReadAll(res.Body)
    if err != nil {
        panic(err)
    }

    result := &pb.Response{}
    if err := proto.Unmarshal(body, result); err != nil {
        panic(err)
    }
    c <- response{result, snoothApiError{""}}
}

func (s *snoothApiService) searchWines(params *pb.Parameters_WineSearch) response {
    c := make(chan response)
    defer close(c)

    v := url.Values{}
    go fetchFromSnooth("wines", v.Encode(), c)
    return <-c
}

func main() {
    snooth := snoothApiService{}
    resp := snooth.searchWines(nil)
    fmt.Println(resp)
}

修改

以下是我试图解组的API响应类型的示例:

    {
    "meta": {
        "results": 1489442,
        "returned": 5,
        "errmsg": "",
        "status": 1
    },
    "wines": [
        {
            "name": "Conway Deep Sea Chardonnay la Costa Wine Co",
            "code": "conway-deep-sea-chardonnay-la-costa-wine-co-2008-1",
            "region": "USA > California > Central Coast",
            "winery": "Conway Family Wines",
            "winery_id": "conway-family-wines",
            "varietal": "Chardonnay",
            "price": "21.99",
            "vintage": "2008",
            "type": "White Wine",
            "link": "http:\/\/www.snooth.com\/wine\/conway-deep-sea-chardonnay-la-costa-wine-co-2008-1\/",
            "tags": "",
            "image": "https:\/\/ei.isnooth.com\/multimedia\/0\/2\/8\/image_787698_square.jpeg",
            "snoothrank": 3,
            "available": 0,
            "num_merchants": 0,
            "num_reviews": 10
        },
        {
            "name": "Olmaia Cabernet di Toscana",
            "code": "olmaia-cabernet-di-toscana",
            "region": "Italy > Tuscany > Toscana Igt",
            "winery": "Col D Orcia",
            "winery_id": "col-d-orcia",
            "varietal": "Cabernet Sauvignon",
            "price": "0.00",
            "vintage": "",
            "type": "Red Wine",
            "link": "http:\/\/www.snooth.com\/wine\/olmaia-cabernet-di-toscana\/",
            "tags": "",
            "image": "https:\/\/ei.isnooth.com\/multimedia\/d\/e\/e\/image_790198_square.jpeg",
            "snoothrank": 3.5,
            "available": 0,
            "num_merchants": 0,
            "num_reviews": 25
        },
        {
            "name": "Dominio Dostares Prieto Picudo Vino de la Tierra de Castilla Y León Cumal",
            "code": "dominio-dostares-prieto-picudo-vino-de-la-tierra-de-castilla-y-leon-cumal-2006",
            "region": "Spain > Castilla y León > Vino de la Tierra de Castilla y León",
            "winery": "Bischöfliches Priesterseminar Trier",
            "winery_id": "bischofliches-priesterseminar-trier",
            "varietal": "Prieto Picudo",
            "price": "15.89",
            "vintage": "2006",
            "type": "Red Wine",
            "link": "http:\/\/www.snooth.com\/wine\/dominio-dostares-prieto-picudo-vino-de-la-tierra-de-castilla-y-leon-cumal-2006\/",
            "tags": "",
            "image": "https:\/\/ei.isnooth.com\/multimedia\/d\/0\/4\/image_336595_square.jpeg",
            "snoothrank": "n\/a",
            "available": 0,
            "num_merchants": 0,
            "num_reviews": 1
        },
        {
            "name": "Dominio Dostares Prieto Picudo Vino de la Tierra de Castilla Y León Cumal",
            "code": "dominio-dostares-prieto-picudo-vino-de-la-tierra-de-castilla-y-leon-cumal-2005",
            "region": "Spain > Castilla y León > Vino de la Tierra de Castilla y León",
            "winery": "Bischöfliches Priesterseminar Trier",
            "winery_id": "bischofliches-priesterseminar-trier",
            "varietal": "Prieto Picudo",
            "price": "38.99",
            "vintage": "2005",
            "type": "Red Wine",
            "link": "http:\/\/www.snooth.com\/wine\/dominio-dostares-prieto-picudo-vino-de-la-tierra-de-castilla-y-leon-cumal-2005\/",
            "tags": "",
            "image": "https:\/\/ei.isnooth.com\/multimedia\/1\/d\/a\/image_336596_square.jpeg",
            "snoothrank": "n\/a",
            "available": 0,
            "num_merchants": 0,
            "num_reviews": 1
        },
        {
            "name": "The Little Penguin Chardonnay Premier",
            "code": "the-little-penguin-chardonnay-premier-2010",
            "region": "South East Australia",
            "winery": "The Little Penguin",
            "winery_id": "the-little-penguin",
            "varietal": "Chardonnay",
            "price": "11.99",
            "vintage": "2010",
            "type": "White Wine",
            "link": "http:\/\/www.snooth.com\/wine\/the-little-penguin-chardonnay-premier-2010\/",
            "tags": "",
            "image": "https:\/\/ei.isnooth.com\/multimedia\/2\/c\/4\/image_826282_square.jpeg",
            "snoothrank": "n\/a",
            "available": 0,
            "num_merchants": 0,
            "num_reviews": 7
        }
    ]
}

2 个答案:

答案 0 :(得分:2)

它会解析当时的encoding/json包:https://play.golang.org/p/IQzMm2tDI7w

protoc - 生成的代码Unmarshal解析协议缓冲区编码的字节流,而不是JSON!

答案 1 :(得分:0)

- 更新 -

我现在使用jsonpb.Unmarshal方法根据需要解开了响应。虽然我必须首先解组并使用常规json库编组响应,但是为了绕过响应中的一些转义值(我在Struct中收到了一个错误的值&#39;错误:

resJson, err := ioutil.ReadAll(res.Body)

j := make(map[string]interface{})
if err := json.Unmarshal(resJson, &j); err != nil {
    panic(err)
}

jbytes, err := json.Marshal(j)

result := &pb.Response{}
r := strings.NewReader(string(jbytes))
if err := jsonpb.Unmarshal(r, result); err != nil {
    panic(err)
}