Does `json.NewDecoder().Decode()` Ignore Context Deadlines in Go HTTP Requests?

Go json.NewDecoder().Decode() Disregards Context Deadline?

Problem:

In a Go program with a context deadline, reading a response body using ioutil.ReadAll() results in the expected deadline exceeded error. However, using json.NewDecoder(resp.Body).Decode() returns nil instead.

Code Example:

<code class="go">package main

import (
    "context"
    "encoding/json"
    "fmt"
    "io/ioutil"
    "net/http"
    "time"
)

var url string = "http://ip.jsontest.com/"

func main() {
    readDoesntFail()
    readFails()
}

type IpResponse struct {
    Ip string
}

func readDoesntFail() {
    ctx, _ := context.WithTimeout(context.Background(), time.Second*5)

    req, err := http.NewRequestWithContext(ctx, http.MethodGet, url, nil)
    if err != nil {
        panic(err)
    }
    resp, err := http.DefaultClient.Do(req)
    if err != nil {
        panic(err)
    }

    ipResponse := new(IpResponse)
    time.Sleep(time.Second * 6)
    fmt.Println("before reading response body, context error is:", ctx.Err())
    err = json.NewDecoder(resp.Body).Decode(ipResponse)
    if err != nil {
        panic(err)
    }
    fmt.Println("Expected panic but there was none")
}

func readFails() {
    ctx, _ := context.WithTimeout(context.Background(), time.Second*5)
    req, err := http.NewRequestWithContext(ctx, http.MethodGet, url, nil)
    if err != nil {
        panic(err)
    }
    resp, err := http.DefaultClient.Do(req)
    if err != nil {
        panic(err)
    }

    time.Sleep(time.Second * 6)
    fmt.Println("before reading response body, context error is:", ctx.Err())
    _, err = ioutil.ReadAll(resp.Body)
    if err != nil {
        fmt.Println("received expected error", err)
    }
}</code>

Answer:

In the net/http package, buffers may be used to process requests. Consequently, the incoming response body may be partially or entirely read and buffered prior to your attempt to read it. As a result, an expiring context may not obstruct you from completing the reading of the body. This is precisely what occurs in this situation.

To better understand, let's alter the code to create a test HTTP server that intentionally postpones the response:

<code class="go">ts := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
    s := []byte(`{"ip":"12.34.56.78"}`)
    w.Write(s[:10])
    if f, ok := w.(http.Flusher); ok {
        f.Flush()
    }
    time.Sleep(time.Second * 6)
    w.Write(s[10:])
}))
defer ts.Close()
url = ts.URL

readDoesntFail()
readFails()</code>

The modified example sends a JSON object similar to the response of ip.jsontest.com, but transmits only the first 10 bytes of the body before flushing it. It then suspends the transmission for 6 seconds, giving the client an opportunity to time out.

When we execute readDoesntFail(), we observe the following behavior:

before reading response body, context error is: context deadline exceeded
panic: Get "http://127.0.0.1:38230": context deadline exceeded

goroutine 1 [running]:
main.readDoesntFail()
    /tmp/sandbox721114198/prog.go:46 +0x2b4
main.main()
    /tmp/sandbox721114198/prog.go:28 +0x93

In this scenario, json.Decoder.Decode() attempts to read from the connection because the data hasn't been buffered yet. Once the context expires, further reading from the connection results in a deadline exceeded error. However, in the original example, json.Decoder.Decode() is reading already buffered data, rendering the expired context irrelevant.

The above is the detailed content of Does `json.NewDecoder().Decode()` Ignore Context Deadlines in Go HTTP Requests?. For more information, please follow other related articles on the PHP Chinese website!