Split out from #9358 as requested in #9358 (comment). Same defect class as the otelhttp report, confirmed independently against otelecho.
Symptom
When a client disconnects mid-request, the handler observes the cancelled request context and typically returns an error, which surfaces as a 500. otelecho then records http.response.status_code=500 and sets the span status to Error, and sets no error.type at all, so a client disconnect is indistinguishable from a genuine server fault.
Middleware reads the status from c.Response().Status after next(c) and never inspects the request context:
|
err := next(c) |
|
if err != nil { |
|
span.SetAttributes(attribute.String("echo.error", err.Error())) |
|
cfg.OnError(c, err) |
|
} |
|
|
|
status := c.Response().Status |
|
span.SetStatus(semconvSrv.Status(status)) |
|
span.SetAttributes(semconvSrv.ResponseTraceAttrs(semconv.ResponseTelemetry{ |
|
StatusCode: status, |
|
WriteBytes: c.Response().Size, |
|
})...) |
|
|
|
// Record the server-side attributes. |
|
var additionalAttributes []attribute.KeyValue |
|
if cfg.MetricAttributeFn != nil { |
|
additionalAttributes = append(additionalAttributes, cfg.MetricAttributeFn(request)...) |
|
} |
|
if cfg.EchoMetricAttributeFn != nil { |
|
additionalAttributes = append(additionalAttributes, cfg.EchoMetricAttributeFn(c)...) |
|
} |
|
|
|
semconvSrv.RecordMetrics(ctx, semconv.ServerMetricData{ |
|
ServerName: serverName, |
|
ResponseSize: c.Response().Size, |
|
MetricAttributes: semconv.MetricAttributes{ |
|
Req: request, |
|
StatusCode: status, |
|
Route: c.Path(), |
|
AdditionalAttributes: additionalAttributes, |
|
}, |
|
MetricData: semconv.MetricData{ |
|
RequestSize: request.ContentLength, |
|
RequestDuration: time.Since(requestStartTime), |
|
}, |
|
}) |
otelecho has an extra wrinkle worth fixing in the same pass. When the handler returns an error it records the full error string as a non-semconv echo.error attribute:
if err != nil {
span.SetAttributes(attribute.String("echo.error", err.Error()))
cfg.OnError(c, err)
}
That is unbounded-cardinality free text on a span attribute, and it is the only place the failure cause is captured — error.type is never set. Idiomatic Echo handlers return the error rather than writing a status themselves, so this is the common path, not an edge case. RecordMetrics carries StatusCode only, with no error.type.
Reproduction
Confirmed on main at 5f30960 with a real HTTP/1.1 disconnect, in both handler shapes. The client issues the request with a cancellable context and cancels it after 100ms.
Handler returns the context error (idiomatic Echo):
e.Use(otelecho.Middleware("srv", otelecho.WithTracerProvider(tp)))
e.GET("/hello", func(c echo.Context) error {
<-c.Request().Context().Done()
return c.Request().Context().Err()
})
span name = "GET /hello"
span status = Error ""
attr http.route = /hello
attr echo.error = context canceled
attr http.response.body.size = 36
attr http.response.status_code = 500
Handler writes the status itself (matching the #9358 repro):
e.GET("/hello", func(c echo.Context) error {
<-c.Request().Context().Done()
return c.String(http.StatusInternalServerError, "cancelled")
})
span name = "GET /hello"
span status = Error ""
attr http.route = /hello
attr http.response.body.size = 9
attr http.response.status_code = 500
Neither sets error.type. In the first case the cause is present but only as free-text echo.error; in the second it is lost entirely.
Suggested fix
Mirroring the triage in #9358 (comment), adapted to Echo:
- Prefer an observed failure cause over the bare status code — the error returned by
next(c), and the request-context error.
- Set span status to
Error and add a predictable, low-cardinality error.type for that cause, alongside (or instead of) the free-text echo.error attribute.
- Preserve the observed/selected HTTP status code; for an error status with no more specific cause, use the status-code string as
error.type.
- Add
error.type to the metric attributes recorded by RecordMetrics, per the HTTP metric conventions.
- Add a real disconnect regression test covering both handler shapes above.
Worth deciding separately whether echo.error should stay as-is, be reduced to a bounded value, or be dropped in favour of error.type — happy to split that into its own issue if preferred.
Note on shared code
otelecho/internal/semconv/server.go is generated from internal/shared/semconv/server.go.tmpl and is byte-identical to the copy vendored into otelhttp (verified by diff). The same is true for otelgin, otelmux and otelrestful. So whatever error.type classification lands in the shared template is picked up by all of them, but each middleware still needs its own wiring to detect and pass the failure cause, plus its own regression test.
Relevant requirements: HTTP span conventions, HTTP metric conventions, and the dropped-connection example.
Split out from #9358 as requested in #9358 (comment). Same defect class as the
otelhttpreport, confirmed independently againstotelecho.Symptom
When a client disconnects mid-request, the handler observes the cancelled request context and typically returns an error, which surfaces as a 500.
otelechothen recordshttp.response.status_code=500and sets the span status toError, and sets noerror.typeat all, so a client disconnect is indistinguishable from a genuine server fault.Middlewarereads the status fromc.Response().Statusafternext(c)and never inspects the request context:opentelemetry-go-contrib/instrumentation/github.com/labstack/echo/otelecho/echo.go
Lines 96 to 131 in 5f30960
otelechohas an extra wrinkle worth fixing in the same pass. When the handler returns an error it records the full error string as a non-semconvecho.errorattribute:That is unbounded-cardinality free text on a span attribute, and it is the only place the failure cause is captured —
error.typeis never set. Idiomatic Echo handlers return the error rather than writing a status themselves, so this is the common path, not an edge case.RecordMetricscarriesStatusCodeonly, with noerror.type.Reproduction
Confirmed on
mainat 5f30960 with a real HTTP/1.1 disconnect, in both handler shapes. The client issues the request with a cancellable context and cancels it after 100ms.Handler returns the context error (idiomatic Echo):
Handler writes the status itself (matching the #9358 repro):
Neither sets
error.type. In the first case the cause is present but only as free-textecho.error; in the second it is lost entirely.Suggested fix
Mirroring the triage in #9358 (comment), adapted to Echo:
next(c), and the request-context error.Errorand add a predictable, low-cardinalityerror.typefor that cause, alongside (or instead of) the free-textecho.errorattribute.error.type.error.typeto the metric attributes recorded byRecordMetrics, per the HTTP metric conventions.Worth deciding separately whether
echo.errorshould stay as-is, be reduced to a bounded value, or be dropped in favour oferror.type— happy to split that into its own issue if preferred.Note on shared code
otelecho/internal/semconv/server.gois generated frominternal/shared/semconv/server.go.tmpland is byte-identical to the copy vendored intootelhttp(verified bydiff). The same is true forotelgin,otelmuxandotelrestful. So whatevererror.typeclassification lands in the shared template is picked up by all of them, but each middleware still needs its own wiring to detect and pass the failure cause, plus its own regression test.Relevant requirements: HTTP span conventions, HTTP metric conventions, and the dropped-connection example.