Istio distributed tracing with Jaeger not working

Question

I'm trying to set up a local k8s cluster and on minikube with installed istio and I have an issue with enabling distributed tracing with Jaeger. I have 3 microservices A -> B -> C. I am propagating the all the headers that are needed:

{"x-request-id", "x-b3-traceid", "x-b3-spanid", "x-b3-parentspanid", "x-b3-sampled", "x-b3-flags", "x-ot-span-context"}

But on Jaeger interface, I can only see the request to the service A and I cannot see the request going to service B.

I have logged the headers that are sent in the request. Headers from service A:

Header - x-request-id: c2804368-2ff0-9d90-a2aa-972537968924
Header - x-b3-traceid: 3a2400b40bbe5ed8
Header - x-b3-spanid: 3a2400b40bbe5ed8
Header - x-b3-parentspanid: 
Header - x-b3-sampled: 1
Header - x-b3-flags: 
Header - x-ot-span-context: 

Headers from service B:

Header - x-request-id: c2804368-2ff0-9d90-a2aa-972537968924
Header - x-b3-traceid: 3a2400b40bbe5ed8
Header - x-b3-spanid: 3a2400b40bbe5ed8
Header - x-b3-parentspanid:
Header - x-b3-sampled: 1
Header - x-b3-flags:
Header - x-ot-span-context:

So the x-request-id, x-b3-traceid, x-b3-sampled, and x-b3-spanid mathces. There are some headers that aren't set. Also, I'm accessing service A via k8s Service IP of type LoadBalancer, not via ingress. Don't know if this could be the issue.

UPD: I have setup istio gateway so now I'm accessing service A via istio gateway. However the result is the same, I can see the trace for gateway->A but no any further tracing

Answer 1

Some web frameworks return empty string if a non-existent header is queried. I have seen this in Spring Boot and KoaJS.

If any of the tracing headers is not sent by Istio, this header logic causes us to send empty string for those non-existent headers which breaks tracing.

My suggestion is after getting the values for headers filter out the ones with empty string as their values and propogate the remaining ones.