How can i improve video quality of pjsip pjsua2 sample Android App?

Question

Current default sample Android pjsip pjsua2 sample app sends a very bad video quality and wish to improve them to atleast Hd quality. I have tried using below methods , but

Answer 1

I still can't answer you're question with the given information yet.

SDP is used a the payload type of the SIP protocol.

You can see that in your (part) SIP log here:

Content-Type: application/sdp

SDP is a offer / answer protocol.

Given the incomplete log snip you have given the SIP INVITE (I assume as you haven't given the WHOLE sip message), so you have given the OFFER only of the SDP procotol. So get a complete picture you need to supply both the OFFER and ANSWER.

Also it would be good to also include the other PJSIP logging as well around the video encoder/decoder setup as well.

In your offer it says:

m=video 4013 RTP/AVP 97

there means is can send / recieve video with the parameters:

a=rtpmap:97 H264/90000
a=fmtp:97 profile-level-id=42e01e; packetization-mode=1

This means is can send/recieve H264 with a sample rate of 90000 (i.e. 90kHz).

The H264 parameters setup is: a=fmtp:97 profile-level-id=42e01e; packetization-mode=1

So...

profile-level-id=42e01e

https://tools.ietf.org/html/rfc6184

  profile-level-id:
     A base16 [7] (hexadecimal) representation of the following
     three bytes in the sequence parameter set NAL unit is specified
     in [1]: 1) profile_idc, 2) a byte herein referred to as
     profile-iop, composed of the values of constraint_set0_flag,
     constraint_set1_flag, constraint_set2_flag,
     constraint_set3_flag, constraint_set4_flag,
     constraint_set5_flag, and reserved_zero_2bits in bit-
     significance order, starting from the most-significant bit, and
     3) level_idc.  Note that reserved_zero_2bits is required to be
     equal to 0 in [1], but other values for it may be specified in
     the future by ITU-T or ISO/IEC.

profile_idc: 0x42 (66) profile-iop: 0xE0 (binary 11100000) level_idc: 0x1E (30)

https://en.wikipedia.org/wiki/Advanced_Video_Coding

profile_idc: 66

Baseline Profile (BP, 66) Primarily for low-cost applications that require additional data loss robustness, this profile is used in some videoconferencing and mobile applications. This profile includes all features that are supported in the Constrained Baseline Profile, plus three additional features that can be used for loss robustness (or for other purposes such as low-delay multi-point video stream compositing). The importance of this profile has faded somewhat since the definition of the Constrained Baseline Profile in 2009. All Constrained Baseline Profile bitstreams are also considered to be Baseline Profile bitstreams, as these two profiles share the same profile identifier code value.

profile-iop: binary 11100000

This means:

constraint_set0_flag=1 (Constrained Baseline profile)
constraint_set1_flag=1
constraint_set2_flag=1

These two values IDC & constraint flags are used to setup the video encoders based on what the decoders can support.

Levels: 30 i.e. 3.0

Level: 3.0 Maximum decoding speed (macroblocks/s): 40,500 Maximum
frame size (macroblocks): 1,620 Maximum video bit rate for video
coding layer (VCL): 10,000 Examples for high resolution @ highest
frame rate (maximum stored frames): 
  352×480@61.4 (12) 
  352×576@51.1 (10) 
  720×480@30.0 (6) 
  720×576@25.0 (5)

The profile level does not specify a video resolution, is manually specifies a maximum frame size / bitrate. ANY combination of resolution / framerate that can "fit" within the these constraits are valid. This is where this is a list of resoulation/framerates are listed as valid.

So 720×480 @ 30fps OR 720×576 @ 25fps are valid to send for level 3.0 profile.

What is offer is telling the other side is that:

1. This side can only DECODE Constrained Baseline Profile H264 encoded stream.
2. This side can only DECODE up to Level 3.0 bitrates (i.e. the list of resolutions/fps combos above)

The offer does not tell WHAT the device will send to the other side are this will depend on your local setup combined with what the other side says it can DECODE.

PJSIP will "pick" the best resolution / fps it can send based on your setup and the offer decode supported (which is why you can to see the PJSIP logs about the encoder setup) to know what it's sending based on the SDP ANSWER (not supplied).

The video doesn't have to be symmetrical. i.e. depending on the camera/screen H/W you may be to display different resolutions that what you can send.

This is also not taking into account things like resolutions changing dynamically during the streaming (e.g. portrait / landscape flipping or resolution increasing / decreasing based on network bandwidth changes from RTCP reports). The only way to look into this may be to capture and decode the H264 stream to undersatnd what it's doing. The PJSIP log may also tell you.

UPDATE

Looking at the pjsip logging output you can see both the SDP offer in the INVITE and the answer in the 200 OK.

I: 11:13:36.176           pjsua_core.c  .RX 1119 bytes Response msg 200/INVITE/cseq=22580 (rdata0x6f73203b18) from TLS 95.128.80.3:5061:
I: SIP/2.0 200 OK
I: To: <sip:0074444@james.com>;tag=61c5c92f
I: Via: SIP/2.0/TLS 146.4.49.20:49305;received=146.4.49.20;rport=49305;branch=z9hG4bKPjdad60ffa-6072-4c6d-8eb1-4a32ab26443a;alias
I: Record-Route: <sip:95.128.80.5;r2=on;lr=on;did=e8.cc62>,<sip:95.128.80.3:5061;transport=tls;r2=on;lr=on;did=e8.cc62>
I: CSeq: 22580 INVITE
I: Call-ID: 0e7676b2-1ca2-48b2-9696-f7e6dc7e1ec9
I: From: <sip:0677777@james.com>;tag=0b4094bb-b47e-4132-960c-ac564015efa0
I: Content-Type: application/sdp
I: Contact: <sip:peoplefone@95.128.80.93:5060;alias=95.128.80.93~5060~1>
I: Content-Length: 535
I: v=0
I: o=- 3819003211 3819003212 IN IP4 95.128.80.5
I: s=pjmedia
I: b=AS:352
I: t=0 0
I: a=X-nat:0
I: m=audio 20918 RTP/AVP 96 120
I: c=IN IP4 95.128.80.5
I: b=TIAS:64000
I: a=rtpmap:96 speex/16000
I: a=rtpmap:120 telephone-event/16000
I: a=fmtp:120 0-16
I: a=ssrc:1254727526 cname:496ca0741b8de59f
I: a=sendrecv
I: a=rtcp:20919
I: m=video 20956 RTP/AVP 97
I: c=IN IP4 95.128.80.5
I: b=TIAS:256000
I: a=rtpmap:97 H264/90000
I: a=fmtp:97 profile-level-id=42e01e; packetization-mode=1
I: a=ssrc:977888024 cname:496ca0741b8de59f
I: a=rtcp-fb:* nack pli
I: a=sendrecv
I: a=rtcp:20957
I: --end msg--

From the answer you can see that it answered with the same H264 parameters as the offer:

I: m=video 20956 RTP/AVP 97
...
I: a=rtpmap:97 H264/90000
I: a=fmtp:97 profile-level-id=42e01e; packetization-mode=1

So it will accept up to H264 level 3.0 bitrates.

If you look at the initialisation of the capture device (camera) we see these logs:

I: 11:13:36.270             vid_port.c  .........Opening device OpenGL renderer [OpenGL] for render: format=I420, size=352x288 @15:1 fps

This means the android front camera has been opened at a resolution of 352x288 @ 15fps.

I assume this is the cause of the video quailty you are talking about.

Looking at the source code for pjsip it enumerates the cameras with there supported parameters.

The supported capture resolution size is then determined by the default capture resolution size downsized based on the "allowed" resolution sizes that can be sent.

Since the allowed sizes is greater than 352x288@15 then I can only assume that your Andriod default capture resolution of your front camera is 352x288@15.

You can try using the back camera instead of the front camera to see if you get a getter solution or another Andriod device that has a better front camera.

PjSip uses the android.hardware.Camera API to access and use the camera devices. See PjCameraInfo and PjCamera for details of how pjsip uses the camera devices.