问题
Current default sample Android pjsip pjsua2 sample app sends a very bad video quality and wish to improve them to atleast Hd quality. I have tried using below methods , but it keep on showing a very low video quality. How can i improve outgoing video quality. This sample app is able to receive upto 355 * 288 video quality from other sip video call ,but it sends a very poor video quality. Currently i have tried to achieve hd video by updating below value from MediaFormatvideo file , just before making outgoing call. And it doesnt help at all to improve outgoing video . Am i updating those attribute at wrong places ?
Currently it sending video quality below 192 * 144 and doesnt accept above 352 * 288 video quality . how can i update it to support atleast 1280*720 video quality
MediaFormatVideo mf=new MediaFormatVideo();
mf.setFpsNum(30);
mf.setFpsDenum(1);
mf.setAvgBps(512000);
mf.setMaxBps(1024000);
mf.setHeight(720);
mf.setWidth(1280);
i am updating those setting as below .
MyCall call = new MyCall(account, -1);
CallOpParam prm = new CallOpParam(true);
AccountVideoConfig avc=new AccountVideoConfig();
MediaFormatVideo mf=new MediaFormatVideo();
Log.e("javan-video",String.valueOf(avc.getAutoShowIncoming()));
Log.e("javan-videofps",String.valueOf(mf.getFpsNum()));
mf.setFpsNum(30);
mf.setFpsDenum(1);
mf.setAvgBps(512000);
mf.setMaxBps(1024000);
mf.setHeight(720);
mf.setWidth(1280);
Log.e("javan-videofps",String.valueOf(mf.getFpsNum()));
try {
call.makeCall("sip:"+dialno+"@peoplefone.ch", prm);
AudioManager am = (AudioManager) getSystemService(Context.AUDIO_SERVICE);
am.setSpeakerphoneOn(true);
// startRinging();
} catch (Exception e) {
call.delete();
return;
}
currentCall = call;
showCallActivity();
}
i have found a documentation ,tried to implement ..but i couldnt able to improve video quality
Framerate
Specify number of frames processed per second.
For encoding direction, configured via pjmedia_vid_codec_param.enc_fmt.det.vid.fps, e.g:
/* Sending @30fps */
param.enc_fmt.det.vid.fps.num = 30;
param.enc_fmt.det.vid.fps.denum = 1;
Note:
that there is a possibility that the value will be adjusted to follow remote capability. For example, if remote signals that maximum framerate supported is 10fps and locally the encoding direction framerate is set to 30fps, then 10fps will be used.
limitation: if preview is enabled before call is established, capture device will opened using default framerate of the device, and subsequent calls that use that device will use this framerate regardless of the configured encoding framerate that is set above. Currently the only solution is to disable preview before establishing media and re-enable it once the video media is established.
For decoding direction, two steps are needed:
pjmedia_vid_codec_param.dec_fmt.det.vid.fps should be set to the highest value expected for incoming video framerate.
signalling to remote, configured via codec specific SDP format parameter (fmtp): pjmedia_vid_codec_param.dec_fmtp.
H263-1998, maximum framerate is specified per size/resolution basis, check here for more info.
/* 3000/(1.001*2) fps for CIF */
param.dec_fmtp.param[m].name = pj_str("CIF");
param.dec_fmtp.param[m].val = pj_str("2");
/* 3000/(1.001*1) fps for QCIF */
param.dec_fmtp.param[n].name = pj_str("QCIF");
param.dec_fmtp.param[n].val = pj_str("1");
H264, similar to size/resolution, the framerate is implicitly specified in H264 level (check the standard specification or this) and the H264 level is signalled via H264 SDP fmtp profile-level-id, e.g:
/* Can receive up to 1280×720 @30fps */
param.dec_fmtp.param[n].name = pj_str("profile-level-id");
param.dec_fmtp.param[n].val = pj_str("xxxx1f");
Bitrate
Specify bandwidth requirement for video payloads stream delivery.
This is configurable via pjmedia_vid_codec_param.enc_fmt.det.vid.avg_bps and pjmedia_vid_codec_param.enc_fmt.det.vid.max_bps, e.g:
/* Bitrate range preferred: 512-1024kbps */
param.enc_fmt.det.vid.avg_bps = 512000;
param.enc_fmt.det.vid.max_bps = 1024000;
Notes:
This setting is applicable for encoding and decoding direction, currently there is no way to set asymmetric bitrate. By decoding direction, actually it just means that this setting will be queried when generating bandwidth info for local SDP (see next point).
The bitrate setting of all codecs will be enumerated and the highest value will be signalled in bandwidth info in local SDP (see ticket #1244).
There is a possibility that the encoding bitrate will be adjusted to follow remote bitrate setting, i.e: read from SDP bandwidth info (b=TIAS line) in remote SDP. For example, if remote signals that maximum bitrate is 128kbps and locally the bitrate is set to 512kbps, then 128kbps will be used.
If codec specific bitrate setting signalling (via SDP fmtp) is desired, e.g: MaxBR for H263, application should put the SDP fmtp manually, for example:
/* H263 specific maximum bitrate 512kbps */
param.dec_fmtp.param[n].name = pj_str("MaxBR");
param.dec_fmtp.param[n].val = pj_str("5120"); /* = max_bps / 100 */
documentation link : enter link description here
From: "0525512904" <sip:0525512904@95.128.80.93>;tag=1609930889511
I: To: <sip:90761330096@95.128.80.5>;tag=c6ce5331-3a35-44c8-bb80-23b6ec664085
I: CSeq: 1 INVITE
I: Allow: PRACK, INVITE, ACK, BYE, CANCEL, UPDATE, INFO, SUBSCRIBE, NOTIFY, REFER, MESSAGE, OPTIONS
I: Contact: <sip:90761330096@146.4.49.20:45483;transport=TLS;ob>
I: Supported: replaces, 100rel, timer, norefersub
I: Content-Type: application/sdp
I: Content-Length: 580
I: v=0
I: o=- 3818919690 3818919691 IN IP4 192.168.3.135
I: s=pjmedia
I: b=AS:352
I: t=0 0
I: a=X-nat:0
I: m=audio 4012 RTP/AVP 96 120
I: c=IN IP4 192.168.3.135
I: b=TIAS:64000
I: a=rtcp:4031 IN IP4 192.168.3.135
I: a=sendrecv
I: a=rtpmap:96 speex/16000
I: a=rtpmap:120 telephone-event/16000
I: a=fmtp:120 0-16
I: a=ssrc:1510027056 cname:365aaa4f448493db
I: m=video 4013 RTP/AVP 97
I: c=IN IP4 192.168.3.135
I: b=TIAS:256000
I: a=rtcp:4033 IN IP4 192.168.3.135
I: a=sendrecv
I: a=rtpmap:97 H264/90000
I: a=fmtp:97 profile-level-id=42e01e; packetization-mode=1
I: a=ssrc:1146236185 cname:365aaa4f448493db
I: a=rtcp-fb:* nack pli
I: --end msg--
E: ringing call
complete log link sip call complete log
回答1:
I still can't answer you're question with the given information yet.
SDP is used a the payload type of the SIP protocol.
You can see that in your (part) SIP log here:
Content-Type: application/sdp
SDP is a offer / answer protocol.
Given the incomplete log snip you have given the SIP INVITE (I assume as you haven't given the WHOLE sip message), so you have given the OFFER only of the SDP procotol. So get a complete picture you need to supply both the OFFER and ANSWER.
Also it would be good to also include the other PJSIP logging as well around the video encoder/decoder setup as well.
In your offer it says:
m=video 4013 RTP/AVP 97
there means is can send / recieve video with the parameters:
a=rtpmap:97 H264/90000
a=fmtp:97 profile-level-id=42e01e; packetization-mode=1
This means is can send/recieve H264 with a sample rate of 90000 (i.e. 90kHz).
The H264 parameters setup is: a=fmtp:97 profile-level-id=42e01e; packetization-mode=1
So...
profile-level-id=42e01e
https://tools.ietf.org/html/rfc6184
profile-level-id: A base16 [7] (hexadecimal) representation of the following three bytes in the sequence parameter set NAL unit is specified in [1]: 1) profile_idc, 2) a byte herein referred to as profile-iop, composed of the values of constraint_set0_flag, constraint_set1_flag, constraint_set2_flag, constraint_set3_flag, constraint_set4_flag, constraint_set5_flag, and reserved_zero_2bits in bit- significance order, starting from the most-significant bit, and 3) level_idc. Note that reserved_zero_2bits is required to be equal to 0 in [1], but other values for it may be specified in the future by ITU-T or ISO/IEC.
profile_idc: 0x42 (66) profile-iop: 0xE0 (binary 11100000) level_idc: 0x1E (30)
https://en.wikipedia.org/wiki/Advanced_Video_Coding
profile_idc: 66
Baseline Profile (BP, 66) Primarily for low-cost applications that require additional data loss robustness, this profile is used in some videoconferencing and mobile applications. This profile includes all features that are supported in the Constrained Baseline Profile, plus three additional features that can be used for loss robustness (or for other purposes such as low-delay multi-point video stream compositing). The importance of this profile has faded somewhat since the definition of the Constrained Baseline Profile in 2009. All Constrained Baseline Profile bitstreams are also considered to be Baseline Profile bitstreams, as these two profiles share the same profile identifier code value.
profile-iop: binary 11100000
This means:
constraint_set0_flag=1 (Constrained Baseline profile)
constraint_set1_flag=1
constraint_set2_flag=1
These two values IDC & constraint flags are used to setup the video encoders based on what the decoders can support.
Levels: 30 i.e. 3.0
Level: 3.0 Maximum decoding speed (macroblocks/s): 40,500 Maximum
frame size (macroblocks): 1,620 Maximum video bit rate for video
coding layer (VCL): 10,000 Examples for high resolution @ highest
frame rate (maximum stored frames):
352×480@61.4 (12)
352×576@51.1 (10)
720×480@30.0 (6)
720×576@25.0 (5)
The profile level does not specify a video resolution, is manually specifies a maximum frame size / bitrate. ANY combination of resolution / framerate that can "fit" within the these constraits are valid. This is where this is a list of resoulation/framerates are listed as valid.
So 720×480 @ 30fps OR 720×576 @ 25fps are valid to send for level 3.0 profile.
What is offer is telling the other side is that:
- This side can only DECODE Constrained Baseline Profile H264 encoded stream.
- This side can only DECODE up to Level 3.0 bitrates (i.e. the list of resolutions/fps combos above)
The offer does not tell WHAT the device will send to the other side are this will depend on your local setup combined with what the other side says it can DECODE.
PJSIP will "pick" the best resolution / fps it can send based on your setup and the offer decode supported (which is why you can to see the PJSIP logs about the encoder setup) to know what it's sending based on the SDP ANSWER (not supplied).
The video doesn't have to be symmetrical. i.e. depending on the camera/screen H/W you may be to display different resolutions that what you can send.
This is also not taking into account things like resolutions changing dynamically during the streaming (e.g. portrait / landscape flipping or resolution increasing / decreasing based on network bandwidth changes from RTCP reports). The only way to look into this may be to capture and decode the H264 stream to undersatnd what it's doing. The PJSIP log may also tell you.
UPDATE
Looking at the pjsip logging output you can see both the SDP offer in the INVITE and the answer in the 200 OK.
I: 11:13:36.176 pjsua_core.c .RX 1119 bytes Response msg 200/INVITE/cseq=22580 (rdata0x6f73203b18) from TLS 95.128.80.3:5061:
I: SIP/2.0 200 OK
I: To: <sip:0074444@james.com>;tag=61c5c92f
I: Via: SIP/2.0/TLS 146.4.49.20:49305;received=146.4.49.20;rport=49305;branch=z9hG4bKPjdad60ffa-6072-4c6d-8eb1-4a32ab26443a;alias
I: Record-Route: <sip:95.128.80.5;r2=on;lr=on;did=e8.cc62>,<sip:95.128.80.3:5061;transport=tls;r2=on;lr=on;did=e8.cc62>
I: CSeq: 22580 INVITE
I: Call-ID: 0e7676b2-1ca2-48b2-9696-f7e6dc7e1ec9
I: From: <sip:0677777@james.com>;tag=0b4094bb-b47e-4132-960c-ac564015efa0
I: Content-Type: application/sdp
I: Contact: <sip:peoplefone@95.128.80.93:5060;alias=95.128.80.93~5060~1>
I: Content-Length: 535
I: v=0
I: o=- 3819003211 3819003212 IN IP4 95.128.80.5
I: s=pjmedia
I: b=AS:352
I: t=0 0
I: a=X-nat:0
I: m=audio 20918 RTP/AVP 96 120
I: c=IN IP4 95.128.80.5
I: b=TIAS:64000
I: a=rtpmap:96 speex/16000
I: a=rtpmap:120 telephone-event/16000
I: a=fmtp:120 0-16
I: a=ssrc:1254727526 cname:496ca0741b8de59f
I: a=sendrecv
I: a=rtcp:20919
I: m=video 20956 RTP/AVP 97
I: c=IN IP4 95.128.80.5
I: b=TIAS:256000
I: a=rtpmap:97 H264/90000
I: a=fmtp:97 profile-level-id=42e01e; packetization-mode=1
I: a=ssrc:977888024 cname:496ca0741b8de59f
I: a=rtcp-fb:* nack pli
I: a=sendrecv
I: a=rtcp:20957
I: --end msg--
From the answer you can see that it answered with the same H264 parameters as the offer:
I: m=video 20956 RTP/AVP 97
...
I: a=rtpmap:97 H264/90000
I: a=fmtp:97 profile-level-id=42e01e; packetization-mode=1
So it will accept up to H264 level 3.0 bitrates.
If you look at the initialisation of the capture device (camera) we see these logs:
I: 11:13:36.270 vid_port.c .........Opening device OpenGL renderer [OpenGL] for render: format=I420, size=352x288 @15:1 fps
This means the android front camera has been opened at a resolution of 352x288 @ 15fps.
I assume this is the cause of the video quailty you are talking about.
Looking at the source code for pjsip it enumerates the cameras with there supported parameters.
The supported capture resolution size is then determined by the default capture resolution size downsized based on the "allowed" resolution sizes that can be sent.
Since the allowed sizes is greater than 352x288@15 then I can only assume that your Andriod default capture resolution of your front camera is 352x288@15.
You can try using the back camera instead of the front camera to see if you get a getter solution or another Andriod device that has a better front camera.
PjSip uses the android.hardware.Camera API to access and use the camera devices. See PjCameraInfo and PjCamera for details of how pjsip uses the camera devices.
来源：https://stackoverflow.com/questions/65410821/how-can-i-improve-video-quality-of-pjsip-pjsua2-sample-android-app